Bohdana Ratitch

Missing data in clinical trials: from clinical assumptions to statistical analysis using pattern mixture models.

The need to use rigorous, transparent, clearly interpretable, and scientifically justified methodology for preventing and dealing with missing data in clinical trials has been a focus of much attention from regulators, practitioners, and academicians over the past years. New guidelines and recommendations emphasize the importance of minimizing the amount of missing data and carefully selecting primary analysis methods on the basis of assumptions regarding the missingness mechanism suitable for the study at hand, as well as the need to stress-test the results of the primary analysis under different sets of assumptions through a range of sensitivity analyses. Some methods that could be effectively used for dealing with missing data have not yet gained widespread usage, partly because of their underlying complexity and partly because of lack of relatively easy approaches to their implementation. In this paper, we explore several strategies for missing data on the basis of pattern mixture models that embody clear and realistic clinical assumptions. Pattern mixture models provide a statistically reasonable yet transparent framework for translating clinical assumptions into statistical analyses. Implementation details for some specific strategies are provided in an Appendix (available online as Supporting Information), whereas the general principles of the approach discussed in this paper can be used to implement various other analyses with different sets of assumptions regarding missing data.

Missing Data: Turning Guidance Into Action

Recent research has fostered new guidance on preventing and treating missing data. This article is the consensus opinion of the Drug Information Associations Scientific Working Group on Missing Data. Common elements from recent guidance are distilled and means for putting the guidance into action are proposed. The primary goal is to maximize the proportion of patients that adhere to the protocol specified interventions. In so doing, trial design and trial conduct should be considered. Completion rate should be focused upon as much as enrollment rate, with particular focus on minimizing loss to follow-up. Whether or not follow-up data after discontinuation of the originally randomized medication and/or initiation of rescue medication contribute to the primary estimand depends on the context. In outcomes trials (intervention thought to influence disease process) follow-up data are often included in the primary estimand, whereas in symptomatic trials (intervention alters symptom severity but does not change underlying disease) follow-up data are often not included. Regardless of scenario, the confounding influence of rescue medications can render follow-up data of little use in understanding the causal effects of the randomized interventions. A sensible primary analysis can often be formulated in the missing at random (MAR) framework. Sensitivity analyses assessing robustness to departures from MAR are crucial. Plausible sensitivity analyses can be prespecified using controlled imputation approaches to either implement a plausibly conservative analysis or to stress test the primary result, and used in combination with other model-based MNAR approaches such as selection, shared parameter, and pattern-mixture models. The example dataset and analyses used in this article are freely available for public use at www.missingdata.org.uk.

Recent Developments in the Prevention and Treatment of Missing Data

Recent research has fostered new guidance on preventing and treating missing data, most notably the landmark expert panel report from the National Research Council (NRC) that was commissioned by FDA. One of the findings from that panel was the need for better software tools to conduct missing data sensitivity analyses and frameworks for drawing inference from them. In response to the NRC recommendations, a Scientific Working Group was formed under the Auspices of the Drug Information Association (DIASWG). The present paper is from work of the DIASWG. Specifically, the NRC panel’s 18 recommendations are distilled into 3 pillars for dealing with missing data: (1) providing clearly stated objectives and causal estimands; (2) preventing as much missing data as possible; and (3) combining a sensible primary analysis with sensitivity analyses to assess robustness of inferences to missing data assumptions. Sample data sets are used to illustrate how sensitivity analyses can be used to assess robustness of inferences to missing data assumptions. The suite of software tools used to conduct the sensitivity analyses are freely available for public use at www.missingdata.org.uk.

Multiplicity issues in exploratory subgroup analysis

ABSTRACT The general topic of subgroup identification has attracted much attention in the clinical trial literature due to its important role in the development of tailored therapies and personalized medicine. Subgroup search methods are commonly used in late-phase clinical trials to identify subsets of the trial population with certain desirable characteristics. Post-hoc or exploratory subgroup exploration has been criticized for being extremely unreliable. Principled approaches to exploratory subgroup analysis based on recent advances in machine learning and data mining have been developed to address this criticism. These approaches emphasize fundamental statistical principles, including the importance of performing multiplicity adjustments to account for selection bias inherent in subgroup search. This article provides a detailed review of multiplicity issues arising in exploratory subgroup analysis. Multiplicity corrections in the context of principled subgroup search will be illustrated using the family of SIDES (subgroup identification based on differential effect search) methods. A case study based on a Phase III oncology trial will be presented to discuss the details of subgroup search algorithms with resampling-based multiplicity adjustment procedures.

Subgroup Identification in Clinical Trials by Stochastic SIDEScreen Methods

ABSTRACT Subgroup identification for personalized medicine has become very popular in the last decade. Efficient recursive partitioning procedures adapted from machine learning are natural approaches for performing subgroup identification based on pre-defined biomarkers since they provide subgroups as terminal nodes in the decision tree. However, recursive partitioning is also known as a potentially unstable procedure with results being quite sensitive to normal sampling variability in the data. One common approach, borrowed from ensemble learning, to overcome such instability is application of recursive partitioning to multiple data sets sampled from the observed data followed by averaging the results over the collection of subgroups. This article proposes an alternative approach to subgroup identification in clinical trials that first evaluates the predictive strength of biomarkers based on variable importance and then applies recursive partitioning to the biomarkers with the highest variable importance scores. A deterministic version of this idea was implemented in the Adaptive SIDEScreen method that generates a collection of patient subgroups by retaining multiple candidate splits of each parent group by different biomarkers (Lipkovich and Dmitrienko 2014a, 2014b). Now, we extend the Adaptive SIDEScreen and introduce the Stochastic SIDEScreen method. The key idea is to introduce randomness in the subgroup generation process, borrowing from bagging methods, to produce a broader collection of subgroups. Specifically, the SIDES method, where the most promising biomarkers are selected for each parent group from a set of candidate biomarkers, is applied to multiple bootstrap samples of the data. This new approach leads to a more reliable biomarker selection process, which is especially important for smaller, early phase studies when biomarker selection is typically carried out. The method is illustrated using clinical trial examples.

Points to consider for analyzing efficacy outcomes in long-term extension clinical trials

This article focuses on 2 objectives in the analysis of efficacy in long-term extension studies of chronic diseases: (1) defining and discussing estimands of interest in such studies and (2) evaluating the performance of several multiple imputation methods that may be useful in estimating some of these estimands. Specifically, 4 estimands are defined and their clinical utility and inferential ramifications discussed. The performance of several multiple imputation methods and approaches were evaluated using simulated data. Results suggested that when interest is in a binary outcome derived from an underlying continuous measurement, it is preferable to impute the underlying continuous value that is subsequently dichotomized rather than to directly impute the binary outcome. Results also demonstrated that multivariate Gaussian models with Markov chain Monte Carlo imputation and sequential regression have minimal bias and the anticipated confidence interval coverage, even in settings with ordinal data where departures from normality are a concern. These approaches are further illustrated using a long-term extension study in psoriasis.