Peter W. Lane

Recommendations for the Primary Analysis of Continuous Endpoints in Longitudinal Clinical Trials

This position paper summarizes relevant theory and current practice regarding the analysis of longitudinal clinical trials intended to support regulatory approval of medicinal products, and it reviews published research regarding methods for handling missing data. It is one strand of the PhRMA initiative to improve efficiency of late-stage clinical research and gives recommendations from a cross-industry team. We concentrate specifically on continuous response measures analyzed using a linear model, when the goal is to estimate and test treatment differences at a given time point. Traditionally, the primary analysis of such trials handled missing data by simple imputation using the last, or baseline, observation carried forward method (LOCF, BOCF) followed by analysis of (co)variance at the chosen time point. However, the general statistical and scientific community has moved away from these simple methods in favor of joint analysis of data from all time points based on a multivariate model (eg, of a mixed-effects type). One such newer method, a likelihood-based mixed-effects model repeated measures (MMRM) approach, has received considerable attention in the clinical trials literature. We discuss specific concerns raised by regulatory agencies with regard to MMRM and review published evidence comparing LOCF and MMRM in terms of validity, bias, power, and type I error. Our main conclusion is that the mixed model approach is more efficient and reliable as a method of primary analysis, and should be preferred to the inherently biased and statistically invalid simple imputation approaches. We also summarize other methods of handling missing data that are useful as sensitivity analyses for assessing the potential effect of data missing not at random.

Statistical approaches for conducting network meta-analysis in drug development.

We introduce health technology assessment and evidence synthesis briefly, and then concentrate on the statistical approaches used for conducting network meta-analysis (NMA) in the development and approval of new health technologies. NMA is an extension of standard meta-analysis where indirect as well as direct information is combined and can be seen as similar to the analysis of incomplete-block designs. We illustrate it with an example involving three treatments, using fixed-effects and random-effects models, and using frequentist and Bayesian approaches. As most statisticians in the pharmaceutical industry are familiar with SAS® software for analyzing clinical trials, we provide example code for each of the methods we illustrate. One issue that has been overlooked in the literature is the choice of constraints applied to random effects, and we show how this affects the estimates and standard errors and propose a symmetric set of constraints that is equivalent to most current practice. Finally, we discuss the role of statisticians in planning and carrying out NMAs and the strategy for dealing with important issues such as heterogeneity.

Meta-analysis of incidence of rare events

This is a review of methods for the meta-analysis of incidence of rare events using summary-level data. It is motivated and illustrated by the dataset used in a published analysis of cardiovascular safety in rosiglitazone trials. This review compares available methods for binary data, considering risk-difference, relative-risk and odds-ratio scales, fixed-effect and random-effects models, and frequentist and Bayesian approaches. Particular issues in this dataset include low incidence rates, the occurrence of studies with no events under one or all treatments, and discrepancy among results achieved using different statistical methodologies. The common method of adding a correction factor to handle zeroes may introduce bias where the incidence of events is small, as in this case. Alternative analyses on the log-odds scale are shown to give similar results, but the choice between them is less important than the potential sources of bias in any meta-analysis arising from limitations in the underlying dataset. It is important to present results carefully, including numerical and graphical summaries on the natural scale of risk when the analysis is on a statistically appropriate scale such as log-odds: the incidence rates should accompany an estimated ratio (of odds or risk) to put the analysis into the proper context. Beyond the statistical methodologies which are the focus of this paper, this dataset highlights the importance of understanding the limitations of the data being combined. Because the rosiglitazone dataset contains clinically heterogeneous trials with low event rates that were not designed or intended to assess cardiovascular outcomes, the findings of any meta-analysis of such trials should be considered hypothesis-generating.

The efficacy and safety of retigabine and other adjunctive treatments for refractory partial epilepsy: A systematic review and indirect comparison

INTRODUCTION Retigabine (RTG) is now approved in Europe and the US for the adjunctive treatment of partial-onset seizures in adults with epilepsy. To support submissions to EU reimbursement authorities, we explored its efficacy and tolerability relative to selected antiepileptic drugs (AEDs). METHODS A systematic review was conducted to identify placebo-controlled trials of RTG and selected AEDs approved for use in a similar position in the management pathway of partial epilepsy (eslicarbazepine acetate [ESL], lacosamide [LCM], pregabalin [PGB], tiagabine [TGB] and zonisamide [ZNS]). Using conventional and network meta-analyses as appropriate, we report efficacy and tolerability outcomes for each AED versus placebo and the performance of RTG relative to other AEDs. RESULTS Twenty studies met the inclusion criteria: three each for RTG, ESL, LCM, TGB and ZNS; five for PGB. Comparisons comprised 1-5 studies per AED. In the network meta-analysis, RTG was not found to be different from the other AEDs for responder rate (maintenance period), seizure freedom (maintenance period and double-blind period), withdrawals due to adverse events, and incidences of ataxia, dizziness, fatigue and nausea. Differences between RTG and other AEDs were found for a few comparisons, which did not reveal any trends: RTG was associated with a lower responder rate than PGB during the double-blind period, higher withdrawal rate due to any reason than ESL and a higher incidence of somnolence than TGB. CONCLUSIONS Findings suggest that the risk/benefit for RTG is similar to that for comparator AEDs. However, results should be interpreted in the context of the limitations of the analyses.

A tool to assess the quality of a meta-analysis

BACKGROUND Because meta-analyses are increasingly prevalent and cited in the medical literature, it is important that tools are available to assess their methodological quality. When performing an empirical study of the quality of published meta-analyses, we found that existing tools did not place a strong emphasis on statistical and interpretational issues. METHODS We developed a quality-assessment tool using existing materials and expert judgment as a starting point, followed by multiple iterations of input from our working group, piloting, and discussion. After having used the tool for our empirical study, agreement for four key items in the tool was measured using weighted kappa coefficients. RESULTS Our tool contained 43 items divided into four key areas (data sources, analysis of individual studies, meta-analysis methods, and interpretation), and each area ended with a summary question. We also produced guidance for completing the tool. Agreement between raters was fair to moderate. CONCLUSIONS The tool should usefully inform subsequent initiatives to develop quality-assessment tools for meta-analysis. We advocate use of consensus between independent raters when assessing statistical appropriateness and adequacy of interpretation in meta-analyses.

Missing Data: Turning Guidance Into Action

Recent research has fostered new guidance on preventing and treating missing data. This article is the consensus opinion of the Drug Information Associations Scientific Working Group on Missing Data. Common elements from recent guidance are distilled and means for putting the guidance into action are proposed. The primary goal is to maximize the proportion of patients that adhere to the protocol specified interventions. In so doing, trial design and trial conduct should be considered. Completion rate should be focused upon as much as enrollment rate, with particular focus on minimizing loss to follow-up. Whether or not follow-up data after discontinuation of the originally randomized medication and/or initiation of rescue medication contribute to the primary estimand depends on the context. In outcomes trials (intervention thought to influence disease process) follow-up data are often included in the primary estimand, whereas in symptomatic trials (intervention alters symptom severity but does not change underlying disease) follow-up data are often not included. Regardless of scenario, the confounding influence of rescue medications can render follow-up data of little use in understanding the causal effects of the randomized interventions. A sensible primary analysis can often be formulated in the missing at random (MAR) framework. Sensitivity analyses assessing robustness to departures from MAR are crucial. Plausible sensitivity analyses can be prespecified using controlled imputation approaches to either implement a plausibly conservative analysis or to stress test the primary result, and used in combination with other model-based MNAR approaches such as selection, shared parameter, and pattern-mixture models. The example dataset and analyses used in this article are freely available for public use at www.missingdata.org.uk.

Methodological quality of meta-analyses : matched-pairs comparison over time and between industry sponsored and academic-sponsored reports

CONTEXT Meta-analyses are regularly used to inform healthcare decisions. Concerns have been expressed about the quality of meta-analyses and, in particular, about those supported by the pharmaceutical industry. OBJECTIVE The objective of this study is to compare the quality of pharmaceutical-industry-supported meta-analyses with academic meta-analyses and of meta-analyses published before and after companies started to disclose their data. DATA SOURCES We identified industry-supported meta-analyses by searching the Scopus bibliographic database, using author affiliations. We matched each industry-supported meta-analysis with an academic meta-analysis using high-level MeSH terms in PubMed. STUDY SELECTION We included meta-analyses of randomized trials assessing the efficacy or safety of any pharmaceutical intervention in humans, published in 2002-2004 or 2008-2009. Cochrane reviews were excluded. Two individuals independently selected papers, with discrepancies resolved by two further individuals. ASSESSMENT We developed and piloted a quality-assessment tool, consisting of 43 questions in four domains, with a key summary question covering each domain. Two individuals independently assessed each meta-analysis. RESULTS We examined 126 meta-analysis publications in 63 matched pairs. The average quality was low, with fewer than 50% adequate in three of the four domains. Industry-supported meta-analyses less often demonstrated adequate methods for locating studies and assessing their quality (odds ratio 0.44, 95% confidence interval 0.21 to 0.92), for analysing the included studies (0.52, 0.25 to 1.06), for undertaking meta-analyses (0.82, 0.40 to 1.68) and in reaching sound conclusions (0.62, 0.30 to 1.28). Quality generally improved over time, particularly for some aspects of industry reports. CONCLUSIONS Academic meta-analysis papers are generally of higher quality than industry-supported ones. This is largely due to less detailed reporting in industry-supported meta-analyses and a tendency for them to take the included studies at face value, probably arising from the implicit assumption that these studies already have high methodological standards to meet licensing requirements. The improved quality over time does not appear to be due to the use of data disclosed by industry. The main limitations of this study are the small sample of papers and the subjective nature of some of the assessment processes.

Graphics for Safety Analysis

We present a range of graphics designed for reporting the analysis of safety data. For adverse events (AEs), we show a comparative dot-and-interval plot of all the main AEs in a trial and also comparative plots of cumulative incidence and hazard rate for individual AEs. For laboratory data, we show a scatterplot designed to help identify potential liver toxicity and 2 trellis plots that can show several laboratory measurements in a single graph, showing changes from baseline or the relationship between the measurements. We also give an example of a profile plot for individual patients. Finally, for ECG data we show a comparative cumulative distribution plot and a comparative boxplot profile showing how distributions change over time. We also show a simple comparative profile plot of means of an ECG measurement over time. We produced each graph using one of GenStat™, SAS™ and S-PLUS™ (using code very similar to R), as indicated in the text, and the programs and data are available from the Web site associated with the book.

Graphics for Meta-Analysis

We present 4 types of graphic used in meta-analysis. The commonest is the forest plot, and we discuss important aspects of the basic form of this plot. We present 2 enhanced versions, one displaying the results of subgroup analysis, and the second displaying absolute risks alongside relative risks from a meta-analysis of a binary outcome. The funnel plot is a well-established graph for assessing publication bias. We show some alternative forms, including a recently suggested enhancement using contours. The third type is a bubble plot used to summarize the results of meta-regression. Finally, we show a graphic designed for network meta-analysis, presenting rankings of the treatments that are compared. We prepared programs and graphs using GenStat™, R, RevMan™, SAS™ and Stata™, and these are available from the website.