Jayne Tierney

To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data.

Systematic reviews and meta-analyses that obtain original research data on individual participants enrolled in trials have been described as the gold standard of review. However, they may take longer and be more resource intensive than other types of review. The authors describe potential advantages and disadvantages of the individual patient data (IPD) approach, including benefits from improved data quality, benefits afforded by the type of analyses that can be done, and advantages in achieving consensus around results and interpretation by an international multi disciplinary team. Disadvantages and barriers relating to resource and expertise, negotiating collaboration, and software requirements are also discussed. At the outset, reviewers should consider the methodological factors likely to influence results in their particular review setting, together with time and resource constraints, so that an active decision can be made about whether to extract data frompublished reports, collect addi tional or replacement summary data from trialists, or collect IPD.

Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data The PRISMA-IPD Statement

IMPORTANCEnSystematic reviews and meta-analyses of individual participant data (IPD) aim to collect, check, and reanalyze individual-level data from all studies addressing a particular research question and are therefore considered a gold standard approach to evidence synthesis. They are likely to be used with increasing frequency as current initiatives to share clinical trial data gain momentum and may be particularly important in reviewing controversial therapeutic areas.nnnOBJECTIVEnTo develop PRISMA-IPD as a stand-alone extension to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement, tailored to the specific requirements of reporting systematic reviews and meta-analyses of IPD. Although developed primarily for reviews of randomized trials, many items will apply in other contexts, including reviews of diagnosis and prognosis.nnnDESIGNnDevelopment of PRISMA-IPD followed the EQUATOR Network framework guidance and used the existing standard PRISMA Statement as a starting point to draft additional relevant material. A web-based survey informed discussion at an international workshop that included researchers, clinicians, methodologists experienced in conducting systematic reviews and meta-analyses of IPD, and journal editors. The statement was drafted and iterative refinements were made by the project, advisory, and development groups. The PRISMA-IPD Development Group reached agreement on the PRISMA-IPD checklist and flow diagram by consensus.nnnFINDINGSnCompared with standard PRISMA, the PRISMA-IPD checklist includes 3 new items that address (1) methods of checking the integrity of the IPD (such as pattern of randomization, data consistency, baseline imbalance, and missing data), (2) reporting any important issues that emerge, and (3) exploring variation (such as whether certain types of individual benefit more from the intervention than others). A further additional item was created by reorganization of standard PRISMA items relating to interpreting results. Wording was modified in 23 items to reflect the IPD approach.nnnCONCLUSIONS AND RELEVANCEnPRISMA-IPD provides guidelines for reporting systematic reviews and meta-analyses of IPD.

Quantifying, displaying and accounting for heterogeneity in the meta-analysis of RCTs using standard and generalised Q statistics

BackgroundClinical researchers have often preferred to use a fixed effects model for the primary interpretation of a meta-analysis. Heterogeneity is usually assessed via the well known Q and I2 statistics, along with the random effects estimate they imply. In recent years, alternative methods for quantifying heterogeneity have been proposed, that are based on a generalised Q statistic.MethodsWe review 18 IPD meta-analyses of RCTs into treatments for cancer, in order to quantify the amount of heterogeneity present and also to discuss practical methods for explaining heterogeneity.ResultsDiffering results were obtained when the standard Q and I2 statistics were used to test for the presence of heterogeneity. The two meta-analyses with the largest amount of heterogeneity were investigated further, and on inspection the straightforward application of a random effects model was not deemed appropriate. Compared to the standard Q statistic, the generalised Q statistic provided a more accurate platform for estimating the amount of heterogeneity in the 18 meta-analyses.ConclusionsExplaining heterogeneity via the pre-specification of trial subgroups, graphical diagnostic tools and sensitivity analyses produced a more desirable outcome than an automatic application of the random effects model. Generalised Q statistic methods for quantifying and adjusting for heterogeneity should be incorporated as standard into statistical software. Software is provided to help achieve this aim.

Addition of docetaxel or bisphosphonates to standard of care in men with localised or metastatic, hormone-sensitive prostate cancer: a systematic review and meta-analyses of aggregate data

Summary Background Results from large randomised controlled trials combining docetaxel or bisphosphonates with standard of care in hormone-sensitive prostate cancer have emerged. In order to investigate the effects of these therapies and to respond to emerging evidence, we aimed to systematically review all relevant trials using a framework for adaptive meta-analysis. Methods For this systematic review and meta-analysis, we searched MEDLINE, Embase, LILACS, and the Cochrane Central Register of Controlled Trials, trial registers, conference proceedings, review articles, and reference lists of trial publications for all relevant randomised controlled trials (published, unpublished, and ongoing) comparing either standard of care with or without docetaxel or standard of care with or without bisphosphonates for men with high-risk localised or metastatic hormone-sensitive prostate cancer. For each trial, we extracted hazard ratios (HRs) of the effects of docetaxel or bisphosphonates on survival (time from randomisation until death from any cause) and failure-free survival (time from randomisation to biochemical or clinical failure or death from any cause) from published trial reports or presentations or obtained them directly from trial investigators. HRs were combined using the fixed-effect model (Mantel-Haenzsel). Findings We identified five eligible randomised controlled trials of docetaxel in men with metastatic (M1) disease. Results from three (CHAARTED, GETUG-15, STAMPEDE) of these trials (2992 [93%] of 3206 men randomised) showed that the addition of docetaxel to standard of care improved survival. The HR of 0·77 (95% CI 0·68–0·87; p<0·0001) translates to an absolute improvement in 4-year survival of 9% (95% CI 5–14). Docetaxel in addition to standard of care also improved failure-free survival, with the HR of 0·64 (0·58–0·70; p<0·0001) translating into a reduction in absolute 4-year failure rates of 16% (95% CI 12–19). We identified 11 trials of docetaxel for men with locally advanced disease (M0). Survival results from three (GETUG-12, RTOG 0521, STAMPEDE) of these trials (2121 [53%] of 3978 men) showed no evidence of a benefit from the addition of docetaxel (HR 0·87 [95% CI 0·69–1·09]; p=0·218), whereas failure-free survival data from four (GETUG-12, RTOG 0521, STAMPEDE, TAX 3501) of these trials (2348 [59%] of 3978 men) showed that docetaxel improved failure-free survival (0·70 [0·61–0·81]; p<0·0001), which translates into a reduced absolute 4-year failure rate of 8% (5–10). We identified seven eligible randomised controlled trials of bisphosphonates for men with M1 disease. Survival results from three of these trials (2740 [88%] of 3109 men) showed that addition of bisphosphonates improved survival (0·88 [0·79–0·98]; p=0·025), which translates to 5% (1–8) absolute improvement, but this result was influenced by the positive result of one trial of sodium clodronate, and we found no evidence of a benefit from the addition of zoledronic acid (0·94 [0·83–1·07]; p=0·323), which translates to an absolute improvement in survival of 2% (−3 to 7). Of 17 trials of bisphosphonates for men with M0 disease, survival results from four trials (4079 [66%] of 6220 men) showed no evidence of benefit from the addition of bisphosphonates (1·03 [0·89–1·18]; p=0·724) or zoledronic acid (0·98 [0·82–1·16]; p=0·782). Failure-free survival definitions were too inconsistent for formal meta-analyses for the bisphosphonate trials. Interpretation The addition of docetaxel to standard of care should be considered standard care for men with M1 hormone-sensitive prostate cancer who are starting treatment for the first time. More evidence on the effects of docetaxel on survival is needed in the M0 disease setting. No evidence exists to suggest that zoledronic acid improves survival in men with M1 or M0 disease, and any potential benefit is probably small. Funding Medical Research Council UK.

Surrogate endpoints for overall survival in chemotherapy and radiotherapy trials in operable and locally advanced lung cancer: A re-analysis of meta-analyses of individual patients' data

Summary Background The gold standard endpoint in clinical trials of chemotherapy and radiotherapy for lung cancer is overall survival. Although reliable and simple to measure, this endpoint takes years to observe. Surrogate endpoints that would enable earlier assessments of treatment effects would be useful. We assessed the correlations between potential surrogate endpoints and overall survival at individual and trial levels. Methods We analysed individual patients data from 15u2008071 patients involved in 60 randomised clinical trials that were assessed in six meta-analyses. Two meta-analyses were of adjuvant chemotherapy in non-small-cell lung cancer, three were of sequential or concurrent chemotherapy, and one was of modified radiotherapy in locally advanced lung cancer. We investigated disease-free survival (DFS) or progression-free survival (PFS), defined as the time from randomisation to local or distant relapse or death, and locoregional control, defined as the time to the first local event, as potential surrogate endpoints. At the individual level we calculated the squared correlations between distributions of these three endpoints and overall survival, and at the trial level we calculated the squared correlation between treatment effects for endpoints. Findings In trials of adjuvant chemotherapy, correlations between DFS and overall survival were very good at the individual level (ρ2=0·83, 95% CI 0·83–0·83 in trials without radiotherapy, and 0·87, 0·87–0·87 in trials with radiotherapy) and excellent at trial level (R2=0·92, 95% CI 0·88–0·95 in trials without radiotherapy and 0·99, 0·98–1·00 in trials with radiotherapy). In studies of locally advanced disease, correlations between PFS and overall survival were very good at the individual level (ρ2 range 0·77–0·85, dependent on the regimen being assessed) and trial level (R2 range 0·89–0·97). In studies with data on locoregional control, individual-level correlations were good (ρ2=0·71, 95% CI 0·71–0·71 for concurrent chemotherapy and ρ2=0·61, 0·61–0·61 for modified vs standard radiotherapy) and trial-level correlations very good (R2=0·85, 95% CI 0·77–0·92 for concurrent chemotherapy and R2=0·95, 0·91–0·98 for modified vs standard radiotherapy). Interpretation We found a high level of evidence that DFS is a valid surrogate endpoint for overall survival in studies of adjuvant chemotherapy involving patients with non-small-cell lung cancers, and PFS in those of chemotherapy and radiotherapy for patients with locally advanced lung cancers. Extrapolation to targeted agents, however, is not automatically warranted. Funding Programme Hospitalier de Recherche Clinique, Ligue Nationale Contre le Cancer, British Medical Research Council, Sanofi-Aventis.

Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use

Jayne Tierney and colleagues offer guidance on how to spot a well-designed and well-conducted individual participant data meta-analysis.

Can the published data tell us about the effectiveness of neoadjuvant chemotherapy for locally advanced cancer of the uterine cervix

The effect of neoadjuvant chemotherapy on survival of patients with locally advanced cervical cancer was investigated by conducting a systematic review and meta-analysis of the published data. Of the 21 randomised trials that we identified, only 15 were published. Furthermore, 2-year survival data could be extracted from only seven trial reports and 3-year survival from only nine trial reports. Meta-analyses of the published data at 2 and 3 years are neither clearly in favour of neoadjuvant chemotherapy nor control (2 years: odds ratio (OR) = 1.09, 95% confidence interval (CI) = 0.83-1.45, P = 0.37; 3 years: OR = 0.96, 95% confidence interval (CI) = 0.73-1.25, P = 0.45). Being restricted to only some of the data from a relatively small fraction of the randomised trials, these analyses potentially suffer from a number of biases and are therefore inconclusive. The only reliable way to judge the value of neoadjuvant chemotherapy in this disease is to perform a meta-analysis of centrally collected, updated, individual data on all patients from all known randomised trials. Such an analysis is currently being carried out by an international collaborative group.

Adding abiraterone to androgen deprivation therapy in men with metastatic hormone-sensitive prostate cancer: a systematic review and meta-analysis

Background There is a need to synthesise the results of numerous randomised controlled trials evaluating the addition of therapies to androgen deprivation therapy (ADT) for men with metastatic hormone-sensitive prostate cancer (mHSPC). This systematic review aims to assess the effects of adding abiraterone acetate plus prednisone/prednisolone (AAP) to ADT. Methods Using our framework for adaptive meta-analysis (FAME), we started the review process before trials had been reported and worked collaboratively with trial investigators to anticipate when eligible trial results would emerge. Thus, we could determine the earliest opportunity for reliable meta-analysis and take account of unavailable trials in interpreting results. We searched multiple sources for trials comparing AAP plus ADT versus ADT in men with mHSPC. We obtained results for the primary outcome of overall survival (OS), secondary outcomes of clinical/radiological progression-free survival (PFS) and grade III–IV and grade V toxicity direct from trial teams. Hazard ratios (HRs) for the effects of AAP plus ADT on OS and PFS, Peto Odds Ratios (Peto ORs) for the effects on acute toxicity and interaction HRs for the effects on OS by patient subgroups were combined across trials using fixed-effect meta-analysis. Findings We identified three eligible trials, one of which was still recruiting (PEACE-1 (NCT01957436)). Results from the two remaining trials (LATITUDE (NCT01715285) and STAMPEDE (NCT00268476)), representing 82% of all men randomised to AAP plus ADT versus ADT (without docetaxel in either arm), showed a highly significant 38% reduction in the risk of death with AAP plus ADT (HR = 0.62, 95% confidence interval [CI] = 0.53–0.71, p = 0.55 × 10−10), that translates into a 14% absolute improvement in 3-year OS. Despite differences in PFS definitions across trials, we also observed a consistent and highly significant 55% reduction in the risk of clinical/radiological PFS (HR = 0.45, 95% CI = 0.40–0.51, p = 0.66 × 10−36) with the addition of AAP, that translates to a 28% absolute improvement at 3 years. There was no evidence of a difference in the OS benefit by Gleason sum score, performance status or nodal status, but the size of the benefit may vary by age. There were more grade III–IV acute cardiac, vascular and hepatic toxicities with AAP plus ADT but no excess of other toxicities or death. Interpretation Adding AAP to ADT is a clinically effective treatment option for men with mHSPC, offering an alternative to docetaxel for men who are starting treatment for the first time. Future research will need to address which of these two agents or whether their combination is most effective, and for whom.

Is there bias in the publication of individual patient data meta-analyses?

OBJECTIVEnThere is increasing empirical evidence for the existence of bias in the publication of primary clinical research, with statistically significant results being published more readily, more quickly, and in higher impact journals. Meta-analysis of individual patient data (IPD) may represent a gold standard of secondary clinical research, giving the best possible summary of current evidence for a particular question, but publication of these may also be subject to bias. This study aimed to explore which factors might be associated with publication of IPD meta-analyses and to identify potential sources of bias.nnnMETHODSnFor all known IPD meta-analysis projects in cancer, the responsible investigator was surveyed by means of a questionnaire to determine descriptive characteristics of the meta-analysis, the nature of the results, and details of the publication history.nnnRESULTSnThere is no good evidence that overall publication status of meta-analyses in cancer is dependent on the statistical or clinical significance of the results. However, those meta-analyses with nonsignificant results did seem to take longer to publish and were published in lower impact journals compared with those with more striking results.nnnCONCLUSIONSnBased on the current data, there seems to be no strong association between the results of IPD meta-analyses in cancer and publication.

Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach?

Identifying which individuals benefit most from particular treatments or other interventions underpins so-called personalised or stratified medicine. However, single trials are typically underpowered for exploring whether participant characteristics, such as age or disease severity, determine an individual’s response to treatment. A meta-analysis of multiple trials, particularly one where individual participant data (IPD) are available, provides greater power to investigate interactions between participant characteristics (covariates) and treatment effects. We use a published IPD meta-analysis to illustrate three broad approaches used for testing such interactions. Based on another systematic review of recently published IPD meta-analyses, we also show that all three approaches can be applied to aggregate data as well as IPD. We also summarise which methods of analysing and presenting interactions are in current use, and describe their advantages and disadvantages. We recommend that testing for interactions using within-trials information alone (the deft approach) becomes standard practice, alongside graphical presentation that directly visualises this.