James Weir

Prediction of outcome after moderate and severe traumatic brain injury: External validation of the International Mission on Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models*

Objective: The International Mission on Prognosis and Analysis of Clinical Trials and Corticoid Randomisation After Significant Head injury prognostic models predict outcome after traumatic brain injury but have not been compared in large datasets. The objective of this is study is to validate externally and compare the International Mission on Prognosis and Analysis of Clinical Trials and Corticoid Randomisation after Significant Head injury prognostic models for prediction of outcome after moderate or severe traumatic brain injury. Design: External validation study. Patients: We considered five new datasets with a total of 9,036 patients, comprising three randomized trials and two observational series, containing prospectively collected individual traumatic brain injury patient data. Measurements and Main Results: Outcomes were mortality and unfavorable outcome, based on the Glasgow Outcome Score at 6 months after injury. To assess performance, we studied the discrimination of the models (by area under the receiver operating characteristic curves), and calibration (by comparison of the mean observed to predicted outcomes and calibration slopes). The highest discrimination was found in the Trauma Audit and Research Network trauma registry (area under the receiver operating characteristic curves between 0.83 and 0.87), and the lowest discrimination in the Pharmos trial (area under the receiver operating characteristic curves between 0.65 and 0.71). Although differences in predictor effects between development and validation populations were found (calibration slopes varying between 0.58 and 1.53), the differences in discrimination were largely explained by differences in case mix in the validation studies. Calibration was good, the fraction of observed outcomes generally agreed well with the mean predicted outcome. No meaningful differences were noted in performance between the International Mission on Prognosis and Analysis of Clinical Trials and Corticoid Randomisation After Significant Head injury models. More complex models discriminated slightly better than simpler variants. Conclusions: Since both the International Mission on Prognosis and Analysis of Clinical Trials and the Corticoid Randomisation After Significant Head injury prognostic models show good generalizability to more recent data, they are valid instruments to quantify prognosis in traumatic brain injury.

IMPACT recommendations for improving the design and analysis of clinical trials in moderate to severe traumatic brain injury

SummaryClinical trials in traumatic brain injury (TBI) pose complex methodological challenges, largely related to the heterogeneity of the population. The International Mission on Prognosis and Clinical Trial Design in TBI study group has explored approaches for dealing with this heterogeneity with the aim to optimize clinical trials in TBI. Extensive prognostic analyses and simulation studies were conducted on individual patient data from eight trials and three observational studies. Here, we integrate the results of these studies into the International Mission on Prognosis and Clinical Trial Design in TBI recommendations for design and analysis of trials in TBI:• Details of the major baseline prognostic characteristics should be provided in every report on a TBI study; in trials they should be differentiated per treatment group. We also advocate the reporting of the baseline prognostic risk as determined by validated prognostic models.• Inclusion criteria should be as broad as is compatible with the current understanding of the mechanisms of action of the intervention being evaluated. This will maximize recruitment rates and enhance the generalizability of the results.• The statistical analysis should incorporate prespecified covariate adjustment to mitigate the effects of the heterogeneity.• The statistical analysis should use an ordinal approach, based on either sliding dichotomy or proportional odds methodology. Broad inclusion criteria, prespecified covariate adjustment, and an ordinal analysis will promote an efficient trial, yielding gains in statistical efficiency of more than 40%. This corresponds to being able to detect a 7% treatment effect with the same number of patients needed to demonstrate a 10% difference with an unadjusted analysis based on the dichotomized Glasgow outcome scale.

Advancing care for traumatic brain injury: findings from the IMPACT studies and perspectives on future research

Research in traumatic brain injury (TBI) is challenging for several reasons; in particular, the heterogeneity between patients regarding causes, pathophysiology, treatment, and outcome. Advances in basic science have failed to translate into successful clinical treatments, and the evidence underpinning guideline recommendations is weak. Because clinical research has been hampered by non-standardised data collection, restricted multidisciplinary collaboration, and the lack of sensitivity of classification and efficacy analyses, multidisciplinary collaborations are now being fostered. Approaches to deal with heterogeneity have been developed by the IMPACT study group. These approaches can increase statistical power in clinical trials by up to 50% and are also relevant to other heterogeneous neurological diseases, such as stroke and subarachnoid haemorrhage. Rather than trying to limit heterogeneity, we might also be able to exploit it by analysing differences in treatment and outcome between countries and centres in comparative effectiveness research. This approach has great potential to advance care in patients with TBI.

A simulation study evaluating approaches to the analysis of ordinal outcome data in randomized controlled trials in traumatic brain injury: results from the IMPACT Project

Background Clinical trials in traumatic brain injury have a disappointing track record, with a long history of ‘negative’ Phase III trials. One contributor to this lack of success is almost certainly the low efficiency of the conventional approach to the analysis, which discards information by dichotomizing an ordinal outcome scale. Purpose Our goal was to evaluate the potential efficiency gains, which can be achieved by using techniques, which extract additional information from ordinal outcome data — the proportional odds model and the sliding dichotomy. In addition, we evaluated the additional efficiency gains, which can be achieved through covariate adjustment. Methods The study was based on simulations, which were built around a database of patient-level data extracted from eight Phase III trials and three observational studies in traumatic brain injury. Two different putative treatment effects were explored, one which followed the proportional odds model, and the other which assumed that the effect of the intervention was to reduce the risk of death without changing the distribution of outcomes within survivors. The results are expressed as efficiency gains, reported as the percentage reduction in sample size that can be used with the ordinal analyses without loss of statistical power relative to the conventional binary analysis. Results The simulation results show substantial efficiency gains. Use of the sliding dichotomy allows sample sizes to be reduced by up to 40% without loss of statistical power. The proportional odds model gives modest additional gains over and above the gains achieved by use of the sliding dichotomy. Limitations As with any simulation study, it is difficult to know how far the findings may be extrapolated beyond the actual situations that were modeled. Conclusions Both ordinal techniques offer substantial efficiency gains relative to the conventional binary analysis. The choice between the two techniques involves subtle value judgments. In the situations examined, the proportional odds model gave efficiency gains over and above the sliding dichotomy, but arguably, the sliding dichotomy is more intuitive and clinically appealing. Clinical Trials 2010; 7: 44—57. http://ctj.sagepub.com

Large between-center differences in outcome after moderate and severe traumatic brain injury in the international mission on prognosis and clinical trial design in traumatic brain injury (IMPACT) study.

BACKGROUND:Differences between centers in patient outcome after traumatic brain injury are of importance for multicenter studies and have seldom been studied. OBJECTIVE:To quantify the differences in centers enrolling patients in randomized clinical trials (RCTs) and surveys. METHODS:We analyzed individual patient data from 9578 patients with moderate and severe traumatic brain injury enrolled in 10 RCTs and 3 observational studies. We used random-effects logistic regression models to estimate the between-center differences in unfavorable outcome (dead, vegetative state, or severe disability measured with the Glasgow Outcome Scale) at 6 months adjusted for differences in patient characteristics. We calculated the difference in odds of unfavorable outcome between the centers at the higher end vs those at the lower end of the outcome distribution. We analyzed the total database, Europe and the United States separately, and 4 larger RCTs. RESULTS:The 9578 patients were enrolled at 265 centers, and 4629 (48%) had an unfavorable outcome. After adjustment for patient characteristics, there was a 3.3-fold difference in the odds of unfavorable outcome between the centers at the lower end of the outcome distribution (2.5th percentile) vs those at the higher end of the outcome distribution (97.5th percentile; P < .001). In the 4 larger RCTs, the differences between centers were similar. However, differences were smaller between centers in the United States (2.4-fold) than between centers in Europe (3.8-fold). CONCLUSION:Outcome after traumatic brain injury differs substantially between centers, particularly in Europe. Further research is needed to study explanations for these differences to suggest where quality of care might be improved.

Prognostic value of major extracranial injury in traumatic brain injury: an individual patient data meta-analysis in 39,274 patients.

BACKGROUND: Major extracranial injury (MEI) is common in traumatic brain injury (TBI) patients, but the effect on outcome is controversial. OBJECTIVE: To assess the prognostic value of MEI on mortality after TBI in an individual patient data meta-analysis of 3 observational TBI studies (International Mission on Prognosis and Clinical Trial Design in TBI [IMPACT]), a randomized controlled trial (Corticosteroid Randomization After Significant Head Injury [CRASH]), and a trauma registry (Trauma Audit and Research Network [TARN]). METHODS: MEI (extracranial injury with an Abbreviated Injury Scale ≥ 3 or requiring hospital admission) was related to mortality with logistic regression analysis, adjusted for age, Glasgow Coma Scale motor score, and pupil reactivity and stratified by TBI severity. We pooled odds ratios (ORs) with random-effects meta-analysis. RESULTS: We included 39 274 patients. Mortality was 25%, and 32% had MEI. MEI was a strong predictor for mortality in TARN, with adjusted odds ratios of 2.81 (95% confidence interval [CI], 2.44-3.23) in mild, 2.18 (95% CI, 1.80-2.65) in moderate, and 2.14 (95% CI, 1.95-2.35) in severe TBI patients. The prognostic effect was smaller in IMPACT and CRASH, with pooled adjusted odds ratios of 2.14 (95% CI, 0.93-4.91) in mild, 1.46 (95% CI, 1.14-1.85) in moderate, and 1.18 (95% CI, 1.03-1.55) in severe TBI. When patients who died within 6 hours after injury were excluded from TARN, the effect of MEI was comparable with IMPACT and CRASH. CONCLUSION: MEI is an important prognostic factor for mortality in TBI patients. However, the effect varies by population, which explains the controversy in the literature. The strength of the effect is smaller in patients with more severe brain injury and depends on time of inclusion in a study.

Does the Extended Glasgow Outcome Scale Add Value to the Conventional Glasgow Outcome Scale

The Glasgow Outcome Scale (GOS) is firmly established as the primary outcome measure for use in Phase III trials of interventions in traumatic brain injury (TBI). However, the GOS has been criticized for its lack of sensitivity to detect small but clinically relevant changes in outcome. The Glasgow Outcome Scale-Extended (GOSE) potentially addresses this criticism, and in this study we estimate the efficiency gain associated with using the GOSE in place of the GOS in ordinal analysis of 6-month outcome. The study uses both simulation and the reanalysis of existing data from two completed TBI studies, one an observational cohort study and the other a randomized controlled trial. As expected, the results show that using an ordinal technique to analyze the GOS gives a substantial gain in efficiency relative to the conventional analysis, which collapses the GOS onto a binary scale (favorable versus unfavorable outcome). We also found that using the GOSE gave a modest but consistent increase in efficiency relative to the GOS in both studies, corresponding to a reduction in the required sample size of the order of 3-5%. We recommend that the GOSE be used in place of the GOS as the primary outcome measure in trials of TBI, with an appropriate ordinal approach being taken to the statistical analysis.

Practical methods for ordinal data meta-analysis in stroke

Ordinal outcomes are common in medical research. However, in meta-analyses, they are routinely analysed as binary outcomes. The aim of this study was to compare fixed effects meta-analyses using ordinal methods (proportional odds) with binary methods (binary logistic regression) and assess any changes in conclusions and gains in precision. The Modified Rankin Scale (mRS) is a commonly used outcome measure in stroke trials. It comprises six disability levels, and death. We examined all 132 systematic reviews of interventions published by the Cochrane Stroke Group in the Cochrane Library, 2010 Issue 11, to find included trials that measured the mRS. The final analysis was based on 216 studies from 24 systematic reviews. Studies reported mRS results to a varying level of detail, from binary only, to all seven possible categories. Initially, all studies were treated as if the outcome was binary. They were then analysed using three categories, four categories, and so on. If a study did not report enough detail for a given analysis, it was included using as much detail as possible. The standard error (SE) of the estimated pooled common odds ratio for the three-category analysis was smaller than two categories (mean ratio of SEs=0.94). Four categories was marginally better than three (mean ratio of SEs=0.99) but adding further categories was not further beneficial. In 2/24 (8%) reviews, the conclusions changed from favouring control or treatment group to no evidence of a difference. In the remainder of reviews, the conclusions did not change. Ordinal methods should be considered when performing meta-analyses.