Jesper Brok

Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses

OBJECTIVES To evaluate meta-analyses with trial sequential analysis (TSA). TSA adjusts for random error risk and provides the required number of participants (information size) in a meta-analysis. Meta-analyses not reaching information size are analyzed with trial sequential monitoring boundaries analogous to interim monitoring boundaries in a single trial. STUDY DESIGN AND SETTING We applied TSA on meta-analyses performed in Cochrane Neonatal reviews. We calculated information sizes and monitoring boundaries with three different anticipated intervention effects of 30% relative risk reduction (TSA(30%)), 15% (TSA(15%)), or a risk reduction suggested by low-bias risk trials of the meta-analysis corrected for heterogeneity (TSA(LBHIS)). RESULTS A total of 174 meta-analyses were eligible; 79 out of 174 (45%) meta-analyses were statistically significant (P<0.05). In the significant meta-analyses, TSA(30%) showed firm evidence in 61%. TSA(15%) and TSA(LBHIS) found firm evidence in 33% and 73%, respectively. The remaining significant meta-analyses had potentially spurious evidence of effect. In the 95 statistically nonsignificant (P>or=0.05) meta-analyses, TSA(30%) showed absence of evidence in 80% (insufficient information size). TSA(15%) and TSA(LBHIS) found that 95% and 91% had absence of evidence. The remaining nonsignificant meta-analyses had evidence of lack of effect. CONCLUSION TSA reveals insufficient information size and potentially false positive results in many meta-analyses.

Estimating required information size by quantifying diversity in random-effects model meta-analyses

BackgroundThere is increasing awareness that meta-analyses require a sufficiently large information size to detect or reject an anticipated intervention effect. The required information size in a meta-analysis may be calculated from an anticipated a priori intervention effect or from an intervention effect suggested by trials with low-risk of bias.MethodsInformation size calculations need to consider the total model variance in a meta-analysis to control type I and type II errors. Here, we derive an adjusting factor for the required information size under any random-effects model meta-analysis.ResultsWe devise a measure of diversity (D2) in a meta-analysis, which is the relative variance reduction when the meta-analysis model is changed from a random-effects into a fixed-effect model. D2 is the percentage that the between-trial variability constitutes of the sum of the between-trial variability and a sampling error estimate considering the required information size. D2 is different from the intuitively obvious adjusting factor based on the common quantification of heterogeneity, the inconsistency (I2), which may underestimate the required information size. Thus, D2 and I2 are compared and interpreted using several simulations and clinical examples. In addition we show mathematically that diversity is equal to or greater than inconsistency, that is D2 ≥ I2, for all meta-analyses.ConclusionWe conclude that D2 seems a better alternative than I2 to consider model variation in any random-effects meta-analysis despite the choice of the between trial variance estimator that constitutes the model. Furthermore, D2 can readily adjust the required information size in any random-effects model meta-analysis.

Apparently conclusive meta-analyses may be inconclusive—Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses

BACKGROUND Random error may cause misleading evidence in meta-analyses. The required number of participants in a meta-analysis (i.e. information size) should be at least as large as an adequately powered single trial. Trial sequential analysis (TSA) may reduce risk of random errors due to repetitive testing of accumulating data by evaluating meta-analyses not reaching the information size with monitoring boundaries. This is analogous to sequential monitoring boundaries in a single trial. METHODS We selected apparently conclusive (P </= 0.05) Cochrane neonatal meta-analyses. We applied heterogeneity-adjusted and unadjusted TSA on these meta-analyses by calculating the information size, the monitoring boundaries, and the cumulative Z-statistic after each trial. We identified the proportion of meta-analyses that did not reach the required information size and the proportion of these meta-analyses in which the Z-curve did not cross the monitoring boundaries. RESULTS Of 54 apparently conclusive meta-analyses, 39 (72%) did not reach the heterogeneity-adjusted information size required to accept or reject an intervention effect of 25% relative risk reduction. Of these 39, 19 meta-analyses (49%) were considered inconclusive, because the cumulative Z-curve did not cross the monitoring boundaries. The median number of participants required to reach the required information size was 1591 (range, 339-6149). TSA without heterogeneity adjustment largely confirmed these results. CONCLUSIONS Many apparently conclusive Cochrane neonatal meta-analyses may become inconclusive when the statistical analyses take into account the risk of random error due to repetitive testing.

Effect of hepatitis B immunisation in newborn infants of mothers positive for hepatitis B surface antigen: systematic review and meta-analysis

Abstract Objective To evaluate the effects of hepatitis B vaccine and immunoglobulin in newborn infants of mothers positive for hepatitis B surface antigen. Design Systematic review and meta-analysis of randomised clinical trials. Data sources Electronic databases and hand searches. Review methods Randomised clinical trials were assessed for methodological quality. Meta-analysis was undertaken on three outcomes: the relative risks of hepatitis B occurrence, antibody levels to hepatitis B surface antigen, and adverse events. Results 29 randomised clinical trials were identified, five of which were considered high quality. Only three trials reported inclusion of mothers negative for hepatitis B e antigen. Compared with placebo or no intervention, vaccination reduced the occurrence of hepatitis B (relative risk 0.28, 95% confidence interval 0.20 to 0.40; four trials). No significant difference in hepatitis B occurrence was found between recombinant vaccine and plasma derived vaccine (1.00, 0.71 to 1.42; four trials) and between high dose versus low dose vaccine (plasma derived vaccine 0.97, 0.55 to 1.68, three trials; recombinant vaccine 0.78, 0.31 to 1.94, one trial). Compared with placebo or no intervention, hepatitis B immunoglobulin or the combination of plasma derived vaccine and hepatitis B immunoglobulin reduced hepatitis B occurrence (immunoglobulin 0.50, 0.41 to 0.60, one trial; vaccine and immunoglobulin 0.08, 0.03 to 0.17, three trials). Compared with vaccine alone, vaccine plus hepatitis B immunoglobulin reduced hepatitis B occurrence (0.54, 0.41 to 0.73; 10 trials). Hepatitis B vaccine and hepatitis B immunoglobulin seem safe, but few trials reported adverse events. Conclusion Hepatitis B vaccine, hepatitis B immunoglobulin, and vaccine plus immunoglobulin prevent hepatitis B occurrence in newborn infants of mothers positive for hepatitis B surface antigen.

Inhaled nitric oxide for acute respiratory distress syndrome and acute lung injury in adults and children: a systematic review with meta-analysis and trial sequential analysis.

BACKGROUND: Acute hypoxemic respiratory failure, defined as acute lung injury and acute respiratory distress syndrome, are critical conditions associated with frequent mortality and morbidity in all ages. Inhaled nitric oxide (iNO) has been used to improve oxygenation, but its role remains controversial. We performed a systematic review with meta-analysis and trial sequential analysis of randomized clinical trials (RCTs). We searched CENTRAL, Medline, Embase, International Web of Science, LILACS, the Chinese Biomedical Literature Database, and CINHAL (up to January 31, 2010). Additionally, we hand-searched reference lists, contacted authors and experts, and searched registers of ongoing trials. Two reviewers independently selected all parallel group RCTs comparing iNO with placebo or no intervention and extracted data related to study methods, interventions, outcomes, bias risk, and adverse events. All trials, irrespective of blinding or language status were included. Retrieved trials were evaluated with Cochrane methodology. Disagreements were resolved by discussion. Our primary outcome measure was all-cause mortality. We performed subgroup and sensitivity analyses to assess the effect of iNO in adults and children and on various clinical and physiological outcomes. We assessed the risk of bias through assessment of trial methodological components. We assessed the risk of random error by applying trial sequential analysis. RESULTS: We included 14 RCTs with a total of 1303 participants; 10 of these trials had a high risk of bias. iNO showed no statistically significant effect on overall mortality (40.2%versus 38.6%) (relative risks [RR] 1.06, 95% confidence interval [CI] 0.93 to 1.22; I2 = 0) and in several subgroup and sensitivity analyses, indicating robust results. Limited data demonstrated a statistically insignificant effect of iNO on duration of ventilation, ventilator-free days, and length of stay in the intensive care unit and hospital. We found a statistically significant but transient improvement in oxygenation in the first 24 hours, expressed as the ratio of PO2 to fraction of inspired oxygen (mean difference [MD] 15.91, 95% CI 8.25 to 23.56; I2 = 25%). However, iNO appears to increase the risk of renal impairment among adults (RR 1.59, 95% CI 1.17 to 2.16; I2 = 0) but not the risk of bleeding or methemoglobin or nitrogen dioxide formation. CONCLUSION: iNO cannot be recommended for patients with acute hypoxemic respiratory failure. iNO results in a transient improvement in oxygenation but does not reduce mortality and may be harmful.

Antithrombin III in critically ill patients : systematic review with meta-analysis and trial sequential analysis

Objective To evaluate the benefits and harms of antithrombin III in critically ill patients. Design Systematic review and meta-analysis of randomised trials. Data sources CENTRAL, Medline, Embase, International Web of Science, LILACS, the Chinese Biomedical Literature Database, and CINHAL (to November 2006); hand search of reference lists, contact with authors and experts, and search of registers of ongoing trials. Review methods Two reviewers independently selected parallel group randomised clinical trials comparing antithrombin with placebo or no intervention and extracted data related to study methods, interventions, outcomes, bias risk, and adverse events. Disagreements were resolved by discussion. Trials in any type of critically ill patients in intensive care were eligible. All trials, irrespective of blinding or language status, that compared any antithrombin III regimen with no intervention or placebo were included. Trials were considered to be at low risk of bias if they had adequate randomisation procedure, blinding, and used intention to treat analysis. Risk ratios with 95% confidence intervals were estimated with fixed and random effects models according to heterogeneity. Main outcome measures Mortality, length of stay in intensive care or hospital, quality of life, severity of sepsis, respiratory failure, duration of mechanical ventilation, incidence of surgical intervention, intervention effect among various populations, and adverse events (such as bleeding). Results 20 trials randomly assigning 3458 patients met inclusion criteria. Eight trials had low risk of bias. Compared with placebo or no intervention, antithrombin III did not reduce overall mortality (relative risk 0.96, 95% confidence interval 0.89 to 1.03). No subgroup analyses on risk of bias, populations of patients, or with and without adjuvant heparin yielded significant results. Antithrombin III increased the risk of bleeding events (1.52, 1.30 to 1.78). Heterogeneity was observed in only a few analyses. Conclusion Antithrombin III cannot be recommended for critically ill patients based on the available evidence.

Monitoring patients at risk of massive transfusion with Thrombelastography or Thromboelastometry: a systematic review.

Thrombelastography (TEG) and Thrombelastometry (ROTEM) are viscoelastic whole‐blood assays evaluating the haemostatic capacity of blood. These devices are used in algorithms to guide transfusion of haemostatic blood components.

Cochrane systematic review: pegylated interferon plus ribavirin vs. interferon plus ribavirin for chronic hepatitis C.

Background  About 170 million patients worldwide have chronic hepatitis C. Pegylated interferon plus ribavirin is currently the recommended therapy.

Prophylactic Probiotics for Preterm Infants: A Systematic Review and Meta-Analysis of Observational Studies

Background: Necrotizing enterocolitis (NEC) is a major morbidity and cause of mortality in preterm neonates. Probiotics seem to have a beneficial role in preventing NEC, which is confirmed in meta-analyses of randomized controlled trials (RCTs). We therefore aimed to review and confirm the efficacy of probiotics in preterm neonates obtained in observational studies. Objective: To assess the effects of prophylactic probiotics in preterm infants. Methods: A meta-analysis was performed searching PubMed, EMBASE, CENTRAL (the Cochrane Library) and www.clinicaltrials.gov. Reference lists of reviews of RCTs were also searched. Included studies were observational studies that enrolled preterm infants <37 weeks of gestational age. Trials were included if they administered any probiotics and measured at least one clinical outcome (e.g. NEC, all-cause mortality, sepsis or long-term development scores). Two authors extracted characteristics and outcomes from included studies. The Newcastle-Ottawa Scale was used for quality assessment. A random-effects meta-analysis model was used, and heterogeneity was assessed by the I2 test. Results: We included 12 studies with 10,800 premature neonates (5,144 receiving prophylactic probiotics and 5,656 controls). The meta-analysis showed a significantly decreased incidence of NEC (risk ratio, RR = 0.55, 95% confidence interval, 95% CI, 0.39-0.78; p = 0.0006) and mortality (RR = 0.72, 95% CI, 0.61-0.85; p < 0.0001). Sepsis did not differ significantly between the two groups (RR = 0.86, 95% CI, 0.74-1.00; p = 0.05). Conclusions: Probiotic supplementation reduces the risk of NEC and mortality in preterm infants. The effect sizes are similar to findings in meta-analyses of RCTs. However, the optimal strain, dose and timing need further investigation.

Ribavirin Monotherapy for Chronic Hepatitis C Infection: A Cochrane Hepato-Biliary Group Systematic Review and Meta-Analysis of Randomized Trials

OBJECTIVES:Adding ribavirin to interferon improves treatment response for patients with chronic hepatitis C, but the effects of ribavirin monotherapy are unclear. We conducted a systematic review to assess the benefits and harms of ribavirin monotherapy for patients with chronic hepatitis C.METHODS:Randomized trials were identified through the Cochrane Hepato-Biliary Group Controlled Trials Register, The Cochrane Library, MEDLINE, and EMBASE (last search May 2005). The primary outcomes were sustained virological response (loss of HCV RNA) and liver-related morbidity plus all-cause mortality. Secondary outcomes included end-of-treatment virological response, biochemical response (normalization of transaminases), histological response, and adverse events.RESULTS:We included 11 randomized trials with 521 patients. Ten trials had unclear control of bias. Ribavirin had no significant effect on sustained (risk difference (RD), 0%; 95% confidence intervals (CI), −2% to 3%) or end-of-treatment virological response (RD, 0%; 95% CI, −3% to 3%). Ribavirin had no significant effect on liver-related morbidity plus mortality (RD, 0%; 95% CI, −2% to 3%). Ribavirin significantly improved histological response and end-of-treatment biochemical response, but not sustained biochemical response. Ribavirin significantly increased the risk of anemia and treatment discontinuation.CONCLUSIONS:We found no evidence to support ribavirin monotherapy for patients with chronic hepatitis C, but trials were small and type II errors cannot be excluded. Patients intolerant to interferon should not continue treatment with ribavirin alone outside randomized trials.