José Antonio López-López

Three-level meta-analysis of dependent effect sizes

Although dependence in effect sizes is ubiquitous, commonly used meta-analytic methods assume independent effect sizes. We describe and illustrate three-level extensions of a mixed effects meta-analytic model that accounts for various sources of dependence within and across studies, because multilevel extensions of meta-analytic models still are not well known. We also present a three-level model for the common case where, within studies, multiple effect sizes are calculated using the same sample. Whereas this approach is relatively simple and does not require imputing values for the unknown sampling covariances, it has hardly been used, and its performance has not been empirically investigated. Therefore, we set up a simulation study, showing that also in this situation, a three-level approach yields valid results: Estimates of the treatment effects and the corresponding standard errors are unbiased.

Meta-analysis of multiple outcomes: a multilevel approach

In meta-analysis, dependent effect sizes are very common. An example is where in one or more studies the effect of an intervention is evaluated on multiple outcome variables for the same sample of participants. In this paper, we evaluate a three-level meta-analytic model to account for this kind of dependence, extending the simulation results of Van den Noortgate, López-López, Marín-Martínez, and Sánchez-Meca Behavior Research Methods, 45, 576–594 (2013) by allowing for a variation in the number of effect sizes per study, in the between-study variance, in the correlations between pairs of outcomes, and in the sample size of the studies. At the same time, we explore the performance of the approach if the outcomes used in a study can be regarded as a random sample from a population of outcomes. We conclude that although this approach is relatively simple and does not require prior estimates of the sampling covariances between effect sizes, it gives appropriate mean effect size estimates, standard error estimates, and confidence interval coverage proportions in a variety of realistic situations.

Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review.

Objectives To evaluate the effects on non-specific and all cause mortality, in children under 5, of Bacillus Calmette-Guérin (BCG), diphtheria-tetanus-pertussis (DTP), and standard titre measles containing vaccines (MCV); to examine internal validity of the studies; and to examine any modifying effects of sex, age, vaccine sequence, and co-administration of vitamin A. Design Systematic review, including assessment of risk of bias, and meta-analyses of similar studies. Study eligibility criteria Clinical trials, cohort studies, and case-control studies of the effects on mortality of BCG, whole cell DTP, and standard titre MCV in children under 5. Data sources Searches of Medline, Embase, Global Index Medicus, and the WHO International Clinical Trials Registry Platform, supplemented by contact with experts in the field. To avoid overlap in children studied across the included articles, findings from non-overlapping birth cohorts were identified. Results Results from 34 birth cohorts were identified. Most evidence was from observational studies, with some from short term clinical trials. Most studies reported on all cause (rather than non-specific) mortality. Receipt of BCG vaccine was associated with a reduction in all cause mortality: the average relative risks were 0.70 (95% confidence interval 0.49 to 1.01) from five clinical trials and 0.47 (0.32 to 0.69) from nine observational studies at high risk of bias. Receipt of DTP (almost always with oral polio vaccine) was associated with a possible increase in all cause mortality on average (relative risk 1.38, 0.92 to 2.08) from 10 studies at high risk of bias; this effect seemed stronger in girls than in boys. Receipt of standard titre MCV was associated with a reduction in all cause mortality (relative risks 0.74 (0.51 to 1.07) from four clinical trials and 0.51 (0.42 to 0.63) from 18 observational studies at high risk of bias); this effect seemed stronger in girls than in boys. Seven observational studies, assessed as being at high risk of bias, have compared sequences of vaccines; results of a subset of these suggest that administering DTP with or after MCV may be associated with higher mortality than administering it before MCV. Conclusions Evidence suggests that receipt of BCG and MCV reduce overall mortality by more than would be expected through their effects on the diseases they prevent, and receipt of DTP may be associated with an increase in all cause mortality. Although efforts should be made to ensure that all children are immunised on schedule with BCG, DTP, and MCV, randomised trials are needed to compare the effects of different sequences.

A Comparison of Procedures to Test for Moderators in Mixed-Effects Meta-Regression Models

Several alternative methods are available when testing for moderators in mixed-effects meta-regression models. A simulation study was carried out to compare different methods in terms of their Type I error and statistical power rates. We included the standard (Wald-type) test, the method proposed by Knapp and Hartung (2003) in 2 different versions, the Huber-White method, the likelihood ratio test, and the permutation test in the simulation study. These methods were combined with 7 estimators for the amount of residual heterogeneity in the effect sizes. Our results show that the standard method, applied in most meta-analyses up to date, does not control the Type I error rate adequately, sometimes leading to overly conservative, but usually to inflated, Type I error rates. Of the different methods evaluated, only the Knapp and Hartung method and the permutation test provide adequate control of the Type I error rate across all conditions. Due to its computational simplicity, the Knapp and Hartung method is recommended as a suitable option for most meta-analyses.

Estimation of the predictive power of the model in mixed-effects meta-regression: A simulation study

Several methods are available to estimate the total and residual amount of heterogeneity in meta-analysis, leading to different alternatives when estimating the predictive power in mixed-effects meta-regression models using the formula proposed by Raudenbush (1994, 2009). In this paper, a simulation study was conducted to compare the performance of seven estimators of these parameters under various realistic scenarios in psychology and related fields. Our results suggest that the number of studies (k) exerts the most important influence on the accuracy of the results, and that precise estimates of the heterogeneity variances and the model predictive power can only be expected with at least 20 and 40 studies, respectively. Increases in the average within-study sample size (N¯) also improved the results for all estimators. Some differences among the accuracy of the estimators were observed, especially under adverse (small k and N¯) conditions, while the results for the different methods tended to convergence for more optimal scenarios.

Oral anticoagulants for primary prevention, treatment and secondary prevention of venous thromboembolic disease, and for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis and cost-effectiveness analysis

BACKGROUND Warfarin is effective for stroke prevention in atrial fibrillation (AF), but anticoagulation is underused in clinical care. The risk of venous thromboembolic disease during hospitalisation can be reduced by low-molecular-weight heparin (LMWH): warfarin is the most frequently prescribed anticoagulant for treatment and secondary prevention of venous thromboembolism (VTE). Warfarin-related bleeding is a major reason for hospitalisation for adverse drug effects. Warfarin is cheap but therapeutic monitoring increases treatment costs. Novel oral anticoagulants (NOACs) have more rapid onset and offset of action than warfarin, and more predictable dosing requirements. OBJECTIVE To determine the best oral anticoagulant/s for prevention of stroke in AF and for primary prevention, treatment and secondary prevention of VTE. DESIGN Four systematic reviews, network meta-analyses (NMAs) and cost-effectiveness analyses (CEAs) of randomised controlled trials. SETTING Hospital (VTE primary prevention and acute treatment) and primary care/anticoagulation clinics (AF and VTE secondary prevention). PARTICIPANTS Patients eligible for anticoagulation with warfarin (stroke prevention in AF, acute treatment or secondary prevention of VTE) or LMWH (primary prevention of VTE). INTERVENTIONS NOACs, warfarin and LMWH, together with other interventions (antiplatelet therapy, placebo) evaluated in the evidence network. MAIN OUTCOME MEASURES Efficacy Stroke, symptomatic VTE, symptomatic deep-vein thrombosis and symptomatic pulmonary embolism. Safety Major bleeding, clinically relevant bleeding and intracranial haemorrhage. We also considered myocardial infarction and all-cause mortality and evaluated cost-effectiveness. DATA SOURCES MEDLINE and PREMEDLINE In-Process & Other Non-Indexed Citations, EMBASE and The Cochrane Library, reference lists of published NMAs and trial registries. We searched MEDLINE and PREMEDLINE In-Process & Other Non-Indexed Citations, EMBASE and The Cochrane Library. The stroke prevention in AF review search was run on the 12 March 2014 and updated on 15 September 2014, and covered the period 2010 to September 2014. The search for the three reviews in VTE was run on the 19 March 2014, updated on 15 September 2014, and covered the period 2008 to September 2014. REVIEW METHODS Two reviewers screened search results, extracted and checked data, and assessed risk of bias. For each outcome we conducted standard meta-analysis and NMA. We evaluated cost-effectiveness using discrete-time Markov models. RESULTS Apixaban (Eliquis®, Bristol-Myers Squibb, USA; Pfizer, USA) [5 mg bd (twice daily)] was ranked as among the best interventions for stroke prevention in AF, and had the highest expected net benefit. Edoxaban (Lixiana®, Daiichi Sankyo, Japan) [60 mg od (once daily)] was ranked second for major bleeding and all-cause mortality. Neither the clinical effectiveness analysis nor the CEA provided strong evidence that NOACs should replace postoperative LMWH in primary prevention of VTE. For acute treatment and secondary prevention of VTE, we found little evidence that NOACs offer an efficacy advantage over warfarin, but the risk of bleeding complications was lower for some NOACs than for warfarin. For a willingness-to-pay threshold of > £5000, apixaban (5 mg bd) had the highest expected net benefit for acute treatment of VTE. Aspirin or no pharmacotherapy were likely to be the most cost-effective interventions for secondary prevention of VTE: our results suggest that it is not cost-effective to prescribe NOACs or warfarin for this indication. CONCLUSIONS NOACs have advantages over warfarin in patients with AF, but we found no strong evidence that they should replace warfarin or LMWH in primary prevention, treatment or secondary prevention of VTE. LIMITATIONS These relate mainly to shortfalls in the primary data: in particular, there were no head-to-head comparisons between different NOAC drugs. FUTURE WORK Calculating the expected value of sample information to clarify whether or not it would be justifiable to fund one or more head-to-head trials. STUDY REGISTRATION This study is registered as PROSPERO CRD42013005324, CRD42013005331 and CRD42013005330. FUNDING The National Institute for Health Research Health Technology Assessment programme.

Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis

Abstract Objective To compare the efficacy, safety, and cost effectiveness of direct acting oral anticoagulants (DOACs) for patients with atrial fibrillation. Design Systematic review, network meta-analysis, and cost effectiveness analysis. Data sources Medline, PreMedline, Embase, and The Cochrane Library. Eligibility criteria for selecting studies Published randomised trials evaluating the use of a DOAC, vitamin K antagonist, or antiplatelet drug for prevention of stroke in patients with atrial fibrillation. Results 23 randomised trials involving 94 656 patients were analysed: 13 compared a DOAC with warfarin dosed to achieve a target INR of 2.0-3.0. Apixaban 5 mg twice daily (odds ratio 0.79, 95% confidence interval 0.66 to 0.94), dabigatran 150 mg twice daily (0.65, 0.52 to 0.81), edoxaban 60 mg once daily (0.86, 0.74 to 1.01), and rivaroxaban 20 mg once daily (0.88, 0.74 to 1.03) reduced the risk of stroke or systemic embolism compared with warfarin. The risk of stroke or systemic embolism was higher with edoxaban 60 mg once daily (1.33, 1.02 to 1.75) and rivaroxaban 20 mg once daily (1.35, 1.03 to 1.78) than with dabigatran 150 mg twice daily. The risk of all-cause mortality was lower with all DOACs than with warfarin. Apixaban 5 mg twice daily (0.71, 0.61 to 0.81), dabigatran 110 mg twice daily (0.80, 0.69 to 0.93), edoxaban 30 mg once daily (0.46, 0.40 to 0.54), and edoxaban 60 mg once daily (0.78, 0.69 to 0.90) reduced the risk of major bleeding compared with warfarin. The risk of major bleeding was higher with dabigatran 150 mg twice daily than apixaban 5 mg twice daily (1.33, 1.09 to 1.62), rivaroxaban 20 mg twice daily than apixaban 5 mg twice daily (1.45, 1.19 to 1.78), and rivaroxaban 20 mg twice daily than edoxaban 60 mg once daily (1.31, 1.07 to 1.59). The risk of intracranial bleeding was substantially lower for most DOACs compared with warfarin, whereas the risk of gastrointestinal bleeding was higher with some DOACs than warfarin. Apixaban 5 mg twice daily was ranked the highest for most outcomes, and was cost effective compared with warfarin. Conclusions The network meta-analysis informs the choice of DOACs for prevention of stroke in patients with atrial fibrillation. Several DOACs are of net benefit compared with warfarin. A trial directly comparing DOACs would overcome the need for indirect comparisons to be made through network meta-analysis. Systematic review registration PROSPERO CRD 42013005324.

Corticosteroids in septic shock: a systematic review and network meta-analysis

BackgroundMultiple corticosteroids and treatment regimens have been used as adjuncts in the treatment of septic shock. Qualitative and quantitative differences exist at cellular and tissular levels between the different drugs and their patterns of delivery. The objective of this study was to elucidate any differences between the drugs and their treatment regimens regarding outcomes for corticosteroid use in adult patients with septic shock.MethodsNetwork meta-analysis of the data used for the recently conducted Cochrane review was performed. Studies that included children and were designed to assess respiratory function in pneumonia and acute respiratory distress syndrome, as well as cross-over studies, were excluded. Network plots were created for each outcome, and all analyses were conducted using a frequentist approach assuming a random-effects model.ResultsComplete data from 22 studies and partial data from 1 study were included. Network meta-analysis provided no clear evidence that any intervention or treatment regimen is better than any other across the spectrum of outcomes. There was strong evidence of differential efficacy in only one area: shock reversal. Hydrocortisone boluses and infusions were more likely than methylprednisolone boluses and placebo to result in shock reversal.ConclusionsThere was no clear evidence that any one corticosteroid drug or treatment regimen is more likely to be effective in reducing mortality or reducing the incidence of gastrointestinal bleeding or superinfection in septic shock. Hydrocortisone delivered as a bolus or as an infusion was more likely than placebo and methylprednisolone to result in shock reversal.

Some recommended statistical analytic practices when reliability generalization studies are conducted

Precursors of the reliability generalization (RG) meta-analytic approach have not established a single preferred analytic method. By means of five real RG examples, we examine how using different statistical methods to integrate coefficients alpha can influence results in RG studies. Specifically, we compare thirteen different statistical models for averaging reliability coefficients and searching for moderator variables that differ in terms of: (a) whether to transform or not the coefficients alpha, and (b) the statistical model assumed, distinguishing between ordinary least squares methods, the fixed-effect (FE) model, the varying coefficient (VC) model, and several versions of the random-effects (RE) model. The results obtained with the different methods exhibited important discrepancies, especially regarding moderator analyses. The main criterion for the model choice should be the extent to which the meta-analyst intends to generalize the results. RE models are the most appropriate when the meta-analyst aims to generalize to a hypothetical population of past or future studies, while FE and VC models are the most appropriate when the interest focuses on generalizing the results to a population of studies identical to those included in the meta-analysis. Finally, some guidelines are proposed for selecting the statistical model when conducting an RG study.

Alternatives for Mixed-Effects Meta-Regression Models in the Reliability Generalization Approach A Simulation Study

Since heterogeneity between reliability coefficients is usually found in reliability generalization studies, moderator analyses constitute a crucial step for that meta-analytic approach. In this study, different procedures for conducting mixed-effects meta-regression analyses were compared. Specifically, four transformation methods for the reliability coefficients, two estimators of the residual between-studies variance, and two methods for testing regression coefficients significance were combined in a Monte Carlo simulation study. The different methods were compared in terms of bias and mean square error (MSE) of the slope estimates, and Type I error and statistical power rates for the slope statistical tests. The results of the simulation study did not vary as a function of the residual variance estimator. All transformation methods provided negatively biased estimates, but both bias and MSE were reasonably small in all cases. In contrast, important differences were found regarding statistical tests, with the method proposed by Knapp and Hartung showing a better adjustment to the nominal significance level and higher power rates than the standard method.