Areti Angeliki Veroniki

Evaluation of inconsistency in networks of interventions

BACKGROUND The assumption of consistency, defined as agreement between direct and indirect sources of evidence, underlies the increasingly popular method of network meta-analysis. No evidence exists so far regarding the extent of inconsistency in full networks of interventions or the factors that control its statistical detection. METHODS In this paper we assess the prevalence of inconsistency from data of 40 published networks of interventions involving 303 loops of evidence. Inconsistency is evaluated in each loop by contrasting direct and indirect estimates and by employing an omnibus test of consistency for the entire network. We explore whether different effect measures for dichotomous outcomes are associated with differences in inconsistency, and evaluate whether different ways to estimate heterogeneity affect the magnitude and detection of inconsistency. RESULTS Inconsistency was detected in from 2% to 9% of the tested loops, depending on the effect measure and heterogeneity estimation method. Loops that included comparisons informed by a single study were more likely to show inconsistency. About one-eighth of the networks were found to be inconsistent. The proportions of inconsistent loops do not materially change when different effect measures are used. Important heterogeneity or the overestimation of heterogeneity was associated with a small decrease in the prevalence of statistical inconsistency. CONCLUSIONS The study suggests that changing the effect measure might improve statistical consistency, and that an analysis of sensitivity to the assumptions and an estimator of heterogeneity might be needed before reaching a conclusion about the absence of statistical inconsistency, particularly in networks with few studies.

Methods to estimate the between-study variance and its uncertainty in meta-analysis

Meta‐analyses are typically used to estimate the overall/mean of an outcome of interest. However, inference about between‐study variability, which is typically modelled using a between‐study variance parameter, is usually an additional aim. The DerSimonian and Laird method, currently widely used by default to estimate the between‐study variance, has been long challenged. Our aim is to identify known methods for estimation of the between‐study variance and its corresponding uncertainty, and to summarise the simulation and empirical evidence that compares them. We identified 16 estimators for the between‐study variance, seven methods to calculate confidence intervals, and several comparative studies. Simulation studies suggest that for both dichotomous and continuous data the estimator proposed by Paule and Mandel and for continuous data the restricted maximum likelihood estimator are better alternatives to estimate the between‐study variance. Based on the scenarios and results presented in the published studies, we recommend the Q‐profile method and the alternative approach based on a ‘generalised Cochran between‐study variance statistic’ to compute corresponding confidence intervals around the resulting estimates. Our recommendations are based on a qualitative evaluation of the existing literature and expert consensus. Evidence‐based recommendations require an extensive simulation study where all methods would be compared under the same scenarios.

Comparative efficacy of serotonin (5-HT3

BackgroundSerotonin (5-HT3) receptor antagonists are commonly used to decrease nausea and vomiting for surgery patients. We conducted a systematic review on the comparative efficacy of 5-HT3 receptor antagonists.MethodsSearches were done in MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials to identify studies comparing 5-HT3 receptor antagonists with each other, placebo, and/or combined with other antiemetic agents for patients undergoing surgical procedures. Screening search results, data abstraction, and risk of bias assessment were conducted by two reviewers independently. Random-effects pairwise meta-analysis and network meta-analysis (NMA) were conducted. PROSPERO registry number: CRD42013003564.ResultsOverall, 450 studies and 80,410 patients were included after the screening of 7,608 citations and 1,014 full-text articles. Significantly fewer patients experienced nausea with any drug relative to placebo, except for ondansetron plus metoclopramide in a NMA including 195 RCTs and 24,230 patients. Significantly fewer patients experienced vomiting with any drug relative to placebo except for palonosetron plus dexamethasone in NMA including 238 RCTs and 12,781 patients. All agents resulted in significantly fewer patients with postoperative nausea and vomiting versus placebo in a NMA including 125 RCTs and 16,667 patients.ConclusionsGranisetron plus dexamethasone was often the most effective antiemetic, with the number needed to treat ranging from two to nine.

Characteristics of networks of interventions: a description of a database of 186 published networks.

Systematic reviews that employ network meta-analysis are undertaken and published with increasing frequency while related statistical methodology is evolving. Future statistical developments and evaluation of the existing methodologies could be motivated by the characteristics of the networks of interventions published so far in order to tackle real rather than theoretical problems. Based on the recently formed network meta-analysis literature we aim to provide an insight into the characteristics of networks in healthcare research. We searched PubMed until end of 2012 for meta-analyses that used any form of indirect comparison. We collected data from networks that compared at least four treatments regarding their structural characteristics as well as characteristics of their analysis. We then conducted a descriptive analysis of the various network characteristics. We included 186 networks of which 35 (19%) were star-shaped (treatments were compared to a common comparator but not between themselves). The median number of studies per network was 21 and the median number of treatments compared was 6. The majority (85%) of the non-star shaped networks included at least one multi-arm study. Synthesis of data was primarily done via network meta-analysis fitted within a Bayesian framework (113 (61%) networks). We were unable to identify the exact method used to perform indirect comparison in a sizeable number of networks (18 (9%)). In 32% of the networks the investigators employed appropriate statistical methods to evaluate the consistency assumption; this percentage is larger among recently published articles. Our descriptive analysis provides useful information about the characteristics of networks of interventions published the last 16 years and the methods for their analysis. Although the validity of network meta-analysis results highly depends on some basic assumptions, most authors did not report and evaluate them adequately. Reviewers and editors need to be aware of these assumptions and insist on their reporting and accuracy.

No role for initial severity on the efficacy of antidepressants: results of a multi-meta-analysis

IntroductionDuring the last decade, a number of meta-analyses questioned the clinically relevant efficacy of antidepressants. Part of the debate concerned the method used in each of these meta-analyses as well as the quality of the data set.Materials and methodsThe Kirsch data set was analysed with a number of different methods, and eight key questions were tackled. We fit random effects models in both Bayesian and frequentist statistical frameworks using raw mean difference and standardised mean difference scales. We also compare between-study heterogeneity estimates and produce treatment rank probabilities for all antidepressants. The role of the initial severity is further examined using meta-regression methods.ResultsThe results suggest that antidepressants have a standardised effect size equal to 0.34 which is lower but comparable to the effect of antipsychotics in schizophrenia and acute mania. The raw HDRS difference from placebo is 2.82 with the value of 3 included in the confidence interval (2.21–3.44). No role of initial severity was found after partially controlling for the effect of structural (mathematical) coupling. Although data are not definite, even after controlling for baseline severity, there is a strong possibility that venlafaxine is superior to fluoxetine, with the other two agents positioned in the middle. The decrease in the difference between the agent and placebo in more recent studies in comparison to older ones is attributed to baseline severity alone.DiscussionThe results reported here conclude the debate on the efficacy of antidepressants and suggest that antidepressants are clearly superior to placebo. They also suggest that baseline severity cannot be utilized to dictate whether the treatment should include medication or not. Suggestions like this, proposed by guidelines or institutions (e.g. the NICE), should be considered mistaken.

Safety, effectiveness, and cost effectiveness of long acting versus intermediate acting insulin for patients with type 1 diabetes: systematic review and network meta-analysis

Objective To examine the safety, effectiveness, and cost effectiveness of long acting insulin for type 1 diabetes. Design Systematic review and network meta-analysis. Data sources Medline, Cochrane Central Register of Controlled Trials, Embase, and grey literature were searched through January 2013. Study selection Randomized controlled trials or non-randomized studies of long acting (glargine, detemir) and intermediate acting (neutral protamine Hagedorn (NPH), lente) insulin for adults with type 1 diabetes were included. Results 39 studies (27 randomized controlled trials including 7496 patients) were included after screening of 6501 titles/abstracts and 190 full text articles. Glargine once daily, detemir once daily, and detemir once/twice daily significantly reduced hemoglobin A1c compared with NPH once daily in network meta-analysis (26 randomized controlled trials, mean difference −0.39%, 95% confidence interval −0.59% to −0.19%; −0.26%, −0.48% to −0.03%; and −0.36%, −0.65% to −0.08%; respectively). Differences in network meta-analysis were observed between long acting and intermediate acting insulin for severe hypoglycemia (16 randomized controlled trials; detemir once/twice daily versus NPH once/twice daily: odds ratio 0.62, 95% confidence interval 0.42 to 0.91) and weight gain (13 randomized controlled trials; detemir once daily versus NPH once/twice daily: mean difference 4.04 kg, 3.06 to 5.02 kg; detemir once/twice daily versus NPH once daily: −5.51 kg, −6.56 to −4.46 kg; glargine once daily versus NPH once daily: −5.14 kg, −6.07 to −4.21). Compared with NPH, detemir was less costly and more effective in 3/14 cost effectiveness analyses and glargine was less costly and more effective in 2/8 cost effectiveness analyses. The remaining cost effectiveness analyses found that detemir and glargine were more costly but more effective than NPH. Glargine was not cost effective compared with detemir in 2/2 cost effectiveness analyses. Conclusions Long acting insulin analogs are probably superior to intermediate acting insulin analogs, although the difference is small for hemoglobin A1c. Patients and their physicians should tailor their choice of insulin according to preference, cost, and accessibility. Systematic review registration PROSPERO CRD42013003610.

Comparisons of Interventions for Preventing Falls in Older Adults: A Systematic Review and Meta-analysis

Importance Falls result in substantial burden for patients and health care systems, and given the aging of the population worldwide, the incidence of falls continues to rise. Objective To assess the potential effectiveness of interventions for preventing falls. Data Sources MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and Ageline databases from inception until April 2017. Reference lists of included studies were scanned. Study Selection Randomized clinical trials (RCTs) of fall-prevention interventions for participants aged 65 years and older. Data Extraction and Synthesis Pairs of reviewers independently screened the studies, abstracted data, and appraised risk of bias. Pairwise meta-analysis and network meta-analysis were conducted. Main Outcomes and Measures Injurious falls and fall-related hospitalizations. Results A total of 283 RCTs (159 910 participants; mean age, 78.1 years; 74% women) were included after screening of 10 650 titles and abstracts and 1210 full-text articles. Network meta-analysis (including 54 RCTs, 41 596 participants, 39 interventions plus usual care) suggested that the following interventions, when compared with usual care, were associated with reductions in injurious falls: exercise (odds ratio [OR], 0.51 [95% CI, 0.33 to 0.79]; absolute risk difference [ARD], −0.67 [95% CI, −1.10 to −0.24]); combined exercise and vision assessment and treatment (OR, 0.17 [95% CI, 0.07 to 0.38]; ARD, −1.79 [95% CI, −2.63 to −0.96]); combined exercise, vision assessment and treatment, and environmental assessment and modification (OR, 0.30 [95% CI, 0.13 to 0.70]; ARD, −1.19 [95% CI, −2.04 to −0.35]); and combined clinic-level quality improvement strategies (eg, case management), multifactorial assessment and treatment (eg, comprehensive geriatric assessment), calcium supplementation, and vitamin D supplementation (OR, 0.12 [95% CI, 0.03 to 0.55]; ARD, −2.08 [95% CI, −3.56 to −0.60]). Pairwise meta-analyses for fall-related hospitalizations (2 RCTs; 516 participants) showed no significant association between combined clinic- and patient-level quality improvement strategies and multifactorial assessment and treatment relative to usual care (OR, 0.78 [95% CI, 0.33 to 1.81]). Conclusions and Relevance Exercise alone and various combinations of interventions were associated with lower risk of injurious falls compared with usual care. Choice of fall-prevention intervention may depend on patient and caregiver values and preferences.

The rank-heat plot is a novel way to present the results from a network meta-analysis including multiple outcomes

OBJECTIVES To present a novel and simple graphical approach to improve the presentation of the treatment ranking in a network meta-analysis (NMA) including multiple outcomes. STUDY DESIGN AND SETTING NMA simultaneously compares many relevant interventions for a clinical condition from a network of trials, and allows ranking of the effectiveness and/or safety of each intervention. There are numerous ways to present the NMA results, which can challenge their interpretation by research users. The rank-heat plot is a novel graph that can be used to quickly recognize which interventions are most likely the best or worst interventions with respect to their effectiveness and/or safety for a single or multiple outcome(s) and may increase interpretability. RESULTS Using empirical NMAs, we show that the need for a concise and informative presentation of results is imperative, particularly as the number of competing treatments and outcomes in an NMA increases. CONCLUSION The rank-heat plot is an efficient way to present the results of ranking statistics, particularly when a large amount of data is available, and it is targeted to users from various backgrounds.

Comparative safety of serotonin (5-HT3) receptor antagonists in patients undergoing surgery: a systematic review and network meta-analysis

BackgroundSerotonin (5-HT3) receptor antagonists are commonly used to decrease nausea and vomiting for surgery patients, but these agents may be harmful. We conducted a systematic review on the comparative safety of 5-HT3 receptor antagonists.MethodsSearches were done in MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials to identify studies comparing 5-HT3 receptor antagonists with each other, placebo, and/or other antiemetic agents for patients undergoing surgical procedures. Screening search results, data abstraction, and risk of bias assessment were conducted by two reviewers independently. Random-effects pairwise meta-analysis and network meta-analysis (NMA) were conducted. PROSPERO registry number: CRD42013003564.ResultsOverall, 120 studies and 27,787 patients were included after screening of 7,608 citations and 1,014 full-text articles. Significantly more patients receiving granisetron plus dexamethasone experienced an arrhythmia relative to placebo (odds ratio (OR) 2.96, 95 % confidence interval (CI) 1.11–7.94), ondansetron (OR 3.23, 95 % CI 1.17–8.95), dolasetron (OR 4.37, 95 % CI 1.51–12.62), tropisetron (OR 3.27, 95 % CI 1.02–10.43), and ondansetron plus dexamethasone (OR 5.75, 95 % CI 1.71–19.34) in a NMA including 31 randomized clinical trials (RCTs) and 6,623 patients of all ages. No statistically significant differences in delirium frequency were observed across all treatment comparisons in a NMA including 18 RCTs and 3,652 patients.ConclusionGranisetron plus dexamethasone increases the risk of arrhythmia.

Characteristics of a loop of evidence that affect detection and estimation of inconsistency: a simulation study

BackgroundThe assumption of consistency, defined as agreement between direct and indirect sources of evidence, underlies the increasingly popular method of network meta-analysis. This assumption is often evaluated by statistically testing for a difference between direct and indirect estimates within each loop of evidence. However, the test is believed to be underpowered. We aim to evaluate its properties when applied to a loop typically found in published networks.MethodsIn a simulation study we estimate type I error, power and coverage probability of the inconsistency test for dichotomous outcomes using realistic scenarios informed by previous empirical studies. We evaluate test properties in the presence or absence of heterogeneity, using different estimators of heterogeneity and by employing different methods for inference about pairwise summary effects (Knapp-Hartung and inverse variance methods).ResultsAs expected, power is positively associated with sample size and frequency of the outcome and negatively associated with the presence of heterogeneity. Type I error converges to the nominal level as the total number of individuals in the loop increases. Coverage is close to the nominal level in most cases. Different estimation methods for heterogeneity do not greatly impact on test performance, but different methods to derive the variances of the direct estimates impact on inconsistency inference. The Knapp-Hartung method is more powerful, especially in the absence of heterogeneity, but exhibits larger type I error. The power for a ‘typical’ loop (comprising of 8 trials and about 2000 participants) to detect a 35% relative change between direct and indirect estimation of the odds ratio was 14% for inverse variance and 21% for Knapp-Hartung methods (with type I error 5% in the former and 11% in the latter).ConclusionsThe study gives insight into the conditions under which the statistical test can detect important inconsistency in a loop of evidence. Although different methods to estimate the uncertainty of the mean effect may improve the test performance, this study suggests that the test has low power for the ‘typical’ loop. Investigators should interpret results very carefully and always consider the comparability of the studies in terms of potential effect modifiers.