Anna Chaimani

Graphical Tools for Network Meta-Analysis in STATA

Network meta-analysis synthesizes direct and indirect evidence in a network of trials that compare multiple interventions and has the potential to rank the competing treatments according to the studied outcome. Despite its usefulness network meta-analysis is often criticized for its complexity and for being accessible only to researchers with strong statistical and computational skills. The evaluation of the underlying model assumptions, the statistical technicalities and presentation of the results in a concise and understandable way are all challenging aspects in the network meta-analysis methodology. In this paper we aim to make the methodology accessible to non-statisticians by presenting and explaining a series of graphical tools via worked examples. To this end, we provide a set of STATA routines that can be easily employed to present the evidence base, evaluate the assumptions, fit the network meta-analysis model and interpret its results.

Evaluating the quality of evidence from a network meta-analysis.

Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.

Using network meta‐analysis to evaluate the existence of small‐study effects in a network of interventions

Suggested methods for exploring the presence of small-study effects in a meta-analysis and the possibility of publication bias are associated with important limitations. When a meta-analysis comprises only a few studies, funnel plots are difficult to interpret, and regression-based approaches to test and account for small-study effects have low power. Assuming that the cause of funnel plot asymmetry is likely to affect an entire research field rather than only a particular comparison of interventions, we suggest that network meta-regression is employed to account for small-study effects in a set of related meta-analyses. We present several possible models for the direction and distribution of small-study effects and we describe the methods by re-analysing two published networks. Copyright

The quality of reporting methods and results in network meta-analyses: an overview of reviews and suggestions for improvement.

Introduction Some have suggested the quality of reporting of network meta-analyses (a technique used to synthesize information to compare multiple interventions) is sub-optimal. We sought to review information addressing this claim. Objective To conduct an overview of existing evaluations of quality of reporting in network meta-analyses and indirect treatment comparisons, and to compile a list of topics which may require detailed reporting guidance to enhance future reporting quality. Methods An electronic search of Medline and the Cochrane Registry of methodologic studies (January 2004–August 2013) was performed by an information specialist. Studies describing findings from quality of reporting assessments were sought. Screening of abstracts and full texts was performed by two team members. Descriptors related to all aspects of reporting a network meta-analysis were summarized. Results We included eight reports exploring the quality of reporting of network meta-analyses. From past reviews, authors found several aspects of network meta-analyses were inadequately reported, including primary information about literature searching, study selection, and risk of bias evaluations; statement of the underlying assumptions for network meta-analysis, as well as efforts to verify their validity; details of statistical models used for analyses (including information for both Bayesian and Frequentist approaches); completeness of reporting of findings; and approaches for summarizing probability measures as additional important considerations. Conclusions While few studies were identified, several deficiencies in the current reporting of network meta-analyses were observed. These findings reinforce the need to develop reporting guidance for network meta-analyses. Findings from this review will be used to guide next steps in the development of reporting guidance for network meta-analysis in the format of an extension of the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) Statement.

Characteristics of networks of interventions: a description of a database of 186 published networks.

Systematic reviews that employ network meta-analysis are undertaken and published with increasing frequency while related statistical methodology is evolving. Future statistical developments and evaluation of the existing methodologies could be motivated by the characteristics of the networks of interventions published so far in order to tackle real rather than theoretical problems. Based on the recently formed network meta-analysis literature we aim to provide an insight into the characteristics of networks in healthcare research. We searched PubMed until end of 2012 for meta-analyses that used any form of indirect comparison. We collected data from networks that compared at least four treatments regarding their structural characteristics as well as characteristics of their analysis. We then conducted a descriptive analysis of the various network characteristics. We included 186 networks of which 35 (19%) were star-shaped (treatments were compared to a common comparator but not between themselves). The median number of studies per network was 21 and the median number of treatments compared was 6. The majority (85%) of the non-star shaped networks included at least one multi-arm study. Synthesis of data was primarily done via network meta-analysis fitted within a Bayesian framework (113 (61%) networks). We were unable to identify the exact method used to perform indirect comparison in a sizeable number of networks (18 (9%)). In 32% of the networks the investigators employed appropriate statistical methods to evaluate the consistency assumption; this percentage is larger among recently published articles. Our descriptive analysis provides useful information about the characteristics of networks of interventions published the last 16 years and the methods for their analysis. Although the validity of network meta-analysis results highly depends on some basic assumptions, most authors did not report and evaluate them adequately. Reviewers and editors need to be aware of these assumptions and insist on their reporting and accuracy.

Repetitive Transcranial Magnetic Stimulation for the Acute Treatment of Major Depressive Episodes: A Systematic Review With Network Meta-analysis

Importance Although several strategies of repetitive transcranial magnetic stimulation (rTMS) have been investigated as treatment of major depressive disorder (MDD), their comparative efficacy and acceptability is unknown. Objective To establish the relative efficacy and acceptability of the different modalities of rTMS used for MDD by performing a network meta-analysis, obtaining a clinically meaningful treatment hierarchy. Data Sources PubMed/MEDLINE, EMBASE, PsycInfo, and Web of Science were searched up until October 1, 2016. Study Selection Randomized clinical trials that compared any rTMS intervention with sham or another rTMS intervention. Trials performing less than 10 sessions were excluded. Data Extraction and Synthesis Two independent reviewers used standard forms for data extraction and quality assessment. Random-effects, standard pairwise, and network meta-analyses were performed to synthesize data. Main Outcomes and Measures Response rates and acceptability (dropout rate). Remission was the secondary outcome. Effect sizes were reported as odds ratios (ORs) with 95% CIs. Results Eighty-one studies (4233 patients, 59.1% women, mean age of 46 years) were included. The interventions more effective than sham were priming low-frequency (OR, 4.66; 95% CI, 1.70-12.77), bilateral (OR, 3.96; 95% CI, 2.37-6.60), high-frequency (OR, 3.07; 95% CI, 2.24-4.21), &thgr;-burst stimulation (OR, 2.54; 95% CI, 1.07-6.05), and low-frequency (OR, 2.37; 95% CI, 1.52-3.68) rTMS. Novel rTMS interventions (accelerated, synchronized, and deep rTMS) were not more effective than sham. Except for &thgr;-burst stimulation vs sham, similar results were obtained for remission. All interventions were at least as acceptable as sham. The estimated relative ranking of treatments suggested that priming low-frequency and bilateral rTMS might be the most efficacious and acceptable interventions among all rTMS strategies. However, results were imprecise and relatively few trials were available for interventions other than low-frequency, high-frequency, and bilateral rTMS. Conclusions and Relevance Few differences were found in clinical efficacy and acceptability between the different rTMS modalities, favoring to some extent bilateral rTMS and priming low-frequency rTMS. These findings warrant the design of larger RCTs investigating the potential of these approaches in the short-term treatment of MDD. Current evidence cannot support novel rTMS interventions as a treatment for MDD. Trial Registration clinicaltrials.gov Identifier: PROSPERO CRD42015019855.

Placebo response rates in antidepressant trials: a systematic review of published and unpublished double-blind randomised controlled studies

BACKGROUND Previous studies have shown that placebo response rates in antidepressant trials have been increasing since the 1970s. However, these studies have been based on outdated or limited datasets and have used inappropriate statistical methods. We did a systematic review of placebo-controlled randomised controlled trials of antidepressants to examine associations between placebo-response rates and study and patient characteristics. METHODS In this systematic review, we searched for published and unpublished double-blind randomised placebo-controlled trials of first-generation and second-generation antidepressants for acute treatment of major depression in adults (update: Jan 8, 2016). The log-transformed proportions of placebo response, defined as 50% or greater reduction in depression severity score from baseline, were meta-analytically synthesised for each year. We then looked for a structural break point in the secular changes in these characteristics through the years and examined the influence of the study year and other trial and patient characteristics on the response rates through meta-regression. FINDINGS We identified 252 placebo-controlled trials (26 324 patients on placebo) done between 1978 and 2015. There was a structural break in 1991, and since then, the average placebo response rates in antidepressant trials have remained constant in the range between 35% and 40% (relative risk [RR] 1·00, 95% CI 0·97-1·03, p=0·99, for every 5-year increase). The length of the study and the number of study centres were significant factors (RR 1·03, 95% CI 1·01-1·05 for 1 more week in trial length; 1·32, 1·11-1·57 for multicentre vs single-centre trials). INTERPRETATION Contrary to the widely held belief, the average placebo response rates in antidepressant trials have been stable for more than 25 years. This new evidence should have an effect on the interpretation of the scientific literature and the future of psychopharmacology, both from a clinical and methodological point of view. FUNDING Japan Society for Promotion of Science, Great Britain Sasakawa Foundation.

Network meta-analyses should be the highest level of evidence in treatment guidelines

randomised trials comparing two treatments directly (socalled direct evidence). The major criticism has been that meta-analysis compares “apples and oranges”; are trials sufficiently similar so that they can be summarised or are they “heterogeneous”? Network meta-analysis (also called multiple-treatments meta-analysis) additionally uses “indirect evidence”. For example, if in schizophrenia there were trials that compared olanzapine with quetiapine and trials that compared olanzapine with aripiprazole, but no trials comparing quetiapine with aripiprazole directly, we can estimate quetiapine versus aripiprazole indirectly from the other two direct comparisons (see Fig. 1). There are several strengths and added values of this approach: (a) the indirect evidence can fill in the gaps in the evidence matrix, which allows to come up with hierarchies of which drug is probably the best, second best, third best and so on. This information is urgently needed by guidelines, but cannot really After initial hesitancy due to fears that this procedure might lead to “cookbook medicine” and others, evidencebased medicine (EBM) is now an accepted principle in all fields of medicine including psychiatry. The essence of the evidence is used by many treatment guidelines to inform clinicians in their daily practice. One not entirely resolved issue is, however, which study or evidence synthesis design should be considered as the highest level of evidence. Early statements from McMaster University in Canada [5] (together with the Cochrane Collaboration, the “cradle” of EBM) suggested systematic reviews with meta-analysis can provide the most robust and reliable evidence, but not all guideline producers are in agreement. This is a timely debate, fuelled by the increasing publication of network meta-analyses, a novel approach which takes the assumptions of meta-analysis one step further [3]. Conventional meta-analyses only average the

The effects of excluding treatments from network meta-analyses: survey.

Objective To examine whether the exclusion of individual treatment comparators, including placebo/no treatment, affects the results of network meta-analysis. Design Survey of networks with individual trial data. Data sources PubMed and communication with authors of network meta-analyses. Study selection and methods We included networks that had five or more treatments, contained at least two closed loops, had at least twice as many studies as treatments, and had trial level data available. Investigators abstracted information about study design, participants, outcomes, network geometry, and the exclusion of eligible treatments. Results Among 18 eligible networks involving 757 randomised controlled trials with 750 possible treatment comparisons, 11 had upfront decided not to consider all treatment comparators and only 10 included placebo/no treatment nodes. In 7/18 networks, there was at least one node whose removal caused a more than 1.10-fold average relative change in the estimated treatments effects, and switches in the top three treatments were observed in 9/18 networks. Removal of placebo/no treatment caused large relative changes of the treatment effects (average change 1.16-3.10-fold) for four of the 10 networks that had originally included placebo/no treatment nodes. Exclusion of current uncommonly used drugs resulted in substantial changes of the treatment effects (average 1.21-fold) in one of three networks on systemic treatments for advanced malignancies. Conclusion Excluding treatments in network meta-analyses sometimes can have important effects on their results and can diminish the usefulness of the research to clinicians if important comparisons are missing.

Dietary Supplements and Risk of Cause-Specific Death, Cardiovascular Disease, and Cancer: A Systematic Review and Meta-Analysis of Primary Prevention Trials

Our aim was to assess the efficacy of dietary supplements in the primary prevention of cause-specific death, cardiovascular disease (CVD), and cancer by using meta-analytical approaches. Electronic and hand searches were performed until August 2016. Inclusion criteria were as follows: 1) minimum intervention period of 12 mo; 2) primary prevention trials; 3) mean age ≥18 y; 4) interventions included vitamins, fatty acids, minerals, supplements containing combinations of vitamins and minerals, protein, fiber, prebiotics, and probiotics; and 5) primary outcome of all-cause mortality and secondary outcomes of mortality or incidence from CVD or cancer. Pooled effects across studies were estimated by using random-effects meta-analysis. Overall, 49 trials (69 reports) including 287,304 participants met the inclusion criteria. Thirty-two trials were judged as low risk–, 15 trials as moderate risk–, and 2 trials as high risk–of-bias studies. Supplements containing vitamin E (RR: 0.88; 95% CI: 0.80, 0.96) significantly reduced cardiovascular mortality risk, whereas supplements with folic acid reduced the risk of CVD (RR: 0.81; 95% CI: 0.70, 0.94). Vitamins D, C, and K; selenium; zinc; magnesium; and eicosapentaenoic acid showed no significant risk reduction for any of the outcomes. On the contrary, vitamin A was linked to an increased cancer risk (RR: 1.16; 95% CI: 1.00, 1.35). Supplements with β-carotene showed no significant effect; however, in the subgroup with β-carotene given singly, an increased risk of all-cause mortality by 6% (RR: 1.06; 95% CI: 1.02, 1.10) was observed. Taken together, we found insufficient evidence to support the use of dietary supplements in the primary prevention of cause-specific death, incidence of CVD, and incidence of cancer. The application of some supplements generated small beneficial effects; however, the heterogeneous types and doses of supplements limit the generalizability to the overall population.