Dan Jackson

Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies

Meta-analyses that simultaneously compare multiple treatments (usually referred to as network meta-analyses or mixed treatment comparisons) are becoming increasingly common. An important component of a network meta-analysis is an assessment of the extent to which different sources of evidence are compatible, both substantively and statistically. A simple indirect comparison may be confounded if the studies involving one of the treatments of interest are fundamentally different from the studies involving the other treatment of interest. Here, we discuss methods for addressing inconsistency of evidence from comparative studies of different treatments. We define and review basic concepts of heterogeneity and inconsistency, and attempt to introduce a distinction between ‘loop inconsistency’ and ‘design inconsistency’. We then propose that the notion of design-by-treatment interaction provides a useful general framework for investigating inconsistency. In particular, using design-by-treatment interactions successfully addresses complications that arise from the presence of multi-arm trials in an evidence network. We show how the inconsistency model proposed by Lu and Ades is a restricted version of our full design-by-treatment interaction model and that there may be several distinct Lu–Ades models for any particular data set. We introduce novel graphical methods for depicting networks of evidence, clearly depicting multi-arm trials and illustrating where there is potential for inconsistency to arise. We apply various inconsistency models to data from trials of different comparisons among four smoking cessation interventions and show that models seeking to address loop inconsistency alone can run into problems. Copyright

Extending DerSimonian and Laird's methodology to perform multivariate random effects meta-analyses.

Multivariate meta-analysis is increasingly used in medical statistics. In the univariate setting, the non-iterative method proposed by DerSimonian and Laird is a simple and now standard way of performing random effects meta-analyses. We propose a natural and easily implemented multivariate extension of this procedure which is accessible to applied researchers and provides a much less computationally intensive alternative to existing methods. In a simulation study, the proposed procedure performs similarly in almost all ways to the more established iterative restricted maximum likelihood approach. The method is applied to some real data sets and an extension to multivariate meta-regression is described.

Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression†

Network meta-analysis (multiple treatments meta-analysis, mixed treatment comparisons) attempts to make the best use of a set of studies comparing more than two treatments. However, it is important to assess whether a body of evidence is consistent or inconsistent. Previous work on models for network meta-analysis that allow for heterogeneity between studies has either been restricted to two-arm trials or followed a Bayesian framework. We propose two new frequentist ways to estimate consistency and inconsistency models by expressing them as multivariate random-effects meta-regressions, which can be implemented in some standard software packages. We illustrate the approach using the mvmeta package in Stata. Copyright

Quantifying the impact of between-study heterogeneity in multivariate meta-analyses

Measures that quantify the impact of heterogeneity in univariate meta-analysis, including the very popular I2 statistic, are now well established. Multivariate meta-analysis, where studies provide multiple outcomes that are pooled in a single analysis, is also becoming more commonly used. The question of how to quantify heterogeneity in the multivariate setting is therefore raised. It is the univariate R2 statistic, the ratio of the variance of the estimated treatment effect under the random and fixed effects models, that generalises most naturally, so this statistic provides our basis. This statistic is then used to derive a multivariate analogue of I2, which we call . We also provide a multivariate H2 statistic, the ratio of a generalisation of Cochrans heterogeneity statistic and its associated degrees of freedom, with an accompanying generalisation of the usual I2 statistic, . Our proposed heterogeneity statistics can be used alongside all the usual estimates and inferential procedures used in multivariate meta-analysis. We apply our methods to some real datasets and show how our statistics are equally appropriate in the context of multivariate meta-regression, where study level covariate effects are included in the model. Our heterogeneity statistics may be used when applying any procedure for fitting the multivariate random effects model. Copyright

Incidence of Schizophrenia and Other Psychoses in England, 1950–2009: A Systematic Review and Meta-Analyses

Background We conducted a systematic review of incidence rates in England over a sixty-year period to determine the extent to which rates varied along accepted (age, sex) and less-accepted epidemiological gradients (ethnicity, migration and place of birth and upbringing, time). Objectives To determine variation in incidence of several psychotic disorders as above. Data Sources Published and grey literature searches (MEDLINE, PSycINFO, EMBASE, CINAHL, ASSIA, HMIC), and identification of unpublished data through bibliographic searches and author communication. Study Eligibility Criteria Published 1950–2009; conducted wholly or partially in England; original data on incidence of non-organic adult-onset psychosis or one or more factor(s) pertaining to incidence. Participants People, 16–64 years, with first -onset psychosis, including non-affective psychoses, schizophrenia, bipolar disorder, psychotic depression and substance-induced psychosis. Study Appraisal and Synthesis Methods Title, abstract and full-text review by two independent raters to identify suitable citations. Data were extracted to a standardized extraction form. Descriptive appraisals of variation in rates, including tables and forest plots, and where suitable, random-effects meta-analyses and meta-regressions to test specific hypotheses; rate heterogeneity was assessed by the I2-statistic. Results 83 citations met inclusion. Pooled incidence of all psychoses (N = 9) was 31.7 per 100,000 person-years (95%CI: 24.6–40.9), 23.2 (95%CI: 18.3–29.5) for non-affective psychoses (N = 8), 15.2 (95%CI: 11.9–19.5) for schizophrenia (N = 15) and 12.4 (95%CI: 9.0–17.1) for affective psychoses (N = 7). This masked rate heterogeneity (I2: 0.54–0.97), possibly explained by socio-environmental factors; our review confirmed (via meta-regression) the typical age-sex interaction in psychosis risk, including secondary peak onset in women after 45 years. Rates of most disorders were elevated in several ethnic minority groups compared with the white (British) population. For example, for schizophrenia: black Caribbean (pooled RR: 5.6; 95%CI: 3.4–9.2; N = 5), black African (pooled RR: 4.7; 95%CI: 3.3–6.8; N = 5) and South Asian groups in England (pooled RR: 2.4; 95%CI: 1.3–4.5; N = 3). We found no evidence to support an overall change in the incidence of psychotic disorder over time, though diagnostic shifts (away from schizophrenia) were reported. Limitations Incidence studies were predominantly cross-sectional, limiting causal inference. Heterogeneity, while evidencing important variation, suggested pooled estimates require interpretation alongside our descriptive systematic results. Conclusions and Implications of Key Findings Incidence of psychotic disorders varied markedly by age, sex, place and migration status/ethnicity. Stable incidence over time, together with a robust socio-environmental epidemiology, provides a platform for developing prediction models for health service planning.

Multivariate meta‐analysis: Potential and promise

The multivariate random effects model is a generalization of the standard univariate model. Multivariate meta-analysis is becoming more commonly used and the techniques and related computer software, although continually under development, are now in place. In order to raise awareness of the multivariate methods, and discuss their advantages and disadvantages, we organized a one day ‘Multivariate meta-analysis’ event at the Royal Statistical Society. In addition to disseminating the most recent developments, we also received an abundance of comments, concerns, insights, critiques and encouragement. This article provides a balanced account of the days discourse. By giving others the opportunity to respond to our assessment, we hope to ensure that the various view points and opinions are aired before multivariate meta-analysis simply becomes another widely used de facto method without any proper consideration of it by the medical statistics community. We describe the areas of application that multivariate meta-analysis has found, the methods available, the difficulties typically encountered and the arguments for and against the multivariate methods, using four representative but contrasting examples. We conclude that the multivariate methods can be useful, and in particular can provide estimates with better statistical properties, but also that these benefits come at the price of making more assumptions which do not result in better inference in every case. Although there is evidence that multivariate meta-analysis has considerable potential, it must be even more carefully applied than its univariate counterpart in practice. Copyright

Methods to estimate the between-study variance and its uncertainty in meta-analysis

Meta‐analyses are typically used to estimate the overall/mean of an outcome of interest. However, inference about between‐study variability, which is typically modelled using a between‐study variance parameter, is usually an additional aim. The DerSimonian and Laird method, currently widely used by default to estimate the between‐study variance, has been long challenged. Our aim is to identify known methods for estimation of the between‐study variance and its corresponding uncertainty, and to summarise the simulation and empirical evidence that compares them. We identified 16 estimators for the between‐study variance, seven methods to calculate confidence intervals, and several comparative studies. Simulation studies suggest that for both dichotomous and continuous data the estimator proposed by Paule and Mandel and for continuous data the restricted maximum likelihood estimator are better alternatives to estimate the between‐study variance. Based on the scenarios and results presented in the published studies, we recommend the Q‐profile method and the alternative approach based on a ‘generalised Cochran between‐study variance statistic’ to compute corresponding confidence intervals around the resulting estimates. Our recommendations are based on a qualitative evaluation of the existing literature and expert consensus. Evidence‐based recommendations require an extensive simulation study where all methods would be compared under the same scenarios.

A design‐by‐treatment interaction model for network meta‐analysis with random inconsistency effects

Network meta-analysis is becoming more popular as a way to analyse multiple treatments simultaneously and, in the right circumstances, rank treatments. A difficulty in practice is the possibility of ‘inconsistency’ or ‘incoherence’, where direct evidence and indirect evidence are not in agreement. Here, we develop a random-effects implementation of the recently proposed design-by-treatment interaction model, using these random effects to model inconsistency and estimate the parameters of primary interest. Our proposal is a generalisation of the model proposed by Lumley and allows trials with three or more arms to be included in the analysis. Our methods also facilitate the ranking of treatments under inconsistency. We derive R and I2 statistics to quantify the impact of the between-study heterogeneity and the inconsistency. We apply our model to two examples.

The exact distribution of Cochran's heterogeneity statistic in one-way random effects meta-analysis†‡

The presence and impact of heterogeneity in the standard one-way random effects model in meta-analysis are often assessed using the Q statistic due to Cochran. We derive the exact distribution of this statistic under the assumptions of the random effects model, and also suggest two moment-based approximations and a saddlepoint approximation for Q. The exact and approximate distributions are then applied to obtain the corresponding distributions of the recently proposed heterogeneity measures I(2) and H(M)(2), the power of the standard test for the presence of heterogeneity and confidence intervals for the between-study variance parameter when the DerSimonian-Laird or the Hartung-Makambi estimator is used. The methodology is illustrated by revisiting a recent simulation study concerning the heterogeneity measures and applying all the proposed methods to four published meta-analyses.

Current Evidence Is Insufficient to Define an Optimal Threshold for Intervention in Isolated Type II Endoleak After Endovascular Aneurysm Repair

Purpose To report a systematic review and meta-regression of the association between the threshold for intervention in patients with isolated type II endoleak after endovascular aneurysm repair (EVAR) and the fate of the aneurysm sac. Methods Medline, trial registries, conference proceedings, and article reference lists were searched to identify case series reporting sac outcomes following a specific treatment threshold for isolated type II endoleak. Articles were classified by the threshold for intervention as conservative, selective (intervention for >5-mm sac expansion or persistent type II endoleak >6 months), or aggressive (any type II endoleak or persistent for >3 months) and sac outcomes were extracted for review. Standard meta-regression to estimate the pooled odds ratios (OR), presented with the 95% confidence interval (CI), was performed to identify whether an aggressive, selective, or conservative threshold for intervention was associated with sac expansion or sac regression. Results Ten series were analyzed that reported the outcomes of isolated type II endoleak in 231 patients; of these, 56 patients were treated at an aggressive threshold, 104 at a selective threshold, and 71 at a conservative threshold. The majority (194/231,84.0%) demonstrated either stable or shrinking sacs during follow-up. No ruptures occurred. Meta-regression demonstrated no evidence that any strategy, compared to using a conservative approach, reduced sac expansion (aggressive estimated OR 0.70, 95% CI 0.15 to 3.31, p=0.60; selective estimated OR 1.72, 95% CI 0.49 to 6.00, p=0.34) or improved sac regression (aggressive estimated OR 0.55, 95% CI 0.02 to 16.94, p=0.69; selective estimated OR 5.54, 95% CI 0.39 to 79.21, p=0.17). Conclusion There is inadequate information to support any one threshold for intervention. The rarity of rupture and sac expansion confirms the predominantly benign nature of isolated type II endoleak. In the absence of statistical support for a uniform approach to this problem, patient and physician preference remain key. Prospective data are still needed to investigate whether an optimum management algorithm can be devised.