Jaime Peters

Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials.

Funnel plots, and tests for funnel plot asymmetry, have been widely used to examine bias in the results of meta-analyses. Funnel plot asymmetry should not be equated with publication bias, because it has a number of other possible causes. This article describes how to interpret funnel plot asymmetry, recommends appropriate tests, and explains the implications for choice of meta-analysis model

The prevalence of co-morbid depression in adults with Type 2 diabetes: a systematic review and meta-analysis

Aim  To conduct a systematic literature review in order to estimate the prevalence and odds ratio of clinically relevant depression in adults with Type 2 diabetes compared with those without.

The prevalence of co-morbid depression in adults with Type 2 diabetes: a systematic review and meta-analysis.

Aim  To conduct a systematic literature review in order to estimate the prevalence and odds ratio of clinically relevant depression in adults with Type 2 diabetes compared with those without.

Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry

OBJECTIVES To present the contour-enhanced funnel plot as an aid to differentiating asymmetry due to publication bias from that due to other factors. STUDY DESIGN AND SETTING An enhancement to the usual funnel plot is proposed that allows the statistical significance of study estimates to be considered. Contour lines indicating conventional milestones in levels of statistical significance (e.g., <0.01, <0.05, <0.1) are added to funnel plots. RESULTS This contour overlay aids the interpretation of the funnel plot. For example, if studies appear to be missing in areas of statistical nonsignificance, then this adds credence to the possibility that the asymmetry is due to publication bias. Conversely, if the supposed missing studies are in areas of higher statistical significance, this would suggest the cause of the asymmetry may be more likely to be due to factors other than publication bias, such as variable study quality. CONCLUSIONS We believe this enhancement to funnel plots (i) is simple to implement, (ii) is widely applicable, (iii) greatly improves interpretability, and (iv) should be used routinely.

Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study

BackgroundIn meta-analysis, the presence of funnel plot asymmetry is attributed to publication or other small-study effects, which causes larger effects to be observed in the smaller studies. This issue potentially mean inappropriate conclusions are drawn from a meta-analysis. If meta-analysis is to be used to inform decision-making, a reliable way to adjust pooled estimates for potential funnel plot asymmetry is required.MethodsA comprehensive simulation study is presented to assess the performance of different adjustment methods including the novel application of several regression-based methods (which are commonly applied to detect publication bias rather than adjust for it) and the popular Trim & Fill algorithm. Meta-analyses with binary outcomes, analysed on the log odds ratio scale, were simulated by considering scenarios with and without i) publication bias and; ii) heterogeneity. Publication bias was induced through two underlying mechanisms assuming the probability of publication depends on i) the study effect size; or ii) the p-value.ResultsThe performance of all methods tended to worsen as unexplained heterogeneity increased and the number of studies in the meta-analysis decreased. Applying the methods conditional on an initial test for the presence of funnel plot asymmetry generally provided poorer performance than the unconditional use of the adjustment method. Several of the regression based methods consistently outperformed the Trim & Fill estimators.ConclusionRegression-based adjustments for publication bias and other small study effects are easy to conduct and outperformed more established methods over a wide range of simulation scenarios.

Effectiveness of screening and monitoring tests for diabetic retinopathy--a systematic review.

SUMMARY

The Effectiveness and Cost-Effectiveness of Donepezil, Galantamine, Rivastigmine and Memantine for the Treatment of Alzheimer's Disease (Review of Technology Appraisal No. 111): A Systematic Review and Economic Model

BACKGROUND Alzheimer’s disease (AD) is the most commonly occurring form of dementia. It is predominantly a disease of later life, affecting 5% of those over 65 in the UK. OBJECTIVES Review and update guidance to the NHS in England and Wales on the clinical effectiveness and cost-effectiveness of donepezil, galantamine, rivastigmine [acetylcholinesterase inhibitors (AChEIs)] and memantine within their licensed indications for the treatment of AD, which was issued in November 2006 (amended September 2007 and August 2009). DATA SOURCES Electronic databases were searched for systematic reviews and/or metaanalyses, randomised controlled trials (RCTs) and ongoing research in November 2009 and updated in March 2010; this updated search revealed no new includable studies. The databases searched included The Cochrane Library (2009 Issue 4, Cochrane Database of Systematic Reviews and Cochrane Central Register of Controlled Trials), MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, PsycINFO, EconLit, ISI Web of Science Databases--Science Citation Index, Conference Proceedings Citation Index, and BIOSIS; the Centre for Reviews and Dissemination (CRD) databases--NHS Economic Evaluation Database, Health Technology Assessment, and Database of Abstracts of Reviews of Effects. REVIEW METHODS The clinical effectiveness systematic review was undertaken following the principles published by the NHS CRD. We included RCTs whose population was people with AD. The intervention and comparators depended on disease severity, measured by the Mini Mental State Examination (MMSE). INTERVENTIONS mild AD (MMSE 21-26)--donepezil, galantamine and rivastigmine; moderate AD (MMSE 10-20)--donepezil, galantamine, rivastigmine and memantine; severe AD (MMSE < 10)--memantine. Comparators: mild AD (MMSE 21-26)--placebo or best supportive care (BSC); moderate AD (MMSE 10-20)--donepezil, galantamine, rivastigmine, memantine, placebo or BSC; severe AD (MMSE < 10)--placebo or BSC. The outcomes were clinical, global, functional, behavioural, quality of life, adverse events, costs and cost-effectiveness. Where appropriate, data were pooled using pair-wise meta-analysis, multiple outcome measures, metaregression and mixedtreatment comparisons. The decision model was based broadly on the structure of the three-state Markov model described in the previous technology assessment report, based upon time to institutionalisation, parameterised with updated estimates of effectiveness, costs and utilities. RESULTS Notwithstanding the uncertainty of our results, we found in the base case that the AChEIs are probably cost saving at a willingness-to-pay (WTP) of £’30,000 per qualityadjusted life-year (QALY) for people with mild-to-moderate AD. For this class of drugs, there is a > 99% probability that the AChEIs are more cost-effective than BSC. These analyses assume that the AChEIs have no effect on survival. For the AChEIs, in people with mild to moderate AD, the probabilistic sensitivity analyses suggested that donepezil is the most cost-effective, with a 28% probability of being the most cost-effective option at a WTP of £’30,000 per QALY (27% at a WTP of £’20,000 per QALY). In the deterministic results, donepezil dominates the other drugs and BSC, which, along with rivastigmine patches, are associated with greater costs and fewer QALYs. Thus, although galantamine has a slightly cheaper total cost than donepezil (£’69,592 vs £’69,624), the slightly greater QALY gains from donepezil (1.616 vs 1.617) are enough for donepezil to dominate galantamine.The probability that memantine is cost-effective in a moderate to severe cohort compared with BSC at a WTP of £’30,000 per QALY is 38% (and 28% at a WTP of £’20,000 per QALY). The deterministic ICER for memantine is £’32,100 per/QALY and the probabilistic ICER is £’36,700 per/QALY. LIMITATIONS Trials were of 6 months maximum follow-up, lacked reporting of key outcomes, provided no subgroup analyses and used insensitive measures. Searches were limited to English language, The model does not include behavioural symptoms and there is uncertainty about the model structure and parameters. CONCLUSIONS The additional clinical effectiveness evidence identified continues to suggest clinical benefit from the AChEIs in alleviating AD symptoms, although there is debate about the magnitude of the effect. Although there is also new evidence on the effectiveness of memantine, it remains less supportive of this drug’s use than the evidence for AChEIs. The conclusions concerning cost-effectiveness are quite different from the previous assessment. This is because both the changes in effectiveness and costs between drug use and non-drug use underlying the ICERs are very small. This leads to highly uncertain results, which are very sensitive to change. RESEARCH PRIORITIES: RCTs to include mortality, time to institutionalisation and quality of life, powered for subgroup analysis. FUNDING The National Institute for Health Research Health Technology Assessment programme.

A Systematic Review of Systematic Reviews and Meta-Analyses of Animal Experiments with Guidelines for Reporting

To maximize the findings of animal experiments to inform likely health effects in humans, a thorough review and evaluation of the animal evidence is required. Systematic reviews and, where appropriate, meta-analyses have great potential in facilitating such an evaluation, making efficient use of the animal evidence while minimizing possible sources of bias. The extent to which systematic review and meta-analysis methods have been applied to evaluate animal experiments to inform human health is unknown. Using systematic review methods, we examine the extent and quality of systematic reviews and meta-analyses of in vivo animal experiments carried out to inform human health. We identified 103 articles meeting the inclusion criteria: 57 reported a systematic review, 29 a systematic review and a meta-analysis, and 17 reported a meta-analysis only. The use of these methods to evaluate animal evidence has increased over time. Although the reporting of systematic reviews is of adequate quality, the reporting of meta-analyses is poor. The inadequate reporting of meta-analyses observed here leads to questions on whether the most appropriate methods were used to maximize the use of the animal evidence to inform policy or decision-making. We recommend that guidelines proposed here be used to help improve the reporting of systematic reviews and meta-analyses of animal experiments. Further consideration of the use and methodological quality and reporting of such studies is needed.

Assessing publication bias in meta-analyses in the presence of between-study heterogeneity

Between-study heterogeneity and publication bias are common features of a meta-analysis that can be present simultaneously. When both are suspected, consideration must be made of each in the assessment of the other. We consider extended funnel plot tests for detecting publication bias, and selection modelling and trim-and-fill methods to adjust for publication bias in the presence of between-study heterogeneity. These methods are applied to two example data sets. Results indicate that ignoring between-study heterogeneity when assessing publication bias can be misleading, but that methods to test or adjust for publication bias in the presence of heterogeneity may not be powerful when the meta-analysis is not large. It is therefore unrealistic to expect to disentangle the effects of publication bias and heterogeneity reliably in all except the largest meta-analyses. Copyright (c) 2010 Royal Statistical Society.

How valuable are multiple treatment comparison methods in evidence-based health-care evaluation?

OBJECTIVES To compare the use of pair-wise meta-analysis methods to multiple treatment comparison (MTC) methods for evidence-based health-care evaluation to estimate the effectiveness and cost-effectiveness of alternative health-care interventions based on the available evidence. METHODS Pair-wise meta-analysis and more complex evidence syntheses, incorporating an MTC component, are applied to three examples: 1) clinical effectiveness of interventions for preventing strokes in people with atrial fibrillation; 2) clinical and cost-effectiveness of using drug-eluting stents in percutaneous coronary intervention in patients with coronary artery disease; and 3) clinical and cost-effectiveness of using neuraminidase inhibitors in the treatment of influenza. We compare the two synthesis approaches with respect to the assumptions made, empirical estimates produced, and conclusions drawn. RESULTS The difference between point estimates of effectiveness produced by the pair-wise and MTC approaches was generally unpredictable-sometimes agreeing closely whereas in other instances differing considerably. In all three examples, the MTC approach allowed the inclusion of randomized controlled trial evidence ignored in the pair-wise meta-analysis approach. This generally increased the precision of the effectiveness estimates from the MTC model. CONCLUSIONS The MTC approach to synthesis allows the evidence base on clinical effectiveness to be treated as a coherent whole, include more data, and sometimes relax the assumptions made in the pair-wise approaches. However, MTC models are necessarily more complex than those developed for pair-wise meta-analysis and thus could be seen as less transparent. Therefore, it is important that model details and the assumptions made are carefully reported alongside the results.