Ba' Pham

Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?

BACKGROUND Few meta-analyses of randomised trials assess the quality of the studies included. Yet there is increasing evidence that trial quality can affect estimates of intervention efficacy. We investigated whether different methods of quality assessment provide different estimates of intervention efficacy evaluated in randomised controlled trials (RCTs). METHODS We randomly selected 11 meta-analyses that involved 127 RCTs on the efficacy of interventions used for circulatory and digestive diseases, mental health, and pregnancy and childbirth. We replicated all the meta-analyses using published data from the primary studies. The quality of reporting of all 127 clinical trials was assessed by means of component and scale approaches. To explore the effects of quality on the quantitative results, we examined the effects of different methods of incorporating quality scores (sensitivity analysis and quality weights) on the results of the meta-analyses. FINDINGS The quality of trials was low. Masked assessments provided significantly higher scores than unmasked assessments (mean 2.74 [SD 1.10] vs 2.55 [1.20]). Low-quality trials (score < or = 2), compared with high-quality trials (score > 2), were associated with an increased estimate of benefit of 34% (ratio of odds ratios [ROR] 0.66 [95% CI 0.52-0.83]). Trials that used inadequate allocation concealment, compared with those that used adequate methods, were also associated with an increased estimate of benefit (37%; ROR=0.63 [0.45-0.88]). The average treatment benefit was 39% (odds ratio [OR] 0.61 [0.57-0.65]) for all trials, 52% (OR 0.48 [0.43-0.54]) for low-quality trials, and 29% (OR 0.71 [0.65-0.77]) for high-quality trials. Use of all the trial scores as quality weights reduced the effects to 35% (OR 0.65 [0.59-0.71]) and resulted in the least statistical heterogeneity. INTERPRETATION Studies of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality.

Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses?

BACKGROUND The inclusion of only a subset of all available evidence in a meta-analysis may introduce biases and threaten its validity; this is particularly likely if the subset of included studies differ from those not included, which may be the case for published and grey literature (unpublished studies, with limited distribution). We set out to examine whether exclusion of grey literature, compared with its inclusion in meta-analysis, provides different estimates of the effectiveness of interventions assessed in randomised trials. METHODS From a random sample of 135 meta-analyses, we identified and retrieved 33 publications that included both grey and published primary studies. The 33 publications contributed 41 separate meta-analyses from several disease areas. General characteristics of the meta-analyses and associated studies and outcome data at the trial level were collected. We explored the effects of the inclusion of grey literature on the quantitative results using logistic-regression analyses. FINDINGS 33% of the meta-analyses were found to include some form of grey literature. The grey literature, when included, accounts for between 4.5% and 75% of the studies in a meta-analysis. On average, published work, compared with grey literature, yielded significantly larger estimates of the intervention effect by 15% (ratio of odds ratios=1.15 [95% CI 1.04-1.28]). Excluding abstracts from the analysis further compounded the exaggeration (1.33 [1.10-1.60]). INTERPRETATION The exclusion of grey literature from meta-analyses can lead to exaggerated estimates of intervention effectiveness. In general, meta-analysts should attempt to identify, retrieve, and include all reports, grey and published, that meet predefined inclusion criteria.

Tenofovir and entecavir are the most effective antiviral agents for chronic hepatitis B: a systematic review and Bayesian meta-analyses.

BACKGROUND & AIMS The relative efficacies of licensed antiviral therapies for treatment-naive chronic hepatitis B (CHB) infection in randomized controlled trials have not been determined. We evaluated the relative efficacies of the first 12 months of CHB treatments. METHODS Drugs evaluated were lamivudine, pegylated interferon, adefovir, entecavir, telbivudine, and tenofovir, as monotherapies and combination therapies, in treatment-naive individuals. Databases were searched for randomized controlled trials of the first 12 months of therapy in hepatitis B e antigen (HBeAg)-positive and/or HBeAg-negative patients with CHB published in English before October 31, 2009. Bayesian mixed treatment comparisons were used to calculate the odds ratios, including 95% credible intervals and predicted probabilities of surrogate outcomes to determine the relative effects of each treatment. RESULTS In HBeAg-positive patients, tenofovir was most effective in inducing undetectable levels of HBV DNA (predicted probability, 88%), normalization of alanine aminotransferase (ALT) levels (66%), HBeAg seroconversion (20%), and hepatitis B surface antigen loss (5%); it ranked third in histologic improvement of the liver (53%). Entecavir was most effective in improving liver histology (56%), second for inducing undetectable levels of HBV DNA (61%) and normalization of ALT levels (70%), and third in loss of hepatitis B surface antigen (1%). In HBeAg-negative patients, tenofovir was the most effective in inducing undetectable levels of HBV DNA (94%) and improving liver histology (65%); it ranked second for normalization of ALT levels (73%). CONCLUSIONS In the first year of treatment for CHB, tenofovir and entecavir are the most potent oral antiviral agents for HBeAg-positive patients; tenofovir is most effective for HBeAg-negative patients.

Carotid endarterectomy for asymptomatic carotid stenosis: a meta-analysis

Abstract Objective: To assess the value of carotid endarterectomy for prevention of stroke in patients with asymptomatic carotid stenosis. Design: Systematic review and meta-analysis of randomised controlled trials in patients with asymptomatic carotid stenosis in which subjects were allocated to carotid endarterectomy or to medical treatment alone. Subjects: Five trials enrolled 2440 patients with stenosis 50%. Main outcome measures: Stroke ipsilateral to the stenosis, all strokes, and perioperative complications (stroke or death). Results: In patients who underwent carotid endarterectomy (n=1215) there was a significant reduction in the odds of ipsilateral stroke plus perioperative stroke or death (odds ratio 0.62; 95% confidence interval 0.44 to 0.86), corresponding to a 2% absolute risk reduction over about 3.1 years. The prevalence of stroke in any location was also reduced (0.68; 0.51 to 0.9) in patients undergoing carotid endarterectomy. During the immediate postoperative period there was an increased prevalence of stroke or death among such patients (4.51; 2.36 to 8.64). Conclusion: Carotid endarterectomy in patients with asymptomatic carotid stenosis unequivocally reduces the incidence of ipsilateral stroke, though the absolute benefit is relatively small. Given the modest benefit of surgery for unselected patients with asymptomatic carotid artery stenosis carotid endarterectomy cannot be routinely recommended for these patients pending reliable identification of high risk subgroups, and medical management is a sensible alternative for most patients.

Should meta-analysts search Embase in addition to Medline?

It is widely accepted that meta-analysts should search multiple databases. The selection of databases is ideally based on the potential contribution of each database to the project or on the potential for bias if a database is excluded, as supported by research evidence. We explore whether searching Embase yields additional trials that influence a meta-analysis. We identified meta-analyses that searched Medline and Embase. A random-effects weighted mean method was used to estimate the intervention effect in articles indexed only in Embase compared with those indexed elsewhere. On average, Embase-unique trials yielded significantly smaller estimates by 29% (ratio of odds ratio [ROR] 0.71, 95% confidence interval [CI] 0.56-0.90) but influenced the pooled estimate by an average of only 6% (ROR 0.94, 95% CI 0.88-0.99). Searching Medline but not Embase risks biasing a meta-analysis by finding studies that show larger estimates, but their prevalence seems low enough that the risk may be slight, provided the rest of the search is comprehensive.

A comparison of the quality of Cochrane reviews and systematic reviews published in paper-based journals.

This study set out to compare Cochrane reviews and reviews published in paper-based journals. Two assessment tools were used to collect the data, a 23-itemchecklist developed by Sacks and a nine-itemscale developed by Oxman. Cochrane reviews were found to be better at reporting some items and paper-based reviews at reporting others. The overall quality was found to be low. This represents a serious situation because clinicians, health policy makers, and consumers are often told that systematic reviews represent “the best available evidence.” In the period since this study, the Cochrane Collaboration has taken steps to improve the quality of its reviews through, for example, more thorough prepublication refereeing, developments in the training and support offered to reviewers, and improvements in the system for postpublication peer review. In addition, the use of evidence-based criteria (i.e., the QUOROM statement) for reporting systematic reviews may help further to improve their quality.

Pharmacological management of intermittent claudication: a meta-analysis of randomised trials.

Intermittent claudication, a symptom of atherosclerosis in the large vessels of the lower limbs, greatly affects patient mobility and quality of life. Medical therapy for a moderate form of this condition includes vasodilators, antiplatelet agents and alternative treatments such as ginkgo biloba.A meta-analysis of results from 52 trials (including 5088 patients) was conducted for all current medical therapies for intermittent claudication. After 24 weeks, some of the medical therapies were found to be more effective than placebo for the primary end-points of either pain-free walking distance or maximum walking distance.Vasodilators presented the best results in walking distance. Pentoxifylline offered better results than naftidrofuryl, although the treatment benefit, measured in additional metres walked with treatment than without, was modest. Antiplatelets, ginkgo biloba and levocarnitine were slightly more effective than placebo, although the treatment benefit was of limited clinical importance. On average, patients walked 60m further with therapy than without, and only about half of that added distance was pain-free. Very little consistent information was available for other clinical end-points, such as overall mortality and adverse effects.These data suggest that some of the medical therapy, pentoxifylline in particular, can only modestly increase functional status in patients with moderate intermittent claudication. There is a need for uniformity in research design and reporting of trials. A future trial comparing medical therapy with physical therapy is indicated.

Cost‐effectiveness of biologic response modifiers compared to disease‐modifying antirheumatic drugs for rheumatoid arthritis: A systematic review

Rheumatoid arthritis (RA) is a chronic systemic autoimmune disease that affects approximately 1% of the population (1,2). The course of RA varies, but for a substantial proportion of patients it is characterized by persistent pain and stiffness, progressive joint destruction, functional disability, and premature mortality (3). RA also presents a serious socioeconomic burden in terms of direct medical costs (associated with resources consumed to research, detect, and treat RA) and indirect costs (associated with lost productivity, early mortality, and time contributed by caregivers) (4–9). The pharmacologic management of RA has been transformed with the introduction of disease-modifying antirheumatic drugs (DMARDs), a large class of drugs that includes hydroxychloroquine, leflunomide, methotrexate, and sulfasalazine. Whereas drugs such as nonsteroidal antiinflammatory drugs (NSAIDs) and corticosteroids control symptoms, DMARDs slow the progression of joint damage that leads to loss of function (10,11). Guidelines advocate treatment with DMARDs as soon as RA is diagnosed to control symptoms and delay disease progression (12). Newly developed biologic response modifiers (biologics) offer even more hope, having a greater potential to suppress disease activity, improve quality of life, and inhibit joint destruction (13–15). But while biologics may have the greatest potential to slow the course of RA, these drugs cost substantially more than DMARDs. Consequently, current guidelines recommend biologics for patients with inadequate responses to DMARDs largely because higher costs preclude their widespread early use (12,16–18). Therefore, at the core of the debate is whether the superior clinical outcomes achieved with biologics are worth their higher costs. Should earlier treatment with biologics be considered, given their potential to slow disease progression and extend productivity, thereby reducing downstream direct costs associated with health care utilization and indirect costs associated with lost productivity? Since the introduction of cyclooxygenase 2–inhibiting NSAIDs and DMARDs, RA drug costs have more than doubled, and now with the recent introduction of biologics, these costs are expected to increase (19). Not surprisingly, many agencies (including the National Institutes of Health in the US) have identified the cost-effectiveness of biologics as one of the highest-priority research topics in the pharmacologic treatment of RA. Decision makers in public and private health care systems need a synopsis of current economic evidence upon which to base funding decisions. An understanding of the existing literature is also essential to identify gaps in the current evidence and to inform the development of future economic evaluations. We therefore undertook a review of the literature to identify and critically appraise existing economic evaluations of biologics versus DMARDs for adults with RA and to determine whether the incremental cost-effectiveness is within the range of generally accepted medical interventions. Gabrielle van der Velde, DC, PhD, Claire Bombardier, MD, FRCPC: Toronto Health Economics and Technology Assessment Collaborative and Institute for Work and Health, Toronto, Ontario, Canada; Ba’ Pham, PhD (Candidate), Márcio Machado, PharmD, PhD (current address: GlaxoSmithKline, Rio de Janeiro, Brazil), Luciano Ieraci, MSc, William Witteman, MIS, Murray Krahn, MD, MSc: Toronto Health Economics and Technology Assessment Collaborative, Toronto, Ontario, Canada. Dr. Bombardier has received consultant fees, speaking fees, and/or honoraria (less than

Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study

10,000 each) from AstraZeneca, Pfizer, Biogen Idec, PESI Healthcare, and Roche, and (more than

Is there a "best" way to detect and minimize publication bias? An empirical evaluation.

10,000 each) from Abbott Canada and Abbott International, Wyeth (Merck), and Schering (Merck). Address correspondence to Gabrielle van der Velde, DC, PhD, THETA Collaborative, Leslie L. Dan Pharmacy Building, University of Toronto, 6th Floor, Room 658, 144 College Street, Toronto, Ontario, M5S 3M2 Canada. E-mail: gabrielle. vandervelde@theta.utoronto.ca. Submitted for publication January 29, 2010; accepted in revised form August 19, 2010. Arthritis Care & Research Vol. 63, No. 1, January 2011, pp 65–78 DOI 10.1002/acr.20338