Tianjing Li

A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis

Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.

Network meta-analysis-highly attractive but more methodological research is needed

Network meta-analysis, in the context of a systematic review, is a meta-analysis in which multiple treatments (that is, three or more) are being compared using both direct comparisons of interventions within randomized controlled trials and indirect comparisons across trials based on a common comparator. To ensure validity of findings from network meta-analyses, the systematic review must be designed rigorously and conducted carefully. Aspects of designing and conducting a systematic review for network meta-analysis include defining the review question, specifying eligibility criteria, searching for and selecting studies, assessing risk of bias and quality of evidence, conducting a network meta-analysis, interpreting and reporting findings. This commentary summarizes the methodologic challenges and research opportunities for network meta-analysis relevant to each aspect of the systematic review process based on discussions at a network meta-analysis methodology meeting we hosted in May 2010 at the Johns Hopkins Bloomberg School of Public Health. Since this commentary reflects the discussion at that meeting, it is not intended to provide an overview of the field.

Comparative Effectiveness of Drug Treatments to Prevent Fragility Fractures: A Systematic Review and Network Meta-Analysis

CONTEXT Osteoporosis and osteopenia are associated with increased fracture incidence. OBJECTIVE The aim of this study was to determine the comparative effectiveness of different pharmacological agents in reducing the risk of fragility fractures. DATA SOURCES We searched multiple databases through 12/9/2011. STUDY SELECTION Eligible studies were randomized controlled trials enrolling individuals at risk of developing fragility fractures and evaluating the efficacy of bisphosphonates, teriparatide, selective estrogen receptor modulators, denosumab, or calcium and vitamin D. DATA EXTRACTION Reviewers working independently and in duplicate determined study eligibility and collected descriptive, methodological quality, and outcome data. DATA SYNTHESIS This network meta-analysis included 116 trials (139,647 patients; median age, 64 yr; 86% females and 88% Caucasians; median follow-up, 24 months). Trials were at low to moderate risk of bias. Teriparatide had the highest risk reduction of fractures (odds ratios, 0.42, 0.30, and 0.50 for hip, vertebral, and nonvertebral fractures, respectively) and the highest probability of being ranked first for efficacy (probabilities of 42, 49, and 79% for hip, vertebral, and nonvertebral fractures, respectively). However, differences to denosumab, zoledronate, risedronate, ibandronate, and alendronate were not statistically significant. Raloxifene and bazedoxifene were likely less effective, although these data were limited. Calcium and vitamin D were ineffective given separately but reduced the risk of hip fractures if given in combination (odds ratio, 0.81; 95% confidence interval, 0.68–0.96). CONCLUSIONS Teriparatide, bisphosphonates, and denosumab are most effective in reducing the risk of fragility fractures. Differences in efficacy across drugs are small; therefore, patients and clinicians need to consider their associated harms and costs.

Differences in Reporting of Analyses in Internal Company Documents Versus Published Trial Reports: Comparisons in Industry-Sponsored Trials in Off-Label Uses of Gabapentin

Using documents obtained through litigation, S. Swaroop Vedula and colleagues compared internal company documents regarding industry-sponsored trials of off-label uses of gabapentin with the published trial reports and find discrepancies in reporting of analyses.

Comparative Effectiveness of First-Line Medications for Primary Open-Angle Glaucoma: A Systematic Review and Network Meta-analysis.

TOPIC Primary open-angle glaucoma (POAG) is a highly prevalent condition worldwide and the most common cause of irreversible sight loss. The objective is to assess the comparative effectiveness of first-line medical treatments in patients with POAG or ocular hypertension through a systematic review and network meta-analysis, and to provide relative rankings of these treatments. CLINICAL RELEVANCE Treatment for POAG currently relies completely on lowering the intraocular pressure (IOP). Although topical drops, lasers, and surgeries can be considered in the initial treatment of glaucoma, most patients elect to start treatment with eye drops. METHODS We included randomized controlled trials (RCTs) that compared a single active topical medication with no treatment/placebo or another single topical medication. We searched CENTRAL, MEDLINE, EMBASE, and the Food and Drug Administrations website. Two individuals independently assessed trial eligibility, abstracted data, and assessed the risk of bias. We performed Bayesian network meta-analyses. RESULTS We included 114 RCTs with data from 20 275 participants. The overall risk of bias of the included trials is mixed. The mean reductions (95% credible intervals) in IOP in millimeters of mercury at 3 months ordered from the most to least effective drugs were as follows: bimatoprost 5.61 (4.94; 6.29), latanoprost 4.85 (4.24; 5.46), travoprost 4.83 (4.12; 5.54), levobunolol 4.51 (3.85; 5.24), tafluprost 4.37 (2.94; 5.83), timolol 3.70 (3.16; 4.24), brimonidine 3.59 (2.89; 4.29), carteolol 3.44 (2.42; 4.46), levobetaxolol 2.56 (1.52; 3.62), apraclonidine 2.52 (0.94; 4.11), dorzolamide 2.49 (1.85; 3.13), brinzolamide 2.42 (1.62; 3.23), betaxolol 2.24 (1.59; 2.88), and unoprostone 1.91 (1.15; 2.67). CONCLUSIONS All active first-line drugs are effective compared with placebo in reducing IOP at 3 months. Bimatoprost, latanoprost, and travoprost are among the most efficacious drugs, although the within-class differences were small and may not be clinically meaningful. All factors, including adverse effects, patient preferences, and cost, should be considered in selecting a drug for a given patient.

Outcomes in Cochrane Systematic Reviews Addressing Four Common Eye Conditions: An Evaluation of Completeness and Comparability

Introduction Choice of outcomes is critical for clinical trialists and systematic reviewers. It is currently unclear how systematic reviewers choose and pre-specify outcomes for systematic reviews. Our objective was to assess the completeness of pre-specification and comparability of outcomes in all Cochrane reviews addressing four common eye conditions. Methods We examined protocols for all Cochrane reviews as of June 2013 that addressed glaucoma, cataract, age-related macular degeneration (AMD), and diabetic retinopathy (DR). We assessed completeness and comparability for each outcome that was named in ≥25% of protocols on those topics. We defined a completely-specified outcome as including information about five elements: domain, specific measurement, specific metric, method of aggregation, and time-points. For each domain, we assessed comparability in how individual elements were specified across protocols. Results We identified 57 protocols addressing glaucoma (22), cataract (16), AMD (15), and DR (4). We assessed completeness and comparability for five outcome domains: quality-of-life, visual acuity, intraocular pressure, disease progression, and contrast sensitivity. Overall, these five outcome domains appeared 145 times (instances). Only 15/145 instances (10.3%) were completely specified (all five elements) (median = three elements per outcome). Primary outcomes were more completely specified than non-primary (median = four versus two elements). Quality-of-life was least completely specified (median = one element). Due to largely incomplete outcome pre-specification, conclusive assessment of comparability in outcome usage across the various protocols per condition was not possible. Discussion Outcome pre-specification was largely incomplete; we encourage systematic reviewers to consider all five elements. This will indicate the importance of complete specification to clinical trialists, on whose work systematic reviewers depend, and will indirectly encourage comparable outcome choice to reviewers undertaking related research questions. Complete pre-specification could improve efficiency and reduce bias in data abstraction and analysis during a systematic review. Ultimately, more completely specified and comparable outcomes could make systematic reviews more useful to decision-makers.

Flavored Tobacco Products in the United States: A Systematic Review Assessing Use and Attitudes

OBJECTIVES We systematically reviewed research examining use of and attitudes toward nonmenthol-flavored tobacco products to provide information relevant to a decision to regulate these products in the future. METHODS To identify eligible studies, we searched PubMed, CINHAL, Embase, LILACS, and PsycINFO on September 19, 2013, without date restrictions. We obtained additional studies via gray literature searches, expert contacts, and hand-searching citations of included articles. We included participants of all ages. We conducted a qualitative synthesis for included studies. RESULTS The 32 studies included in this review exhibited substantial heterogeneity and were of varied methodological quality. Findings from observational, experimental, and quasiexperimental studies suggest that flavored tobacco use is associated with young age and that consumers may perceive flavored products more favorably than nonflavored products. Evidence from qualitative studies indicates that flavoring in tobacco is viewed favorably by users and nonusers of these products. CONCLUSIONS The Food and Drug Administration has expressed interest in regulating flavored tobacco products. This systematic review strengthens the evidence base relating to this issue by synthesizing the literature from the United States on the use of and attitudes toward flavored tobacco. To address gaps in the literature, more research is needed to understand how flavoring impacts tobacco use over time. The evidence base would further be strengthened with the collection of brand-, flavor-, and product-specific data.

Systematic review with network meta‐analysis: pharmacological prophylaxis against post‐ ERCP pancreatitis

The efficacy of many pharmacological agents for preventing post‐ERCP pancreatitis (PEP) has been evaluated in randomised controlled trials (RCTs), but it is unclear which agent(s) should be used in clinical practice. Network meta‐analyses of RCTs are used to simultaneously compare several agents to determine their relative efficacy and identify priority agents for comparison in future RCTs.

Support of personalized medicine through risk-stratified treatment recommendations - an environmental scan of clinical practice guidelines

BackgroundRisk-stratified treatment recommendations facilitate treatment decision-making that balances patient-specific risks and preferences. It is unclear if and how such recommendations are developed in clinical practice guidelines (CPGs). Our aim was to assess if and how CPGs develop risk-stratified treatment recommendations for the prevention or treatment of common chronic diseases.MethodsWe searched the United States National Guideline Clearinghouse for US, Canadian and National Institute for Health and Clinical Excellence (United Kingdom) CPGs for heart disease, stroke, cancer, chronic obstructive pulmonary disease and diabetes that make risk-stratified treatment recommendations. We included only those CPGs that made risk-stratified treatment recommendations based on risk assessment tools. Two reviewers independently identified CPGs and extracted information on recommended risk assessment tools; type of evidence about treatment benefits and harms; methods for linking risk estimates to treatment evidence and for developing treatment thresholds; and consideration of patient preferences.ResultsWe identified 20 CPGs that made risk-stratified treatment recommendations out of 133 CPGs that made any type of treatment recommendations for the chronic diseases considered in this study. Of the included 20 CPGs, 16 (80%) used evidence about treatment benefits from randomized controlled trials, meta-analyses or other guidelines, and the source of evidence was unclear in the remaining four (20%) CPGs. Nine CPGs (45%) used evidence on harms from randomized controlled trials or observational studies, while 11 CPGs (55%) did not clearly refer to harms. Nine CPGs (45%) explained how risk prediction and evidence about treatments effects were linked (for example, applying estimates of relative risk reductions to absolute risks), but only one CPG (5%) assessed benefit and harm quantitatively and three CPGs (15%) explicitly reported consideration of patient preferences.ConclusionsOnly a small proportion of CPGs for chronic diseases make risk-stratified treatment recommendations with a focus on heart disease and stroke prevention, diabetes and breast cancer. For most CPGs it is unclear how risk-stratified treatment recommendations were developed. As a consequence, it is uncertain if CPGs support patients and physicians in finding an acceptable benefit- harm balance that reflects both profile-specific outcome risks and preferences.

Benefits and harms of roflumilast in moderate to severe COPD

Background Roflumilast, a phosphodiesterase 4 inhibitor, has been approved for the prevention of chronic obstructive pulmonary disease (COPD) exacerbations. It is unclear which patients will have a favourable benefit–harm balance with roflumilast. Our aim was to quantitatively assess the benefits and harms of roflumilast (500 µg/day) compared with placebo. Methods We used summary data released by the US Food and Drug Administration to estimate the treatment effects of roflumilast. Data from trials and observational studies were used to estimate the baseline risks for COPD exacerbations and gastrointestinal, neurological and psychiatric harms associated with roflumilast. Using simulation, we calculated the probability that roflumilast provides net benefit. We examined the impacts of different baseline risks for exacerbations and the severity of exacerbations, and varied weights (ie, relative importance) for outcomes and treated death as a competing risk in the analyses. Results The probability that roflumilast provides net benefit approximates 0% across different age categories of men and women with varying baseline risks for exacerbations. Using different weights for outcomes did not change the probability that roflumilast provides a net benefit. Only in the sensitivity analysis restricted to the prevention of severe exacerbations was there a probability of >50% that roflumilast provides a net benefit if the baseline risk of having at least one severe exacerbation per year exceeds 22%. Conclusions Our results suggest that roflumilast only provides a net benefit to patients at a high risk of severe exacerbations. Guideline developers should consider different recommendations for patients with COPD at different baseline risks for exacerbations.