Jonathan R Treadwell

AHRQ Series Paper 5: Grading the strength of a body of evidence when comparing medical interventions—Agency for Healthcare Research and Quality and the Effective Health-Care Program

OBJECTIVE To establish guidance on grading strength of evidence for the Evidence-based Practice Center (EPC) program of the U.S. Agency for Healthcare Research and Quality. STUDY DESIGN AND SETTING Authors reviewed authoritative systems for grading strength of evidence, identified domains and methods that should be considered when grading bodies of evidence in systematic reviews, considered public comments on an earlier draft, and discussed the approach with representatives of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group. RESULTS The EPC approach is conceptually similar to the GRADE system of evidence rating; it requires assessment of four domains: risk of bias, consistency, directness, and precision. Additional domains to be used when appropriate include dose-response association, presence of confounders that would diminish an observed effect, strength of association, and publication bias. Strength of evidence receives a single grade: high, moderate, low, or insufficient. We give definitions, examples, mechanisms for scoring domains, and an approach for assigning strength of evidence. CONCLUSION EPCs should grade strength of evidence separately for each major outcome and, for comparative effectiveness reviews, all major comparisons. We will collaborate with the GRADE group to address ongoing challenges in assessing the strength of evidence.

Systematic Review and Meta-Analysis of Bariatric Surgery for Pediatric Obesity

Objective:The prevalence of morbid obesity has risen sharply in recent years, even among pediatric patients. Bariatric surgery is used increasingly in an effort to induce weight loss, improve medical comorbidities, enhance quality of life, and extend survival. We performed a systematic review and meta-analysis of all published evidence pertaining specifically to bariatric surgery in pediatric patients. Methods:We systematically searched MEDLINE, EMBASE, 13 other databases, and article bibliographies to identify relevant evidence. Included studies must have reported outcome data for ≥3 patients aged ≤21, representing ≥50% of pediatric patients enrolled at that center. We only included English language articles on currently performed procedures when data were separated by procedure, and there was a minimum 1-year follow-up for weight and body mass index (BMI). Results:Eight studies of laparoscopic adjustable gastric banding (LAGB) reported data on 352 patients (mean BMI 45.8); 6 studies of Roux-en-Y gastric bypass (RYGB) included 131 patients (mean BMI 51.8); 5 studies of other surgical procedures included 158 patients (mean BMI 48.8). Average patient age was 16.8 years (range, 9–21). Meta-analyses of BMI reductions at longest follow-up indicated sustained and clinically significant BMI reductions for both LAGB and RYGB. Comorbidity resolution was sparsely reported, but surgery did appear to resolve some medical conditions including diabetes and hypertension. For LAGB, band slippage and micronutrient deficiency were the most frequently reported complications, with sporadic cases of band erosion, port/tube dysfunction, hiatal hernia, wound infection, and pouch dilation. For RYGB, more severe complications have been documented, such as pulmonary embolism, shock, intestinal obstruction, postoperative bleeding, staple line leak, and severe malnutrition. Conclusions:Bariatric surgery in pediatric patients results in sustained and clinically significant weight loss, but also has the potential for serious complications.

Long-Term Opioid Therapy for Chronic Noncancer Pain: A Systematic Review and Meta-Analysis of Efficacy and Safety

Opioid therapy for chronic noncancer pain (CNCP) is controversial due to concerns regarding long-term efficacy and adverse events (including addiction). We systematically reviewed the clinical evidence on patients treated with opioids for CNCP for at least six months. Of 115 studies identified by our search of eleven databases (through April 7, 2007), 17 studies (patients [n]=3,079) met inclusion criteria. Studies evaluated oral (studies [k]=7; n=1,504), transdermal (k=3; n=1, 993), and/or intrathecal (k=8; n=177) opioids. Many patients withdrew from the clinical trials due to adverse effects (oral: 32.5% [95% confidence interval (CI), 26.1%-39.6%]; intrathecal: 6.3% [95% CI, 2.9%-13.1%]; transdermal: 17.5% [95% CI, 6.5%-39.0%]), or due to insufficient pain relief (oral: 11.9% [95% CI, 7.8%-17.7%]; intrathecal: 10.5% [95% CI, 3.5%-27.4%]; transdermal: 5.8% [95% CI, 4.2%-7.3%]). Signs of opioid addiction were reported in only 0.05% (1/2,042) of patients and abuse in only 0.43% (3/685). There was an insufficient amount of data on transdermal opioids to quantify pain relief. For patients able to remain on oral or intrathecal opioids for at least six months, pain scores were reduced long-term (oral: standardized mean difference [SMD] 1.99, 95% CI, 1.17-2.80; intrathecal: SMD 1.33, 95% CI, 0.97-1.69). We conclude that many patients discontinue long-term opioid therapy due to adverse events or insufficient pain relief; however, weak evidence suggests that oral and intrathecal opioids reduce pain long-term in the relatively small proportion of individuals with CNCP who continue treatment.

Surgical checklists: a systematic review of impacts and implementation

Background Surgical complications represent a significant cause of morbidity and mortality with the rate of major complications after inpatient surgery estimated at 3–17% in industrialised countries. The purpose of this review was to summarise experience with surgical checklist use and efficacy for improving patient safety. Methods A search of four databases (MEDLINE, CINAHL, EMBASE and the Cochrane Database of Controlled Trials) was conducted from 1 January 2000 to 26 October 2012. Articles describing actual use of the WHO checklist, the Surgical Patient Safety System (SURPASS) checklist, a wrong-site surgery checklist or an anaesthesia equipment checklist were eligible for inclusion (this manuscript summarises all but the anaesthesia equipment checklists, which are described in the Agency for Healthcare Research and Quality publication). Results We included a total of 33 studies. We report a variety of outcomes including avoidance of adverse events, facilitators and barriers to implementation. Checklists have been adopted in a wide variety of settings and represent a promising strategy for improving the culture of patient safety and perioperative care in a wide variety of settings. Surgical checklists were associated with increased detection of potential safety hazards, decreased surgical complications and improved communication among operating staff. Strategies for successful checklist implementation included enlisting institutional leaders as local champions, incorporating staff feedback for checklist adaptation and avoiding redundancies with existing systems for collecting information. Conclusions Surgical checklists represent a relatively simple and promising strategy for addressing surgical patient safety worldwide. Further studies are needed to evaluate to what degree checklists improve clinical outcomes and whether improvements may be more pronounced in particular settings.

Systematic Review: Comparative Effectiveness of Core-Needle and Open Surgical Biopsy to Diagnose Breast Lesions

BACKGROUND Most women undergoing breast biopsy are found not to have cancer. PURPOSE To compare the accuracy and harms of different breast biopsy methods in average-risk women suspected of having breast cancer. DATA SOURCES Databases, including MEDLINE and EMBASE, searched from 1990 to September 2009. STUDY SELECTION Studies that compared core-needle biopsy diagnoses with open surgical diagnoses or clinical follow-up. DATA EXTRACTION Data were abstracted by 1 of 3 researchers and verified by the primary investigator. DATA SYNTHESIS 33 studies of stereotactic automated gun biopsy; 22 studies of stereotactic-guided, vacuum-assisted biopsy; 16 studies of ultrasonography-guided, automated gun biopsy; 7 studies of ultrasonography-guided, vacuum-assisted biopsy; and 5 studies of freehand automated gun biopsy met the inclusion criteria. Low-strength evidence showed that core-needle biopsies conducted under stereotactic guidance with vacuum assistance distinguished between malignant and benign lesions with an accuracy similar to that of open surgical biopsy. Ultrasonography-guided biopsies were also very accurate. The risk for severe complications is lower with core-needle biopsy than with open surgical procedures (<1% vs. 2% to 10%). Moderate-strength evidence showed that women in whom breast cancer was initially diagnosed by core-needle biopsy were more likely than women with cancer initially diagnosed by open surgical biopsy to be treated with a single surgical procedure (random-effects odds ratio, 13.7 [95% CI, 5.5 to 34.6]). LIMITATION The strength of evidence was rated low for accuracy outcomes because the studies did not report important details required to assess the risk for bias. CONCLUSION Stereotactic- and ultrasonography-guided core-needle biopsy procedures seem to be almost as accurate as open surgical biopsy, with lower complication rates. PRIMARY FUNDING SOURCE Agency for Healthcare Research and Quality.

Autologous hematopoietic cell transplantation for multiple sclerosis: a systematic review

Background and objectives: The purpose of this systematic review was to evaluate the safety and efficacy of autologous hematopoietic cell transplantation in patients with progressive multiple sclerosis (MS) refractory to conventional medical treatment. Methods: Eight case series met our a priori inclusion criteria for the primary outcome of progression-free survival. Individual study quality was rated using an 11-item scale for case series. The strength of the overall body of evidence for each outcome was rated using a system developed by the ECRI Institute. Data from different studies were statistically combined using meta-analysis. An additional six studies were included for a summary of mortality and morbidity. Results: For secondary progressive MS, immunoablative therapy with autologous bone marrow/peripheral blood stem cell transplantation was associated with higher progression-free survival (up to 3 years following treatment) when using intermediate-intensity conditioning regimens compared with high-intensity conditioning regimens. The evidence was insufficient to determine whether the treatment was effective in patients with other types of MS. Treatment-related mortality was about 2.7%. Conclusions: Patients with secondary progressive MS refractory to conventional medical treatment have longer progression-free survival following autologous stem cell transplantation with intermediate-intensity conditioning regimens than with high-intensity conditioning regimens.

Assessing equivalence and noninferiority

OBJECTIVE For systematic reviews, no guidance exists for what review methods support valid conclusions of equivalence (EQ) and noninferiority (NI). To provide such guidance, we convened a workgroup of 13 experienced systematic reviewers from seven evidence-based practice centers (EPCs) and the Agency for Healthcare Research and Quality (AHRQ). STUDY DESIGN AND SETTING The Lead EPC first performed two methods projects intended to assist the workgroup in clarifying the context, prioritizing the issues, targeting the scope, and summarizing the state of the art. RESULTS Based on expert opinion, we devised guidance in four areas: 1) Unique risk of bias issues for trials self-identifying as EQ-NI trials; 2) Setting the reviewers minimum important difference; 3) Analytic foundations for concluding EQ or NI; and 4) Language considerations when concluding EQ or NI. CONCLUSION This article summarizes the main recommendations, and the full guidance chapter appears on the AHRQ Web site.

Imaging for the Pretreatment Staging of Small cell Lung Cancer

BACKGROUND Small cell lung cancer (SCLC) is an aggressive form of lung cancer. Accurate staging is essential to select the optimal treatment plan to maximize survival. No consensus exists on standard imaging modalities for pretreatment staging of SCLC. MATERIALS AND METHODS We conducted a systematic review of the literature on imaging modalities in the pretreatment staging of SCLC. A systematic search of multiple databases identified relevant studies published from 2000 through June 2015. Outcomes of interest included test concordance, staging accuracy (sensitivity and specificity), choice of treatment, timeliness of treatment, and patient outcomes. RESULTS The search identified 2880 citations; 7 studies met inclusion criteria, n = 408 patients. Six of the seven studies were deemed to have moderate risk of bias, and one was deemed to have high risk of bias. One of the studies reported test concordance, three studies reported comparative accuracy of testing strategies, and four studies reported the accuracy of a single imaging modality. Analysis from these studies revealed that fluorodeoxyglucose-positron emission tomography/computed tomography (FDG-PET/CT) is more sensitive than multidetector CT for detecting osseous metastases, more sensitive than bone scintigraphy for detecting osseous metastases, and more sensitive for detecting any distant metastases. CONCLUSIONS Evidence is sparse on the use of imaging in the pretreatment staging of SCLC. There is a lack of evidence on patient-oriented outcomes and a lack of evidence on whether comparative accuracy or effectiveness is associated with patient factors. We found low-strength evidence suggesting that FDG-PET/CT is more sensitive than CT and bone scintigraphy for detecting osseous metastases.

Recommendations for Assessing the Risk of Bias in Systematic Reviews of Health Care Interventions

OBJECTIVES Risk-of-bias assessment is a central component of systematic reviews, but little conclusive empirical evidence exists on the validity of such assessments. In the context of such uncertainty, we present pragmatic recommendations that promote transparency and reproducibility in processes, address methodological advances in the risk-of-bias assessment, and can be applied consistently across review topics. STUDY DESIGN AND SETTING Epidemiological study design principles; available empirical evidence, risk-of-bias tools, and guidance; and workgroup consensus. RESULTS We developed recommendations for assessing the risk of bias of studies of health-care interventions specific to framing the focus and scope of risk-of-bias assessment; selecting the risk-of-bias categories; choosing assessment instruments; and conducting, analyzing, and presenting results of risk-of-bias assessments. Key recommendations include transparency and reproducibility of judgments, separating risk of bias from other constructs such as applicability and precision, and evaluating the risk of bias per outcome. We recommend against certain past practices, such as focusing on reporting quality, relying solely on study design or numerical quality scores, and automatically downgrading for industry sponsorship. CONCLUSION Risk-of-bias assessment remains a challenging but essential step in systematic reviews. We presented standards to promote transparency of judgments.

How Electronic Clinical Data Can Improve Health Technology Assessment

Health technology assessments represent comprehensive summaries of available evidence and information on a technology. They are used by medical decision makers in a variety of ways, including diagnostic testing, treatment selection, care management, patient perspectives, patient safety, insurance coverage, pharmaceutical innovation, equipment planning, device purchasing, and total cost-of-care. Electronic clinical data, which are captured routinely by clinicians and hospitals, are only rarely incorporated into formal health technology assessments. This disconnect reveals a key opportunity. In this paper, we discuss current uses of electronic clinical data, several benefits of including it in health technology assessments, potential pitfalls of that inclusion, and the implications for better medical decisions.