Beth Devine

Interpreting Indirect Treatment Comparisons and Network Meta-Analysis for Health-Care Decision Making: Report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 1

Evidence-based health-care decision making requires comparisons of all relevant competing interventions. In the absence of randomized, controlled trials involving a direct comparison of all treatments of interest, indirect treatment comparisons and network meta-analysis provide useful evidence for judiciously selecting the best choice(s) of treatment. Mixed treatment comparisons, a special case of network meta-analysis, combine direct and indirect evidence for particular pairwise comparisons, thereby synthesizing a greater share of the available evidence than a traditional meta-analysis. This report from the ISPOR Indirect Treatment Comparisons Good Research Practices Task Force provides guidance on the interpretation of indirect treatment comparisons and network meta-analysis to assist policymakers and health-care professionals in using its findings for decision making. We start with an overview of how networks of randomized, controlled trials allow multiple treatment comparisons of competing interventions. Next, an introduction to the synthesis of the available evidence with a focus on terminology, assumptions, validity, and statistical methods is provided, followed by advice on critically reviewing and interpreting an indirect treatment comparison or network meta-analysis to inform decision making. We finish with a discussion of what to do if there are no direct or indirect treatment comparisons of randomized, controlled trials possible and a health-care decision still needs to be made.

Drug Treatment of Idiopathic Pulmonary Fibrosis: Systematic Review and Network Meta-Analysis

BACKGROUND Idiopathic pulmonary fibrosis (IPF) is a form of chronic progressive fibrosing interstitial lung disease of unknown origin. Recently, nintedanib and pirfenidone demonstrated efficacy in slowing disease progression and were approved by the US Food and Drug Administration. Although numerous treatments have been evaluated in IPF, none have shown significant decreases in mortality. The objective of this study was to identify all pharmacologic treatments evaluated for IPF and analyze their efficacy via Bayesian network meta-analysis and pairwise indirect treatment comparisons. This review did not evaluate the effect of steroid therapy. METHODS We searched MEDLINE, Embase, and the Cochrane Library for studies published on or before August 2014. Studies were required to contain a randomized evaluation of nonsteroidal drug therapy for treatment of IPF and be published in English. Key outcomes of interest for this analysis were pulmonary function as measured by FVC as well as all-cause and respiratory-specific death. All outcomes were analyzed via a Bayesian framework. RESULTS Our review identified 30 eligible studies that evaluated 16 unique treatments. Under both the fixed-effect and random-effect models for respiratory-specific mortality, no treatments performed better than placebo. For all-cause mortality, pirfenidone and nintedanib had effects approaching significance with credible intervals slightly crossing the null under a fixed-effect model. Notably, for respiratory-specific mortality, all-cause mortality, and decline in percent predicted FVC, nintedanib and pirfenidone were virtually indistinguishable and no clear advantage was detected. CONCLUSIONS Although two treatments have been approved for IPF on the basis of reduced decline in pulmonary function, neither one has a clear advantage on mortality outcomes.

Imaging Techniques for the Diagnosis of Hepatocellular Carcinoma: A Systematic Review and Meta-analysis.

Hepatocellular carcinoma (HCC) is the most common primary malignant neoplasm of the liver, usually developing in persons with chronic liver disease. Worldwide, it is the fifth most common type of cancer and the third most common cause of death from cancer (1). There were 25000 deaths attributed to liver and intrahepatic bile duct cancer in the United States in 2011 (2). Common causes of HCC are hepatitis C virus infection, hepatitis B virus infection, and alcohol abuse, although a substantial proportion of cases have no identifiable cause (35). Imaging modalities for HCC include ultrasonography, computed tomography (CT), and magnetic resonance imaging (MRI). Although CT and MRI provide higher-resolution images than ultrasonography, they are also more costly and, in the case of CT, are associated with radiation exposure (5). Because HCC is typically a hypervascular lesion, CT and MRI are performed with arterial-enhancing contrast agents. Microbubble-enhanced ultrasonography can also be performed, although agents are not yet approved by the U.S. Food and Drug Administration for this purpose, and microbubbles are present in the liver for only a limited duration (6). Other technical, patient, and tumor factors may also affect test performance (712). This article reviews the test performance of ultrasonography, MRI, and CT for detection of HCC and for evaluation of focal liver lesions. This was conducted as part of a larger review commissioned by the Agency for Healthcare Research and Quality (AHRQ) on HCC imaging (13). Supplement. Original Version (PDF) Methods Scope of the Review The protocol was developed by using a standardized process with input from experts and the public and was registered in the PROSPERO database (CRD42014007016) (14). The review protocol included key questions on the comparative test performance of imaging for detection of HCC and for evaluation of focal liver lesions. Detailed methods and data for the review, including search strategies, inclusion criteria, and abstraction and quality ratings tables, are available in the full report, which also includes further key questions, full sensitivity and subgroup analyses, and an additional imaging modality (positron emission tomography) (13). Data Sources and Searches A research librarian searched multiple electronic databases, including MEDLINE (1998 to December 2013 for the full report; the update search for the review in this article was performed in December 2014), the Cochrane Library, and Scopus. Additional studies were identified by reviewing reference lists and from peer review suggestions. Study Selection Two investigators independently evaluated each study at the title/abstract and full-text article stages to determine inclusion eligibility (Appendix Table 1). We included studies on the test performance of ultrasonography, CT, or MRI against a reference standard for detection of HCC in surveillance or nonsurveillance settings (for example, imaging performed in patients undergoing treatment for liver disease or in whom HCC was previously diagnosed) or for further evaluation of focal liver lesions. Reference standards were histopathologic examination based on explanted liver or nonexplant histologic specimens, imaging plus clinical follow-up (for example, lesion growth), or a combination of these. Appendix Table 1. Inclusion and Exclusion Criteria We selected studies of ultrasonography (with or without contrast) and contrast-enhanced CT and MRI that met minimum technical criteria (non-multidetector or multidetector spiral CT, or 1.5- or 3.0-T MRI) (7). We excluded studies published before 1998 and those in which imaging began before 1995, unless the imaging methods met minimum technical criteria; studies of MRI with contrast agents no longer commercially produced (for example, superparamagnetic iron oxide [ferumoxides or ferucarbotran] or mangafodipir); and studies of CT arterial portography, CT hepatic angiography, and intraoperative ultrasonography. We included studies of ultrasonography microbubble contrast agents because they are commercially available and commonly used outside the United States, and efforts to obtain approval from the U.S. Food and Drug Administration are ongoing (1517). We excluded studies of diagnostic accuracy for non-HCC malignant lesions, including liver metastases. We included studies that reported results for HCC and cholangiocarcinoma together if cholangiocarcinoma lesions comprised less than 10% of the total. Studies on the accuracy of imaging for distinguishing HCC from a specific type of liver lesion (such as hemangioma or pseudolesion) and on the accuracy of imaging tests used in combination are addressed in the full report (13). We excluded studies published only as conference abstracts and included only English-language articles. The literature flow diagram is shown in Appendix Figure 1. Appendix Figure 1. Summary of evidence search and selection. * Studies of positron emission tomography; effects on clinical decisions, clinical outcomes, or staging; and accuracy for distinguishing hepatocellular carcinoma lesions from another specific type of liver lesion are addressed in the full report (13). Data Abstraction and Quality Rating One investigator abstracted details on the study design, dates of imaging and publication, patient population, country, sample size, imaging method and associated technical factors (Appendix Table 2), and results. Two investigators independently applied the approach recommended in the AHRQ Methods Guide for Medical Test Reviews to assess risk of bias as high, moderate, or low (18, 19). Appendix Table 2. Technical Factors Abstracted, by Imaging Modality Data Synthesis We conducted meta-analysis with a bivariate logistic mixed random-effects model that incorporated the correlation between sensitivity and specificity, using SAS software, version 9.3 (SAS Institute) (20). We assumed bivariate normal distributions for sensitivity and specificity. Statistical heterogeneity was measured with the random-effect variance (2). We calculated positive and negative likelihood ratios by using the summarized sensitivity and specificity (21, 22). We analyzed data separately for each imaging modality; ultrasonography with and without contrast were also analyzed separately. We also separately analyzed studies in which imaging was performed for detection of HCC and for evaluation of focal liver lesions; studies on HCC detection were further stratified by setting (surveillance or nonsurveillance). We separately analyzed test performance by using patients with HCC or by using HCC lesions (one patient can have multiple lesions) as the unit of analysis. Other sensitivity and subgroup analyses were conducted on the reference standard, factors related to risk of bias, country, technical factors (Appendix Table 2), tumor factors (such as HCC lesion size or degree of tumor differentiation), and patient characteristics (for example, severity of underlying liver disease, underlying cause of liver disease, and body mass index). We performed separate analyses on the subset of studies that directly compared 2 or more imaging modalities or techniques in the same population against a common reference standard (23). We used the same bivariate logistic mixed-effects model as described above, with an added indicator variable for imaging modalities. We also performed meta-analyses for within-study comparisons on lesion size, degree of tumor differentiation, and (when data were available) technical factors. We graded the strength of each body of evidence as high, moderate, low, or insufficient on the basis of the aggregate risk of bias, consistency, precision, and directness (24). Role of the Funding Source This research was funded by the AHRQ Effective Health Care Program. Investigators worked with AHRQ staff to develop and refine the review protocol. The AHRQ staff had no role in conducting the review, and the investigators are solely responsible for the content of the manuscript and the decision to submit for publication. Results Of the 5202 citations identified at the title and abstract level, 890 articles seemed to meet inclusion criteria and were selected for further full-text review. After full-text review, 241 studies (Appendix Table 3) met inclusion criteria for the key questions and imaging modalities addressed in this review (Appendix Figure 1). Appendix Table 3. References to Articles That Met the Inclusion Criteria Appendix Table 3Continued. Appendix Table 3Continued. Sixty-eight studies evaluated ultrasonography (Appendix Table 3), 131 evaluated CT (25153), and 125 evaluated MRI (Appendix Table 3). Almost all studies reported sensitivity, but specificity was available in only 139 studies. We rated 5 studies as having low risk of bias (56, 99, 128, 132, 154), 199 as having moderate risk of bias, and 89 as having high risk of bias (13). One hundred twenty-five studies avoided use of a casecontrol design, 160 used blinded design, and 75 were prospective. More studies were conducted in Asia (190 studies) than in Australia, Canada, the United States, or Europe (95 studies in total for these regions). In 166 studies, imaging began in or after 2003 (13). Twenty-eight studies evaluated CT using methods that met minimum technical specifications (8-row multidetector CT; contrast rate 3 mL/s; at least arterial, portal venous, and delayed-phase imaging; delayed-phase imaging performed >120 s after administration of contrast; and enhanced imaging section thickness 5 mm), and 67 studies evaluated MRI using methods that met minimum technical specifications (1.5- or 3.0-T MRI; at least arterial, portal venous, and delayed-phase imaging; delayed-phase imaging performed >120 s after administration of contrast; and enhanced imaging section thickness 5 mm). Seventy-three MRI studies evaluated use of hepatic-specific contrast (for example, gadoxetic acid or gadobenate). Forty-seven ultrasonography studies evaluated use of

Original Research: Diffuse Lung DiseaseDrug Treatment of Idiopathic Pulmonary Fibrosis: Systematic Review and Network Meta-Analysis

BACKGROUND Idiopathic pulmonary fibrosis (IPF) is a form of chronic progressive fibrosing interstitial lung disease of unknown origin. Recently, nintedanib and pirfenidone demonstrated efficacy in slowing disease progression and were approved by the US Food and Drug Administration. Although numerous treatments have been evaluated in IPF, none have shown significant decreases in mortality. The objective of this study was to identify all pharmacologic treatments evaluated for IPF and analyze their efficacy via Bayesian network meta-analysis and pairwise indirect treatment comparisons. This review did not evaluate the effect of steroid therapy. METHODS We searched MEDLINE, Embase, and the Cochrane Library for studies published on or before August 2014. Studies were required to contain a randomized evaluation of nonsteroidal drug therapy for treatment of IPF and be published in English. Key outcomes of interest for this analysis were pulmonary function as measured by FVC as well as all-cause and respiratory-specific death. All outcomes were analyzed via a Bayesian framework. RESULTS Our review identified 30 eligible studies that evaluated 16 unique treatments. Under both the fixed-effect and random-effect models for respiratory-specific mortality, no treatments performed better than placebo. For all-cause mortality, pirfenidone and nintedanib had effects approaching significance with credible intervals slightly crossing the null under a fixed-effect model. Notably, for respiratory-specific mortality, all-cause mortality, and decline in percent predicted FVC, nintedanib and pirfenidone were virtually indistinguishable and no clear advantage was detected. CONCLUSIONS Although two treatments have been approved for IPF on the basis of reduced decline in pulmonary function, neither one has a clear advantage on mortality outcomes.

Lymphovascular Invasion as a Prognostic Indicator in Stage I Non-Small Cell Lung Cancer: A Systematic Review and Meta-Analysis

BACKGROUND Lymphovascular invasion (LVI) is considered a high-risk pathologic feature in resected non-small cell carcinoma (NSCLC). The ability to stratify stage I patients into risk groups may permit refinement of adjuvant treatment recommendations. We performed a systematic review and meta-analysis to evaluate whether the presence of LVI is associated with disease outcome in stage I NSCLC patients. METHODS A systematic search of the literature was performed (1990 to December 2012 in MEDLINE/EMBASE). Two reviewers independently assessed the quality of the articles and extracted data. Pooled hazard ratios (HRs) and 95% confidence intervals (CI) were estimated with a random effects model. Two end points were independently analyzed: recurrence-free survival (RFS) and overall survival (OS). We analyzed unadjusted and adjusted effect estimates, resulting in four separate meta-analyses. RESULTS We identified 20 published studies that reported the comparative survival of stage I patients with and without LVI. The unadjusted pooled effect of LVI was significantly associated with worse RFS (HR, 3.63; 95% CI, 1.62 to 8.14) and OS (HR, 2.38; 95% CI, 1.72 to 3.30). Adjusting for potential confounders yielded similar results, with RFS (HR, 2.52; 95% CI, 1.73 to 3.65) and OS (HR, 1.81; 95% CI, 1.53 to 2.14) both significantly worse for patients exhibiting LVI. CONCLUSIONS The present study indicates that LVI is a strong prognostic indicator for poor outcome for patients with surgically managed stage I lung cancer. Future prospective lung cancer trials with well-defined methods for evaluating LVI are necessary to validate these results.

Outcomes From Health Information Exchange: Systematic Review and Future Research Needs

Background Health information exchange (HIE), the electronic sharing of clinical information across the boundaries of health care organizations, has been promoted to improve the efficiency, cost-effectiveness, quality, and safety of health care delivery. Objective To systematically review the available research on HIE outcomes and analyze future research needs. Methods Data sources included citations from selected databases from January 1990 to February 2015. We included English-language studies of HIE in clinical or public health settings in any country. Data were extracted using dual review with adjudication of disagreements. Results We identified 34 studies on outcomes of HIE. No studies reported on clinical outcomes (eg, mortality and morbidity) or identified harms. Low-quality evidence generally finds that HIE reduces duplicative laboratory and radiology testing, emergency department costs, hospital admissions (less so for readmissions), and improves public health reporting, ambulatory quality of care, and disability claims processing. Most clinicians attributed positive changes in care coordination, communication, and knowledge about patients to HIE. Conclusions Although the evidence supports benefits of HIE in reducing the use of specific resources and improving the quality of care, the full impact of HIE on clinical outcomes and potential harms are inadequately studied. Future studies must address comprehensive questions, use more rigorous designs, and employ a standard for describing types of HIE. Trial Registration PROSPERO Registry No CRD42014013285; http://www.crd.york.ac.uk/PROSPERO/ display_record.asp?ID=CRD42014013285 (Archived by WebCite at http://www.webcitation.org/6dZhqDM8t).

Minimal residual disease prior to allogeneic hematopoietic cell transplantation in acute myeloid leukemia: a meta-analysis

Minimal residual disease prior to allogeneic hematopoietic cell transplantation has been associated with increased risk of relapse and death in patients with acute myeloid leukemia, but detection methodologies and results vary widely. We performed a systematic review and meta-analysis evaluating the prognostic role of minimal residual disease detected by polymerase chain reaction or multiparametric flow cytometry before transplant. We identified 19 articles published between January 2005 and June 2016 and extracted hazard ratios for leukemia-free survival, overall survival, and cumulative incidences of relapse and non-relapse mortality. Pre-transplant minimal residual disease was associated with worse leukemia-free survival (hazard ratio=2.76 [1.90–4.00]), overall survival (hazard ratio=2.36 [1.73–3.22]), and cumulative incidence of relapse (hazard ratio=3.65 [2.53–5.27]), but not non-relapse mortality (hazard ratio=1.12 [0.81–1.55]). These associations held regardless of detection method, conditioning intensity, and patient age. Adverse cytogenetics was not an independent risk factor for death or relapse. There was more heterogeneity among studies using flow cytometry-based than WT1 polymerase chain reaction-based detection (I2=75.1% vs. <0.1% for leukemia-free survival, 67.8% vs. <0.1% for overall survival, and 22.1% vs. <0.1% for cumulative incidence of relapse). These results demonstrate a strong relationship between pre-transplant minimal residual disease and post-transplant relapse and survival. Outcome heterogeneity among studies using flow-based methods may underscore site-specific methodological differences or differences in test performance and interpretation.

Barriers and facilitators to exchanging health information: a systematic review.

OBJECTIVES We conducted a systematic review of studies assessing facilitators and barriers to use of health information exchange (HIE). METHODS We searched MEDLINE, PsycINFO, CINAHL, and the Cochrane Library databases between January 1990 and February 2015 using terms related to HIE. English-language studies that identified barriers and facilitators of actual HIE were included. Data on study design, risk of bias, setting, geographic location, characteristics of the HIE, perceived barriers and facilitators to use were extracted and confirmed. RESULTS Ten cross-sectional, seven multiple-site case studies, and two before-after studies that included data from several sources (surveys, interviews, focus groups, and observations of users) evaluated perceived barriers and facilitators to HIE use. The most commonly cited barriers to HIE use were incomplete information, inefficient workflow, and reports that the exchanged information that did not meet the needs of users. The review identified several facilitators to use. DISCUSSION Incomplete patient information was consistently mentioned in the studies conducted in the US but not mentioned in the few studies conducted outside of the US that take a collective approach toward healthcare. Individual patients and practices in the US may exercise the right to participate (or not) in HIE which effects the completeness of patient information available to be exchanged. Workflow structure and user roles are key but understudied. CONCLUSIONS We identified several facilitators in the studies that showed promise in promoting electronic health data exchange: obtaining more complete patient information; thoughtful workflow that folds in HIE; and inclusion of users early in implementation.

Everolimus and sunitinib for advanced pancreatic neuroendocrine tumors: a matching-adjusted indirect comparison

BackgroundEverolimus and sunitinib have been approved for the treatment advanced pancreatic neuroendocrine tumors, but have not been compared to each other in a randomized trial and have not demonstrated prolonged overall survival compared to placebo. This study aimed to indirectly compare overall and progression-free among everolimus, sunitinib and placebo across separate randomized trials.MethodsA matching adjusted indirect comparison was conducted in which individual patient data from the pivotal trial of everolimus (n = 410) were adjusted to match the inclusion criteria and average baseline characteristics reported for the pivotal trial of sunitinib (n = 171). Prior to matching, trial populations differed in baseline performance status and prior treatments. After matching, these and all other available baseline characteristics were balanced between trials.ResultsCompared to the placebo arm in the sunitinib trial, everolimus was associated with significantly prolonged overall survival (HR = 0.61, 95% CI = 0.38-0.98, p = 0.042).Compared to sunitinib, everolimus was associated with similar progression-free (hazard ratio for death (HR) = 0.84, 95% CI = 0.46–1.53, p = 0.578) and overall survival (HR = 0.81, 95% CI = 0.49–1.31, p = 0.383).ConclusionAfter adjusting for observed cross-trial differences, everolimus treatment was associated with longer overall survival than the placebo arm in the sunitinib trial for advanced pancreatic neuroendocrine tumors.

Mapping from Disease-Specific Measures to Health-State Utility Values in Individuals with Migraine

OBJECTIVE The objective of this study was to develop empirical algorithms that estimate health-state utility values from disease-specific quality-of-life scores in individuals with migraine. METHODS Data from a cross-sectional, multicountry study were used. Individuals with episodic and chronic migraine were randomly assigned to training or validation samples. Spearmans correlation coefficients between paired EuroQol five-dimensional (EQ-5D) questionnaire utility values and both Headache Impact Test (HIT-6) scores and Migraine-Specific Quality-of-Life Questionnaire version 2.1 (MSQ) domain scores (role restrictive, role preventive, and emotional function) were examined. Regression models were constructed to estimate EQ-5D questionnaire utility values from the HIT-6 score or the MSQ domain scores. Preferred algorithms were confirmed in the validation samples. RESULTS In episodic migraine, the preferred HIT-6 and MSQ algorithms explained 22% and 25% of the variance (R(2)) in the training samples, respectively, and had similar prediction errors (root mean square errors of 0.30). In chronic migraine, the preferred HIT-6 and MSQ algorithms explained 36% and 45% of the variance in the training samples, respectively, and had similar prediction errors (root mean square errors 0.31 and 0.29). In episodic and chronic migraine, no statistically significant differences were observed between the mean observed and the mean estimated EQ-5D questionnaire utility values for the preferred HIT-6 and MSQ algorithms in the validation samples. CONCLUSIONS The relationship between the EQ-5D questionnaire and the HIT-6 or the MSQ is adequate to use regression equations to estimate EQ-5D questionnaire utility values. The preferred HIT-6 and MSQ algorithms will be useful in estimating health-state utilities in migraine trials in which no preference-based measure is present.