Margaret Piper

The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group

The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Initiative, established by the National Office of Public Health Genomics at the Centers for Disease Control and Prevention, supports the development and implementation of a rigorous, evidence-based process for evaluating genetic tests and other genomic applications for clinical and public health practice in the United States. An independent, non-federal EGAPP Working Group (EWG), a multidisciplinary expert panel selects topics, oversees the systematic review of evidence, and makes recommendations based on that evidence. This article describes the EGAPP processes and details the specific methods and approaches used by the EWG.

Recombinant human erythropoiesis-stimulating agents and mortality in patients with cancer: a meta-analysis of randomised trials.

BACKGROUND Erythropoiesis-stimulating agents reduce anaemia in patients with cancer and could improve their quality of life, but these drugs might increase mortality. We therefore did a meta-analysis of randomised controlled trials in which these drugs plus red blood cell transfusions were compared with transfusion alone for prophylaxis or treatment of anaemia in patients with cancer. METHODS Data for patients treated with epoetin alfa, epoetin beta, or darbepoetin alfa were obtained and analysed by independent statisticians using fixed-effects and random-effects meta-analysis. Analyses were by intention to treat. Primary endpoints were mortality during the active study period and overall survival during the longest available follow-up, irrespective of anticancer treatment, and in patients given chemotherapy. Tests for interactions were used to identify differences in effects of erythropoiesis-stimulating agents on mortality across prespecified subgroups. FINDINGS Data from a total of 13 933 patients with cancer in 53 trials were analysed. 1530 patients died during the active study period and 4993 overall. Erythropoiesis-stimulating agents increased mortality during the active study period (combined hazard ratio [cHR] 1.17, 95% CI 1.06-1.30) and worsened overall survival (1.06, 1.00-1.12), with little heterogeneity between trials (I(2) 0%, p=0.87 for mortality during the active study period, and I(2) 7.1%, p=0.33 for overall survival). 10 441 patients on chemotherapy were enrolled in 38 trials. The cHR for mortality during the active study period was 1.10 (0.98-1.24), and 1.04 (0.97-1.11) for overall survival. There was little evidence for a difference between trials of patients given different anticancer treatments (p for interaction=0.42). INTERPRETATION Treatment with erythropoiesis-stimulating agents in patients with cancer increased mortality during active study periods and worsened overall survival. The increased risk of death associated with treatment with these drugs should be balanced against their benefits. FUNDING German Federal Ministry of Education and Research, Medical Faculty of University of Cologne, and Oncosuisse (Switzerland).

Diagnostic and Predictive Accuracy of Blood Pressure Screening Methods With Consideration of Rescreening Intervals: A Systematic Review for the U.S. Preventive Services Task Force

Nearly 1 in 3 U.S. adults has high blood pressure (BP), including two thirds of those aged 60 years or older (1). Elevated BP is the largest contributing risk factor to all-cause and cardiovascular mortality (2). Despite the clear importance of accurate diagnosis of high BP, recommendations for BP measurement protocols and rescreening intervals are not based on systematic reviews of the literature (3, 4), and recommended protocols, such as repeated measurements, are rarely followed in routine health care settings (59). To help address these issues, newer measurement methods have been developed to reduce error, simplify performance of repeated measurements, evaluate BP throughout the 24-hour cycle, and allow use in nonmedical settings. Evidence-based measurement methods and rescreening intervals could improve the benefits and efficiency of BP screening. In 2007, the U.S. Preventive Services Task Force (USPSTF) reaffirmed its 2003 A recommendation to screen for high BP in adults aged 18 years or older (10). In 2003, a synthesis of indirect evidence for BP screening found good-quality evidence that treatment of high BP in adults substantially decreases the incidence of cardiovascular events (11). Both reviews found that screening and treatment for high BP cause few major harms (11, 12). Given the strong evidence base for the previous recommendations and recently updated guidelines for BP control (4, 13), the USPSTF did not believe that updating the indirect evidence path was necessary. However, the previous systematic reviews did not identify a BP measurement reference standard, address diagnostic accuracy of BP measurement methods and protocols, or determine the most appropriate rescreening interval. Our evidence review was designed to address these important aspects of screening for high BP and update the direct evidence of benefits and harms of screening. Methods To conduct this review, we developed an analytic framework with 5 key questions (Appendix Figure 1) that examined direct evidence for the benefits and harms of screening for high BP (key questions 1 and 5, respectively), diagnostic accuracy of office BP measurement (OBPM) (key question 2), prediction of cardiovascular events by BP method and diagnostic accuracy of nonoffice measurement (key question 3), and rescreening interval (key question 4). Detailed methods are available in our full evidence report (14). The analytic framework, review questions, and methods for locating and qualifying evidence were posted on the USPSTF Web site for public comment before we started the review, and the final versions reflect public input. Appendix Figure 1. Analytic framework. ABPM = ambulatory blood pressure monitoring; BP = blood pressure; CHD = coronary heart disease; CVD = cardiovascular disease; ESKD = end-stage kidney disease; HBPM = home blood pressure monitoring; HF = heart failure. * Defined as the threshold for pharmacologic treatment. Data Sources and Searches We searched MEDLINE, PubMed, the Cochrane Central Register of Controlled Trials, and CINAHL from 2003 through 8 August 2014 to update benefits and harms of screening for high BP. We searched the same databases (excluding CINAHL) through 24 February 2014 as follows: starting in 1992 (to allow for implementation of the first guidelines for validation of BP monitoring devices [15]) for prediction of cardiovascular events by BP method and diagnostic accuracy of nonoffice measurement, and starting in 1966 (the beginning of MEDLINE) for rescreening interval. On the basis of the findings from these updated searches, we did not further update them because any studies we found would probably not have changed the overall conclusions. We also searched bibliographies of relevant reviews, included studies, and publication lists of highly referenced studies. Study Selection Two investigators independently reviewed abstracts and full-text articles against prespecified inclusion and exclusion criteria (14). We required all studies to have enrolled untreated adults and to have been conducted in countries rated as very high on the 2013 Human Development Index (16). For prediction of cardiovascular events, we allowed studies that included treated patients because a proportion of persons followed over time would inevitably begin treatment. Ambulatory BP monitoring (ABPM) and home BP monitoring (HBPM) devices were eligible for use in confirming an initially elevated OBPM result. For screening benefits and harms, cardiovascular events we analyzed included fatal or nonfatal myocardial infarction; sudden cardiac death; stroke; heart failure; atrial fibrillation; transient ischemic attack; end-stage kidney disease; or a composite of any of the aforementioned events, excluding cardiovascular symptoms, angina, revascularization, carotid intimamedia thickness, and left ventricular hypertrophy. For diagnostic accuracy of OBPM, we included studies that compared different office-based devices or measurement protocols and reported sensitivity, specificity, predictive values, or concordance (for example, ). For diagnostic accuracy of confirmatory BP measurement methods, eligible study populations had an initial elevated office BP at screening, which allowed for reporting or calculation of the positive predictive value (PPV). For prediction of cardiovascular events, eligible studies followed a cohort of patients over time and reported the associations (hazard or risk ratios) of BP as a continuous variable, measured by at least 2 methods at baseline, with data on overall mortality or cardiovascular events collected during follow-up. For rescreening interval, we included studies that followed cohorts of initially nonhypertensive adults over time and reported hypertension incidence at rescreening intervals of up to 6 years. Data Extraction and Quality Assessment One investigator abstracted data from all included studies, and a second checked for accuracy. Two investigators independently assessed the quality of included studies by using predefined, design-specific criteria (1719). We rated study quality as good, fair, or poor and excluded all poor-quality studies (17). We resolved disagreements about quality through discussion with a third investigator. Where reported, studies with various threats to internal validity were downgraded to fair-quality according to USPSTF standards (17). Data Synthesis and Analysis We qualitatively described the results on the benefits and harms of screening. Per our protocol, we first calculated the diagnostic accuracy of OBPM by using the recommendations of the American Heart Association as the reference standard because there is no gold standard for BP measurement (3). With the subsequent identification of ABPM as the best predictor of cardiovascular events, we calculated the diagnostic accuracy of OBPM and confirmatory BP measurement methods by using ABPM as the reference standard where possible. We qualitatively described all diagnostic accuracy results because data were insufficient for quantitative synthesis. For prediction of cardiovascular events, we combined fatal and nonfatal events within outcome categories (cardiovascular, stroke, and cardiac). Risk was most commonly reported as the hazard ratio associated with each 10mm Hg increase in systolic BP and each 5mm Hg increase in diastolic BP. We converted hazard ratios to these common increments if they were reported differently (14). We depicted the hazard ratios in forest plots for qualitative evaluation; because of the small numbers of studies for each outcome and heterogeneity across studies, we did not calculate summary meta-analytic estimates of risk to determine the best BP measurement method for prediction. We conducted exploratory meta-analyses to compare ABPM protocols (24-hour, daytime, and nighttime) by generating estimates of cardiovascular events or mortality risk for each protocol by using the DerSimonianLaird random-effects method (20). In sensitivity analyses, these results were compared to estimates generated by using profile likelihood (21) and KnappHartung methods (22). For rescreening, we pooled reported incidence rates across all studies to generate a weighted mean incidence at yearly intervals (reported within0.5 year). We qualitatively examined within-study comparisons among a priori subgroups of age, BP, sex, body mass index (BMI), smoking status, and race/ethnicity (14). When constructing the overall summary of evidence (Appendix Table 1), we evaluated included studies within the context of each review question for consistency of results for important outcomes and relevance to primary care. Appendix Table 1. Overall Summary of Evidence, by Key Question Role of the Funding Source Staff from the Agency for Healthcare Research and Quality (AHRQ) provided oversight for the project and assisted in external review of the companion draft evidence synthesis. Liaisons for the USPSTF helped to resolve issues about the scope of the review but were not involved in the conduct of the review. Results We reviewed 19309 abstracts and 1171 articles for possible inclusion (Appendix Figure 2). Appendix Figure 2. Summary of evidence search and selection. KQ = key question. * Surveillance search results through August 2014 for trials reporting direct benefits of screening were not included; no additional trials were identified. Benefits of Screening for High BP For direct evidence of screening benefit, we included only randomized, controlled trials (RCTs) that reported changes in health outcomes as a result of screening for hypertension compared with no screening. We identified 1 good-quality cluster RCT of a community pharmacybased BP screening program targeting adults aged 65 years or older (23). Trained volunteer health educators also provided participants with educational materials and resources to support self-management. This trial found fewer annual composite cardiovascular-related hospitalizations in the intervention group than in t

A Rapid-ACCE review of CYP2C9 and VKORC1 alleles testing to inform warfarin dosing in adults at elevated risk for thrombotic events to avoid serious bleeding

Purpose: Summarize evidence regarding genetic testing in adults to inform warfarin dosing to reduce adverse drug events such as serious bleeding.Methods: Review published (and selected gray) literature using the Rapid-ACCE structure that addresses analytic validity, clinical validity, clinical utility, and ethical, legal, and social implications.Results: Preliminary data suggest overall analytic sensitivity and specificity will be 98% or higher for CYP2C9 genotyping, but strength of evidence for analytic validity is low, especially for VKORC1 testing. Strength of evidence is high for the clinical validity of both genes in predicting stable warfarin dose, an intermediate outcome, but is low for the association between CYP2C9 testing and severe bleeding events (clinical sensitivity 46% (95% CI 32–60%); specificity 69% (95% CI 62–75%) and absent for bleeding events associated with VKORC1 testing. No data are available to document clinical utility of genotyping before warfarin dosing.Conclusions: The most important gaps identified are: which variants should be included in a testing panel, lack of data from external proficiency testing, lack of validated dosing algorithm incorporating genetic and nongenetic factors, evidence of clinical utility, reliable economic analyses, and methods to address several ethical, legal, and social implications issues.

Evidence of Clinical Utility: An Unmet Need in Molecular Diagnostics for Patients with Cancer

This article defines and describes best practices for the academic and business community to generate evidence of clinical utility for cancer molecular diagnostic assays. Beyond analytical and clinical validation, successful demonstration of clinical utility involves developing sufficient evidence to demonstrate that a diagnostic test results in an improvement in patient outcomes. This discussion is complementary to theoretical frameworks described in previously published guidance and literature reports by the U.S. Food and Drug Administration, Centers for Disease Control and Prevention, Institute of Medicine, and Center for Medical Technology Policy, among others. These reports are comprehensive and specifically clarify appropriate clinical use, adoption, and payer reimbursement for assay manufacturers, as well as Clinical Laboratory Improvement Amendments–certified laboratories, including those that develop assays (laboratory developed tests). Practical criteria and steps for establishing clinical utility are crucial to subsequent decisions for reimbursement without which high-performing molecular diagnostics will have limited availability to patients with cancer and fail to translate scientific advances into high-quality and cost-effective cancer care. See all articles in this CCR Focus section, “The Precision Medicine Conundrum: Approaches to Companion Diagnostic Co-development.” Clin Cancer Res; 20(6); 1428–44. ©2014 AACR.

Improving the efficiency and relevance of evidence-based recommendations in the era of whole-genome sequencing: an EGAPP methods update

To provide an update on recent revisions to Evaluation of Genomic Applications in Practice and Prevention (EGAPP) methods designed to improve efficiency, and an assessment of the implications of whole genome sequencing for evidence-based recommendation development. Improvements to the EGAPP approach include automated searches for horizon scanning, a quantitative ranking process for topic prioritization, and the development of a staged evidence review and evaluation process. The staged process entails (i) triaging tests with minimal evidence of clinical validity, (ii) using and updating existing reviews, (iii) evaluating clinical validity prior to analytic validity or clinical utility, (iv) using decision modeling to assess potential clinical utility when direct evidence is not available. EGAPP experience to date suggests the following approaches will be critical for the development of evidence based recommendations in the whole genome sequencing era: (i) use of triage approaches and frameworks to improve efficiency, (ii) development of evidence thresholds that consider the value of further research, (iii) incorporation of patient preferences, and (iv) engagement of diverse stakeholders. The rapid advances in genomics present a significant challenge to traditional evidence based medicine, but also an opportunity for innovative approaches to recommendation development.Genet Med 2013:15(1):14–24

The EGAPP initiative: Lessons learned

The Evaluation of Genomic Applications in Practice and Prevention Working Group was first convened in 2005 to develop and test evidence-based methods for the evaluation of genomic tests in transition from research to clinical and public health practice. Over the ensuing years, the Working Group has met 26 times, publishing eight recommendation statements, two methods papers, and one outcomes paper, as well as planning and serving as technical experts on numerous associated systematic reviews. Evaluation of Genomic Applications in Practice and Prevention methods have evolved to address implications of the proliferation of genome-wide association studies and are currently expanding to face challenges expected from clinical implementation of whole-genome sequencing tests. In this article, we review the work of the Evaluation of Genomic Applications in Practice and Prevention Working Group over the first 8 years of its existence with an emphasis on lessons learned throughout the process. It is hoped that in addition to the published methods of the Working Group, the lessons we have learned along the way will be informative to others who are producers and consumers of evidence-based guidelines in the field of genomic medicine.Genet Med 2014:16(3):217–224.

A standardized, evidence-based protocol to assess clinical actionability of genetic disorders associated with genomic variation.

Purpose:Genome and exome sequencing can identify variants unrelated to the primary goal of sequencing. Detecting pathogenic variants associated with an increased risk of a medical disorder enables clinical interventions to improve future health outcomes in patients and their at-risk relatives. The Clinical Genome Resource, or ClinGen, aims to assess clinical actionability of genes and associated disorders as part of a larger effort to build a central resource of information regarding the clinical relevance of genomic variation for use in precision medicine and research.Methods:We developed a practical, standardized protocol to identify available evidence and generate qualitative summary reports of actionability for disorders and associated genes. We applied a semiquantitative metric to score actionability.Results:We generated summary reports and actionability scores for the 56 genes and associated disorders recommended by the American College of Medical Genetics and Genomics for return as secondary findings from clinical genome-scale sequencing. We also describe the challenges that arose during the development of the protocol that highlight important issues in characterizing actionability across a range of disorders.Conclusion:The ClinGen framework for actionability assessment will assist research and clinical communities in making clear, efficient, and consistent determinations of actionability based on transparent criteria to guide analysis and reporting of findings from clinical genome-scale sequencing.Genet Med 18 12, 1258–1268.

A Health Services Research Agenda for Cellular, Molecular and Genomic Technologies in Cancer Care

Background: In recent decades, extensive resources have been invested to develop cellular, molecular and genomic technologies with clinical applications that span the continuum of cancer care. Methods: In December 2006, the National Cancer Institute sponsored the first workshop to uniquely examine the state of health services research on cancer-related cellular, molecular and genomic technologies and identify challenges and priorities for expanding the evidence base on their effectiveness in routine care. Results: This article summarizes the workshop outcomes, which included development of a comprehensive research agenda that incorporates health and safety endpoints, utilization patterns, patient and provider preferences, quality of care and access, disparities, economics and decision modeling, trends in cancer outcomes, and health-related quality of life among target populations. Conclusions: Ultimately, the successful adoption of useful technologies will depend on understanding and influencing the patient, provider, health care system and societal factors that contribute to their uptake and effectiveness in ‘real-world’ settings.

Family history and the natural history of colorectal cancer: Systematic review

Purpose:Family history of colorectal cancer (CRC) is a known risk factor for CRC and encompasses both genetic and shared environmental risks.Methods:We conducted a systematic review to estimate the impact of family history on the natural history of CRC and adherence to screening.Results:We found high heterogeneity in family-history definitions, the most common definition being one or more first-degree relatives. The prevalence of family history may be lower than the commonly cited 10%, and confirms evidence for increasing levels of risk associated with increasing family-history burden. There is evidence for higher prevalence of adenomas and of multiple adenomas in people with family history of CRC but no evidence for differential adenoma location or adenoma progression by family history. Limited data regarding the natural history of CRC by family history suggest a differential age or stage at cancer diagnosis and mixed evidence with respect to tumor location. Adherence to recommended colonoscopy screening was higher in people with a family history of CRC.Conclusion:Stratification based on polygenic and/or multifactorial risk assessment may mature to the point of displacing family history–based approaches, but for the foreseeable future, family history may remain a valuable clinical tool for identifying individuals at increased risk for CRC.Genet Med 17 9, 702–712.