W. David Dotson

The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group

The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Initiative, established by the National Office of Public Health Genomics at the Centers for Disease Control and Prevention, supports the development and implementation of a rigorous, evidence-based process for evaluating genetic tests and other genomic applications for clinical and public health practice in the United States. An independent, non-federal EGAPP Working Group (EWG), a multidisciplinary expert panel selects topics, oversees the systematic review of evidence, and makes recommendations based on that evidence. This article describes the EGAPP processes and details the specific methods and approaches used by the EWG.

Can UGT1A1 genotyping reduce morbidity and mortality in patients with metastatic colorectal cancer treated with irinotecan? An evidence-based review

This evidence-based review addresses the question of whether testing for UGT1A1 mutations in patients with metastatic colorectal cancer treated with irinotecan leads to improvement in outcomes (e.g., irinotecan toxicity, response to treatment, morbidity, and mortality), when compared with no testing. No studies were identified that addressed this question directly. The quality of evidence on the analytic validity of current UGT1A1 genetic testing methods is adequate (scale: convincing, adequate, inadequate), with available data indicating that both analytic sensitivity and specificity for the common genotypes are high. For clinical validity, the quality of evidence is adequate for studies reporting concentration of the active form of irinotecan (SN-38), presence of severe diarrhea, and presence of severe neutropenia stratified by UGT1A1 common genotypes. The strongest association for a clinical endpoint is for severe neutropenia. Patients homozygous for the *28 allele are 3.5 times more likely to develop severe neutropenia compared with individuals with the wild genotype (risk ratio 3.51; 95% confidence interval 2.03–6.07). The proposed clinical utility of UGT1A1 genotyping would be derived from a reduction in drug-related adverse reactions (benefits) while at the same time avoiding declines in tumor response rate and increases in morbidity/mortality (harms). At least three treatment options for reducing this increased risk have been suggested: modification of the irinotecan regime (e.g., reduce initial dose), use of other drugs, and/or pretreatment with colony-stimulating factors. However, we found no prospective studies that examined these options, particularly whether a reduced dose of irinotecan results in a reduced rate of adverse drug events. This is a major gap in knowledge. Although the quality of evidence on clinical utility is inadequate, two of three reviewed studies (and one published since our initial selection of studies for review) found that individuals homozygous for the *28 allele had improved survival. Three reviewed studies found statistically significant higher tumor response rates among individuals homozygous for the *28 allele. We found little or no direct evidence to assess the benefits and harms of modifying irinotecan regimens for patients with colorectal cancer based on their UGT1A1 genotype; however, results of our preliminary modeling of prevalence, acceptance, and effectiveness indicate that reducing the dose would need to be highly effective to have benefits outweigh harms. An alternative is to increase irinotecan dose among wild-type individuals to improve tumor response with minimal increases in adverse drug events. Given the large number of colorectal cancer cases diagnosed each year, a randomized controlled trial of the effects of irinotecan dose modifications in patients with colorectal cancer based on their UGT1A1 genotype is feasible and could clarify the tradeoffs between possible reductions in severe neutropenia and improved tumor response and/or survival in patients with various UGT1A1 genotypes.

Outcomes of interest in evidence-based evaluations of genetic tests.

Genetic tests are increasingly available for use in traditional clinical practice settings and through direct-to-consumer marketing. The need for evidence-based information and guidance on their appropriate use has never been more apparent. The independent Working Group of the Evaluation of Genomic Applications in Practice and Prevention Initiative commissions evidence-based reviews and develops recommendations to inform decision making surrounding the implementation of genetic tests and other applications of genomic technologies into clinical practice. A critical component of this analysis involves the identification and appropriate weighting of relevant health outcomes from genetic testing. Impacts of testing on morbidity and mortality are central considerations although research to document such outcomes can be challenging to conduct. In considering the broader impacts of genetic tests on the individual, familial and societal levels, psychosocial outcomes often take on increasing importance, and their systematic evaluation is a challenge for traditional methods of evidence-based review. Incorporating these types of outcomes in evidence-based processes is possible, however, and necessary to extract balanced and complete (or as complete as available data will allow) information on potential benefits and on potential harms. The framework used by the Evaluation of Genomic Applications in Practice and Prevention Working Group in considering, categorizing, and weighting health-related outcomes as applied to genomic technologies is presented here.

Beyond Base Pairs to Bedside: A Population Perspective on How Genomics Can Improve Health

A decade after the sequencing of the human genome, the National Human Genome Research Institute announced a strategic plan for genomic medicine. It calls for evaluating the structure and biology of genomes, understanding the biology of disease, advancing the science of medicine, and improving the effectiveness of health care. Fulfilling the promise of genomics urgently requires a population perspective to complement the bench-to-bedside model of translation. A population approach should assess the contribution of genomics to health in the context of social and environmental determinants of disease; evaluate genomic applications that may improve health care; design strategies for integrating genomics into practice; address ethical, legal, and social issues; and measure the population health impact of new technologies.

Horizon scanning for new genomic tests

Purpose: The development of health-related genomic tests is decentralized and dynamic, involving government, academic, and commercial entities. Consequently, it is not easy to determine which tests are in development, currently available, or discontinued. We developed and assessed the usefulness of a systematic approach to identifying new genomic tests on the Internet.Methods: We devised targeted queries of Web pages, newspaper articles, and blogs (Google Alerts) to identify new genomic tests. We finalized search and review procedures during a pilot phase that ended in March 2010. Queries continue to run daily and are compiled weekly; selected data are indexed in an online database, the Genomic Applications in Practice and Prevention Finder.Results: After the pilot phase, our scan detected approximately two to three new genomic tests per week. Nearly two thirds of all tests (122/188, 65%) were related to cancer; only 6% were related to hereditary disorders. Although 88 (47%) of the tests, including 2 marketed directly to consumers, were commercially available, only 12 (6%) claimed United States Food and Drug Administration licensure.Conclusion: Systematic surveillance of the Internet provides information about genomic tests that can be used in combination with other resources to evaluate genomic tests. The Genomic Applications in Practice and Prevention Finder makes this information accessible to a wide group of stakeholders.

Use of genomic profiling to assess risk for cardiovascular disease and identify individualized prevention strategies—A targeted evidence-based review

Purpose: To address the key question of whether using available “cardiogenomic profiles” leads to improved health outcomes (e.g., reduction in rates of myocardial infarction and stroke) and whether these profiles help in making medical or personal decisions.Methods: A targeted evidence-based review based on published Evaluation of Genomic Applications in Practice and Prevention methodologies.Results: No study addressed the overarching question directly. Evidence for the analytic validity of genomic profiles was inadequate for most genes (scale: convincing, adequate, and inadequate), but based on gray data, the analytic sensitivity and specificity might be adequate. For the 29 candidate genes (58 separate associations reviewed), the credibility of evidence for clinical validity was weak (34 associations) to moderate (23 associations), based on limited evidence, potential biases, and/or variability between included studies. The association of 9p21 variants with heart disease had strong credibility with odds ratios of 0.80 (95% confidence interval: 0.77–0.82) and 1.25 (95% confidence interval: 1.21–1.30), respectively, for individuals with no, or two, at-risk alleles versus those with one at-risk allele. Using a multiplicative model, we combined information from 24 markers predicting heart disease and from 13 markers for stroke. The areas under the curves (64.7% and 55.2%, respectively), and overall screening performance (detection rates of 24% and 14% at a 10% false-positive rate, respectively) do not warrant use as stand-alone tests.Conclusion: Even if genomic markers were independent of traditional risk factors, reports indicate that cardiovascular disease risk reclassification would be small. Improvement in health could occur with earlier initiation or higher adherence to medical or behavioral interventions, but no prospective studies documented such improvements (clinical utility).

Horizon scanning for translational genomic research beyond bench to bedside

Purpose:The dizzying pace of genomic discoveries is leading to an increasing number of clinical applications. In this report, we provide a method for horizon scanning and 1 year data on translational research beyond bench to bedside to assess the validity, utility, implementation, and outcomes of such applications.Methods:We compiled cross-sectional results of ongoing horizon scanning of translational genomic research, conducted between 16 May 2012 and 15 May 2013, based on a weekly, systematic query of PubMed. A set of 505 beyond bench to bedside articles were collected and classified, including 312 original research articles; 123 systematic and other reviews; 38 clinical guidelines, policies, and recommendations; and 32 articles describing tools, decision support, and educational materials.Results:Most articles (62%) addressed a specific genomic test or other health application; almost half of these (n = 180) were related to cancer. We estimate that these publications account for 0.5% of reported human genomics and genetics research during the same time.Conclusion:These data provide baseline information to track the evolving knowledge base and gaps in genomic medicine. Continuous horizon scanning of the translational genomics literature is crucial for an evidence-based translation of genomics discoveries into improved health care and disease prevention.Genet Med 16 7, 535–538.

Knowledge integration at the center of genomic medicine

Three articles in this issue of Genetics in Medicine describe examples of “knowledge integration,” involving methods for generating and synthesizing rapidly emerging information on health-related genomic technologies and engaging stakeholders around the evidence. Knowledge integration, the central process in translating genomic research, involves three closely related, iterative components: knowledge management, knowledge synthesis, and knowledge translation. Knowledge management is the ongoing process of obtaining, organizing, and displaying evolving evidence. For example, horizon scanning and “infoveillance” use emerging technologies to scan databases, registries, publications, and cyberspace for information on genomic applications. Knowledge synthesis is the process of conducting systematic reviews using a priori rules of evidence. For example, methods including meta-analysis, decision analysis, and modeling can be used to combine information from basic, clinical, and population research. Knowledge translation refers to stakeholder engagement and brokering to influence policy, guidelines and recommendations, as well as the research agenda to close knowledge gaps. The ultrarapid production of information requires adequate public and private resources for knowledge integration to support the evidence-based development of genomic medicine.Genet Med advance online publication 3 May 2012.

Evidence synthesis and guideline development in genomic medicine: current status and future prospects

Purpose:With the accelerated implementation of genomic medicine, health-care providers will depend heavily on professional guidelines and recommendations. Because genomics affects many diseases across the life span, no single professional group covers the entirety of this rapidly developing field.Methods:To pursue a discussion of the minimal elements needed to develop evidence-based guidelines in genomics, the Centers for Disease Control and Prevention and the National Cancer Institute jointly held a workshop to engage representatives from 35 organizations with interest in genomics (13 of which make recommendations). The workshop explored methods used in evidence synthesis and guideline development and initiated a dialogue to compare these methods and to assess whether they are consistent with the Institute of Medicine report “Clinical Practice Guidelines We Can Trust.” Results:The participating organizations that develop guidelines or recommendations all had policies to manage guideline development and group membership, and processes to address conflicts of interests. However, there was wide variation in the reliance on external reviews, regular updating of recommendations, and use of systematic reviews to assess the strength of scientific evidence.Conclusion:Ongoing efforts are required to establish criteria for guideline development in genomic medicine as proposed by the Institute of Medicine.Genet Med advance online publication 19 June 2014

Clinical utility of gene-expression profiling in women with early breast cancer: an overview of systematic reviews

Purpose:This overview systematically evaluates the clinical utility of using Oncotype DX and MammaPrint gene-expression profiling tests to direct treatment decisions in women with breast cancer. The findings are intended to inform an updated recommendation from the Evaluation of Genomic Applications in Practice and Prevention Working Group.Methods:Evidence reported in systematic reviews evaluating the clinical utility of Oncotype DX and MammaPrint, as well as the ability to predict treatment outcomes, change in treatment decisions, and cost-effectiveness, was qualitatively synthesized.Results:Five systematic reviews found no direct evidence of clinical utility for either test. Indirect evidence showed Oncotype DX was able to predict treatment effects of adjuvant chemotherapy, whereas no evidence of predictive value was found for MammaPrint. Both tests influenced a change in treatment recommendations in 21 to 74% of participants. The cost-effectiveness of Oncotype DX varied with the alternative compared. For MammaPrint, lack of evidence of the predictive value led to uncertainty in the cost-effectiveness.Conclusion:No studies were identified that provided direct evidence that using gene-expression profiling tests to direct treatment decisions improved outcomes in women with breast cancer. Three ongoing studies may provide direct evidence for determining the clinical utility of gene-expression profiling testing.Genet Med 17 7, 519–532.