Patricia A. Deverka

Evidence of Clinical Utility: An Unmet Need in Molecular Diagnostics for Patients with Cancer

This article defines and describes best practices for the academic and business community to generate evidence of clinical utility for cancer molecular diagnostic assays. Beyond analytical and clinical validation, successful demonstration of clinical utility involves developing sufficient evidence to demonstrate that a diagnostic test results in an improvement in patient outcomes. This discussion is complementary to theoretical frameworks described in previously published guidance and literature reports by the U.S. Food and Drug Administration, Centers for Disease Control and Prevention, Institute of Medicine, and Center for Medical Technology Policy, among others. These reports are comprehensive and specifically clarify appropriate clinical use, adoption, and payer reimbursement for assay manufacturers, as well as Clinical Laboratory Improvement Amendments–certified laboratories, including those that develop assays (laboratory developed tests). Practical criteria and steps for establishing clinical utility are crucial to subsequent decisions for reimbursement without which high-performing molecular diagnostics will have limited availability to patients with cancer and fail to translate scientific advances into high-quality and cost-effective cancer care. See all articles in this CCR Focus section, “The Precision Medicine Conundrum: Approaches to Companion Diagnostic Co-development.” Clin Cancer Res; 20(6); 1428–44. ©2014 AACR.

Integrating comparative effectiveness design elements and endpoints into a phase III, randomized clinical trial (SWOG S1007) evaluating oncotypeDX-guided management for women with breast cancer involving lymph nodes ☆

Women with breast cancer involving the lymph nodes are typically treated with cytotoxic chemotherapy. Retrospective evaluations of prior studies suggest that the 21-gene test (OncotypeDX®), may allow identification of those who can safely avoid chemotherapy. To better understand the performance of the 21-gene test, the RxPONDER (Rx for Positive Node, Endocrine Responsive breast cancer) study was designed, a multicenter Phase III trial randomizing women with hormone receptor-positive and HER2-negative breast cancer involving 1-3 lymph nodes and a 21-gene assay recurrence score (RS) of 25 or less to endocrine therapy alone versus chemotherapy followed by endocrine therapy. As one of the first large-scale comparative-effectiveness studies in oncology, RxPONDER utilized an external stakeholder group to help inform the design of the trial. Stakeholders met with representatives of SWOG over several months through a structured discussion process. The stakeholder engagement process resulted in several changes being made to the trial design. In addition, stakeholder representatives from the health insurance industry provided guidance regarding a mechanism whereby the costs of OncotypeDX® would be paid by the majority of health insurers as part of the trial. The process may serve as a template for future studies evaluating the comparative effectiveness of genomic tests in oncology, particularly those that are conducted within cooperative clinical trials groups.

Stakeholder assessment of the evidence for cancer genomic tests: insights from three case studies.

Purpose:Insufficient evidence on the net benefits and harms of genomic tests in real-world settings is a translational barrier for genomic medicine. Understanding stakeholders’ assessment of the current evidence base for clinical practice and coverage decisions should be a critical step in influencing research, policy, and practice.Methods:Twenty-two stakeholders participated in a workshop exploring the evidence of genomic tests for clinical and coverage decision making. Stakeholders completed a survey prior to and during the meeting. They also discussed if they would recommend for or against current clinical use of each test.Results:At baseline, the level of confidence in the clinical validity and clinical utility of each test varied, although the group expressed greater confidence for epidermal growth factor receptor mutation and Lynch syndrome testing than for Oncotype DX. Following the discussion, survey results reflected even less confidence for Oncotype DX and epidermal growth factor receptor mutation testing, but not for Lynch syndrome testing. The majority of stakeholders would consider clinical use for all three tests, but under the conditions of additional research or a shared clinical decision–making approach.Conclusion:Stakeholder engagement in unbiased settings is necessary to understand various perspectives about evidentiary thresholds in genomic medicine. Participants recommended the use of various methods for evidence generation and synthesis.Genet Med advance online publication 5 April 2012.

Stakeholder engagement in comparative effectiveness research: how will we measure success?

Stakeholder engagement in comparative effectiveness research continues to gain national attention. While various methods are used to gather stakeholder expertise and form recommendations, evaluation of the stakeholder experience is often missing. The lack of evaluation prohibits assessing how effective and meaningful engagement practices are for enhancing research efforts and limits the ability to identify areas for future improvement. We propose that an evaluation plan of engagement processes be developed before stakeholder involvement begins and be required as part of a request for proposal or research grant where stakeholder input is being sought. Furthermore, we recommend the inclusion of six meta-criteria that represent normative goals of multiple studies: respect, trust, legitimacy, fairness, competence and accountability. To aid in the development of future evaluations, we have developed definitions for and matched specific examples of measuring each meta-criterion to serve a guide for others in the field.

Making genomic medicine evidence-based and patient-centered: a structured review and landscape analysis of comparative effectiveness research

Comparative effectiveness research (CER) in genomic medicine (GM) measures the clinical utility of using genomic information to guide clinical care in comparison to appropriate alternatives. We summarized findings of high-quality systematic reviews that compared the analytic and clinical validity and clinical utility of GM tests. We focused on clinical utility findings to summarize CER-derived evidence about GM and identify evidence gaps and future research needs. We abstracted key elements of study design, GM interventions, results, and study quality ratings from 21 systematic reviews published in 2010 through 2015. More than half (N = 13) of the reviews were of cancer-related tests. All reviews identified potentially important clinical applications of the GM interventions, but most had significant methodological weaknesses that largely precluded any conclusions about clinical utility. Twelve reviews discussed the importance of patient-centered outcomes, although few described evidence about the impact of genomic medicine on these outcomes. In summary, we found a very limited body of evidence about the effect of using genomic tests on health outcomes and many evidence gaps for CER to address.Genet Med advance online publication 13 April 2017

Prioritization in Comparative Effectiveness Research: The CANCERGEN Experience in Cancer Genomics

Background:Systematic approaches to stakeholder-informed research prioritization are a central focus of comparative effectiveness research. Genomic testing in cancer is an ideal area to refine such approaches given rapid innovation and potentially significant impacts on patient outcomes. Objective:To develop and pilot test a stakeholder-informed approach to prioritizing genomic tests for future study in collaboration with the cancer clinical trials consortium SWOG. Methods:We conducted a landscape analysis to identify genomic tests in oncology using a systematic search of published and unpublished studies, and expert consultation. Clinically valid tests suitable for evaluation in a comparative study were presented to an external stakeholder group. Domains to guide the prioritization process were identified with stakeholder input, and stakeholders ranked tests using multiple voting rounds. Results:A stakeholder group was created including representatives from patient-advocacy groups, payers, test developers, regulators, policy makers, and community-based oncologists. We identified 9 domains for research prioritization with stakeholder feedback: population impact; current standard of care, strength of association; potential clinical benefits, potential clinical harms, economic impacts, evidence of need, trial feasibility, and market factors. The landscape analysis identified 635 studies; of 9 tests deemed to have sufficient clinical validity, 6 were presented to stakeholders. Two tests in lung cancer (ERCC1 and EGFR) and 1 test in breast cancer (CEA/CA15-3/CA27.29) were identified as top research priorities. Conclusions:Use of a diverse stakeholder group to inform research prioritization is feasible in a pragmatic and timely manner. Additional research is needed to optimize search strategies, stakeholder group composition, and integration with existing prioritization mechanisms.

Payer decision making for next-generation sequencing-based genetic tests: insights from cell-free DNA prenatal screening

Purpose:Cell-free DNA (cfDNA) prenatal screening tests have been rapidly adopted into clinical practice, due in part to positive insurance coverage. We evaluated the framework payers used in making coverage decisions to describe a process that should be informative for other sequencing tests.Methods:We analyzed coverage policies from the 19 largest US private payers with publicly available policies through February 2016, building from the University of California San Francisco TRANSPERS Payer Coverage Policy Registry.Results:All payers studied cover cfDNA screening for detection of trisomies 21, 18, and 13 in high-risk, singleton pregnancies, based on robust clinical validity (CV) studies and modeled evidence of clinical utility (CU). Payers typically evaluated the evidence for each chromosomal abnormality separately, although results are offered as part of a panel. Starting in August 2015, 8 of the 19 payers also began covering cfDNA screening in average-risk pregnancies, citing recent CV studies and updated professional guidelines. Most payers attempted, but were unable, to independently assess analytic validity (AV).Conclusion:Payers utilized the standard evidentiary framework (AV/CV/CU) when evaluating cfDNA screening but varied in their interpretation of the sufficiency of the evidence. Professional guidelines, large CV studies, and decision analytic models regarding health outcomes appeared highly influential in coverage decisions.Genet Med advance online publication 22 September 2016

Generating and evaluating evidence of the clinical utility of molecular diagnostic tests in oncology

Purpose:Enthusiasm for molecular diagnostic (MDx) testing in oncology is constrained by the gaps in required evidence regarding its impact on patient outcomes (clinical utility (CU)). This effectiveness guidance document proposes recommendations for the design and evaluation of studies intended to reflect the evidence expectations of payers, while also reflecting information needs of patients and clinicians.Methods:Our process included literature reviews and key informant interviews followed by iterative virtual and in-person consultation with an expert technical working group and an advisory group comprising life-sciences industry experts, public and private payers, patients, clinicians, regulators, researchers, and other stakeholders.Results:Treatment decisions in oncology represent high-risk clinical decision making, and therefore the recommendations give preference to randomized controlled trials (RCTs) for demonstrating CU. The guidance also describes circumstances under which alternatives to RCTs could be considered, specifying conditions under which test developers could use prospective-retrospective studies with banked biospecimens, single-arm studies, prospective observational studies, or decision-analytic modeling techniques that make a reasonable case for CU.Conclusion:Using a process driven by multiple stakeholders, we developed a common framework for designing and evaluating studies of the clinical validity and CU of MDx tests, achieving a balance between internal validity of the studies and the relevance, feasibility, and timeliness of generating the desired evidence.Genet Med 18 8, 780–787.

Prioritization in comparative effectiveness research: the CANCERGEN Experience.

Background:Systematic approaches to stakeholder-informed research prioritization are a central focus of comparative effectiveness research. Genomic testing in cancer is an ideal area to refine such approaches given rapid innovation and potentially significant impacts on patient outcomes. Objective:To develop and pilot test a stakeholder-informed approach to prioritizing genomic tests for future study in collaboration with the cancer clinical trials consortium SWOG. Methods:We conducted a landscape analysis to identify genomic tests in oncology using a systematic search of published and unpublished studies, and expert consultation. Clinically valid tests suitable for evaluation in a comparative study were presented to an external stakeholder group. Domains to guide the prioritization process were identified with stakeholder input, and stakeholders ranked tests using multiple voting rounds. Results:A stakeholder group was created including representatives from patient-advocacy groups, payers, test developers, regulators, policy makers, and community-based oncologists. We identified 9 domains for research prioritization with stakeholder feedback: population impact; current standard of care, strength of association; potential clinical benefits, potential clinical harms, economic impacts, evidence of need, trial feasibility, and market factors. The landscape analysis identified 635 studies; of 9 tests deemed to have sufficient clinical validity, 6 were presented to stakeholders. Two tests in lung cancer (ERCC1 and EGFR) and 1 test in breast cancer (CEA/CA15-3/CA27.29) were identified as top research priorities. Conclusions:Use of a diverse stakeholder group to inform research prioritization is feasible in a pragmatic and timely manner. Additional research is needed to optimize search strategies, stakeholder group composition, and integration with existing prioritization mechanisms.

Payer coverage policies for multigene tests

VOLUME 35 NUMBER 7 JULY 2017 NATURE BIOTECHNOLOGY conditions; other analyses have examined coverage for panels including BRCA1/2 (ref. 5), coverage of non-invasive prenatal screening tests6, and the evidence cited in coverage decisions7. Our initial version (Version 1.0) of the registry includes publicly available national policies from the five largest US private (commercial) payers, representing 112 million enrollees8, though these policies do not necessarily reflect regional polices pertaining to particular benefit designs of states, unions, or individual private companies. (To our knowledge, it would be unlikely that such policies would single out multigene tests for separate coverage.) Policies current as of June 2015 were systematically coded by two authors (M.P.D. and P.A.D., with review by a third author K.A.P.) (Supplementary Methods). As noted previously9, policies are contextspecific and thus we classified tests into the following categories: tumor profiling; inherited disease testing for neurologic, cancer, cardiovascular, or biochemical disorders; drug metabolism testing; whole exome or whole genome sequencing; and prenatal testing or carrier testing (Table 2). Policies discuss multigene tests using either a general description for the type of panel (e.g., “cancer susceptibility testing”) or a specific brand name (e.g., “BreastNext”). Tests may be included (‘mentioned’) or not included in a policy. When included, tests can be determined to be covered or not covered. Thus, some tests may have unknown coverage because they are not included or mentioned in a policy. Tests were placed into one of three mutually exclusive categories. 1. Covered when mentioned. Test is covered for at least one indication in that policy and not specifically excluded in any other policy. Multigene tests within the same policy that were covered for some clinical indications but not others were coded as “covered.” 2. Not covered when mentioned. Not covered in any policies that specifically mention the test. Payer coverage policies for multigene tests