Robert M. Califf

Need for a National Evaluation System for Health Technology

Federal regulatory frameworks governing medical products are designed to (1) provide evidence that a product benefits patients when used as intended and should be available despite accompanying risks and (2) ensure timely access to needed therapies and diagnostics. Historically, policy makers and product developers have viewed these objectives as being in tension. However, ensuring safety, expediting patient access, and enabling innovation can be complementary goals within a regulatory framework for medical devices. The US standard for marketing a medical device is “reasonable assurance of safety and effectiveness” (RASE).1 Generally, clinical studies must be conducted to demonstrate RASE for both high-risk and innovative lower-risk devices and US patients and clinicians have greater assurances that the benefits of devices outweigh the potential risks. In contrast, other countries apply a standard of safety and performance with limited clinical data. The greater evidentiary burden of RASE may create disincentives for manufacturers to bring important medical devices to the United States or may delay access to devices. For example, the first transcatheter aortic valve replacement device was available for clinical use in Europe several years before it was available in the United States. However, there are examples of unsafe and ineffective devices that never made it to the US market; these can be found in a report2 from the US Food and Drug Administration (FDA). A key dilemma for device regulation is how to ensure timely access while also providing evidence to guide safe and appropriate use. When a device is approved for the US market, residual uncertainty about benefit and risk is typically addressed through postmarket evaluation. Premarket studies often do not fully reflect how a device will be used in practice, and participants enrolled in such studiesmaynotrepresenttheentirespectrumofpatients likely to receive the device. The effects of operator experience, user learning curves, or skill level of the individual who implants the device and the supporting team also cannot be assessed until the device is in wider use. However, current approaches to postmarket evaluation have limitations. Even though the FDA can require device makers to perform postmarket studies, patients have few incentives to enroll in a study once a device is marketed, and many FDA-mandated postmarket studies for devices have been delayed, scaled back, or never finished. Generally, if the company makes a good-faith effort in performing postmarket studies, there are no penalties. Furthermore, reporting of adverse events and device malfunctions currently depends on clinicians identifying and reporting a possible association; therefore, underreporting is likely common. Spontaneous reporting also fails to capture numerators and denominators that allow reliable risk estimation. Safety issues are therefore often not identified until many patients have been exposed to risks, leading to greater potential for avoidable harm as well as greater liability and loss of consumer confidence in the manufacturer. Spontaneous reporting is not systematic and can be biased by extraneous factors such as news reports. Other safety issues also depend on companies appropriately assimilating and reporting data. However, a strategic approach to linking and using clinically based data sources, such as registries, electronic health records (EHRs), and claims data, could potentially reduce the burdens of obtaining appropriate evidence across the life cycle of a device. By leveraging clinical data and applying advanced analytics and flexible regulatory approaches tailored to the unique data needs and innovation cycles of specific device types, a more comprehensive and accurate framework could be created for assessing the risks and benefits of devices.

Biomarkers and Surrogate Endpoints: Developing Common Terminology and Definitions

Biomarkers, surrogate endpoints, and clinical outcome assessments provide an essential set of tools needed to translate scientific concepts into diagnostic and therapeutic approaches and technologies. Recently, biomarkers have been promoted as offering significant potential for accelerating basic science, drug discovery, and medical product development, as well as improving clinical care.1 Examples of common biomarkers include breast cancer genes 1 and 2, prostate-specific antigen, and hepatitis C virus ribonucleic acid. Although new analytical approaches and omics-based technologies have yielded a rapidly expanding array of biomarkers, precisely how and when they should be used has not been clearly described.

Engaging Patients Across the Spectrum of Medical Product Development: View From the US Food and Drug Administration.

The complex tasks of developing, evaluating, and determining the appropriate use of medical technologies occur in an evolving ecosystem of diverse stakeholders. However, as new medical therapies and diagnostics are designed and tested, the preferences and views of the patients and care partners who are most directly affected by these treatments are all too often overlooked. Individual patients often experience different effects of diseases and may have unique preferences about treatments or diagnostic procedures that differ from those of other patients or of their physicians or other health care practitioners; they may also have differing views about what kinds and degrees of risk are tolerable. As patients weigh the balance of benefits and risks, their decisions are informed by their experiences, backgrounds, and personal circumstances. In addition, patients are no longer passive recipients of care; instead, they are empowered consumers of medical products and partners in the process by which those products are developed. Patients increasingly act as advocates for new treatments, and many are fully engaged in making decisions about their care. Programs recently enacted at the US Food and Drug Administration (FDA) are focused on including patient perspectives throughout the continuum of medical product development.1 In this Viewpoint, we describe ongoing efforts at several FDA centers and offer views on a conceptual framework within the context of the Precision Medicine Initiative that could lead to improved health by more effectively matching medical products to the needs and preferences of patients and care partners.

Benefit-Risk Assessments at the US Food and Drug Administration: Finding the Balance

The US Food and Drug Administration’s (FDA’s) independent evaluation of medical products for safety and effectiveness prior to granting approval for marketing or new labeled indications is fundamental to assuring the public and clinicians that marketed products have a positive balance of benefit to risk when used according to labeling. Ever since this foundational protection was established in 1962, a body of evidence has supported the use of a flexible but consistent standard when making decisions about marketing approval: Do the benefits of a product outweigh the risks when used as intended and labeled?

Pragmatic clinical trials: Emerging challenges and new roles for statisticians

Patients, clinicians, and policymakers alike need access to high-quality scientific evidence in order to make informed choices about health and healthcare, but the current national clinical trials enterprise is not yet optimally configured for the efficient creation and dissemination of such evidence. However, new technologies and methods hold significant potential for accelerating the rate at which we are able to translate raw findings gathered from both patient care and clinical research into actionable knowledge. We are now entering a period in which the quantitative sciences are emerging as the critical disciplines for advancing knowledge about health and healthcare, and statisticians will increasingly serve as critical mediators in transforming data into evidence. In this new, data-centric era, biostatisticians not only need to be expert at analyzing data but should also be involved directly in diverse efforts, including the review and analysis of research portfolios in order to optimize the relevance of research questions, the use of “quality by design” principles to improve reliability and validity of each individual trial, and the mining of aggregate knowledge derived from the clinical research enterprise as a whole. In order to meet these challenges, it is imperative that we (1) nurture and build the biostatistical workforce, (2) develop a deeper understanding of the biological and clinical context among statisticians, (3) facilitate collaboration among biostatisticians and other members of the clinical trials enterprise, (4) focus on communication skills in training and education programs, and (5) enhance the quantitative capacity of the research and clinical practice worlds.

The US Food and Drug Administration and the Future of Cardiovascular Medicine

Cardiovascular medicine has led the drive for creativity and innovation with a culture that has been at the forefront of evidence generation. However, we are functioning at only a fraction of our evidence generation capacity. Despite the leadership of cardiovascular medicine, very few guideline recommendations are supported by high levels of evidence, and the proportion of recommendations for which there is no conclusive evidence is substantial. Clinical research has proven to be too slow, unreliable, and expensive as conducted in the past. In the current era, a new model of unlimited information, better access to care, and better payer coverage has the potential to change our evidence base to support clinical guidelines. We now have the opportunity to use volumes of data to support US Food and Drug Administration labeling and practice guidelines. The electronic health record can be used to feed decisions and provide an information feedback loop. In addition, learning health systems can contribute to providing networks and registries for information sharing. In this Special Communication, we summarize opportunities of the cardiovascular community to build on its pioneering leadership in evidence-based medicine through major initiatives now under way. By joining in broad efforts to create an efficient national evidence generation system, larger proportions of clinical practice can be guided by high-quality evidence; clinicians and their practice organizations will be increasingly able to focus on interpreting, applying, and communicating research findings to improve outcomes; and patients and consumers will be increasingly informed and empowered to play active roles in managing their own health and health care.

Overcoming the Challenges of Conducting Early Feasibility Studies of Medical Devices in the United States

Initial clinical studies of new medical technologies involve a complex balance of research participant benefits versus risks and costs of uncertainty when novel concepts are tested. The Food and Drug Administration Center for Devices and Radiological Health has recently introduced the Early Feasibility Study (EFS) Program for facilitating the conduct of these studies under the Investigational Device Exemption regulations. However, a systematic approach is needed to successfully implement this program while affording appropriate preservation of the rights and interests of patients. For this to succeed, a holistic reform of the clinical studies ecosystem for performing early-stage clinical research in the United States is necessary. The authors review the current landscape of the U.S. EFS and make recommendations for developing an efficient EFS process to meet the goal of improving access to early-stage, potentially beneficial medical devices in the United States.

The Future of Cardiovascular Medicine From the Regulatory Perspective.

![Figure][1] For many years, the American public and the entire world have benefited from the U.S. Food and Drug Administration’s (FDA) regulatory system for medical products. The field of cardiovascular medicine has been an exemplary partner with the FDA for much of this time—a period

The FDA in the 21st Century: How Is the FDA Responding to the New World? A Focus on Heart Failure Devices.

Regulatory decisions to approve or deny marketing applications for new drugs and devices for heart failure (HF) care often garner significant publicity and highlight the impact of the U.S. Food and Drug Administration (FDA) on HF care. However, there are a number of other activities and discussions

Knowing When and How to Use Medical Products: A Shared Responsibility for the FDA and CMS

Before a medical product can be widely used in the United States, it generally must first be approved or cleared for marketing by the US Food and Drug Administration (FDA). Then, payers such as the Centers for Medicare & Medicaid Services (CMS) must decide whether the product merits coverage and payment. Because the statutes governing these agencies evolved to meet the exigencies of particular moments in the history of medical product development, the degree of convergence in standards and in the underlying evidence needed to support regulatory and payment decisions is not always immediately obvious. The resulting fragmentation—perceived or real—has led to questions about whether FDA approval or clearance for marketing will necessarily result in approval for coverage and payment. Despite these challenges, changes in the organization of health care and in the larger information ecosystem shouldallowtheFDAandCMStomoveincreasinglytoward use of shared sources of evidence while still applying the most appropriate criteria to their decision making. Such a move should help reduce current gaps in evidence that create uncertainty surrounding the approval or clearance of new therapies and their subsequent use in practice. It should also enable greater efficiency in medical product development and provide the higher-quality evidence needed in the emerging era of precision medicine. Although different standards are applied for product approval or clearance for marketing (in the case of the FDA) and coverage and payment (in the case of CMS), both agencies base their determinations on scientific evidence. The FDA bases drug approval for marketing on “substantial evidence” derived from at least 1 wellconducted clinical trial,1 whereas CMS uses “reasonable and necessary” as the standard for coverage.2 In the increasing number of situations in which “coverage with evidence development” is used by CMS, coverage decisions are based on “... formal review of the medical literature ... only in the context of an approved clinical study or when additional clinical data are collected to assess the appropriateness of an item or service for use with a particular beneficiary.”3 For much of the modern era of medicine, research intended to support marketing approval or clearance by the FDA has usually examined the effects of therapeutics in narrow, strictly delineated populations enrolled in trials conducted under conditions that may not reflect the clinical practice settings in which the product will actually be used. There are excellent reasons that this system evolved, including the need to control confounding variables and ensure the internal validity of study findings. Given that approximately 90% of candidate drugs that reach the point of human trials fail before attaining marketing approval4 (as do many devices designed to treat serious medical illnesses),demonstratingthatatherapycanworkevenwhen administered under a strict protocol in a carefully defined population represents a major challenge. However, when a product demonstrates promise in the rarefied setting of earlier-phase clinical research, developers should rapidly pivot toward evaluating that product in the clinical contexts for which it is intended. In other words, the United States needs to develop highquality evidence about the risks and benefits of tangible health outcomes in clinical settings and among patients representative of those who will actually use these products. This expansion of scope earlier in the product life cycle will help establish how a given product is likely to perform once it is marketed and how best to administer that product. Early involvement of health systems and payers will also help to understand and address the kinds of evidence needed to incorporate the new product into practice, place it on formulary or device inventory, and decide whether and how much to pay for its use, thereby facilitating implementation and adoption. Notably, the European Medicines Agency has recently come to the same conclusion in concert with its health technology assessment organizations.5 Before the Kefauver-Harris Amendments were enacted by Congress in 1962, drugs were required to undergo evaluation only for safety, not efficacy, and many were sold on the basis of anecdote. However, following passage of the amendments, the FDA sought to ascertain whether drugs met new standards for effectiveness.6 This program was a tremendous success in terms of requiring that drugs demonstrate efficacy in specific, well-defined populations, and similar systems have developed for highrisk devices. But the unintended results of limitations arising from the separate evolution of clinical and research environments and the absence of a means for efficient data exchange between these 2 spheres were trials that often do not adequately inform practice and product labeling that sometimes lacks optimal empirical information to guide use of the product in clinical practice. It is imperative that the US regulatory groups bridge this gap between evidence that is generated through trials performed primarily to obtain marketing approval for drugs and high-risk devices and the optimal kinds of evidence needed to support the intended uses of these products in practice. Therefore, to ensure that patients, clinicians, and payers have adequate evidence to guide their choices, the FDA and CMS are focusing on the following approaches: • First, these agencies are clarifying the need for including diverse populations and measuring relevant clinical outcomes within the sphere of trials conducted for regulatory approval and to inform labeling. This task is particularly important in the case of accelerated approvals VIEWPOINT