Spencer Phillips Hey

Are outcome-adaptive allocation trials ethical?

Randomization is firmly established as a cornerstone of clinical trial methodology. Yet, the ethics of randomization continues to generate controversy. The default, and most efficient, allocation scheme randomizes patients equally (1:1) across all arms of study. However, many randomized trials are using outcome-adaptive allocation schemes, which dynamically adjust the allocation ratio in favor of the better performing treatment arm. Advocates of outcome-adaptive allocation contend that it better accommodates clinical equipoise and promotes informed consent, since such trials limit patient-subject exposure to sub-optimal care. In this essay, we argue that this purported ethical advantage of outcome-adaptive allocation does not stand up to careful scrutiny in the setting of two-armed studies and/or early-phase research.

Benefit, Risk, and Outcomes in Drug Development: A Systematic Review of Sunitinib

BACKGROUND Little is known about the total patient burden associated with clinical development and where burdens fall most heavily during a drug development program. Our goal was to quantify the total patient burden/benefit in developing a new drug. METHODS We measured risk using drug-related adverse events that were grade 3 or higher, benefit by objective response rate, and trial outcomes by whether studies met their primary endpoint with acceptable safety. The differences in risk (death rate) and benefit (overall response rate) between industry and nonindustry trials were analyzed with an inverse-variance weighted fixed effects meta-analysis implemented as a weighted regression analysis. All statistical tests were two-sided. RESULTS We identified 103 primary publications of sunitinib monotherapy, representing 9092 patients and 3991 patient-years of involvement over 10 years and 32 different malignancies. In total, 1052 patients receiving sunitinib monotherapy experienced objective tumor response (15.7% of intent-to-treat population, 95% confidence interval [CI] = 15.3% to 16.0%), 98 died from drug-related toxicities (1.08%, 95% CI = 1.02% to 1.14%), and at least 1245 experienced grade 3-4 drug-related toxicities (13.7%, 95% CI = 13.3% to 14.1%). Risk/benefit worsened as the development program matured, with several instances of replicated negative studies and almost no positive trials after the first responding malignancies were discovered. CONCLUSIONS Even for a successful drug, the risk/benefit balance of trials was similar to phase I cancer trials in general. Sunitinib monotherapy development showed worsening risk/benefit, and the testing of new indications responded slowly to evidence that sunitinib monotherapy would not extend to new malignancies. Research decision-making should draw on evidence from whole research programs rather than a narrow band of studies in the same indication.

The questionable use of unequal allocation in confirmatory trials

Randomization is the standard means for addressing known and unknown confounders within the patient population in clinical trials. Although random assignment to treatment arms on a 1:1 basis has long been the norm, many 2-armed confirmatory trials now use unequal allocation schemes where the number of patients receiving investigational interventions exceeds those in the comparator arm. In what follows, we offer 3 arguments for why investigators, institutional review boards, and data and safety monitoring boards should exercise caution when planning or reviewing 2-armed confirmatory trials involving unequal allocation ratios. We close by laying out some of the conditions where uneven allocation can be justified ethically.

The risk-escalation model: a principled design strategy for early-phase trials.

Should first-in-human trials be designed to maximize the prospect of therapeutic benefit for volunteers, prioritize avoidance of unintended harms, or aim for some happy medium between the two? Perennial controversies surrounding initiation and design of early-phase trials hinge on how this question is resolved. In this paper, we build on the premise that the task of early-phase testing is to optimize various components of a potential therapy so that later, confirmatory trials have the maximal probability of informing drug development and clinical care. We then explore three strategies that investigators might use to manage trial risks while optimizing a therapy, using cell therapy for Amyotrophic Lateral Sclerosis (ALS) as an example. We argue that an iterative application of maximin strategies over successive cohorts and trials, which we call the “risk-escalation model,” establishes a moral principle that should guide decision-making in early-phase trials.

The Question of Clinical Equipoise and Patients' Best Interests.

Clinical equipoise—the idea that the community of medical experts is uncertain about the relative therapeutic merits of the arms of a clinical trial at its outset—mitigates physicians’ responsibility for patients’ poor outcomes when patients are assigned to the control arm or are harmed by an investigational agent.

Countering imprecision in precision medicine

Better coordination is needed to study complex interventions The goal of precision medicine (PM) is to “ensure that the right treatment is delivered to the right patient at the right time” (1). Predictive biomarker diagnostics are critical to this effort. Yet despite substantial promise, PM has been plagued with problems (2). Many commercially available biomarker diagnostics have not been adequately validated (3); the scientific literature is flooded with low-quality and unreliable reports (2); and even ostensibly successful PMs, such as trastuzumab (Herceptin) chemotherapy in HER2-expressing breast cancer, have been characterized by uncertainty about how to use and interpret diagnostic test results (4). These disappointing features of PM research can be explained by three obstacles inherent in the science: (i) Biological theories play a central role in the testing methodology; therefore, pivotal trials of PMs cannot be agnostic about underlying mechanisms. (ii) Interventions are complex with many components and degrees of uncertainty that need to be resolved before clinical use. (iii) No single stakeholder controls the biomarkers or coordinates the research program. Although some new regulatory (5) and evidence synthesis (6) efforts are designed to address these problems, we believe that meaningful progress in PM requires new mechanisms of scientific oversight.

Assay Sensitivity and the Epistemic Contexts of Clinical Trials

This article examines the concept of assay sensitivity in clinical research. Defined as the ability of a clinical trial to distinguish between an effective and ineffective treatment, the need for assay sensitivity has been taken to support the claim that placebos are methodologically superior to active control treatments. The demands of doing good clinical science must trump the physician-researcher’s ethical duty to provide all trial participants with nothing less than competent medical care. We argue that this supposed implication of assay sensitivity rests on (1) collapsing the distinction between biological efficacy and clinical effectiveness, and (2) conflating the epistemic contexts of a trial-as-designed and a trial-as-executed. Once these errors are corrected, it becomes clear that placebos grant no epistemological advantage over active controls, and there is therefore no longer a tension between the epistemic and ethical demands of research. We suggest that the legitimate worries behind assay sensitivity can be better understood as the need for researchers to articulate their experimental heuristics and to demonstrate a robust pattern of evidence across a series of trials.

Accumulating Evidence and Research Organization (AERO) model: a new tool for representing, analyzing, and planning a translational research program

BackgroundMaximizing efficiency in drug development is important for drug developers, policymakers, and human subjects. Limited funds and the ethical imperative of risk minimization demand that researchers maximize the knowledge gained per patient-subject enrolled. Yet, despite a common perception that the current system of drug development is beset by inefficiencies, there remain few approaches for systematically representing, analyzing, and communicating the efficiency and coordination of the research enterprise. In this paper, we present the first steps toward developing such an approach: a graph-theoretic tool for representing the Accumulating Evidence and Research Organization (AERO) across a translational trajectory.MethodsThis initial version of the AERO model focuses on elucidating two dimensions of robustness: (1) the consistency of results among studies with an identical or similar outcome metric; and (2) the concordance of results among studies with qualitatively different outcome metrics. The visual structure of the model is a directed acyclic graph, designed to capture these two dimensions of robustness and their relationship to three basic questions that underlie the planning of a translational research program: What is the accumulating state of total evidence? What has been the translational trajectory? What studies should be done next?ResultsWe demonstrate the utility of the AERO model with an application to a case study involving the antibacterial agent, moxifloxacin, for the treatment of drug-susceptible tuberculosis. We then consider some possible elaborations for the AERO model and propose a number of ways in which the tool could be used to enhance the planning, reporting, and analysis of clinical trials.ConclusionThe AERO model provides an immediate visual representation of the number of studies done at any stage of research, depicting both the robustness of evidence and the relationship of each study to the larger translational trajectory. In so doing, it makes some of the invisible or inchoate properties of the research system explicit – helping to elucidate judgments about the accumulating state of evidence and supporting decision-making for future research.

Heuristics and Meta-heuristics in Scientific Judgement

Despite the increasing recognition that heuristics may be involved in myriad scientific activities, much about how to use them prudently remains obscure. As typically defined, heuristics are efficient rules or procedures for converting complex problems into simpler ones. But this increased efficiency and problem-solving power comes at the cost of a systematic bias. As Wimsatt ( [1980] , [2007] ) showed, biased modelling heuristics can conceal errors, leading to poor decisions or inaccurate models. This liability to produce errors presents a fundamental challenge to the philosophical value of heuristic analyses. Heuristics may be powerful and efficient procedures, but their practical value for science and epistemology is significantly mitigated if we do not have a principled methodology for knowing how to use them wisely. In this essay, I extend Wimsatt’s analyses to argue that this challenge for a heuristic methodology can be met by appealing to second-order, or meta-, heuristics—that is, practical guidelines that prescribe the appropriate conditions for a first-order heuristic’s use. 1   Introduction 2   Background 3   Heuristics and Meta-heuristics 4   A Hierarchical Model and Three Applications    4.1   Decision making in emergency medicine    4.2   Mathematical explanation in physics    4.3   Resilience in ecological management 5   Conclusion 1   Introduction 2   Background 3   Heuristics and Meta-heuristics 4   A Hierarchical Model and Three Applications    4.1   Decision making in emergency medicine    4.2   Mathematical explanation in physics    4.3   Resilience in ecological management    4.1   Decision making in emergency medicine    4.2   Mathematical explanation in physics    4.3   Resilience in ecological management 5   Conclusion

Robust and Discordant Evidence: Methodological Lessons from Clinical Research

The concordance of results that are “robust” across multiple scientific modalities is widely considered to play a critical role in the epistemology of science. But what should we make of those cases where such multimodal evidence is discordant? Jacob Stegenga has recently argued that robustness is “worse than useless” in these cases, suggesting that “different kinds of evidence cannot be combined in a coherent way.” In this article I respond to this critique and illustrate the critical methodological role that robustness plays as an aim of scientific inquiry.