Thomas Permutt

Sensitivity analysis for missing data in regulatory submissions.

The National Research Council Panel on Handling Missing Data in Clinical Trials recommended that sensitivity analyses have to be part of the primary reporting of findings from clinical trials. Their specific recommendations, however, seem not to have been taken up rapidly by sponsors of regulatory submissions. The NRC reports detailed suggestions are along rather different lines than what has been called sensitivity analysis in the regulatory setting up to now. Furthermore, the role of sensitivity analysis in regulatory decision-making, although discussed briefly in the NRC report, remains unclear. This paper will examine previous ideas of sensitivity analysis with a view to explaining how the NRC panels recommendations are different and possibly better suited to coping with present problems of missing data in the regulatory setting. It will also discuss, in more detail than the NRC report, the relevance of sensitivity analysis to decision-making, both for applicants and for regulators. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

Editorial: Dealing With the Missing Data Challenge in Clinical Trials

Correspondence Address Thomas Permuff. 10903 New Hampshire Ave.. Silver Spring, MD 20993 (email: Missing data remains a pervasive and important problem in clinical studies. How to properly include information from a patient who did not Thomas. Permutt @ fda.hhs.gov), complete the study will often depend o n unverifiable assumptions about the underlying missing data mechanism. Different assumptions and associated analysis methods may lead to conflicting conclusions about the statistical significance and magnitude of treatment benefit. Understandably, there is considerable interest from key stakeholders in drug development (including regulatory agencies, pharmaceutical companies, and academic institutions) to reach a n agreement on acceptable and recommended approaches for dealing with missing data. In recognition of the relevance of the topic, one of the two tracks in the Second FDA-DIA Statistics Forum (held in Bethesda, Maryland, on April 14-16,2008) was dedicated to missing data, being cochaired by the two of us. Renowned experts on missing data methods presented at the conference and led lively discussions with the participants o n the pros and cons of different approaches. This special issue of the Drug Information Journal intends to document the most relevant topics presented and discussed a t the meeting, serving as a reference for future research and debate in the field. We start, in this editorial, with ou r thoughts on framing and addressing the problem of missing data in clinical trials. T H E P R O B L E M O R P R O B L E M S Consider t he following five possible outcomes for a patient in a clinical trial.

A Note on Stratification in Clinical Trials

Stratification is sometimes proposed to deal with problems of influential covariates in clinical trials. The word stratification, however, may refer to any of four different methods of design and analysis. The methods are capable of addressing three different problems. Which problem and which method are being discussed is often misunderstood. Consequently, the method adopted may not solve the problem that provoked its consideration.

Effects in Adherent Subjects

ABSTRACT Dropouts confound the treatment effect when the outcome and the dropout process both depend on subject characteristics. If dropout is unrelated to treatment, there is an unconfounded effect, but it is the effect in a principal stratum, rather than a de jure effect. There are at least two different definitions of the effect if all subjects adhered, giving effects different numerically from each other and from the effect in the adherent principal stratum. Estimation of either of these two effects requires an assumption (MAR) different from but sometimes confused with the assumption that dropout is unrelated to treatment.

Considerations for Developing Targeted Therapies in Low‐Frequency Molecular Subsets of a Disease

Advances in our understanding of the molecular underpinnings of disease have spurred the development of targeted therapies and the use of precision medicine approaches in patient care. While targeted therapies have improved our capability to provide effective treatments to patients, they also present additional challenges to drug development and benefit–risk assessment such as identifying the subset(s) of patients likely to respond to the drug, assessing heterogeneity in response across molecular subsets of a disease, and developing diagnostic tests to identify patients for treatment. These challenges are particularly difficult to address when targeted therapies are developed to treat diseases with multiple molecular subtypes that occur at low frequencies. To help address these challenges, the US Food and Drug Administration recently published a draft guidance entitled “Developing Targeted Therapies in Low‐Frequency Molecular Subsets of a Disease.” Here we provide additional information on specific aspects of targeted therapy development in diseases with low‐frequency molecular subsets.

Author's reply to comments on “A taxonomy of estimands for regulatory clinical trials with discontinuations”

The better‐half or trimmed‐mean estimand has a dual interpretation as a summary of half the distribution or of the whole distribution with half of it compressed. As Doi et al point out, in the second interpretation, the numerical difference between groups is the first one diluted by a factor of 2. In a different but related context, a similar duality has been exploited to make inference about one of the two parameters from information about the other and about the extent of dilution. A mathematically similar distinction arises for utility estimands, as Doi et al note. As they say, a far more important issue in that setting is the possibility that death is related to treatment. I agree that in that case the difference in outcome for survivors will often be much less important than the difference in survival. I would not say, however, that the utility estimand is not of interest, because it reflects the difference in survival as well as the difference in outcome for survivors. Rather, it would be seriously misleading in that case to attempt to estimate the utility estimand by the difference in outcome for survivors only.

Comments on ‘Estimands in clinical trials – broadening the perspective’

In Akacha’s formulation, the third question is divided between nonadherents due to toxicity and nonadherents due to lack of efficacy, but this division is both too specific and not specific enough. For example, a patient might have no improvement in symptoms and a mild toxicity, but if her symptoms had improved, she might have put up with the toxicity. So, did she drop out because of toxicity or because of lack of efficacy? It doesn’t really matter. What matters a lot is what happened to patients who dropped out. Did they suffer severe consequences either of the treatment itself or the withholding of a better one? If so, the severity of these consequences, not just the number of nonadherents, must be taken into account. On the other hand, if the likely consequence of inadherence is merely an expeditious switch to an effective drug, the tripartite formulation of the problem becomes especially attractive because inadherence is simply one mode of failure. There is then not much to be gained by following up inadherent patients just to see precisely how badly the treatment had failed them, nor by inventing a numerical value for what was not observed. This tripartite formulation helps draw attention to deficiencies in current practice. It makes immediately clear that inadherence itself is sometimes the relevant outcome for a given patient. In contrast, methods treating outcomes for inadherent patients as missing make no use of the fact of inadherence and all too much use of a supposed latent outcome, at best unobserved and at worst absurd (such as the value for a dead patient had he been alive). The argument in Sections 2 and 3, however, relies heavily on substituting the word effect in a nonstandard sense where I used outcomes above. The effect of a treatment is the outcome on that treatment compared to the outcome on some other treatment. This, not just the likely outcome on treatment, is what is relevant to patients, prescribers, regulators, and payers alike. If my physician tells me I will likely be dead in 10 years if I take her advice, that advice is incomplete without her also telling me I will likely be dead in a year otherwise. Some rather strong conclusions are offered in Sections 2 and 3 before the central difficulty is even mentioned: outcomes among adherents may be well defined, but the effect among adherents is not, because different patients may adhere to the test treatment than to the control. The effect of the test drug compared to control, as discussed in Section 4, may be partly to change some patients who would not have adhered to control into adherents on the test drug; partly to change other patients who would have adhered to control into nonadherents on the test drug; and partly to change the outcomes for patients who would have adhered to either treatment. This difficulty is correctly though succinctly addressed in Section 4.2. A simple comparison of adherents in the two groups does not compare like to like; it does not estimate a drug effect at all, but a drug