University of Pennsylvania 11th annual conference on statistical issues in clinical trials: Estimands, missing data and sensitivity analysis (afternoon panel session)

Abstract

Roderick J. Little: There has been quite a bit of discussion about the National Academy of Sciences (NAS) study. I think the main focus of that study was on internal validity rather than external validity, although external validity is clearly important. Key points of the study are as follows. (1) NAS Study Panel defined data as missing if the missingness hides quantities that are meaningful for analysis. (2) The clinical trialists on the NAS Panel made the point very strongly that the best solution to missing data is to design and implement the study in a way that limits to the degree possible the amount of missing data. The reason is that any analysis method for handling missing data comes with unverifiable assumptions, and in confirmatory trials we want to limit those kinds of assumptions. (3) The message from the NAS Panel to prevent missing data has had some impact on practice, with pharmaceutical companies paying far more attention to avoiding it in the conduct of trials. (4) Analysis methods need to be driven by plausible scientific assumptions. If you look at analysis methods in this conference, it is clear that we’ve come a long way from ‘‘last observation carried forward’’ imputation. I think that is good news. (5) The NAS Panel recommended sensitivity analyses to assess impact of alternative assumptions about missing data—I’ll say a little bit about this later. Sensitivity analyses make pharmaceutical companies nervous, but concerns about lack of robustness of findings to sensitivity analysis reinforce the idea that it is important not to have too much missing data, because a large amount of missing data tends to reduce the robustness of a treatment effect.