Discussion on “Improving precision and power in randomized trials for COVID‐19 treatments using covariate adjustment, for binary, ordinal, and time‐to‐event outcomes” by David Benkeser, Ivan Diaz, Alex Luedtke, Jodi Segal, Daniel Scharfstein, and Michael Rosenblum

Abstract

The paper entitled “Precision and Power in Randomized Trials for COVID-19 Treatments Using Covariate Adjustment, for Binary, Ordinal, and Time-to-Event Outcomes” is a welcome addition to the literature on covariate adjustment in clinical trials conducted for medical research. The authors’ work has the potential to make a substantial impact on drug development through increased use of an extremely underutilized tool for improving precision in clinical trials, thereby reducing the number of studies that fail due to insufficient power. And the timeliness of this research could not be better, with the number of clinical trials currently being planned or already in the field to combat COVID-19. Both the urgency with which safe and effective treatments are needed in a pandemic and the competition that inherently follows multiple trials launched at the same time, for the same purpose (i.e., finding those safe and effective treatments), result in an unusually high need for accurately sized trials optimized to reach their goals. The advantages of covariate adjustment in the much simpler linear model context have been known for a while, and the method is popular with clinical trialists for its ability to improve the precision of treatment effect estimates with minimal assumptions. I have written before (LaVange 2014, 2019) of my quandary, after arriving at FDA in 2011, about the lack of a covariate adjustment guidance document, only to find that one had been drafted in the early 2000s but never published. The guidance was apparently shelved because, although noncontroversial for linear models, regulators were concerned that covariate adjustment would be misused in nonlinear models without appropriate guidance for that more complicated setting. As Biostatistics Office Director in the Center for Drug Evaluation and Research, I was able to prioritize an update of this guidance, which was completed soon after I left and issued in 2019 (FDA, 2019). An examination of the delay in FDA’s issuance of a covariate adjustment guidance helps to explain the importance of the Benkeser et al. paper to the drug development enterprise. The International Council on Harmonisation (ICH) published the guideline, E9 Statistical Principles for Clinical Trials (ICH, 1999), calling for the adjustment of covariates, measured before randomization, that were correlated with the primary trial outcomes. Purposes of this adjustment were twofold, to improve precision and adjust for imbalances between treatment groups. The European Medicines Agency (EMA) followed with a Points to Consider document in 2003 and a guidance document in 2015, both providing similar advice on covariate adjustment for trials regulated in the European Union (EU) region. The FDA guidance followed much later—20 years after ICH E9—and the primary reason was the inability to endorse a simple analytical tool like analysis of covariance when the analysis model was nonlinear. As the FDA guidance makes clear, prespecification of any covariate adjustments is required to ensure that the chance of making an erroneous conclusion about drug effects is not increased due to experimenting with different model adjustments after the trial concludes. The guidance also makes clear that even if the analysis model is inaccurate, the advantages of