Statistical successes and failures during the COVID‐19 pandemic: Comments on Ellenberg and Morris

Abstract

The ongoing HIV/AIDS and COVID-19 pandemics have exerted an extraordinary effect on human health, and they are prime examples of the unique and ever-present threat of emerging infectious diseases. In their article, Ellenberg and Morris describe the critical role of statistical thinking in responding to both pandemics. This includes modeling the outbreak, tracking reporting trends, characterizing the natural history of disease, and evaluating interventions. Statisticians have a unique opportunity to contribute during public health crises, to ensure that analyses are rigorous, inference is sound, and policy decisions are driven by the data. Though we remain in the midst of the COVID-19 pandemic, sufficient time has elapsed that we can reflect upon some successes and failures. These reveal lessons that we may carry forward into the future. A Success—Vaccines and Leveraging Existing Networks. The rapid identification of safe and efficacious COVID-19 vaccines is a standout achievement for the scientific community. A noteworthy contributor to this success was that the COVID-19 Prevention Network (CoVPN), formed by the US National Institutes of Health, relied heavily upon prior experience testing HIV vaccines.1 For over 20 years, the HIV Vaccine Trials Network (HVTN) has been hard at work implementing trials and developing associated statistical methods.2 HVTN statisticians played a critical role in CoVPN, advising vaccine companies working with Operation Warp Speed on their trial protocols. The end result is higher quality and more standardized results, enabling easier comparisons for regulators and for the public. Repurposing infrastructure put in place for other diseases has been an important strategy to efficiently conduct pandemic research. Many systems for influenza have been extended to include COVID-19. The Seattle Flu Study, created in 2018, detected some of the earliest SARS-CoV-2 infections in the United States.3 Leveraging the established relationships investigators had with their community, they were able to quickly stand up new SARS-CoV-2 studies in households and homeless shelters. Another positive example of a study that quickly adapted is the REMAP-CAP platform trial for simultaneous evaluation of multiple treatments for community-acquired pneumonia.4 In response to the COVID-19 pandemic, they implemented a pandemic appendix that enabled already participating trial sites to begin testing COVID-19 therapeutics. These successes highlight the value of building multi-functional studies, including planning in advance how to accommodate emerging infectious diseases. A Failure—National COVID Data and Missing Standardization. Unfortunately, there have also been failures in our response. Ellenberg and Morris document a range of data challenges familiar to statisticians even outside the context of a fast-moving pandemic—missing information, measurement error, lagged reporting, and lack of standardization. Infectious diseases are inherently local, so we like to study trends on a fine spatial scale. We want to disaggregate across ages, occupations, and racial/ethnic groups to track risk and target interventions. Individuals may have diverse reasons for seeking testing, thus it is useful to distinguish between those presenting with and without symptoms. Yet it has been incredibly challenging to find summary data with this level of detail. Notably, different states use different definitions for reporting, including different categorizations for age and racial/ethnic group.5 Though the Centers for Disease Control is rapidly expanding the functionality of their online dashboard,6 this utility should have been available much earlier.