Commentary on Hay et al.: Can clinical trials data collection be improved by administrative data elements?

Abstract

Properly conducted, randomized controlled trials (RCTs) are considered the gold standard for evaluating the effectiveness of medical interventions. Primary data collection for RCTs requires significant time and resources and presents major challenges even under the most well-designed and carefully managed RCTs. Most notably, due to the protracted nature of follow-up time in RCTs, obstacles to tracking patient data (such as when patients visit multiple health institutions) can lead to missing information on baseline conditions, intervention compliance, and outcomes, which in turn can affect the validity of results. In order to improve data collection in clinical trials, some researchers have championed the use of clinical and administrative data to address missing data and potentially replace current costly data collection practices. However, for the use of these data sources in clinical trials to be feasible, researchers must ensure their quality and validity. To provide insight regarding the feasibility of using administrative data to enhance and validate clinical trials, Hay et al. measured the accuracy of selected outcomes of two published RCTs conducted by the Canadian Cancer Trials Group using routinely collected administrative data from the Institute of Clinical Evaluative Sciences (ICES) Ontario. Because of the lack of a unique identifier in their clinical trial, they relied on probabilistic linkages and were able to link their databases for slightly more than 90% of patients. After probabilistic linkage, they compared death, hospitalization, and emergency visit outcomes observed in their trials to the outcomes available at ICES Ontario. They observed very good agreement for death outcomes, but lower agreement for hospitalization and emergency visit outcomes. Notably, the use of administrative data allowed the identification of more outcome events both during the trial and outside the trial follow-up time window. The authors conclude that the use of administrative data could enhance clinical trial data collection. These results are encouraging as the accuracy of mortality as an outcome seems sufficient to be used reliably in a real-world clinical trial. The agreement for the two other reported outcomes is lower, but one may argue that the data obtained by ICES were more complete, thus enabling potentially more accurate reflection and estimations of effect for each of the outcomes. Moreover, in hospital systems where unique identifiers can be linked to trial participants, the linkage rates could be even higher. Despite these encouraging results, there are many aspects of using administrative data in clinical trials that need further validation before being able to implement these strategies routinely. First, it remains unclear whether the additional outcomes reported by the administrative datasets represent true novel events. Additional validation may be required to determine if data collection is complete and accurate enough to be able to assume that an outcome occurred when present in the data and did not occur when absent. Second, although the administrative data sources for this study seemed to provide fairly reliable outcome data, this still needs to be confirmed for other outcomes and other data elements such as baseline patient comorbidities or diagnoses. Indeed, since these data may come from different source systems with varying data collection strategies (for example, manual data entry vs automated data extraction), it is possible that the use of administrative data would not be appropriate for all type of data elements. This study was also conducted in the context of two oncology trials. The accuracy of outcome measures may be different in different study populations such as in ambulatory patients or critically ill patients. Finally,