Validity of using cancer registry data for comparative effectiveness research

Abstract

Purpose: Researchers often analyze cancer registry data to assess for differences in survival among cancer treatments. However, the retrospective non-random design of these analyses raise questions about study validity. The purpose of this study was to determine the extent to which comparative effectiveness analyses using cancer registry data produces results concordant with randomized clinical trials. \n\nMethods: We identified 141 randomized clinical trials referenced in the National Comprehensive Cancer Network Clinical Practice Guidelines for 8 common solid tumor types. For each trial we identified subjects from the National Cancer Database (NCDB) matching the eligibility criteria of the randomized trial. With each trial we used three Cox regression models to determine hazard ratios (HRs) for overall survival including univariable, multivariable, and propensity score adjusted models. Multivariable and propensity score analyses controlled for potential confounders including demographic, comorbidity, clinical, treatment, and tumor-related variables. Each NCDB survival analysis was defined as discordant if the HR for the NCDB analysis fell outside the 95% confidence interval of the corresponding randomized trial.\n\nResults: NCDB analyses produced HRs for survival discordant with randomized trials in 62 (44%) univariable analyses, 43 (30%) multivariable analyses, and 51 (36%) propensity score models. NCDB analyses produced p-values discordant with randomized trials in 83 (59%) univariable analyses, 76 (54%) multivariable analyses, and 78 (55%) propensity score models. We did not identify any clinical trial characteristic associated with discordance between NCDB analyses and randomized trials including disease site, type of clinical intervention, or severity of cancer.\n\nConclusion: Comparative effectiveness research with cancer registry data often produces survival outcomes discordant to randomized clinical data. These findings help provide context for providers interpreting observational comparative effectiveness research in oncology.