Data challenges for evaluating new treatments

Abstract

Between 2009 and 2011, the US Food and Drug Administration (FDA) approved multiple new immunotherapy and targeted agents for the treatment of advanced adenocarcinoma of the lung, melanoma, breast cancer, prostate cancer, and renal cell carcinoma based on data from randomized controlled trials. However, little is known about the uptake of these new treatment options, treatment adherence, toxicity, and the impact on survival in patients treated in the real world. Understanding the generalizability of clinical trial findings to community practice is critical for improving health care delivery, especially when treatments are expensive, frequently with annual list prices in excess of $100,000, and can result in significant patient out-of-pocket cost. Patients with cancer in clinical trials differ substantially from those treated in community settings on characteristics associated with survival outcomes, including age, comorbidity burden, socioeconomic status, treatment adherence, and health insurance coverage. Providers and facilities participating in clinical trials may differ from those in community practice as well. In this issue of Cancer, Pulte and colleagues use data from the Surveillance, Epidemiology, and End Results (SEER) cancer registries to conduct a timely study comparing the survival of patients who were diagnosed before and after new treatment options were approved for selected advanced cancers. The study reports a modest improvement in survival for patients newly diagnosed with advanced adenocarcinoma of the lung, melanoma, breast cancer, prostate cancer, and renal cell carcinoma. Overall, point estimates of 2-year survival increased by approximately 3 percentage points for patients diagnosed in 2011 and 2012, compared with patients diagnosed in 2007 and 2008. The authors acknowledged several limitations of the study, principal among them the absence of relevant treatment information in SEER. Comprehensive patient data about tumor markers required for treatment eligibility, receipt of systemic treatment, specific agent(s) received, duration of treatment, and treatment response were unavailable. Instead, the authors used the year of diagnosis as a proxy for receipt of newly approved cancer treatments. Use of the date of diagnosis as a proxy for receipt of cancer treatment is especially problematic for immunotherapy, for which only small percentages of patients are eligible. In an attempt to address this limitation, the authors restricted the analysis to only patients with recorded chemotherapy in SEER (ranging from 54% of patients with non–small cell lung cancer to 5% of those with prostate cancer). However, as explained by the authors, SEER contains only limited information regarding chemotherapy, and immunotherapy is especially subject to misclassification. The unexpected finding of a decrease in survival between 2007 to 2008 and 2011 to 2012 for advanced melanoma and prostate cancer when analyses were restricted to only patients receiving chemotherapy reinforces concerns about this approach. Ultimately, without patient-level treatment information, changes in population-level estimates of survival cannot be attributed to newly introduced treatment options because they may simply reflect increased dissemination of other forms of treatments introduced in prior years. Another limitation of this study, not mentioned by the authors, is the choice of 2-year overall survival as the outcome. First, the authors do not provide background evidence for hypothesizing that the new treatments might increase the number of patients experiencing prolonged survival. The median improvement in overall survival reported in trials for these new treatment options is approximately 2 months. A small benefit observed in a trial may vanish entirely in community practice because of differences in the patients, providers, and facilities. Second, there is growing concern about the initially underestimated toxicity of these treatments. Evaluations of pivotal and postmarketing clinical trials suggest that the likelihood of harm from these new therapies can be substantial.