Russell Knoth

Validity and Reliability of Value Assessment Frameworks for New Cancer Drugs

BACKGROUND Several organizations have developed frameworks to systematically assess the value of new drugs. These organizations include the American Society of Clinical Oncology (ASCO), the European Society for Medical Oncology (ESMO), the Institute for Clinical and Economic Review (ICER), and the National Comprehensive Cancer Network (NCCN). OBJECTIVES To understand the extent to which these four tools can facilitate value-based treatment decisions in oncology. METHODS In this pilot study, eight panelists conducted value assessments of five advanced lung cancer drugs using the ASCO, ESMO, and ICER frameworks. The panelists received instructions and published clinical data required to complete the assessments. Published NCCN framework scores were abstracted. The Kendalls W coefficient was used to measure convergent validity among the four frameworks. Intraclass correlation coefficients were used to measure inter-rater reliability among the ASCO, ESMO, and ICER frameworks. Sensitivity analyses were conducted. RESULTS Drugs were ranked similarly by the four frameworks, with Kendalls W of 0.703 (P = 0.006) across all the four frameworks. Pairwise, Kendalls W was the highest for ESMO-ICER (W = 0.974; P = 0.007) and ASCO-NCCN (W = 0.944; P = 0.022) and the lowest for ICER-NCCN (W = 0.647; P = 0.315) and ESMO-NCCN (W = 0.611; P = 0.360). Intraclass correlation coefficients (confidence interval [CI]) for the ASCO, ESMO, and ICER frameworks were 0.786 (95% CI 0.517-0.970), 0.804 (95% CI 0.545-0.973), and 0.281 (95% CI 0.055-0.799), respectively. When scores were rescaled to 0 to 100, the ICER framework provided the narrowest band of scores. CONCLUSIONS The ASCO, ESMO, ICER, and NCCN frameworks demonstrated convergent validity, despite differences in conceptual approaches used. The ASCO inter-rater reliability was high, although potentially at the cost of user burden. The ICER inter-rater reliability was poor, possibly because of its failure to distinguish differential value among the sample of drugs tested. Refinements of all frameworks should continue on the basis of further testing and stakeholder feedback.

Measuring the Value of New Drugs: Validity and Reliability of 4 Value Assessment Frameworks in the Oncology Setting

BACKGROUND Several organizations have developed frameworks to systematically assess the value of new drugs. OBJECTIVE To evaluate the convergent validity and interrater reliability of 4 value frameworks to understand the extent to which these tools can facilitate value-based treatment decisions in oncology. METHODS Eight panelists used the American Society of Clinical Oncology (ASCO), European Society for Medical Oncology (ESMO), Institute for Clinical and Economic Review (ICER), and National Comprehensive Cancer Network (NCCN) frameworks to conduct value assessments of 15 drugs for advanced lung and breast cancers and castration-refractory prostate cancer. Panelists received instructions and published clinical data required to complete the assessments, assigning each drug a numeric or letter score. Kendalls Coefficient of Concordance for Ranks (Kendalls W) was used to measure convergent validity by cancer type among the 4 frameworks. Intraclass correlation coefficients (ICCs) were used to measure interrater reliability for each framework across cancers. Panelists were surveyed on their experiences. RESULTS Kendalls W across all 4 frameworks for breast, lung, and prostate cancer drugs was 0.560 (P= 0.010), 0.562 (P = 0.010), and 0.920 (P < 0.001), respectively. Pairwise, Kendalls W for breast cancer drugs was highest for ESMO-ICER and ICER-NCCN (W = 0.950, P = 0.019 for both pairs) and lowest for ASCO-NCCN (W = 0.300, P = 0.748). For lung cancer drugs, W was highest pairwise for ESMO-ICER (W = 0.974, P = 0.007) and lowest for ASCO-NCCN (W = 0.218, P = 0.839); for prostate cancer drugs, pairwise W was highest for ICER-NCCN (W = 1.000, P < 0.001) and lowest for ESMO-ICER and ESMO-NCCN (W = 0.900, P = 0.052 for both pairs). When ranking drugs on distinct framework subdomains, Kendalls W among breast cancer drugs was highest for certainty (ICER, NCCN: W = 0.908, P = 0.046) and lowest for clinical benefit (ASCO, ESMO, NCCN: W = 0.345, P = 0.436). Among lung cancer drugs, W was highest for toxicity (ASCO, ESMO, NCCN: W = 0. 944, P < 0.001) and lowest for certainty (ICER, NCCN: W = 0.230, P = 0.827); and among prostate cancer drugs, it was highest for quality of life (ASCO, ESMO: W = 0.986, P = 0.003) and lowest for toxicity (ASCO, ESMO, NCCN: W = 0.200, P = 0.711). ICC (95% CI) for ASCO, ESMO, ICER, and NCCN were 0.800 (0.660-0.913), 0.818 (0.686-0.921), 0.652 (0.466-0.834), and 0.153 (0.045-0.371), respectively. When scores were rescaled to 0-100, NCCN provided the narrowest band of scores. When asked about their experiences using the ASCO, ESMO, ICER, and NCCN frameworks, panelists generally agreed that the frameworks were logically organized and reasonably easy to use, with NCCN rated somewhat easier. CONCLUSIONS Convergent validity among the ASCO, ESMO, ICER, and NCCN frameworks was fair to excellent, increasing with clinical benefit subdomain concordance and simplicity of drug trial data. Interrater reliability, highest for ASCO and ESMO, improved with clarity of instructions and specificity of score definitions. Continued use, analyses, and refinements of these frameworks will bring us closer to the ultimate goal of using value-based treatment decisions to improve patient care and outcomes. DISCLOSURES This work was funded by Eisai Inc. Copher and Knoth are employees of Eisai Inc. Bentley, Lee, Zambrano, and Broder are employees of Partnership for Health Analytic Research, a health services research company paid by Eisai Inc. to conduct this research. For this study, Cohen, Huynh, and Neville report fees from Partnership for Health Analytic Research. Outside of this study, Cohen receives grants and direct consulting fees from various companies that manufacture and market pharmaceuticals. Mei reports a grant from Eisai Inc. during this study. The other authors have no disclosures to report. Study concept and design were contributed by Bentley and Broder, with assistance from Elkin and Cohen. Bentley took the lead in data collection, along with Elkin, Huynh, Mukherjea, Neville, Mei, Popescu, Lee, and Zambrano. Data interpretation was performed by Bentley and Broder, along with Elkin, Cohen, Copher, and Knoth. The manuscript was written primarily by Bentley, along with Elkin and Broder, and revised by Bentley, Broder, Elkin, Cohen, Copher, and Knoth. Select components of this works methods were presented at ISPOR 19th Annual European Congress held in Vienna, Austria, October 29-November 2, 2016, and Society for Medical Decision Making 38th Annual North American Meeting held in Vancouver, Canada, October 23-26, 2016.

Understanding the impact of settings of care and the development of chemotherapy-induced nausea and vomiting within a population of commercially insured patients.

298 Background: Research evaluating the impact of different clinical practice patterns according to settings of care and oncology patient outcomes is limited. This study describes chemotherapy-induced nausea and vomiting (CINV) rates in chemotherapy (CT) naïve cancer patients starting CT in a hospital outpatient (HOP) or community outpatient (COP) setting. METHODS Using the Optum Normative Health Information Database, patients with a new claim of CT and ≥1 outpatient cancer diagnosis claim between 1/1/06 - 6/30/12 were identified. Patients with previous inpatient cancer diagnoses, multi-day CT regimens or Medicare/Medicaid patients were excluded. CINV was defined using relevant claims-based ICD-9-CM diagnosis and procedure codes or a prescription claim for antiemetics within days 2-7 of 1st 8 CT cycles or 1st 6 months following the index CT claim. CINV events were evaluated descriptively and using regression models Results: Patients receiving CT in HOP vs. COP were similar in age. Patients receiving CT in COP vs. HOP setting were more likely to be female (78.7% vs. 62.8%), breast cancer patients (66.8% vs. 46.7%), live in the South region (49.7% vs. 44.1%) and have higher baseline healthcare costs (mean

Investigating rates of CINV across settings of care in Medicare: A retrospective claims-based analysis.

24,950 vs.

The impact of 5HT3RA antiemetics on the incidence of chemotherapy-induced nausea and vomiting (CINV), treatment adherence, and delay of therapy in early-stage breast cancer (BC) patients treated with moderately/highly emetogenic chemotherapy (MEC/HEC).

24,629) (all p<0.05). Patients in the HOP vs. COP settings had higher Charlson Comorbidity Index scores (mean 3.9 vs. 3.3, p < 0.05). More CINV events were reported for patients in COP vs. those in HOP settings (p < 0.05) (Table). After adjusting for clinical and demographic factors, number of CINV events remained higher for COP vs. HOP settings. However, we were unable to control for antiemetic prophylaxis use or CT emetogenic potential, due to coding irregularities. CONCLUSIONS Results suggest cancer patients starting CT in COP vs. HOP settings may have more CINV events. However, further analyses are needed to explore the impact of antiemetic prophylaxis use or CT emetogenic potential on CINV events between settings of care. [Table: see text].

Relationship between age and health care utilization in patients with myelodysplastic syndrome receiving supportive care.

297 Background: Clinical practice may differ according to settings of care and may impact both the quality of care delivered and, ultimately, patient outcomes. This study describes the differences in chemotherapy-induced nausea and vomiting (CINV) rates between chemotherapy (CT) naïve Medicare cancer patients starting CT in a hospital outpatient (HOP) or community outpatient (COP) setting. METHODS Using the 5% Medicare Fee-for-Service standard analytic files, patients with a new claim of CT and ≥1 outpatient cancer diagnosis claim between 1/1/10 - 6/30/11 were identified. Patients with a previous inpatient cancer diagnosis, multi-day CT cycles or who switched CT relevant to emetogenic potential were excluded. CINV was defined using relevant claims-based ICD-9-CM diagnosis and procedure codes within days 2-7 of the first 8 single-day CT cycles or the first 6 months following the index CT claim. CINV events were evaluated descriptively and using regression models. RESULTS Medicare patients receiving CT in HOP (n=1,007) vs. COP (n=1,080) were similar in demographics such as age, race, and baseline healthcare costs. However, Medicare patients receiving CT in COP compared to those in HOP settings were more likely to be female (57.0% vs. 44.7%), breast cancer patients (27.6% vs. 16.0%), live in the South region (37.7% vs. 32.3%), have higher Charlson Comorbidity Index scores (mean 5.2 vs. 4.8) and receive moderately to highly emetogenic CT (44.1% vs. 36.0%) (all p<0.05). Overall, 13.9% had any CINV in the evaluation period. More CINV events per patient were reported among those in COP compared to those in HOP settings (0.43 vs 0.27, p <0.05). However, differences between settings of care were not shown to be significantly different in adjusted regression analyses (p=0.177). CONCLUSIONS We found the population characteristics between Medicare patients treated with CT in HOP and COP to vary on a number of factors. However, after controlling for these differences, our results suggest the number of CINV events was similar across settings of care. Future research should further clarify how differences in quality of care for antiemesis between COP and HOP settings may impact the incidence of CINV events in this population.

Health care utilization and costs in patients with early onset myelodysplastic syndrome in a commercially insured population.

75 Background: The effectiveness of 5-HT3receptor antagonists (5HTs) for the prevention of CINV is well established. Less is known about the impact of these drugs on chemotherapy adherence (AD) and preventing delayed therapy (DT). We examined the impact of palonosetron (PAL) vs. other 5HTs on CINV incidence, treatment AD and DT in early-stage BC. METHODS An observational nested case-control study was conducted using the HealthCore Integrated Research Database (HIRD). Female patients (pts) were identified by their first claim for IV MEC/HEC between 1/1/02 and 10/31/10 (index). Inclusion criteria were: ≥ 12 months health plan eligibility pre-index (baseline) and ≥ 6 months post-index (follow-up); ≥ 1 baseline claim for BC; no claim for secondary or multiple primary neoplasms; and ≥ 1 claim for 5HT during follow-up. CINV was defined using ICD-9 codes for nausea, vomiting, or related events, and use of rescue medications for 5 days after MEC/HEC. AD was defined as receiving the requisite number of cycles within the recommended NCCN timeframe. DT was defined as exceeding 2x the NCCN-recommended cycle length between MEC/HEC claims. Outcomes were assessed using descriptive analysis and multivariate logistic regression, controlling for demographics, baseline medical conditions, and index therapy. RESULTS We identified 682/696 [MEC] and 1,782/3,103 [HEC] pts who received PAL or other 5HTs, respectively. For PAL vs. 5HT, 33.7% vs. 49.7% (OR: 0.51; 95% CI 0.41-0.65; p<0.01) had CINV (mean 0.27 vs. 0.61 events/cycle) in the MEC group; in the HEC group, 27.3% vs. 35.7% (OR: 0.61; 95% CI 0.53-0.7; p<0.01) had CINV (mean 0.21 vs. 0.34 events/cycle). PAL users delayed MEC at a lower rate than those using other 5HTs (2.6% vs. 8.2%; OR: 0.37; CI 0.21-0.65; p<0.01) but had similar AD (41.8% vs. 37.2%; OR: 1.2; CI 0.95-1.5; p=0.12). No differences in AD or DT were found among pts on HEC. CONCLUSIONS Among early-stage BC pts initiating a MEC or HEC agent, PAL was associated with fewer CINV events, a lower (MEC) or equal (HEC) rate of delayed therapy, and equal adherence compared to other 5HTs.

Chemotherapy induced nausea and vomiting in breast cancer treated with antiemetic prophylaxis as recommended by the ASCO antiemesis guidelines.

6560 Background: Myelodysplastic syndrome (MDS) is rare in people under 50. Little is known about the disease in this group, particularly among those patients who receive supportive care only. METHODS This was a descriptive cohort study using a large commercial claims database. The study included patients with an initial MDS claim (ICD-9-CM 238.72-238.75) between 2/1/2007 and 7/31/2008, who were continuously enrolled for 6 months prior to and 12 months following the index claim. Patients were excluded it they were treated with either hypomethylating agents (HMA) or thalidomide analogues (TA). Once identified, patients were stratified into two group, those aged <50 and those ≥50. Demographic variables and utilization and costs were calculated in the pre- and postindex period, respectively. RESULTS The study identified 1,209 patients newly diagnosed with MDS and continuously enrolled in the health plan. After excluding 76 (6.3%) who were treated with HMA/TA, the final cohort contained 1,133 patients. Of these, 221 (19.5%) were <50 and 912 (80.5%) were ≥50. In the first year after diagnosis, patients <50 had significantly fewer office visits than the older group (17.5 vs. 24.2, p<.001) but were no different in the proportion hospitalized (25.8% vs. 29.2%, respectively, p=.52) or in length of stay (7.8 vs. 9.0 days, p=.42). Mean total health care charges were

Validity and reliability of four value frameworks for cancer drugs.

30,177 (SD

A Novel Method for Evaluating Value Assessment Frameworks

53,550, Md