Sabine Grimm

EQ-5D-5L versus EQ-5D-3L: The Impact on Cost Effectiveness in the United Kingdom

OBJECTIVESnTo model the relationship between the three-level (3L) and the five-level (5L) EuroQol five-dimensional questionnaire and examine how differences have an impact on cost effectiveness in case studies.nnnMETHODSnWe used two data sets that included the 3L and 5L versions from the same respondents. The EuroQol Group data set (n = 3551) included patients with different diseases and a healthy cohort. The National Data Bank data set included patients with rheumatoid disease (n = 5205). We estimated a system of ordinal regressions in each data set using copula models to link responses of the 3L instrument to those of the 5L instrument and its UK tariff, and vice versa. Results were applied to nine cost-effectiveness studies.nnnRESULTSnBest-fitting models differed between the EuroQol Group and the National Data Bank data sets in terms of the explanatory variables, copulas, and coefficients. In both cases, the coefficients of the covariates and latent factors between the 3L and the 5L instruments were significantly different, indicating that moving between instruments is not simply a uniform re-alignment of the response levels for most dimensions. In the case studies, moving from the 3L to the 5L caused a decrease of up to 87% in incremental quality-adjusted life-years gained from effective technologies in almost all cases. Incremental cost-effectiveness ratios increased, often substantially. Conversely, one technology with a significant mortality gain saw increased incremental quality-adjusted life-years.nnnCONCLUSIONSnThe 5L shifts mean utility scores up the utility scale toward full health and compresses them into a smaller range, compared with the 3L. Improvements in quality of life are valued less using the 5L than using the 3L. The 3L and the 5L can produce substantially different estimates of cost effectiveness. There is no simple proportional adjustment that can be made to reconcile these differences.

Assessing the Expected Value of Research Studies in Reducing Uncertainty and Improving Implementation Dynamics

Background. With low implementation of cost-effective health technologies being a problem in many health systems, it is worth considering the potential effects of research on implementation at the time of health technology assessment. Meaningful and realistic implementation estimates must be of dynamic nature. Objective. To extend existing methods for assessing the value of research studies in terms of both reduction of uncertainty and improvement in implementation by considering diffusion based on expert beliefs with and without further research conditional on the strength of evidence. Methods. We use expected value of sample information and expected value of specific implementation measure concepts accounting for the effects of specific research studies on implementation and the reduction of uncertainty. Diffusion theory and elicitation of expert beliefs about the shape of diffusion curves inform implementation dynamics. We illustrate use of the resulting dynamic expected value of research in a preterm birth screening technology and results are compared with those from a static analysis. Results. Allowing for diffusion based on expert beliefs had a significant impact on the expected value of research in the case study, suggesting that mistakes are made where static implementation levels are assumed. Incorporating the effects of research on implementation resulted in an increase in the expected value of research compared to the expected value of sample information alone. Conclusions. Assessing the expected value of research in reducing uncertainty and improving implementation dynamics has the potential to complement currently used analyses in health technology assessments, especially in recommendations for further research. The combination of expected value of research, diffusion theory, and elicitation described in this article is an important addition to the existing methods of health technology assessment.

When Future Change Matters: Modeling Future Price and Diffusion in Health Technology Assessments of Medical Devices.

BACKGROUNDnHealth technology assessments (HTAs) that take account of future price changes have been examined in the literature, but the important issue of price reductions that are generated by the reimbursement decision has been ignored.nnnOBJECTIVESnTo explore the impact of future price reductions caused by increasing uptake on HTAs and decision making for medical devices.nnnMETHODSnWe demonstrate the use of a two-stage modeling approach to derive estimates of technology price as a consequence of changes in technology uptake over future periods on the basis of existing theory and supported by empirical studies. We explore the impact on cost-effectiveness and expected value of information analysis in an illustrative example on the basis of a technology in development for preterm birth screening.nnnRESULTSnThe application of our approach to the case study technology generates smaller incremental cost-effectiveness ratios compared with the commonly used single cohort approach. The extent of this reduction in the incremental cost-effectiveness ratio depends on the magnitude of the modeled price reduction, the speed of diffusion, and the length of the assumed technology life horizon. Results of value of information analysis are affected through changes in the expected net benefit calculation, the addition of uncertain parameters, and the diffusion-adjusted estimate of the affected patient population.nnnCONCLUSIONSnBecause modeling future changes in price and uptake has the potential to affect HTA outcomes, modeling techniques that can address such changes should be considered for medical devices that may otherwise be rejected.

The HTA Risk Analysis Chart: Visualising the Need for and Potential Value of Managed Entry Agreements in Health Technology Assessment

BackgroundRecent changes to the regulatory landscape of pharmaceuticals may sometimes require reimbursement authorities to issue guidance on technologies that have a less mature evidence base. Decision makers need to be aware of risks associated with such health technology assessment (HTA) decisions and the potential to manage this risk through managed entry agreements (MEAs).ObjectiveThis work develops methods for quantifying risk associated with specific MEAs and for clearly communicating this to decision makers.MethodsWe develop the ‘HTA risk analysis chart’, in which we present the payer strategy and uncertainty burden (P-SUB) as a measure of overall risk. The P-SUB consists of the payer uncertainty burden (PUB), the risk stemming from decision uncertainty as to which is the truly optimal technology from the relevant set of technologies, and the payer strategy burden (PSB), the additional risk of approving a technology that is not expected to be optimal. We demonstrate the approach using three recent technology appraisals from the UK National Institute for Health and Clinical Excellence (NICE), each of which considered a price-based MEA.ResultsThe HTA risk analysis chart was calculated using results from standard probabilistic sensitivity analyses. In all three HTAs, the new interventions were associated with substantial risk as measured by the P-SUB. For one of these technologies, the P-SUB was reduced to zero with the proposed price reduction, making this intervention cost effective with near complete certainty. For the other two, the risk reduced substantially with a much reduced PSB and a slightly increased PUB.ConclusionsThe HTA risk analysis chart shows the risk that the healthcare payer incurs under unresolved decision uncertainty and when considering recommending a technology that is not expected to be optimal given current evidence. This allows the simultaneous consideration of financial and data-collection MEA schemes in an easily understood format. The use of HTA risk analysis charts will help to ensure that MEAs are considered within a standard utility-maximising health economic decision-making framework.

EMA and NICE appraisal processes for cancer drugs : current status and uncertainties

The Evidence Review Group members that contributed to this editorial are funded by the UK NIHR HTA Programme. The views and opinions expressed are those of the authors and do not necessarily reflect those of the UK Department of Health and the NIHR.

Adalimumab for Treating Moderate-to-Severe Hidradenitis Suppurativa: An Evidence Review Group Perspective of a NICE Single Technology Appraisal

As part of its single technology appraisal (STA) process, the UK National Institute for Health and Care Excellence (NICE) invited the manufacturer of adalimumab (AbbVie) to submit evidence on the clinical effectiveness and cost effectiveness of adalimumab for the treatment of moderate-to-severe hidradenitis suppurativa (HS). The appraisal assessed adalimumab as monotherapy in adult patients with an inadequate response to conventional systemic HS therapy. The School of Health and Related Research Technology Appraisal Group was commissioned to act as the independent Evidence Review Group (ERG). The ERG produced a critical review of the evidence for the clinical effectiveness and cost effectiveness of the technology based on the company’s submission to NICE. The evidence was mainly derived from three randomised controlled trials comparing adalimumab with placebo in adults with moderate-to-severe HS. The clinical-effectiveness review found that significantly more patients achieved a clinical response in the adalimumab groups than in the control groups but that the treatment effect varied between trials and there was uncertainty regarding its impact on a range of other relevant outcomes as well as long-term efficacy. The company’s submitted Markov model assessed the incremental cost effectiveness of adalimumab versus standard care for the treatment of HS from the perspective of the UK NHS and Personal Social Services (PSS) over a lifetime horizon. The original submitted model, including a patient access scheme (PAS), suggested that the incremental cost-effectiveness ratio (ICER) for adalimumab versus standard care was expected to be £16,162 per quality-adjusted life-year (QALY) gained. Following a critique of the model, the ERG’s preferred base case, which corrected programming errors and structural problems surrounding discontinuation rules and incorporated a lower unit cost for HS surgery, resulted in a probabilistic ICER of £29,725 per QALY gained. Based on additional analyses undertaken by the company and the ERG following the publication of the appraisal consultation document (ACD), the Appraisal Committee concluded that the maximum possible ICER for adalimumab compared with supportive care was between £28,500 and £33,200 per QALY gained but was likely to be lower. The Appraisal Committee recommended adalimumab (with the PAS) for the treatment of active moderate-to-severe HS in adults whose disease has not responded to conventional systemic therapy.

Estimating Future Health Technology Diffusion Using Expert Beliefs Calibrated to an Established Diffusion Model

OBJECTIVESnEstimates of future health technology diffusion, or future uptake over time, are a requirement for different analyses performed within health technology assessments. Methods for obtaining such estimates include constant uptake estimates based on expert opinion or analogous technologies and on extrapolation from initial data points using parametric curves-but remain divorced from established diffusion theory and modeling. We propose an approach to obtaining diffusion estimates using experts beliefs calibrated to an established diffusion model to address this methodologic gap.nnnMETHODSnWe performed an elicitation of experts beliefs on future diffusion of a new preterm birth screening illustrative case study technology. The elicited quantities were chosen such that they could be calibrated to yield the parameters of the Bass model of new product growth, which was chosen based on a review of the diffusion literature.nnnRESULTSnWith the elicitation of only three quantities per diffusion curve, our approach enabled us to quantify uncertainty about diffusion of the new technology in different scenarios. Pooled results showed that the attainable number of adoptions was predicted to be relatively low compared with what was thought possible. Further research evidence improved the attainable number of adoptions only slightly but resulted in greater speed of diffusion.nnnCONCLUSIONSnThe proposed approach of eliciting experts beliefs about diffusion and informing the Bass model has the potential to fill the methodologic gap evident in value of implementation and research, as well as budget impact and some cost-effectiveness analyses.

Nivolumab for Treating Metastatic or Unresectable Urothelial Cancer: An Evidence Review Group Perspective of a NICE Single Technology Appraisal

As part of its single technology appraisal (STA) process, the National Institute for Health and Care Excellence (NICE) invited the manufacturer (Bristol-Myers Squibb) of nivolumab (Opdivo®) to submit evidence of its clinical and cost effectiveness for metastatic or unresectable urothelial cancer. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG), which produced a detailed review of the evidence for the clinical and cost effectiveness of the technology, based on the company’s submission to NICE. Nivolumab was compared with docetaxel, paclitaxel, best supportive care and retreatment with platinum-based chemotherapy (cisplatin plus gemcitabine, but only for patients whose disease has had an adequate response in first-line treatment). Two ongoing, phase I/II, single-arm studies for nivolumab were identified, but no studies directly compared nivolumab with any specified comparator. Evidence from directly examining the single arms of the trial data indicated little difference between the outcomes measured from the nivolumab and comparator studies. A simulated treatment comparison (STC) analysis was used in an attempt to reduce the bias induced by naïve comparison, but there was no clear evidence that risk of bias was reduced. Multiple limitations in the STC were identified and remained. The effect of an analysis based on different combinations of covariates in the prediction model remains unknown. The ERG’s concerns regarding the economic analysis included the use of a non-established response-based survival analysis method, which introduced additional uncertainty. The use of time-dependent hazard ratios produced overfitting and was not represented in the probabilistic sensitivity analysis. The use of a treatment stopping rule to cap treatment cost left treatment effectiveness unaltered. A relevant comparator was excluded from the base-case analysis. The revised ERG deterministic base-case incremental cost-effectiveness ratios based on the company’s Appraisal Consultation Document response were £58,791, £78,869 and £62,352 per quality-adjusted life-year gained versus paclitaxel, docetaxel and best supportive care, respectively. Nivolumab was dominated by cisplatin plus gemcitabine in the ERG base case. Substantial uncertainties about the relative treatment effectiveness comparing nivolumab against all comparators remained. NICE did not recommend nivolumab, within its marketing authorisation, as an option for treating locally advanced, unresectable or metastatic urothelial carcinoma in adults who have had platinum-containing therapy, and considered that nivolumab was not suitable for use within the Cancer Drugs Fund.

Risk scores to guide referral decisions for people with suspected ovarian cancer in secondary care: a systematic review and cost-effectiveness analysis

BACKGROUNDnOvarian cancer is the sixth most common cancer in UK women and can be difficult to diagnose, particularly in the early stages. Risk-scoring can help to guide referral to specialist centres.nnnOBJECTIVESnTo assess the clinical and cost-effectiveness of risk scores to guide referral decisions for women with suspected ovarian cancer in secondary care.nnnMETHODSnTwenty-one databases, including MEDLINE and EMBASE, were searched from inception to November 2016. Review methods followed published guidelines. The meta-analysis using weighted averages and random-effects modelling was used to estimate summary sensitivity and specificity with 95% confidence intervals (CIs). The cost-effectiveness analysis considered the long-term costs and quality-adjusted life-years (QALYs) associated with different risk-scoring methods, and subsequent care pathways. Modelling comprised a decision tree and a Markov model. The decision tree was used to model short-term outcomes and the Markov model was used to estimate the long-term costs and QALYs associated with treatment and progression.nnnRESULTSnFifty-one diagnostic cohort studies were included in the systematic review. The Risk of Ovarian Malignancy Algorithm (ROMA) score did not offer any advantage over the Risk of Malignancy Index 1 (RMI 1). Patients with borderline tumours or non-ovarian primaries appeared to account for disproportionately high numbers of false-negative, low-risk ROMA scores. (Confidential information has been removed.) To achieve similar levels of sensitivity to the Assessment of Different NEoplasias in the adneXa (ADNEX) model and the International Ovarian Tumour Analysis (IOTA) groups simple ultrasound rules, a very low RMI 1 decision threshold (25) would be needed; the summary sensitivity and specificity estimates for the RMI 1 at this threshold were 94.9% (95% CI 91.5% to 97.2%) and 51.1% (95% CI 47.0% to 55.2%), respectively. In the base-case analysis, RMI 1 (threshold of 250) was the least effective [16.926 life-years (LYs), 13.820 QALYs] and the second cheapest (£5669). The IOTA groups simple ultrasound rules (inconclusive, assumed to be malignant) were the cheapest (£5667) and the second most effective [16.954 LYs, 13.841 QALYs], dominating RMI 1. The ADNEX model (threshold of 10%), costing £5699, was the most effective (16.957 LYs, 13.843 QALYs), and compared with the IOTA groups simple ultrasound rules, resulted in an incremental cost-effectiveness ratio of £15,304 per QALY gained. At thresholds of up to £15,304 per QALY gained, the IOTA groups simple ultrasound rules are cost-effective; the ADNEX model (threshold of 10%) is cost-effective for higher thresholds.nnnLIMITATIONSnInformation on the downstream clinical consequences of risk-scoring was limited.nnnCONCLUSIONSnBoth the ADNEX model and the IOTA groups simple ultrasound rules may offer increased sensitivity relative to current practice (RMI 1); that is, more women with malignant tumours would be referred to a specialist multidisciplinary team, although more women with benign tumours would also be referred. The cost-effectiveness model supports prioritisation of sensitivity over specificity. Further research is needed on the clinical consequences of risk-scoring.nnnSTUDY REGISTRATIONnThis study is registered as PROSPERO CRD42016053326.nnnFUNDING DETAILSnThe National Institute for Health Research Health Technology Assessment programme.

Clinically inappropriate post hoc exclusion of study participants from test accuracy calculations: the ROMA score, an example from a recent NICE diagnostic assessment:

Objective To explore how the definition of the target condition and post hoc exclusion of participants can limit the usefulness of diagnostic accuracy studies. Methods We used data from a systematic review, conducted for a NICE diagnostic assessment of risk scores to inform secondary care decisions about specialist referral for women with suspected ovarian cancer, to explore how the definition of the target condition and post hoc exclusion of participants can limit the usefulness of diagnostic accuracy studies to inform clinical practice. Results Fourteen of the studies evaluated the ROMA score, nine used Abbott ARCHITECT tumour marker assays, five used Roche Elecsys. The summary sensitivity estimate (Abbott ARCHITECT) was highest, 95.1% (95% CI: 92.4 to 97.1%), where analyses excluded participants with borderline tumours or malignancies other than epithelial ovarian cancer and lowest, 75.0% (95% CI: 60.4 to 86.4%), where all participants were included. Results were similar for Roche Elecsys tumour marker assays. Although the number of patients involved was small, data from studies that reported diagnostic accuracy for both the whole study population and with post hoc exclusion of those with borderline or non-epithelial malignancies suggested that patients with borderline or malignancies other than epithelial ovarian cancer accounts for between 50 and 85% of false-negative ROMA scores. Conclusions Our results illustrate the potential consequences of inappropriate population selection in diagnostic studies; women with non-epithelial ovarian cancers or non-ovarian primaries, and those borderline tumours may be disproportionately represented among those with false negative, ‘low risk’ ROMA scores. These observations highlight the importance of giving careful consideration to how the target condition has been defined when assessing whether the diagnostic accuracy estimates reported in clinical studies will translate into clinical utility in real-world settings.