Kevin Marsh

Assessing the Value of Healthcare Interventions Using Multi-Criteria Decision Analysis: A Review of the Literature

The objective of this study is to support those undertaking a multi-criteria decision analysis (MCDA) by reviewing the approaches adopted in healthcare MCDAs to date, how these varied with the objective of the study, and the lessons learned from this experience. Searches of EMBASE and MEDLINE identified 40 studies that provided 41 examples of MCDA in healthcare. Data were extracted on the objective of the study, methods employed, and decision makers’ and study authors’ reflections on the advantages and disadvantages of the methods. The recent interest in MCDA in healthcare is mirrored in an increase in the application of MCDA to evaluate healthcare interventions. Of the studies identified, the first was published in 1990, but more than half were published since 2011. They were undertaken in 18 different countries, and were designed to support investment (coverage and reimbursement), authorization, prescription, and research funding allocation decisions. Many intervention types were assessed: pharmaceuticals, public health interventions, screening, surgical interventions, and devices. Most used the value measurement approach and scored performance using predefined scales. Beyond these similarities, a diversity of different approaches were adopted, with only limited correspondence between the approach and the type of decision or product. Decision makers consulted as part of these studies, as well as the authors of the studies are positive about the potential of MCDA to improve decision making. Further work is required, however, to develop guidance for those undertaking MCDA.

Multiple Criteria Decision Analysis for Health Care Decision Making—Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force

Health care decisions are complex and involve confronting trade-offs between multiple, often conflicting objectives. Using structured, explicit approaches to decisions involving multiple criteria can improve the quality of decision making. A set of techniques, known under the collective heading, multiple criteria decision analysis (MCDA), are useful for this purpose. In 2014, ISPOR established an Emerging Good Practices Task Force. The task forces first report defined MCDA, provided examples of its use in health care, described the key steps, and provided an overview of the principal methods of MCDA. This second task force report provides emerging good-practice guidance on the implementation of MCDA to support health care decisions. The report includes: a checklist to support the design, implementation and review of an MCDA; guidance to support the implementation of the checklist; the order in which the steps should be implemented; illustrates how to incorporate budget constraints into an MCDA; provides an overview of the skills and resources, including available software, required to implement MCDA; and future research directions.

Amplifying Each Patient's Voice: A Systematic Review of Multi-criteria Decision Analyses Involving Patients.

BackgroundQualitative methods tend to be used to incorporate patient preferences into healthcare decision making. However, for patient preferences to be given adequate consideration by decision makers they need to be quantified. Multi-criteria decision analysis (MCDA) is one way to quantify and capture the patient voice. The objective of this review was to report on existing MCDAs involving patients to support the future use of MCDA to capture the patient voice.MethodsMEDLINE and EMBASE were searched in June 2014 for English-language papers with no date restriction. The following search terms were used: ‘multi-criteria decision*’, ‘multiple criteria decision*’, ‘MCDA’, ‘benefit risk assessment*’, ‘risk benefit assessment*’, ‘multicriteri* decision*’, ‘MCDM’, ‘multi-criteri* decision*’. Abstracts were included if they reported the application of MCDA to assess healthcare interventions where patients were the source of weights. Abstracts were excluded if they did not apply MCDA, such as discussions of how MCDA could be used; or did not evaluate healthcare interventions, such as MCDAs to assess the level of health need in a locality. Data were extracted on weighting method, variation in patient and expert preferences, and discussion on different weighting techniques.ResultsThe review identified ten English-language studies that reported an MCDA to assess healthcare interventions and involved patients as a source of weights. These studies reported 12 applications of MCDA. Different methods of preference elicitation were employed: direct weighting in workshops; discrete choice experiment surveys; and the analytical hierarchy process using both workshops and surveys. There was significant heterogeneity in patient responses and differences between patients, who put greater weight on disease characteristics and treatment convenience, and experts, who put more weight on efficacy. The studies highlighted cognitive challenges associated with some weighting methods, though patients’ views on their ability to undertake weighting tasks was positive.ConclusionThis review identified several recent examples of MCDA used to elicit patient preferences, which support the feasibility of using MCDA to capture the patient voice. Challenges identified included, how best to reflect the heterogeneity of patient preferences in decision making and how to manage the cognitive burden associated with some MCDA tasks.

International society for pharmacoeconomics and outcomes research comments on the American society of clinical oncology value framework

As members of the International Society for Pharmacoeconomics and Outcomes Research, we read with great interest the new American Society of Clinical Oncology (ASCO) conceptual framework to assess the value of cancer treatment options.1 We applaud the Value in Cancer Care Task Force for proposing a conceptual framework to support clinicians and patients in assessing the value of new cancer treatments. We acknowledge the challenges facing clinician–patient decision making, particularly concerning cancer treatments. Like ASCO, we recognize that the cost of treatments is increasingly being placed on patients through cost sharing and that engaging patients as part of making individual treatment decisions is of high importance. The ASCO framework highlights the growing tension among patients, insurance companies, and product manufacturers in a dynamic health care environment. In that light, the framework deserves a field test, and we look forward to seeing the outcome of that experience. We also appreciate the opportunity to offer comments and suggestions on the ASCO framework at this early stage, and our membership stands ready to support ASCO in future enhancements.

Model-Based Cost-Effectiveness Analyses for the Treatment of Chronic Lymphocytic Leukaemia: A Review of Methods to Model Disease Outcomes and Estimate Utility

Assessing the economic value of treatments for chronic lymphocytic leukaemia (CLL) is necessary to support healthcare decision makers; however, it poses a number of challenges. This paper reviews economic models of CLL treatment to learn the lessons from this experience and support ongoing model efforts. A search of databases and submissions to key health technology assessment agencies identified nine models. The modelling approaches adopted across these studies were fairly similar, with most models adopting a cohort Markov structure, though one example of a discrete event simulation was identified. While the cohort Markov approach has been acceptable to the National Institute for Health and Care Excellence, the review identifies a number of key uncertainties with these models, including the extrapolation of survival outcomes beyond the period observed by the trial, the effectiveness of second-line therapies, and estimates of health state utility. Further work is required to overcome these uncertainties, including comprehensive sensitivity analysis, systematic review of the evidence on the natural progression of CLL, and the collection of longer-term trial and registry data.

MCDA swing weighting and discrete choice experiments for elicitation of patient benefit‐risk preferences: a critical assessment

Multiple criteria decision analysis swing weighting (SW) and discrete choice experiments (DCE) are appropriate methods for capturing patient preferences on treatment benefit‐risk trade‐offs. This paper presents a qualitative comparison of the 2 methods.

Model-Based Cost-Effectiveness Analyses for the Treatment of Chronic Myeloid Leukaemia: A Review and Summary of Challenges

Assessing the economic value of treatments for chronic myeloid leukaemia (CML) is important but poses a number of challenges. This paper reviews economic models of CML treatment to learn lessons from this experience and support ongoing efforts to model CML. A search of databases and submissions to key health technology assessment agencies identified 12 studies that reported 22 models. Common practice included the use of cohort Markov models—most models used health states organised around the key stages in CML: chronic phase, accelerated phase and blast phase—and the use of utility estimates in the literature that correspond with the National Institute for Health and Care Excellence reference case. Two key areas of uncertainty were the extrapolation of survival outcomes beyond the period observed by the trial; and the effectiveness of second-line therapies. Further work is required to overcome these uncertainties in existing models, such as longer-term trial data collection, including trials of second-line therapies; validation of health-related quality-of-life instruments; and the testing of alternative modelling approaches. In the meantime, it is important that the impact of uncertainties is tested through the use of sensitivity and scenario analysis.

The Use of MCDA in HTA: Great Potential, but More Effort Needed

The potential for multi-criteria decision analysis (MCDA) to support health technology assessment (HTA) has been much discussed, and various HTA agencies are piloting or applying MCDA. Alongside these developments, good practice guidelines for the application of MCDA in health care have been developed. An assessment of current applications of MCDA to HTA in light of good practice guidelines reveals, however, that many have methodologic flaws that undermine their usefulness. Three challenges are considered: the use of additive models, a lack of connection between criteria scales and weights, and the use of MCDA in economic evaluation. More attention needs to be paid to MCDA good practice by researchers, journal editors, and decision makers and further methodologic developments are required if MCDA is to achieve its potential to support HTA.

Incorporating Preferences and Priorities into MCDA: Selecting an Appropriate Scoring and Weighting Technique

A key component of many multi-criteria decision analyses (MCDAs) is the elicitation of stakeholder preferences in the form or scores and weights. A challenge to the MCDA practitioner is that there is little guidance about how to choose between the many scoring and weighting techniques. This chapter describes and illustrates the four commonly used methods – direct rating (specifically an instance of the use of the Evidence and Value: Impact on Decision Making (EVIDEM) framework), Keeney-Raiffa MCDA, the analytical hierarchy process and discrete choice experiment – and identifies key differences between these techniques in order to support researchers to determine the most appropriate technique in different circumstances. It is concluded that there is no ‘best’ MCDA method, with the pertinence of methods depending on the objective of the analysis.

PATIENT-CENTERED DECISION MAKING: LESSONS FROM MULTI-CRITERIA DECISION ANALYSIS FOR QUANTIFYING PATIENT PREFERENCES

OBJECTIVES Patient preferences should be a central consideration in healthcare decision making. However, stories of patients challenging regulatory and reimbursement decisions has led to questions on whether patient voices are being considered sufficiently during those decision making processes. This has led some to argue that it is necessary to quantify patient preferences before they can be adequately considered. METHODS This study considers the lessons from the use of multi-criteria decision analysis (MCDA) for efforts to quantify patient preferences. It defines MCDA and summarizes the benefits it can provide to decision makers, identifies examples of MCDAs that have involved patients, and summarizes good practice guidelines as they relate to quantifying patient preferences. RESULTS The guidance developed to support the use of MCDA in healthcare provide some useful considerations for the quantification of patient preferences, namely that researchers should give appropriate consideration to: the heterogeneity of patient preferences, and its relevance to decision makers; the cognitive challenges posed by different elicitation methods; and validity of the results they produce. Furthermore, it is important to consider how the relevance of these considerations varies with the decision being supported. CONCLUSIONS The MCDA literature holds important lessons for how patient preferences should be quantified to support healthcare decision making.