Axel C. Mühlbacher

Constructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force

Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods-and particularly discrete-choice experiments (DCEs)-have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health-A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task forces draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).

ISPOR task force reportConstructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force

Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods-and particularly discrete-choice experiments (DCEs)-have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health-A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task forces draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).

Patient Preferences Versus Physicians’ Judgement: Does it Make a Difference in Healthcare Decision Making?

Clinicians and public health experts make evidence-based decisions for individual patients, patient groups and even whole populations. In addition to the principles of internal and external validity (evidence), patient preferences must also influence decision making. Great Britain, Australia and Germany are currently discussing methods and procedures for valuing patient preferences in regulatory (authorization and pricing) and in health policy decision making. However, many questions remain on how to best balance patient and public preferences with physicians’ judgement in healthcare and health policy decision making. For example, how to define evaluation criteria regarding the perceived value from a patient’s perspective? How do physicians’ fact-based opinions also reflect patients’ preferences based on personal values? Can empirically grounded theories explain differences between patients and experts—and, if so, how? This article aims to identify and compare studies that used different preference elicitation methods and to highlight differences between patient and physician preferences. Therefore, studies comparing patient preferences and physician judgements were analysed in a review. This review shows a limited amount of literature analysing and comparing patient and physician preferences for healthcare interventions and outcomes. Moreover, it shows that methodology used to compare preferences is diverse. A total of 46 studies used the following methods—discrete-choice experiments, conjoint analyses, standard gamble, time trade-offs and paired comparisons—to compare patient preferences with doctor judgements. All studies were published between 1985 and 2011. Most studies reveal a disparity between the preferences of actual patients and those of physicians. For most conditions, physicians underestimated the impact of intervention characteristics on patients’ decision making. Differentiated perceptions may reflect ineffective communication between the provider and the patient. This in turn may keep physicians from fully appreciating the impact of certain medical conditions on patient preferences. Because differences exist between physicians’ judgement and patient preferences, it is important to incorporate the needs and wants of the patient into treatment decisions.

Analysis of physicians' perspectives versus patients' preferences: direct assessment and discrete choice experiments in the therapy of multiple myeloma.

BackgroundAgainst the background of patient involvement, understanding patients’ preferences for treatments is crucial: Do physicians have the same or a different perception of the patients’ preferences? As there is currently no cure for patients with multiple myeloma, primary objectives of treatment are to extend survival at the best possible quality of life. In this study, physicians’ beliefs about patients’ preferences regarding the treatment of multiple myeloma (MM) were explored in a direct assessment and a discrete choice experiment (DCE), and were compared to the previously explored patients’ views. How much do physicians know about their patients’ preferences?MethodsIn a preceding study with German multiple myeloma patients, relevant attributes of an ideal multiple myeloma treatment were collected by reviewing the literature and by conducting a qualitative study with focus groups. The attributes were analyzed using both a direct measurement (16 items on a five-point Likert scale) and a DCE (eight pairs with eight characteristics). For the present study, 30 German physicians reviewed the treatment attributes from the previous study for completeness. A total of 243 physicians participated in the study (including the 30 participants in the pre-test). The direct assessment and the DCE covered four major preference dimensions that both the literature review and the focus groups revealed: Aspects of medical effectiveness (including prolonged life expectancy, effectiveness and long duration of effect), side effects, quality of life (including social, physical and emotional quality of life) and flexibility (breaks in therapy and further treatment options).ResultsIn the direct measurement of patients’ preferences, physicians rated physical quality of life (specified as “reduced mobility or good mobility”), rare side effects and effectiveness aspects (duration of effect, maximal prolonged life expectancy and effectiveness) as the most important attributes from the patients’ perspective, followed by emotional quality of life (specified as “Not always think of the disease”) and therapy-free intervals. Especially further treatment options and dosage were more important to patients than physicians believed. In this case, the physicians had quite obviously underestimated the importance of these attributes from the perspective of those affected. Physicians ranked prolonged life expectancy as relatively the most important and significantly more important than all other treatment attributes. Further treatment options were the second most important attribute and significant compared to the attributes breaks in therapy and physical quality of life, whereas the patients ordered these two attributes in reverse order. Similarly, the patients gave the opposite relative importance to the next two priorities: self-application of treatment and emotional quality of life.ConclusionsAsking patients or physicians about the multiple myeloma patients’ treatment preferences, the combination of direct assessment and DCE proves to be a valid survey technique. Over a broad range of treatment attributes, the physicians’ perceptions of preferences were very close to those of multiple myeloma patients. Both the direct assessment of importance in order to rank the patient perceptions and the DCE provide important insights into the preference structure of patients with multiple myeloma. The findings can subsequently be used as a basis for tailoring health care services for multiple myeloma patients in reference to their preferences.

Making Good Decisions in Healthcare with Multi-Criteria Decision Analysis: The Use, Current Research and Future Development of MCDA

Healthcare decision making is usually characterized by a low degree of transparency. The demand for transparent decision processes can be fulfilled only when assessment, appraisal and decisions about health technologies are performed under a systematic construct of benefit assessment. The benefit of an intervention is often multidimensional and, thus, must be represented by several decision criteria. Complex decision problems require an assessment and appraisal of various criteria; therefore, a decision process that systematically identifies the best available alternative and enables an optimal and transparent decision is needed. For that reason, decision criteria must be weighted and goal achievement must be scored for all alternatives. Methods of multi-criteria decision analysis (MCDA) are available to analyse and appraise multiple clinical endpoints and structure complex decision problems in healthcare decision making. By means of MCDA, value judgments, priorities and preferences of patients, insurees and experts can be integrated systematically and transparently into the decision-making process. This article describes the MCDA framework and identifies potential areas where MCDA can be of use (e.g. approval, guidelines and reimbursement/pricing of health technologies). A literature search was performed to identify current research in healthcare. The results showed that healthcare decision making is addressing the problem of multiple decision criteria and is focusing on the future development and use of techniques to weight and score different decision criteria. This article emphasizes the use and future benefit of MCDA.

Choice Experiments to Quantify Preferences for Health and Healthcare: State of the Practice

Stated-preference methods increasingly are used to quantify preferences in health economics, health technology assessment, benefit-risk analysis and health services research. The objective of stated-preference studies is to acquire information about trade-off preferences among treatment outcomes, prioritization of clinical decision criteria, likely uptake or adherence to healthcare products and acceptability of healthcare services or policies. A widely accepted approach to eliciting preferences is discrete-choice experiments. Patient, physician, insurant or general-public respondents choose among constructed, experimentally controlled alternatives described by decision-relevant features or attributes. Attributes can represent complete health states, sets of treatment outcomes or characteristics of a healthcare system. The observed pattern of choice reveals how different respondents or groups of respondents implicitly weigh, value and assess different characteristics of treatments, products or services. An important advantage of choice experiments is their foundation in microeconomic utility theory. This conceptual framework provides tests of internal validity, guidance for statistical analysis of latent preference structures, and testable behavioural hypotheses. Choice experiments require expertise in survey-research methods, random-utility theory, experimental design and advanced statistical analysis. This paper should be understood as an introduction to setting up a basic experiment rather than an exhaustive critique of the latest findings and procedures. Where appropriate, we have identified topics of active research where a broad consensus has not yet been established.

Experimental measurement of preferences in health and healthcare using best-worst scaling: an overview

Best-worst scaling (BWS), also known as maximum-difference scaling, is a multiattribute approach to measuring preferences. BWS aims at the analysis of preferences regarding a set of attributes, their levels or alternatives. It is a stated-preference method based on the assumption that respondents are capable of making judgments regarding the best and the worst (or the most and least important, respectively) out of three or more elements of a choice-set. As is true of discrete choice experiments (DCE) generally, BWS avoids the known weaknesses of rating and ranking scales while holding the promise of generating additional information by making respondents choose twice, namely the best as well as the worst criteria. A systematic literature review found 53 BWS applications in health and healthcare. This article expounds possibilities of application, the underlying theoretical concepts and the implementation of BWS in its three variants: ‘object case’, ‘profile case’, ‘multiprofile case’. This paper contains a survey of BWS methods and revolves around study design, experimental design, and data analysis. Moreover the article discusses the strengths and weaknesses of the three types of BWS distinguished and offered an outlook. A companion paper focuses on special issues of theory and statistical inference confronting BWS in preference measurement.

Can patients diagnosed with schizophrenia complete choice-based conjoint analysis tasks?

AbstractBackground: Schizophrenia is a severe mental illness associated with hallucinations, delusions, apathy, poor social functioning, and impaired cognition. Researchers and funders have been hesitant to focus efforts on treatment preferences of patients with schizophrenia because of the perceived cognitive burden that research methods, such as conjoint analysis, place on them. Objective: The objective of this study was to test if patients diagnosed with schizophrenia were able to complete a choice-based conjoint analysis (often referred to as discrete-choice experiments) and to test if meaningful trade-offs were being made. Methods: German outpatients diagnosed with schizophrenia were eligible to participate in this study if they were aged 18–65 years, had received treatment for at least 1 year and were not experiencing acute symptoms. Conjoint analysis tasks were based on six attributes, each with two levels, which were identified via a literature review and focus groups. A psychologist in a professional interview facility presented each respondent with the eight tasks with little explanation. All interviews were recorded, transcribed, and analyzed to verify that respondents understood the tasks. Preferences were assessed using logistic regression, with a correction for clustering. Results: We found evidence that the 21 patients diagnosed with schizophrenia participating in the study could complete conjoint analysis tasks in a meaningful way. Patients not only related to the scenarios presented in conjoint tasks, but explicitly stated that they used their own preferences to judge which scenarios were better. Statistical analysis confirmed all hypotheses about the attributes (i.e. all attributes had the expected sign). Having a supportive physician, not feeling slowed, and improvements in stressful situations (p<0.01) were the most important attributes. Conclusions: We found that patients diagnosed with schizophrenia can complete conjoint analysis tasks, that they base their decisions on their own preferences, and that patients make trade-offs between attributes.

International experience with comparative effectiveness research: case studies from England/Wales and Germany.

PURPOSE Although the US has lagged behind international developments in health technology assessment (HTA), renewed interest in HTA in the US has been fueled by the appropriation of

A discrete-choice experiment to determine patient preferences for injectable multiple sclerosis treatments in Germany.

1.1 billion comparative effectiveness research (CER) in 2009 and the debate over health care reform. APPROACH To inform CER practices in the US, we present case studies of HTA from England/Wales and Germany: contrasting methods; relevance to the US; and impact on innovation. FINDINGS The National Institute of Health and Clinical Excellence (NICE) was established in 1999 to inform trusts within the National Health Service of England and Wales. It uses cost-effectiveness analysis to guide the allocation resource across preventative and curative interventions. In Germany, the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) was established in 2004 to inform reimbursement and pricing policies for the statutory sickness funds set by the Gemeinsamer Bundesaursschuss (G-BA). IQWiG evaluates competing technologies within specific therapeutic areas, placing more weight on clinical evidence and the relative efficiency of competing therapies. PRACTICAL IMPLICATIONS Although having deep political and cultural antecedents, differences between NICE and IQWiG can be explained by perspective: the former guiding resource allocation across an entire system (macro-evaluation), the latter focusing on efficiency within the bounds of a particular therapeutic area (micro-evaluation). Given the decentralized nature of the US health care system, and the relative powers of different medical specialties, the IQWiG model presents a more suitable case study to guided CER efforts in the US.