F. Reed Johnson

Conjoint analysis applications in health - A checklist: A report of the ISPOR Good Research Practices for Conjoint Analysis Task Force

BACKGROUND The application of conjoint analysis (including discrete-choice experiments and other multiattribute stated-preference methods) in health has increased rapidly over the past decade. A wider acceptance of these methods is limited by an absence of consensus-based methodological standards. OBJECTIVE The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Good Research Practices for Conjoint Analysis Task Force was established to identify good research practices for conjoint-analysis applications in health. METHODS The task force met regularly to identify the important steps in a conjoint analysis, to discuss good research practices for conjoint analysis, and to develop and refine the key criteria for identifying good research practices. ISPOR members contributed to this process through an extensive consultation process. A final consensus meeting was held to revise the article using these comments, and those of a number of international reviewers. RESULTS Task force findings are presented as a 10-item checklist covering: 1) research question; 2) attributes and levels; 3) construction of tasks; 4) experimental design; 5) preference elicitation; 6) instrument design; 7) data-collection plan; 8) statistical analyses; 9) results and conclusions; and 10) study presentation. A primary question relating to each of the 10 items is posed, and three sub-questions examine finer issues within items. CONCLUSIONS Although the checklist should not be interpreted as endorsing any specific methodological approach to conjoint analysis, it can facilitate future training activities and discussions of good research practices for the application of conjoint-analysis methods in health care studies.

Constructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force

Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods-and particularly discrete-choice experiments (DCEs)-have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health-A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task forces draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).

ISPOR task force reportConstructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force

Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods-and particularly discrete-choice experiments (DCEs)-have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health-A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task forces draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).

Conjoint Analysis Applications in Health - How are Studies being Designed and Reported?: An Update on Current Practice in the Published Literature between 2005 and 2008.

Despite the increased popularity of conjoint analysis in health outcomes research, little is known about what specific methods are being used for the design and reporting of these studies. This variation in method type and reporting quality sometimes makes it difficult to assess substantive findings. This review identifies and describes recent applications of conjoint analysis based on a systematic review of conjoint analysis in the health literature. We focus on significant unanswered questions for which there is neither compelling empirical evidence nor agreement among researchers.We searched multiple electronic databases to identify English-language articles of conjoint analysis applications in human health studies published since 2005 through to July 2008. Two independent reviewers completed the detailed data extraction, including descriptive information, methodological details on survey type, experimental design, survey format, attributes and levels, sample size, number of conjoint scenarios per respondent, and analysis methods. Review articles and methods studies were excluded. The detailed extraction form was piloted to identify key elements to be included in the database using a standardized taxonomy.We identified 79 conjoint analysis articles that met the inclusion criteria. The number of applied studies increased substantially over time in a broad range of clinical applications, cancer being the most frequent. Most used a discrete-choice survey format (71%), with the number of attributes ranging from 3 to 16. Most surveys included 6 attributes, and 73% presented 7–15 scenarios to each respondent. Sample size varied substantially (minimum = 13, maximum = 1258), with most studies (38%) including between 100 and 300 respondents. Cost was included as an attribute to estimate willingness to pay in approximately 40% of the articles across all years.Conjoint analysis in health has expanded to include a broad range of applications and methodological approaches. Although we found substantial variation in methods, terminology, and presentation of findings, our observations on sample size, the number of attributes, and number of scenarios presented to respondents should be helpful in guiding researchers when planning a new conjoint analysis study in health.

Willingness to pay for improved respiratory and cardiovascular health: a multiple-format, stated-preference approach

This study uses stated-preference (SP) analysis to measure willingness to pay (WTP) to reduce acute episodes of respiratory and cardiovascular ill health. The SP survey employs a modified version of the health state descriptions used in the Quality of Well Being (QWB) Index. The four health state attributes are symptom, episode duration, activity restrictions and cost. Preferences are elicited using two different SP formats: graded-pair and discrete-choice. The different formats cause subjects to focus on different evaluation strategies. Combining two elicitation formats yields more valid and robust estimates than using only one approach. Estimates of indirect utility function parameters are obtained using advanced panel econometrics for each format separately and jointly. Socio-economic differences in health preferences are modelled by allowing the marginal utility of money relative to health attributes to vary across respondents. Because the joint model captures the combined preference information provided by both elicitation formats, these model estimates are used to calculate WTP. The results demonstrate the feasibility of estimating meaningful WTP values for policy-relevant respiratory and cardiac symptoms, even from subjects who never have personally experienced these conditions. Furthermore, because WTP estimates are for individual components of health improvements, estimates can be aggregated in various ways depending upon policy needs. Thus, using generic health attributes facilitates transferring WTP estimates for benefit-cost analysis of a variety of potential health interventions.

Valuing Public Goods: Discrete versus Continuous Contingent-Valuation Responses

Independent applications of open-ended and dichotomous-choice formats are compared using tests of means, estimating joint likelihood functions and nonparametric tests of distributions. The null hypothesis of no difference in the open-ended and dichotomous-choice estimates of central tendency cannot be rejected for two out of three data sets, while estimated standard deviations are significantly different for all three data sets. In addition, actual dichotomous-choice means and standard deviations exceed those from comparable synthetic dichotomous-choice data sets, suggesting either open-ended questions underestimate values or dichotomous-choice bid structures may lead to systematic overestimates.

Multiple sclerosis patients—benefit-risk preferences: Serious adverse event risks versus treatment efficacy

Objective:The aim of this study is to estimate the willingness of multiple sclerosis (MS) patients to accept life-threatening adverse event risks in exchange for improvements in their MS related health outcomes.Methods:MS patients completed a survey questionnaire that included a series of choice-format conjoint tradeoff tasks. Patients chose hypothetical treatments from pairs of treatment alternatives with varying levels of clinical efficacy and associated risks.Results:Among the 651 patients who completed the survey, delay in years to disability progression was the most important factor in treatment preferences. In return for decreases in relapse rates from 4 to 1 and increases in delay in progression from 3 to 5 years, patients were willing to accept a 0.38% annual risk of death or disability from PML, a 0.39% annual risk of death from liver failure or a 0.48% annual risk of death from leukemia.Conclusions:Medical interventions carry risks of adverse outcomes that must be evaluated against their clinical benefits. Most MS patients indicated they are willing to accept risks in exchange for clinical efficacy. Patient preferences for potential benefits and risks can assist in decision-making.

Can public information programs affect risk perceptions

This article provides the first controlled evaluation of how different information materials explaining the risks from radon influenced peoples perceptions of these risks. Using a panel study, it was possible to observe how stated risk perceptions responded to information about indoor radon concentrations and brochures explaining the radon readings. The findings indicate that risk communication policies can be effective in modifying risk perceptions. Moreover, they have three specific implications for radon policy: (1) Public officials should not adopt strategies that provide minimal risk information to the public as a means of avoiding undue alarm, for this can have the reverse effect; (2) measures of the effectiveness of risk communication will depend on how education and behavior change are defined; (3) categorical guidelines about risk without quantitative information can lead people to treat the levels as thresholds, creating an artificial discontinuity in their responses to small changes in risk perceptions.

Valuing morbidity: An integration of the willingness-to-pay and health-status index literatures

Placing dollar values on human health has long been a controversial aspect of policy analysis and remains difficult given the relatively small number of morbidity-valuation studies available. By combining both the economic and health literature, this paper offers an alternative approach to morbidity valuation and provides estimates for a wide range of short-term health conditions.

Multiple Destination Trip Bias in Recreation Benefit Estimation

Balanced and multiple use of the public lands has become the theme of much of federal land use policy.1 As additional and alternative uses of the public lands are proposed, the measurement of the value of publicly managed recreation facilities is necessary in order to make rational comparisons. For the most part these alternative uses have quantifiable benefits and costs determined through market operations. As a matter of social policy, however, access to many publicly managed recreation facilities is often provided at prices less than the market prices that would otherwise prevail. Since prices determined in a market economy usually provide the basis for calculating the benefits and costs of various commodities or activities, situations involving commodities or activities acquired without the use of market exchange pose certain difficulties for value estimation. It is necessary in these cases to derive visitor willingness-to-pay from observed nonmarket data or from direct consumer responses to questions about recreation values. One widely accepted technique for estimating the willingness-to-pay for a recreation site in cost-benefit studies is the Clawson-Knetsch travel cost method.2 Recently published regulations implementing revised Principles and Standards for Planning Water and Related Land Resources will result in increased application of this technique in federal planning. The advantages and disadvantages of using the travel cost method for estimating recreation benefits have been widely discussed (e.g., Pearce [1978]). The travel cost method assumes, among other things, that all travel costs are incurred exclusively to obtain access to the single specific recreation site being valued and thus differences in travel cost may be treated as a proxy for price. This study (1) examines the consequences of multiple destination trips on travel cost estimates of benefits, (2) suggests a means of correcting the bias that arises under such circumstances, and (3) obtains nearly identical value estimates from the travel cost method and from a direct question on willingness-to-pay. The magnitude of multiple destination bias is