Vikram Kilambi

Constructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force

Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods-and particularly discrete-choice experiments (DCEs)-have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health-A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task forces draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).

ISPOR task force reportConstructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force

Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods-and particularly discrete-choice experiments (DCEs)-have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health-A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task forces draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).

Eliciting Benefit-Risk Preferences and Probability-Weighted Utility Using Choice-Format Conjoint Analysis

This study applies conjoint analysis to estimate health-related benefit-risk tradeoffs in a non-expected-utility framework. We demonstrate how this method can be used to test for and estimate nonlinear weighting of adverse-event probabilities and we explore the implications of nonlinear weighting on maximum acceptable risk (MAR) measures of risk tolerance. We obtained preference data from 570 Crohn’s disease patients using a web-enabled conjoint survey. Respondents were presented with choice tasks involving treatment options that involve different efficacy benefits and different mortality risks for 3 possible side effects. Using conditional logit maximum likelihood estimation, we estimate preference parameters using 3 models that allow for nonlinear preference weighting of risks—a categorical model, a simple-weighting model, and a rank dependent utility (RDU) model. For the second 2 models we specify and jointly estimate 1- and 2-parameter probability weighting functions. Although the 2-parameter functions are more flexible, estimation of the 1-parameter functions generally performed better. Despite well-known conceptual limitations, the simple-weighting model allows us to estimate weighting function parameters that vary across 3 risk types, and we find some evidence of statistically significant differences across risks. The parameter estimates from RDU model with the single-parameter weighting function provide the most robust estimates of MAR. For an improvement in Crohn’s symptom severity from moderate and mild, we estimate maximum 10-year mortality risk tolerances ranging from 2.6% to 7.1%. Our results provide further the evidence that quantitative benefit-risk analysis used to evaluate medical interventions should account explicitly for the nonlinear probability weighting of preferences.

Modeling the allocation system: principles for robust design before restructuring.

The United Network for Organ Sharing is poised to resolve geographic disparity in liver transplantation and promote allocation based on medical urgency. At the time of writing, United Network for Organ Sharing is considering redistricting the organ procurement and transplantation network so that patient model for end-stage liver disease scores at transplant is more uniform across regions. We review the proposal with a systems-engineering focus and find that although the proposal is promising, it currently lacks evidence that it would perform effectively under realistic departures from its underlying data and assumptions. Moreover, we caution against prematurely focusing on redistricting as the only method to mitigate disparity. We describe system modeling principles which, if followed, will ensure that the redesigned allocation system is effective and efficient in achieving the intended goals.

Changes in geographic disparity in kidney transplantation since the final rule.

Background The national organ allocation system for deceased-donor kidney transplant will endure increased burden as the waitlist expands and organ shortage persists. The Department of Health and Human Services issued the “Final Rule” in 1998 that states “Organs and tissues ought to be distributed on the basis of objective priority criteria and not on the basis of accidents of geography.” However, it has not been addressed whether the rule was effective in encouraging regions to share the additional burden equitably. Objective To assess the significance of changes of geographic disparities for four metrics since the rule’s adoption: waiting times, transplant rates, pretransplant mortality, and organ quality. Methods Using Organ Procurement and Transplant Network data from 1988 through 2009, annual ranges of the metrics were calculated for all donor service areas and United Network for Organ Sharing regions. Time series analyses were used to compare the metrics before and after the enactment of the Final Rule. Results A total of 412,127 kidney transplant candidates and 178,163 deceased-donor recipients were analyzed. Demographics varied significantly by region. The ranges of the four metrics have worsened by approximately 30% or more after the Final Rule at both the regional and donor service area levels. Conclusion Increasing geographic disparity in allocation procedures may yield diverging outcomes and experiences in different locations for otherwise similar candidates. Consensus for measuring allocation discrepancies and policy interventions are required to mitigate the inequities.

The Effect of the Statewide Sharing Variance on Geographic Disparity in Kidney Transplantation in the United States

BACKGROUND AND OBJECTIVES The Statewide Sharing variance to the national kidney allocation policy allocates kidneys not used within the procuring donor service area (DSA), first within the state, before the kidneys are offered regionally and nationally. Tennessee and Florida implemented this variance. Known geographic differences exist between the 58 DSAs, in direct violation of the Final Rule stipulated by the US Department of Health and Human Services. This study examined the effect of Statewide Sharing on geographic allocation disparity over time between DSAs within Tennessee and Florida and compared them with geographic disparity between the DSAs within a state for all states with more than one DSA (California, New York, North Carolina, Ohio, Pennsylvania, Texas, and Wisconsin). DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS A retrospective analysis from 1987 to 2009 was conducted using Organ Procurement and Transplant Network data. Five previously used indicators for geographic allocation disparity were applied: deceased-donor kidney transplant rates, waiting time to transplantation, cumulative dialysis time at transplantation, 5-year graft survival, and cold ischemic time. RESULTS Transplant rates, waiting time, dialysis time, and graft survival varied greatly between deceased-donor kidney recipients in DSAs in all states in 1987. After implementation of Statewide Sharing in 1992, disparity indicators decreased by 41%, 36%, 31%, and 9%, respectively, in Tennessee and by 28%, 62%, 34%, and 19%, respectively in Florida, such that the geographic allocation disparity in Tennessee and Florida almost completely disappeared. Statewide kidney allocations incurred 7.5 and 5 fewer hours of cold ischemic time in Tennessee and Florida, respectively. Geographic disparity between DSAs in all the other states worsened or improved to a lesser degree. CONCLUSIONS As sweeping changes to the kidney allocation system are being discussed to alleviate geographic disparity--changes that are untested run the risk of unintended consequences--more limited changes, such as Statewide Sharing, should be further studied and considered.

Patient and parent preferences for immunoglobulin treatments: a conjoint analysis.

Abstract Objectives: The purpose was to quantify patient and parent preferences for administration attributes of immunoglobulin (IG) treatments; and determine which administration attributes were most important to users of IG treatment and whether patients and parents have similar preferences for administration attributes. Methods: US adult patients and parents of children with a self-reported physician diagnosis of a primary immunodeficiency disorder completed a best-practice web-enabled choice-format conjoint survey that presented a series of 12 choice questions, each including a pair of hypothetical IG-treatment profiles. After reviewing current therapies, each profile was defined by mode of administration, frequency, location, number of needle sticks, and treatment duration. Before answering the choice questions, respondents were told to assume all treatments worked equally well. Choice questions were based on a D-efficient experimental design. Preference weights for attribute levels were estimated using random-parameters logit for each sample (adult patients and parents). Tests were performed to determine potential interactions among the administration attributes. All respondents provided online informed consent. Results: In total, 252 patients and 66 parents completed the choice questions appropriately. Overall, both groups preferred a home setting, monthly frequency, fewer needle sticks, and shorter treatment durations of IG treatment relative to alternative choices (p < 0.05). Mode of administration was the least important attribute to both samples; however, parents strongly preferred self-administration to an appointment with a healthcare professional (p < 0.05), whereas patients slightly preferred self-administration but were indifferent to the two modes. Limitations: Respondents evaluate hypothetical treatments and differences can arise between stated and actual choices. Conclusions: Considering the hypothetical treatments evaluated, IG treatments that provide the option of a home setting, monthly frequency, fewer needle sticks, and shorter treatment durations may address the needs of both patients and parents. Patients and parents have different preferences for administration attributes of IG treatments.

Improving Liver Allocation Using Optimized Neighborhoods

BackgroundGeographic disparities persist in access to liver transplantation. Candidates with similar urgency experience varying opportunities for transplants across the United States. Policymakers are poised to act and 1 proposal entails reorganizing the current Organ Procurement and Transplant Network (OPTN) of 11 regions into 8 districts. However, redistricting has the shortcomings that Organ Procurement Organizations (OPOs) are disconnected from their immediate neighbors by district borders and that it is not easily responsive to uncertainty resulting from variability in donor and listing rates. MethodsWe introduce the notion of an OPOs neighborhood—a collection of donor service areas (DSA) surrounding the OPO that acts as the OPOs region in the current local-regional-national framework. Districts and concentric circles are special cases. We design 58 neighborhoods for the DSAs with several attractive properties and optimize them to balance supplies and demands using 10 years of Organ Procurement and Transplant Network data. We conduct a simulation experiment comparing current allocation, redistricting, and neighborhoods under current sharing policies with respect to the following metrics: total mortalities, DSA-average model for end-stage liver disease (MELD) at transplant, DSA-average MELD standard deviation, and average organ transport distance. Liver-simulated allocation model cannot accommodate neighborhoods, so we programmed a discrete-event simulator, LivSim, to approximate liver-simulated allocation model. ResultsWe exhibited a neighborhood solution. Compared with the current allocation, simulation results showed that neighborhoods reduce the DSA-average MELD standard deviation by 29% and save about 65 lives annually. Compared with redistricting, the neighborhoods had smaller average transport distances that were more uniform across DSAs, saved about 20 additional lives, and reduced DSA-average MELD standard deviation by an additional 17%. ConclusionsAlternatives to redistricting with desirable properties and performance are possible and should be considered.

Livsim: An Open-source Simulation Software Platform for Community Research and Development for Liver Allocation Policies

This study developed LivSim, an open-source software alternative to the Liver Simulated Allocation Model (LSAM v Aug 2014) created by the Scientific Registry of Transplant Recipients.

A Concentric-Neighborhoods Solution to Disparity in Liver Access that contains current UNOS Districts

Background Policymakers are deliberating reforms to reduce geographic disparity in liver allocation. Public comments and the United Network for Organ Sharing Liver and Intestinal Committee have expressed interest in refining the neighborhoods approach. Share 35 and Share 15 policies affect geographic disparity. Methods We construct concentric neighborhoods superimposing the current 11 regions. Using concepts from concentric circles, we construct neighborhoods for each donor service area (DSA) that consider all DSAs within 400, 500, or 600 miles as neighbors. We consider limiting each neighborhood to 10 DSAs and use no metrics for liver supplies and demands. We change Model for End-Stage Liver Disease (MELD) thresholds for the Share 15 policy to 18 or 20 and apply 3- and 5-point MELD proximity boosts to enhance local priority, control travel distances, and reduce disparity. We conduct simulations comparing current allocation with the neighborhoods and sharing policies. Results Concentric neighborhoods structures provide an array of solutions where simulation results indicate that they reduce geographic disparity, annual mortalities, and the airplane travel distances by varying degrees. Tuning of the parameters and policy combinations can lead to beneficial improvements with acceptable transplant volume loss and reductions in geographic disparity and travel distance. Particularly, the 10-DSA, 500-mile neighborhood solution with Share 35, Share 15, and 0-point MELD boost achieves such while limiting transplant volume losses to below 10%. Conclusions The current 11 districts can be adapted systematically by adding neighboring DSAs to improve geographic disparity, mortality, and airplane travel distance. Modifications to Share 35 and Share 15 policies result in further improvements. The solutions may be refined further for implementation.