Janine Astrid van Til

From efficacy to equity: Literature review of decision criteria for resource allocation and healthcare decisionmaking.

ObjectivesResource allocation is a challenging issue faced by health policy decisionmakers requiring careful consideration of many factors. Objectives of this study were to identify decision criteria and their frequency reported in the literature on healthcare decisionmaking.MethodAn extensive literature search was performed in Medline and EMBASE to identify articles reporting healthcare decision criteria. Studies conducted with decisionmakers (e.g., focus groups, surveys, interviews), conceptual and review articles and articles describing multicriteria tools were included. Criteria were extracted, organized using a classification system derived from the EVIDEM framework and applying multicriteria decision analysis (MCDA) principles, and the frequency of their occurrence was measured.ResultsOut of 3146 records identified, 2790 were excluded. Out of 356 articles assessed for eligibility, 40 studies included. Criteria were identified from studies performed in several regions of the world involving decisionmakers at micro, meso and macro levels of decision and from studies reporting on multicriteria tools. Large variations in terminology used to define criteria were observed and 360 different terms were identified. These were assigned to 58 criteria which were classified in 9 different categories including: health outcomes; types of benefit; disease impact; therapeutic context; economic impact; quality of evidence; implementation complexity; priority, fairness and ethics; and overall context. The most frequently mentioned criteria were: equity/fairness (32 times), efficacy/effectiveness (29), stakeholder interests and pressures (28), cost-effectiveness (23), strength of evidence (20), safety (19), mission and mandate of health system (19), organizational requirements and capacity (17), patient-reported outcomes (17) and need (16).ConclusionThis study highlights the importance of considering both normative and feasibility criteria for fair allocation of resources and optimized decisionmaking for coverage and use of healthcare interventions. This analysis provides a foundation to develop a questionnaire for an international survey of decisionmakers on criteria and their relative importance. The ultimate objective is to develop sound multicriteria approaches to enlighten healthcare decisionmaking and priority-setting.

A Comparison of Analytic Hierarchy Process and Conjoint Analysis Methods in Assessing Treatment Alternatives for Stroke Rehabilitation

AbstractBackground: With growing emphasis on patient involvement in health technology assessment, there is a need for scientific methods that formally elicit patient preferences. Analytic hierarchy process (AHP) and conjoint analysis (CA) are two established scientific methods — albeit with very different objectives. Objective: The objective of this study was to compare the performance of AHP and CA in eliciting patient preferences for treatment alternatives for stroke rehabilitation. Methods: Five competing treatments for drop-foot impairment in stroke were identified. One survey, including the AHP and CA questions, was sent to 142 patients, resulting in 89 patients for final analysis (response rate 63%). Standard software was used to calculate attribute weights from both AHP and CA. Performance weights for the treatments were obtained from an expert panel using AHP. Subsequently, the mean predicted preference for each of the five treatments was calculated using the AHP and CA weights. Differences were tested using non-parametric tests. Furthermore, all treatments were rank ordered for each individual patient, using the AHP and CA weights. Results: Important attributes in both AHP and CA were the clinical outcome (0.3 in AHP and 0.33 in CA) and risk of complications (about 0.2 in both AHP and CA). Main differences between the methods were found for the attributes ‘impact of treatment’ (0.06 for AHP and 0.28 for two combined attributes in CA) and ‘cosmetics and comfort’ (0.28 for two combined attributes in AHP and 0.05 for CA). On a group level, the most preferred treatments were soft tissue surgery (STS) and orthopedic shoes (OS). However, STS was most preferred using AHP weights versus OS using CA weights p< 0.001). This difference was even more obvious when interpreting the individual treatment ranks. Nearly all patients preferred STS according to the AHP predictions, while >50% of the patients chose OS instead of STS, as most preferred treatment using CA weights. Conclusion: While we found differences between AHP and CA, these differences were most likely caused by the labeling of the attributes and the elicitation of performance judgments. CA scenarios are built using the level descriptions, and hence provide realistic treatment scenarios. In AHP, patients only compared less concrete attributes such as ‘impact of treatment.’ This led to less realistic choices, and thus overestimation of the preference for the surgical scenarios. Several recommendations are given on how to use AHP and CA in assessing patient preferences.

The Use of the Analytic Hierarchy Process to Aid Decision Making in Acquired Equinovarus Deformity

OBJECTIVE To increase the transparency of decision making about treatment in patients with equinovarus deformity poststroke. DESIGN The analytic hierarchy process (AHP) was used as a structured methodology to study the subjective rationale behind choice of treatment. SETTING An 8-hour meeting at a centrally located rehabilitation center in The Netherlands, during which a patient video was shown to all participants (using a personal computer and a large screen) and the patient details were provided on paper. PARTICIPANTS A panel of 10 health professionals from different backgrounds. INTERVENTIONS Not applicable. MAIN OUTCOME MEASURES The performance of the applicable treatments on outcome, impact, comfort, cosmetics, daily effort, and risks and side effects of treatment, as well as the relative importance of criteria in the choice of treatment. RESULTS According to the model, soft-tissue surgery (.413) ranked first as the preferred treatment, followed by orthopedic footwear (.181), ankle-foot orthosis (.147), surface electrostimulation (.137), and finally implanted electrostimulation (.123). Outcome was the most influential consideration affecting treatment choice (.509), followed by risk and side effects (.194), comfort (.104), daily effort (.098), cosmetics (.065), and impact of treatment (.030). CONCLUSIONS Soft-tissue surgery was judged best on outcome, daily effort, comfortable shoe wear, and cosmetically acceptable result and was thereby preferred as a treatment alternative by the panel in this study. In contrast, orthosis and orthopedic footwear are usually preferred in daily practice. The AHP method was found to be suitable methodology for eliciting subjective opinions and quantitatively comparing treatments in the absence of scientific evidence.

A multicriteria decision analysis of augmentative treatment of upper limbs in persons with tetraplegia

This study supported the evaluation by a rehabilitation team of the performance of two treatment options that improve the arm-hand function in subjects with sixth cervical vertebra (C6) level Motor Group 2 tetraplegia. The analytic hierarchy process, a technique for multicriteria decision analysis, was used by a rehabilitation team and potential recipients to quantitatively compare a new technology, Functional Elec trical Stimulation (FES), with conventional surgery. Perform-ance was measured by functional improvement, treatment load, risks, user-friendliness, and social outcomes. Functional improvement after FES was considered better than that after conventional surgery. However, the rehabilitation teams overall rating for conventional surgery was slightly higher than that for FES (57% vs 44%). Compared with the rehabilitation team, potential recipients gave greater weight to burden of treatment and less weight to functional improvement. This study shows that evaluation of new technology must be more comprehensive than the evaluation of functional improvement alone, and that patient preferences may differ from those of the rehabilitation team.

Does technique matter; a pilot study exploring weighting techniques for a multi-criteria decision support framework

BackgroundThere is an increased interest in the use of multi-criteria decision analysis (MCDA) to support regulatory and reimbursement decision making. The EVIDEM framework was developed to provide pragmatic multi-criteria decision support in health care, to estimate the value of healthcare interventions, and to aid in priority-setting. The objectives of this study were to test 1) the influence of different weighting techniques on the overall outcome of an MCDA exercise, 2) the discriminative power in weighting different criteria of such techniques, and 3) whether different techniques result in similar weights in weighting the criteria set proposed by the EVIDEM framework.MethodsA sample of 60 Dutch and Canadian students participated in the study. Each student used an online survey to provide weights for 14 criteria with two different techniques: a five-point rating scale and one of the following techniques selected randomly: ranking, point allocation, pairwise comparison and best worst scaling.ResultsThe results of this study indicate that there is no effect of differences in weights on value estimates at the group level. On an individual level, considerable differences in criteria weights and rank order occur as a result of the weight elicitation method used, and the ability of different techniques to discriminate in criteria importance. Of the five techniques tested, the pair-wise comparison of criteria has the highest ability to discriminate in weights when fourteen criteria are compared.ConclusionsWhen weights are intended to support group decisions, the choice of elicitation technique has negligible impact on criteria weights and the overall value of an innovation. However, when weights are used to support individual decisions, the choice of elicitation technique influences outcome and studies that use dissimilar techniques cannot be easily compared. Weight elicitation through pairwise comparison of criteria is preferred when taking into account its superior ability to discriminate between criteria and respondents’ preferences.

Comparison of Two Multi-Criteria Decision Techniques for Eliciting Treatment Preferences in People with Neurological Disorders

AbstractObjective: To present and compare two multi-criteria decision techniques (analytic hierarchy process [AHP] and conjoint analysis [CA]) for eliciting preferences in patients with cervical spinal cord injury (SCI) who are eligible for surgical augmentation of hand function, either with or without implantation of a neuroprosthesis. The methods were compared in respect to attribute weights, overall preference, and practical experiences. Methods: Two previously designed and administered multi-criteria decision surveys in patients with SCI were compared and further analysed. Attributes and their weights in the AHP experiment were determined by an expert panel, followed by determination of the weights in the patient group. Attributes for the CA were selected and validated using an expert panel, piloted in six patients with SCI and subsequently administered to the same group of patients as participated in the AHP experiment. Results: Both experiments showed the importance of non-outcome-related factors such as inpatient stay and number of surgical procedures. In particular, patients were less concerned with clinical outcomes in actual decision making. Overall preference in both the AHP and CA was in favor of tendon reconstruction (0.6 vs 0.4 for neuroprosthetic implantation). Both methods were easy to apply, but AHP was less easily explained and understood. Conclusions: Both the AHP and CA methods produced similar outcomes, which may have been caused by the obvious preferences of patients. CA may be preferred because of the holistic approach of considering all treatment attributes simultaneously and, hence, its power in simulating real market decisions. On the other hand, the AHP method is preferred as a hands-on, easy-to-implement task with immediate feedback to the respondent. This flexibility allows AHP to be used in shared decision making. However, the way the technique is composed results in many inconsistencies. Patients preferred CA but complained about the number of choice tasks.

Medical devices early assessment methods: systematic literature review.

OBJECTIVES The aim of this study was to get an overview of current theory and practice in early assessments of medical devices, and to identify aims and uses of early assessment methods used in practice. METHODS A systematic literature review was conducted in September 2013, using computerized databases (PubMed, Science Direct, and Scopus), and references list search. Selected articles were categorized based on their type, objective, and main target audience. The methods used in the application studies were extracted and mapped throughout the early stages of development and for their particular aims. RESULTS Of 1,961 articles identified, eighty-three studies passed the inclusion criteria, and thirty were included by searching reference lists. There were thirty-one theoretical papers, and eighty-two application papers included. Most studies investigated potential applications/possible improvement of medical devices, developed early assessment framework or included stakeholder perspective in early development stages. Among multiple qualitative and quantitative methods identified, only few were used more than once. The methods aim to inform strategic considerations (e.g., literature review), economic evaluation (e.g., cost-effectiveness analysis), and clinical effectiveness (e.g., clinical trials). Medical devices were often in the prototype product development stage, and the results were usually aimed at informing manufacturers. CONCLUSIONS This study showed converging aims yet widely diverging methods for early assessment during medical device development. For early assessment to become an integral part of activities in the development of medical devices, methods need to be clarified and standardized, and the aims and value of assessment itself must be demonstrated to the main stakeholders for assuring effective and efficient medical device development.

Why Should Regulators Consider Using Patient Preferences in Benefit-risk Assessment?

There is an increased interest in including the patient perspective in regulatory decision making. At present, regulatory agencies such as the European Medicines Agency (EMA) in the EU and the US FDA use a combination of patient consultation and patient participation in decision committees to ensure that the patient perspective is taken into account in the decision-making process. A relatively new area of interest is the assessment of quantitative patient preferences in larger patient populations. The aim of preference elicitation is to estimate the impact and importance of known adverse effects of a new drug compared with its benefits from a patient perspective, a trade-off that is now implicitly made by the decision committee. The opportunity offered by stated preference methods is to elicit preferences in a large and representative sample of the patient population, thereby improving the reliability and validity of the preferences itself. This paper discusses the potential of patient-based preference assessment of benefits and risks in the approval process for new healthcare technologies.

A preliminary economic evaluation of percutaneous neuromuscular electrical stimulation in the treatment of hemiplegic shoulder pain

Objective. The objective of this study was to compare the cost-effectiveness of various treatment modalities for hemiplegic shoulder pain. Design. A stage II economic evaluation. Main outcome measures. Incremental cost effectiveness ratio of P-NMES, compared to slings and anti-inflammatory injections. Results. The incremental cost effectiveness ratio (ICER) of p-NMES, compared to anti-inflammatory injections is €6,061 (±3,285). The incremental cost of the first quality-adjusted life year after implantation of the P-NMES device compared to anti-inflammatory injections is €33,007 (±5,434). This decreases to ≈ €7,000 after 5 years, and to ≈ €5,000 after 10 survival years. Conclusion. In this early evaluation, P-NMES seems to be cost-effective according to known guidelines. Treatment with P-NMES is recommended for patients with chronic HSP.

The use of multi-criteria decision analysis weight elicitation techniques in patients with mild cognitive impairment: a pilot study.

Objective: To test the applicability of multi-criteria decision analysis preference elicitation techniques in cognitively impaired individuals.Method: A convenience sample of 16 cognitively impaired subjects and 12 healthy controls was asked to participate in a small pilot study. The subjects determined the relative importance of four decision criteria using five different weight elicitation techniques, namely simple multi-attribute rating technique, simple multi-attribute rating technique using swing weights, Kepner-Tregoe weighting, the analytical hierarchical process, and conjoint analysis.Results: Conjoint analysis was judged to be the easiest method for weight elicitation in the control group (Z = 10.00; p = 0.04), while no significant differences in difficulty rating between methods was found in cognitively impaired subjects. Conjoint analysis elicitates weights and rankings significantly different from other methods. Subjectively, cognitively impaired subjects were positive about the use of the weight elicitation techniques. However, it seems the use of swing weights can result in the employment of shortcut strategies.Conclusion: The results of this pilot study suggest that individuals with mild cognitive impairment are willing and able to use multi-criteria elicitation methods to determine criteria weights in a decision context, although no preference for a method was found. The same methodologic and practical issues can be identified in cognitively impaired individuals as in healthy controls and the choice of method is mostly determined by the decision context.