Annette M. O'Connor

Developing a quality criteria framework for patient decision aids: online international Delphi consensus process

Abstract Objective To develop a set of quality criteria for patient decision support technologies (decision aids). Design and setting Two stage web based Delphi process using online rating process to enable international collaboration. Participants Individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed evidence summaries and rated the importance of 80 criteria in 12 quality domains ona1to9 scale. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones. Main outcome measure Aggregate ratings for each criterion calculated using medians weighted to compensate for different numbers in stakeholder groups; criteria rated between 7 and 9 were retained. Results 212 nominated people were invited to participate. Of those invited, 122 participated in the first round (77 researchers, 21 patients, 10 practitioners, 14 policy makers); 104/122 (85%) participated in the second round. 74 of 83 criteria were retained in the following domains: systematic development process (9/9 criteria); providing information about options (13/13); presenting probabilities (11/13); clarifying and expressing values (3/3); using patient stories (2/5); guiding/coaching (3/5); disclosing conflicts of interest (5/5); providing internet access (6/6); balanced presentation of options (3/3); using plain language (4/6); basing information on up to date evidence (7/7); and establishing effectiveness (8/8). Conclusions Criteria were given the highest ratings where evidence existed, and these were retained. Gaps in research were highlighted. Developers, users, and purchasers of patient decision aids now have a checklist for appraising quality. An instrument for measuring quality of decision aids is being developed.

Decision aids for patients facing health treatment or screening decisions: systematic review

Abstract Objective: To conduct a systematic review of randomised trials of patient decision aids in improving decision making and outcomes. Design: We included randomised trials of interventions providing structured, detailed, and specific information on treatment or screening options and outcomes to aid decision making. Two reviewers independently screened and extracted data on several evaluation criteria. Results were pooled by using weighted mean differences and relative risks. Results: 17 studies met the inclusion criteria. Compared with the controls, decision aids produced higher knowledge scores (weighted mean difference=19/100, 95% confidence interval 14 to 25); lower decisional conflict scores (weighted mean difference=−0.3/5, −0.4 to −0.1); more active patient participation in decision making (relative risk = 2.27, 95% confidence interval 1.3 to 4); and no differences in anxiety, satisfaction with decisions (weighted mean difference=0.6/100, −3 to 4), or satisfaction with the decision making process (2/100,−3 to 7). Decision aids had a variable effect on decisions. When complex decision aids were compared with simpler versions, they were better at reducing decisional conflict, improved knowledge marginally, but did not affect satisfaction. Conclusions: Decision aids improve knowledge, reduce decisional conflict, and stimulate patients to be more active in decision making without increasing their anxiety. Decision aids have little effect on satisfaction and a variable effect on decisions. The effects on outcomes of decisions (persistence with choice, quality of life) remain uncertain.

A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation

Although postmenopausal women are advised to consider their values when deliberating about potential benefits and risks of hormone therapy (HRT), feasible, effective methods of decision support in primary care have yet to be established. Using an explicit decision support framework, we developed a self-administered HRT decision aid and evaluated it in a before/after study of 94 women from six family practices. An audiotape guided women through an illustrated booklet including: detailed information about HRT benefits and risks tailored to a womans clinical risk, and a values clarification exercise to promote informed decision making consistent with personal values. After using the decision aid participants: had better general knowledge and more realistic personal expectations of HRT benefits and risks; and, felt more certain, informed, clear about values, and supported in decision making. Womens values elicited in the clarification exercise were 84% accurate in discriminating between decisions. Women with polarized preferences at baseline did not change their minds, but were better informed. Changes in preferences occurred in the uncertain group, with equal numbers accepting or declining HRT. Most participants found the decision aid comprehensible, acceptable in length and pace, and balanced. Decision aids are useful in preparing women for decision making about this complex, personal issue.

Validation of a Decision Regret Scale

Background. As patients become more involved in health care decisions, there may be greater opportunity for decision regret. The authors could not find a validated, reliable tool for measuring regret after health care decisions. Methods. A5- item scale was administered to 4 patient groups making different health care decisions. Convergent validity was deter- mined by examining the scales correlation with satisfaction measures, decisional conflict, and health outcome measures. Results. The scale showed good internal consistency (Cronbachs = 0.81 to 0.92). It correlated strongly with decision satisfaction (r = -0.40 to -0.60), decisional conflict (r = 0.31 to 0.52), and overall rated quality of life (r = -0.25 to - 0.27). Groups differing on feelings about a decision also differed on rated regret: F(2, 190) = 31.1, P < 0.001. Regret was greater among those who changed their decisions than those who did not, t(175) = 16.11, P < 0.001. Conclusions. The scale is a useful indicator of health care decision regret at a given point in time.

Assessing the Quality of Decision Support Technologies Using the International Patient Decision Aid Standards instrument (IPDASi)

Objectives To describe the development, validation and inter-rater reliability of an instrument to measure the quality of patient decision support technologies (decision aids). Design Scale development study, involving construct, item and scale development, validation and reliability testing. Setting There has been increasing use of decision support technologies – adjuncts to the discussions clinicians have with patients about difficult decisions. A global interest in developing these interventions exists among both for-profit and not-for-profit organisations. It is therefore essential to have internationally accepted standards to assess the quality of their development, process, content, potential bias and method of field testing and evaluation. Methods Scale development study, involving construct, item and scale development, validation and reliability testing. Participants Twenty-five researcher-members of the International Patient Decision Aid Standards Collaboration worked together to develop the instrument (IPDASi). In the fourth Stage (reliability study), eight raters assessed thirty randomly selected decision support technologies. Results IPDASi measures quality in 10 dimensions, using 47 items, and provides an overall quality score (scaled from 0 to 100) for each intervention. Overall IPDASi scores ranged from 33 to 82 across the decision support technologies sampled (n = 30), enabling discrimination. The inter-rater intraclass correlation for the overall quality score was 0.80. Correlations of dimension scores with the overall score were all positive (0.31 to 0.68). Cronbachs alpha values for the 8 raters ranged from 0.72 to 0.93. Cronbachs alphas based on the dimension means ranged from 0.50 to 0.81, indicating that the dimensions, although well correlated, measure different aspects of decision support technology quality. A short version (19 items) was also developed that had very similar mean scores to IPDASi and high correlation between short score and overall score 0.87 (CI 0.79 to 0.92). Conclusions This work demonstrates that IPDASi has the ability to assess the quality of decision support technologies. The existing IPDASi provides an assessment of the quality of a DSTs components and will be used as a tool to provide formative advice to DSTs developers and summative assessments for those who want to compare their tools against an existing benchmark.

Do Patient Decision Aids Meet Effectiveness Criteria of the International Patient Decision Aid Standards Collaboration? A Systematic Review and Meta-analysis

Objective. To describe the extent to which patient decision aids (PtDAs) meet effectiveness standards of the International Patient Decision Aids Collaboration (IPDAS). Data sources. Five electronic databases (to July 2006) and personal contacts (to December 2006). Results. Among 55 randomized controlled trials, 38 (69%) used at least 1 measure that mapped onto an IPDAS effectiveness criterion. Measures of decision quality were knowledge scores (27 trials), accurate risk perceptions (12 trials), and value congruence with the chosen option (3 trials). PtDAs improved knowledge scores relative to usual care (weighted mean difference [WMD] = 15.2%, 95% confidence interval [CI] = 11.7 to 18.7); detailed PtDAs were somewhat more effective than simpler PtDAs (WMD = 4.6%, 95% CI = 3.0 to 6.2). PtDAs with probabilities improved accurate risk perceptions relative to those without probabilities (relative risk = 1.6, 95% CI = 1.4 to 1.9). Relative to simpler PtDAs, detailed PtDAs improved value congruence with the chosen option. Only 2 of 6 IPDAS decision process criteria were measured: feeling informed (15 trials) and feeling clear about values (13 trials). PtDAs improved these process measures relative to usual care (feeling uninformed WMD = —8.4, 95% CI = —11.9 to —4.8; unclear values WMD = —6.3, 95% CI = —10.0 to —2.7). There was no difference in process measures when detailed and simple PtDAs were compared. Conclusions. PtDAs improve decision quality and the decision processs measures of feeling informed and clear about values; however, the size of the effect varies across studies. Several IPDAS decision process measures have not been used. Future trials need to use a minimum data set of IPDAS evaluation measures. The degree of detail PtDAs require for positive effects on IPDAS criteria should be explored.

Randomized Trial of a Portable, Self-administered Decision Aid for Postmenopausal Women Considering Long-term Preventive Hormone Therapy

Although practice guidelines suggest that postmenopausal women learn about the benefits and nsks and consider their values when deciding about hormone therapy, the optimal decision-support method has not been established. In a randomized controlled trial, the authors compared the efficacy of a general educational pamphlet with that of a tailored decision aid. The pamphlet briefly summarized benefits, risks, and likely beneficiaries in general terms. The decision aid, delivered via booklet and audiotape, provided: detailed benefits and risks using functional terms and probabilities tailored to clinical risk; and steps for considering the issue in a womans own situation, including a value-clarification exercise. Compared with the pamphlet group, the decision-aid group had statistically significant (p < 0.05) improvements in terms of realistic personal expectations of the benefits and nsks, decisional conflict, and perceived acceptability of the intervention. Levels of general knowledge about the main benefits and nsks were comparable for the two interventions It is concluded that tailored decision aids prepare women for decision making better than do general pamphlets. Key words: decision making; choice behavior; informed consent; decision-support techniques; woman education; woman participation; health education; consumer satisfaction; hormone replacement therapy; menopause. (Med Decis Making 1998;18:295-303)

Toward Minimum Standards for Certifying Patient Decision Aids A Modified Delphi Consensus Process

Objective. The IPDAS Collaboration has developed a checklist and an instrument (IPDASi v3.0) to assess the quality of patient decision aids (PDAs) in terms of their development process and shared decision-making design components. Certification of PDAs is of growing interest in the US and elsewhere. We report a modified Delphi consensus process to agree on IPDASi (v3.0) items that should be considered as minimum standards for PDA certification, for inclusion in the refined IPDASi (v4.0). Methods. A 2-stage Delphi voting process considered the inclusion of IPDASi (v3.0) items as minimum standards. Item scores and qualitative comments were analyzed, followed by expert group discussion. Results. One hundred and one people voted in round 1; 87 in round 2. Forty-seven items were reduced to 44 items across 3 new categories: 1) qualifying criteria, which are required in order for an intervention to be considered a decision aid (6 items); 2) certification criteria, without which a decision aid is judged to have a high risk of harmful bias (10 items); and 3) quality criteria, believed to strengthen a decision aid but whose omission does not present a high risk of harmful bias (28 items). Conclusions. This study provides preliminary certification criteria for PDAs. Scoring and rating processes need to be tested and finalized. However, the process of appraising the quality of the clinical evidence reported by the PDA should be used to complement these criteria; the proposed standards are designed to rate the quality of the development process and shared decision-making design elements, not the quality of the PDA’s clinical content.

A survey of the decision-making needs of Canadians faced with complex health decisions.

Objective To describe the decision‐making needs of Canadians when faced with ‘complex’ health decisions characterized by balancing advantages against disadvantages. Although a national report emphasized that public confidence in the health‐care system depends on support for personal knowledge and decision‐making, there has been no systematic investigation of the Canadian populations decision‐making needs.

Decisional Conflict in Patients and Their Physicians: A Dyadic Approach to Shared Decision Making:

Background. Decisional conflict is defined as personal uncertainty about which course of action to take when choice among competing options involves risk, regret, or challenge to personal life values. It is influenced by inadequate knowledge, unclear values, inadequate support, and the perception that an ineffective decision has been made. Until recently, it has been studied at the individual level, which ignores the interpersonal system between patients and physicians. Objective. To explore the effect of feeling uninformed, unclear values, inadequate support, and the perception that an ineffective decision has been made on one owns outcome (actor effect) and on the other persons outcome (partner effect). Methods. After a clinical encounter, modifiable deficits and personal uncertainty were measured in physicians and patients using the Decisional Conflict Scale. Structural equation modeling was used to measure the parameters of the Actor-Partner Interdependence Model. Results. A total of 112 dyads of physicians and patients were included in the analysis. For both patients and physicians, 2 actor effects, unclear values (P < 0:0001) and the perception that an ineffective decision has been made (P < 0:0001), were found to be positively correlated with personal uncertainty. One partner effect, feeling uninformed (P=0:03), was found to be negatively correlated with personal uncertainty. Conclusions. Personal uncertainty of patients and physicians is influenced not only by their respective deficits but also by the deficits of the other member of the dyad. Our results indicate that the more unclear the expression of their own values and the more they perceive that an ineffective choice had been made, the more both physicians and patients experience personal uncertainty. They also indicate that the less uninformed they feel, the more both physicians and patients experience personal uncertainty.