Dawn Stacey

Developing a quality criteria framework for patient decision aids: online international Delphi consensus process

Abstract Objective To develop a set of quality criteria for patient decision support technologies (decision aids). Design and setting Two stage web based Delphi process using online rating process to enable international collaboration. Participants Individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed evidence summaries and rated the importance of 80 criteria in 12 quality domains ona1to9 scale. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones. Main outcome measure Aggregate ratings for each criterion calculated using medians weighted to compensate for different numbers in stakeholder groups; criteria rated between 7 and 9 were retained. Results 212 nominated people were invited to participate. Of those invited, 122 participated in the first round (77 researchers, 21 patients, 10 practitioners, 14 policy makers); 104/122 (85%) participated in the second round. 74 of 83 criteria were retained in the following domains: systematic development process (9/9 criteria); providing information about options (13/13); presenting probabilities (11/13); clarifying and expressing values (3/3); using patient stories (2/5); guiding/coaching (3/5); disclosing conflicts of interest (5/5); providing internet access (6/6); balanced presentation of options (3/3); using plain language (4/6); basing information on up to date evidence (7/7); and establishing effectiveness (8/8). Conclusions Criteria were given the highest ratings where evidence existed, and these were retained. Gaps in research were highlighted. Developers, users, and purchasers of patient decision aids now have a checklist for appraising quality. An instrument for measuring quality of decision aids is being developed.

Assessing the Quality of Decision Support Technologies Using the International Patient Decision Aid Standards instrument (IPDASi)

Objectives To describe the development, validation and inter-rater reliability of an instrument to measure the quality of patient decision support technologies (decision aids). Design Scale development study, involving construct, item and scale development, validation and reliability testing. Setting There has been increasing use of decision support technologies – adjuncts to the discussions clinicians have with patients about difficult decisions. A global interest in developing these interventions exists among both for-profit and not-for-profit organisations. It is therefore essential to have internationally accepted standards to assess the quality of their development, process, content, potential bias and method of field testing and evaluation. Methods Scale development study, involving construct, item and scale development, validation and reliability testing. Participants Twenty-five researcher-members of the International Patient Decision Aid Standards Collaboration worked together to develop the instrument (IPDASi). In the fourth Stage (reliability study), eight raters assessed thirty randomly selected decision support technologies. Results IPDASi measures quality in 10 dimensions, using 47 items, and provides an overall quality score (scaled from 0 to 100) for each intervention. Overall IPDASi scores ranged from 33 to 82 across the decision support technologies sampled (n = 30), enabling discrimination. The inter-rater intraclass correlation for the overall quality score was 0.80. Correlations of dimension scores with the overall score were all positive (0.31 to 0.68). Cronbachs alpha values for the 8 raters ranged from 0.72 to 0.93. Cronbachs alphas based on the dimension means ranged from 0.50 to 0.81, indicating that the dimensions, although well correlated, measure different aspects of decision support technology quality. A short version (19 items) was also developed that had very similar mean scores to IPDASi and high correlation between short score and overall score 0.87 (CI 0.79 to 0.92). Conclusions This work demonstrates that IPDASi has the ability to assess the quality of decision support technologies. The existing IPDASi provides an assessment of the quality of a DSTs components and will be used as a tool to provide formative advice to DSTs developers and summative assessments for those who want to compare their tools against an existing benchmark.

Do Patient Decision Aids Meet Effectiveness Criteria of the International Patient Decision Aid Standards Collaboration? A Systematic Review and Meta-analysis

Objective. To describe the extent to which patient decision aids (PtDAs) meet effectiveness standards of the International Patient Decision Aids Collaboration (IPDAS). Data sources. Five electronic databases (to July 2006) and personal contacts (to December 2006). Results. Among 55 randomized controlled trials, 38 (69%) used at least 1 measure that mapped onto an IPDAS effectiveness criterion. Measures of decision quality were knowledge scores (27 trials), accurate risk perceptions (12 trials), and value congruence with the chosen option (3 trials). PtDAs improved knowledge scores relative to usual care (weighted mean difference [WMD] = 15.2%, 95% confidence interval [CI] = 11.7 to 18.7); detailed PtDAs were somewhat more effective than simpler PtDAs (WMD = 4.6%, 95% CI = 3.0 to 6.2). PtDAs with probabilities improved accurate risk perceptions relative to those without probabilities (relative risk = 1.6, 95% CI = 1.4 to 1.9). Relative to simpler PtDAs, detailed PtDAs improved value congruence with the chosen option. Only 2 of 6 IPDAS decision process criteria were measured: feeling informed (15 trials) and feeling clear about values (13 trials). PtDAs improved these process measures relative to usual care (feeling uninformed WMD = —8.4, 95% CI = —11.9 to —4.8; unclear values WMD = —6.3, 95% CI = —10.0 to —2.7). There was no difference in process measures when detailed and simple PtDAs were compared. Conclusions. PtDAs improve decision quality and the decision processs measures of feeling informed and clear about values; however, the size of the effect varies across studies. Several IPDAS decision process measures have not been used. Future trials need to use a minimum data set of IPDAS evaluation measures. The degree of detail PtDAs require for positive effects on IPDAS criteria should be explored.

Patients’ Perceptions of Sharing in Decisions

AbstractBackground: Shared decision making is the process in which a healthcare choice is made jointly by the health professional and the patient. Little is known about what patients view as effective or ineffective strategies to implement shared decision making in routine clinical practice. Objective: This systematic review evaluates the effectiveness of interventions to improve health professionals’ adoption of shared decision making in routine clinical practice, as seen by patients. Data Sources: We searched electronic databases (PubMed, the Cochrane Library, EMBASE, CINAHL, and PsycINFO) from their inception to mid-March 2009. We found additional material by reviewing the reference lists of the studies found in the databases; systematic reviews of studies on shared decision making; the proceedings of various editions of the International Shared Decision Making Conference; and the transcripts of the Society for Medical Decision Making’s meetings. Study Selection: In our study selection, we included randomized controlled trials, controlled clinical trials, controlled before-and-after studies, and interrupted time series analyses in which patients evaluated interventions to improve health professionals’ adoption of shared decision making. The interventions in question consisted of the distribution of printed educational material; educational meetings; audit and feedback; reminders; and patient-mediated initiatives (e.g. patient decision aids). Study Appraisal: Two reviewers independently screened the studies and extracted data. Statistical analyses considered categorical and continuous process measures. We computed the standardized effect size for each outcome at the 95% confidence interval. The primary outcome of interest was health professionals’ adoption of shared decision making as reported by patients in a self-administered questionnaire. Results: Of the 6764 search results, 21 studies reported 35 relevant comparisons. Overall, the quality of the studies ranged from 0% to 83%. Only three of the 21 studies reported a clinically significant effect for the primary outcome that favored the intervention. The first study compared an educational meeting and a patient-mediated intervention with another patient-mediated intervention (median improvement of 74%). The second compared an educational meeting, a patient-mediated intervention, and audit and feedback with an educational meeting on an alternative topic (improvement of 227%). The third compared an educational meeting and a patient-mediated intervention with usual care (p = 0.003). All three studies were limited to the patient-physician dyad. Limitations: To reduce bias, future studies should improve methods and reporting, and should analyze costs and benefits, including those associated with training of health professionals. Conclusions: Multifaceted interventions that include educating health professionals about sharing decisions with patients and patient-mediated interventions, such as patient decision aids, appear promising for improving health professionals’ adoption of shared decision making in routine clinical practice as seen by patients.

Validating a conceptual model for an inter-professional approach to shared decision making: a mixed methods study

Rationale, aims and objectives Following increased interest in having inter-professional (IP) health care teams engage patients in decision making, we developed a conceptual model for an IP approach to shared decision making (SDM) in primary care. We assessed the validity of the model with stakeholders in Canada. Methods In 15 individual interviews and 7 group interviews with 79 stakeholders, we asked them to: (1) propose changes to the IP-SDM model; (2) identify barriers and facilitators to the models implementation in clinical practice; and (3) assess the model using a theory appraisal questionnaire. We performed a thematic analysis of the transcripts and a descriptive analysis of the questionnaires. Results Stakeholders suggested placing the patient at its centre; extending the concept of family to include significant others; clarifying outcomes; highlighting the concept of time; merging the micro, meso and macro levels in one figure; and recognizing the influence of the environment and emotions. The most common barriers identified were time constraints, insufficient resources and an imbalance of power among health professionals. The most common facilitators were education and training in inter-professionalism and SDM, motivation to achieve an IP approach to SDM, and mutual knowledge and understanding of disciplinary roles. Most stakeholders considered that the concepts and relationships between the concepts were clear and rated the model as logical, testable, having clear schematic representation, and being relevant to inter-professional collaboration, SDM and primary care. Conclusions Stakeholders validated the new IP-SDM model for primary care settings and proposed few modifications. Future research should assess if the model helps implement SDM in IP clinical practice.

Toward Minimum Standards for Certifying Patient Decision Aids A Modified Delphi Consensus Process

Objective. The IPDAS Collaboration has developed a checklist and an instrument (IPDASi v3.0) to assess the quality of patient decision aids (PDAs) in terms of their development process and shared decision-making design components. Certification of PDAs is of growing interest in the US and elsewhere. We report a modified Delphi consensus process to agree on IPDASi (v3.0) items that should be considered as minimum standards for PDA certification, for inclusion in the refined IPDASi (v4.0). Methods. A 2-stage Delphi voting process considered the inclusion of IPDASi (v3.0) items as minimum standards. Item scores and qualitative comments were analyzed, followed by expert group discussion. Results. One hundred and one people voted in round 1; 87 in round 2. Forty-seven items were reduced to 44 items across 3 new categories: 1) qualifying criteria, which are required in order for an intervention to be considered a decision aid (6 items); 2) certification criteria, without which a decision aid is judged to have a high risk of harmful bias (10 items); and 3) quality criteria, believed to strengthen a decision aid but whose omission does not present a high risk of harmful bias (28 items). Conclusions. This study provides preliminary certification criteria for PDAs. Scoring and rating processes need to be tested and finalized. However, the process of appraising the quality of the clinical evidence reported by the PDA should be used to complement these criteria; the proposed standards are designed to rate the quality of the development process and shared decision-making design elements, not the quality of the PDA’s clinical content.

Shared decision making models to inform an interprofessional perspective on decision making: A theory analysis

OBJECTIVE To conduct a theory analysis of shared decision making (SDM) conceptual models and determine the extent to which the models are relevant to interprofessional collaboration in clinical practice. METHODS Theory analysis of SDM models identified from three systematic reviews and personal files. Eligible publications: model of SDM; described concepts with relational statements. Two independently appraised models. RESULTS Of 54 publications, 15 unique models included 18 core concepts. Of two models that included more than one health professional collaborating with the patient, one included 3 of 10 elements of interprofessional collaboration and the other included 1 element. Fourteen were rated as having no logical fallacies, 10 as parsimonious, 7 had been empirically tested, 4 provided testable hypotheses, and 3 described the development process. CONCLUSION Most SDM models failed to encompass an interprofessional approach. Those that included at least two professionals met few of the elements of interprofessional collaboration and had limited description of SDM processes. Although models were rated as logically adequate and parsimonious, only half were tested and few were developed using an explicit process. PRACTICE IMPLICATIONS Appraisal of SDM models highlights the need for a model that is more inclusive of an interprofessional approach.

Establishing the effectiveness of patient decision aids: key constructs and measurement instruments

BackgroundEstablishing the effectiveness of patient decision aids (PtDA) requires evidence that PtDAs improve the quality of the decision-making process and the quality of the choice made, or decision quality. The aim of this paper is to review the theoretical and empirical evidence for PtDA effectiveness and discuss emerging practical and research issues in the measurement of effectiveness.MethodsThis updated overview incorporates: a) an examination of the instruments used to measure five key decision-making process constructs (i.e., recognize decision, feel informed about options and outcomes, feel clear about goals and preferences, discuss goals and preferences with health care provider, and be involved in decisions) and decision quality constructs (i.e., knowledge, realistic expectations, values-choice agreement) within the 86 trials in the Cochrane review; and b) a summary of the 2011 Cochrane Collaboration’s review of PtDAs for these key constructs. Data on the constructs and instruments used were extracted independently by two authors from the 86 trials and any disagreements were resolved by discussion, with adjudication by a third party where required.ResultsThe 86 studies provide considerable evidence that PtDAs improve the decision-making process and decision quality. A majority of the studies (76/86; 88%) measured at least one of the key decision-making process or decision quality constructs. Seventeen different measurement instruments were used to measure decision-making process constructs, but no single instrument covered all five constructs. The Decisional Conflict Scale was most commonly used (n = 47), followed by the Control Preference Scale (n = 9). Many studies reported one or more constructs of decision quality, including knowledge (n = 59), realistic expectation of risks and benefits (n = 21), and values-choice agreement (n = 13). There was considerable variability in how values-choice agreement was defined and determined. No study reported on all key decision-making process and decision quality constructs.ConclusionsEvidence of PtDA effectiveness in improving the quality of the decision-making process and decision quality is strong and growing. There is not, however, consensus or standardization of measurement for either the decision-making process or decision quality. Additional work is needed to develop and evaluate measurement instruments and further explore theoretical issues to advance future research on PtDA effectiveness.

Ten years of the International Patient Decision Aid Standards Collaboration: evolution of the core dimensions for assessing the quality of patient decision aids

In 2003, the International Patient Decision Aid Standards (IPDAS) Collaboration was established to enhance the quality and effectiveness of patient decision aids by establishing an evidence-informed framework for improving their content, development, implementation, and evaluation. Over this 10 year period, the Collaboration has established: a) the background document on 12 core dimensions to inform the original modified Delphi process to establish the IPDAS checklist (74 items); b) the valid and reliable IPDAS instrument (47 items); and c) the IPDAS qualifying (6 items), certifying (6 items + 4 items for screening), and quality criteria (28 items). The objective of this paper is to describe the evolution of the IPDAS Collaboration and discuss the standardized process used to update the background documents on the theoretical rationales, evidence and emerging issues underlying the 12 core dimensions for assessing the quality of patient decision aids.

Decision Coaching to Prepare Patients for Making Health Decisions A Systematic Review of Decision Coaching in Trials of Patient Decision Aids

Background. Decision coaching is individualized, nondirective facilitation of patient preparation for shared decision making. Purpose. To explore characteristics and effectiveness of decision coaching evaluated within trials of patient decision aids (PtDAs) for health decisions. Data Sources. A subanalysis of trials included in the 2011 Cochrane Review of PtDAs. Study Selection. Eligible trials allowed the effectiveness of decision coaching to be compared with another intervention and/or usual care. Data Extraction. Two reviewers independently screened 86 trials, extracted data, and appraised quality. Data Synthesis. Ten trials were eligible. Decision coaching was provided by genetic counselors, nurses, pharmacists, physicians, psychologists, or health educators. Coaching compared with usual care (n = 1 trial) improved knowledge. Coaching plus PtDA compared with usual care (n = 4) improved knowledge and participation in decision making without reported dissatisfaction. Coaching compared with PtDA alone (n = 4) increased values-choice agreement and improved satisfaction with the decision-making process without any difference in knowledge or participation in decision making. Coaching plus PtDA compared with PtDA alone (n = 4) had no difference in knowledge, values-choice agreement, participation in decision making, or satisfaction with the process. Decision coaching plus PtDA was more cost-effective compared with PtDA alone or usual care (n = 1). Limitations. Methodological quality, number of trials, and description of decision coaching. Conclusions. Compared with usual care, decision coaching improved knowledge. However, the improvement in knowledge was similar when coaching was compared with PtDA alone. Outcomes for other comparisons are more variable, some trials showing positive effects and other trials reporting no difference. Given the small number of trials and variability in results, further research is required to determine the effectiveness of decision coaching.