Margaret Holmes-Rovner

Developing a quality criteria framework for patient decision aids: online international Delphi consensus process

Abstract Objective To develop a set of quality criteria for patient decision support technologies (decision aids). Design and setting Two stage web based Delphi process using online rating process to enable international collaboration. Participants Individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed evidence summaries and rated the importance of 80 criteria in 12 quality domains ona1to9 scale. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones. Main outcome measure Aggregate ratings for each criterion calculated using medians weighted to compensate for different numbers in stakeholder groups; criteria rated between 7 and 9 were retained. Results 212 nominated people were invited to participate. Of those invited, 122 participated in the first round (77 researchers, 21 patients, 10 practitioners, 14 policy makers); 104/122 (85%) participated in the second round. 74 of 83 criteria were retained in the following domains: systematic development process (9/9 criteria); providing information about options (13/13); presenting probabilities (11/13); clarifying and expressing values (3/3); using patient stories (2/5); guiding/coaching (3/5); disclosing conflicts of interest (5/5); providing internet access (6/6); balanced presentation of options (3/3); using plain language (4/6); basing information on up to date evidence (7/7); and establishing effectiveness (8/8). Conclusions Criteria were given the highest ratings where evidence existed, and these were retained. Gaps in research were highlighted. Developers, users, and purchasers of patient decision aids now have a checklist for appraising quality. An instrument for measuring quality of decision aids is being developed.

Decision aids for patients facing health treatment or screening decisions: systematic review

Abstract Objective: To conduct a systematic review of randomised trials of patient decision aids in improving decision making and outcomes. Design: We included randomised trials of interventions providing structured, detailed, and specific information on treatment or screening options and outcomes to aid decision making. Two reviewers independently screened and extracted data on several evaluation criteria. Results were pooled by using weighted mean differences and relative risks. Results: 17 studies met the inclusion criteria. Compared with the controls, decision aids produced higher knowledge scores (weighted mean difference=19/100, 95% confidence interval 14 to 25); lower decisional conflict scores (weighted mean difference=−0.3/5, −0.4 to −0.1); more active patient participation in decision making (relative risk = 2.27, 95% confidence interval 1.3 to 4); and no differences in anxiety, satisfaction with decisions (weighted mean difference=0.6/100, −3 to 4), or satisfaction with the decision making process (2/100,−3 to 7). Decision aids had a variable effect on decisions. When complex decision aids were compared with simpler versions, they were better at reducing decisional conflict, improved knowledge marginally, but did not affect satisfaction. Conclusions: Decision aids improve knowledge, reduce decisional conflict, and stimulate patients to be more active in decision making without increasing their anxiety. Decision aids have little effect on satisfaction and a variable effect on decisions. The effects on outcomes of decisions (persistence with choice, quality of life) remain uncertain.

Do Patient Decision Aids Meet Effectiveness Criteria of the International Patient Decision Aid Standards Collaboration? A Systematic Review and Meta-analysis

Objective. To describe the extent to which patient decision aids (PtDAs) meet effectiveness standards of the International Patient Decision Aids Collaboration (IPDAS). Data sources. Five electronic databases (to July 2006) and personal contacts (to December 2006). Results. Among 55 randomized controlled trials, 38 (69%) used at least 1 measure that mapped onto an IPDAS effectiveness criterion. Measures of decision quality were knowledge scores (27 trials), accurate risk perceptions (12 trials), and value congruence with the chosen option (3 trials). PtDAs improved knowledge scores relative to usual care (weighted mean difference [WMD] = 15.2%, 95% confidence interval [CI] = 11.7 to 18.7); detailed PtDAs were somewhat more effective than simpler PtDAs (WMD = 4.6%, 95% CI = 3.0 to 6.2). PtDAs with probabilities improved accurate risk perceptions relative to those without probabilities (relative risk = 1.6, 95% CI = 1.4 to 1.9). Relative to simpler PtDAs, detailed PtDAs improved value congruence with the chosen option. Only 2 of 6 IPDAS decision process criteria were measured: feeling informed (15 trials) and feeling clear about values (13 trials). PtDAs improved these process measures relative to usual care (feeling uninformed WMD = —8.4, 95% CI = —11.9 to —4.8; unclear values WMD = —6.3, 95% CI = —10.0 to —2.7). There was no difference in process measures when detailed and simple PtDAs were compared. Conclusions. PtDAs improve decision quality and the decision processs measures of feeling informed and clear about values; however, the size of the effect varies across studies. Several IPDAS decision process measures have not been used. Future trials need to use a minimum data set of IPDAS evaluation measures. The degree of detail PtDAs require for positive effects on IPDAS criteria should be explored.

Implementing shared decision‐making in routine practice: barriers and opportunities

Determine feasibility of shared decision‐making programmes in fee‐for‐service hospital systems including physicians’ offices and in‐patient facilities.

An Educational Intervention as Decision Support for Menopausal Women

The purpose of this study was to develop and test a decision support intervention (DSI) to assist women to make and act on informed decisions that are consistent with their values in the area of menopause and hormone replacement therapy (HRT). Mode and intensity of intervention were tested in midlife women (N = 248), randomly assigned to one of three intervention formats: written information only, guided discussion, or personalized decision exercise. Data were collected over 12 months. Knowledge, decisional conflict, satisfaction with health care provider, and self-efficacy improved following intervention and were maintained for 12 months for all groups. Womens adherence to their own plans over 12 months was 59% (exercise), 76% (calcium intake), and 89% (HRT). Carefully written information is effective in promoting knowledge, adherence, and satisfaction among well-educated, interested women. It was concluded that women can understand complex information, including tradeoffs regarding treatment options. Women will adhere to their own plans, suggesting that consumer rather than provider plans may be the more appropriate gold standard for measuring adherence.

Patient education materials about the treatment of early-stage prostate cancer: a critical review.

Prostate cancer is the most prevalent noncutaneous malignant condition and, after lung cancer, the second leading cause of cancer deaths among U.S. men (1). Localized prostate cancer is typically treated with watchful waiting, radical prostatectomy, radiation therapy (external and brachytherapy), and, more rarely, hormone therapy. Men receiving a diagnosis of localized prostate cancer face a difficult treatment decision because most randomized, controlled trials are still in progress. However, a recent Swedish randomized, controlled trial found that while radical prostatectomy reduced mortality from prostate cancer more than did watchful waiting, men who had surgery lived no longer than those who were treated with watchful waiting (2). Because specialists disagree and the literature is inconclusive, patients are increasingly expected to participate in treatment decisions. Patients must learn the risks and benefits of each treatment and choose which risks they can tolerate and which benefits they prefer. Decision aids have become the evidence-based medicine information source for patients who face choices that 1) have clinically important differences in the balance between outcomes and complication rates, 2) require tradeoffs between near-term and long-term outcomes, 3) include treatments that may result in a grave outcome, or 4) have only marginal differences in outcomes. Decision aids are one form of patient education materials. A recent Cochrane review of randomized trials of decision aids showed that compared with usual care, decision aids improve patient knowledge and realistic expectations and reduce decisional conflict. Exposure to decision aids also seemed to increase patient participation in decisions and improve decision quality (3). We sought to determine the adequacy of publicly available patient education materials to offer these identified advantages to patients. More particularly, we first evaluated whether the needed information was presented (content review) and then evaluated how well the information was presented (quality review). Methods Development of Review Criteria We developed the criteria for evaluating the content of patient education materials by first examining the empirical and theoretical literature on informed consent and decision aids. For a primary-level analysis, we adopted the Cochrane definition of balanced presentation: 1) Are all options presented (including, if appropriate, watchful waiting)? and 2) Are potential harms as well as potential benefits presented (4)? We used the Cochrane criteria to eliminate all patient education materials that 1) did not discuss the 4 standard prostate cancer treatments (watchful waiting, radical prostatectomy, radiation therapy, and hormone therapy), 2) discussed only cancer in general, or 3) discussed only prostate cancer screening. In a second-level detailed analysis, we applied previously developed generic criteria specific to prostate cancer (5). Additional items to reflect patient needs were obtained through a literature review and through review of our criteria from a local prostate cancer support group and from prostate cancer experts (urologists, oncologists, nurses, researchers, and activists) from the Michigan Cancer Consortium Prostate Cancer Action Committee. Four of the authors independently developed common definitions and coded test documents. An initial list of 85 criteria was reduced to 54 criteria by consensus and was operationalized (Tables 1, 2, and 3). A weighting system was considered but was not implemented because no gold standard could be identified from the literature on decision aids or risk communication. Similarly, the Cochrane criteria provide an empirically validated set of criteria for the second-level detailed analysis. Table 1. Frequency of Clinical Condition Content Table 2. Frequency of Treatment Option Content Table 3. Frequency of Quantitative Elements and Other Factors In previous work, we have shown that decision aids can produce decisions that better reflect patient preferences (6, 7). The parallel criteria selected for prostate cancer are the categories of treatment outcomes and side effects. Traditional patient education elements of description of the anatomy and disease progression were also included. Identification of Patient Education Materials Print Materials, Videotapes, and CD-ROMs To identify these materials (identified between September and December 2001), we asked national organizations (including patient advocacy groups, government organizations, pharmaceutical companies, insurance companies, health maintenance organizations, universities, and comprehensive cancer centers) for their materials. To be included, all materials must be widely available to the public at no cost. Internet Search Strategy We first reviewed Web sites of prominent organizations (including all of those identified during the print material, videotape, and CD-ROM search). Second, we reviewed Web sites of pharmaceutical companies that had received approval from the U.S. Food and Drug Administration to produce prostate cancer drugs. Third, a naive patient with prostate cancer strategy used an open (broad-based) search strategy with Google and Yahoo, which located more than 300 000 Web sites. We reviewed at least the first 100 links provided and discontinued our search when relevance decreased. We did not evaluate Web sites composed of links to other Web sites or duplicated print materials previously reviewed (for example, the Web site of the National Cancer Institute). The initial review set of 546 materials was reduced by 502 materials after applying the Cochrane criteria for all standard treatment options to be presented (see the Appendix Table, for list of print materials not eligible for review). The remaining 44 materials (19 print materials, 19 Web sites, 4 videotapes, and 2 CD-ROMs) underwent a formal content and quality review (Table 4). Table 4. Full Review Documents Appendix Table. Print Materials Rejected for Formal Analysis Content Review Three coding teams composed of 2 coders were each responsible for reviewing one third of the materials. Thus, each material was independently scored by 2 coders. Criteria were rated as present if they were mentioned in the text, even if briefly (8). Paired ratings were compared, and differences were resolved by a second review and, in rare occasions, by a third reviewer. Percentage agreement achieved by the teams ranged from 91% to 94%. Quality Review The quality review evaluated 1) the accuracy of the information contained in the patient education material, 2) whether presentation of treatment options was balanced, and 3) whether the information was comprehensible to the average reader. An experienced clinician with previous prostate cancer research experience performed the accuracy and balance review. The clinician did not participate in the content review and was blind to content scores. Accuracy criteria were developed from the prostate cancer literature on the natural history of the disease and the efficacy and side effects of treatments. The estimates of balance were subjective, graded from minimal to marked bias toward any particular treatment method. To further evaluate the best materials identified through the simple content inclusion criteria, a health literacy expert performed an extensive plain-language review on the top 5 print materials and top 5 Web sites. To determine these top-rated materials, we developed a scoring system that identified how many of the 54 essential criteria each piece of patient education material contained but did not prioritize further. Those materials that contained inaccurate information or were strongly biased toward a particular treatment were excluded from consideration. Only 1 of the print materials was eliminated from the top rating because of presence of bias. The review assessed characteristics of text and design that affect reading ease and comprehension, incorporating the widely used Suitability Assessment of Materials system (9). Criteria include 1) readability; 2) amount and organization of content; 3) writing style as it affects literacy demands; 4) graphics, layout, and typography; 5) evidence of learning stimulation; and 6) cultural appropriateness. Each criterion was evaluated according to specific subcharacteristics rated on a 0- to 2-point scale (0 = unsuitable; 1 = adequate; 2 = superior; or not applicable). Final scores were calculated as percentages based on a denominator of 44 possible points. The 0- to 2-point scale was then translated into grade level. Web sites were evaluated by using similar criteria. Updating the Materials Several Web sites changed during the 4-month review period. The review described later in this paper reflects assessment of Web sites between 1 September 2001 and 14 December 2001. In April 2003, we rescored the top 5 and bottom 5 Web sites and brochures to determine whether they had changed substantially since our review. Role of the Funding Source The funding source was not involved in the design, conduct, or reporting of the study or in the decision to submit the manuscript for publication. Results Tables 1, 2, and 3 show the proportion of criteria (in consolidated form) scored as present in the 44 patient education materials. Although videotape and the CD-ROM materials are reported in Tables 1, 2, and 3, discussion in the paper is restricted to print materials and Web sites because these are the tools most available to public audiences. We found some differences while rescoring the top 5 and bottom 5 print materials and Web sites, but they were small and did not change the rankings of the materials. Content Review Disease Process Most patient education materials included basic information on prostate anatomy and physiology (95% and 80% of print materials and Web sites, respectively). Most print materials and Web sites also discussed prostate cancer staging (100% and 95%, respectively)

Evidence-based patient choice: A prostate cancer decision aid in plain language

BackgroundDecision aids (DA) to assist patients in evaluating treatment options and sharing in decision making have proliferated in recent years. Most require high literacy and do not use plain language principles. We describe one of the first attempts to design a decision aid using principles from reading research and document design. The plain language DA prototype addressed treatment decisions for localized prostate cancer. Evaluation assessed impact on knowledge, decisions, and discussions with doctors in men newly diagnosed with prostate cancer.MethodsDocument development steps included preparing an evidence-based DA in standard medical parlance, iteratively translating it to emphasize shared decision making and plain language in three formats (booklet, Internet, and audio-tape). Scientific review of medical content was integrated with expert health literacy review of document structure and design. Formative evaluation methods included focus groups (n = 4) and survey of a new sample of men newly diagnosed with prostate cancer (n = 60), compared with historical controls (n = 184).ResultsA transparent description of the development process and design elements is reported. Formative evaluation among newly diagnosed prostate cancer patients found the DA to be clear and useful in reaching a decision. Newly diagnosed patients reported more discussions with doctors about treatment options, and showed increases in knowledge of side effects of radiation therapy.ConclusionThe plain language DA presenting medical evidence in text and numerical formats appears acceptable and useful in decision-making about localized prostate cancer treatment. Further testing should evaluate the impact of all three media on decisions made and quality of life in the survivorship period, especially among very low literacy men.

Addressing health literacy in patient decision aids.

BackgroundEffective use of a patient decision aid (PtDA) can be affected by the user’s health literacy and the PtDA’s characteristics. Systematic reviews of the relevant literature can guide PtDA developers to attend to the health literacy needs of patients. The reviews reported here aimed to assess:1. a) the effects of health literacy / numeracy on selected decision-making outcomes, and b) the effects of interventions designed to mitigate the influence of lower health literacy on decision-making outcomes, and2. the extent to which existing PtDAs a) account for health literacy, and b) are tested in lower health literacy populations.MethodsWe reviewed literature for evidence relevant to these two aims. When high-quality systematic reviews existed, we summarized their evidence. When reviews were unavailable, we conducted our own systematic reviews.ResultsAim 1: In an existing systematic review of PtDA trials, lower health literacy was associated with lower patient health knowledge (14 of 16 eligible studies). Fourteen studies reported practical design strategies to improve knowledge for lower health literacy patients. In our own systematic review, no studies reported on values clarity per se, but in 2 lower health literacy was related to higher decisional uncertainty and regret. Lower health literacy was associated with less desire for involvement in 3 studies, less question-asking in 2, and less patient-centered communication in 4 studies; its effects on other measures of patient involvement were mixed. Only one study assessed the effects of a health literacy intervention on outcomes; it showed that using video to improve the salience of health states reduced decisional uncertainty. Aim 2: In our review of 97 trials, only 3 PtDAs overtly addressed the needs of lower health literacy users. In 90% of trials, user health literacy and readability of the PtDA were not reported. However, increases in knowledge and informed choice were reported in those studies in which health literacy needs were addressed.ConclusionLower health literacy affects key decision-making outcomes, but few existing PtDAs have addressed the needs of lower health literacy users. The specific effects of PtDAs designed to mitigate the influence of low health literacy are unknown. More attention to the needs of patients with lower health literacy is indicated, to ensure that PtDAs are appropriate for lower as well as higher health literacy patients.

Cost-effectiveness analysis in heart disease, part III: Ischemia, congestive heart failure, and arrhythmias

Cost-effectiveness analyses were reviewed in the following diagnostic and treatment categories: acute myocardial infarction (MI) and diagnostic strategies for coronary artery disease (CAD), coronary artery bypass graft (CABG) surgery, percutaneous transluminal coronary angioplasty (PTCA), congestive heart failure (CHF), and arrhythmias. In the case of acute MI, coronary care units, as presently used, are rather expensive but could be made much more efficient with more effective triage and resource utilization; reperfusion via thrombolysis is cost-effective, as are beta-blockers and angiotensin-converting enzyme (ACE) inhibitors post-MI in appropriate patients. Cost-effectiveness of CAD screening tests depends strongly on the prevalence of disease in the population studied. Cost-effectiveness of CABG surgery depends on targeting; eg, it is highly effective for such conditions as left-main and three-vessel disease but not for lesser disease. PTCA appears to be cost-effective in situations where there is clinical consensus for its use, eg, severe ischemia and one-vessel disease, but requires further analysis based on randomized data; coronary stents also appear to be cost-effective. In preliminary analysis, ACE inhibition for CHF dominates, ie, saves both money and lives. Cardiac transplant appears to be cost-effective but requires further study. For arrhythmias, implantable cardioverter defibrillators are cost-effective, especially the transvenous device, in life-threatening situations; radiofrequency ablation is also cost-effective in patients with Wolff-Parkinson-White syndrome apart from asymptomatic individuals; and pacemakers have not been analyzed except in the case of biofascicular block, where results were variable depending on the situation and preceding tests.

PATIENT DECISION SUPPORT INTERVENTION : INCREASED CONSISTENCY WITH DECISION ANALYTIC MODELS

BACKGROUND Patient Decision Support (PDS) tools assist patients in using medical evidence to make choices consistent that are with their values and in using evidence about consequences of medical alternatives. OBJECTIVE To evaluate a PDS intervention for perimenopausal hormone replacement therapy. We assessed the impact of the PDS on (1) consistency between the decision to take estrogen replacement therapy (ERT) or progesterone/estrogen replacement therapy (PERT) and the expected utility of treatment and (2) likelihood to take ERT and PERT pre- and postintervention. DESIGN Content of the PDS was standardized. Randomized trial of three intensities of intervention: (1) brochure; (2) lecture/discussion; and (3) active decision support. SUBJECTS Participants were perimenopausal community volunteers between the ages of 40 and 65 (n = 248). MEASURES (1) Consistent with values (correlation between expected utility (EU) and likelihood of taking hormones); and (2) Likelihood to take hormone replacement therapy. RESULTS (1) The brochure group was less consistent with the decision analytic model than the lecture/discussion and active decision support groups. (2) Influence on decisions: PDS tools increased the number of women certain about whether or not to take hormones. There were no differences among experimental groups. Of 99 women uncertain about ERT pre-PDS, 65% changed. Twenty-one (32%) decided against ERT and 44 (68%) decided for ERT. (3) More intensive interventions produced modest gains in a normative direction. CONCLUSIONS PDSs using any of 3 formats reduce uncertainty and assist women to make informed decisions. Increased consistency with decision analytic models appears to be driven by better estimates of likelihood of outcomes.