Albert G. Mulley

Developing a quality criteria framework for patient decision aids: online international Delphi consensus process

Abstract Objective To develop a set of quality criteria for patient decision support technologies (decision aids). Design and setting Two stage web based Delphi process using online rating process to enable international collaboration. Participants Individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed evidence summaries and rated the importance of 80 criteria in 12 quality domains ona1to9 scale. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones. Main outcome measure Aggregate ratings for each criterion calculated using medians weighted to compensate for different numbers in stakeholder groups; criteria rated between 7 and 9 were retained. Results 212 nominated people were invited to participate. Of those invited, 122 participated in the first round (77 researchers, 21 patients, 10 practitioners, 14 policy makers); 104/122 (85%) participated in the second round. 74 of 83 criteria were retained in the following domains: systematic development process (9/9 criteria); providing information about options (13/13); presenting probabilities (11/13); clarifying and expressing values (3/3); using patient stories (2/5); guiding/coaching (3/5); disclosing conflicts of interest (5/5); providing internet access (6/6); balanced presentation of options (3/3); using plain language (4/6); basing information on up to date evidence (7/7); and establishing effectiveness (8/8). Conclusions Criteria were given the highest ratings where evidence existed, and these were retained. Gaps in research were highlighted. Developers, users, and purchasers of patient decision aids now have a checklist for appraising quality. An instrument for measuring quality of decision aids is being developed.

Patient reactions to a program designed to facilitate patient participation in treatment decisions for benign prostatic hyperplasia.

Patients often want considerable information about their conditions, and enhanced patient participation might reduce unwanted practice variation and improve medical decisions. The authors assessed how men with benign prostatic hyperplasia reacted to an educational program designed to facilitate participation in decisionmaking, and how strongly ratings of their symptom state and the prospect of complications predicted their treatment choice. A prospective cohort study was conducted in three hospital-based urology practices: two in prepaid group practices, and one Veterans Administration clinic. Four hundred twenty-one men with symptomatic benign prostatic hyperplasia without prior prostatectomy or benign prostatic hyperplasia complications were enrolled, and 373 provided usable ratings. Subjects participated in an interactive videodisc-based shared decisionmaking program about benign prostatic hyperplasia and its treatment options, prostatectomy, and “watchful waiting.” They rated the length, clarity, balance, and value of the program and were followed for 3 months to determine if they underwent surgery. Patients rated the program as generally clear, informative, and balanced. Across all three sites, 77% of patients were very positive and 16% were generally positive about the programs usefulness in making a treatment decision. Logistic models predicting choice of surgical treatment documented the independent importance of negative ratings of the current symptom state (odds ratio 7.0, 95% confidence interval 2.9–16.6), as well as the prospect of postoperative sexual dysfunction (odds ratio 0.20, 95% confidence interval 0.08–0.48) in decisionmaking. Patients rated the Shared Decisionmaking Program very positively and made decisions consistent with their assessed preferences. These results suggest that patients can be helped to participate in treatment decisions, and support a randomized trial of the Shared Decisionmaking Program.

Medical Intensive Care: Indications, Interventions, and Outcomes

To evaluate current practices regarding intensive-care units (ICUs), we collected data on 2693 consecutive admissions to a medical ICU during a two-year period and studied indications for admission, specific interventions, costs, and outcomes. The need for noninvasive monitoring rather than immediate major interventions prompted 77 per cent of the admissions. Only 10 per cent of monitored patients had subsequent indications for major interventions. The 23 per cent who required immediate interventions accounted for disproportionate shares of total charges (37 per cent) and deaths during hospitalization (58 per cent). Demographic and diagnostic data indicate that the aged and chronically ill have become the principal consumers of intensive care. Overall mortality during hospitalization was 10 per cent; cumulative mortality during follow-up study (mean duration, 15 months) was 25 per cent. We conclude that identification of sensitive predictors of complications and specific predictors of mortality can lead to more efficient and effective ICU practices.

Prognosis, survival, and the expenditure of hospital resources for patients in an intensive-care unit.

To define more precisely the factors determining the allocation of resources to critically ill patients, we asked physicians to estimate at the time of admission the short-term prognosis of patients who accounted for 1831 admissions to a medical intensive-care and coronary-care unit. We then examined the relations between this prognosis, the actual outcome, and the resource expenditure during a single hospitalization. We found that the care of nonsurvivors involved a significantly higher mean expenditure than did the care of survivors (P less than 0.01). Among nonsurvivors, expenditure positively correlated with the probability of survival estimated at the time of admission (P less than 0.001). Among survivors, expenditure negatively correlated with the probability of survival (P less than 0.001). Among both nonsurvivors and survivors, total expenditure and expenditure per day were greatest for patients whose outcome were most unexpected. We conclude that prognostic uncertainty is important in determining resource expenditures for the critically ill. This factor warrants greater consideration in future studies of expenditure for the care of catastrophically ill patients.

Developing shared decision-making programs to improve the quality of health care.

We strongly believe in the importance of patient involvement in a medical decision. The interactive SDPs appear to be an effective way to facilitate this involvement. One key to the acceptance of these programs by patients and physicians is that they be--and be perceived as--fair, accurate, and balanced. Herein we have described the well-defined protocol for developing, evaluating, and updating SDPs. The first of the foundations programs dealing with benign prostatic hyperplasia has been well received by patients and clinicians and has been demonstrated to have an impact on practice patterns. Efforts are under way to evaluate four additional programs, leading to widespread availability of the first five SDPs by fall of 1992.

Rationing Intensive Care — Physician Responses to a Resource Shortage

To determine how physicians ration limited critical resources, we studied the allocation of intensive-care-unit (ICU) beds during a shortage caused by a lack of nurses. As the bed capacity of the medical ICU decreased from 18 to 8, the percentage of days on which one or more beds were available decreased from 95 to 55 per cent, and monthly admissions decreased from 122 to 95. Physicians responded by restricting ICU admissions to acutely ill patients and reducing the proportion of patients admitted primarily for monitoring. Among patients admitted because of chest pain, the proportion actually sustaining a myocardial infarction increased linearly with the restriction in bed capacity. Although more patients with myocardial infarction were admitted to non-intensive-care areas, there was no increase in mortality. In addition, physicians transferred patients out of the ICU sooner. There was no apparent withdrawal of care from dying patients. Our results suggest that physicians can respond to moderate resource limitations by more efficient use of intensive-care resources.

Indications for Use of Hepatitis B Vaccine, Based on Cost-Effectiveness Analysis

To formulate indications for the use of hepatitis B vaccine, we examined the cost effectiveness of three strategies: vaccinating everyone; screening everyone and vaccinating those without evidence of immunity; and neither vaccinating nor screening, but passively immunizing those with known exposure. Estimates of the hepatitis attack rate, prevalence of immunity, and frequency of known exposure were made for three representative populations: homosexual men, surgical residents, and the general population of the United States. Screening followed by vaccination of homosexual men and vaccination without prior screening of surgical residents would result in savings of medical costs. Neither screening nor vaccination is the lowest-cost strategy for the general population. Vaccination of susceptible persons will save medical costs for populations with annual attack rates above 5 per cent. Vaccination may be considered cost effective (or cost saving when indirect costs are included) for populations with attack rates as low as 1 to 2 per cent.

Involving patients in clinical decisions: Impact of an interactive video program on use of back surgery

Background.Back surgery rates are rapidly rising in the United States. This surgery is usually elective, so patient preferences are important in the treatment decision. Objectives.The objective of this study was to determine the impact on outcomes and surgical choices of an interactive, diagnosis-specific videodisk program for informing patients about treatment choices. Research Design.This was a randomized, controlled trial at 2 sites comparing the interactive video plus a booklet with the booklet alone. Subjects.Elective surgery candidates (n = 393) included 171 patients with herniated disks, 110 with spinal stenosis, and 112 with other diagnoses. Measures.Mailed questionnaires were used to assess outcomes and satisfaction; surgery rates were determined by questionnaires and automated records. Results.Symptom and functional outcomes at 3 months and 1 year were similar between study groups. The overall surgery rate was 22% lower in the videodisk group (26% versus 33%, P = 0.08). Among patients with herniated disks, those in the video group underwent significantly less surgery (32% versus 47%, P = 0.05 by Kaplan-Meier test). Among patients with spinal stenosis, surgery rates in the video group were higher (39% for the video group, 29% for the booklet group;P = 0.4). There was little effect on patient satisfaction, but patients in the video group felt better informed. Conclusions.The program appears to facilitate decision making and may help to ensure informed consent. For patients with herniated disks, it reduced the surgery rate without diminishing patient outcomes. Its impact on costs of care depends on the proportion of patients with various diagnoses and on local surgery rates.

CLINICAL GUIDELINES: PART II: Early Detection of Prostate Cancer: Part II: Estimating the Risks, Benefits, and Costs

In this paper, we synthesize the evidence on the epidemiology of prostate cancer and the effectiveness of screening tests that were presented in part I of this paper [1]. We also examine evidence of treatment effectiveness to estimate the cost-effectiveness of one-time screening for prostate cancer with digital rectal examination and measurement of prostate-specific antigen (PSA). Before presenting estimates of cost-effectiveness, we review the most widely used strategies for treating clinically localized prostate cancer. A more detailed discussion of treatment issues is reported elsewhere [2]. Strategies for Treating Clinically Localized Prostate Cancer Expectant Management or Watchful Waiting The strategy of expectant management includes several approaches that range from reserving palliative interventions for patients who develop symptomatic local progression or metastatic disease to withholding potentially curative therapy until signs of disease progression are seen [3]. Expectant management is currently the most favored strategy outside of the United States [4]. Data on the long-term outcomes of this strategy are limited ([5-12]; Johansson JE. Natural history in early primary untreated prostate cancer [Presented paper]. 85th Annual Meeting of the American Urological Association; 1993; San Antonio, Texas), and almost no data are available on the distant prognosis of tumors that are discovered by PSA measurement. In a structured review of the literature [13], disease-specific mortality and metastatic rates that were seen with expectant management did not differ significantly from those seen in published, uncontrolled surgical series. The rates associated with expectant management were actually lower than those reported for radiotherapy. However, a larger proportion of patients who were treated with radiotherapy had poorly differentiated tumors, reflecting important differences in patient selection. If disparities in the extent of pathologic staging are taken into account, patients who are treated by nonsurgical methods almost certainly have more advanced disease than can be appreciated on clinical grounds. Such analyses highlight the hazards of nonrandomized treatment comparisons [13-15]. Adolfsson and colleagues [16] identified seven studies of men who had palpable, clinically localized cancer that had been treated since 1980 and who had received expectant management. The 10-year disease-specific survival rate (which was measured in only two of the studies) was lower for expectant management (84%) than for radical prostatectomy (93%) but was lower still for radiotherapy (74%). The prospective study by Johansson and colleagues of 223 men (mean age, 72 years) in Sweden is one of few published studies that is population based [11]. Persons who had clinically localized disease were enrolled between 1977 and 1984. After a mean follow-up of 12.5 years, mortality related to prostate cancer was 10%, whereas 66% of the group died from other causes. Ten-year metastasis-free survival corrected for deaths from other causes was 83%. This study has been criticized for enrolling too many older men and men who had incidental tumors diagnosed at transurethral resection of the prostate [17]. However, similar results were reported for a subset of men who were potential candidates for radical prostatectomy [11, 18]. More recent 15-year population-based data from Connecticut support the findings of earlier analyses of expectant management [19]. A patient-level meta-analysis of 762 men (mean age, 70 years) who were enrolled in six studies of expectant management and showed the following 10-year prostate cancer-specific survival rates by histologic grade: 87% for a well-differentiated tumor, 87% for a moderately differentiated tumor, and 34% for a poorly differentiated tumor [20]. In contrast, a more recent Scandinavian series [21] suggested poorer outcomes with expectant management. However, this study used a retrospective design, unlike the prospective cohort approach of Johansson and colleagues [11] and the retrospective cohort design of Albertsen and coworkers [19]. In addition, cause of death was not determined using blinded reviewers in the study by Aus [21] as it was in the study by Albertsen and coworkers [19]. Nonetheless, for 108 men who had clinically localized tumors that were well-differentiated or moderately differentiated, a 10-year crude survival rate of 78% was reported; 40% of all deaths in this group were caused by prostate cancer. Men who receive expectant management are more likely than men who receive radical surgery to have local progression of prostate cancer. However, the clinical significance of local progression and its effect on quality of life have not been well studied [22]. Of the patients who were followed by Johansson (Johansson JE. Natural history in early primary untreated prostate cancer [Presented paper]. 85th Annual Meeting of the American Urological Association; 1993; San Antonio, Texas), 22% had cancer that progressed to clinical stage T3 on digital rectal examination after 10 years. However, only six cases were associated with local problems that were considered substantial. Radiation Therapy The effectiveness of radiation therapy has not been established in controlled trials [13]. A randomized trial of 97 men who had stage A2 or B cancer that was treated with radiation or surgery had a significantly longer time to recurrence after surgery but showed no difference in mortality rate [23, 24]. This analysis was from a less desirable treatment-given perspective rather than an intention-to-treat perspective. Few published observational studies of radiation therapy have stratified outcome by grade and stage [13]. Recent cohort studies of patients receiving radiation therapy for clinically localized cancer indicate that at 10 years, overall survival rates are no different for these patients than for age-matched controls [25-28]. In a recent structured review [13], radiation therapy was associated with higher median rates of metastasis and cancer-specific mortality than was surgery or expectant management. However, such comparative analyses are confounded by selection and staging bias because patients treated with radiation therapy tend to be more likely to have poorly differentiated tumors and occult nodal involvement. Potential complications of radiation therapy include death (0.2% to 0.5%) and various gastrointestinal and genitourinary illnesses, including incontinence and sexual dysfunction, that may become chronic [13]. Radical Prostatectomy Substantial improvements in the technique of radical retropubic prostatectomy have been reported [24]. Studies suggest that men who are found to have pathologically organ-confined disease after radical prostatectomy have a low risk for recurrence and essentially normal life expectancy. However, a recent analysis of data from national Medicare claims [29] indicates that approximately 25% of men who are reported to have pathologically localized disease at the time of radical prostatectomy require additional treatment within 5 years. Radical prostatectomy seems to reduce the probability of developing future problems with local disease even in patients who are found to have extracapsular disease during surgery [30]. The benefit that can be attributed to radical prostatectomy remains uncertain. Excellent prognoses in radical prostatectomy series done at single institutions [31] may be an effect of better case selection [32]. Outcomes from surgical series reported in 1981 to 1993 do not differ statistically from outcomes reported for expectant management [13]; however, differences in patient characteristics make such comparisons hazardous. One randomized trial [33] directly compared expectant management with radical prostatectomy. No difference in survival was seen after 15 years of follow-up. However, because this study had only 111 patients, its power to detect clinically important differences in outcome was limited [34]. The risks of radical prostatectomy include perioperative death, early cardiovascular complications, and chronic impairments of urinary and sexual function [3, 13]. Risk may be greater for patients who are treated in community settings. In a survey of a national probability sample of 1070 Medicare-eligible men who had had radical prostatectomy between 1988 and 1990, Fowler and associates [35] found that 30% were wearing pads or other devices to manage urinary incontinence 2 to 4 years after surgery. Sixty-one percent reported having no or only partial erections since surgery (> 90% said that they had been potent before surgery). However, Medicare claims from one state documented only a 0.7% risk for death 30 days after radical prostatectomy [36]. Litwin and colleagues [37] recently documented that patients who have had radical prostatectomy are more likely to develop impairment in urinary and sexual function than are patients who have received radiation therapy or expectant management. On the other hand, Fowler and coworkers [38] found that almost all of the men in their survey would have radical prostatectomy again, despite the complications. Methods Estimating the Benefits and Harms of Early Detection Decision models of prostate cancer screening or treatment have yielded conflicting results [39-47]. The absence of definitive data for many important variables, particularly the prognosis of clinically localized cancer that is not treated and the effectiveness of aggressive treatment for such cancer, can result in models that support a wide array of screening policies. Using available data that have already been reviewed, we estimated the risks and maximum benefits of one-time testing using digital rectal examination and PSA measurement for men who are older than 50 years of age. We believe that the available data are insufficient to model the current American Cancer Society recommendation of an annual digital rectal examination and PSA mea

Assessing Patients' Utilities: Can the Ends Justify the Means?

Each of the elements of a utility assessment strategy—defining and describing health states of interest, identifying subjects, choosing a scaling task, aggregating across subjects, determining reliability and validity—is controversial. The controversy is in part explained by the interdisciplinary nature of the problem; different disciplinary conceptualizations of utility lead to different priorities for methodologic problem solving. Controversy is further explained by widely divergent potential applications of utility assessments, including individual decisions made with and without (or by) an agent, and decisions made for populations that may be homogeneous or heterogeneous with regard to utilities for the same health states. Issues can be clarified by focusing on the purpose of the utility assessment and, in the case of clinical decision making, on the most relevant disease-specific outcomes. The prostatectomy decision is an example. Although questions of measurement validity need continuing attention, more attention should be paid to validating uses of utility assessments: Can utility assessments distinguish prospectively, among patients who subsequently experience the same health outcome, those for whom it is associated with a high or low level of well-being? Can utility assessments be used to predict behavior? Can a decision process that includes utility assessments affect decisions in a manner that improves overall well-being? Approaches to such questions are complicated by changes in utilities over time, departures from the normative model of decision making, the effects of decision-making responsibility, and biases introduced by the decision-making process.