Steven H. Woolf

Potential benefits, limitations, and harms of clinical guidelines

This is the first in a series of four articles on issues in the development and use of clinical guidelines Over the past decade, clinical guidelines have increasingly become a familiar part of clinical practice. Every day, clinical decisions at the bedside, rules of operation at hospitals and clinics, and health spending by governments and insurers are being influenced by guidelines. As defined by the Institute of Medicine, clinical guidelines are “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”1 They may offer concise instructions on which diagnostic or screening tests to order, how to provide medical or surgical services, how long patients should stay in hospital, or other details of clinical practice The broad interest in clinical guidelines that is stretching across Europe, North America, Australia, New Zealand, and Africa (box) has its origin in issues that most healthcare systems face: rising healthcare costs, fueled by increased demand for care, more expensive technologies, and an ageing population; variations in service delivery among providers, hospitals, and geographical regions and the presumption that at least some of this variation stems from inappropriate care, either overuse or underuse of services; and the intrinsic desire of healthcare professionals to offer, and of patients to receive, the best care possible. Clinicians, policy makers, and payers see guidelines as a tool for making care more consistent and efficient and for closing the gap between what clinicians do and what scientific evidence supports. As guidelines diffuse into medicine, there are important lessons to learn from the firsthand experience of those who develop, evaluate, and use them.3 This article, the first of a four part series to reflect on these lessons, examines the potential benefits, limitations, and harms of clinical guidelines. Future articles will review lessons learned …

Current methods of the U.S. Preventive Services Task Force: A review of the process

The U.S. Preventive Services Task Force (USPSTF/Task Force) represents one of several efforts to take a more evidence-based approach to the development of clinical practice guidelines. As methods have matured for assembling and reviewing evidence and for translating evidence into guidelines, so too have the methods of the USPSTF. This paper summarizes the current methods of the third USPSTF, supported by the Agency for Healthcare Research and Quality (AHRQ) and two of the AHRQ Evidence-based Practice Centers (EPCs). The Task Force limits the topics it reviews to those conditions that cause a large burden of suffering to society and that also have available a potentially effective preventive service. It focuses its reviews on the questions and evidence most critical to making a recommendation. It uses analytic frameworks to specify the linkages and key questions connecting the preventive service with health outcomes. These linkages, together with explicit inclusion criteria, guide the literature searches for admissible evidence. Once assembled, admissible evidence is reviewed at three strata: (1) the individual study, (2) the body of evidence concerning a single linkage in the analytic framework, and (3) the body of evidence concerning the entire preventive service. For each stratum, the Task Force uses explicit criteria as general guidelines to assign one of three grades of evidence: good, fair, or poor. Good or fair quality evidence for the entire preventive service must include studies of sufficient design and quality to provide an unbroken chain of evidence-supported linkages, generalizable to the general primary care population, that connect the preventive service with health outcomes. Poor evidence contains a formidable break in the evidence chain such that the connection between the preventive service and health outcomes is uncertain. For services supported by overall good or fair evidence, the Task Force uses outcomes tables to help categorize the magnitude of benefits, harms, and net benefit from implementation of the preventive service into one of four categories: substantial, moderate, small, or zero/negative. The Task Force uses its assessment of the evidence and magnitude of net benefit to make a recommendation, coded as a letter: from A (strongly recommended) to D (recommend against). It gives an I recommendation in situations in which the evidence is insufficient to determine net benefit. The third Task Force and the EPCs will continue to examine a variety of methodologic issues and document work group progress in future communications.Abstract Abstract: The U.S. Preventive Services Task Force (USPSTF/Task Force) represents one of several efforts to take a more evidence-based approach to the development of clinical practice guidelines. As methods have matured for assembling and reviewing evidence and for translating evidence into guidelines, so too have the methods of the USPSTF. This paper summarizes the current methods of the third USPSTF, supported by the Agency for Healthcare Research and Quality (AHRQ) and two of the AHRQ Evidence-based Practice Centers (EPCs). mThe Task Force limits the topics it reviews to those conditions that cause a large burden of suffering to society and that also have available a potentially effective preventive service. It focuses its reviews on the questions and evidence most critical to making a recommendation. It uses analytic frameworks to specify the linkages and key questions connecting the preventive service with health outcomes. These linkages, together with explicit inclusion criteria, guide the literature searches for admissible evidence. mOnce assembled, admissible evidence is reviewed at three strata: (1) the individual study, (2) the body of evidence concerning a single linkage in the analytic framework, and (3) the body of evidence concerning the entire preventive service. For each stratum, the Task Force uses explicit criteria as general guidelines to assign one of three grades of evidence: good, fair, or poor. Good or fair quality evidence for the entire preventive service must include studies of sufficient design and quality to provide an unbroken chain of evidence-supported linkages, generalizable to the general primary care population, that connect the preventive service with health outcomes. Poor evidence contains a formidable break in the evidence chain such that the connection between the preventive service and health outcomes is uncertain. For services supported by overall good or fair evidence, the Task Force uses outcomes tables to help categorize the magnitude of benefits, harms, and net benefit from implementation of the preventive service into one of four categories: substantial, moderate, small, or zero/negative. mThe Task Force uses its assessment of the evidence and magnitude of net benefit to make a recommendation, coded as a letter: from A (strongly recommended) to D (recommend against). It gives an I recommendation in situations in which the evidence is insufficient to determine net benefit. mThe third Task Force and the EPCs will continue to examine a variety of methodologic issues and document work group progress in future communications.

Clinical guidelines: Developing guidelines

The methods of guideline development should ensure that treating patients according to the guidelines will achieve the outcomes that are desired. This article presents a combination of the literature about guideline development and the results of our combined experience in guideline development in North America and Britain. It considers the five steps in the initial development of an evidence based guideline. The dissemination, implementation, and evaluation of practice guidelines will be discussed in the final article in this series.

Pediatric Vesicoureteral Reflux Guidelines Panel Summary Report on the Management of Primary Vesicoureteral Reflux in Children

PURPOSE The American Urological Association convened the Pediatric Vesicoureteral Reflux Guidelines Panel to analyze the literature regarding available methods for treating vesicoureteral reflux diagnosed following a urinary tract infection in children and to make practice policy recommendations based on the treatment outcomes data insofar as the data permit. MATERIALS AND METHODS The panel searched the MEDLINE data base for all articles from 1965 to 1994 on vesicoureteral reflux and systematically analyzed outcomes data for 7 treatment alternatives: 1) intermittent antibiotic therapy, 2) bladder training, 3) continuous antibiotic prophylaxis, 4) antibiotic prophylaxis and bladder training, 5) antibiotic prophylaxis, anticholinergics and bladder training, 6) open surgical repair and 7) endoscopic repair. Key outcomes identified were probability of reflux resolution, likelihood of developing pyelonephritis and scarring, and possibility of complications of medical and surgical treatment. RESULTS Available outcomes data on the various treatment alternatives were summarized in tabular form and graphically, and the relative probabilities of possible outcomes were compared for each alternative. Treatment recommendations were based on scientific evidence and expert opinion. The panel concluded that only a few recommendations can be derived purely from scientific evidence of a beneficial effect on health outcomes. CONCLUSIONS For most children the panel recommended continuous antibiotic prophylaxis as initial treatment. Surgery was recommended for children with persistent reflux and other indications, as specified in the document.

Informed decision making: what is its role in cancer screening?

Interest in informed decision making (IDM) has grown in recent years. Greater patient involvement in decision making is consistent with recommendations to improve health care quality. This report provides an overview of IDM; clarifies the differences between IDM, shared decision making (SDM), and informed consent; and reviews the evidence to date about IDM for cancer screening. The authors also make recommendations for research. We define IDM as occurring when an individual understands the disease or condition being addressed and comprehends what the clinical service involves, including its benefits, risks, limitations, alternatives, and uncertainties; has considered his or her preferences and makes a decision consistent with them; and believes he or she has participated in decision making at the level desired. IDM interventions are used to facilitate informed decisions. The authors reviewed the evidence to date for IDM and cancer screening based primarily on published meta‐analyses and a recent report for the Centers for Disease Control and Preventions Guide to Community Preventive Services. IDM and SDM interventions, such as decision aids, result in improved knowledge, beliefs, risk perceptions, and combinations of these. Little or no evidence exists, however, regarding whether these interventions result in 1) participation in decision making at a level consistent with patient preferences or 2) effects on patient satisfaction with the decision‐making process. These variables generally either were not assessed or were not reported in the articles reviewed. Results of interventions on uptake of screening were variable. After exposure to IDM/SDM interventions, most studies showed small decreases in prostate cancer screening, whereas four studies on breast and colorectal cancer screening showed small increases. Few data are available by which to evaluate current practices in cancer screening IDM. Patient participation in IDM should be facilitated for those who prefer it. More research is needed to assess the benefits of IDM/SDM interventions and to tailor interventions to individuals who are most likely to desire and benefit from them. There are many system barriers to IDM/SDM and few tools. More work is needed in this area as well. In addition, research is needed to learn how to incorporate IDM into ongoing clinical practice and to determine whether there are unintended negative consequences of IDM. Cancer 2004.

Assessing the clinical effectiveness of preventive maneuvers: analytic principles and systematic methods in reviewing evidence and developing clinical practice recommendations. A report by the Canadian Task Force on the Periodic Health Examination.

This paper examines a process for evaluating clinical effectiveness and developing recommendations in which systematic methods are used to review evidence from published clinical research and to reach sound conclusions about appropriate medical policy. The methodology addresses four important components of the analytic process: (1) the criteria that must be satisfied for a clinical practice to be considered effective; (2) proper methods for reviewing evidence from published clinical research to determine whether a clinical practice meets these criteria (including methods for performing comprehensive literature reviews, for judging the quality of individual studies, and for synthesizing or pooling the results of multiple studies); (3) theoretical and practical concerns in translating the results of the scientific review into sound clinical practice recommendations; and (4) the importance of documentation, guidelines, and other safeguards to minimize the effect the reviewers themselves have on the objectivity and consistency of the analytic methods.

Patient Safety Is Not Enough: Targeting Quality Improvements To Optimize the Health of the Population

Patient safety is elemental to the Hippocratic oath. Nothing is more contrary to the ethos of medicine than harming individuals who search for care and compassion. Safety and medical errors have gripped public attention ever since the release of the Institute of Medicine report To Err Is Human in 1999 (1). That report, which generated the stunning headline that 44 000 to 98 000 Americans die each year because of medical errors, galvanized concern and prompted a groundswell of safety programs, research, and coalitions. The intensity of the patient safety movement calls attention to the potential downside of focusing too much on a single disease or health theme: Concentrating on 1 niche of medicine can ultimately compromise the overall health of the population if it comes at the expense of other problems that are more threatening to health. A balanced approach to improving health would transcend theme areas, such as patient safety, and examine the big picture. If failure to provide preventive care claims 100 times as many lives as do lapses in safety, then a system that is preoccupied with safety and does little about preventive care will see more of its patients die. Too much of a good thing in quality improvement paradoxically threatens health by giving inadequate attention to more important needs. For example, the U.S. Agency for Healthcare Research and Quality, the principal funding source for research on the uninsured, racial disparities, and other aspects of quality, allocates 60% of its grants to patient safety. Funding for all other categories of health services research, now described as nonpatient safety grants, was cut by 15% to finance this investment (2). As of April 2003, Congress was considering 20 bills on patient safety (3), but only minimal progress was expected on improving care for disadvantaged individuals (4, 5). The Notion of Proportionality The issue is not whether patient safety should receive attention, as it surely must, but whether that attention is proportionate. Unless attention and resources are allocated to safety and other quality improvement areas in proportion to their relative effect on public health, an excess of the population may die or sustain morbidity. For every patient harmed by lapses in patient safety, more will experience or die of deficient health care and flawed delivery systems, which are problems that a perfect safety record will not take away. People are less likely to die of an overdose of warfarin (a lapse in safety) than of not receiving warfarin at all (6). The attention that policymakers give to safety should be coupled with a proportionately larger effort to deal with defects in health care that affect more lives. Before this discussion proceeds further, the central premise for this concernthat there is only so much attention to go aroundmust be defended. Some might argue that it is unnecessary for patient safety to hinder other forms of quality improvement and that one has little to do with the other. But, in fact, resources for quality improvement are often fixed. Providers, practices, and health systems can give only so much time, human energy, and money to making health care better, and they cannot fix all problems at once. This means that choices must be made, with resources applied to one quality arena not going to another. Systems preoccupied with safety may find little energy remaining to recognize, let alone rectify, deeper design flaws that compromise caring. As task forces convene to work on patient safety, are other aspects of quality left unattended? At the national level, is the focus on patient safety coming at the expense of efforts to help the uninsured (7) or reduce disparities in the care of minorities (8)? Patient Safety in the Big Picture Imagine 4 concentric circles representing all deficiencies in the quality of health care (Figure). At the center is the problem of patient safety, the most extreme violation of quality, in which the attempt to heal has the perverse effect of causing harm. The center blurs into the larger circle of medical errors, which connotes a larger category of mistakes that endanger patients in domains other than safety, such as mistakes that create deficiencies in managing chronic disease (9). That circle of errors, in turn, blurs into the larger circle of gaps in quality, encompassing not only individual errors but also the broader organizational causes of inadequate care, including restricted access to care (7), racial and ethnic disparities (8), systemic quality defects introduced by insurance and management policies, faulty information systems, and flawed system designs. This circle encompasses underuse, overuse, and misuse of care (10). Some envision a fourth circle: the extent to which the technical elements of care seem good on the basis of performance indicators but ultimately fail to be caring because of deficiencies not captured by these measures (11, 12). The rudeness or insensitivity that patients encounter or the frustrations they experience in obtaining information and control over treatment decisions illustrate gaps in quality, of deep concern to the public (13), that often are not measured under normative standards. Figure. Organizational framework for deficiencies in health care service Center. Second circle. center Third circle. first second circles Fourth circle. center second third circles Implicit in this model is the proposition that patient safety is a subset of everything else. Quite the opposite has been suggested of late, with patient safety being used as shorthand for medical errors (or worse, all deficiencies in care) (14-17). But safety, defined as freedom from injury (1), connotes a certain kind of injury brought on by clinical intervention, such as a drug overdose or a surgical mishap, in which patients find themselves with a new problem that did not exist before the clinical encounter. This additive risk differs from errors of inadequate treatment for problems brought to the clinical encounter, in which injury occurs by default. If a patient arrives with a glycohemoglobin level of 10% and the provider does not act, the provider does not give the patient a new problem (making the visit unsafe) but causes harm by not reducing an existing risk. Putting Health Priorities in Epidemiologic Perspective These errors in addressing extant risks arguably are more threatening to health than lapses in safety. Although To Err Is Human (1) suggested that 44 000 to 98 000 Americans die each year because of medical errors, more careful analyses suggest that only a fraction of these deaths, perhaps fewer than 5%, are causally linked to errors (18-20). Only a subset of adverse drug events, the first concern of the patient safety movement, causes serious harm (21). A cohort study of Medicare beneficiaries noted 5 life-threatening or fatal preventable adverse drug events for every 1000 person-years of observation (22). By comparison, modeling studies estimate that hundreds of thousands of Americans die because of inadequate treatment of cardiovascular disease, cancer, and other conditions (23, 24). Failure to address extant risks in just 1 niche of medicine, the use of -blockers after acute myocardial infarction, claims 4300 to 17 000 lives each year (23, 25). The numbers grow larger when deaths due to omissions in other cardiac drug regimens and in managing other diseases (for example, stroke, cancer, and diabetes) are added (26). The National Committee for Quality Assurance recently estimated that 57 000 deaths occur annually because of shortcomings in delivering recommended care (27). Many more deathspotentially more than 700 000 per year (23)result from gaps in screening, immunizations, and reducing risk factors (for example, obesity and unhealthy diet) in the population (28-30). Tobacco use alone accounts for more than 400 000 deaths per year (31). This matters because getting the balance wrong can cost lives. If poor control of blood pressure or serum lipid levels accounts for more deaths than do illegible drug prescriptions (21, 23, 32), a quality improvement program that is preoccupied with computerized prescription entry but ignores the large proportion of patients with uncontrolled hypertension or hyperlipidemia costs more lives than a program with reverse priorities. This contention relies to some extent on a false dichotomy, because common etiologies often cause lapses in both safety and quality and similar solutions apply. The same reminder system that corrects drug errors can also recognize overdue mammography (33). Work conditions that cause confusion induce mistakes, whether it is reaching for the wrong vial (threatening safety) or overlooking a needed treatment (not reducing existing risks). Understanding what causes individuals to commit errors can therefore bear fruit in both domains. Although safety initiatives can thereby identify system solutions that kill 2 birds with 1 stone, they may overlook system defects unrelated to safety that are more threatening to health. Designing Systems for Much More than Safety Ominous flaws in the structure of the health care system severely compromise the health of Americans (34). The system delivers only 55% of recommended care (35). Patients cannot access providers, services are not covered, and features of the delivery system impede efforts to obtain what patients need precisely when they need it. System solutions to improve access, reduce inequities, and provide the infrastructure for health promotion and chronic disease management (36, 37) may do little for safety but are urgent public health priorities that could lower morbidity and mortality on a scale unattainable by safety interventions (10). The urgency of these redesigns is apparent to those who take a global perspective and measure progress in terms of overall population health. The urgency may be less palpable to those preoccupied with health nichessuch as patient safety, asthma, or

Patient Reports of Preventable Problems and Harms in Primary Health Care

BACKGROUND Despite recent attention given to medical errors, little is known about the kinds and importance of medical errors in primary care. The principal aims of this study were to develop patient-focused typologies of medical errors and harms in primary care settings and to discern which medical errors and harms seem to be the most important. METHODS Thirty-eight in-depth anonymous interviews of adults from rural, suburban, and urban locales in Virginia and Ohio were conducted to solicit stories of preventable problems with primary health care that led to physical or psychological harm. Transcriptions were analyzed to identify, name, and organize the stories of errors and harms. RESULTS The 38 narratives described 221 problematic incidents that predominantly involved breakdowns in the clinician-patient relationship (n = 82, 37%) and access to clinicians (n = 63, 29%). There were several reports of perceived racism. The incidents were linked to 170 reported harms, 70% of which were psychological, including anger, frustration, belittlement, and loss of relationship and trust in one’s clinician. Physical harms accounted for 23% of the total and included pain, bruising, worsening medical condition, and adverse drug reactions. DISCUSSION The errors reported by interviewed patients suggest that breakdowns in access to and relationships with clinicians may be more prominent medical errors than are technical errors in diagnosis and treatment. Patients were more likely to report being harmed psychologically and emotionally, suggesting that the current preoccupation of the patient safety movement with adverse drug events and surgical mishaps could overlook other patient priorities.

When should clinical guidelines be updated

Considerable resources are being expended internationally on the development of clinical practice guidelines.1 Although consensus is increasing about methods for developing evidence based guidelines,2 less attention has been paid to the process for assessing when guidelines should be updated. The most common advice is for guidelines to include a scheduled review date. This could result in wasted resources, however, if a full update is undertaken prematurely in a slowly evolving field, or in guidelines in a rapidly evolving field becoming out of date before the scheduled review. Some guidelines state that they should be updated when new information becomes available. It is unclear, however, how this should be done, and we are unaware of any systematic attempts to devise a method. In this paper we propose a set of principles and a pragmatic model for assessing whether guidelines need to be updated. #### Summary points Changes in evidence, the values placed on evidence, the resources available for health care, and improvements in current performance are all possible reasons for updating clinical guidelines The need for an efficient mechanism for identifying when guidelines require updating is urgent A possible model for assessing validity of guidelines is based on a combination of multidisciplinary expert opinion and limited literature searches ### Changes in evidence on the existing benefits and harms of interventions New information about the magnitude of benefits and harms may make the pre-existing guideline invalid. The surgical risk of carotid endarterectomy, for example, has fallen substantially over the past 30 years, altering the risk-benefit ratio in favour of performing the operation for selected patients with symptomatic, high grade carotid stenosis.3–5 ### Changes in outcomes considered important New evidence may identify as important outcomes that were previously unappreciated or wholly unrecognised. Quality of life, for example, an end point often not considered in earlier research and guidelines, is receiving increasing …

A String of Mistakes: The Importance of Cascade Analysis in Describing, Counting, and Preventing Medical Errors

BACKGROUND Notions about the most common errors in medicine currently rest on conjecture and weak epidemiologic evidence. We sought to determine whether cascade analysis is of value in clarifying the epidemiology and causes of errors and whether physician reports are sensitive to the impact of errors on patients. METHODS Eighteen US family physicians participating in a 6-country international study filed 75 anonymous error reports. The narratives were examined to identify the chain of events and the predominant proximal errors. We tabulated the consequences to patients, both reported by physicians and inferred by investigators. RESULTS A chain of errors was documented in 77% of incidents. Although 83% of the errors that ultimately occurred were mistakes in treatment or diagnosis, 2 of 3 were set in motion by errors in communication. Fully 80% of the errors that initiated cascades involved informational or personal miscommunication. Examples of informational miscommunication included communication breakdowns among colleagues and with patients (44%), misinformation in the medical record (21%), mishandling of patients’ requests and messages (18%), inaccessible medical records (12%), and inadequate reminder systems (5%). When asked whether the patient was harmed, physicians answered affirmatively in 43% of cases in which their narratives described harms. Psychological and emotional effects accounted for 17% of physician-reported consequences but 69% of investigator-inferred consequences. CONCLUSIONS Cascade analysis of physicians’ error reports is helpful in understanding the precipitant chain of events, but physicians provide incomplete information about how patients are affected. Miscommunication appears to play an important role in propagating diagnostic and treatment mistakes.