Ann G. Lawthers

Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I.

BACKGROUND As part of an interdisciplinary study of medical injury and malpractice litigation, we estimated the incidence of adverse events, defined as injuries caused by medical management, and of the subgroup of such injuries that resulted from negligent or substandard care. METHODS We reviewed 30,121 randomly selected records from 51 randomly selected acute care, nonpsychiatric hospitals in New York State in 1984. We then developed population estimates of injuries and computed rates according to the age and sex of the patients as well as the specialties of the physicians. RESULTS Adverse events occurred in 3.7 percent of the hospitalizations (95 percent confidence interval, 3.2 to 4.2), and 27.6 percent of the adverse events were due to negligence (95 percent confidence interval, 22.5 to 32.6). Although 70.5 percent of the adverse events gave rise to disability lasting less than six months, 2.6 percent caused permanently disabling injuries and 13.6 percent led to death. The percentage of adverse events attributable to negligence increased in the categories of more severe injuries (Wald test chi 2 = 21.04, P less than 0.0001). Using weighted totals, we estimated that among the 2,671,863 patients discharged from New York hospitals in 1984 there were 98,609 adverse events and 27,179 adverse events involving negligence. Rates of adverse events rose with age (P less than 0.0001). The percentage of adverse events due to negligence was markedly higher among the elderly (P less than 0.01). There were significant differences in rates of adverse events among categories of clinical specialties (P less than 0.0001), but no differences in the percentage due to negligence. CONCLUSIONS There is a substantial amount of injury to patients from medical management, and many injuries are the result of substandard care.

Preventing medical injury.

Although adverse events (AEs) are not uncommon in hospitalized patients, they are by no means inevitable. A review of records from a population-based study in New York revealed that nearly 4% of hospitalized patients suffered AEs. Two-thirds of those events were considered to be caused by errors in management, most of which were not because of negligence. A large part of the reason preventable AEs occur is that todays medical care is extremely complex, involving a variety of personnel, equipment, and procedures. By seeking to eliminate errors in the system of providing care and raising the awareness of health care providers about the potential for AEs, programs can be established to address and possibly remedy this serious problem.

Identifying Adverse Events Caused by Medical Care: Degree of Physician Agreement in a Retrospective Chart Review

Retrospective case review has long been a main-stay of peer review. It supports scientific studies and medical audits as well as assessments of the appropriateness, effectiveness, and quality of health care provided by physicians, hospitals, or regions. As part of quality assurance, hospitals and clinics regularly use formal and informal case review. Insurers and managed care organizations rely on case review when making decisions about coverage. All forms of case review depend heavily on expert opinion. Case review also underlies current and proposed systems of compensating patients for injuries caused by medical care. Under the current litigation system, the patient must prove, with the support of expert medical opinion, that medical care contributed to the injury (causation) and fell below the standards of practice in the community (negligence). Under proposed no-fault alternatives to litigation, entitlement to compensation and liability for payment might also depend on an experts opinion as to whether the patients outcome was caused by medical care rather than by a preexisting disease or condition [1-4]. Critics have identified several problems with case review. First, experts cannot form a consensus about which outcomes are adverse. Second, medical technology changes rapidly and creates uncertainty about the appropriateness and effectiveness of practices. Third, administrative or transaction costs in making individualized determinations of causation might be high [5-8]. The American College of Physicians [9] and others [10] have called for further demonstration projects. Building on previous research on the reliability of clinical judgments, we used a large sample of physician reviews of medical records to estimate the degree of agreement on the cause of adverse patient outcomes. We also discuss the implications of the results for quality assurance, performance assessment, and proposals for no-fault patient compensation. Methods Cases were obtained from the Medical Practice Study, a project designed to estimate the rate of adverse events occurring among inpatients in a random sample of 31 429 medical records from 51 health care facilities in New York State. We defined an adverse event as an injury that 1) was caused at least in part by medical management and 2) required or prolonged hospitalization or led to disability after discharge. The injury could result from a providers action or inaction in either inpatient or outpatient settings or from a drug or medical device. The medical management did not have to be substandard or inappropriate; the injury could follow an unexpected complication. Adverse outcomes caused solely by underlying disease or by the intended consequences of treatment were not considered to be adverse events. For example, an injury to the recurrent laryngeal nerve during partial thyroidectomy (an unplanned and unintended but recognized complication) would be considered an adverse event, but the intentional destruction of the same nerve in a radical thyroid resection for cancer would not. A broken experimental balloon that led to an embolus and stroke during cardiac catheterization would, as a complication of treatment, also be an adverse event, especially if the patients risk was unknown. This result would apply even in a study approved by a Human Subjects Committee. Other aspects of the Medical Practice Study and the general methods have been widely reported [11-15]. The following methods are relevant to our report. Record Review Records were reviewed in two stages. In stage 1 (which is not the subject of this report), nurses and medical records administrators used a single review per case to screen the entire sample of records for the presence of 1 or more of 18 explicit criteria (Figure 2). These criteria were based primarily on previous research [16] and were revised by the physician investigators of the Medical Practice Study. Although explicit, the criteria were broad and open to interpretation. The nurses and records administrators received an extensive manual, which contained detailed examples of the criteria, and 2 hours of focused classroom training from team leaders chosen for this project. To increase the efficiency and accuracy of screening, the nurses and records administrators used preprinted forms generated by the project management team. Nurses were instructed to refer any questionable cases for stage 2 review. Questions of a more general nature were referred to supervisors and then to the project office for consistent responses. The estimated negative predictive value of the screening was 99.5% [17]. Figure 2. Screening Criteria Implemented at Stage 1 Review by Nurses and Medical Records Administrators. Judgments on adverse events by pairs of physician-reviewers and rate of agreement on occurrence of adverse events compared with extreme disagreement. If a = cases of extreme disagreement (one reviewer scored the outcome as 0 [no possible adverse event] and the other scored the case as 4, 5, or 6) and b = cases for which both reviewers found adverse events (both scored the case as 4, 5, or 6), then the reported rate of agreement = a/(a + b). Bars represent exact binomial 95% CIs. Numbers in parentheses are the population-weighted estimates of the number of cases in New York State in 1984 that are represented by the sampled cases reported in this figure. In stage 2, each record that had or may have had at least one criterion present was further analyzed by two physicians who worked independently. Physicians were recruited primarily from New York State through a network of personal contacts of the study investigators. The physicians could not review records at the hospitals in which they practiced. Most were board certified in surgery (23%) or internal medicine (68%); the remaining were certified in obstetrics and gynecology, family practice, pediatrics, urology, or emergency medicine. Eighty-five percent were male. Most physicians were in the early stage of their careers: Fifty-five percent had received board certification within the 10 years before the study began. All physicians had telephone access to a panel of experts. A separate manual and a structured abstraction form guided the stage 2 review. As described previously [17], both were revised repeatedly after extensive pilot testing. This 65-page manual included explicit instructions on several types of adverse events. According to the manual, for example, all surgical wound infections were almost invariably adverse events, as were all falls and all drug reactions that prolonged hospitalization or caused disability. A 14-page abstraction form first asked the physician reviewer to assess whether an adverse event might have occurred. If the physician found no possible adverse event, the review was stopped and the case received a score of 0. If an adverse event might have occurred, the reviewer considered a list of factors on the cause of the injury and rated his or her confidence about the occurrence of an adverse event on an interval scale of 1 to 6 (Figure 1). For a confidence score of 2 (slight to modest evidence of an adverse event) or greater, the reviewer indicated the type of event (fall, drug reaction, wound infection, error of omission, or failure to diagnose), the number of additional days of hospitalization (if applicable), and the degree of disability over and above the underlying disease. Finally, the reviewers considered whether the error amounted to negligence. Within this structure, however, the physician could be discreet in judging the cause of the injury, hospitalization, or disability (a structured implicit review). All physician reviewers identified themselves by number, with the understanding that their confidential opinions would not be used for quality assurance, peer review, or litigation. Copies of the abstraction booklet are available from the authors. Figure 1. Our report focuses on the two independent expert opinions obtained during stage 2 review as to whether an adverse outcome identified during stage 1 had been caused at least in part by medical management. Results of each assessment of causation were linked to the patients computerized discharge data summary to identify the patients age, diagnosis, and discharge status. Statistical Analysis Agreement between Reviewers We calculated a rate of agreement between the two physician reviewers in each pair on adverse events using a statistic described by Grant [18] for assessing agreement on abnormal tracings from electronic fetal monitoring. In our application, the numerator of this statistic was the number of cases in which both reviewers assessed their confidence in an adverse event as more likely than not or greater. This assessment corresponded to a score of 4, 5, or 6. The denominator was the sum of the numerator and the number of cases of extreme disagreement, for which one reviewer scored the case as 4, 5, or 6 and the other physician found no possible adverse event (a score of 0). The statistic therefore compared the number of cases with agreed-upon adverse events with the number of clear disagreements. This statistic does not include cases for which both physicians agreed that no adverse event had occurred. It recognizes that agreement about whether a patients condition is normal (no adverse event) is usually greater than agreement about whether a patient has disease or an abnormal condition [19-22]. The statistic is also not affected by the number of clearly normal cases in the samples of cases for review. In our study, the number of cases clearly without adverse events at stage 2 was influenced by the coarseness of the previous screening process. The stage 1 reviewers were cautioned to avoid false-negative determinations if they were in doubt, so that adverse events would not be overlooked. This statistic also facilitated comparisons of rates of agreement across subsets of such adverse events as drug reactions, which are

Physicians' Perceptions of the Risk of Being Sued

We explore the deterrent effect of the tort system by assessing physician perceptions of the risk of being sued and the impact of those perceptions on their own practice. The data are from a mailed survey conducted in 1989 of a random sample of physicians who were practicing in New York State in 1984. The survey results were compared to the actual risk of suit using the between-group (Wald) test and logistic regression methods. We also surveyed physicians about practice changes undertaken in the last ten years, factors influencing practice standards, and the costs of being sued and included these in the analysis. On average, physicians estimate that 19.5 out of one hundred of their colleagues will be sued in a given year, approximately three times the actual rate, with significant differences by specialty, location, and suit history. Perceived risk is associated with self-reported changes in test-ordering frequency and reduction in practice scope. The median number of days lost from practice to defend a malpractice suit was three to five, and 6 percent of the physicians surveyed incurred some out-of-pocket expenses. These findings suggest that physicians respond to the messages sent by litigation in a manner consistent with the deterrent theory of tort litigation.

Developing and evaluating performance measures for ambulatory care quality: a preliminary report of the DEMPAQ project.

Because of the focus on technical quality, the content of the DEMPAQ performance measures is clinically detailed and oriented toward processes of care relevant to the everyday practice of medicine in the ambulatory setting. This emphasis is crucial if the performance measures are to be useful to practicing physicians.

Designing and using measures of quality based on physician office records

This article presents our principles for developing performance measures to assess the quality of ambulatory care. The measures were developed as part of a project for developing and evaluating methods to promote ambulatory care quality (DEMPAQ). We describe our design for the performance measures, present examples of the DEMPAQ review criteria, and show the formats we used to feed back information to physicians. We conclude by presenting the results of our appralsal of the performance measures showing how evaluation can aid in the interpretation of measurement findings.

Developing a Quality Improvement Database Using Health Insurance Data: A Guided Tour with Application to Medicare's National Claims History File

Health policy researchers are increasingly turning to insurance claims to provide timely information on cost, utilization, and quality trends in health care markets. This research offers an in-depth description of how to systematically transform raw inpatient and ambulatory claims data into useful information for health care management and research using the Health Care Financing Administrations National Claims History file as an example. The topics covered include: (a) understanding the contents and architecture of claims data, (b) creating analytic files from raw claims, (c) technical innovations for health policy studies, (d) assessing data accuracy, (d) the costs of using claims data, and (e) ensuring confidentiality. In summary, claims data are found to have great potential for quality of care analysis. As in any analysis, careful development of a database is required for scientific research. The methods outlined in this study offer health data novices as well as experienced analysts a series of strategies to maximize the value of claims data for health policy analysis.

Education to improve the quality of clinical care in group practices.

Intervening to improve the quality of medical care is an essential component of an effective quality improvement program. Although interventions may take many forms, most have some educational component. This article discusses different types of educational strategies that may be used in group practices and describes our experience with a continuing medical education (CME) program based on the results of clinical performance measurement. Data are presented showing the effectiveness and acceptability of the educational intervention in the DEMPAQ project.