Jane M. Geraci

International Classification of Diseases, 9th Revision, Clinical Modification codes in discharge abstracts are poor measures of complication occurrence in medical inpatients.

OBJECTIVES The authors tested the ability of International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes in discharge abstracts to identify medical inpatients who experienced an in-hospital complication, using complications identified through chart review as the gold standard. METHODS Two sets of ICD-9-CM codes were used: an inclusive set including many medical diagnoses that may also be coexistent complicating conditions on admission rather than complications and an exclusive set consisting primarily of ICD-9-CM-specified complication and adverse drug event codes. RESULTS Neither set performed well as a diagnostic test for complication occurrence according to receiver operating characteristic analysis (ROC areas were 0.61 for the inclusive set and 0.55 for the exclusive set). Sensitivities of the ICD-9-CM codes for complications were 0.34 for the inclusive set and 0.14 for the exclusive set. Corresponding positive predictive values were 0.32 and 0.37, respectively. Sensitivities of code definitions for individual complications were generally poor, less than 0.5 in most cases. CONCLUSIONS The authors conclude that ICD-9-CM codes in discharge abstracts are poor measures of complication occurrence.

Predicting the Occurrence of Adverse Events after Coronary Artery Bypass Surgery

Abbreviation HCFAHealth Care Financing Administration The frequency of coronary artery bypass graft surgery increased by 60% in the Medicare population between 1986 and 1988 [1]. The federal government estimated that 135 000 Medicare beneficiaries would have bypass surgery at more than 700 hospitals in 1991, at a total cost to Medicare of over

Postoperative adverse events of common surgical procedures in the Medicare population

3 billion [1]. Assessing the quality of care provided to Medicare recipients is a priority of the Health Care Financing Administration (HCFA) [2]. Evaluation of the quality of care of hospital patients has recently focused on hospital-specific mortality rates [3-7]. This approach is facilitated by the availability of mortality statistics from administrative data sets and by the widespread acceptance of death as a readily measured and undesirable outcome of hospital care. However, the low frequency of postoperative death among surgical patients makes mortality an insensitive tool for evaluating surgery at many hospitals [6, 8]. Substandard patient care may result in problems that are common and serious but have little effect on 30-day mortality. Donabedian [9] and others [10] have suggested studying outcomes more closely related to specific conditions or procedures. This approach was recently used by Luft and colleagues [11] in a study in which nonfatal adverse outcomes of patient hospitalization were used as a measure of quality and by the HCFA in a recent report of adverse-outcome rates for selected procedures, including bypass surgery, in the Medicare population [2]. Another concern related to using hospital mortality or adverse-event rates to compare providers is the need to account for differences in illness severity at admission. When there is inadequate adjustment for severity of illness, conclusions about quality of care from mortality or adverse-event rates may be unjustified. In our study, we used chart-abstracted data to define and predict adverse events that occurred after bypass surgery in Medicare patients. The use of clinical information recorded at patient admission enables adjustment for severity of illness as well as the identification of the relation between illness severity and postoperative adverse events. Methods Study Population Medicare patients 65 years of age or more composed our study sample; 2213 patients who underwent bypass surgery between January 1985 and June 1986 in seven states (Alabama, Arizona, Indiana, New York, Pennsylvania, Utah, and Wisconsin) were chosen from a larger study of hospitalized patients conducted by the HCFA in 1987. Peer review organizations in each of the seven states received identifiers for randomly chosen patients with specific ICD-9-CM diagnosis or procedure codes. These organizations directed the abstraction of clinical information according to the MedisGroups protocol [12-18]. MedisGroups is a proprietary software product developed by MediQual Systems, Inc. (Westborough, Massachusetts); MediQual staff trained the chart abstractors. Thomas and Ashcraft [16] recently found MedisGroups to have a high inter-rater reliability, comparable to that of APACHE II and diagnostic-related groups [16]. Table 1. Characteristics of the Study Patients MedisGroups mandates the collection of about 250 key clinical findings representing the results of the patient history, the physical examination, laboratory tests, pathologic examination, and radiologic evaluation. These key clinical findings are clinical abnormalities recorded during reviews of specific periods of a patients hospital stay. Separate reviews are done at admission and after surgery; we used only clinical data recorded in the MedisGroups admission review to adjust for severity of illness. In this first review, clinical information is collected from admission through hospital day 2 or the time of surgery, whichever occurs first. We identified key clinical findings that might indicate postoperative adverse events using the date of bypass surgery and the dates of the MedisGroups reviews to verify the temporal sequence. This description and the data used for this study pertain to the 1987 version of MedisGroups, which was modified for the HCFA. Information from Medicare, Part A, files (dates of death, hospital admission, and discharge; and ICD-9-CM diagnosis and procedure codes) was merged with clinical data to create the analytic files. Entrance into our study required the presence of one of the following ICD-9-CM codes for the bypass procedure: 36.10-36.16, 36.19, 36.20, and 36.30. Definition of Adverse Events Adverse events were conceptualized as serious postoperative complications potentially related to quality of care, resulting in a high likelihood of increased morbidity, subsequent intensive therapy, or prolonged hospital stay. Definitions of adverse events in the postoperative period were developed through review of the clinical literature on bypass surgery, consultation with medical and surgical subspecialists, and review of recommendations by expert panels convened by the HCFA. In addition to death within 30 days of hospital admission, 13 nonfatal adverse events occurring in the postoperative period were identified and defined as follows: new myocardial infarction by electrocardiogram; cardiorespiratory arrest for the first time; new congestive heart failure by chest radiograph; acute graft failure; new-onset thromboembolism; new-onset stroke; coma; mechanical ventilation for more than 48 hours; wound infection; bacteremia; acute renal failure as manifested by first-time peritoneal dialysis or hemodialysis or by a rise in the serum creatinine level from normal at admission to 442 mol/L (5.0 mg/dL) or greater; transfusion of more than 6 units of whole blood or packed red blood cells; and an unplanned return to surgery. Over 60 key clinical findings relating to severity of illness at admission were evaluated as predictors of the outcome (any adverse event) (Appendix Table 1). To be studied, key clinical findings had to be of strong clinical interest or present in at least 10 patients at admission. Potential indicators of illness severity included elements of the medical history, such as a history of bypass surgery, diabetes, or stroke; laboratory abnormalities such as an abnormal serum potassium level; the presence of an infiltrate on the chest radiograph; and diagnostic test findings such as number of coronary arteries with stenosis of 50% or more. All key clinical findings were handled as dichotomous variables (that is, presence or absence of a clinical characteristic). Bypass surgery was defined as emergent when done on the day of admission. Acute myocardial infarction was considered present at admission when the electrocardiogram indicated acute myocardial infarction or the serum creatine kinase MB fraction was 0.04 or more. Table 2. Postoperative Adverse Events and Death in 2213 Patients* Statistical Analysis Summary descriptive statistics were computed for all variables using the SAS software package (SAS, Cary, North Carolina). These included frequencies, proportions, and means plus standard deviations. Correlates of adverse events were identified using the t-test, chi-square test, and the Fisher exact test. Regression models were developed in several steps to predict the occurrence of any adverse event, including death within 30 days of admission. In addition to selected key clinical findings considered to be indicators of illness severity, age (coded as a linear variable to the age of 85 years) and sex were included as explanatory variables in all models. The individual adverse events were combined into one single outcome variable, any adverse event, because most did not occur frequently enough among the study patients to permit meaningful statistical modeling of single adverse events. Forward selection stepwise, ordinary least-squares regression was used for exploratory models; we have found that this type of analysis yields the same set of predictors as logistic regression and is more efficient. Variables were retained in regression models if their associated P values were 0.05. The database was randomly split in half, and a model predicting any adverse event was developed for each half-sample. Each model was then fit to the other half-sample, allowing computation of a cross-validated R2, which is calculated as 1 SSE/SSM, where SSE is the sum of the squared differences between the outcome indicator and the probability of complication predicted by the regression equation, and SSM is the sum of the squared differences between the outcome indicator variable and the overall mean rate of adverse events [19]. A positive cross-validated R2 was taken as evidence that the variables within the models had predictive power. Finally, all variables from either of the cross-validated models were used as candidates in a stepwise, logistic regression model procedure applied to the entire database. This process of model development was used to reduce the likelihood that explanatory variables would be selected through overfitting rather than systematic association. A c-statistic was computed as a measure of the explanatory power of the logistic model. The c-statistic equals the probability that, for a randomly drawn pair of patients, one with and one without an adverse event, the model assigns a higher probability of having an adverse event to the patient who had one. The c-statistic is also equivalent to the area under a receiver-operating-characteristic (ROC) curve. A value of c near 0.5 indicates no discriminatory power, whereas a c of 1.0 indicates perfect discrimination between patients with and without adverse events [20]. To assess the models goodness of fit, we calculated the Hosmer-Lemeshow statistic [21]; we also calculated expected numbers of patients with any adverse event, by decile of predicted risk. Results Demographic and clinical characteristics of the study sample are shown in Table 1. Eighteen percent of the patients were 75 years of age or more. Se

Mortality after noncardiac surgery: Prediction from administrative versus clinical data

Mortality rates are the most widely used measure in assessing patient outcome from hospitalization. However, they may be an insensitive measure of quality for surgical patients because death is a relatively rare outcome. A random sample of patient data (n = 8126) selected from the Medicare files of seven states was used to identify, through chart abstraction, clinical postoperative complications of surgery that could serve as measures of quality. Four surgical procedures were studied: 1) coronary artery bypass grafting; 2) coronary angioplasty; 3) cholecystectomy; and 4) prostatectomy. Severity at admission was controlled for using severity-of-illness models developed with chart-abstracted data to predict adverse events after these four procedures. 30-day mortality rates ranged from 1.0% to 6.6%, while the prevalence of postoperative adverse events identified from chart review was greater (6.9% to 33.3%). There were significant differences between patients with and without adverse events. For example, coronary artery bypass graft patients with adverse events had prolonged postsurgical lengths of stay (18.5 ± 13.2 vs. 13.2 ± 6.2, P < 0.001) and higher mortality rates (15.2% vs. 2.6%, P < 0.001). The R-square values using clinical indicators at admission to predict the occurrence of any adverse event ranged from 0.05 to 0.13. Clinically meaningful adverse events of surgery can be successfully identified through chart abstraction and appear to be valid measures of postoperative complications among surgical patients. Severity adjustment at admission only modestly predicts the occurrence of these adverse events.

EXPECTED SOURCE OF PAYMENT AND USE OF HOSPITAL SERVICES FOR CORONARY ATHEROSCLEROSIS

Background:Hospital profiles are increasingly constructed using risk-adjusted clinical data abstracted from patient records. Objective:We sought to compare hospital profiles based on risk adjusted death within 30 days of surgery from administrative versus clinical data in a national cohort of surgical patients. Design:This was a cohort study that included 78,546 major noncardiac operations performed between October 1, 1991 and December 31, 1993 in 44 Veterans Affairs hospitals. Administrative data were used to develop and validate multivariable logistic regression models of 30-day postoperative death for all surgery and 4 surgical specialties (general, orthopedic, thoracic, and vascular). Previously developed and validated clinical models were obtained and reproduced for matching operations using data from the VA National Surgical Quality Improvement Program. Observed-to-expected 30-day mortality ratios for administrative and clinical data were calculated and compared for each hospital. Results:In multivariable logistic regression models using administrative data, characteristics such as patient age, race, marital status, admission from a nursing home, interhospital transfer, admission on the weekend, weekend surgery, and risk strata consisting of groups of principal and comorbidity diagnoses were predictive of postoperative mortality (P < 0.05). Correlations of the clinical and administrative observed-to-expected ratios were 0.75, 0.83, 0.64, 0.78, and 0.86 for all surgery, general, orthopedic, thoracic, and vascular surgery, respectively. When compared with clinical models, administrative models identified outlier hospitals with sensitivity of 73%, specificity of 89%, positive predictive value of 51%, and negative predictive value of 96%. Conclusions:Our data suggest that risk adjustment of mortality using administrative data may be useful for screening hospitals for potential quality problems.

In-hospital complications among survivors of admission for congestive heart failure, chronic obstructive pulmonary disease, or diabetes mellitus

To study associations between payer and provision of services for patients hospitalized for coronary atherosclerosis, the authors analyzed abstracts of 24,424 discharges from California acute care hospitals during 1989. Services examined included receipt of coronary artery bypass surgery, percutaneous transluminal coronary angioplasty (PTCA), long length of stay (LOS) without revascularization, and overall LOS. Regression techniques controlled demographic factors and comorbidities. The privately insured were 96% more likely to undergo revascularization (either bypass or PTCA) than Medicaid discharges and 117% more likely than the uninsured. Odds of revascularization for Medicare and health maintenance organization discharges resembled those for the privately insured. Analyzed separately, PTCA was far more likely among the privately insured than Medicaid beneficiaries and the uninsured. In addition, the adjusted odds for PTCA were 52% greater for the privately insured than for health maintenance organization discharges. The greatest likelihood of long LOS without revascularization and the greatest overall LOS was observed for Medicaid discharges. Strong associations, consistent with financial incentives to provide care, exist between payer and provision of services. Future studies need to address whether variations in process result from differences in thresholds for procedure performance, differences in admission practices, or both.

The association of quality of care and occurrence of in-hospital, treatment-related complications.

OBJECTIVE: To determine the frequency of hospital complications among survivors of inpatient treatment for congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD), or diabetes mellitus (DM).DESIGN: Retrospective cohort study.SETTING: Nine Veterans Affairs hospitals in the southern United States.PATIENTS: 1,837 men veterans discharged alive following hospitalization for CHF, COPD, or DM between January 1987 and December 1989. This patient population represents a subset of cases gathered to study the process of care in the hospital and subsequent early readmission; thus, veterans who died in the hospital were not included.MEASUREMENTS: Medical record review to record the occurrence of any of 30 in-hospital complications such as cardiac arrest, nosocomial infections, or delirium (overall agreement between two reviewers=84%, kappa=0.37).RESISTS: Complications occurred in 15.7% of the CHF cases, 13.1% of the COPD cases, and 14.8% of the DM cases. Hypoglycemic reactions were the most frequent individual adverse events in the CHF and DM cases (3.6% and 11.4% of the cases, respectively), and theophylline toxicity was most frequent among the COPD cases (4.9%). Patient age, the presence of comorbid diseases, and the Acute Physiology Score (APS) of APACHE II were associated with complication occurrence. For each disease, the patients who had a complication had significantly longer mean hospital stays than did the patients who did not have complications (14.6 to 14.9 days vs 7.2 to 8.2 days, p<0.01).CONCLUSIONS: Complications are frequent among patients discharged alive with CHF, COPD, or DM. The patients who experienced complications were more ill on admission and had longer hospital stays.

The Demise of Comparative Provider Complication Rates Derived from ICD-9-CM Code Diagnoses

BACKGROUND Little data exist supporting the association of quality of care and nonfatal adverse outcomes in hospitalized patients, yet those outcomes are routinely scrutinized in quality assessment efforts. OBJECTIVE To determine whether measurable differences in quality of care are associated with the occurrence of non-fatal, in-hospital, and treatment-related complications. DESIGN Retrospective cohort study. SUBJECTS A total of 2,268 patients who were discharged alive from 9 Southwestern Veterans Affairs Medical Centers with congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD) or diabetes mellitus. MEASURES Retrospective chart review was performed to collect information on patient severity of illness, in-hospital complication occurrence, and process quality of care. Process quality was assessed as the adherence scores for admission work-up and for treatment during the hospital stay. Process quality represents the proportion of applicable admission or treatment criteria that were met by that patients care providers. Once severity of illness was taken into account Cox proportional hazards regression was used to assess the independent contribution of process quality of care to complication occurrence. RESULTS Higher admission work-up adherence scores for COPD patients and higher treatment adherence scores for COPD and diabetes patients were associated with a lower risk of complication occurrence. The adjusted risk ratios of complications for higher versus lower adherence scores (with 95% CI) were 0.64 (0.43, 0.97) and 0.52 (0.33, 0.80) for admission and treatment, respectively, in COPD patients, and 0.51 (0.31, 0.83) for treatment in diabetics. No significant association was found in CHF patients. CONCLUSION Better admission work-up and treatment quality in COPD patients, as well as treatment quality in diabetic patients, are associated with lower risk of nonfatal treatment-related complications in the study population.

Postoperative Adverse Events of Cholecystectomy in the Medicare Population

Administrative databases containing information derived from bills for health services, typically ICD-9-CM codes for patient diagnoses and procedures, as well as basic demographic characteristics, have been a tremendous source of data on patterns of care and outcomes across populations as small as those of single state, or encompassing all of America. Through innovative analyses of administrative data from both the United States

Identifying complications and low provider adherence to normative practices using administrative data.

We explored the use of postoperative adverse events of cholecystectomy as possible screens for poor quality of care. Retrospective analysis of clinical data ab stracted from hospital charts between 1985-1986 was conducted on a random sample of 3,182 cholecystectomy cases. Severity of illness models were developed pre dicting adverse events following cholecystectomy in pa tients with and without bile duct exploration. Outcome measures included 17 nonfatal adverse events and death within 30 days of admission. Adverse event rates were 23.2% for cases with bile duct exploration and 14.4% for cases without bile duct exploration. Cross-validated R-squareds and C-statistics showed that models had real, although modest, predictive power. We conclude that clinically meaningful adverse events of cholecystec tomy can be successfully identified through chart ab straction.