Jack E. Zimmerman

APACHE II: a severity of disease classification system.

This paper presents the form and validation results of APACHE II, a severity of disease classification system. APACHE II uses a point score based upon initial values of 12 routine physiologic measurements, age, and previous health status to provide a general measure of severity of disease. An increa

APACHE-acute physiology and chronic health evaluation: a physiologically based classification system.

&NA; Investigations describing the utilization pattern and documenting the value of intensive care are limited by the lack of a reliable and valid classification system. In this paper, the authors describe the development and initial validation of acute physiology and chronic health evaluation (APACHE), a physiologically based classification system for measuring severity of illness in groups of critically ill patients. APACHE uses information available in the medical record. In studies on 582 admissions to a university hospital ICU and 223 admissions to a community hospital ICU, APACHE was reliable in classifying ICU admissions. In validation studies involving these 805 admissions, the acute physiology score of APACHE demonstrated consistent agreement with subsequent therapeutic effort and mortality. This was true for a broad range of patient groups using a variety of sensitivity analyses. After successful completion of multi‐institutional validation studies, the APACHE classification system could be used to control for case mix, compare outcomes, evaluate new therapies, and study the utilization of ICUs.

An evaluation of outcome from intensive care in major medical centers

We prospectively studied treatment and outcome in 5030 patients in intensive care units at 13 tertiary care hospitals. We stratified each hospitals patients by individual risk of death using diagnosis, indication for treatment, and Acute Physiology and Chronic Health Evaluation (APACHE) II score. We then compared actual and predicted death rates using group results as the standard. One hospital had significantly better results with 69 predicted but 41 observed deaths (p less than 0.0001). Another hospital had significantly inferior results with 58% more deaths than expected (p less than 0.0001). These differences occurred within specific diagnostic categories, for medical patients alone and for medical and surgical patients combined, and were related more to the interaction and coordination of each hospitals intensive care unit staff than to the units administrative structure, amount of specialized treatment used, or the hospitals teaching status. Our findings support the hypothesis that the degree of coordination of intensive care significantly influences its effectiveness.

Prognosis in acute organ-system failure.

This prospective study describes the current prognosis of patients in acute Organ System Failure (OSF). Objective definitions were developed for five OSFs, and then 5677 ICU admissions from 13 hospitals were monitored. The number and duration of OSF were linked to outcome at hospital discharge for each of the 2719 ICU patients (48%) who developed OSF. For all medical and most surgical admissions, a single OSF lasting more than 1 day resulted in a mortality rate approaching 40%. Among both medical and surgical patients, two OSFs for more than 1 day increased death rates to 60%. Advanced chronologic age increased both the probability of developing OSF and the probability of death once OSF occurred. Mortality for 99 patients with three or more OSFs persisting after 3 days was 98%. The two patients who survived were both young, in prior excellent health, and had severe but limited primary diseases. These results emphasize the high death rates associated with acute OSF and the rapidity with which mortality increases over time. The prognostic estimates provide reference data for physicians treating similar patients.

Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients.

Objective:To improve the accuracy of the Acute Physiology and Chronic Health Evaluation (APACHE) method for predicting hospital mortality among critically ill adults and to evaluate changes in the accuracy of earlier APACHE models. Design:Observational cohort study. Setting:A total of 104 intensive care units (ICUs) in 45 U.S. hospitals. Patients:A total of 131,618 consecutive ICU admissions during 2002 and 2003, of which 110,558 met inclusion criteria and had complete data. Interventions:None. Measurements and Main Results:We developed APACHE IV using ICU day 1 information and a multivariate logistic regression procedure to estimate the probability of hospital death for randomly selected patients who comprised 60% of the database. Predictor variables were similar to those in APACHE III, but new variables were added and different statistical modeling used. We assessed the accuracy of APACHE IV predictions by comparing observed and predicted hospital mortality for the excluded patients (validation set). We tested discrimination and used multiple tests of calibration in aggregate and for patient subgroups. APACHE IV had good discrimination (area under the receiver operating characteristic curve = 0.88) and calibration (Hosmer-Lemeshow C statistic = 16.9, p = .08). For 90% of 116 ICU admission diagnoses, the ratio of observed to predicted mortality was not significantly different from 1.0. We also used the validation data set to compare the accuracy of APACHE IV predictions to those using APACHE III versions developed 7 and 14 yrs previously. There was little change in discrimination, but aggregate mortality was systematically overestimated as model age increased. When examined across disease, predictive accuracy was maintained for some diagnoses but for others seemed to reflect changes in practice or therapy. Conclusions:APACHE IV predictions of hospital mortality have good discrimination and calibration and should be useful for benchmarking performance in U.S. ICUs. The accuracy of predictive models is dynamic and should be periodically retested. When accuracy deteriorates they should be revised and updated.

THE PERFORMANCE OF INTENSIVE CARE UNITS : DOES GOOD MANAGEMENT MAKE A DIFFERENCE ?

A significant portion of health care resources are spent in intensive care units with, historically, up to two-fold variation in risk-adjusted mortality. Technological, demographic, and social forces are likely to lead to an increased volume of intensive care in the future. Thus, it is important to identify ways of more efficiently managing intensive care units and reducing the variation in patient outcomes. Based on data collected from 17,440 patients across 42 ICUs, the present study examines the factors associated with risk-adjusted mortality, risk-adjusted average length of stay, nurse turnover, evaluated technical quality of care, and evaluated ability to meet family member needs. Using the Apache III methodology for risk-adjustment, findings reveal that: 1) technological availability is significantly associated with lower risk-adjusted mortality (beta = -.42); 2) diagnostic diversity is significantly associated with greater risk-adjusted mortality (beta = .46); and 3) caregiver interaction comprising the culture, leadership, coordination, communication, and conflict management abilities of the unit is significantly associated with lower risk-adjusted length of stay (beta = .34), lower nurse turnover (beta = -.36), higher evaluated technical quality of care (beta = .81), and greater evaluated ability to meet family member needs (beta = .74). Furthermore, units with greater technological availability are significantly more likely to be associated with hospitals that are more profitable, involved in teaching activities, and have unit leaders actively participating in hospital-wide quality improvement activities. The findings hold a number of important managerial and policy implications regarding technological adoption, specialization, and the quality of interaction among ICU team members. They suggest intervention “leverage points” for care givers, managers, and external policy makers in efforts to continuously improve the outcomes of intensive care.

Variations in Mortality and Length of Stay in Intensive Care Units

Intensive care units (ICUs), first introduced in the 1960s, now account for approximately 7% of total U.S. hospital beds, 20% to 30% of hospital costs, and 1% of the U.S. gross domestic product [1-3]. These economic and institutional consequences have increased the need for outcome evaluation and guidance regarding efficient utilization. Mortality rates, an insensitive measure for an entire hospital [4-6], are high enough in ICUs to serve as one reliable performance indicator. Substantial progress has also been made in identifying clinical risk factors for death and resource utilization for patients in ICUs [7-11]. The objective of this study was to explore the ability to evaluate ICU performance using risk-adjusted in-hospital mortality rates and length of ICU stay. In this report, we focus on the amount of variation that can be accounted for after adjusting for patient characteristics present at admission. We describe the nature and relative importance of these factors and the extent of the remaining variation in outcome performance after such adjustment. Methods Hospital and Intensive Care Unit Selection We used a stratified random process based on geographic region, size, and teaching status (defined by the presence of resident housestaff or of one or more accredited graduate medical training programs) to select 26 hospitals from 1691 nonfederal U.S. hospitals with more than 200 beds. Twenty-three of the 26 hospitals agreed to participate. The reasons for nonparticipation of the three hospitals were the sale of the hospital, a severe nursing shortage (making data collection assistance unlikely), and a poor fiscal condition (making closure imminent). We chose three alternate hospitals from the same strata using an identical process. Fourteen other hospitals, primarily large tertiary care or university teaching hospitals with an interest in the project, also volunteered to participate in the study, giving a total of 40 hospitals. In hospitals with more than one ICU, data collection took place in the unit with the highest annual admission rate. Data were also collected in two ICUs at two volunteer institutions, giving a total of 42 ICUs for analysis. We excluded burn, pediatric, neonatal, and coronary care units from consideration. Thus, data collection took place in adult general medical, general surgical, and combined medical-surgical units. Patient Selection and Data Collection Data collection began in May 1988 and was completed in February 1990; the study period at each ICU averaged 9.7 months (range, 3 to 17 months). In most ICUs, data were collected concurrently for consecutive ICU admissions. A systematic scheme (for example, every second or third patient) was used in 20% of the units when patient volume precluded data collection on consecutive admissions. Patients were excluded from the study if they were admitted for chest pain, rule-out myocardial infarction, coronary artery bypass surgery, or burn injury; were younger than 16 years; or had a length of ICU stay of less than 4 hours. Information collected for each patient included age, an extensive listing of coexisting illnesses, location before ICU admission (emergency, recovery, hospital, or operating room; ICU readmission; or transfer from another ICU or hospital), and surgery status (elective or emergency, which was defined as surgery for an immediately life-threatening condition). We also recorded the primary reason for ICU admission using 78 mutually exclusive disease categories [12]. During the first ICU day, each patients clinical record was reviewed for APACHE III and Therapeutic Intervention Scoring System scoring [12, 13]. The APACHE III score consists of an acute physiology score obtained by applying weights to 17 potential physiologic variables, a weight applied to age, and additional weight to one of seven comorbid conditions that influence the risk for short-term death by reducing immune response. A higher APACHE III score (0 to 299) is associated with a higher risk for in-hospital death. The Therapeutic Intervention Scoring System is also a weighted (1 to 4) scoring system derived from 80 interventions and specific nursing tasks representing the intensity of care provided. During the subsequent 6 ICU days, we recorded all changes in the APACHE III acute physiology score and, using the Therapeutic Intervention Scoring System, in the type and amount of monitoring and treatment received. We also recorded length of stay in the ICU and in the hospital and followed all patients for survival at ICU and hospital discharge and at 30 days after discharge for all Medicare patients and for a 15% random sample of all other patients. Patient data were entered into on-site microcomputers by trained data collectors using specially designed software, and data underwent continuous quality monitoring and review. A formal interobserver reliability study was conducted at 11 of the hospitals [14]. Further details on data collection procedures are available elsewhere [12, 15]. Equations for Predicting In-Hospital Death and Length of ICU Stay For each patient, we estimated the probability of in-hospital death using a multiple logistic regression equation. The variables used in this analysis were preselected based on previous research [15]. The primary determinants of short-term outcome were defined as acute physiologic abnormalities within 24 hours of ICU admission (APACHE III acute physiologic score); the patients physiologic reserve as measured by age and the presence of specific comorbid conditions (as represented in the APACHE III score); the underlying disease prompting ICU admission; the location and duration of treatment immediately before ICU admission; and one institutional characteristic: the nature of hospital discharge practices as measured by the excess mean duration of hospital stay for survivors. This variable was determined for each unit based on a statistical analysis of length of hospital stay for all hospital survivors compared with average stay for all ICUs based on disease and APACHE III score (see footnote in Appendix Table 1). This variable was used to account for variations in triage pressure or practice style, both of which affect hospital discharge patterns: Hospitals that discharge patients later are likely to report more deaths because of the longer time during which their patients could die in the hospital [16]. Appendix Table 1. Relation of Prognostic Factors to In-Hospital Mortality and Length of Intensive Care Unit Stay Table 1. Characteristics of Patients, Hospitals, and Intensive Care Units* Each patients first admission to the ICU within the study period was included in the analysis. Second and subsequent readmissions to the ICU were excluded from the mortality analysis to avoid counting two outcomes for the same individual. The mortality equation was cross-validated using a grouped jackknife approach [17]: All patient data were divided into 10 independent groups using a random-number generator, and 10 different regression models were estimated, with each model excluding one group. Each model was used to calculate predictions for the excluded group. We then compared the predicted risks for individual patients from the excluded groups with the predictions based on the equation estimated on the entire sample. For both the grouped jackknife approach and the equation estimated on all patients, the equation was estimated each time with the same fixed set of predictor variables, without using stepwise variable selection or other search techniques. We developed a separate multiple regression equation to estimate length of ICU stay based on the same patient and institutional characteristics as described above, with the following exceptions. The excess mean adjusted length of hospital stay and the patients length of hospital stay before ICU admission were deleted, and traditional hospital descriptors, such as geographic region, bed size, and teaching status, were added. This analysis excluded patients discharged to another ICU for which there was incomplete data on total length of ICU stay but included the fact that a patient was readmitted to the unit. In cases in which length of ICU stay exceeded 40 days, such stays were rounded down to 40 days and then included in the analysis. To allow for nonlinearities in the relation between continuous variables and length of ICU stay, the method of restricted cubic splines was used [18]. This technique permits the weight attributable to a variable to vary continuously throughout its possible range. The model for length of ICU stay was also cross-validated using the same grouped jackknife approach as has been described for mortality. To measure how much of the variation in mortality and length of stay were accounted for by the equations, we used the area under a receiver operating characteristic curve [19] for the dichotomous outcome, mortality, and the R2 for the continuous variable, length of stay. Except for the comparisons of the full equation with the cross-validated predictions, all these results report associations with the cross-validated predicted risks. Risk-adjusted (Standardized) Ratios for In-Hospital Mortality and Length of ICU Stay To calculate a risk-adjusted mortality rate for each ICU, we added individual patient predictions using the cross-validated equations and then calculated a standardized mortality ratio by dividing actual by mean predicted group death rate at hospital discharge. The units were then ranked by relative performance according to their standardized mortality ratio. We used a chi-square test to determine the significance of differences between actual and predicted survival rates for each unit, and a P value 0.05 at the unit level was considered to be significant. To determine the amount of variation across ICUs that was accounted for by predictions, we estimated a univariate least-squares regression equation across all 42 ICUs, using the observed deat

APACHE II-A Severity of Disease Classification System: Reply

This paper presents the form and validation results of APACHE II, a severity of disease classification system. APACHE II uses a point score based upon initial values of 12 routine physiologic measurements, age, and previous health status to provide a general measure of severity of disease. An increasing score (range 0 to 71) was closely correlated with the subsequent risk of hospital death for 5815 intensive care admissions from 13 hospitals. This relationship was also found for many common diseases.When APACHE II scores are combined with an accurate description of disease, they can prognostically stratify acutely ill patients and assist investigators comparing the success of new or differing forms of therapy. This scoring index can be used to evaluate the use of hospital resources and compare the efficacy of intensive care in different hospitals or over time.

Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited.

Objective:To examine the Hosmer-Lemeshow tests sensitivity in evaluating the calibration of models predicting hospital mortality in large critical care populations. Design:Simulation study. Setting:Intensive care unit databases used for predictive modeling. Patients:Data sets were simulated representing the approximate number of patients used in earlier versions of critical care predictive models (n = 5,000 and 10,000) and more recent predictive models (n = 50,000). Each patient had a hospital mortality probability generated as a function of 23 risk variables. Interventions:None. Measurements and Main Results:Data sets of 5,000, 10,000, and 50,000 patients were replicated 1,000 times. Logistic regression models were evaluated for each simulated data set. This process was initially carried out under conditions of perfect fit (observed mortality = predicted mortality; standardized mortality ratio = 1.000) and repeated with an observed mortality that differed slightly (0.4%) from predicted mortality. Under conditions of perfect fit, the Hosmer-Lemeshow test was not influenced by the number of patients in the data set. In situations where there was a slight deviation from perfect fit, the Hosmer-Lemeshow test was sensitive to sample size. For populations of 5,000 patients, 10% of the Hosmer-Lemeshow tests were significant at p < .05, whereas for 10,000 patients 34% of the Hosmer-Lemeshow tests were significant at p < .05. When the number of patients matched contemporary studies (i.e., 50,000 patients), the Hosmer-Lemeshow test was statistically significant in 100% of the models. Conclusions:Caution should be used in interpreting the calibration of predictive models developed using a smaller data set when applied to larger numbers of patients. A significant Hosmer-Lemeshow test does not necessarily mean that a predictive model is not useful or suspect. While decisions concerning a mortality models suitability should include the Hosmer-Lemeshow test, additional information needs to be taken into consideration. This includes the overall number of patients, the observed and predicted probabilities within each decile, and adjunct measures of model calibration.

Improving intensive care : observations based on organizational case studies in nine intensive care units : a prospective, multicenter study

To examine organizational practices associated with higher and lower intensive care unit (ICU) outcome performance. Design:Prospective multicenter study. On-site organizational analysis; prospective inception cohort. Setting:Nine ICUs (one medical, two surgical, six medical-surgical) at five teaching and four nonteaching hospitals. Participants:A sample of 3,672 ICU admissions; 316 nurses and 202 physicians. Materials and Methods:Interviews and direct observations by a team of clinical and organizational researchers. Demographic, physiologic, and outcome data for an average of 408 admissions per ICU; and questionnaires on ICU structure and organization. The ratio of actual/predicted hospital death rate was used to measure ICU effectiveness; the ratio of actual/predicted length of ICU stay was used to assess efficiency. Measurements and Main Results:ICUs with superior risk-adjusted survival could not be distinguished by structural and organizational questionnaires or by global judgment following on-site analysis. Superior organizational practices among these ICUs were related to a patient-centered culture, strong medical and nursing leadership, effective communication and coordination, and open, collaborative approaches to solving problems and managing conflict. Conclusions:The best and worst organizational practices found in this study can be used by ICU leaders as a checklist for improving ICU management. (Crit Care Med 1993; 21:1443–1451)