Andrew A. Kramer

Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients.

Objective:To improve the accuracy of the Acute Physiology and Chronic Health Evaluation (APACHE) method for predicting hospital mortality among critically ill adults and to evaluate changes in the accuracy of earlier APACHE models. Design:Observational cohort study. Setting:A total of 104 intensive care units (ICUs) in 45 U.S. hospitals. Patients:A total of 131,618 consecutive ICU admissions during 2002 and 2003, of which 110,558 met inclusion criteria and had complete data. Interventions:None. Measurements and Main Results:We developed APACHE IV using ICU day 1 information and a multivariate logistic regression procedure to estimate the probability of hospital death for randomly selected patients who comprised 60% of the database. Predictor variables were similar to those in APACHE III, but new variables were added and different statistical modeling used. We assessed the accuracy of APACHE IV predictions by comparing observed and predicted hospital mortality for the excluded patients (validation set). We tested discrimination and used multiple tests of calibration in aggregate and for patient subgroups. APACHE IV had good discrimination (area under the receiver operating characteristic curve = 0.88) and calibration (Hosmer-Lemeshow C statistic = 16.9, p = .08). For 90% of 116 ICU admission diagnoses, the ratio of observed to predicted mortality was not significantly different from 1.0. We also used the validation data set to compare the accuracy of APACHE IV predictions to those using APACHE III versions developed 7 and 14 yrs previously. There was little change in discrimination, but aggregate mortality was systematically overestimated as model age increased. When examined across disease, predictive accuracy was maintained for some diagnoses but for others seemed to reflect changes in practice or therapy. Conclusions:APACHE IV predictions of hospital mortality have good discrimination and calibration and should be useful for benchmarking performance in U.S. ICUs. The accuracy of predictive models is dynamic and should be periodically retested. When accuracy deteriorates they should be revised and updated.

Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited.

Objective:To examine the Hosmer-Lemeshow tests sensitivity in evaluating the calibration of models predicting hospital mortality in large critical care populations. Design:Simulation study. Setting:Intensive care unit databases used for predictive modeling. Patients:Data sets were simulated representing the approximate number of patients used in earlier versions of critical care predictive models (n = 5,000 and 10,000) and more recent predictive models (n = 50,000). Each patient had a hospital mortality probability generated as a function of 23 risk variables. Interventions:None. Measurements and Main Results:Data sets of 5,000, 10,000, and 50,000 patients were replicated 1,000 times. Logistic regression models were evaluated for each simulated data set. This process was initially carried out under conditions of perfect fit (observed mortality = predicted mortality; standardized mortality ratio = 1.000) and repeated with an observed mortality that differed slightly (0.4%) from predicted mortality. Under conditions of perfect fit, the Hosmer-Lemeshow test was not influenced by the number of patients in the data set. In situations where there was a slight deviation from perfect fit, the Hosmer-Lemeshow test was sensitive to sample size. For populations of 5,000 patients, 10% of the Hosmer-Lemeshow tests were significant at p < .05, whereas for 10,000 patients 34% of the Hosmer-Lemeshow tests were significant at p < .05. When the number of patients matched contemporary studies (i.e., 50,000 patients), the Hosmer-Lemeshow test was statistically significant in 100% of the models. Conclusions:Caution should be used in interpreting the calibration of predictive models developed using a smaller data set when applied to larger numbers of patients. A significant Hosmer-Lemeshow test does not necessarily mean that a predictive model is not useful or suspect. While decisions concerning a mortality models suitability should include the Hosmer-Lemeshow test, additional information needs to be taken into consideration. This includes the overall number of patients, the observed and predicted probabilities within each decile, and adjunct measures of model calibration.

Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III).

Objective:To update the Mortality Probability Model at intensive care unit (ICU) admission (MPM0-II) using contemporary data. Design:Retrospective analysis of data from 124,855 patients admitted to 135 ICUs at 98 hospitals participating in Project IMPACT between 2001 and 2004. Independent variables considered were 15 MPM0-II variables, time before ICU admission, and code status. Univariate analysis and multivariate logistic regression were used to identify risk factors associated with hospital mortality. Setting:One hundred thirty-five ICUs at 98 hospitals. Patients:Patients in the Project IMPACT database eligible for MPM0-II scoring. Interventions:None. Measurements and Main Results:Hospital mortality rate in the current data set was 13.8% vs. 20.8% in the MPM0-II cohort. All MPM0-II variables remained associated with mortality. Clinical conditions with high relative risks in MPM0-II also had high relative risks in MPM0-III. Gastrointestinal bleeding is now associated with lower mortality risk. Two factors have been added to MPM0-III: “full code” resuscitation status at ICU admission, and “zero factor” (absence of all MPM0-II risk factors except age). Seven two-way interactions between MPM0-II variables and age were included and reflect the declining marginal contribution of acute and chronic medical conditions to mortality risk with increasing age. Lead time before ICU admission and pre-ICU location influenced individual outcomes but did not improve model discrimination or calibration. MPM0-III calibrates well by graphic comparison of actual vs. expected mortality, overall standardized mortality ratio (1.018; 95% confidence interval, 0.996–1.040) and a low Hosmer-Lemeshow goodness-of-fit statistic (11.62; p = .31). The area under the receiver operating characteristic curve was 0.823. Conclusions:MPM0-II risk factors remain relevant in predicting ICU outcome, but the 1993 model significantly overpredicts mortality in contemporary practice. With the advantage of a much larger sample size and the addition of new variables and interaction effects, MPM0-III provides more accurate comparisons of actual vs. expected ICU outcomes.

Inter-hospital variability in post-cardiac arrest mortality

AIM A growing body of evidence suggests that variability in post-cardiac arrest care contributes to differential outcomes of patients with initial return of spontaneous circulation after cardiac arrest. We examined hospital-level variation in mortality of patients admitted to United States intensive care units (ICUs) with a diagnosis of cardiac arrest. METHODS Patients with a primary ICU admission diagnosis of cardiac arrest were identified in the 2002--2005 Acute Physiology and Chronic Health Evaluation (APACHE) IV dataset, a multicenter clinical registry of ICU patients. RESULTS We identified 4674 patients from 39 hospitals. The median number of annual patients was 33 per hospital (range: 12-116). Mean APACHE score was 94 (+/-38), and overall mortality was 56.8%. Age, severity of illness (acute physiology score), and admission Glasgow Coma Scale were all associated with increased mortality (p<0.001). There was no survival difference for patients admitted from the emergency department vs. the inpatient floor. Among institutions, unadjusted in-hospital mortality ranged from 41% to 81%. After adjusting for age and severity of illness, institutional mortality ranged from 46% to 68%. Patients treated at higher volume centers were significantly less likely to die in the hospital. CONCLUSIONS We demonstrate hospital-level variation in severity adjusted mortality among patients admitted to the ICU after cardiac arrest. We identify a volume-outcome relationship showing lower mortality among patients admitted to ICUs that treat a high volume of post-cardiac arrest patients. Prospective studies should identify hospital-level and patient care factors that contribute to post-cardiac arrest survival.

Nighttime Intensivist Staffing and Mortality among Critically Ill Patients

BACKGROUND Hospitals are increasingly adopting 24-hour intensivist physician staffing as a strategy to improve intensive care unit (ICU) outcomes. However, the degree to which nighttime intensivists are associated with improvements in the quality of ICU care is unknown. METHODS We conducted a retrospective cohort study involving ICUs that participated in the Acute Physiology and Chronic Health Evaluation (APACHE) clinical information system from 2009 through 2010, linking a survey of ICU staffing practices with patient-level outcomes data from adult ICU admissions. Multivariate models were used to assess the relationship between nighttime intensivist staffing and in-hospital mortality among ICU patients, with adjustment for daytime intensivist staffing, severity of illness, and case mix. We conducted a confirmatory analysis in a second, population-based cohort of hospitals in Pennsylvania from which less detailed data were available. RESULTS The analysis with the use of the APACHE database included 65,752 patients admitted to 49 ICUs in 25 hospitals. In ICUs with low-intensity daytime staffing, nighttime intensivist staffing was associated with a reduction in risk-adjusted in-hospital mortality (adjusted odds ratio for death, 0.62; P=0.04). Among ICUs with high-intensity daytime staffing, nighttime intensivist staffing conferred no benefit with respect to risk-adjusted in-hospital mortality (odds ratio, 1.08; P=0.78). In the verification cohort, there was a similar relationship among daytime staffing, nighttime staffing, and in-hospital mortality. The interaction between nighttime staffing and daytime staffing was not significant (P=0.18), yet the direction of the findings were similar to those in the APACHE cohort. CONCLUSIONS The addition of nighttime intensivist staffing to a low-intensity daytime staffing model was associated with reduced mortality. However, a reduction in mortality was not seen in ICUs with high-intensity daytime staffing. (Funded by the National Heart, Lung, and Blood Institute.).

Intensive care unit length of stay: Benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV*

Objective:To revise and update the Acute Physiology and Chronic Health Evaluation (APACHE) model for predicting intensive care unit (ICU) length of stay. Design:Observational cohort study. Setting:One hundred and four ICUs in 45 U.S. hospitals. Patients:Patients included 131,618 consecutive ICU admissions during 2002 and 2003, of which 116,209 met inclusion criteria. Interventions:None. Measurements and Main Results:The APACHE IV model for predicting ICU length of stay was developed using ICU day 1 patient data and a multivariate linear regression procedure to estimate the precise ICU stay for randomly selected patients who comprised 60% of the database. New variables were added to the previous APACHE III model, and advanced statistical modeling techniques were used. Accuracy was assessed by comparing mean observed and mean predicted ICU stay for the excluded 40% of patients. Aggregate mean observed ICU stay was 3.86 days and mean predicted 3.78 days (p < .001), a difference of 1.9 hrs. For 108 (93%) of 116 diagnoses, there was no significant difference between mean observed and mean predicted ICU stay. The model accounted for 21.5% of the variation in ICU stay across individual patients and 62% across ICUs. Correspondence between mean observed and mean predicted length of stay was reduced for patients with a short (≤1.7 days) or long (≥9.4 days) ICU stay and a low (<20%) or high (>80%) risk of death on ICU day 1. Conclusions:The APACHE IV model provides clinically useful ICU length of stay predictions for critically ill patient groups, but its accuracy and utility are limited for individual patients. APACHE IV benchmarks for ICU stay are useful for assessing the efficiency of unit throughput and support examination of structural, managerial, and patient factors that affect ICU stay.

Effect of a rapid response system for patients in shock on time to treatment and mortality during 5 years

Objective:Treatment of nontraumatic shock is often delayed or inadequate due to insufficient knowledge or skills of front-line healthcare providers, limited hospital resources, and lack of institution-wide systems to ensure application of best practice. As a result, mortality from shock remains high. We designed a study to determine whether outcomes will be improved by a hospital-wide system that educates and empowers clinicians to rapidly identify and treat patients in shock with a multidisciplinary team using evidenced-based protocols. Design:Single-center trial before and after implementation of a hospital-wide rapid response system for early identification and treatment of patients in shock. Setting:A 180-bed regional referral center in northern California. Patients:A total of 511 adult patients who met criteria for shock during a 7-yr period. Interventions:We designed a rapid response system that included a comprehensive educational program for clinicians on earlier recognition of shock, empowerment of front-line providers using specific criteria to initiate therapy, mobilization of the rapid response team, protocol goal-directed therapy, and early transfer to the intensive care unit. Outcome feedback was provided to foster adoption. Measurements and Main Results:We measured times to key interventions and hospital mortality 2.5 yrs before and until 5 yrs after system initiation. Times to interventions and mortality decreased significantly over time before and after adjusting for confounding factors. Interventions times, including shock alert activation, infusion of 2 L of fluid, central venous catheter placement, and antibiotic administration, were significant predictors of mortality (p < .05). Overall and septic subgroup mortality decreased from before system implementation through protocol year 5 from 40% to 11.8% and from 50% to 10%, respectively (p < .001). Conclusion:Over time, a rapid response system for patients in shock continued to reduce time to treatment, resulting in a continued decrease in mortality. By year 5, only three patients needed to be treated to save one additional life.

Changes in hospital mortality for United States intensive care unit admissions from 1988 to 2012

IntroductionA decrease in disease-specific mortality over the last twenty years has been reported for patients admitted to United States (US) hospitals, but data for intensive care patients are lacking. The aim of this study was to describe changes in hospital mortality and case-mix using clinical data for patients admitted to multiple US ICUs over the last 24 years.MethodsWe carried out a retrospective time series analysis of hospital mortality using clinical data collected from 1988 to 2012. We also examined the impact of ICU admission diagnosis and other clinical characteristics on mortality over time. The potential impact of hospital discharge destination on mortality was also assessed using data from 2001 to 2012.ResultsFor 482,601 ICU admissions there was a 35% relative decrease in mortality from 1988 to 2012 despite an increase in age and severity of illness. This decrease varied greatly by diagnosis. Mortality fell by > 60% for patients with chronic obstructive pulmonary disease, seizures and surgery for aortic dissection and subarachnoid hemorrhage. Mortality fell by 51% to 59% for six diagnoses, 41% to 50% for seven diagnoses, and 10% to 40% for seven diagnoses. The decrease in mortality from 2001 to 2012 was accompanied by an increase in discharge to post-acute care facilities and a decrease in discharge to home.ConclusionsHospital mortality for patients admitted to US ICUs has decreased significantly over the past two decades despite an increase in the severity of illness. Decreases in mortality were diagnosis specific and appear attributable to improvements in the quality of care, but changes in discharge destination and other confounders may also be responsible.

Use of intravenous infusion sedation among mechanically ventilated patients in the United States

Objectives: Many studies compare the efficacy of different forms of intravenous infusion sedation for critically ill patients, but little is known about the actual use of these medications. We sought to describe current use of intravenous infusion sedation in mechanically ventilated patients in U.S. intensive care units. Design: Retrospective cohort study of intravenous infusion sedation among mechanically ventilated patients. Intravenous sedatives examined included benzodiazepines (midazolam and lorazepam), propofol, and dexmedetomidine. Use was defined as having received an intravenous infusion for any time period during the stay in intensive care. Setting: One hundred seventy-four intensive care units contributing data to Project IMPACT from 2001 through 2007. Patients: All patients who received mechanical ventilation. Interventions: None. Measurements and Main Results: Of 109,671 mechanically ventilated patients, 56,443 (51.5%, 95% confidence interval 51.2–51.8) received one or more intravenous infusion sedatives. Sedative use increased over time, from 39.7% (38.7–40.6) of patients in 2001 to 66.7% (65.7–67.7) in 2007 (p < .001). Most patients who received intravenous infusion sedation received propofol (82.2%, 81.9–82.5) vs. benzodiazepines (31.1%, 30.7–31.5) or dexmedetomidine (4.0%, 3.8–4.2). Of the patients, 66.2% (65.8–66.6) received only propofol, and 16.2% (15.9–16.5) only benzodiazepines. Among patients mechanically ventilated >96 hrs, propofol infusions were more common. Intravenous infusion narcotics (fentanyl, morphine, or hydromorphone) were used more frequently among patients who received benzodiazepines (70.1%, 69.1–71.0) compared with propofol (23.9%, 23.5–24.3), p < .001. Conclusions: The percentage of mechanically ventilated patients receiving intravenous infusion sedation has increased over time. Sedation with an infusion of propofol was much more common than with benzodiazepines or dexmedetomidine, even for patients mechanically ventilated beyond 96 hrs.

Intensive care unit readmissions in U.s. hospitals: Patient characteristics, risk factors, and outcomes*

Objective:To examine which patient characteristics increase the risk for intensive care unit readmission and assess the association of readmission with case-mix adjusted mortality and resource use. Design:Retrospective cohort study. Setting:Ninety-seven intensive and cardiac care units at 35 hospitals in the United States. Patients:A total of 229,375 initial intensive care unit admissions during 2001 through 2009 who met inclusion criteria. Interventions:None. Measurements and Main Results:For patients who were discharged alive and candidates for readmission, we compared the characteristics of those with and without a readmission. A multivariable logistic regression analysis was used to identify potential patient-level characteristics that increase the risk for subsequent readmission. We also evaluated case-mix adjusted outcomes by comparing observed and predicted values of mortality and length of stay for patients with and without intensive care unit readmission. Among 229,375 first admissions that met inclusion criteria, 13,980 (6.1%) were eventually readmitted. Risk factors associated with the highest multivariate odds ratio for unit readmission included location before intensive care unit admission, age, comorbid conditions, diagnosis, intensive care unit length of stay, physiologic abnormalities at intensive care discharge, and discharge to a step-down unit. After adjustment for risk factors, patients who were readmitted had a four-fold greater probability of hospital mortality and a 2.5-fold increase in hospital stay compared to patients without readmission. Conclusions:Intensive care readmission is associated with patient factors that reflect a greater severity and complexity of illness, resulting in a higher risk for hospital mortality and a longer hospital stay. To improve patient safety, physicians should consider these risk factors when making intensive care discharge decisions. Because intensive care unit readmission correlates with more complex and severe illness, readmission rates require case-mix adjustment before they can be properly interpreted as quality measures.