Rondall K. Lane

Variation in ICU Risk-Adjusted Mortality: Impact of Methods of Assessment and Potential Confounders

BACKGROUND Federal and state agencies are considering ICU performance assessment and public reporting; however, an accurate method for measuring performance must be selected. In this study, we determine whether a substantial variation in ICU mortality performance still exists in modern ICUs, and compare the predictive accuracy, reliability, and data burden of existing ICU risk-adjustment models. METHODS A retrospective chart review of 11,300 ICU patients from 35 California hospitals from 2001 to 2004 was performed. We calculated standardized mortality ratios (SMRs) for each hospital using the mortality probability model III (MPM(0) III), the simplified acute physiology score (SAPS) II, and the acute physiology and chronic health evaluation (APACHE) IV risk-adjustment models. We compared discrimination, calibration, data reliability, and abstraction time for the models. RESULTS Regardless of the model used, there was a large variation in SMRs among the ICUs studied. The discrimination and calibration were adequate for all risk-adjustment models. APACHE IV had the best discrimination (area under the receiver operating characteristic curve [AUC], 0.892) compared to MPM(0) III (AUC, 0.809), and SAPS II (AUC, 0.873; p < 0.001). The models differed substantially in data abstraction times, as follows: MPM(0)III, 11.1 min (95% confidence interval [CI], 8.7 to 13.4); SAPS II, 19.6 min (95% CI, 17.0 to 22.2); and APACHE IV, 37.3 min (95% CI, 28.0 to 46.6). CONCLUSIONS We found substantial variation in the ICU risk-adjusted mortality rates that persisted regardless of the risk-adjustment model. With unlimited resources, the APACHE IV model offers the best predictive accuracy. If constrained by cost and manual data collection, the MPM(0) III model offers a viable alternative without a substantial loss in accuracy.

Mortality Probability Model III and Simplified Acute Physiology Score II: Assessing Their Value in Predicting Length of Stay and Comparison to APACHE IV

BACKGROUND To develop and compare ICU length-of-stay (LOS) risk-adjustment models using three commonly used mortality or LOS prediction models. METHODS Between 2001 and 2004, we performed a retrospective, observational study of 11,295 ICU patients from 35 hospitals in the California Intensive Care Outcomes Project. We compared the accuracy of the following three LOS models: a recalibrated acute physiology and chronic health evaluation (APACHE) IV-LOS model; and models developed using risk factors in the mortality probability model III at zero hours (MPM(0)) and the simplified acute physiology score (SAPS) II mortality prediction model. We evaluated models by calculating the following: (1) grouped coefficients of determination; (2) differences between observed and predicted LOS across subgroups; and (3) intraclass correlations of observed/expected LOS ratios between models. RESULTS The grouped coefficients of determination were APACHE IV with coefficients recalibrated to the LOS values of the study cohort (APACHE IVrecal) [R(2) = 0.422], mortality probability model III at zero hours (MPM(0) III) [R(2) = 0.279], and simplified acute physiology score (SAPS II) [R(2) = 0.008]. For each decile of predicted ICU LOS, the mean predicted LOS vs the observed LOS was significantly different (p <or= 0.05) for three, two, and six deciles using APACHE IVrecal, MPM(0) III, and SAPS II, respectively. Plots of the predicted vs the observed LOS ratios of the hospitals revealed a threefold variation in LOS among hospitals with high model correlations. CONCLUSIONS APACHE IV and MPM(0) III were more accurate than SAPS II for the prediction of ICU LOS. APACHE IV is the most accurate and best calibrated model. Although it is less accurate, MPM(0) III may be a reasonable option if the data collection burden or the treatment effect bias is a consideration.

Original ResearchCritical Care MedicineMortality Probability Model III and Simplified Acute Physiology Score II: Assessing Their Value in Predicting Length of Stay and Comparison to APACHE IV

BACKGROUND To develop and compare ICU length-of-stay (LOS) risk-adjustment models using three commonly used mortality or LOS prediction models. METHODS Between 2001 and 2004, we performed a retrospective, observational study of 11,295 ICU patients from 35 hospitals in the California Intensive Care Outcomes Project. We compared the accuracy of the following three LOS models: a recalibrated acute physiology and chronic health evaluation (APACHE) IV-LOS model; and models developed using risk factors in the mortality probability model III at zero hours (MPM(0)) and the simplified acute physiology score (SAPS) II mortality prediction model. We evaluated models by calculating the following: (1) grouped coefficients of determination; (2) differences between observed and predicted LOS across subgroups; and (3) intraclass correlations of observed/expected LOS ratios between models. RESULTS The grouped coefficients of determination were APACHE IV with coefficients recalibrated to the LOS values of the study cohort (APACHE IVrecal) [R(2) = 0.422], mortality probability model III at zero hours (MPM(0) III) [R(2) = 0.279], and simplified acute physiology score (SAPS II) [R(2) = 0.008]. For each decile of predicted ICU LOS, the mean predicted LOS vs the observed LOS was significantly different (p <or= 0.05) for three, two, and six deciles using APACHE IVrecal, MPM(0) III, and SAPS II, respectively. Plots of the predicted vs the observed LOS ratios of the hospitals revealed a threefold variation in LOS among hospitals with high model correlations. CONCLUSIONS APACHE IV and MPM(0) III were more accurate than SAPS II for the prediction of ICU LOS. APACHE IV is the most accurate and best calibrated model. Although it is less accurate, MPM(0) III may be a reasonable option if the data collection burden or the treatment effect bias is a consideration.

Central line infections.

Central venous catheters are commonly used in the critical care setting. Unfortunately, their use is often associated with complications, including fatal infections. Making the diagnosis of central venous catheter infection can be difficult. Additionally, resistance among the more common organisms that cause catheter-related infection is increasing. However, our understanding of the pathogenesis of catheter infection is improving through examination of biofilms. Also, our ability to diagnose catheter-related infections more accurately is improving with new techniques. There is new hope for ruling out catheter-related infection before removal by several methods, including a rapid enzyme-linked immunosorbent assay and the use of time differential for microbial growth between blood cultures obtained from a peripheral site and the catheter itself. Prevention through the use of barrier techniques and antimicrobial-coated catheters has been demonstrated to be of value in reducing catheter-related infection with these devices.

Advance care planning norms may contribute to hospital variation in end-of-life ICU use: a simulation study.

Background. There is wide variation in end-of-life (EOL) intensive care unit (ICU) use among academic medical centers (AMCs). Our objective was to develop hypotheses regarding medical decision-making factors underlying this variation. Methods. This was a high-fidelity simulation experiment involving a critically and terminally ill elder, followed by a survey and debriefing cognitive interview and evaluated using triangulated quantitative-qualitative comparative analysis. The study was conducted in 2 AMCs in the same state and health care system with disparate EOL ICU use. Subjects were hospital-based physicians responsible for ICU admission decisions. Measurements included treatment plan, prognosis, diagnosis, qualitative case perceptions, and clinical reasoning. Results. Sixty-seven of 111 (60%) eligible physicians agreed to participate; 48 (72%) could be scheduled. There were no significant between-AMC differences in 3-month prognosis or treatment plan, but there were systematic differences in perceptions of the case. Case perceptions at the low-intensity AMC seemed to be influenced by the absence of a do-not-resuscitate order in the context of norms of universal code status discussion and documentation upon admission, whereas case perceptions at the high-intensity AMC seemed to be influenced by the patient’s known metastatic gastric cancer in the context of norms of oncologists’ avoiding code status discussions. Conclusions: In this simulation study of 2 AMCs, hospital-based physicians had different perceptions of an identical case. We hypothesize that different advance care planning norms may have influenced their decision-making heuristics.

Predictors of early postdischarge mortality in critically ill patients: a retrospective cohort study from the California Intensive Care Outcomes project.

PURPOSE Existing intensive care unit (ICU) mortality measurement systems address in-hospital mortality only. However, early postdischarge mortality contributes significantly to overall 30-day mortality. Factors associated with early postdischarge mortality are unknown. METHODS We performed a retrospective study of 8484 ICU patients. Our primary outcome was early postdischarge mortality: death after hospital discharge and 30 days or less from ICU admission. Cox regression models assessed the association between patient, hospital, and utilization factors and the primary outcome. RESULTS In multivariate analyses, the hazard for early postdischarge mortality increased with rising severity of illness and decreased with full-code status (hazard ratio [HR], 0.33; 95% confidence interval [CI], 0.21-0.49). Compared with discharges home, early postdischarge mortality was highest for acute care transfers (HR, 3.18; 95% CI, 2.45-4.12). Finally, patients with very short ICU length of stay (<1 day) had greater early postdischarge mortality (HR, 1.86; 95% CI; 1.32-2.61) than those with longest stays (≥7 days). CONCLUSIONS Early postdischarge mortality is associated with patient preferences (full-code status) and decisions regarding timing and location of discharge. These findings have important implications for anyone attempting to measure or improve ICU performance and who rely on in-hospital mortality measures to do so.