Benjamin J. Turk

Stratification of Risk of Early-Onset Sepsis in Newborns ≥34 Weeks’ Gestation

OBJECTIVE: To define a quantitative stratification algorithm for the risk of early-onset sepsis (EOS) in newborns ≥34 weeks’ gestation. METHODS: We conducted a retrospective nested case-control study that used split validation. Data collected on each infant included sepsis risk at birth based on objective maternal factors, demographics, specific clinical milestones, and vital signs during the first 24 hours after birth. Using a combination of recursive partitioning and logistic regression, we developed a risk classification scheme for EOS on the derivation dataset. This scheme was then applied to the validation dataset. RESULTS: Using a base population of 608 014 live births ≥34 weeks’ gestation at 14 hospitals between 1993 and 2007, we identified all 350 EOS cases <72 hours of age and frequency matched them by hospital and year of birth to 1063 controls. Using maternal and neonatal data, we defined a risk stratification scheme that divided the neonatal population into 3 groups: treat empirically (4.1% of all live births, 60.8% of all EOS cases, sepsis incidence of 8.4/1000 live births), observe and evaluate (11.1% of births, 23.4% of cases, 1.2/1000), and continued observation (84.8% of births, 15.7% of cases, incidence 0.11/1000). CONCLUSIONS: It is possible to combine objective maternal data with evolving objective neonatal clinical findings to define more efficient strategies for the evaluation and treatment of EOS in term and late preterm infants. Judicious application of our scheme could result in decreased antibiotic treatment in 80 000 to 240 000 US newborns each year.

An electronic Simplified Acute Physiology Score-based risk adjustment score for critical illness in an integrated healthcare system.

Objective:Risk adjustment is essential in evaluating the performance of an ICU; however, assigning scores is time-consuming. We sought to create an automated ICU risk adjustment score, based on the Simplified Acute Physiology Score 3, using only data available within the electronic medical record (Kaiser Permanente HealthConnect). Design, Setting, and Patients:The eSimplified Acute Physiology Score 3 was developed by adapting Kaiser Permanente HealthConnect structured data to Simplified Acute Physiology Score 3 criteria. The model was tested among 67,889 first-time ICU admissions at 21 hospitals between 2007 and 2011 to predict hospital mortality. Model performance was evaluated using published Simplified Acute Physiology Score 3 global and North American coefficients; a first-level customized version of the eSimplified Acute Physiology Score 3 was also developed in a 40% derivation cohort and tested in a 60% validation cohort. Measurements:Electronic variables were considered “directly” available if they could be mapped exactly within Kaiser Permanente HealthConnect; they were considered “adapted” if no exact electronic corollary was identified. Model discrimination was evaluated with area under receiver operating characteristic curves; calibration was assessed using Hosmer–Lemeshow goodness-of-fit tests. Main Results:Mean age at ICU admission was 65 ± 17 yrs. Mortality in the ICU was 6.2%; total in-hospital mortality was 11.2%. The majority of Simplified Acute Physiology Score 3 variables were considered “directly” available; others required adaptation based on diagnosis coding, medication records, or procedure tables. Mean eSimplified Acute Physiology Score 3 scores were 45 ± 13. Using published Simplified Acute Physiology Score 3 global and North American coefficients, the eSimplified Acute Physiology Score 3 demonstrated good discrimination (area under the receiver operating characteristic curve, 0.80–0.81); however, it overpredicted mortality. The customized eSimplified Acute Physiology Score 3 score demonstrated good discrimination (area under the receiver operating characteristic curve, 0.82) and calibration (Hosmer–Lemeshow goodness-of-fit chi-square p = 0.57) in the validation cohort. The eSimplified Acute Physiology Score 3 demonstrated stable performance when cohorts were limited to specific hospitals and years. Conclusions:The customized eSimplified Acute Physiology Score 3 shows good potential for providing automated risk adjustment in the intensive care unit.

Automated identification of pneumonia in chest radiograph reports in critically ill patients.

BackgroundPrior studies demonstrate the suitability of natural language processing (NLP) for identifying pneumonia in chest radiograph (CXR) reports, however, few evaluate this approach in intensive care unit (ICU) patients.MethodsFrom a total of 194,615 ICU reports, we empirically developed a lexicon to categorize pneumonia-relevant terms and uncertainty profiles. We encoded lexicon items into unique queries within an NLP software application and designed an algorithm to assign automated interpretations (‘positive’, ‘possible’, or ‘negative’) based on each report’s query profile. We evaluated algorithm performance in a sample of 2,466 CXR reports interpreted by physician consensus and in two ICU patient subgroups including those admitted for pneumonia and for rheumatologic/endocrine diagnoses.ResultsMost reports were deemed ‘negative’ (51.8%) by physician consensus. Many were ‘possible’ (41.7%); only 6.5% were ‘positive’ for pneumonia. The lexicon included 105 terms and uncertainty profiles that were encoded into 31 NLP queries. Queries identified 534,322 ‘hits’ in the full sample, with 2.7 ± 2.6 ‘hits’ per report. An algorithm, comprised of twenty rules and probability steps, assigned interpretations to reports based on query profiles. In the validation set, the algorithm had 92.7% sensitivity, 91.1% specificity, 93.3% positive predictive value, and 90.3% negative predictive value for differentiating ‘negative’ from ‘positive’/’possible’ reports. In the ICU subgroups, the algorithm also demonstrated good performance, misclassifying few reports (5.8%).ConclusionsMany CXR reports in ICU patients demonstrate frank uncertainty regarding a pneumonia diagnosis. This electronic tool demonstrates promise for assigning automated interpretations to CXR reports by leveraging both terms and uncertainty profiles.

Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU

BACKGROUND Patients in general medical-surgical wards who experience unplanned transfer to the intensive care unit (ICU) show evidence of physiologic derangement 6-24h prior to their deterioration. With increasing availability of electronic medical records (EMRs), automated early warning scores (EWSs) are becoming feasible. OBJECTIVE To describe the development and performance of an automated EWS based on EMR data. MATERIALS AND METHODS We used a discrete-time logistic regression model to obtain an hourly risk score to predict unplanned transfer to the ICU within the next 12h. The model was based on hospitalization episodes from all adult patients (18years) admitted to 21 Kaiser Permanente Northern California (KPNC) hospitals from 1/1/2010 to 12/31/2013. Eligible patients met these entry criteria: initial hospitalization occurred at a KPNC hospital; the hospitalization was not for childbirth; and the EMR had been operational at the hospital for at least 3months. We evaluated the performance of this risk score, called Advanced Alert Monitor (AAM) and compared it against two other EWSs (eCART and NEWS) in terms of their sensitivity, specificity, negative predictive value, positive predictive value, and area under the receiver operator characteristic curve (c statistic). RESULTS A total of 649,418 hospitalization episodes involving 374,838 patients met inclusion criteria, with 19,153 of the episodes experiencing at least one outcome. The analysis data set had 48,723,248 hourly observations. Predictors included physiologic data (laboratory tests and vital signs); neurological status; severity of illness and longitudinal comorbidity indices; care directives; and health services indicators (e.g. elapsed length of stay). AAM showed better performance compared to NEWS and eCART in all the metrics and prediction intervals. The AAM AUC was 0.82 compared to 0.79 and 0.76 for eCART and NEWS, respectively. Using a threshold that generated 1 alert per day in a unit with a patient census of 35, the sensitivity of AAM was 49% (95% CI: 47.6-50.3%) compared to the sensitivities of eCART and NEWS scores of 44% (42.3-45.1) and 40% (38.2-40.9), respectively. For all three scores, about half of alerts occurred within 12h of the event, and almost two thirds within 24h of the event. CONCLUSION The AAM score is an example of a score that takes advantage of multiple data streams now available in modern EMRs. It highlights the ability to harness complex algorithms to maximize signal extraction. The main challenge in the future is to develop detection approaches for patients in whom data are sparser because their baseline risk is lower.

Piloting electronic medical record–based early detection of inpatient deterioration in community hospitals

Patients who deteriorate in the hospital outside the intensive care unit (ICU) have higher mortality and morbidity than those admitted directly to the ICU. As more hospitals deploy comprehensive inpatient electronic medical records (EMRs), attempts to support rapid response teams with automated early detection systems are becoming more frequent. We aimed to describe some of the technical and operational challenges involved in the deployment of an early detection system. This 2-hospital pilot, set within an integrated healthcare delivery system with 21 hospitals, had 2 objectives. First, it aimed to demonstrate that severity scores and probability estimates could be provided to hospitalists in real time. Second, it aimed to surface issues that would need to be addressed so that deployment of the early warning system could occur in all remaining hospitals. To achieve these objectives, we first established a rationale for the development of an early detection system through the analysis of risk-adjusted outcomes. We then demonstrated that EMR data could be employed to predict deteriorations. After addressing specific organizational mandates (eg, defining the clinical response to a probability estimate), we instantiated a set of equations into a Java application that transmits scores and probability estimates so that they are visible in a commercially available EMR every 6 hours. The pilot has been successful and deployment to the remaining hospitals has begun. Journal of Hospital Medicine 2016;11:S18-S24.

Evaluation Following Staggered Implementation of the "Rethinking Critical Care" ICU Care Bundle in a Multicenter Community Setting.

Objectives:To evaluate process metrics and outcomes after implementation of the “Rethinking Critical Care” ICU care bundle in a community setting. Design:Retrospective interrupted time-series analysis. Setting:Three hospitals in the Kaiser Permanente Northern California integrated healthcare delivery system. Patients:ICU patients admitted between January 1, 2009, and August 30, 2013. Interventions:Implementation of the Rethinking Critical Care ICU care bundle which is designed to reduce potentially preventable complications by focusing on the management of delirium, sedation, mechanical ventilation, mobility, ambulation, and coordinated care. Rethinking Critical Care implementation occurred in a staggered fashion between October 2011 and November 2012. Measurements and Main Results:We measured implementation metrics based on electronic medical record data and evaluated the impact of implementation on mortality with multivariable regression models for 24,886 first ICU episodes in 19,872 patients. After implementation, some process metrics (e.g., ventilation start and stop times) were achieved at high rates, whereas others (e.g., ambulation distance), available late in the study period, showed steep increases in compliance. Unadjusted mortality decreased from 12.3% to 10.9% (p < 0.01) before and after implementation, respectively. The adjusted odds ratio for hospital mortality after implementation was 0.85 (95% CI, 0.73–0.99) and for 30-day mortality was 0.88 (95% CI, 0.80–0.97) compared with before implementation. However, the mortality rate trends were not significantly different before and after Rethinking Critical Care implementation. The mean duration of mechanical ventilation and hospital stay also did not demonstrate incrementally greater declines after implementation. Conclusions:Rethinking Critical Care implementation was associated with changes in practice and a 12–15% reduction in the odds of short-term mortality. However, these findings may represent an evaluation of changes in practices and outcomes still in the midimplementation phase and cannot be directly attributed to the elements of bundle implementation.

The Association Between Sepsis and Potential Medical Injury Among Hospitalized Patients

BACKGROUND Patient safety remains a national priority, but the role of disease-specific characteristics in safety is not well characterized. METHODS We identified potentially preventable medical injuries using patient safety indicators (PSIs) and annual data from the Nationwide Inpatient Sample between 2003 and 2007. We compared the rate of selected PSIs among patients hospitalized with and without sepsis. Among patients with sepsis, we also compared PSI rates across severity strata. Using multivariable case-control matching and regression analyses, we estimated the excess adverse outcomes associated with PSI events in patients with sepsis. RESULTS Patients hospitalized with sepsis accounted for 2% to 4% of hospital discharges; however, they accounted for 9% to 26% of all potential medical injuries. PSI rates varied considerably; among patients hospitalized for sepsis, they were lowest for accidental puncture or laceration and highest for postoperative respiratory failure. Nearly all PSI rates were higher among patients with sepsis compared with patients without sepsis. Among those with sepsis, most PSI rates increased as sepsis severity increased. Compared with matched sepsis control subjects, increased length of stay and hospital charges were associated with PSI events in sepsis cases. However, only decubitus ulcer, iatrogenic pneumothorax, and postoperative metabolic and physiologic derangement or respiratory failure were associated with excess mortality. CONCLUSION Patients hospitalized for sepsis, compared with the general hospital population, were at a substantially increased risk of potential medical injury; their risk rose as disease severity increased. Future patient safety efforts may benefit from focusing on medically vulnerable populations.