Larry Nielsen

Predicting ICU hemodynamic instability using continuous multiparameter trends

Background: Identifying hemodynamically unstable patients in a timely fashion in intensive care units (ICUs) is crucial because it can lead to earlier interventions and thus to potentially better patient outcomes. Current alert algorithms are typically limited to detecting dangerous conditions only after they have occurred and suffer from high false alert rates. Our objective was to predict hemodynamic instability at least two hours before a major clinical intervention (e.g., vasopressor administration), while maintaining a low false alert rate. Study population: From the MIMIC II database, containing ICU minute-by-minute heart rate (HR) and invasive arterial blood pressure (BP) monitoring trend data collected between 2001 and 2005, we identified 132 stable and 104 unstable patients that met our stability-instability criteria and had sufficient data points. Method: We first derived additional physiological parameters of shock index, rate pressure product, heart rate variability, and two measures of trending based on HR and BP. Then we developed 220 statistical features and systematically selected a small set to use for classification. We applied multi-variable logistic regression modeling to do classification and implemented validation via bootstrapping. Results: Area under receiver-operating curve (ROC) 0.83±0.03, sensitivity 0.75±0.06, and specificity 0.80±0.07; if the specificity is targeted at 0.90, then the sensitivity is 0.57±0.07. Based on our preliminary results, we conclude that the algorithms we developed using HR and BP trend data may provide a promising perspective toward reliable predictive alerts for hemodynamically unstable patients.

Predicting respiratory instability in the ICU

Acute lung injury (ALI) and acute respiratory distress syndrome (ARDS) contribute to the morbidity and mortality of intensive care patients worldwide, and have large associated human and financial costs. We identified a reference data set of 624 mechanically-ventilated patients in the MIMIC-II intensive care database with and without low PaO2/FiO2 ratios (termed respiratory instability), and developed prediction algorithms for distinguishing these patients prior to the critical event. In the end, we had four rule sets using mean airway pressure, plateau pressure, total respiratory rate and oxygen saturation (SpO2), where the specificity/sensitivity rates were either 80%/60% or 90%/50%.

Hemodynamic Instability Prediction Through Continuous Multiparameter Monitoring in ICU

Current algorithms identifying hemodynamically unstable intensive care unit patients typically are limited to detecting existing dangerous conditions and suffer from high false alert rates. Our objective was to predict hemodynamic instability at least two hours before patient deterioration while maintaining a low false alert rate, using minute-by-minute heart rate (HR) and blood pressure (BP) data. We identified 66 stable and 104 unstable patients meeting our stability-instability criteria from the MIMIC II database, and developed multi-parameter measures using HR and BP. An instability index combining measures of BP, shock index, rate pressure product, and HR variation was developed from a multivariate regression model to predict hemodynamic instability (ROC of 0.82±0.03, sensitivity of 0.57±0.07 when the specificity was targeted at 0.90; the alert rate ratio of unstable to stable patients was 7.62). We conclude that these algorithms could form the basis for reliable predictive clinical alerts which identify patients likely to become hemodynamically unstable within the next few hours so that the clinicians can proactively manage these patients and provide necessary care.

Estimation of the patient monitor alarm rate for a quantitative analysis of new alarm settings.

In many critical care units, default patient monitor alarm settings are not fine-tuned to the vital signs of the patient population. As a consequence there are many alarms. A large fraction of the alarms are not clinically actionable, thus contributing to alarm fatigue. Recent attention to this phenomenon has resulted in attempts in many institutions to decrease the overall alarm load of clinicians by altering the trigger thresholds for monitored parameters. Typically, new alarm settings are defined based on clinical knowledge and patient population norms and tried empirically on new patients without quantitative knowledge about the potential impact of these new settings. We introduce alarm regeneration as a method to estimate the alarm rate of new alarm settings using recorded patient monitor data. This method enables evaluation of several alarm setting scenarios prior to using these settings in the clinical setting. An expression for the alarm rate variance is derived for the calculation of statistical confidence intervals on the results.

Heuristics to determine ventilation times of ICU patients from the MIMIC-II database

Mechanical ventilation is an important life support tool for patients in intensive care units (ICU). For various research purposes related to patient hemodynamic and cardiopulmonary monitoring, it is important to know when a patient is on a ventilator. Unfortunately, the widely used MIMIC-II database contains results from user charted data, where the user did not always store ventilation on and off times explicitly and accurately. The resulting ventilation-related data are subject to error. Therefore, there are no simple rules to define ventilation times retrospectively for this dataset. Hence, we designed a simple set of rules to determine the ventilation times using multiple sources of mechanical ventilator-related settings and physiological measurements by expert heuristics. The rules worked well in comparison with nursing notes regarding ventilation events. We conclude that our rule sets for determining ventilation times may be useful in assisting with MIMIC-II database analysis.

A Methodology for Evaluating the Performance of Alerting and Detection Algorithms Running on Continuous Patient Data

Objectives Clinicians in the intensive care unit (ICU) are presented with a large number of physiological data consisting of periodic and frequently sampled measurements, such as heart rate and blood pressure, as well as aperiodic measurements, such as noninvasive blood pressure and laboratory studies. Because this data can be overwhelming, there is considerable interest in designing algorithms that help integrate and interpret this data and assist ICU clinicians in detecting or predicting in advance patients who may be deteriorating. In order to decide whether to deploy such algorithms in a clinical trial, it is important to evaluate these algorithms using retrospective data. However, the fact that these algorithms will be running continuously, i.e., repeatedly sampling incoming patient data, presents some novel challenges for algorithm evaluation. Commonly used measures of performance such as sensitivity and positive predictive value (PPV) are easily applied to static “snapshots” of patient data, but can be very misleading when applied to indicators or alerting algorithms that are running on continuous data. Our objective is to create a method for evaluating algorithm performance on retrospective data with the algorithm running continuously throughout the patient’s stay as it would in a real ICU. Methods We introduce our evaluation methodology in the context of evaluating an algorithm, a Hemodynamic Instability Indicator (HII), for assisting bedside ICU clinicians with the early detection of hemodynamic instability before the onset of acute hypotension. Each patient’s ICU stay is divided into segments that are labelled as hemodynamically stable or unstable based on clinician interventions typically aimed at treating hemodynamic instability. These segments can be of varying length with varying degrees of exposure to potential alerts, whether true positive or false positive. Furthermore, to simulate how clinicians might interact with the alerting algorithm, we use a dynamic alert supervision mechanism which suppresses subsequent alerts unless the indicator has significantly deteriorated since the prior alert. Under these conditions determining what counts as a positive or negative instance, and calculations of sensitivity, specificity, and positive predictive value can be problematic. We introduce a methodology for consistently counting positive and negative instances. The methodology distinguishes between counts based on alerting events and counts based on sub-segments, and show how they can be applied in calculating measures of performance such as sensitivity, specificity, positive predictive value. Results The introduced methodology is applied to retrospective evaluation of two algorithms, HII and an alerting algorithm based on systolic blood pressure. We use a database, consisting of data from 41,707 patients from 25 US hospitals, to evaluate the algorithms. Both algorithms are evaluated running continuously throughout each patient’s stay as they would in a real ICU setting. We show how the introduced performance measures differ for different algorithms and for different assumptions. Discussion The standard measures of diagnostic tests in terms of true positives, false positives, etc. are based on certain assumptions which may not apply when used in the context of measuring the performance on an algorithm running continuously, and thus repeatedly sampling from the same patient. When such measures are being reported it is important that the underlying assumptions be made explicit; otherwise, the results can be very misleading. Conclusion We introduce a methodology for evaluating how an alerting algorithm or indicator will perform running continuously throughout every patient’s ICU stay, not just for a subset of patients for selected episodes.