Grant S. Fletcher

International Validity of the HOSPITAL Score to Predict 30-Day Potentially Avoidable Hospital Readmissions

IMPORTANCE Identification of patients at a high risk of potentially avoidable readmission allows hospitals to efficiently direct additional care transitions services to the patients most likely to benefit. OBJECTIVE To externally validate the HOSPITAL score in an international multicenter study to assess its generalizability. DESIGN, SETTING, AND PARTICIPANTS International retrospective cohort study of 117 065 adult patients consecutively discharged alive from the medical department of 9 large hospitals across 4 different countries between January 2011 and December 2011. Patients transferred to another acute care facility were excluded. EXPOSURES The HOSPITAL score includes the following predictors at discharge: hemoglobin, discharge from an oncology service, sodium level, procedure during the index admission, index type of admission (urgent), number of admissions during the last 12 months, and length of stay. MAIN OUTCOMES AND MEASURES 30-day potentially avoidable readmission to the index hospital using the SQLape algorithm. RESULTS Overall, 117 065 adults consecutively discharged alive from a medical department between January 2011 and December 2011 were studied. Of all medical discharges, 16 992 of 117 065 (14.5%) were followed by a 30-day readmission, and 11 307 (9.7%) were followed by a 30-day potentially avoidable readmission. The discriminatory power of the HOSPITAL score to predict potentially avoidable readmission was good, with a C statistic of 0.72 (95% CI, 0.72-0.72). As in the derivation study, patients were classified into 3 risk categories: low (n = 73 031 [62.4%]), intermediate (n = 27 612 [23.6%]), and high risk (n = 16 422 [14.0%]). The estimated proportions of potentially avoidable readmission for each risk category matched the observed proportion, resulting in an excellent calibration (Pearson χ2 test P = .89). CONCLUSIONS AND RELEVANCE The HOSPITAL score identified patients at high risk of 30-day potentially avoidable readmission with moderately high discrimination and excellent calibration when applied to a large international multicenter cohort of medical patients. This score has the potential to easily identify patients in need of more intensive transitional care interventions to prevent avoidable hospital readmissions.

Transition from paper to electronic inpatient physician notes

UW Medicine teaching hospitals have seen a move from paper to electronic physician inpatient notes, after improving the availability of workstations, and wireless laptops and the technical infrastructure supporting the electronic medical record (EMR). The primary driver for the transition was to unify the medical record for all disciplines in one location. The main barrier faced was the time required to enter notes, which was addressed with data-rich templates tailored to rounding workflow, simplified login and other measures. After a 2-year transition, nearly all physician notes for hospitalized patients are now entered electronically, approximately 1500 physician notes per day. Remaining challenges include time for note entry, and the perception that notes may be more difficult to understand and to find within the EMR. In general, the transition from paper to electronic notes has been regarded as valuable to patient care and hospital operations.

Use of more than one electronic medical record system within a single health care organization

Healthcare organizations vary in the number of electronic medical record (EMR) systems they use. Some use a single EMR for nearly all care they provide, while others use EMRs from more than one vendor. These strategies create a mixture of advantages, risks and costs. Based on our experience in two organizations over a decade, we analyzed use of more than one EMR within our two health care organizations to identify advantages, risks and costs that use of more than one EMR presents. We identified the data and functionality types that pose the greatest challenge to patient safety and efficiency. We present a model to classify patterns of use of more than one EMR within a single healthcare organization, and identified the most important 28 data types and 4 areas of functionality that in our experience present special challenges and safety risks with use of more than one EMR within a single healthcare organization. The use of more than one EMR in a single organization may be the chosen approach for many reasons, but in our organizations the limitations of this approach have also become clear. Those who use and support EMRs realize that to safely and efficiently use more than one EMR, a considerable amount of IT work is necessary. Thorough understanding of the challenges in using more than one EMR is an important prerequisite to minimizing the risks of using more than one EMR to care for patients in a single healthcare organization.

The Hospital Score Predicts Potentially Preventable 30-day Readmissions in Conditions Targeted by the Hospital Readmissions Reduction Program

Background/Objectives: New tools to accurately identify potentially preventable 30-day readmissions are needed. The HOSPITAL score has been internationally validated for medical inpatients, but its performance in select conditions targeted by the Hospital Readmission Reduction Program (HRRP) is unknown. Design: Retrospective cohort study. Setting: Six geographically diverse medical centers. Participants/Exposures: All consecutive adult medical patients discharged alive in 2011 with 1 of the 4 medical conditions targeted by the HRRP (acute myocardial infarction, chronic obstructive pulmonary disease, pneumonia, and heart failure) were included. Potentially preventable 30-day readmissions were identified using the SQLape algorithm. The HOSPITAL score was calculated for all patients. Measurements: A multivariable logistic regression model accounting for hospital effects was used to evaluate the accuracy (Brier score), discrimination (c-statistic), and calibration (Pearson goodness-of-fit) of the HOSPITAL score for each 4 medical conditions. Results: Among the 9181 patients included, the overall 30-day potentially preventable readmission rate was 13.6%. Across all 4 diagnoses, the HOSPITAL score had very good accuracy (Brier score of 0.11), good discrimination (c-statistic of 0.68), and excellent calibration (Hosmer-Lemeshow goodness-of-fit test, P=0.77). Within each diagnosis, performance was similar. In sensitivity analyses, performance was similar for all readmissions (not just potentially preventable) and when restricted to patients age 65 and above. Conclusions: The HOSPITAL score identifies a high-risk cohort for potentially preventable readmissions in a variety of practice settings, including conditions targeted by the HRRP. It may be a valuable tool when included in interventions to reduce readmissions within or across these conditions.

Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU

Objectives We validate a machine learning-based sepsis-prediction algorithm (InSight) for the detection and prediction of three sepsis-related gold standards, using only six vital signs. We evaluate robustness to missing data, customisation to site-specific data using transfer learning and generalisability to new settings. Design A machine-learning algorithm with gradient tree boosting. Features for prediction were created from combinations of six vital sign measurements and their changes over time. Setting A mixed-ward retrospective dataset from the University of California, San Francisco (UCSF) Medical Center (San Francisco, California, USA) as the primary source, an intensive care unit dataset from the Beth Israel Deaconess Medical Center (Boston, Massachusetts, USA) as a transfer-learning source and four additional institutions’ datasets to evaluate generalisability. Participants 684 443 total encounters, with 90 353 encounters from June 2011 to March 2016 at UCSF. Interventions None. Primary and secondary outcome measures Area under the receiver operating characteristic (AUROC) curve for detection and prediction of sepsis, severe sepsis and septic shock. Results For detection of sepsis and severe sepsis, InSight achieves an AUROC curve of 0.92 (95% CI 0.90 to 0.93) and 0.87 (95% CI 0.86 to 0.88), respectively. Four hours before onset, InSight predicts septic shock with an AUROC of 0.96 (95% CI 0.94 to 0.98) and severe sepsis with an AUROC of 0.85 (95% CI 0.79 to 0.91). Conclusions InSight outperforms existing sepsis scoring systems in identifying and predicting sepsis, severe sepsis and septic shock. This is the first sepsis screening system to exceed an AUROC of 0.90 using only vital sign inputs. InSight is robust to missing data, can be customised to novel hospital data using a small fraction of site data and retains strong discrimination across all institutions.

Effect of a Real-Time Electronic Dashboard on a Rapid Response System

A rapid response system (RRS) may have limited effectiveness when inpatient providers fail to recognize signs of early patient decompensation. We evaluated the impact of an electronic medical record (EMR)-based alerting dashboard on outcomes associated with RRS activation. We used a repeated treatment study in which the dashboard display was successively turned on and off each week for ten 2-week cycles over a 20-week period on the inpatient acute care wards of an academic medical center. The Rapid Response Team (RRT) dashboard displayed all hospital patients in a single view ranked by severity score, updated in real time. The dashboard could be seen within the EMR by any provider, including RRT members. The primary outcomes were the incidence rate ratio (IRR) of all RRT activations, unexpected ICU transfers, cardiopulmonary arrests and deaths on general medical-surgical wards (wards). We conducted an exploratory analysis of first RRT activations. There were 6736 eligible admissions during the 20-week study period. There was no change in overall RRT activations (IRR = 1.14, p = 0.07), but a significant increase in first RRT activations (IRR = 1.20, p = 0.04). There were no significant differences in unexpected ICU transfers (IRR = 1.15, p = 0.25), cardiopulmonary arrests on general wards (IRR = 1.46, p = 0.43), or deaths on general wards (IRR = 0.96, p = 0.89). The introduction of the RRT dashboard was associated with increased initial RRT activations but not overall activations, unexpected ICU transfers, cardiopulmonary arrests, or death. The RRT dashboard is a novel tool to help providers recognize patient decompensation and may improve initial RRT notification.

Selection and Implementation of an Electronic Health Record

Electronic health records (EHRs) are now the standard of practice in most communities, because of transition to reimbursement that increasingly focuses on risk sharing and quality measurement, and government EHR incentive programs. The selection and implementation of an EHR is one of the most important decisions a practice faces. Organizing the search for an EHR that fits a practice, negotiating a contract, planning and successfully implementing an EHR are best accomplished with a well‐informed, strong, multidisciplinary team using project management techniques. Focusing on the best match between your practices needs and available commercial systems, and creating a strong relationship with your vendor will be key to leveraging the EHR to improve the experience of your patients and the quality of care they receive, and to the efficiency of your practice.

Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting

Algorithm–based clinical decision support (CDS) systems associate patient-derived health data with outcomes of interest, such as in-hospital mortality. However, the quality of such associations often depends on the availability of site-specific training data. Without sufficient quantities of data, the underlying statistical apparatus cannot differentiate useful patterns from noise and, as a result, may underperform. This initial training data burden limits the widespread, out-of-the-box, use of machine learning–based risk scoring systems. In this study, we implement a statistical transfer learning technique, which uses a large “source” data set to drastically reduce the amount of data needed to perform well on a “target” site for which training data are scarce. We test this transfer technique with AutoTriage, a mortality prediction algorithm, on patient charts from the Beth Israel Deaconess Medical Center (the source) and a population of 48 249 adult inpatients from University of California San Francisco Medical Center (the target institution). We find that the amount of training data required to surpass 0.80 area under the receiver operating characteristic (AUROC) on the target set decreases from more than 4000 patients to fewer than 220. This performance is superior to the Modified Early Warning Score (AUROC: 0.76) and corresponds to a decrease in clinical data collection time from approximately 6 months to less than 10 days. Our results highlight the usefulness of transfer learning in the specialization of CDS systems to new hospital sites, without requiring expensive and time-consuming data collection efforts.

Simplification of the HOSPITAL score for predicting 30-day readmissions.

Objective The HOSPITAL score has been widely validated and accurately identifies high-risk patients who may mostly benefit from transition care interventions. Although this score is easy to use, it has the potential to be simplified without impacting its performance. We aimed to validate a simplified version of the HOSPITAL score for predicting patients likely to be readmitted. Design and setting Retrospective study in 9 large hospitals across 4 countries, from January through December 2011. Participants We included all consecutively discharged medical patients. We excluded patients who died before discharge or were transferred to another acute care facility. Measurements The primary outcome was any 30-day potentially avoidable readmission. We simplified the score as follows: (1) ‘discharge from an oncology division’ was replaced by ‘cancer diagnosis or discharge from an oncology division’; (2) ‘any procedure’ was left out; (3) patients were categorised into two risk groups (unlikely and likely to be readmitted). The performance of the simplified HOSPITAL score was evaluated according to its overall accuracy, its discriminatory power and its calibration. Results Thirty-day potentially avoidable readmission rate was 9.7% (n=11 307/117 065 patients discharged). Median of the simplified HOSPITAL score was 3 points (IQR 2–5). Overall accuracy was very good with a Brier score of 0.08 and discriminatory power remained good with a C-statistic of 0.69 (95% CI 0.68 to 0.69). The calibration was excellent when comparing the expected with the observed risk in the two risk categories. Conclusions The simplified HOSPITAL score has good performance for predicting 30-day readmission. Prognostic accuracy was similar to the original version, while its use is even easier. This simplified score may provide a good alternative to the original score depending on the setting.

Multicenter validation of a machine learning algorithm for 48 hour all-cause mortality prediction

Purpose This study evaluates a machine-learning-based mortality prediction tool. Materials and Methods We conducted a retrospective study with data drawn from three academic health centers. Inpatients of at least 18 years of age and with at least one observation of each vital sign were included. Predictions were made at 12, 24, and 48 hours before death. Models fit to training data from each institution were evaluated on hold-out test data from the same institution and data from the remaining institutions. Predictions were compared to those of qSOFA and MEWS using area under the receiver operating characteristic curve (AUROC). Results For training and testing on data from a single institution, machine learning predictions averaged AUROCs of 0.97, 0.96, and 0.95 across institutional test sets for 12-, 24-, and 48-hour predictions, respectively. When trained and tested on data from different hospitals, the algorithm achieved AUROC up to 0.95, 0.93, and 0.91, for 12-, 24-, and 48-hour predictions, respectively. MEWS and qSOFA had average 48-hour AUROCs of 0.86 and 0.82, respectively. Conclusion This algorithm may help identify patients in need of increased levels of clinical care.