R. Andrew Taylor

Prediction of In-hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data-Driven, Machine Learning Approach.

OBJECTIVES Predictive analytics in emergency care has mostly been limited to the use of clinical decision rules (CDRs) in the form of simple heuristics and scoring systems. In the development of CDRs, limitations in analytic methods and concerns with usability have generally constrained models to a preselected small set of variables judged to be clinically relevant and to rules that are easily calculated. Furthermore, CDRs frequently suffer from questions of generalizability, take years to develop, and lack the ability to be updated as new information becomes available. Newer analytic and machine learning techniques capable of harnessing the large number of variables that are already available through electronic health records (EHRs) may better predict patient outcomes and facilitate automation and deployment within clinical decision support systems. In this proof-of-concept study, a local, big data-driven, machine learning approach is compared to existing CDRs and traditional analytic methods using the prediction of sepsis in-hospital mortality as the use case. METHODS This was a retrospective study of adult ED visits admitted to the hospital meeting criteria for sepsis from October 2013 to October 2014. Sepsis was defined as meeting criteria for systemic inflammatory response syndrome with an infectious admitting diagnosis in the ED. ED visits were randomly partitioned into an 80%/20% split for training and validation. A random forest model (machine learning approach) was constructed using over 500 clinical variables from data available within the EHRs of four hospitals to predict in-hospital mortality. The machine learning prediction model was then compared to a classification and regression tree (CART) model, logistic regression model, and previously developed prediction tools on the validation data set using area under the receiver operating characteristic curve (AUC) and chi-square statistics. RESULTS There were 5,278 visits among 4,676 unique patients who met criteria for sepsis. Of the 4,222 patients in the training group, 210 (5.0%) died during hospitalization, and of the 1,056 patients in the validation group, 50 (4.7%) died during hospitalization. The AUCs with 95% confidence intervals (CIs) for the different models were as follows: random forest model, 0.86 (95% CI = 0.82 to 0.90); CART model, 0.69 (95% CI = 0.62 to 0.77); logistic regression model, 0.76 (95% CI = 0.69 to 0.82); CURB-65, 0.73 (95% CI = 0.67 to 0.80); MEDS, 0.71 (95% CI = 0.63 to 0.77); and mREMS, 0.72 (95% CI = 0.65 to 0.79). The random forest model AUC was statistically different from all other models (p ≤ 0.003 for all comparisons). CONCLUSIONS In this proof-of-concept study, a local big data-driven, machine learning approach outperformed existing CDRs as well as traditional analytic techniques for predicting in-hospital mortality of ED patients with sepsis. Future research should prospectively evaluate the effectiveness of this approach and whether it translates into improved clinical outcomes for high-risk sepsis patients. The methods developed serve as an example of a new model for predictive analytics in emergency care that can be automated, applied to other clinical outcomes of interest, and deployed in EHRs to enable locally relevant clinical predictions.

Point-of-care focused cardiac ultrasound for the assessment of thoracic aortic dimensions, dilation, and aneurysmal disease.

OBJECTIVES   Thoracic aortic aneurysm and thoracic aortic dissection are related and potentially deadly diseases that present with nonspecific symptoms. Transthoracic echocardiography (TTE) may detect thoracic aortic pathology and is being increasingly performed by the emergency physician at the bedside; however, the accuracy of point-of-care (POC) focused cardiac ultrasound (FOCUS) for thoracic aortic aneurysm and thoracic aortic dissection has not been studied. The objective of this pilot study was to explore the agreement, sensitivity, and specificity of FOCUS for thoracic aortic dimensions, dilation, and aneurysm compared with CT angiography (CTA) as the reference standard. METHODS   This study was a retrospective pilot analysis of image and chart data on consecutive patients presenting to an urban, academic emergency department (ED) between January 2008 and June 2010, who had both a FOCUS and a CTA for suspicion of thoracic aorta pathology. Thoracic aorta dimensions were measured from recordings by three ultrasound-trained emergency physicians blinded to any initial FOCUS and CTA results. CTA measurements were obtained by a radiologist blinded to the FOCUS results. Using cutoffs of 40 and 45 mm, we calculated the sensitivity and specificity of FOCUS for aortic dilation and aneurysm with the largest measurement on CT as the reference standard. Bland-Altman plots with 95% limits of agreement were used to demonstrate agreement for aortic measurements, kappa statistics to assess the degree of agreement between tests for aortic dilation, and intraclass correlation for interobserver and intraobserver variability. RESULTS   Ninety-two patients underwent both FOCUS and CTA during the study period. Ten FOCUS studies had inadequate visualization for all measurements areas. Eighty-two patients were included in the final analysis. Mean (±SD) age was 58.1 (±16.6) years and 58.5% were male. Sensitivity, specificity, and the observed kappa value (95% confidence interval [CI]) between FOCUS and CTA for the presence of aortic dilation at the 40-mm cutoff were 0.77 (95% CI = 0.58 to 0.98), 0.95 (95% CI = 0.84 to 0.99), and 0.74 (95% CI = 0.58 to 0.90), respectively. The mean difference (95% limits of agreement) for the Bland-Altman plots was 0.6 mm (-5.3 to 6.5) for the sinuses of Valsalva, 4 mm (-2.7 to 10.7) for the sinotubular junction, 1.5 mm (-5.8 to 8.8) for the ascending aorta, and 2.2 mm (-5.9 to 10.3) for the descending aorta. CONCLUSIONS   In this retrospective pilot study, FOCUS demonstrated good agreement with CTA measurements of maximal thoracic aortic diameter. FOCUS appears to be specific for aortic dilation and aneurysm when compared to CTA, but requires further prospective study.

Point-of-Care Focused Cardiac Ultrasound for Prediction of Pulmonary Embolism Adverse Outcomes

BACKGROUND In patients with pulmonary embolism (PE), right ventricular strain (RVS) on transthoracic echocardiography by Cardiology has been shown to be an independent predictor of 30-day adverse outcomes. However, it is not known how emergency practitioner-performed point-of-care focused cardiac ultrasound (FOCUS) with assessment for RVS compares with other prognostic methods in the Emergency Department (ED). OBJECTIVES To determine whether RVS on FOCUS is a significant predictor of in-hospital adverse outcomes when compared to other risk factors and scoring systems. METHODS Retrospective chart review of patients who were diagnosed with PE and had a FOCUS examination during January 1, 2007 through January 1, 2011 in an urban, academic center with a well-developed ultrasound program. Adverse outcomes were defined as shock, respiratory failure requiring intubation, death, recurrent venous thromboembolism, transition to higher level of care, or major bleeding during hospital admission. Statistical analysis included univariate and multivariate analysis to assess for prognostic significance. RESULTS One hundred sixty-one patients were included in the final analysis. A total of 25 (16%) patients had an adverse outcome during hospitalization. On univariate analysis, only the presence of altered mental status conferred a higher positive likelihood ratio (6.4 vs. 4.0) than RVS, whereas absence of RVS had the lowest negative likelihood ratio (0.45). On multivariate analysis, RVS and cardiopulmonary disease were the only predictors of adverse outcomes that achieved statistical significance, with odds ratio of 9.2 and 3.4, respectively. CONCLUSION In this retrospective chart review, a FOCUS examination for RVS performed by emergency care practitioners of varying experience level was a significant predictor of in-hospital adverse outcomes among patients diagnosed with PE in the ED. Future research should be directed at exploring ways to incorporate RVS assessment into ED prognostic models for pulmonary embolism.

A decision analysis to determine a testing threshold for computed tomographic angiography and d-dimer in the evaluation of aortic dissection

OBJECTIVE The objective of this study is to determine at what probability of thoracic aortic dissection (TAD) to use a computed tomographic angiography (CTA) or a d-dimer test. METHODS We used decision analysis software to determine the testing threshold (TT) for 3 hypothetical decisions when evaluating for TAD: (1) no testing vs CTA, (2) no testing vs D-dimer, and (3) CTA vs D-dimer. One- and 2-way sensitivity analyses were performed to determine which variables were drivers of the TTs. RESULTS We found TTs of 0.03%, 0.013%, and 0.6% for decisions 1, 2, and 3, respectively. For all 3 decisions, patient age and the annual rate of cancer were major drivers of the TT. In decisions 1 and 2, the probability of acute renal failure requiring renal replacement therapy was a major driver, whereas d-dimer sensitivity was a major driver for decision 3. CONCLUSION The TTs for TAD are low and reflect the large mortality benefit from diagnosis and treatment when compared with the small risks of CTA. However, given the low prevalence of TAD (~0.05% among emergency department patients presenting with symptoms previously attributed to TAD), our results suggest that without high-risk features, clinicians should not order a CTA test for TAD. Depending on age, CTA should be considered for those patients with a disease probability greater than 0.3% to 2.1%, whereas d-dimer testing is appropriate in the range of pretest probabilities from 0.01% to 0.6%. Future studies should focus on clinical decision rules that place disease probabilities below, between, and above the calculated TTs.

Emergency physician focused cardiac ultrasound improves diagnosis of ascending aortic dissection

STUDY OBJECTIVE Ascending aortic dissection (AAD) is an uncommon, time-sensitive, and deadly diagnosis with a nonspecific presentation. Ascending aortic dissection is associated with aortic dilation, which can be determined by emergency physician focused cardiac ultrasound (EP FOCUS). We seek to determine if patients who receive EP FOCUS have reduced time to diagnosis for AAD. METHODS We performed a retrospective review of patients treated at 1 of 3 affiliated emergency departments, March 1, 2013, to May 1, 2015, diagnosed as having AAD. All autopsies were reviewed for missed cases. Primary outcome measure was time to diagnosis. Secondary outcomes were time to disposition, misdiagnosis rate, and mortality. RESULTS Of 386547 ED visits, targeted review of 123 medical records and 194 autopsy reports identified 32 patients for inclusion. Sixteen patients received EP FOCUS and 16 did not. Median time to diagnosis in the EP FOCUS group was 80 (interquartile range [IQR], 46-157) minutes vs 226 (IQR, 109-1449) minutes in the non-EP FOCUS group (P = .023). Misdiagnosis was 0% (0/16) in the EP FOCUS group vs 43.8% (7/16) in the non-EP FOCUS group (P = .028). Mortality, adjusted for do-not-resuscitate status, for EP FOCUS vs non-EP FOCUS was 15.4% vs 37.5% (P = .24). Median rooming time to disposition was 134 (IQR, 101-195) minutes for EP FOCUS vs 205 (IQR, 114-342) minutes for non-EP FOCUS (P = .27). CONCLUSIONS Patients who receive EP FOCUS are diagnosed faster and misdiagnosed less compared with patients who do not receive EP FOCUS. We recommend assessment of the thoracic aorta be performed routinely during cardiac ultrasound in the emergency department.

Improving emergency physician performance using audit and feedback: a systematic review

BACKGROUND Audit and feedback can decrease variation and improve the quality of care in a variety of health care settings. There is a growing literature on audit and feedback in the emergency department (ED) setting. Because most studies have been small and not focused on a single clinical process, systematic assessment could determine the effectiveness of audit and feedback interventions in the ED and which specific characteristics improve the quality of emergency care. OBJECTIVE The objective of the study is to assess the effect of audit and feedback on emergency physician performance and identify features critical to success. METHODS We adhered to the PRISMA statement to conduct a systematic review of the literature from January 1994 to January 2014 related to audit and feedback of physicians in the ED. We searched Medline, EMBASE, PsycINFO, and PubMed databases. We included studies that were conducted in the ED and reported quantitative outcomes with interventions using both audit and feedback. For included studies, 2 reviewers independently assessed methodological quality using the validated Downs and Black checklist for nonrandomized studies. Treatment effect and heterogeneity were to be reported via meta-analysis and the I2 inconsistency index. RESULTS The search yielded 4332 articles, all of which underwent title review; 780 abstracts and 131 full-text articles were reviewed. Of these, 24 studies met inclusion criteria with an average Downs and Black score of 15.6 of 30 (range, 6-22). Improved performance was reported in 23 of the 24 studies. Six studies reported sufficient outcome data to conduct summary analysis. Pooled data from studies that included 41,124 patients yielded an average treatment effect among physicians of 36% (SD, 16%) with high heterogeneity (I2=83%). CONCLUSION The literature on audit and feedback in the ED reports positive results for interventions across numerous clinical conditions but without standardized reporting sufficient for meta-analysis. Characteristics of audit and feedback interventions that were used in a majority of studies were feedback that targeted errors of omission and that was explicit with measurable instruction and a plan for change delivered in the clinical setting greater than 1 week after the audited performance using a combination of media and types at both the individual and group levels. Future work should use standardized reporting to identify the specific aspects of audit or feedback that drive effectiveness in the ED.

Redefining Overuse to Include Costs: A Decision Analysis for Computed Tomography in Minor Head Injury

BACKGROUND A study was conducted to (1) determine the testing threshold for head computed tomography (CT) in minor head injury in the emergency department using decision analysis with and without costs included in the analysis, (2) to determine which variables have significant impact on the testing threshold, and (3) to compare this calculated testing threshold to the pretest risk estimate previously reported when the Canadian CT Head Rule (CCHR) was applied. It was hypothesized that the CCHR might not identify all patients above the testing threshold. METHODS A decision analytic model was constructed using commercially available software and data from published literature. Outcomes were assigned values on the basis of quality-adjusted life-years (QALYs) and cost. Two testing thresholds were calculated, the first based only on the effectiveness of either strategy, the second on the overall net monetary benefit. Two-way sensitivity analyses were performed to determine which variables most affected the testing threshold. RESULTS When only effectiveness (QALYs) was considered, the testing threshold for obtaining head CT was 0.039%. This threshold increased to 0.421% when the net monetary benefit was considered in lieu of QALYs. Age, probability of lesion on CT requiring neurosurgery, and cost of CT were the main drivers of the model. CONCLUSION If only effectiveness is considered, current clinical decision rules might not provide a sufficient degree of certainty to ensure identification of all patients for whom the benefits of CT outweigh its risks. However, inclusion of cost in the analysis increases the testing threshold by an order of magnitude and well outside the range of uncertainty of current clinical decision rules. These results suggest that the term overuse should be redefined to include the provision of medical services with no benefits or for which harms including cost outweigh benefits.

Use of Point-of-Care Ultrasound in the Emergency Department Insights From the 2012 Medicare National Payment Data Set

Point‐of‐care ultrasound is a valuable tool with potential to expedite diagnoses and improve patient outcomes in the emergency department. However, little is known about national patterns of adoption. This study examined nationwide point‐of‐care ultrasound reimbursement among emergency medicine (EM) practitioners and examined regional and practitioner level variations.

Impact of point-of-care ultrasonography on ED time to disposition for patients with nontraumatic shock

STUDY OBJECTIVE Nontraumatic shock in the emergency department (ED) has multiple causes and carries in-hospital mortality approaching 20%, underscoring the need for prompt diagnosis and treatment. Diagnostic ultrasonography at the point of care is one method that may improve the ability of ED physicians to quickly diagnose and treat. This study assesses the effect of the use and timing of point-of-care (POC) ultrasonography on time to disposition request. METHODS This retrospective study across 4 Connecticut EDs compared propensity score matched shock patients who did and did not receive POC ultrasonography. Two propensity score matches were performed: the first using covariates of time to disposition from previous literature and the second using 25 novel covariates identified from electronic health records using machine learning to reduce variable selection biases. RESULTS A total of 3834 unique patients presented with shock during an 18-month period, and 703 (18.3%) patients received POC ultrasonography. Mean time to disposition for all patients was 255.4minutes (interquartile range, 163.8). After propensity score matching, patients had a mean reduction of 26.7minutes (95% confidence interval [CI], 2.8-58.3) in time to disposition when POC ultrasonography was performed within 1hour of ED arrival and a lesser reduction of 16.7minutes (95% CI, -2.8 to 35.5) when POC ultrasonography was performed within 2hours. There was no evidence of reduction in time to disposition when ultrasonography was performed after 2hours (16.7minutes; 95% CI, -14.3 to 29.9). Propensity score models using machine learning-selected variables yielded similar results. CONCLUSION Performance of POC ultrasonography likely improves time to disposition when performed early on ED patients with shock.

The Association Between Physician Empathy and Variation in Imaging Use.

BACKGROUND Variation in emergency physician computed tomography (CT) imaging utilization is well described, but little is known about what drives it. Physician empathy has been proposed as a potential characteristic affecting CT utilization. OBJECTIVES The objective was to describe empathy in a cohort of emergency physicians and evaluate its association with CT utilization. We also sought to compare emergency physician performance on an empathy psychometric test with performance on other psychometric tests previously proposed as predictors of CT utilization. METHODS This cross-sectional study included two parts: 1) a secondary analysis of emergency department (ED) CT imaging utilization data in a large health system from July 2013 to June 2014 and 2) a survey study of the cohort of physicians responsible for this imaging using four psychometric scales: the Jefferson Scale of Empathy (JSE), a risk-taking subset of the Jackson Personality Index (RTS), the Stress from Uncertainty Scale (SUS), and the Malpractice Fear Scale (MFS). The study included data and physicians from four EDs: one urban, academic ED, two community, and one free-standing. A hierarchical, mixed-effects regression model was used to evaluate the association between emergency physician performance on the four scales and risk-adjusted CT imaging utilization. The model incorporated physician-specific CT utilization rates adjusted for propensity scores that were calculated using over 500 patient-level variables via random forest methods, physician demographics, and a random provider effect to account for the clustering of observations. RESULTS CT variation analysis included 113,517 patients seen during the study period by the 74 eligible emergency physician survey respondents; 20,972 (18.5%) of these patients had at least one CT. The survey response rate was 74 of 82 (90.2%). Correlation coefficients between JSE and the other scales were not statistically significant. In subset analysis, there was a trend toward a physicians number of years in practice and RTS score contributing to CT utilization for traumatic head CT. There were no significant associations between performance on any of the psychometric scales and CT utilization. CONCLUSIONS Performance on the JSE, RTS, SUS, or MFS was not predictive of risk-adjusted CT utilization in the ED. The underlying physician-based factors that mediate interphysician variation remain to be clearly identified.