Ilan Rubinfeld

Looking Beyond Historical Patient Outcomes to Improve Clinical Models

Clinical models can be improved by decreasing the importance assigned to fitting historical patient outcomes in often small and imperfectly characterized derivation cohorts. When Less Is More Clinical models play an important role in guiding patient care at the bedside, improving our understanding of diseases, and performing objective assessments of healthcare quality. The typical approach to developing these models places great importance on fitting historical patient outcomes in derivation data sets (such as those obtained from clinical studies or patient registries). However, for a fairly broad range of medical applications, these derivation data sets may be small after accounting for inclusionary and exclusionary criteria, and additionally may be imperfectly characterized due to noise and variations in the rates of patient outcomes. Collecting more data offers one approach to address this issue, but is challenging due to the costs and complexity of increasing the size of clinical cohorts. In the setting of small and imperfectly characterized data sets, approaches to developing clinical models that rely exclusively on fitting historical patient outcomes suffer from the implicit assumption that the derivation data sets are representative. Instead, as the new study by Chia et al. explores, the process of developing clinical models can be improved by decreasing the importance placed on fitting historical patient outcomes, and by supplementing these models with information about the extent to which patients differ from the statistical distribution of clinical characteristics within the derivation data set. When evaluated using data from three different clinical applications [patients with acute coronary syndrome enrolled in the DISPERSE2-TIMI33 and MERLIN-TIMI36 trials, patients undergoing inpatient surgery in the National Surgical Quality Improvement Program (NSQIP) registry, and patients undergoing percutaneous coronary intervention in the Blue Cross Blue Shield of Michigan Cardiovascular Consortium (BMC2) registry], this approach of treating derivation data for clinical models as simultaneously labeled and unlabeled consistently improved discrimination between high- and low-risk patients according to different statistical metrics. The idea of decreasing the importance assigned to fitting historical outcomes allows for better clinical models, and ultimately for improvements in the use of these models to study diseases, choose therapies, or evaluate healthcare providers. Conventional algorithms for modeling clinical events focus on characterizing the differences between patients with varying outcomes in historical data sets used for the model derivation. For many clinical conditions with low prevalence and where small data sets are available, this approach to developing models is challenging due to the limited number of positive (that is, event) examples available for model training. Here, we investigate how the approach of developing clinical models might be improved across three distinct patient populations (patients with acute coronary syndrome enrolled in the DISPERSE2-TIMI33 and MERLIN-TIMI36 trials, patients undergoing inpatient surgery in the National Surgical Quality Improvement Program registry, and patients undergoing percutaneous coronary intervention in the Blue Cross Blue Shield of Michigan Cardiovascular Consortium registry). For each of these cases, we supplement an incomplete characterization of patient outcomes in the derivation data set (uncensored view of the data) with an additional characterization of the extent to which patients differ from the statistical support of their clinical characteristics (censored view of the data). Our approach exploits the same training data within the derivation cohort in multiple ways to improve the accuracy of prediction. We position this approach within the context of traditional supervised (2-class) and unsupervised (1-class) learning methods and present a 1.5-class approach for clinical decision-making. We describe a 1.5-class support vector machine (SVM) classification algorithm that implements this approach, and report on its performance relative to logistic regression and 2-class SVM classification with cost-sensitive weighting and oversampling. The 1.5-class SVM algorithm improved prediction accuracy relative to other approaches and may have value in predicting clinical events both at the bedside and for risk-adjusted quality of care assessment.

Laparoscopic versus open adrenalectomy: Another look at outcome using the Clavien classification system

BACKGROUNDnA laparoscopic approach to adrenalectomy has become the procedure of choice for most adrenal resections. We hypothesized that laparoscopic adrenalectomy is less likely to result in intensive care unit (ICU) level complications or death than open adrenalectomy, despite baseline comorbidity mix.nnnMETHODSnUsing the National Surgical Quality Improvement Program (NSQIP) participant use files for 2005-2009, all laparoscopic and open adrenalectomies were identified by current procedural terminology. Adverse outcomes tracked in NSQIP were mapped to Clavien level based on need for ICU care or death. Univariate and multivariate analysis were used to compare groups.nnnRESULTSnThere were 1,980 laparoscopic and 592 open procedures. Clavien 4 and 5 complications occurred in 45 (7.6%) of open and 35 (1.8%) of laparoscopic operations. The univariate odds ratio showed a 4.6-fold greater likelihood that a patient would have an ICU level complication (P < .001), and 4.9 odds ratio of death (P < .001) if an open rather than laparoscopic operation was performed. Regression modeling showed persistence of the protective effect of laparoscopy after adjusting for comorbidities with a multivariate odds ratio of 3.3 (P < .001).nnnCONCLUSIONnThe laparoscopic approach to adrenalectomy has an independent protective effect on ICU level complications and mortality when compared with open procedures. This correlation persists after correcting for multiple comorbidities.

Prolonged Length of Stay after Esophageal Resection: Identifying Drivers of Increased Length of Stay Using the NSQIP Database

BACKGROUNDnAlthough influence of technical complications in association to hospital length of stay has been studied extensively in esophageal resection, nontechnical factors responsible for prolonged length of stay have not been reported. Using the NSQIP dataset, we hypothesized that we would be able to identify factors associated with prolonged length of stay after esophagectomy.nnnSTUDY DESIGNnNational Surgical Quality Improvement Program data from 2005 to 2012 were reviewed for CPT codes for esophagectomy. Outlier status for length of stay was defined as >75th percentile. Logistic regression was used to predict outlier status and linear regression to discern factors contributing to longer lengths of stay.nnnRESULTSnA total of 3,538 cases were reviewed. The 75th percentile for length of stay was 17 days. Preoperative predictors of hospital stay outliers include emergency surgery and frailty index (odds ratiosxa0= 3.7 and 3.6; p < 0.001). Deep organ space infection and progressive renal insufficiency had the highest likelihood of prolonged length of stay (odds ratiosxa0= 5.2 and 5.1; pxa0<xa00.001). Failure to wean off of ventilator in 48 hours, urinary tract infection, and pneumonia were associated with length of stay outlier (odds ratiosxa0= 3.7, 2.7, and 2.7; all p < 0.001).nnnCONCLUSIONSnUrinary tract infection and pneumonia after esophagectomy are associated with longer hospital stays. Although meticulous surgical technique remains paramount, our study demonstrates that postoperative nontechnical complications factor into prolonged hospital stays. Focus on such factors can lead to reductions in hospital stays.

The differential effects of surgical harm in elderly populations. Does the adage: “they tolerate the operation, but not the complications” hold true?

BACKGROUNDnElderly patients are thought to tolerate surgical complications poorly because of low physiologic reserve. The purpose of the study was to evaluate the differential effects of surgical harm in patients over 80 years old.nnnMETHODSnThree years of data from a harm-reduction campaign were used to identify inpatient surgeries performed on patients older than 50. The rates of harm, death, cost, and length of stay (LOS) were analyzed using SPSS 21 (IBM, New York, NY).nnnRESULTSnA total of 22,710 patients were identified. Rates of harm and mortality increased with increasing age. Harmed patients over age 80 had increased mortality (9.5% vs 7%), but lower cost, intensive care unit days, and LOS versus those aged 50 to 80. Linear regression showed increased cost with harm (

e-Portfolio Competency Metadata: Pilot Study for a Call to Action

24,000) and decreased cost with age above 80 (-

Personalized risk stratification for adverse surgical outcomes: innovation at the boundaries of medicine and computation

7,000).nnnCONCLUSIONSnIn the elderly surgical population, there is more harm and harm events are associated with higher mortality rates, but less additional cost and LOS. Differing goals or aggressiveness of care may explain cost avoidance in the elderly.

Ultrasonographic Evaluation of Sinusitis During Microgravity in a Novel Animal Model

The six competency domains required by the Accreditation Council for Graduate Medical Education (ACGME) have led to a proliferation of measurement tools, assessment methods, and all forms of data from paper to electronic. The need exists to develop a standardized electronic (e)-portfolio to provide the aggregate data to improve education and patient care. This process requires a sound methodology using XML metadata to allow portability of e-portfolio data. We surveyed publicly available metadata and developed an e-portfolio system for the Henry Ford Hospital General Surgery Residency Program. Based on our implementation of e-portfolios for 70 physicians, we call upon the ACGME, the Residency Review Committees, and the American Board of Medical Specialties to establish a method to formalize and develop a standard for residency competency metadata. Using an approach similar to that of our study can streamline data and lead to improved medical education and ultimately better patient care.

Patient safety curriculum for surgical residency programs: Results of a national consensus conference

Patients undergoing surgery exhibit a highly variable risk of mortality and morbidity, even when undergoing similar procedures. Accurately quantifying this risk is critical for preoperative decision-making to ensure patients recieve treatment that is optimal for their individual profile, and for guiding intraoperative and postoperative care. Despite the considerable attention this issue has received, existing models for surgical risk stratification remain grounded in traditional statistical methods and in problem statements that have not evolved significantly over the years. This article explores recent innovations in machine learning and data mining to advance these efforts. Risk-stratification models based on sophisticated computational techniques hold the promise of a new generation of predictive analytical tools that are highly accurate and widely deployable.

Unsupervised Risk Stratification in Clinical Datasets: Identifying Patients at Risk of Rare Outcomes

OBJECTIVESnTo develop an animal model of rhinosinusitis in microgravity, to characterize the behavior of intracavitary fluid in microgravity, and to assess the accuracy of ultrasonographic (US) diagnosis in microgravity.nnnDESIGNnAn animal model of acute sinusitis was developed in anesthetized swine by creating a window into a frontal sinus to allow unilateral catheter placement and injection of fluid. We performed US examinations in normal and microgravity environments on control and sinusitis conditions and recorded these for later interpretation.nnnSETTINGnHenry Ford Hospital and the National Aeronautics and Space Administration (NASA) Microgravity Research Facility in Houston, Texas.nnnSUBJECTSnGround (normal-gravity) experiments were conducted on anesthetized swine (n = 4) at Henry Ford Hospital before the microgravity experiments (n = 4) conducted in the NASA Microgravity Research Facility.nnnMAIN OUTCOME MEASUREnUltrasound visualization of fluid cavity.nnnRESULTSnResults of bilateral US examinations before fluid injection demonstrated typical air-filled sinuses. After unilateral injection of 1 mL of fluid, a consistent air-fluid interface was observed on the catheterized side at ground conditions. Microgravity conditions caused the rapid (<10-second) dissolution of the air-fluid interface, associated with uniform dispersion of the fluid to the walls of the sinus. The air-fluid interface reformed on return to normal gravity.nnnCONCLUSIONSnThe US appearance of fluid in nasal sinuses during microgravity is characterized in the large animal model. On the introduction of microgravity, the typical air-fluid interface disassociates, and fluid lining the sinus can be observed. Such fluid behavior can be used to develop diagnostic criteria for acute bacterial rhinosinusitis in the microgravity environment.