Kevin Chagin

Development and validation of nomograms to provide individualised predictions of seizure outcomes after epilepsy surgery: a retrospective analysis

BACKGROUND Half of patients who have resective brain surgery for drug-resistant epilepsy have recurrent postoperative seizures. Although several single predictors of seizure outcome have been identified, no validated method incorporates a patients complex clinical characteristics into an instrument to predict an individuals post-surgery seizure outcome. METHODS We developed nomograms to predict complete freedom from seizures and Engel score of 1 (eventual freedom from seizures allowing for some initial postoperative seizures, or seizures occurring only with physiological stress such as drug withdrawal) at 2 years and 5 years after surgery on the basis of sex, seizure frequency, secondary seizure generalisation, type of surgery, pathological cause, age at epilepsy onset, age at surgery, epilepsy duration at time of surgery, and surgical side. We designed the models from a development cohort of patients who had resective surgery at the Cleveland Clinic (Cleveland, OH, USA) between 1996 and 2011. We then tested the nomograms in an external validation cohort operated on over a similar period in four epilepsy surgery centres, in Brazil, France, Italy, and the USA. We assessed performance of the nomogram by calculating concordance statistics and assessing the calibration of predicted freedom from seizures with the reported freedom from seizures and Engel score of 1. FINDINGS The development cohort included 846 patients and the validation cohort included 604 patients. Variables included in the nomograms were sex, seizure frequency, secondary seizure generalisation, type of surgery, and pathological cause. In the development cohort, the baseline risk of complete freedom from seizures was 0·57 at 2 years and 0·40 at 5 years. The baseline risk of Engel score of 1 was 0·69 at 2 years and 0·62 at 5 years. In the validation cohort, the models had a concordance statistic of 0·60 for complete freedom from seizures and 0·61 for Engel score of 1. Calibration curves showed adequate calibration (judged by eye) of predicted and reported freedom from seizures, throughout the range of seizure outcomes. INTERPRETATION If validated in prospective cohorts, these nomograms could be used to predict seizure outcomes in patients who have been judged eligible for epilepsy surgery. FUNDING Cleveland Clinic Epilepsy Center.

Strategies for handling missing data in electronic health record derived data.

Electronic health records (EHRs) present a wealth of data that are vital for improving patient-centered outcomes, although the data can present significant statistical challenges. In particular, EHR data contains substantial missing information that if left unaddressed could reduce the validity of conclusions drawn. Properly addressing the missing data issue in EHR data is complicated by the fact that it is sometimes difficult to differentiate between missing data and a negative value. For example, a patient without a documented history of heart failure may truly not have disease or the clinician may have simply not documented the condition. Approaches for reducing missing data in EHR systems come from multiple angles, including: increasing structured data documentation, reducing data input errors, and utilization of text parsing / natural language processing. This paper focuses on the analytical approaches for handling missing data, primarily multiple imputation. The broad range of variables available in typical EHR systems provide a wealth of information for mitigating potential biases caused by missing data. The probability of missing data may be linked to disease severity and healthcare utilization since unhealthier patients are more likely to have comorbidities and each interaction with the health care system provides an opportunity for documentation. Therefore, any imputation routine should include predictor variables that assess overall health status (e.g. Charlson Comorbidity Index) and healthcare utilization (e.g. number of encounters) even when these comorbidities and patient encounters are unrelated to the disease of interest. Linking the EHR data with other sources of information (e.g. National Death Index and census data) can also provide less biased variables for imputation. Additional methodological research with EHR data and improved epidemiological training of clinical investigators is warranted.

Glycopeptides versus β-lactams for the prevention of surgical site infections in cardiovascular and orthopedic surgery: a meta-analysis.

OBJECTIVE To compare the efficacy of glycopeptides and β-lactams in preventing surgical site infections (SSIs) in cardiac, vascular, and orthopedic surgery. BACKGROUND The cost-effectiveness of switching from β-lactams to glycopeptides for preoperative antibiotic prophylaxis has been controversial. β-Lactams are generally recommended in clean surgical procedures, but they are ineffective against resistant gram-positive bacteria. METHODS PubMed, International Pharmaceuticals Abstracts, Scopus, and Cochrane were searched for randomized clinical trials comparing glycopeptides and β-lactams for prophylaxis in adults undergoing cardiac, vascular, or orthopedic surgery. Abstracts and conference proceedings were included. Two independent reviewers performed study selection, data extraction, and assessment of risk of bias. RESULTS Fourteen studies with a total of 8952 patients were analyzed. No difference was detected in overall SSIs between antibiotic types. However, compared with β-lactams, glycopeptides reduced the risk of resistant staphylococcal SSIs by 48% (relative risk, 0.52; 95% confidence interval, 0.29-0.93; P = 0.03) and enterococcal SSIs by 64% (relative risk, 0.36; 95% confidence interval, 0.16-0.80; P = 0.01), but increased respiratory tract infections by 54% (relative risk, 1.54; 95% confidence interval, 1.19-2.01; P ≤ 0.01). Subgroup analysis of cardiac procedures showed superiority of β-lactams in preventing superficial and deep chest SSIs, susceptible staphylococcal SSIs, and respiratory tract infections. CONCLUSIONS Glycopeptides reduce the risk of resistant staphylococcal SSIs and enterococcal SSIs, but increase the risk of respiratory tract infections. Additional high-quality randomized clinical trials are needed as these results are limited by high risk of bias.

The Wisdom of Crowds of Doctors: Their Average Predictions Outperform Their Individual Ones

Background. Evidence suggests that the average prediction across groups is more accurate than for individuals. Our goals were therefore to investigate accuracy of the average predictions for groups of clinicians and to compare this accuracy with a published statistical prediction model. Methods. Twenty-four expert clinicians attending an advisory board meeting were asked to make predictions for 25 patients from a research registry regarding the probability of having a positive bone scan 1 year from today if left untreated. Comparisons were made between the accuracy of average responses and that of an appropriate previously published statistical prediction model. Results. This study suggests that the mean of the clinicians’ predictions can quickly approach the accuracy of the best clinician using as few as 5 clinicians. When all 24 clinicians’ predictions were averaged, the concordance index reached 0.750, still far below that of the published statistical model with 0.812. Conclusions. Averaging clinician predictions may have merit over individual clinician predictions but still not reasonably replace a carefully built statistical model. However, averaging clinician predictions could prove helpful in situations where statistical models do not yet exist or where existing models are inadequate.

Intensification of Diabetes Therapy and Time Until A1C Goal Attainment Among Patients With Newly Diagnosed Type 2 Diabetes Who Fail Metformin Monotherapy Within a Large Integrated Health System

OBJECTIVE “Clinical inertia” has been used to describe the delay in the intensification of type 2 diabetes treatment among patients with poor glycemic control. Previous studies may have exaggerated the prevalence of clinical inertia by failing to adequately monitor drug dose changes and nonmedication interventions. This project evaluated the intensification of diabetes therapy and hemoglobin A1c (A1C) goal attainment among patients with newly diagnosed type 2 diabetes when metformin monotherapy failed. RESEARCH DESIGN AND METHODS The electronic health record at Cleveland Clinic was used to identify patients with newly diagnosed type 2 diabetes between 2005 and 2013 who failed to reach the A1C goal after 3 months of metformin monotherapy. A time-dependent survival analysis was used to compare the time until A1C goal attainment in patients who received early intensification of therapy (within 6 months of metformin failure) or late intensification. The analysis was performed for A1C goals of 7% (n = 1,168), 7.5% (n = 679), and 8% (n = 429). RESULTS Treatment was intensified early in 62%, 69%, and 72% of patients when poor glycemic control was defined as an A1C >7%, >7.5%, and >8%, respectively. The probability of undergoing an early intensification was greater the higher the A1C category. Time until A1C goal attainment was shorter among patients who received early intensification regardless of the A1C goal (all P < 0.05). CONCLUSIONS A substantial number of patients with newly diagnosed type 2 diabetes fail to undergo intensification of therapy within 6 months of metformin monotherapy failure. Early intervention in patients when metformin monotherapy failed resulted in more rapid attainment of A1C goals.

Probability of developing proximal deep-vein thrombosis and/or pulmonary embolism after distal deep-vein thrombosis

Isolated distal deep-vein thrombosis (DDVT) of the lower extremities can be associated with subsequent proximal deep-vein thrombosis (PDVT) and/or acute pulmonary embolism (PE). We aimed to develop a model predicting the probability of developing PDVT and/or PE within three months after an isolated episode of DDVT. We conducted a retrospective cohort study of patients with symptomatic DDVT confirmed by lower extremity vein ultrasounds between 2001-2012 in the Cleveland Clinic Health System. We reviewed all the ultrasounds, chest ventilation/perfusion and computed tomography scans ordered within three months after the initial DDVT to determine the incidence of PDVT and/or PE. A multiple logistic regression model was built to predict the rate of developing these complications. The final model included 450 patients with isolated DDVT. Within three months, 30 (7 %) patients developed an episode of PDVT and/or PE. Only two factors predicted subsequent thromboembolic complications: inpatient status (OR, 6.38; 95 % CI, 2.17 to 18.78) and age (OR, 1.02 per year; 95 % CI, 0.99 to 1.05). The final model had a bootstrap bias-corrected c-statistic of 0.72 with a 95 % CI (0.64 to 0.79). Outpatients were at low risk (< 4 %) of developing PDVT/PE. Inpatients aged ≥ 60 years were at high risk (> 10 %). Inpatients aged < 60 were at intermediate risk. We created a simple model that can be used to risk stratify patients with isolated DDVT based on inpatient status and age. The model might be used to choose between anticoagulation and monitoring with serial ultrasounds.

Predicting Risk of Urinary Incontinence and Adverse Events After Midurethral Sling Surgery in Women.

OBJECTIVE: To construct and validate models that predict a patients risk of developing stress and urgency urinary incontinence and adverse events 12 months after sling surgery. METHODS: This was a secondary analysis of four randomized trials. Twenty-five candidate predictors (patient characteristics and urodynamic variables) were identified from the National Institute of Diabetes and Digestive and Kidney Diseases Trial of Mid-Urethral Slings (N=597). Multiple logistic models were fit to predict four different outcomes: 1) bothersome stress urinary incontinence; 2) a positive stress test; 3) bothersome urgency urinary incontinence; and 4) any adverse event up to 12 months after sling surgery. Model discrimination was measured using a concordance index. Each models concordance index was internally validated using 1,000 bootstrap samples and calibration curves were plotted. Final models were externally validated on a separate data set (n=902) from a combination of three different multicenter randomized trials. RESULTS: Four best models discriminated on internal validation between women with bothersome stress urinary incontinence (concordance index 0.728, 95% confidence interval [CI] 0.683–0.773), a positive stress test (concordance index 0.712, 95% CI 0.669–0.758), bothersome urgency urinary incontinence (concordance index 0.722, 95% CI 0.680–0.764), and any adverse event (concordance index 0.640, 95% CI 0.595–0.681) after sling surgery. Each models concordance index was reduced as expected when important variables were removed for external validation, but model discrimination remained stable with bothersome stress urinary incontinence (concordance index 0.548), a positive stress test (concordance index 0.656), bothersome urgency urinary incontinence (concordance index 0.621), and any adverse event (concordance index 0.567). Predicted probabilities are closest to actual probabilities when predictions are less than 50%. CONCLUSION: Four best and modified models discriminate between women who will and will not develop urinary incontinence and adverse events 12 months after midurethral sling surgery 64–73% and 55–66% of the time, respectively.

Using the landmark method for creating prediction models in large datasets derived from electronic health records

With the integration of electronic health records (EHRs), health data has become easily accessible and abounded. The EHR has the potential to provide important healthcare information to researchers by creating study cohorts. However, accessing this information comes with three major issues: 1) Predictor variables often change over time, 2) Patients have various lengths of follow up within the EHR, and 3) the size of the EHR data can be computationally challenging. Landmark analyses provide a perfect complement to EHR data and help to alleviate these three issues. We present two examples that utilize patient birthdays as landmark times for creating dynamic datasets for predicting clinical outcomes. The use of landmark times help to solve these three issues by incorporating information that changes over time, by creating unbiased reference points that are not related to a patient’s exposure within the EHR, and reducing the size of a dataset compared to true time-varying analysis. These techniques are shown using two example cohort studies from the Cleveland Clinic that utilized 4.5 million and 17,787 unique patients, respectively.

Patient Characteristics Associated With Severe Hypoglycemia in a Type 2 Diabetes Cohort in a Large, Integrated Health Care System From 2006 to 2015

OBJECTIVE To identify severe hypoglycemia events, defined as emergency department visits or hospitalizations for hypoglycemia, in patients with type 2 diabetes receiving care in a large health system and to identify patient characteristics associated with severe hypoglycemia events. RESEARCH DESIGN AND METHODS This was a retrospective cohort study from January 2006 to December 2015 using the electronic medical record in the Cleveland Clinic Health System (CCHS). Participants included 50,439 patients with type 2 diabetes receiving care in the CCHS. Number of severe hypoglycemia events and associated patient characteristics were identified. RESULTS The incidence proportion of severe hypoglycemia increased from 0.12% in 2006 to 0.31% in 2015 (P = 0.01). Compared with patients who did not experience severe hypoglycemia, those with severe hypoglycemia had similar median glycosylated hemoglobin (HbA1c) levels. More patients with severe hypoglycemia versus those without had a prior diagnosis of nonsevere hypoglycemia (9% vs. 2%, P < 0.001). Logistic regression confirmed an increased odds for severe hypoglycemia with insulin, sulfonylureas, increased number of diabetes medications, history of nonsevere hypoglycemia (odds ratio [OR] 3.01, P < 0.001), HbA1c <6% (42 mmol/mol) (OR 1.95, P < 0.001), black race, and increased Charlson comorbidity index. Lower odds of severe hypoglycemia were noted with higher BMI and use of metformin, dipeptidyl peptidase 4 inhibitors, and glucagon-like peptide 1 agonists. CONCLUSIONS In this retrospective study of patients with type 2 diabetes with severe hypoglycemia, patient characteristics were identified. Patients with severe hypoglycemia had previous nonsevere hypoglycemia diagnoses more frequently than those without. Identifying patients at high risk at the point of care can allow for change in modifiable risk factors and prevention of severe hypoglycemia events.

External validation of the TIMI risk score for secondary cardiovascular events among patients with recent myocardial infarction

BACKGROUND AND AIMS Risk stratification of patients with recent myocardial infarction (MI) for subsequent cardiovascular (CV) events helps identify patients most likely to benefit from secondary prevention therapies. This study externally validated a new risk score (TRS2˚P) for secondary events derived from the TRA2°P-TIMI 50 trial among post-MI patients from two large health care systems. METHODS This retrospective cohort study included 9618 patients treated for acute MI at either the Cleveland Clinic (CC) or Geisinger Health System (GHS) between 2008 and 2013. Patients with a clinic visit within 2-52 weeks of MI were included and followed for CV death, repeat MI, and ischemic stroke through electronic medical records (EMR). The TRS2˚P is based on nine factors determined through EMR documentation. Discrimination and calibration of the TRS2˚P were quantified in both patient populations. RESULTS MI patients at CC and GHS were older, had more comorbidities, received fewer medications, and had higher 3-year event rates compared to subjects in the TRA2°P trial: 31% (CC), 33% (GHS), and 10% (TRA2°P-TIMI 50). The proposed risk score had similar discrimination across the three cohorts with c-statistics of 0.66 (CC), 0.66 (GHS), and 0.67 (TRA2°P-TIMI 50). A strong graded relationship between the risk score and event rates was observed in all cohorts, though 3-year event rates were consistently higher within TRS2°P strata in the CC and GHS cohorts relative to TRA2˚P-TIMI 50. CONCLUSIONS The TRS2˚P demonstrated consistent risk discrimination across trial and non-trial patients with recent MI, but event rates were consistently higher in the non-trial cohorts.