Allison R. Dahlke

Association of the 2011 ACGME resident duty hour reform with general surgery patient outcomes and with resident examination performance.

IMPORTANCE In 2011, the Accreditation Council for Graduate Medical Education (ACGME) restricted resident duty hour requirements beyond those established in 2003, leading to concerns about the effects on patient care and resident training. OBJECTIVE To determine if the 2011 ACGME duty hour reform was associated with a change in general surgery patient outcomes or in resident examination performance. DESIGN, SETTING, AND PARTICIPANTS Quasi-experimental study of general surgery patient outcomes 2 years before (academic years 2009-2010) and after (academic years 2012-2013) the 2011 duty hour reform. Teaching and nonteaching hospitals were compared using a difference-in-differences approach adjusted for procedural mix, patient comorbidities, and time trends. Teaching hospitals were defined based on the proportion of cases at which residents were present intraoperatively. Patients were those undergoing surgery at hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). General surgery resident performance on the annual in-training, written board, and oral board examinations was assessed for this same period. EXPOSURES National implementation of revised resident duty hour requirements on July 1, 2011, in all ACGME accredited residency programs. MAIN OUTCOMES AND MEASURES Primary outcome was a composite of death or serious morbidity; secondary outcomes were other postoperative complications and resident examination performance. RESULTS In the main analysis, 204,641 patients were identified from 23 teaching (n = 102,525) and 31 nonteaching (n = 102,116) hospitals. The unadjusted rate of death or serious morbidity improved during the study period in both teaching (11.6% [95% CI, 11.3%-12.0%] to 9.4% [95% CI, 9.1%-9.8%], P < .001) and nonteaching hospitals (8.7% [95% CI, 8.3%-9.0%] to 7.1% [95% CI, 6.8%-7.5%], P < .001). In adjusted analyses, the 2011 ACGME duty hour reform was not associated with a significant change in death or serious morbidity in either postreform year 1 (OR, 1.12; 95% CI, 0.98-1.28) or postreform year 2 (OR, 1.00; 95% CI, 0.86-1.17) or when both postreform years were combined (OR, 1.06; 95% CI, 0.93-1.20). There was no association between duty hour reform and any other postoperative adverse outcome. Mean (SD) in-training examination scores did not significantly change from 2010 to 2013 for first-year residents (499.7 [ 85.2] to 500.5 [84.2], P = .99), for residents from other postgraduate years, or for first-time examinees taking the written or oral board examinations during this period. CONCLUSIONS AND RELEVANCE Implementation of the 2011 ACGME duty hour reform was not associated with a change in general surgery patient outcomes or differences in resident examination performance. The implications of these findings should be considered when evaluating the merit of the 2011 ACGME duty hour reform and revising related policies in the future.

Food Insecurity in Relation to Changes in Hemoglobin A1c, Self-Efficacy, and Fruit/Vegetable Intake During a Diabetes Educational Intervention

OBJECTIVE Food insecurity is hypothesized to make diabetes self-management more difficult. We conducted a longitudinal assessment of food insecurity with several diabetes self-care measures. RESEARCH DESIGN AND METHODS We conducted a secondary, observational analysis of 665 low-income patients with diabetes, all of whom received self-management support as part of a larger diabetes educational intervention. We analyzed baseline food insecurity (measured by the U.S. Department of Agriculture Food Security module) in relation to changes in hemoglobin A1c (HbA1c) as well as self-reported diabetes self-efficacy and daily fruit and vegetable intake. We examined longitudinal differences using generalized estimating equation linear regression models, controlling for time, age, sex, race, income, and intervention arm. RESULTS Overall, 57% of the sample had an income <

Development of the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial Protocol: A National Cluster-Randomized Trial of Resident Duty Hour Policies

15,000. Participants who were food insecure (33%) were younger, had less income, and were more likely to be unemployed compared with participants who were food secure. At baseline, those who were food insecure had higher mean HbA1c values (8.4% vs. 8.0%) and lower self-efficacy and fruit and vegetable intake than those who were food secure (all P < 0.05). Compared with food-secure individuals, participants who were food insecure had significantly greater improvements in HbA1c over time (0.38% decrease compared with 0.01% decrease; P value for interaction <0.05) as well as in self-efficacy (P value for interaction <0.01). There was no significant difference in HbA1c by food security status at follow-up. CONCLUSIONS Participants experiencing food insecurity had poorer diabetes-related measures at baseline but made significant improvements in HbA1c and self-efficacy. Low-income patients who were food insecure may be particularly receptive to diabetes self-management support, even if interventions are not explicitly structured to address finances or food security challenges.

Patient Factors That Affect Quality of Colonoscopy Preparation

IMPORTANCE Debate continues regarding whether to further restrict resident duty hour policies, but little high-level evidence is available to guide policy changes. OBJECTIVE To inform decision making regarding duty hour policies, the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial is being conducted to evaluate whether changing resident duty hour policies to permit greater flexibility in work hours affects patient postoperative outcomes, resident education, and resident well-being. DESIGN, SETTING, AND PARTICIPANTS Pragmatic noninferiority cluster-randomized trial of general surgery residency programs with 2 study arms. Participating in the study are Accreditation Council for Graduate Medical Education (ACGME)-approved US general surgery residency programs (n = 118), their affiliated hospitals (n = 154), surgical residents and program directors, and general surgery patients from July 1, 2014, to June 30, 2015, with additional patient safety outcomes collected through June 30, 2016. The data collection platform for patient outcomes is the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP), thus only hospitals participating in the ACS NSQIP were included. INTERVENTIONS In the usual care arm, programs adhered to current ACGME resident duty hour standards. In the intervention arm, programs were allowed to deviate from current standards regarding maximum shift lengths and minimum time off between shifts through an ACGME waiver. MAIN OUTCOMES AND MEASURES Death or serious morbidity within 30 days of surgery measured through ACS NSQIP, as well as resident satisfaction and well-being measured through a survey delivered at the time of the 2015 American Board of Surgery in Training Examination (ABSITE). RESULTS A total of 118 general surgery residency programs and 154 hospitals were enrolled in the FIRST Trial and randomized. Fifty-nine programs (73 hospitals) were randomized to the usual care arm and 59 programs (81 hospitals) were randomized to the intervention arm. Intent-to-treat analysis will be used to estimate the effectiveness of assignment to the intervention arm on patient outcomes, resident education, and resident well-being compared with the usual care arm. Several sensitivity analyses will be performed to determine whether there were differential effects when examining only inpatients, high-risk patients, and emergent/urgent cases. CONCLUSIONS AND RELEVANCE To our knowledge, the FIRST Trial is the first national randomized clinical trial of duty hour policies. Results of this study may be informative to policymakers and other stakeholders engaged in restructuring graduate medical training to enhance the quality of patient care and resident education. TRIAL REGISTRATION clinicaltrials.org Identifier: NCT02050789.

Evaluation of Initial Participation in Public Reporting of American College of Surgeons NSQIP Surgical Outcomes on Medicare's Hospital Compare Website

BACKGROUND & AIMS Optimal colonoscopy preparation requires patients to adhere to written instructions and be activated to complete the task. Among patients with chronic disease, health literacy and patient activation have been associated with outcome, but these factors have not been studied for colonoscopy. We examined the association between health literacy, patient activation, and quality of bowel preparation. METHODS We analyzed outpatient colonoscopy results from 462 adults, 55-74 years old (mean, 62 ± 6 years), who previously completed extensive neurocognitive assessments as part of a prospective study (Health Literacy and Cognitive Function in Older Adults). We collected information on cecal intubation, polyp detection, bowel preparation quality, and histopathology. RESULTS One-third of the patients (n = 134) had suboptimal quality of bowel preparation; 15% (n = 62) had fair quality, and 17% (n = 72) had poor quality. Limited health literacy was associated with a lower level of education (P < .001), diabetes (P < .001), and a higher number of chronic conditions (P < .001), but not quality of colonoscopy preparation. No baseline characteristics were associated with patient activation. In multivariable analysis, after adjusting for demographics and clinical characteristics, diabetes (odds ratio [OR], 2.45; 95% confidence interval [CI], 1.14-5.25) and patient activation (OR, 2.12; 95% CI, 1.30-3.45) were independent predictors of suboptimal bowel preparation quality, but limited health literacy was not (OR, 0.76; 95% CI, 0.38-1.52). CONCLUSIONS We investigated the relationship between health literacy, patient activation, and colonoscopy preparation quality. Lower patient activation was an independent predictor of suboptimal bowel preparation quality. Interventions to improve colonoscopy preparation quality should consider the importance of patient activation within their design.

Association of the 2011 ACGME Resident Duty Hour Reform with Postoperative Patient Outcomes in Surgical Specialties

Background In October 2012, The Centers for Medicare and Medicaid Services (CMS) began publicly reporting American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) surgical outcomes on its public reporting website, Hospital Compare. Participation in this CMS-NSQIP initiative is voluntary. Our objective was to compare CMS-NSQIP participating hospitals to ACS NSQIP hospitals that elected not to participate. Study Design Hospital Compare and American Hospital Association Annual Survey data were merged to compare CMS-NSQIP participants to non-participants. Regression models were developed to assess predictors of participation and to assess if hospitals differed on 32 process, 10 patient experience (HCAHPS), and 16 outcome (Hospital Compare & AHRQ) measures. Additionally, performance on two waves of publicly reported ACS NSQIP surgical outcome measures was compared. Results Of the 452 ACS NSQIP hospitals, 80 (18%) participated in CMS-NSQIP public reporting. Participating hospitals had more beds, admissions, operations, and were more often accredited (Commission on Cancer and the Council of Teaching Hospitals (COTH) (P<0.05). Only COTH membership remained significant in adjusted analyses (OR 2.45, 95% CI 1.12–5.35). Hospital performance on process, HCAHPS, and outcome measures were not associated with CMS-NSQIP participation for 54 of 58 measures examined. Hospitals with “better-than-average” performance were more likely to publicly report the Elderly Surgery measure (P<0.05). In wave two, an increased proportion of new participants reported “worse-than-average” outcomes. Conclusions There were few measurable differences between CMS-NSQIP participating and non-participating hospitals. The decision to voluntarily publicly report may be related to the hospital’s culture of quality improvement and transparency.

The Mini Mental Status Exam as a Surrogate Measure of Health Literacy

BACKGROUND The 2011 ACGME resident duty hour reform implemented additional restrictions to existing duty hour policies. Our objective was to determine the association between this reform and patient outcomes among several surgical specialties. STUDY DESIGN Patients from 5 surgical specialties (neurosurgery, obstetrics/gynecology, orthopaedic surgery, urology, and vascular surgery) were identified from the American College of Surgeons NSQIP. Data from 1 year before and 2 years after the reform was implemented were obtained for teaching and nonteaching hospitals. Hospital teaching status was defined based on the percentage of operations with a resident present intraoperatively. Difference-in-differences models were developed separately for each specialty and adjusted for patient demographics, comorbidities, procedural case-mix, and time trends. The association between duty hour reform and a composite measure of death or serious morbidity within 30 days of surgery was estimated for each specialty. RESULTS The unadjusted rate of death or serious morbidity decreased during the study period in both teaching and nonteaching hospitals for all surgical specialties. In multivariable analyses, there were no significant associations between duty hour reform and the composite outcomes of death or serious morbidity in the 2 years post-reform for any surgical specialty evaluated (neurosurgery: odds ratio [OR] = 0.90; 95% CI, 0.75-1.08; p = 0.26; obstetrics/gynecology: OR = 0.96; 95% CI, 0.71-1.30; p = 0.80; orthopaedic surgery: OR = 0.95; 95% CI, 0.74-1.22; p = 0.70; urology: OR = 1.16; 95% CI, 0.89-1.51; p = 0.26; vascular surgery: OR = 1.07; 95% CI, 0.93-1.22; p = 0.35). CONCLUSIONS Implementation of the 2011 ACGME resident duty hour reform was not associated with a significant change in patient outcomes for several surgical specialties in the 2 years after reform.

Comparison of postoperative complication risk prediction approaches based on factors known preoperatively to surgeons versus patients

ABSTRACTBACKGROUNDStudies have documented strong associations between cognitive function, health literacy skills, and health outcomes, such that outcome performance may be partially explained by cognitive ability. Common cognitive assessments such as the Mini Mental Status Exam (MMSE) therefore may be measuring the same latent construct as existing health literacy tools.OBJECTIVESWe evaluated the potential of the MMSE as a surrogate measure of health literacy by comparing its convergent and predictive validity to the three most commonly used health literacy assessments and education.SUBJECTS827 older adults recruited from an academic general internal medicine ambulatory care clinic or one of five federally qualified health centers in Chicago, IL. Non-English speakers and those with severe cognitive impairment were excluded.MEASURESPearson correlations were completed to test the convergent validity of the MMSE with assessments of health literacy and education. Receiver Operating Characteristic (ROC) curves and the d statistic were calculated to determine the optimal cut point on the MMSE for classifying participants with limited health literacy. Multivariate logistic regression models were completed to measure the predictive validity of the new MMSE cut point.KEY RESULTSThe MMSE was found to have moderate to high convergent validity with the existing health literacy measures. The ROC and d statistic analyses suggested an optimal cut point of ≤ 27 on the MMSE. The new threshold score was found to predict health outcomes at least as well as, or better than, existing health literacy measures or education alone.CONCLUSIONSThe MMSE has considerable face validity as a health literacy measure that could be easily administered in the healthcare setting. Further research should aim to validate this cut point and examine the constructs being measured by the MMSE and other literacy assessments.Studies have documented strong associations between cognitive function, health literacy skills, and health outcomes, such that outcome performance may be partially explained by cognitive ability. Common cognitive assessments such as the Mini Mental Status Exam (MMSE) therefore may be measuring the same latent construct as existing health literacy tools. We evaluated the potential of the MMSE as a surrogate measure of health literacy by comparing its convergent and predictive validity to the three most commonly used health literacy assessments and education. 827 older adults recruited from an academic general internal medicine ambulatory care clinic or one of five federally qualified health centers in Chicago, IL. Non-English speakers and those with severe cognitive impairment were excluded. Pearson correlations were completed to test the convergent validity of the MMSE with assessments of health literacy and education. Receiver Operating Characteristic (ROC) curves and the d statistic were calculated to determine the optimal cut point on the MMSE for classifying participants with limited health literacy. Multivariate logistic regression models were completed to measure the predictive validity of the new MMSE cut point. The MMSE was found to have moderate to high convergent validity with the existing health literacy measures. The ROC and d statistic analyses suggested an optimal cut point of ≤ 27 on the MMSE. The new threshold score was found to predict health outcomes at least as well as, or better than, existing health literacy measures or education alone. The MMSE has considerable face validity as a health literacy measure that could be easily administered in the healthcare setting. Further research should aim to validate this cut point and examine the constructs being measured by the MMSE and other literacy assessments.

Use and Underlying Reasons for Duty Hour Flexibility in the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial

BACKGROUND Estimating the risk of postoperative complications can be performed by surgeons with detailed clinical information or by patients with limited information. Our objective was to compare three estimation models: (1) the All Information Model, using variables available from the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP); (2) the Surgeon Assessment Model, using variables available to surgeons preoperatively, and (3) the Patient-Entered Model, using information that patients know about their own health. STUDY DESIGN Using the ACS NSQIP 2011 data for general and colon surgery, standard ACS NSQIP regression methods were used to develop models. Each model examined Overall and Serious Morbidity as outcomes. The models were assessed using the c-statistic, Hosmer-Lemshow statistic, and Akaike Information Criterion. RESULTS The overall morbidity rate was 13.0%, and the serious morbidity rate was 10.5% for patients undergoing general surgery (colon surgery: 31.8% and 26.0%, respectively). There was a small decrement in the c-statistic as the number of predictors decreased. The Akaike Information Criterion likelihood ratio increased between the All Information and Surgeon Assessment models, but decreased in the Patient-Entered Model. The Hosmer-Lemshow statistic suggested good model fit for five colon surgery models and one general surgery model. CONCLUSION Although a small decline in model performance was observed, the magnitude suggests that it may not be clinically meaningful as the risk predictions offered are superior to simply providing unadjusted complications rates. The Surgeon Assessment and Patient-Entered models with fewer predictors can be used with relative confidence to predict a patients risk.

Public reporting in surgery an emerging opportunity to improve care and inform patients

BACKGROUND The Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial randomly assigned surgical residency programs to either standard duty hour policies or flexible policies that eliminated caps on shift lengths and time off between shifts. Our objectives were to assess adherence to duty hour requirements in the Standard Policy arm and examine how often and why duty hour flexibility was used in the Flexible Policy arm. STUDY DESIGN A total of 3,795 residents in the FIRST trial completed a survey in January 2016 (response rate >95%) that asked how often and why they exceeded current standard duty hour limits in both study arms. RESULTS Flexible Policy interns worked more than 16 hours continuously at least once in a month more frequently than Standard Policy residents (86% vs 37.8%). Flexible Policy residents worked more than 28 hours once in a month more frequently than Standard Policy residents (PGY1: 64% vs 2.9%; PGY2 to 3: 62.4% vs 41.9%; PGY4 to 5: 52.2% vs 36.6%), but this occurred most frequently only 1 to 2 times per month. Although residents reported working more than 80 hours in a week 3 or more times in the most recent month more frequently under Flexible Policy vs Standard Policy (19.9% vs 16.2%), the difference was driven by interns (30.9% vs 19.6%), and there were no significant differences in exceeding 80 hours among PGY2 to 5 residents. The most common reasons reported for extending duty hours were facilitating care transitions (76.6%), stabilizing critically ill patients (70.7%), performing routine responsibilities (67.9%), and operating on patients known to the trainee (62.0%). CONCLUSIONS There were differences in duty hours worked by residents in the Flexible vs Standard Policy arms of the FIRST trial, but it appeared that residents generally used the flexibility for patient care and educational opportunities selectively.