Jordan D. Bohnen

The Feasibility of Real-Time Intraoperative Performance Assessment With SIMPL (System for Improving and Measuring Procedural Learning): Early Experience From a Multi-institutional Trial

PURPOSE Intraoperative performance assessment of residents is of growing interest to trainees, faculty, and accreditors. Current approaches to collect such assessments are limited by low participation rates and long delays between procedure and evaluation. We deployed an innovative, smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), to make real-time intraoperative performance assessment feasible for every case in which surgical trainees participate, and hypothesized that SIMPL could be feasibly integrated into surgical training programs. METHODS Between September 1, 2015 and February 29, 2016, 15 U.S. general surgery residency programs were enrolled in an institutional review board-approved trial. SIMPL was made available after 70% of faculty and residents completed a 1-hour training session. Descriptive and univariate statistics analyzed multiple dimensions of feasibility, including training rates, volume of assessments, response rates/times, and dictation rates. The 20 most active residents and attendings were evaluated in greater detail. RESULTS A total of 90% of eligible users (1267/1412) completed training. Further, 13/15 programs began using SIMPL. Totally, 6024 assessments were completed by 254 categorical general surgery residents (n = 3555 assessments) and 259 attendings (n = 2469 assessments), and 3762 unique operations were assessed. There was significant heterogeneity in participation within and between programs. Mean percentage (range) of users who completed ≥1, 5, and 20 assessments were 62% (21%-96%), 34% (5%-75%), and 10% (0%-32%) across all programs, and 96%, 75%, and 32% in the most active program. Overall, response rate was 70%, dictation rate was 24%, and mean response time was 12 hours. Assessments increased from 357 (September 2015) to 1146 (February 2016). The 20 most active residents each received mean 46 assessments by 10 attendings for 20 different procedures. CONCLUSIONS SIMPL can be feasibly integrated into surgical training programs to enhance the frequency and timeliness of intraoperative performance assessment. We believe SIMPL could help facilitate a national competency-based surgical training system, although local and systemic challenges still need to be addressed.

Implementing a pilot leadership course for internal medicine residents: design considerations, participant impressions, and lessons learned.

BackgroundEffective clinical leadership is associated with better patient care. We implemented and evaluated a pilot clinical leadership course for second year internal medicine residents at a large United States Academic Medical Center that is part of a multi-hospital health system.MethodsThe course met weekly for two to three hours during July, 2013. Sessions included large group discussions and small group reflection meetings. Topics included leadership styles, emotional intelligence, and leading clinical teams. Course materials were designed internally and featured “business school style” case studies about everyday clinical medicine which explore how leadership skills impact care delivery. Participants evaluated the course’s impact and quality using a post-course survey. Questions were structured in five point likert scale and free text format. Likert scale responses were converted to a 1-5 scale (1 = strongly disagree; 3 = neither agree nor disagree; 5 = strongly agree), and means were compared to the value 3 using one-way T-tests. Responses to free text questions were analyzed using the constant comparative method.ResultsAll sixteen pilot course participants completed the survey. Participants overwhelmingly agreed that the course provided content and skills relevant to their clinical responsibilities and leadership roles. Most participants also acknowledged that taking the course improved their understanding of their strengths and weaknesses as leaders, different leadership styles, and how to manage interpersonal conflict on clinical teams. 88% also reported that the course increased their interest in pursuing additional leadership training.ConclusionsA clinical leadership course for internal medicine residents designed by colleagues, and utilizing case studies about clinical medicine, resulted in significant self-reported improvements in clinical leadership competencies.

Readiness of US General Surgery Residents for Independent Practice

Objective: This study evaluates the current state of the General Surgery (GS) residency training model by investigating resident operative performance and autonomy. Background: The American Board of Surgery has designated 132 procedures as being “Core” to the practice of GS. GS residents are expected to be able to safely and independently perform those procedures by the time they graduate. There is growing concern that not all residents achieve that standard. Lack of operative autonomy may play a role. Methods: Attendings in 14 General Surgery programs were trained to use a) the 5-level System for Improving and Measuring Procedural Learning (SIMPL) Performance scale to assess resident readiness for independent practice and b) the 4-level Zwisch scale to assess the level of guidance (ie, autonomy) they provided to residents during specific procedures. Ratings were collected immediately after cases that involved a categorical GS resident. Data were analyzed using descriptive statistics and supplemented with Bayesian ordinal model-based estimation. Results: A total of 444 attending surgeons rated 536 categorical residents after 10,130 procedures. Performance: from the first to the last year of training, the proportion of Performance ratings for Core procedures (n = 6931) at “Practice Ready” or above increased from 12.3% to 77.1%. The predicted probability that a typical trainee would be rated as Competent after performing an average Core procedure on an average complexity patient during the last week of residency training is 90.5% (95% CI: 85.7%–94%). This falls to 84.6% for more complex patients and to less than 80% for more difficult Core procedures. Autonomy: for all procedures, the proportion of Zwisch ratings indicating meaningful autonomy (“Passive Help” or “Supervision Only”) increased from 15.1% to 65.7% from the first to the last year of training. For the Core procedures performed by residents in their final 6 months of training (cholecystectomy, inguinal/femoral hernia repair, appendectomy, ventral hernia repair, and partial colectomy), the proportion of Zwisch ratings (n = 357) indicating near-independence (“Supervision Only”) was 33.3%. Conclusions: US General Surgery residents are not universally ready to independently perform Core procedures by the time they complete residency training. Progressive resident autonomy is also limited. It is unknown if the amount of autonomy residents do achieve is sufficient to ensure readiness for the entire spectrum of independent practice.

Reconceiving the Morbidity and Mortality Conference in an Era of Big Data: An "Unexpected" Outcomes Approach.

S ince their introduction by Dr. Ernest A. Codman in the early 20th century, morbidity and mortality (M&M) conferences have become standard practice in modern medicine. Many institutions now uphold the M&M tradition by presenting all complications and deaths, whereas others have applied new formats, for example, by mapping learning lessons to improvement aims or core competencies. Advances in ‘‘big data’’ and outcomes research offer new quality improvement (QI) tools to facilitate comparison of healthcare outcomes. For example, big data analyses have identified patientspecific risk factors for the development of disease; provider-specific performance metrics; and population-level risk-adjustment algorithms to compare complication rates across institutions and regions. However, the application of big data and outcomes research to QI has not been fully realized. One area that may be particularly conducive to big data approaches is the M&M conference.

Collaborating internationally on physician leadership development: why now?

Purpose This paper aims to highlight the importance of leadership development for all physicians within a competency-based medical education (CBME) framework. It describes the importance of timely international collaboration as a key strategy in promoting physician leadership development. Design/methodology/approach The paper explores published and Grey literature around physician leadership development and proposes that international collaboration will meet the expanding call for development of leadership competencies in postgraduate medical learners. Two grounding frameworks were used: complexity science supports adding physician leadership training to the current momentum of CBME adoption, and relational cultural theory supports the engagement of diverse stakeholders in multiple jurisdictions around the world to ensure inclusivity in leadership education development. Findings An international collaborative identified key insights regarding the need to frame physician leadership education within a competency-based model. Practical implications International collaboration can be a vehicle for developing a globally relevant, generalizable physician leadership curriculum. This model can be expanded to encourage innovation, scholarship and program evaluation. Originality/value A competency-based leadership development curriculum is being designed by an international collaborative. The curriculum is based on established leadership and education frameworks. The international collaboration model provides opportunities for ongoing sharing, networking and diversification.

Uniting Evidence-based Evaluation with the Acgme Plastic Surgery Milestones: A Simple and Reliable Assessment of Resident Operative Performance.

Background: Milestones evaluations in plastic surgery reflect a shift toward competency-based training but have created a number of challenges. The authors have incorporated the smartphone application evaluation tool, System for Improving and Measuring Procedural Learning (SIMPL), that was recently developed by a multi-institutional research collaborative. In this pilot study, the authors hypothesize that SIMPL can improve resident evaluation and also collect granular performance data to simplify compliance with the plastic surgery Milestones. Methods: SIMPL was prospectively piloted with a plastic surgery resident and faculty surgeon at Massachusetts General Hospital in this institutional review board–approved study. The study period was a 2-month interval corresponding to the resident’s rotation. Results: The resident-faculty combination performed 20 cases together. All cases were evaluated with SIMPL. SIMPL evaluations uniformly took under 1 minute to submit. The average time to completed evaluation from surgery completion was 5 hours (<0.5 hour to 12 hours). Concrete, objective, and specific data about resident performance were collected for every case and presented in a concise format. Conclusions: SIMPL is an innovative, evidence-based evaluation system that makes performance assessment feasible for every procedure in which a plastic surgery resident participates. SIMPL’s competency-based framework can be easily scaled to facilitate data collection and reporting of mandatory Milestones evaluations at the program and national levels. This technology will support a shared vocabulary between residents and faculty to enhance intraoperative education.

Collaborating internationally on physician leadership education: first steps.

Purpose Physicians are often ill-equipped for the leadership activities their work demands. In part, this is due to a gap in traditional medical education. An emergent international network is developing a globally relevant leadership curriculum for postgraduate medical education. The purpose of this article is to share key learnings from this process to date. Design/methodology/approach The Toronto International Summit on Leadership Education for Physicians (TISLEP) was hosted by the Royal College of Physicians and Surgeons of Canada, and the University of Torontos Faculty of Medicine and Institute of Health Policy, Management and Evaluation. Of 64 attendees from eight countries, 34 joined working groups to develop leadership competencies. The CanMEDS Competency Framework, stage of learner development and venue of learning formed the scaffold for the work. Emotional intelligence was selected as the topic to test the feasibility of fruitful international collaboration; results were presented at TISLEP 2015. Findings Dedicated international stakeholders engaged actively and constructively through defined working groups to develop a globally relevant, competency-based curriculum for physician leadership education. Eleven principles are recommended for consideration in physician leadership curriculum development. Defining common language and taxonomy is essential for a harmonized product. The importance of establishing an international network to support implementation, evaluation, sustainability and dissemination of the work was underscored. Originality/value International stakeholders are collaborating successfully on a graduated, competency-based leadership curriculum for postgraduate medical learners. The final product will be available for adaptation to local needs. An international physician leadership education network is being developed to support and expand the work underway.

Sex Differences in Faculty Rank Among Academic Surgeons in the United States in 2014

Objective: The aim of this study was to evaluate sex differences in full professorship among a comprehensive, contemporary cohort of US academic surgeons. Summary of Background Data: Previous work demonstrates that women are less likely than men to be full professors in academic medicine, and in certain surgical subspecialties. Whether sex differences in academic rank exist across all surgical fields, and after adjustment for confounders, is not known. Methods: A comprehensive list of surgeons with faculty appointments at US medical schools in 2014 was obtained from Association of American Medical Colleges (AAMC) faculty roster and linked to a comprehensive physician database from Doximity, an online physician networking website, which contained the following data for all physicians: sex, age, years since residency, publication number (total and first/last author), clinical trials participation, National Institutes of Health grants, and surgical subspecialty. A 20% sample of 2013 Medicare payments for care was added to this dataset. Multivariable regression models were used to estimate sex differences in full professorship, adjusting for these variables and medical school-specific fixed effects. Results: Among 11,549 surgeon faculty at US medical schools in 2014, 1692 (14.7%) were women. Women comprised 19.4% of assistant professors (1072/5538), 13.8% of associate professors (404/2931), and 7.0% of full professors (216/3080). After multivariable analysis, women were less likely to be full professors than men (adjusted odds ratio: 0.76, 95% confidence interval: 0.6–0.9). Conclusion: Among surgical faculty at US medical schools in 2014, women were less likely than men to be full professors after adjustment for multiple factors known to impact faculty rank.

Perioperative risk factors impact outcomes in emergency versus nonemergency surgery differently: Time to separate our national risk-adjustment models?

BACKGROUND Emergency surgery (ES) is acknowledged to be riskier than nonemergency surgery (NES). Yet, little is known about the relative impact of individual perioperative risk factors on 30-day outcomes in ES versus NES. METHODS Using the 2011–2012 American College of Surgeons National Surgical Quality Improvement Program nationwide database, the 20 most common ES procedures were identified by Current Procedural Terminology code. Current Procedural Terminology codes with less than 300 observations in either ES or NES were excluded. Emergency surgery cases were defined as “emergent” and “nonelective” per American College of Surgeons National Surgical Quality Improvement Program criteria. Multivariable regression models were constructed to identify predictors of 30-day major morbidity and mortality (MMM) in each group, controlling for demographics, American Society of Anesthesiologists class, comorbidities, preoperative laboratory values, and procedure type. The odds ratios of independent predictors of MMM in ES and NES were derived then individually compared between the two groups; “effect modification” of procedure status (ES vs. NES) on each risk factor was subsequently calculated. RESULTS Of 986,034 patients, 170,131 met inclusion criteria (59,949 ES; 110,182 NES). The overall risk of MMM was significantly higher in ES versus NES (16.75% vs. 9.73%, p < 0.001; odds ratio, 1.18; 95% confidence interval, 1.12–1.24; p < 0.001). Of 40 ES- and 38 NES-identified independent risk factors, preoperative transfusion and white blood cell count of 4.5 × 103/&mgr;L or less carried significantly higher relative risk of MMM in ES versus NES. Conversely, ascites, preoperative anemia, and white blood cell count of 11 × 103/&mgr;L to 25 × 103/&mgr;L carried greater relative risk for MMM in NES. Four procedures (laparoscopic cholecystectomy, laparotomy, and umbilical and incisional herniorrhaphy) were inherently riskier in ES versus NES. The effect modification of ES (vs. NES) ranged between 0.68 (0.52–0.88) for ascites and 2.56 (1.67–3.92) for umbilical hernia repair. CONCLUSIONS Perioperative risk factors and procedure type impact postoperative morbidity and mortality differently in ES versus NES. Instead of using the same risk-adjustment model for both ES and NES, as currently practiced, our findings strongly suggest the need to benchmark emergent and elective surgeries separately. LEVEL OF EVIDENCE Prognostic/epidemiologic study, level III.

Derivation and Validation of a Novel Physiological Emergency Surgery Acuity Score (PESAS)

BackgroundWe present a novel and abbreviated Physiological Emergency Surgery Acuity Score (PESAS) that assesses the severity of disease at presentation in patients undergoing Emergency Surgery (ES).MethodsUsing the 2011 ACS-NSQIP database, we identified all patients who underwent “emergent” surgery. The following methodology was designed: (1) identification of independent predictors of 30-day mortality that are markers of acuity; (2) derivation of PESAS based on the relative impact (i.e., odds ratio) of each predictor; and (3) measurement of the c-statistic. The PESAS was validated using the 2012 ACS-NSQIP database.ResultsFrom 24,702 ES cases, a 15-point score was derived. This included 10 components with a range of 0 and 15 points. Its c-statistic was 0.80. Mortality gradually increased from 1.7 to 40.6 to 100% at scores of 0, 8, and 15, respectively. In the validation phase, PESAS c-statistic remained stable.ConclusionPESAS is a novel score that assesses the acuity of disease at presentation in ES patients and strongly correlates with postoperative mortality risk. PESAS could prove useful for preoperative counseling and for risk-adjusted benchmarking.