Carla M. Pugh

Quantifying technical skills during open operations using video-based motion analysis

INTRODUCTION Objective quantification of technical operative skills in surgery remains poorly defined, although the delivery of and training in these skills is essential to the profession of surgery. Attempts to measure hand kinematics to quantify operative performance primarily have relied on electromagnetic sensors attached to the surgeons hand or instrument. We sought to determine whether a similar motion analysis could be performed with a marker-less, video-based review, allowing for a scalable approach to performance evaluation. METHODS We recorded six reduction mammoplasty operations-a plastic surgery procedure in which the attending and resident surgeons operate in parallel. Segments representative of surgical tasks were identified with Multimedia Video Task Analysis software. Video digital processing was used to extract and analyze the spatiotemporal characteristics of hand movement. RESULTS Attending plastic surgeons appear to use their nondominant hand more than residents when cutting with the scalpel, suggesting more use of countertraction. While suturing, attendings were more ambidextrous, with smaller differences in movement between their dominant and nondominant hands than residents. Attendings also seem to have more conservation of movement when performing instrument tying than residents, as demonstrated by less nondominant hand displacement. These observations were consistent within procedures and between the different attending plastic surgeons evaluated in this fashion. CONCLUSION Video motion analysis can be used to provide objective measurement of technical skills without the need for sensors or markers. Such data could be valuable in better understanding the acquisition and degradation of operative skills, providing enhanced feedback to shorten the learning curve.

Surgical data science for next-generation interventions

Interventional healthcare will evolve from an artisanal craft based on the individual experiences, preferences and traditions of physicians into a discipline that relies on objective decision-making on the basis of large-scale data from heterogeneous sources.Lena Maier-Hein, Swaroop Vedula, Stefanie Speidel, Nassir Navab, Ron Kikinis, Adrian Park, Matthias Eisenmann, Hubertus Feussner, Germain Forestier, Stamatia Giannarou, Makoto Hashizume, Darko Katic, Hannes Kenngott, Michael Kranzfelder, Anand Malpani, Keno März, Thomas Neumuth, Nicolas Padoy, Carla Pugh, Nicolai Schoch, Danail Stoyanov, Russell Taylor, Martin Wagner, Gregory D. Hager, Pierre Jannin

Use of Mannequin-Based Simulation to Decrease Student Anxiety Prior to Interacting With Male Teaching Associates

Background: Previous studies have compared the usefulness of teaching associates versus mannequin trainers for learning physical exam skills. Little work has been done to assess the usefulness of mannequin trainers prior to students’ interaction with teaching associates. Purpose: We studied the effects of mannequin-based simulators on student comfort levels toward learning the male genitourinary examination. Methods: First-year medical students (N = 346) were surveyed before and after a mannequin-based curriculum to assess their comfort levels toward learning the male genitourinary examination. Results: The mannequin-based curriculum significantly increased (p < .001) student comfort levels toward the male genitourinary exam. However, the pre–post improvements were small, and on average students only progressed from being “very uncomfortable” to “somewhat comfortable.” The intimate nature of the examination was the top cause of anxiety toward learning the male genitourinary exam. Students were least comfortable with the digital rectal examination at the beginning of class. Conclusions: We suggest that mannequin-based simulators be used prior to students’ experience with male teaching associates when learning the male genitourinary exam.

The use of error analysis to assess resident performance

BACKGROUND The aim of this study was to assess validity of a human factors error assessment method for evaluating resident performance during a simulated operative procedure. METHODS Seven postgraduate year 4-5 residents had 30 minutes to complete a simulated laparoscopic ventral hernia (LVH) repair on day 1 of a national, advanced laparoscopic course. Faculty provided immediate feedback on operative errors and residents participated in a final product analysis of their repairs. Residents then received didactic and hands-on training regarding several advanced laparoscopic procedures during a lecture session and animate lab. On day 2, residents performed a nonequivalent LVH repair using a simulator. Three investigators reviewed and coded videos of the repairs using previously developed human error classification systems. RESULTS Residents committed 121 total errors on day 1 compared with 146 on day 2. One of 7 residents successfully completed the LVH repair on day 1 compared with all 7 residents on day 2 (P = .001). The majority of errors (85%) committed on day 2 were technical and occurred during the last 2 steps of the procedure. There were significant differences in error type (P ≤ .001) and level (P = .019) from day 1 to day 2. The proportion of omission errors decreased from day 1 (33%) to day 2 (14%). In addition, there were more technical and commission errors on day 2. CONCLUSION The error assessment tool was successful in categorizing performance errors, supporting known-groups validity evidence. Evaluating resident performance through error classification has great potential in facilitating our understanding of operative readiness.

Relationship Between Technical Errors and Decision-Making Skills in the Junior Resident

OBJECTIVE The purpose of this study is to coevaluate resident technical errors and decision-making capabilities during placement of a subclavian central venous catheter (CVC). We hypothesize that there would be significant correlations between scenario-based decision-making skills and technical proficiency in central line insertion. We also predict residents would face problems in anticipating common difficulties and generating solutions associated with line placement. DESIGN Participants were asked to insert a subclavian central line on a simulator. After completion, residents were presented with a real-life patient photograph depicting CVC placement and asked to anticipate difficulties and generate solutions. Error rates were analyzed using chi-square tests and a 5% expected error rate. Correlations were sought by comparing technical errors and scenario-based decision-making skills. SETTING This study was performed at 7 tertiary care centers. PARTICIPANTS Study participants (N = 46) largely consisted of first-year research residents who could be followed longitudinally. Second-year research and clinical residents were not excluded. RESULTS In total, 6 checklist errors were committed more often than anticipated. Residents committed an average of 1.9 errors, significantly more than the 1 error, at most, per person expected (t(44) = 3.82, p < 0.001). The most common error was performance of the procedure steps in the wrong order (28.5%, p < 0.001). Some of the residents (24%) had no errors, 30% committed 1 error, and 46 % committed more than 1 error. The number of technical errors committed negatively correlated with the total number of commonly identified difficulties and generated solutions (r (33) = -0.429, p = 0.021, r (33) = -0.383, p = 0.044, respectively). CONCLUSIONS Almost half of the surgical residents committed multiple errors while performing subclavian CVC placement. The correlation between technical errors and decision-making skills suggests a critical need to train residents in both technique and error management.

Advanced Engineering Technology for Measuring Performance.

The demand for competency-based assessments in surgical training is growing. Use of advanced engineering technology for clinical skills assessment allows for objective measures of hands-on performance. Clinical performance can be assessed in several ways via quantification of an assessees hand movements (motion tracking), direction of visual attention (eye tracking), levels of stress (physiologic marker measurements), and location and pressure of palpation (force measurements). Innovations in video recording technology and qualitative analysis tools allow for a combination of observer- and technology-based assessments. Overall the goal is to create better assessments of surgical performance with robust validity evidence.

Can a virtual reality assessment of fine motor skill predict successful central line insertion

BACKGROUND Due to the increased use of peripherally inserted central catheter lines, central lines are not performed as frequently. The aim of this study is to evaluate whether a virtual reality (VR)-based assessment of fine motor skills can be used as a valid and objective assessment of central line skills. METHODS Surgical residents (N = 43) from 7 general surgery programs performed a subclavian central line in a simulated setting. Then, they participated in a force discrimination task in a VR environment. Hand movements from the subclavian central line simulation were tracked by electromagnetic sensors. Gross movements as monitored by the electromagnetic sensors were compared with the fine motor metrics calculated from the force discrimination tasks in the VR environment. RESULTS Long periods of inactivity (idle time) during needle insertion and lack of smooth movements, as detected by the electromagnetic sensors, showed a significant correlation with poor force discrimination in the VR environment. Also, long periods of needle insertion time correlated to the poor performance in force discrimination in the VR environment. CONCLUSIONS This study shows that force discrimination in a defined VR environment correlates to needle insertion time, idle time, and hand smoothness when performing subclavian central line placement. Fine motor force discrimination may serve as a valid and objective assessment of the skills required for successful needle insertion when placing central lines.

Application of National Testing Standards to Simulation-Based Assessments of Clinical Palpation Skills

With the advent of simulation technology, several types of data acquisition methods have been used to capture hands-on clinical performance. Motion sensors, pressure sensors, and tool-tip interaction software are a few of the broad categories of approaches that have been used in simulation-based assessments. The purpose of this article is to present a focused review of 3 sensor-enabled simulations that are currently being used for patient-centered assessments of clinical palpation skills. The first part of this article provides a review of technology components, capabilities, and metrics. The second part provides a detailed discussion regarding validity evidence and implications using the Standards for Educational and Psychological Testing as an organizational and evaluative framework. Special considerations are given to content domain and creation of clinical scenarios from a developers perspective. The broader relationship of this work to the science of touch is also considered.

A structured, extended training program to facilitate adoption of new techniques for practicing surgeons

IntroductionLaparoscopic inguinal hernia repair has been shown to have significant benefits when compared to open inguinal hernia repair, yet remains underutilized in the United States. The traditional model of short, hands-on, cognitive courses to enhance the adoption of new techniques fails to lead to significant levels of practice implementation for most surgeons. We hypothesized that a comprehensive program would facilitate the adoption of laparoscopic inguinal hernia repair (TEP) for practicing surgeons.MethodsA team of experts in simulation, coaching, and hernia care created a comprehensive training program to facilitate the adoption of TEP. Three surgeons who routinely performed open inguinal hernia repair with greater than 50 cases annually were recruited to participate in the program. Coaches were selected based on their procedural expertise and underwent formal training in surgical coaching. Participants were required to evaluate all aspects of the educational program and were surveyed out to one year following completion of the program to assess for sustained adoption of TEP.ResultsAll three participants successfully completed the first three steps of the seven-step program. Two participants completed the full course, while the third dropped out of the program due to time constraints and low case volume. Participant surgeons rated Orientation (4.7/5), GlovesOn training (5/5), and Preceptored Cases (5/5) as highly important training activities that contributed to advancing their knowledge and technical performance of the TEP procedure. At one year, both participants were performing TEPs for “most of their cases” and were confident in their ability to perform the procedure. The total cost of the program including all travel, personal coaching, and simulation was

Rescuing the Clinical Breast Examination: Advances in Classifying Technique and Assessing Physician Competency.

8638.60 per participant.DiscussionOur comprehensive educational program led to full and sustained adoption of TEP for those who completed the course. Time constraints, travel costs, and case volume are major considerations for successful completion; however, the program is feasible, acceptable, and affordable.