Elaine R. Cohen

The use of error analysis to assess resident performance

BACKGROUND The aim of this study was to assess validity of a human factors error assessment method for evaluating resident performance during a simulated operative procedure. METHODS Seven postgraduate year 4-5 residents had 30 minutes to complete a simulated laparoscopic ventral hernia (LVH) repair on day 1 of a national, advanced laparoscopic course. Faculty provided immediate feedback on operative errors and residents participated in a final product analysis of their repairs. Residents then received didactic and hands-on training regarding several advanced laparoscopic procedures during a lecture session and animate lab. On day 2, residents performed a nonequivalent LVH repair using a simulator. Three investigators reviewed and coded videos of the repairs using previously developed human error classification systems. RESULTS Residents committed 121 total errors on day 1 compared with 146 on day 2. One of 7 residents successfully completed the LVH repair on day 1 compared with all 7 residents on day 2 (P = .001). The majority of errors (85%) committed on day 2 were technical and occurred during the last 2 steps of the procedure. There were significant differences in error type (P ≤ .001) and level (P = .019) from day 1 to day 2. The proportion of omission errors decreased from day 1 (33%) to day 2 (14%). In addition, there were more technical and commission errors on day 2. CONCLUSION The error assessment tool was successful in categorizing performance errors, supporting known-groups validity evidence. Evaluating resident performance through error classification has great potential in facilitating our understanding of operative readiness.

Evaluation of hands-on clinical exam performance using marker-less video tracking

This study investigates the potential of using marker-less video tracking for evaluating hands-on clinical skills. Experienced family practitioners attending a national conference were recruited and asked to conduct a breast examination on a simulator that presents different clinical pathologies. Videos were taken of the clinician’s hands during the exam. Video processing software for tracking and quantifying hand motion kinematics was used. Videos were divided into two segments: a general search segment and a mass exploration segment. The general exploration segments exhibited motion patterns which included 72% faster movement and 73% higher acceleration across clinical pathologies. The most complex pathology exhibited 14% greater displacement for pressing/rubbing than for general exploration. Marker-less video kinematic tracking shows promise in discriminating between different examination procedures, clinicians, and pathologies.