Sarah E. Peyre

Crisis Checklists for the Operating Room: Development and Pilot Testing

BACKGROUND Because operating room crises are rare events, failure to adhere to critical management steps is common. We sought to develop and pilot a tool to improve adherence to lifesaving measures during operating room crises. STUDY DESIGN We identified 12 of the most frequently occurring operating room crises and corresponding evidence-based metrics of essential care for each (46 total process measures). We developed checklists for each crisis based on a previously defined method, which included literature review, multidisciplinary expert consultation, and simulation. After development, 2 operating room teams (11 participants) were each exposed to 8 simulations with random assignment to checklist use or working from memory alone. Each team managed 4 simulations with a checklist available and 4 without. One of the primary outcomes measured through video review was failure to adhere to essential processes of care. Participants were surveyed for perceptions of checklist use and realism of the scenarios. RESULTS Checklist use resulted in a 6-fold reduction in failure of adherence to critical steps in management for 8 scenarios with 2 pilot teams. These results held in multivariate analysis accounting for clustering within teams and adjusting for learning or fatigue effects (11 of 46 failures without the checklist vs 2 of 46 failures with the checklist; adjusted relative risk = 0.15, 95% CI, 0.04-0.60; p = 0.007). All participants rated the overall quality of the checklists and scenarios to be higher than average or excellent. CONCLUSIONS Checklist use can improve safety and management in operating room crises. These findings warrant broader evaluation, including in clinical settings.

Postgame analysis: Using video-based coaching for continuous professional development

BACKGROUND The surgical learning curve persists for years after training, yet existing continuing medical education activities targeting this are limited. We describe a pilot study of a scalable video-based intervention, providing individualized feedback on intraoperative performance. STUDY DESIGN Four complex operations performed by surgeons of varying experience--a chief resident accompanied by the operating senior surgeon, a surgeon with less than 10 years in practice, another with 20 to 30 years in practice, and a surgeon with more than 30 years of experience--were video recorded. Video playback formed the basis of 1-hour coaching sessions with a peer-judged surgical expert. These sessions were audio recorded, transcribed, and thematically coded. RESULTS The sessions focused on operative technique--both technical aspects and decision-making. With increasing seniority, more discussion was devoted to the optimization of teaching and facilitation of the residents technical performance. Coaching sessions with senior surgeons were peer-to-peer interactions, with each discussing his preferred approach. The coach alternated between directing the session (asking probing questions) and responding to specific questions brought by the surgeons, depending on learning style. At all experience levels, video review proved valuable in identifying episodes of failure to progress and troubleshooting alternative approaches. All agreed this tool is a powerful one. Inclusion of trainees seems most appropriate when coaching senior surgeons; it may restrict the dialogue of more junior attendings. CONCLUSIONS Video-based coaching is an educational modality that targets intraoperative judgment, technique, and teaching. Surgeons of all levels found it highly instructive. This may provide a practical, much needed approach for continuous professional development.

Pilot testing of a model for insurer-driven, large-scale multicenter simulation training for operating room teams.

Objective:To test the feasibility of implementing a standardized teamwork training program with full operating room teams in multiple institutions, driven by malpractice insurer support and incentives. Background:Failures in intraoperative teamwork are among the leading causes of preventable patient injury and death in surgical patients. Teamwork training, particularly using simulation, can be an effective intervention but is difficult to scale. Methods:A malpractice insurer convened a collaborative with 4 Harvard-affiliated simulation programs to develop a standardized operating room teamwork training curriculum, including principles of communication, assertiveness, and use of the World Health Organization Surgical Safety Checklist. Participant teams were compensated for lost operative time via malpractice premium discounts, continuing education credits, and compensation for lost wages. The course was delivered through a simulation program involving the management of intraoperative emergency scenarios. Participants were surveyed for their perceptions of the program and of its impact on clinical practice. Results:A total of 221 active operating room staff members participated in the program. Each team contained at least 1 attending surgeon, 1 attending anesthesiologist, and 1 operating room nurse (mean size per team: 7 ± 2 participants). No study dates were cancelled because of lack of attendance. The survey response rate was 99% (218/221). Overall, the vast majority of participants found the scenarios realistic [94% (95% confidence interval: 90.9%, 97.2%)], appropriately challenging [95.4% (92.6%, 98.2%)], relevant to their practice [96.3% (93.8%, 98.8%)], and found the training would help them provide safer patient care [92.6% (89.1%, 96.1%)]. Surgeons reported their greatest personal deficit as communication skills. Operating room nurses and anesthesiologists reported a greater need than surgeons to work on personal assertiveness. Conclusions:A standardized multicenter team training program involving full operative teams is feasible with high-fidelity simulation and modest compensation for lost time. The vast majority of the multidisciplinary participants believed the course to have had a meaningful impact on their approach to clinical practice.

Protecting patients from an unsafe system: the etiology and recovery of intraoperative deviations in care.

Objective:To understand the etiology and resolution of unanticipated events in the operating room (OR). Background:The majority of surgical adverse events occur intraoperatively. The OR represents a complex, high-risk system. The influence of different human, team, and organizational/environmental factors on safety and performance is unknown. Methods:We video-recorded and transcribed 10 high-acuity operations, representing 43.7 hours of patient care. Deviations, defined as delays and/or episodes of decreased patient safety, were identified by majority consensus of a multidisciplinary team. Factors that contributed to each event and/or mitigated its impact were determined and attributed to the patient, providers, or environment/organization. Results:Thirty-three deviations (10 delays, 17 safety compromises, 6 both) occurred—with a mean of 1 every 79.4 minutes. These deviations were multifactorial (mean 3.1 factors). Problems with communication and organizational structure appeared repeatedly at the root of both types of deviations. Delays tended to be resolved with vigilance, communication, coordination, and cooperation, while mediation of safety compromises was most frequently accomplished with vigilance, leadership, communication, and/or coordination. The organization/environment was not found to play a direct role in compensation. Conclusions:Unanticipated events are common in the OR. Deviations result from poor organizational/environmental design and suboptimal team dynamics, with caregivers compensating to avoid patient harm. Although recognized in other high-risk domains, such human resilience has not yet been described in surgery and has major implications for the design of safety interventions.

Deconstructing intraoperative communication failures.

BACKGROUND Communication failure is a common contributor to adverse events. We sought to characterize communication failures during complex operations. METHODS We video recorded and transcribed six complex operations, representing 22 h of patient care. For each communication event, we determined the participants and the content discussed. Failures were classified into four types: audience (key individuals missing), purpose (issue nonresolution), content (insufficient/inaccurate information), and/or occasion (futile timing). We added a systems category to reflect communication occurring at the organizational level. The impact of each identified failure was described. RESULTS We observed communication failures in every case (mean 29, median 28, range 13-48), at a rate of one every 8 min. Cross-disciplinary exchanges resulted in failure nearly twice as often as intradisciplinary ones. Discussions about or mandated by hospital policy (20%), personnel (18%), or other patient care (17%) were most error prone. Audience and purpose each accounted for >40% of failures. A substantial proportion (26%) reflected flawed systems for communication, particularly those for disseminating policy (29% of system failures), coordinating personnel (27%), and conveying the procedure planned (27%) or the equipment needed (24%). In 81% of failures, inefficiency (extraneous discussion and/or work) resulted. Resource waste (19%) and work-arounds (13%) also were frequently seen. CONCLUSIONS During complex operations, communication failures occur frequently and lead to inefficiency. Prevention may be achieved by improving synchronous, cross-disciplinary communication. The rate of failure during discussions about/mandated by policy highlights the need for carefully designed standardized interventions. System-level support for asynchronous perioperative communication may streamline operating room coordination and preparation efforts.

Preoperative Learning Goals Set by Surgical Residents and Faculty

BACKGROUND The operating room (OR) remains the main teaching venue for surgical trainees. The OR is considered a pure-discovery learning environment; the downsides of this can be putatively overcome when faculty and trainee arrive at a shared understanding of learning. This study aimed to better understand preoperative learning goals to identify areas of commonalities and potential barrier to intraoperative teaching. METHODS Brief, structured preoperative interviews were conducted outside the OR with the resident and faculty member who were scheduled to operate together. Answers were analyzed and grouped using grounded theory. RESULTS Twenty-seven resident-faculty pairs were interviewed. Nine residents (33.3%) were junior (PGY 1 and 2) and 18 (66.7%) were senior (PGY 3 through 5). Learning goal categories that emerged from the response analysis were anatomy, basic and advanced surgical skills, general and specific procedural tasks, technical autonomy, and pre-, intra-, and postoperative considerations. Residents articulated fewer learning goals than faculty (1.5 versus 2.4; P = 0.024). The most frequently identified learning goal by both groups was one classifiable under general procedural tasks; the greatest divergence was seen regarding perioperative considerations, which were identified frequently by faculty members but rarely by residents. CONCLUSIONS Faculty articulate significantly more learning goals for the residents they will operate with than residents articulate for themselves. Our data suggest that residents and faculty align on some learning goals for the OR but residents tend to be more limited, focusing predominantly on technical aspects of the operation. Faculty members tend to hold a broader view of the learning potential of the OR. These discrepancies may present barriers to effective intraoperative teaching.

Impact of online education on intern behaviour around joint commission national patient safety goals: a randomised trial

Purpose To compare the effectiveness of two types of online learning methodologies for improving the patient-safety behaviours mandated in the Joint Commission National Patient Safety Goals (NPSG). Methods This randomised controlled trial was conducted in 2010 at Massachusetts General Hospital and Brigham and Womens Hospital (BWH) in Boston USA. Incoming interns were randomised to either receive an online Spaced Education (SE) programme consisting of cases and questions that reinforce over time, or a programme consisting of an online slide show followed by a quiz (SQ). The outcome measures included NPSG-knowledge improvement, NPSG-compliant behaviours in a simulation scenario, self-reported confidence in safety and quality, programme acceptability and programme relevance. Results Both online learning programmes improved knowledge retention. On four out of seven survey items measuring satisfaction and self-reported confidence, the proportion of SE interns responding positively was significantly higher (p<0.05) than the fraction of SQ interns. SE interns demonstrated a mean 4.79 (36.6%) NPSG-compliant behaviours (out of 13 total), while SQ interns completed a mean 4.17 (32.0%) (p=0.09). Among those in surgical fields, SE interns demonstrated a mean 5.67 (43.6%) NPSG-compliant behaviours, while SQ interns completed a mean 2.33 (17.9%) (p=0.015). Focus group data indicates that SE was more contextually relevant than SQ, and significantly more engaging. Conclusion While both online methodologies improved knowledge surrounding the NPSG, SE was more contextually relevant to trainees and was engaging. SE impacted more significantly on both self-reported confidence and the behaviour of surgical residents in a simulated scenario.

Reliability of a procedural checklist as a high-stakes measurement of advanced technical skill

BACKGROUND The purpose of this study was to determine the reliability of a previously validated laparoscopic Nissen fundoplication procedural checklist as a measurement of advanced technical skill. METHODS Five surgeons, using a 65-step procedural checklist, independently evaluated 2 video recordings of expert surgeon operative performances of a laparoscopic Nissen fundoplication. Results were analyzed for percent agreement and Fleiss kappa correlation for each operation independently and combined as a whole. RESULTS Sixty-four of the 65 steps had 80% or higher percent agreement for both operative case A and B independently and when considered overall. The Fleiss kappa coefficients for operative case A (kappa = .95) and operative case B (kappa = .96) and overall (operative case A + B) (kappa = .95). CONCLUSION The percentage agreement and kappa coefficients for all 3 analyses indicate a high degree of reliability (>.80) that would support the use of this instrument for high-stakes assessment of a laparoscopic Nissen fundoplication.

Mini-clinical evaluation exercise as a student assessment tool in a surgery clerkship: Lessons learned from a 5-year experience

BACKGROUND The mini-clinical evaluation exercise (mini-CEX) used for clinical skill assessment in internal medicine provides in-depth assessment of single clinical encounters. The goals of this study were to determine the feasibility and value of implementation of the mini-CEX in a surgery clerkship. METHODS Retrospective review of mini-CEX evaluations collected for surgery clerkship students at our institution between 2005 and 2010. Returned assessment forms were tallied. Qualitative feedback comments were analyzed using grounded theory. Principal components analysis identified thematic clusters. Thematic comment counts were compared to those provided via global assessments. RESULTS For 124 of 137 (90.5%) students, mini-CEX score sheets were available. Thematic clusters identified comments on 8 distinct clinical skill domains. On the mini-CEX, each student received an average of 6.5 ± 2.2 qualitative feedback comments covering 4.5 ± 1.2 separate skills. Of these, 42.7% were critical. Comments provided in global evaluations were fewer (2.9 ± 0.6; P < .001), constrained in scope (0.8 ± 0.2 skills; P < .001), and rarely critical (9.1%). CONCLUSION A mini-CEX can be incorporated into a surgery clerkship. The number and breadth of feedback comments make the mini-CEX a rich assessment tool. Critical and supportive feedback comments, both highly valuable, are provided nearly equally frequently when the mini-CEX is used as an assessment tool.

Mobilizing Faculty for Simulation

Faculty involvement in simulation training is essential for curriculum development, utilization of their clinical expertise in teaching, and ultimately for validating the importance of the training program. Several barriers to faculty involvement exist, including competing demands on time, the challenges in developing curriculum, and teaching using simulation. Through our experiences in implementing a widely expansive program, we have identified several areas to encourage and engage faculty. Further discussion as a medical education community is needed to support the interaction and involvement of our faculty to support and promote ongoing simulation education.