Shabnam Undre

Developing observational measures of performance in surgical teams

Team performance is increasingly recognised as an essential foundation of good surgical care and a determinant of good surgical outcome. To understand team performance and to develop team training, reliable and valid measures of team performance are necessary. Currently there is no firm consensus on how to measure teamwork, partly because of a lack of empirical data to validate measures. The input−process−output model provides a framework for surgical team studies. Objective observational measures are needed in surgery as a basis for interdisciplinary team assessment and training. The “observational teamwork assessment for surgery” (OTAS) tool assesses two facets of the surgical process. Observer 1 monitors specific tasks carried out by team members, under the categories patient, environment, equipment, provisions, and communications. Observer 2 uses a behavioural observation scale to rate behaviour for the three surgical phases (pre-operative, operative, and post-operative) with components of teamwork: cooperation, leadership, coordination, awareness, and communication. Illustrative data from an initial series of 50 cases is presented here. The OTAS tool enables two independent observers, a surgeon and psychologist, to record detailed information both on what the theatre team does and how they do it, and has the potential to identify constraints on performance that might relate to surgical outcome.

Reliability of a revised NOTECHS scale for use in surgical teams

BACKGROUND Recent developments in the surgical literature highlight the need for assessment of nontechnical skills in surgery. We report a revision of the NOn-TECHnical Skills (NOTECHS) scale of the aviation industry for use in surgery and detailed analysis on its reliability. METHODS The original NOTECHS scale assesses (1) Cooperation, (2) Leadership and Managerial Skills, (3) Situation Awareness and Vigilance, and (4) Decision Making. We added a Communication and Interaction dimension and adapted all subscales for use in surgical context. Reliability was assessed in simulation-based training for trainee clinicians. RESULTS Satisfactory reliability (Cronbachs alpha) was obtained (1) across professional groups and trainers and trainees, (2) in separate analyses for trainers and trainees, (3) in successive administrations of the scale, and (4) in surgical, anaesthetic and nursing groups analyzed separately. In the operating department practitioners group, Situation Awareness and Vigilance and Cooperation and Team Skills exhibited lower reliability. CONCLUSIONS Assessment of surgical nontechnical skills is becoming a training priority. The present evidence suggests that the revised NOTECHS scale exhibits good reliability. Further empirical research should assess the validity of the scale.

The simulated operating theatre: comprehensive training for surgical teams

Surgical excellence is traditionally defined in terms of technical performance, with little regard for the importance of interpersonal communication and leadership skills. Studies in the aviation industry have stressed the role of human factors in causing error and, in an attempt to reduce the occurrence of adverse events, led to the organisation of simulation based training scenarios. Similar strategies have recently been employed for the surgical team with the development of a simulated operating theatre project. This enables technical and non-technical performance of the surgeon and circulating staff to be assessed by experts situated in an adjacent control room, and provides an opportunity for constructive feedback. The scenarios have good face validity and junior surgeons can benefit from the process of learning new technical skills in a realistic environment. The effect of external influences such as distractions, new technology, or a crisis scenario can also be defined, with the ultimate aim of reducing the number of adverse events arising in the real operating room.

Observational Teamwork Assessment for Surgery (OTAS): Refinement and Application in Urological Surgery

BackgroundTeamwork in surgical teams is at the forefront of good practice guidelines and empirical research as an important aspect of safe surgery. We have developed a comprehensive assessment for teamwork in surgery—the Observational Teamwork Assessment for Surgery (OTAS)—and we have tested it for general surgical procedures. The aim of the research reported here was to extend the assessment to urology procedures.MethodsAfter refining the original assessment, we used it to observe 50 urology procedures. The OTAS comprises a procedural task checklist that assesses patient, equipment/provisions, and communication tasks as well as ratings on five team behavior constructs (communication, cooperation, coordination, leadership, and monitoring). Teamwork was assessed separately in the surgical, anesthesia, and nursing subteams in the operating theater. We also assessed the reliability of the behavioral scoring.ResultsRegarding task completion, a number of communication and equipment/provisions tasks were not routinely performed during the operations we observed. Regarding teamwork-related behaviors, adequate reliability was obtained in the scoring of behaviors. Anesthetists and nurses obtained their lowest scores on communication. Surgeons’ scores revealed a more complex pattern. In addition to low scores on communication, surgeons’ teamwork behaviors appeared to deteriorate as the procedures were finishing.ConclusionsOur findings suggest that OTAS is applicable to various branches of surgery. Separate assessment of the subteams in the operating theater provides useful information that can be used to build targeted teamwork training aiming to improve surgical patients’ safety and outcomes.

Observational teamwork assessment for surgery: construct validation with expert versus novice raters.

Objective:To test the construct validity of the Observational Teamwork Assessment for Surgery (OTAS) tool. Summary Background Data:Poor teamwork in surgical teams has been implicated in adverse events to patients. The OTAS is a tool that assesses teamwork in real time for the entire surgical team. Existing empirical research on OTAS has yet to explore how expert versus novice tool users use the tool to assess teamwork in the operating room. Methods:Data were collected in 12 elective procedures by an expert/expert (N = 6) and an expert/novice (N = 6) pair of raters. Five teamwork behaviors (communication, coordination, leadership, monitoring, and cooperation) were scored via observation pre, intra, and postoperatively by blind raters. Results:Significant and sizeable correlations were obtained in 12 of 15 behaviors in the expert/expert pair, but only in 3 of 15 behaviors in the expert/novice pair. Significant differences in mean scores were obtained in 3 of 15 behaviors in the expert/expert pair, but in 11 of 15 behaviors in the expert/novice pair. Total OTAS scores exhibited strong correlations and no significant differences in ratings in the expert/expert pair. In the expert/novice pair no correlations were obtained and there were significant differences in mean scores. The overall size of inconsistency in the scoring was 2% for expert/expert versus 15% for expert/novice. Conclusions:OTAS exhibits adequate construct validity as assessed by consistency in the scoring by expert versus novices—ie, expert raters produce significantly more consistent scoring than novice raters. Further validation should assess the learning curve for novices in OTAS. Relationships between OTAS, measures of technical skill, and behavioral responses to surgical crises should also be quantified.

Surgical crisis management skills training and assessment : A stimulation-based approach to enhancing operating room performance

Background:Intraoperative surgical crisis management is learned in an unstructured manner. In aviation, simulation training allows aircrews to coordinate and standardize recovery strategies. Our aim was to develop a surgical crisis simulation and evaluate its feasibility, realism, and validity of the measures used to assess performance. Methods:Surgical trainees were exposed to a bleeding crisis in a simulated operating theater. Assessment of performance consisted of a trainee’s technical ability to control the bleeding and of their team/human factors skills. This assessment was performed in a blinded manner by 2 surgeons and one human factors expert. Other measures consisted of time measures such as time to diagnose the bleeding (TD), inform team members (TT), achieve control (TC), and close the laceration (TL). Blood loss was used as a surrogate outcome measures. Results:There were considerable variations within both senior (n = 10) and junior (n = 10) trainees for technical and team skills. However, while the senior trainees scored higher than the juniors for technical skills (P = 0.001), there were no differences in human factors skills. There were also significant differences between the 2 groups for TD (P = 0.01), TC (P = 0.001), and TL (0.001). The blood loss was higher in the junior group. Conclusions:We have described the development of a novel simulated setting for the training of crisis management skills and the variability in performance both in between and within the 2 groups.

Observational Assessment of Surgical Teamwork: A Feasibility Study

BackgroundTeamwork is fundamental to effective surgery, yet there are currently no measures of teamwork to guide training, evaluate team interventions or assess the impact of teamwork on outcomes. We report the first steps in the development of an observational assessment of teamwork and preliminary findings.MethodWe observed 50 operations in general surgery from a single operating theater using a measure of teamwork specifically developed for use in the operating theater. The OTAS (Observational Teamwork Assessment for Surgery) comprises a procedural task checklist centered on the patient, equipment and communications tasks and ratings on team behavior constructs, namely: communication, co-operation, co-ordination, shared-leadership and monitoring.ResultsRatings of overall team performance were reasonably high, though variable, but there was evidence that clinically significant steps were being missed which at the very least eroded safety margins. There was, for instance, a frequent failure to check both surgical and anesthetic equipment and a failure to confirm the procedure verbally, patient notes were missing in about one-eighth of the cases and delays or changes occurred in over two-thirds of the cases.ConclusionsThis study takes an initial step towards developing measures of team performance in surgery that are defined in relation to tasks and behaviors of the team. The observational method of assessment is feasible and can provide a wealth of potentially valuable research data. However, for these measures to be used for formal assessment, more research is needed to make them robust and standardized.

Simultaneous combined use of flexible ureteroscopy and percutaneous nephrolithotomy to reduce the number of access tracts in the management of complex renal calculi

In a joint study between Duke University, North Carolina and London, a simultaneous combination of flexible ureteroscopy and percutaneous nephrolithotomy was used to reduce the number of access tracts required to manage complex renal calculi. This was found to be an effective method which reduced morbidity and blood loss, without increasing operating time, but without increasing stone‐free rates.

Annoyances, Disruptions, and Interruptions in Surgery: The Disruptions in Surgery Index (DiSI)

BackgroundRecent studies have investigated disruptions to surgical process via observation. We developed the Disruptions in Surgery Index (DiSI) to assess operating room professionals’ self-perceptions of disruptions that affect surgical processes.MaterialsThe DiSI assesses individual issues, operating room environment, communication, coordination/situational awareness, patient-related disruptions, team cohesion, and organizational issues. Sixteen surgeons, 26 nurses, and 20 anesthetists/operating departmental practitioners participated. Participants judged for themselves and for their colleagues how often each disruption occurs, its contribution to error, and obstruction of surgical goals.ResultsWe combined the team cohesion and organizational disruptions to improve reliability. All participants judged that individual issues, operating room environment, and communication issues affect others more often and more severely than one’s self. Surgeons reported significantly fewer disruptions than nurses or anesthetists.ConclusionAlthough operating room professionals acknowledged disruptions and their impact, they attributed disruptions related to individual performance and attitudes more to their colleagues than to themselves. The cross-professional discrepancy in perceived disruptions (surgeons perceiving fewer than the other two groups) suggests that attempts to improve the surgical environment should always start with thorough assessment of the views of all its users. DiSI is useful in that it differentiates between the frequency and the severity of disruptions. Further research should explore correlations of DiSI-assessed perceptions and other observable measures.

Defining the technical skills of teamwork in surgery

Developments in surgical technology and procedure have accelerated and altered the work carried out in the operating theatre/room, but team modelling and training have not co-evolved. Evidence suggests that team structure and role allocation are sometimes unclear and contentious, and coordination and communication are not fully effective. To improve teamwork, clinicians need models that specify team resources, structure, process and tasks. They also need measures to assess performance and methods to train teamwork strategically. An effective training strategy might be to incorporate teamwork with other technical skills training in simulation. However, the measures employed for enhancing teamwork in training and practice will need to vary in their object of analysis, level of technical specificity, and system scope.