Andrew N. Healey

Developing observational measures of performance in surgical teams

Team performance is increasingly recognised as an essential foundation of good surgical care and a determinant of good surgical outcome. To understand team performance and to develop team training, reliable and valid measures of team performance are necessary. Currently there is no firm consensus on how to measure teamwork, partly because of a lack of empirical data to validate measures. The input−process−output model provides a framework for surgical team studies. Objective observational measures are needed in surgery as a basis for interdisciplinary team assessment and training. The “observational teamwork assessment for surgery” (OTAS) tool assesses two facets of the surgical process. Observer 1 monitors specific tasks carried out by team members, under the categories patient, environment, equipment, provisions, and communications. Observer 2 uses a behavioural observation scale to rate behaviour for the three surgical phases (pre-operative, operative, and post-operative) with components of teamwork: cooperation, leadership, coordination, awareness, and communication. Illustrative data from an initial series of 50 cases is presented here. The OTAS tool enables two independent observers, a surgeon and psychologist, to record detailed information both on what the theatre team does and how they do it, and has the potential to identify constraints on performance that might relate to surgical outcome.

Observational Teamwork Assessment for Surgery (OTAS): Refinement and Application in Urological Surgery

BackgroundTeamwork in surgical teams is at the forefront of good practice guidelines and empirical research as an important aspect of safe surgery. We have developed a comprehensive assessment for teamwork in surgery—the Observational Teamwork Assessment for Surgery (OTAS)—and we have tested it for general surgical procedures. The aim of the research reported here was to extend the assessment to urology procedures.MethodsAfter refining the original assessment, we used it to observe 50 urology procedures. The OTAS comprises a procedural task checklist that assesses patient, equipment/provisions, and communication tasks as well as ratings on five team behavior constructs (communication, cooperation, coordination, leadership, and monitoring). Teamwork was assessed separately in the surgical, anesthesia, and nursing subteams in the operating theater. We also assessed the reliability of the behavioral scoring.ResultsRegarding task completion, a number of communication and equipment/provisions tasks were not routinely performed during the operations we observed. Regarding teamwork-related behaviors, adequate reliability was obtained in the scoring of behaviors. Anesthetists and nurses obtained their lowest scores on communication. Surgeons’ scores revealed a more complex pattern. In addition to low scores on communication, surgeons’ teamwork behaviors appeared to deteriorate as the procedures were finishing.ConclusionsOur findings suggest that OTAS is applicable to various branches of surgery. Separate assessment of the subteams in the operating theater provides useful information that can be used to build targeted teamwork training aiming to improve surgical patients’ safety and outcomes.

Observational teamwork assessment for surgery: construct validation with expert versus novice raters.

Objective:To test the construct validity of the Observational Teamwork Assessment for Surgery (OTAS) tool. Summary Background Data:Poor teamwork in surgical teams has been implicated in adverse events to patients. The OTAS is a tool that assesses teamwork in real time for the entire surgical team. Existing empirical research on OTAS has yet to explore how expert versus novice tool users use the tool to assess teamwork in the operating room. Methods:Data were collected in 12 elective procedures by an expert/expert (N = 6) and an expert/novice (N = 6) pair of raters. Five teamwork behaviors (communication, coordination, leadership, monitoring, and cooperation) were scored via observation pre, intra, and postoperatively by blind raters. Results:Significant and sizeable correlations were obtained in 12 of 15 behaviors in the expert/expert pair, but only in 3 of 15 behaviors in the expert/novice pair. Significant differences in mean scores were obtained in 3 of 15 behaviors in the expert/expert pair, but in 11 of 15 behaviors in the expert/novice pair. Total OTAS scores exhibited strong correlations and no significant differences in ratings in the expert/expert pair. In the expert/novice pair no correlations were obtained and there were significant differences in mean scores. The overall size of inconsistency in the scoring was 2% for expert/expert versus 15% for expert/novice. Conclusions:OTAS exhibits adequate construct validity as assessed by consistency in the scoring by expert versus novices—ie, expert raters produce significantly more consistent scoring than novice raters. Further validation should assess the learning curve for novices in OTAS. Relationships between OTAS, measures of technical skill, and behavioral responses to surgical crises should also be quantified.

Observational Assessment of Surgical Teamwork: A Feasibility Study

BackgroundTeamwork is fundamental to effective surgery, yet there are currently no measures of teamwork to guide training, evaluate team interventions or assess the impact of teamwork on outcomes. We report the first steps in the development of an observational assessment of teamwork and preliminary findings.MethodWe observed 50 operations in general surgery from a single operating theater using a measure of teamwork specifically developed for use in the operating theater. The OTAS (Observational Teamwork Assessment for Surgery) comprises a procedural task checklist centered on the patient, equipment and communications tasks and ratings on team behavior constructs, namely: communication, co-operation, co-ordination, shared-leadership and monitoring.ResultsRatings of overall team performance were reasonably high, though variable, but there was evidence that clinically significant steps were being missed which at the very least eroded safety margins. There was, for instance, a frequent failure to check both surgical and anesthetic equipment and a failure to confirm the procedure verbally, patient notes were missing in about one-eighth of the cases and delays or changes occurred in over two-thirds of the cases.ConclusionsThis study takes an initial step towards developing measures of team performance in surgery that are defined in relation to tasks and behaviors of the team. The observational method of assessment is feasible and can provide a wealth of potentially valuable research data. However, for these measures to be used for formal assessment, more research is needed to make them robust and standardized.

Quantifying distraction and interruption in urological surgery

Background: To enhance safety in surgery, it is necessary to develop a variety of tools for measuring and evaluating the system of work. One important consideration for safety in any high-risk work is the frequency and effect of distraction and interruption. Aim: To quantify distraction and interruption to the sterile surgical team in urology. Methods: Observation of the behaviour of the surgical team and their task activity determined distraction and interruption recorded. Using an ordinal scale, an observer rated each salient distraction or interruption observed in relation to the team’s involvement. Results: The frequency of events and their attached ratings were high, deriving from varying degrees of equipment, procedure and environment problems, telephones, bleepers and conversations. Discussion: With further refinement and testing, this method may be useful for distinguishing ordinal levels of work interference in surgery and helpful in raising awareness of its origin for postoperative surgical team debriefing.

Defining the technical skills of teamwork in surgery

Developments in surgical technology and procedure have accelerated and altered the work carried out in the operating theatre/room, but team modelling and training have not co-evolved. Evidence suggests that team structure and role allocation are sometimes unclear and contentious, and coordination and communication are not fully effective. To improve teamwork, clinicians need models that specify team resources, structure, process and tasks. They also need measures to assess performance and methods to train teamwork strategically. An effective training strategy might be to incorporate teamwork with other technical skills training in simulation. However, the measures employed for enhancing teamwork in training and practice will need to vary in their object of analysis, level of technical specificity, and system scope.

The complexity of measuring interprofessional teamwork in the operating theatre

Surgery depends on interprofessional teamwork, which is becoming increasingly specialized. If surgery is to become a highly reliable system, it must adapt and professionals must learn from, and share, tested models of interprofessional teamwork. Trainers also need valid measures of teamwork to assess individual and team performance. However, measurement and assessment of interprofessional teamwork is lacking and interprofessional team training is scarce in the surgical domain. This paper addresses the complexity of measuring interprofessional teamwork in the operating theatre. It focuses mainly on the design and properties of observational assessment tools. The report and analysis serves to inform the researcher or clinician of the issues to consider when designing or choosing from alternative measures of team performance for training or assessment.

Information needs in operating room teams: what is right, what is wrong, and what is needed?

BackgroundSafe surgical care requires effective information transfer between members of the operating room (OR) team. The present study aims to assess directly, systematically, and comprehensively, information needs of all OR team-members.MethodsThirty-three OR team-members (16 surgeons/anesthesiologists, 17 nurses) took part in a mixed-method interview. Participants indicated what information they need, their problems accessing it, and potential interventions to improve information transfer. They also rated the importance of different sources of information and the quality (accuracy, availability, timeliness, completeness, and clarity) of the information that they typically receive. Theme extraction and statistical analyses (descriptive and inferential) were used to analyze the data.ResultsThe patient emerged as the top source of information. Surgeons and anesthesiologists relied more on information from fellow clinicians, as well as information originating from diagnostic and imaging labs. They were also more critical about the quality of the information than nursing personnel. Anesthesiologists emerged as the most reliable source of information, whereas information coming from surgeons was deemed lacking in quality (even by surgeons themselves). Finally, the more time participants had spent working in ORs, the more negative views they had about the information that they receive—an unexpected finding. Communication skills training, standardized communication protocols, and information technology (IT) systems to function as a central information repository were the top three proposed interventions.ConclusionsThis study comprehensively maps information sources, problems, and solutions expressed by OR end-users. Recent developments in skills training modules and patient safety interventions for the OR (Surgical Safety Checklist) are discussed as potential interventions that will ameliorate communication in ORs, with a view to enhance patient safety and surgical care.

Distraction in the urology operating theatre.

Studies investigating adverse events in healthcare have identified a high rate of preventable surgical complications [1]. The traditional approach to identifying the factors that determine surgical outcome has focused on patient pathological risk factors, and the skill of the surgeon and surgical team [2]. However, this fails to account for the many factors inherent in the operating theatre system that might affect human performance and lead to complications [1–3]. These factors include interface design, teamwork, decisionmaking and the operative environment. The impact of distractions in the operating environment on patient outcomes and their effect on the surgical team should also be considered. The research needed to improve the design of this surgical system is termed ‘human factors’ or ergonomic research, and involves adapting the operating theatre to increase efficiency and usability for all theatre-based professionals.

Engineering the system of communication for safer surgery

Communication failures are a leading cause of error in surgery. Researchers and practitioners have therefore developed different interventions to improve communications, such as team briefing and pre-operative patient checklists. These different methods have clear merit. However, they have only dealt with portions of a complex system. Consequently, disparate interventions of varying kinds may not integrate and build an effective system of communication. We argue that a new view of communication is needed to improve safety in surgery; the view that communication is more fundamentally as a property of the whole system of work rather than confined to interpersonal exchanges. Rather than simply add an intervention to the system, interventions should integrate into the system. To achieve this, we propose a practical strategy to re-engineer the system of communication for surgery. This demands an analysis of the immediate informational needs within the system of interest, and an account of the wider system and those ergonomic and human factors shaping the performance of communicators. We illustrate the application of the method and refer to potential improvements in safety.