Douglas A. Wiegmann

Human Error and Commercial Aviation Accidents: An Analysis Using the Human Factors Analysis and Classification System

Objective: The aim of this study was to extend previous examinations of aviation accidents to include specific aircrew, environmental, supervisory, and organizational factors associated with two types of commercial aviation (air carrier and commuter/ on-demand) accidents using the Human Factors Analysis and Classification System (HFACS). Background: HFACS is a theoretically based tool for investigating and analyzing human error associated with accidents and incidents. Previous research has shown that HFACS can be reliably used to identify human factors trends associated with military and general aviation accidents. Method: Using data obtained from both the National Transportation Safety Board and the Federal Aviation Administration, 6 pilot-raters classified aircrew, supervisory, organizational, and environmental causal factors associated with 1020 commercial aviation accidents that occurred over a 13-year period. Results: The majority of accident causal factors were attributed to aircrew and the environment, with decidedly fewer associated with supervisory and organizational causes. Comparisons were made between HFACS causal categories and traditional situational variables such as visual conditions, injury severity, and regional differences. Conclusion: These data will provide support for the continuation, modification, and/or development of interventions aimed at commercial aviation safety. Application: HFACS provides a tool for assessing human factors associated with accidents and incidents.

The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills.

Purpose: To develop and evaluate a participant rating scale for assessing high performance teamwork skills in simulation medicine settings. Methods: In all, 107 participants in crisis resource management (CRM) training in a multidisciplinary medical simulation center generated 273 ratings of key CRM skills after participating in two or three simulation exercises. These data were analyzed using Rasch and traditional psychometric approaches to develop the 16-item Mayo High Performance Teamwork Scale (MHPTS). Sensitivity to change as a result CRM training was also evaluated. Results: The MHPTS showed satisfactory internal consistency and construct validity by Rasch (person reliability = 0.77; person separation = 1.85; item reliability = 0.96; item separation = 5.04) and traditional psychometric (Cronbach’s alpha = 0.85) indicators. The scale demonstrated sensitivity to change as a result of CRM training (pretraining mean = 21.44 versus first posttraining rating mean = 24.37; paired t = −4.15, P < 0.0001; first posttraining mean = 24.63 versus second posttraining mean = 26.83; paired t = −4.31 P < 0.0001). Conclusions: The MHPTS provides a brief, reliable, practical measure of CRM skills that can be used by participants in CRM training to reflect on and evaluate their performance as a team. Further evaluation of validity and appropriateness in other simulation and medical settings is desirable.

Safety Culture: An Integrative Review

Recent years have witnessed a growing concern over the issue of safety culture within aviation and other complex, high-risk industries. The purpose of this review is to summarize and integrate the numerous reports and studies that have been conducted to define and assess safety culture as well as the highly related concept of safety climate. Results of the review indicate that few formally documented efforts have been made to assess safety culture within the aviation industry. Furthermore, there exists considerable disagreement among safety professionals, both within and across industries, as to how safety culture should be defined and whether or not safety culture is inherently different from the concept of safety climate. We conducted a synthesis of these different perspectives. We provide a discussion of key organizational indicators of safety culture and the various methods commonly used to assess these factors. We also present issues that need to be considered when implementing a safety culture assessment program. The hope is that this review will enable researchers and safety professionals to better understand and assess safety culture and that it will facilitate the sharing of information and strategies for improving safety culture across organizations and industries.

Automation Failures on Tasks Easily Performed by Operators Undermine Trust in Automated Aids

Objective: We tested the hypothesis that automation errors on tasks easily performed by humans undermine trust in automation. Background: Research has revealed that the reliability of imperfect automation is frequently misperceived. We examined the manner in which the easiness and type of imperfect automation errors affect trust and dependence. Method: Participants performed a target detection task utilizing an automated aid. In Study 1, the aid missed targets either on easy trials (easy miss group) or on difficult trials (difficult miss group). In Study 2, we manipulated both easiness and type of error (miss vs. false alarm). The aid erred on either difficult trials alone (difficult errors group) or on difficult and easy trials (easy miss group; easy false alarm group). Results: In both experiments, easy errors led to participants mistrusting and disagreeing more with the aid on difficult trials, as compared with those using aids that generated only difficult errors. This resulted in a downward shift in decision criterion for the former, leading to poorer overall performance. Misses and false alarms led to similar effects. Conclusion: Automation errors on tasks that appear “easy” to the operator severely degrade trust and reliance. Application: Potential applications include the implementation of system design solutions that circumvent the negative effects of easy automation errors.

Similarities and differences between human–human and human–automation trust: an integrative review

The trust placed in diagnostic aids by the human operator is a critical psychological factor that influences operator reliance on automation. Studies examining the nature of human interaction with automation have revealed that users have a propensity to apply norms of human–human inter-personal interaction to their interaction with ‘intelligent machines’. Nevertheless, there exist subtle differences in the manner in which humans perceive and react to automated aids compared to human team-mates. In the present paper, the concept of trust in human–automation dyads is compared and contrasted with that of human–human dyads. A theoretical framework that synthesizes and describes the process of trust development in humans vs automated aids is proposed and implications for the design of decision aids are provided. Potential implications of this research include the improved design of decision support systems by incorporating features into automated aids that elicit operator responses mirroring responses in human–human inter-personal interaction. Such interventions will likely facilitate better quantification and prediction of human responses to automation, while improving the quality of human interaction with non-human team-mates.

Human Error Perspectives in Aviation

As aircraft have become more reliable, humans have played a progressively more important causal role in aviation accidents, resulting in the proliferation of human error frameworks and accident investigation schemes. To date, however, few efforts have been made to systematically organize these different approaches based on underlying theoretical similarities, and formalized methods for evaluating the utility of these multiple approaches have not been clearly defined. Many safety professionals, therefore, have been at a loss when choosing which error analysis and prevention approach to use within their organizations. As a result, those tasked with instituting human-centered safety programs often rely primarily on personal experience and intuition to address their needs. The purpose of this article is to help remedy this situation by providing safety practitioners with an overview of the prominent human error perspectives in aviation, as well as a set of objective criteria for evaluating human error frameworks.

Development and Pilot Evaluation of a Preoperative Briefing Protocol for Cardiovascular Surgery

BACKGROUND Preprocedural briefings have been adopted in many high consequence environments, but have not been widely accepted in medicine. We sought to develop, implement, and evaluate a preoperative briefing for cardiovascular surgery. STUDY DESIGN The briefing was developed by using a combined questionnaire and semistructured focus group approach involving five subspecialties of surgical staff (n=55). The results were used to design and implement a preoperative briefing protocol. The briefing was evaluated by monitoring surgical flow disruptions, circulating nurse trips to the core, time spent in the core, and cost-waste reports before and after implementation of the briefing across 16 cardiac surgery cases. RESULTS Focus group data indicated consensus among surgical staff concerning briefing benefits, duration, location, content, and potential barriers. Disagreement arose concerning timing of the brief and the roles of key participants. After implementation of the briefing, there was a reduction in total surgical flow disruptions per case (5.4 preimplementation versus 2.8 postimplementation, p=0.004) and reductions in per case average of procedural knowledge disruptions (4.1 versus 2.17, p=0.004) and miscommunication events (2.5 versus 1.17, p=0.03). There was no significant reduction in disruptions because of equipment preparation or disruptions from patient-related issues. On average, briefed teams experienced fewer trips to the core (10 versus 4.7, p=0.004) and spent less time in the core (397.4 seconds versus 172.3 seconds, p=0.006), and there was a trend toward decreased waste (30% versus 17%, p=0.15). CONCLUSIONS These findings demonstrate the feasibility of creating a specialty-specific preoperative briefing to decrease surgical flow disruptions and improve patient safety in the operating room.

Surgical coaching for individual performance improvement.

T he technical skill of individual surgeons is an important determinant of surgical outcomes, at least for bariatric surgery. In a recent study in The New England Journal of Medicine, Birkmeyer and colleagues1 provide evidence that there is wide variation in technical skill among practicing surgeons and that this variation correlates with outcome. The authors conclude that new approaches to improving individual performance are needed and one-on-one coaching has potential to fill this role. The concept of coaching for performance improvement has been recently described in a variety of health care settings, but it is not well developed and experience is limited.2,3 Surgical coaching has the potential to address limitations in our current approach to continuing medical education, which does not incorporate the critical concepts of adult learning theory. Adults learn best as active participants in learning that builds on individual needs, is tailored to past experiences, and has direct applicability to their daily activities.4 Adult learners should participate in the identification of their own goals and have the opportunity to practice what is learned through self-reflection coupled with constructive feedback. The importance of self-reflection is emphasized by K. Anders Ericsson, who advocates for deliberate practice. Ericsson examines this concept for physicians, suggesting that most clinical practice does not include the critical aspects of deliberate practice, namely, the identification of areas for improvement by reflection on performance, followed by intentional adjustments in approach and evaluation of the resultant impact.5 This deficiency leads many practitioners to plateau in a state of proficiency. In contrast, professionals in other disciplines use coaches to facilitate deliberate practice and continued performance improvement, even among the most elite experts. To further develop the concept of surgical coaching, we examined features of coaching in other disciplines through literature review and observation and interviews of prominent coaches. In what follows, we outline main themes and provide a conceptual framework to synthesize the critical elements.

Identifying methods to improve heart surgery: an operative approach and strategy for implementation on an organizational level

BACKGROUND Previous research has found teamwork failures to be strongly associated with the occurrence of surgical error. There have been few efforts to prospectively collect data regarding teamwork failures and technical errors in order to create interventions that would maximize teamwork effectiveness thereby minimizing technical error. METHODS Thirty-one cardiac surgical cases were prospectively observed by a trained human factors observer. Events were characterized according to human factors theory and included teamwork failures and technical errors. Surgical team structure was also evaluated in an effort to identify if it had an impact on surgical team performance. RESULTS A strong correlation (r=0.67, p<0.001) was recognized between the occurrence of technical error (n=155) and teamwork failures (n=178). Teamwork failures consisted of surgeon-technical team failures (n=90, 51%), procedural information failures (n=36, 20%), surgeon-anesthesiologist failures (n=27, 15%), surgeon-perfusionist failures (n=18, 10%), and failures due to handoffs (n=7, 4%). Teams made up of members that were familiar with the operating surgeon had significantly fewer total event failures (8.6+/-1.6 vs 22+/-3.1, p<0.0001) and teamwork failures (5.6+/-1.8 vs 15.4+/-1.9, p<0.0001) in comparison to those teams where the majority of members were unfamiliar with the operating surgeon. CONCLUSIONS These results indicate that the process of cardiac surgery would benefit from interventions to improve teamwork and communication. Such interventions could include preoperative briefings, revised approach to structuring of operative teams to favor members that have gained familiarity with the operating surgeon, standardized communication practices, and postoperative debriefings.

Development and Evaluation of an Observational Tool for Assessing Surgical Flow Disruptions and Their Impact on Surgical Performance

BackgroundMany researchers have previously explored the correlation between surgical flow disruptions and adverse events in cardiac surgery; however, there is no reliable tool to prospectively categorize surgical flow disruptions and the conditions that predispose a surgical team to adverse events.MethodsTwo independent raters of different medical and human factors expertise observed 12 cardiovascular operations and iteratively designed a surgical flow disruption tool (SFDT) to characterize surgical flow disruptions and the latent factors that contribute to adverse events. Categories to characterize surgical flow disruptions were created based on human factors models of human error. After the design period, both raters observed ten surgical cases using the tool to assess validity and inter-rater reliability.ResultsRating agreement (weighted kappa) for each category across the ten surgeries was moderate to very high, resulting in strong inter-rater reliability for each category on the surgical flow disruption tool. Use of the SFDT was simple and clear for observers of diverse backgrounds, including human factors experts and medical personnel.ConclusionsThis research depicts the development and utility of a tool to analyze surgical flow disruptions in the cardiovascular operating room with satisfactory inter-rater reliability. This tool is an important first step in systematically categorizing and measuring surgical flow disruptions and their impact on patient safety in the operating room.