Emilie Roth

Protecting patients from an unsafe system: the etiology and recovery of intraoperative deviations in care.

Objective:To understand the etiology and resolution of unanticipated events in the operating room (OR). Background:The majority of surgical adverse events occur intraoperatively. The OR represents a complex, high-risk system. The influence of different human, team, and organizational/environmental factors on safety and performance is unknown. Methods:We video-recorded and transcribed 10 high-acuity operations, representing 43.7 hours of patient care. Deviations, defined as delays and/or episodes of decreased patient safety, were identified by majority consensus of a multidisciplinary team. Factors that contributed to each event and/or mitigated its impact were determined and attributed to the patient, providers, or environment/organization. Results:Thirty-three deviations (10 delays, 17 safety compromises, 6 both) occurred—with a mean of 1 every 79.4 minutes. These deviations were multifactorial (mean 3.1 factors). Problems with communication and organizational structure appeared repeatedly at the root of both types of deviations. Delays tended to be resolved with vigilance, communication, coordination, and cooperation, while mediation of safety compromises was most frequently accomplished with vigilance, leadership, communication, and/or coordination. The organization/environment was not found to play a direct role in compensation. Conclusions:Unanticipated events are common in the OR. Deviations result from poor organizational/environmental design and suboptimal team dynamics, with caregivers compensating to avoid patient harm. Although recognized in other high-risk domains, such human resilience has not yet been described in surgery and has major implications for the design of safety interventions.

Deconstructing intraoperative communication failures.

BACKGROUND Communication failure is a common contributor to adverse events. We sought to characterize communication failures during complex operations. METHODS We video recorded and transcribed six complex operations, representing 22 h of patient care. For each communication event, we determined the participants and the content discussed. Failures were classified into four types: audience (key individuals missing), purpose (issue nonresolution), content (insufficient/inaccurate information), and/or occasion (futile timing). We added a systems category to reflect communication occurring at the organizational level. The impact of each identified failure was described. RESULTS We observed communication failures in every case (mean 29, median 28, range 13-48), at a rate of one every 8 min. Cross-disciplinary exchanges resulted in failure nearly twice as often as intradisciplinary ones. Discussions about or mandated by hospital policy (20%), personnel (18%), or other patient care (17%) were most error prone. Audience and purpose each accounted for >40% of failures. A substantial proportion (26%) reflected flawed systems for communication, particularly those for disseminating policy (29% of system failures), coordinating personnel (27%), and conveying the procedure planned (27%) or the equipment needed (24%). In 81% of failures, inefficiency (extraneous discussion and/or work) resulted. Resource waste (19%) and work-arounds (13%) also were frequently seen. CONCLUSIONS During complex operations, communication failures occur frequently and lead to inefficiency. Prevention may be achieved by improving synchronous, cross-disciplinary communication. The rate of failure during discussions about/mandated by policy highlights the need for carefully designed standardized interventions. System-level support for asynchronous perioperative communication may streamline operating room coordination and preparation efforts.

A Cognitive Task Analysis for Cyber Situational Awareness

Cyber Network degradation and exploitation can covertly turn an organizations technological strength into an operational weakness. It has become increasingly imperative, therefore, for an organizations personnel to have an awareness of the state of the Cyber Network that they use to carry out their mission. Recent high-level government initiatives along with hacking and exploitation in the commercial realm highlight this need for general Cyber Situational Awareness (SA). While much of the attention in both the military and commercial cyber security communities is on abrupt and blunt attacks on the network, the most insidious cyber threat to organizations are subtle and persistent attacks leading to compromised databases, processing algorithms, and displays. We recently began an effort developing software tools to support the Cyber SA of users at varying levels of responsibility and expertise (i.e., not just the network administrators). This paper presents our approach and preliminary findings from a CTA we conducted with an operational Subject Matter Expert to uncover the situational awareness requirements of such a tool. Results from our analysis indicate a list of preliminary categories of these requirements, as well as specific questions that will drive the design and development of our SA tool.

Handoffs During Nursing Shift Changes in Acute Care

Handoffs during the nursing shift change were directly observed on two acute care wards each of a private and public hospital, for a total of 236 patient updates by 49 nurses during 14 shift changes. Data from the three wards which conducted audio-taped updates were transcribed. The transcriptions and field notes were analyzed for the existence and frequency of 21 strategies used in high reliability organizations. In addition, we iteratively categorized the interruptions, questions, and statements made during the updates. Finally, we iteratively categorized stances towards decisions communicated during the updates. Implications of the findings are discussed.

Cognitive Complexities Impacting Army Intelligence Analysis

The primary goal of this effort was to understand the problems faced by military intelligence analysis personnel as well as how, and to what degree, the identification of these problems could guide the development of computational support systems. To develop this understanding, we performed a literature review, knowledge elicitation interviews and a cognitive task analysis (CTA) in the domain of Army Intelligence Analysis at the Brigade Combat Team. This effort consisted of identifying: (1) the major functions or cognitive tasks entailed in Army Intelligence Analysis; and (2) the complexities in the domain that pose challenges to performance of these cognitive tasks. Identifying the cognitive tasks and the challenges faced in performing those tasks provided a basis for determining opportunities for more effective support of human information processing and decision-making. In this paper, we document selected results of this analysis effort.

Designing Collaborative Automated Planners for Agile Adaptation to Dynamic Change

A common characteristic of domains that require planning for allocation of scarce resources is the need to dynamically revise the plan as new requirements emerge and priorities change. We describe a prototype decision support system for planning and scheduling airlift that we developed for a military transport organization that enables agile plan adaptation as movement requirements, available airlift assets, and priorities change. The collaborative automated scheduler includes visualizations to foster improved situation awareness of available airlift assets versus total demand on those assets; mechanisms to enable users to communicate informal priorities and changes to those priorities; and mechanisms that enable users to explore alternative scheduling options in response to changes in movement requirements, priorities and available assets. A formal user evaluation study that included 12 participants representing three different organizational groups involved in transportation planning provided evidence that the prototype improves the ability to capture and communicate movement priorities, rapidly reallocate airlift assets to accommodate changes, and communicate/collaborate across organizational boundaries in managing airlift demand versus capacity.

Techniques for Effective Collaborative Automation for Air Mission Replanning

We describe the most recent work-centered design for a military airlift organization. Earlier design cycles produced a set of coordinated visualizations to support synchronized air mission replanning. In this phase of the program automated planning support was incorporated to help C2 staff solve complex constraint problems across multiple missions and airfields. We describe our efforts at designing a replanning tool to work in collaboration with the human operator on complex replanning problems. A prototype was developed and used in an empirical evaluation comparing target user performance on complex constraint problems using either the visualizations alone or the visualizations with embedded automated support (collaborative planner). The collaborative planner significantly improved the speed and quality of dynamic replanning solutions. Post-test questionnaire ratings indicated users trusted the support provided by the automated aid. Design principles for effective collaborative automation are discussed.

User Created Cognitive Artifacts: What can they Teach us about Design of Information Technology?

Cognitive artifacts are created and used to support task performance in many domains. These artifacts may be essential components designed into a process, they may have been created by users as work-arounds to system shortcomings, or they may be extensions to systems that add functionalities to meet evolving needs. Examination of cognitive artifacts may provide insights that complement other cognitive engineering methods such as task analysis. This panel will present findings from study of cognitive artifacts used in high risk domains such as healthcare. Panelists will address questions relating to research methodologies, theoretical frameworks, and design implications, such as: What are the roles of user-designed artifacts for care coordination and patient safety? What can designers learn from artifacts? How can we use artifact analysis to design better health information technologies? Can vendor-designed information tools and user-designed artifacts work in harmony to provide safe care? Panel members will address these questions based on their research studies and experiences within as well as outside the health care area.

Cognitive engineering and health informatics: Applications and intersections

Cognitive engineering is an applied field with roots in both cognitive science and engineering that has been used to support design of information displays, decision support, human-automation interaction, and training in numerous high risk domains ranging from nuclear power plant control to transportation and defense systems. Cognitive engineering provides a set of structured, analytic methods for data collection and analysis that intersect with and complement methods of Cognitive Informatics. These methods support discovery of aspects of the work that make performance challenging, as well as the knowledge, skills, and strategies that experts use to meet those challenges. Importantly, cognitive engineering methods provide novel representations that highlight the inherent complexities of the work domain and traceable links between the results of cognitive analyses and actionable design requirements. This article provides an overview of relevant cognitive engineering methods, and illustrates how they have been applied to the design of health information technology (HIT) systems. Additionally, although cognitive engineering methods have been applied in the design of user-centered informatics systems, methods drawn from informatics are not typically incorporated into a cognitive engineering analysis. This article presents a discussion regarding ways in which data-rich methods can inform cognitive engineering.

Complexities and Challenges in the Use of Bayesian Belief Networks: Informing the Design of Causal Influence Models

Bayesian belief networks (BNs) are well-suited to capturing vague and uncertain knowledge. However, the capture of this knowledge and associated reasoning from human domain experts often requires specialized knowledge engineers responsible for translating the experts communications into BN-based models. Across application domains, we have analyzed how these models are constructed, refined, and validated with domain experts. From this analysis, we have identified key user-centered complexities and challenges that we have used to drive the selection of simplifying assumptions. This led us to develop computational techniques and user interface methods that leverage these same assumptions with the goal of improving the efficiency and ease with which expert knowledge can be expressed, verified, validated, and encoded. In this paper, we present the results of our analysis of BN construction, validation, and use. We discuss how these results motivated the design of a simplified version of BNs called Causal Influence Models (CIMs). In addition, we detail how CIMs enable the design and construction of user interface mechanisms that address complexities identified in our analysis.