Chris Bearman

The Breakdown of Coordinated Decision Making in Distributed Systems

Objective: This article aims to explore the nature and resolution of breakdowns in coordinated decision making in distributed safety-critical systems. Background: In safety-critical domains, people with different roles and responsibilities often must work together to make coordinated decisions while geographically distributed. Although there is likely to be a large degree of overlap in the shared mental models of these people on the basis of procedures and experience, subtle differences may exist. Method: Study 1 involves using Aviation Safety Reporting System reports to explore the ways in which coordinated decision making breaks down between pilots and air traffic controllers and the way in which the breakdowns are resolved. Study 2 replicates and extends those findings with the use of transcripts from the Apollo 13 National Aeronautics and Space Administration space mission. Results: Across both studies, breakdowns were caused in part by different types of lower-level breakdowns (or disconnects), which are labeled as operational, informational, or evaluative. Evaluative disconnects were found to be significantly harder to resolve than other types of disconnects. Conclusion: Considering breakdowns according to the type of disconnect involved appears to capture useful information that should assist accident and incident investigators. The current trend in aviation of shifting responsibilities and providing increasingly more information to pilots may have a hidden cost of increasing evaluative disconnects. Application: The proposed taxonomy facilitates the investigation of breakdowns in coordinated decision making and draws attention to the importance of considering subtle differences between participants’ mental models when considering complex distributed systems.

The structure and function of spontaneous analogising in domain-based problem solving

Laboratory-based studies of problem solving suggest that transfer of solution principles from an analogue to a target arises only minimally without the presence of directive hints. Recently, however, real-world studies indicate that experts frequently and spontaneously use analogies in domain-based problem solving. There is also some evidence that in certain circumstances domain novices can draw analogies designed to illustrate arguments. It is less clear, however, whether domain novices can invoke analogies in the sophisticated manner of experts to enable them to progress problem solving. In the current study groups of novices and experts tackled large-scale management problems. Spontaneous analogising was observed in both conditions, with no marked differences between expertise levels in the frequency, structure, or function of analogising. On average four analogies were generated by groups per hour, with significantly more relational mappings between analogue and target being produced than superficial object-and-attribute mappings. Analogising served two different purposes: problem solving (dominated by relational mappings), and illustration (which for novices was dominated by object-and-attribute mappings). Overall, our novices showed a sophistication in domain-based analogical reasoning that is usually only observed with experts, in addition to a sensitivity to the pragmatics of analogy use.

Socializing the Human Factors Analysis and Classification System: Incorporating Social Psychological Phenomena Into a Human Factors Error Classification System

Objective: The presence of social psychological pressures on pilot decision making was assessed using qualitative analyses of critical incident interviews. Background: Social psychological phenomena have long been known to influence attitudes and behavior but have not been highlighted in accident investigation models. Method: Using a critical incident method, 28 pilots who flew in Alaska were interviewed. The participants were asked to describe a situation involving weather when they were pilot in command and found their skills challenged. They were asked to describe the incident in detail but were not explicitly asked to identify social pressures. Pressures were extracted from transcripts in a bottom-up manner and then clustered into themes. Results: Of the 28 pilots, 16 described social psychological pressures on their decision making, specifically, informational social influence, the foot-in-the-door persuasion technique, normalization of deviance, and impression management and self-consistency motives. Conclusion: We believe accident and incident investigations can benefit from explicit inclusion of common social psychological pressures. Application: We recommend specific ways of incorporating these pressures into the Human Factors Analysis and Classification System.

Developing a research framework for complex multi–team coordination in emergency management

This conceptual paper addresses previous calls for the development of new theoretical frameworks to better account for the multi–agency emergency management coordination required in complex events. It uses, as a departure point, a teamwork model that includes four phases: situation assessment; plan formulation, plan execution and team learning. The thesis put forward here is that we need to move the focus of analysis beyond the team to one of multi–layered multiple team and multiple organisation systems. To further develop this research framework indicators from multi–organisational literature are added to those found in the individual and teamwork literature to develop a more comprehensive account of multi–team multi–organisational coordination. The paper identifies key anchor points for future use in data collection.

Explaining away the negative effects of evaluation on analogical transfer: the perils of premature evaluation.

Four experiments explored effects on analogical transfer of evaluating solutions to base problems. In contrast to reports of positive effects of explanation, evaluation consistently reduced transfer rates and impaired mental representations of base material. This effect was not ameliorated by encoding for a later memory test, summarizing, or engaging in similar processes at encoding and recall. However, providing a prior explanation task removed the inhibitory effect of evaluation. It appears that evaluation leads to encoding of extraneous material that interferes with access to solution-critical analogous information. Prior explanation inoculates against negative effects on transfer by ensuring that new information introduced via evaluation is organized around existing representations of relevant information of the base problem. The results suggest that the source of difficulty in analogical transfer may reside not only in retrieval and mapping but also in the initial encoding of problems.

Breakdowns in coordinated decision making at and above the incident management team level: An analysis of three large scale Australian wildfires

Emergency situations are by their nature difficult to manage and success in such situations is often highly dependent on effective team coordination. Breakdowns in team coordination can lead to significant disruption to an operational response. Breakdowns in coordination were explored in three large-scale bushfires in Australia: the Kilmore East fire, the Wangary fire, and the Canberra Firestorm. Data from these fires were analysed using a top-down and bottom-up qualitative analysis technique. Forty-four breakdowns in coordinated decision making were identified, which yielded 83 disconnects grouped into three main categories: operational, informational and evaluative. Disconnects were specific instances where differences in understanding existed between team members. The reasons why disconnects occurred were largely consistent across the three sets of data. In some cases multiple disconnects occurred in a temporal manner, which suggested some evidence of disconnects creating states that were conducive to the occurrence of further disconnects. In terms of resolution, evaluative disconnects were nearly always resolved however operational and informational disconnects were rarely resolved effectively. The exploratory data analysis and discussion presented here represents the first systematic research to provide information about the reasons why breakdowns occur in emergency management and presents an account of how team processes can act to disrupt coordination and the operational response.

Values and Complexities in Assessing Strategic‐Level Emergency Management Effectiveness

Emergency managers working at a strategic level play a pivotal role in managing operational needs as well the needs of political elites engaged in crisis management and meaning‐making. Evaluating emergency performance has been widely regarded as problematic, in part because of various stakeholder interests and because it is so subjective. This paper explores the values of personnel working in emergency management at a strategic level as well as the conditions that limit their effectiveness as they attempt to align the needs of different stakeholders. The surveys and interviews conducted suggest that what constitutes success is both subjective and contested by different stakeholders. There is a need to establish a broader perspective on what constitutes success when evaluating emergency response.

A day in the life of a volunteer incident commander: errors, pressures and mitigating strategies.

To meet an identified gap in the literature this paper investigates the tasks that a volunteer incident commander needs to carry out during an incident, the errors that can be made and the way that errors are managed. In addition, pressure from goal seduction and situation aversion were also examined. Volunteer incident commanders participated in a two-part interview consisting of a critical decision method interview and discussions about a hierarchical task analysis constructed by the authors. A SHERPA analysis was conducted to further identify potential errors. The results identified the key tasks, errors with extreme risk, pressures from strong situations and mitigating strategies for errors and pressures. The errors and pressures provide a basic set of issues that need to be managed by both volunteer incident commanders and fire agencies. The mitigating strategies identified here suggest some ways that this can be done.

Challenges of NextGen Technologies for Coordinated Decision Making and the Exchange of Information between Pilots and Controllers

Introduction: This paper presents two studies that explore the implications of NextGen technologies for pilot/controller information exchange and coordinated decision making. Method: In Study 1 five participative focus groups were conducted with human factors experts. In Study 2 fifteen interviews were conducted with air traffic controllers. Both studies employed a thematic analysis. Results and Discussion: Results from Study 1 suggest that changes to pilot and controller information acquisition will alter rather than reduce breakdowns in coordinated decision making. Study 2 identified some basic issues in information exchange with NextGen technologies and suggests that some important non-operational information will be lost. Conclusion: The two studies highlight some important challenges that need to be carefully considered as the NextGen technologies move towards maturity.

Human error during the multilevel responses to three Australian bushfire disasters

The scale and complexity associated with the coordinated response to natural disasters inevitably produce human errors. However, little is known about the frequency and distribution of human error at different levels of coordination during disasters. The purpose of this research was to explore this phenomenon for selected catastrophic bushfires in Australia. To accomplish this, we used the Human Factors Analysis and Classification System that has been widely applied to accidents but is untested with respect to the complexity and temporality of disasters. The results identified that decision errors made during these disasters differed depending upon the level of coordination but were associated with information uncertainty, fatigue, coordination complexities, procedural violations, and degraded personal interactions.