Nadine B. Sarter

HOW IN THE WORLD DID WE EVER GET INTO THAT MODE? MODE ERROR AND AWARENESS IN SUPERVISORY CONTROL

New technology is flexible in the sense that it provides practitioners with a large number of functions and options for carrying out a given task under different circumstances. However, this flexibility has a price. Because the human supervisor must select the mode best suited to a particular situation, he or she must know more than before about system operations and the operation of the system as well as satisfy new monitoring and attentional demands to track which mode the automation is in and what it is doing to manage the underlying processes. When designers proliferate modes without supporting these new cognitive demands, new mode-related error forms and failure paths can result. Mode error has been discussed in human-computer interaction for some time; however, the increased capabilities and the high level of autonomy of new automated systems appear to have created new types of mode-related problems. We explore these new aspects based on results from our own and related studies of human-automation interaction. In particular, we draw on empirical data from a series of studies of pilot-automation interaction in commercial glass cockpit aircraft to illustrate the nature, circumstances, and potential consequences of mode awareness problems in supervisory control of automated resources. The result is an expanded view of mode error that takes into account the new demands imposed by more automated systems.

Good Vibrations: Tactile Feedback in Support of Attention Allocation and Human-Automation Coordination in Event-Driven Domains

Observed breakdowns in human-machine communication can be explained, in part, by the nature of current automation feedback, which relies heavily on focal visual attention. Such feedback is not well suited for capturing attention in case of unexpected changes and events or for supporting the parallel processing of large amounts of data in complex domains. As suggested by multiple-resource theory, one possible solution to this problem is to distribute information across various sensory modalities. A simulator study was conducted to compare the effectiveness of visual, tactile, and redundant visual and tactile cues for indicating unexpected changes in the status of an automated cockpit system. Both tactile conditions resulted in higher detection rates for, and faster response times to, uncommanded mode transitions. Tactile feedback did not interfere with, nor was its effectiveness affected by, the performance of concurrent visual tasks. The observed improvement in task-sharing performance indicates that the introduction of tactile feedback is a promising avenue toward better supporting humanmachine communication in event-driven, information-rich domains.

SUPPORTING DECISION MAKING AND ACTION SELECTION UNDER TIME PRESSURE AND UNCERTAINTY: THE CASE OF IN-FLIGHT ICING

Operators in high-risk domains such as aviation often need to make decisions under time pressure and uncertainty. One way to support them in this task is through the introduction of decision support systems (DSSs). The present study examined the effectiveness of two different DSS implementations: status and command displays. Twenty-seven pilots (9 pilots each in a baseline, status, and command group) flew 20 simulated approaches involving icing encounters. Accuracy of the decision aid (a smart icing system), familiarity with the icing condition, timing of icing onset, and autopilot usage were varied within subjects. Accurate information from either decision aid led to improved handling of the icing encounter. However, when inaccurate information was presented, performance dropped below that of the baseline condition. The cost of inaccurate information was particularly high for command displays and in the case of unfamiliar icing conditions. Our findings suggest that unless perfect reliability of a decision aid can be assumed, status displays may be preferable to command displays in high-risk domains (e.g., space flight, medicine, and process control), as the former yield more robust performance benefits and appear less vulnerable to automation biases.

Team play with a powerful and independent agent: a full-mission simulation study.

One major problem with pilot-automation interaction on modern flight decks is a lack of mode awareness; that is, a lack of knowledge and understanding of the current and future status and behavior of the automation. A lack of mode awareness is not simply a pilot problem; rather, it is a symptom of a coordination breakdown between humans and machines. Recent changes in automation design can therefore be expected to have an impact on the nature of problems related to mode awareness. To examine how new automation properties might affect pilot-automation coordination, we performed a full-mission simulation study on one of the most advanced automated aircraft, the Airbus A-320. The results of this work indicate that mode errors and automation surprises still occur on these advanced aircraft. However, there appear to be more opportunities for delayed or missing interventions with undesirable system activities, possibly because of higher system autonomy and coupling.

THE NEED FOR MULTISENSORY INTERFACES IN SUPPORT OF EFFECTIVE ATTENTION ALLOCATION IN HIGHLY DYNAMIC EVENT-DRIVEN DOMAINS: THE CASE OF COCKPIT AUTOMATION

In a variety of domains, automation technology has evolved from passive tools to highly autonomous agents that can initiate actions independent of user input and without explicit operator consent. This evolution brings with it an increased need for effective human-automation communication and coordination to ensure that both agents stay informed about each others goals, activities, and limitations. Yet, most modern systems are not equipped with the skills required to contribute effectively and in a timely manner to the exchange of information on commitments and actions. In particular, systems fail to provide external attentional guidance to their operators in the case of uncommanded changes and events, which can lead to automation surprises and, sometimes, incidents and accidents. To a large extent, these problems can be explained by designers increasing reliance on automation feedback that requires focal visual attention. This article explores the potential of multisensory displays to better support attentional guidance in multidisplay environments and to allow for parallel processing of the considerable amount of information that is available in many complex dynamic domains such as the modern flight deck.

Error Types and Related Error Detection Mechanisms in the Aviation Domain: An Analysis of Aviation Safety Reporting System Incident Reports

Human error is considered a contributing factor in 70% to 80% of all aviation accidents. Because errors can never be eliminated completely, a further reduction of the already low accident rate in this domain will require investments in better support for error management. In particular, a better understanding of the nature and effectiveness of error detection mechanisms is needed. With this goal in mind, NASA Aviation Safety Reporting System incident reports were analyzed in terms of the formal characteristics of underlying errors, the cognitive stage, and the performance level at which these errors occurred, and with respect to the processes that led to their detection and, thus, prevented these incidents from turning into accidents. The majority of incidents involved lapses (i.e., failures to perform a required action) or mistakes, such as errors in intention formation and strategy choice. These errors were most often detected based on routine checks and the observed outcome of an action, respectively. Most slips appear to have been discovered by the crew before they could lead to a problem worth reporting. Our findings suggest a need for more effective feedback in support of data-driven monitoring, especially in the case of errors of omission and for shared knowledge of intent between airborne and ground-based operators to promote the more timely and reliable detection of mistakes.

Stages and Levels of Automation: An Integrated Meta-analysis

Function allocation between human and automation can be represented in terms of the stages & levels taxonomy proposed by Parasuraman, Sheridan & Wickens (2000). Higher degrees of automation (DOA) are achieved both by later stages (e.g., automation decision aiding rather than diagnostic aiding) and higher levels within stages (e.g. executing a choice unless vetoed, versus offering the human several choices). A meta analysis based on data of 14 experiments examines the mediating effects of DOA on routine system performance, performance when the automation fails, workload and situation awareness. The effects of DOA on these four measures are summarized by level of statistical significance. We found: (1) an inverse relationship between routine performance and workload as automation is introduced and DOA increases. (2) a weak positive relationship between routine performance and failure performance, as mediated by DOA. (3) A strong mediating role of situation awareness in improving both routine and failure performance.

Peripheral Visual Feedback: A Powerful Means of Supporting Effective Attention Allocation in Event-Driven, Data-Rich Environments

Breakdowns in human-automation coordination in data-rich, event-driven domains such as aviation can be explained in part by a mismatch between the high degree of autonomy yet low observability of modern technology. To some extent, the latter is the result of an increasing reliance in feedback design on foveal vision - an approach that fails to support pilots in tracking system-induced changes and events in parallel with performing concurrent flight-related tasks. One possible solution to the problem is the distribution of tasks and information across sensory modalities and processing channels. A simulator study is presented that compared the effectiveness of current foveal feedback and two implementations of peripheral visual feedback for keeping pilots informed about un commanded changes in the status of an automated cockpit system. Both peripheral visual displays resulted in higher detection rates and faster response times, without interfering with the performance of concurrent visual tasks any more than does currently available automation feedback. Potential applications include improved display designs that support effective attention allocation in a variety of complex dynamic environments, such as aviation, process control, and medicine.

WHY PILOTS MISS THE GREEN BOX: HOW DISPLAY CONTEXT UNDERMINES ATTENTION CAPTURE

Visual displays often employ the onset or flashing of an element to notify users of important events. Recent research findings and operational experiences in data-rich, event-driven domains, such as aviation, suggest that this design approach, which was supported by findings from early basic research on attention capture, is not always successful. The goal of this study was to examine how display context affects the effectiveness of abrupt onset signals. Participants in this study performed an externally paced visual task while trying to detect abrupt-onset stimuli, which were presented against 5 different display backgrounds and at 2 different eccentricities. The display background varied in terms of its dynamics and its color similarity to the target. Color similarity, the movement of background elements, and increasing target eccentricity resulted in reduced detection performance. The findings from this study help explain why pilots on modern flight decks sometimes miss changes in the status and behavior of their automated systems. More generally, they illustrate the importance of considering display context and the need to adapt findings from laboratory research when designing interfaces for complex environments.

Smart Icing Systems for Aircraft Icing Safety

Ice accretion affects the performance and control of an aircraft and in extreme situations can lead to incidents and accidents. However, changes in performance and control are difficult to sense. As a result, the icing sensors currently in use sense primarily ice accretion, not the effect of the ice. No processed aircraft performance degradation information is available to the pilot. In this paper, the Smart Icing System research program is reviewed and progress towards its development reported. Such a system would sense ice accretion through traditional icing sensors and use modern system identification methods to estimate aircraft performance and control changes. This information would be used to automatically operate ice protection systems, provide aircraft envelope protection and, if icing was severe, adapt the flight controls. All of this would be properly communicated to and coordinated with the flight crew. In addition to describing the basic concept, this paper reviews the research conducted to date in three critical areas; aerodynamics and flight mechanics, aircraft control and identification, and human factors. In addition, the flight simulation development is reviewed, as well as the Twin Otter flight test program that is being conducted in cooperation with NASA Glenn Research Center.