Mark I. Nikolic

Not now! Supporting interruption management by indicating the modality and urgency of pending tasks

Operators in complex event-driven domains must coordinate competing attentional demands in the form of multiple tasks and interactions. This study examined the extent to which this requirement can be supported more effectively through informative interruption cueing (in this case, partial information about the nature of pending tasks). The 48 participants performed a visually demanding air traffic control (ATC) task. They were randomly assigned to 1 of 3 experimental groups that differed in the availability of information (not available, available upon request, available automatically) about the urgency and modality of pending interruption tasks. Within-subject variables included ATC-related workload and the modality, frequency, and priority of interruption tasks. The results show that advance knowledge about the nature of pending tasks led participants to delay visual interruption tasks the longest, which allowed them to avoid intramodal interference and scanning costs associated with performing these tasks concurrently with ATC tasks. The 3 experimental groups did not differ significantly in terms of their interruption task performance; however, the group that automatically received task-related information showed better ATC performance, thus experiencing a net performance gain. Actual or potential applications of this research include the design of interfaces in support of attention and interruption management in a wide range of event-driven environments.

Peripheral Visual Feedback: A Powerful Means of Supporting Effective Attention Allocation in Event-Driven, Data-Rich Environments

Breakdowns in human-automation coordination in data-rich, event-driven domains such as aviation can be explained in part by a mismatch between the high degree of autonomy yet low observability of modern technology. To some extent, the latter is the result of an increasing reliance in feedback design on foveal vision - an approach that fails to support pilots in tracking system-induced changes and events in parallel with performing concurrent flight-related tasks. One possible solution to the problem is the distribution of tasks and information across sensory modalities and processing channels. A simulator study is presented that compared the effectiveness of current foveal feedback and two implementations of peripheral visual feedback for keeping pilots informed about un commanded changes in the status of an automated cockpit system. Both peripheral visual displays resulted in higher detection rates and faster response times, without interfering with the performance of concurrent visual tasks any more than does currently available automation feedback. Potential applications include improved display designs that support effective attention allocation in a variety of complex dynamic environments, such as aviation, process control, and medicine.

WHY PILOTS MISS THE GREEN BOX: HOW DISPLAY CONTEXT UNDERMINES ATTENTION CAPTURE

Visual displays often employ the onset or flashing of an element to notify users of important events. Recent research findings and operational experiences in data-rich, event-driven domains, such as aviation, suggest that this design approach, which was supported by findings from early basic research on attention capture, is not always successful. The goal of this study was to examine how display context affects the effectiveness of abrupt onset signals. Participants in this study performed an externally paced visual task while trying to detect abrupt-onset stimuli, which were presented against 5 different display backgrounds and at 2 different eccentricities. The display background varied in terms of its dynamics and its color similarity to the target. Color similarity, the movement of background elements, and increasing target eccentricity resulted in reduced detection performance. The findings from this study help explain why pilots on modern flight decks sometimes miss changes in the status and behavior of their automated systems. More generally, they illustrate the importance of considering display context and the need to adapt findings from laboratory research when designing interfaces for complex environments.

Supporting Timesharing and Interruption Management Through Multimodal Information Presentation

Operators in complex event-driven domains often need to perform multiple concurrent tasks and handle competing attentional demands, such as interruptions by other human or machine agents. This study examined the effectiveness of distributing tasks across various sensory channels and presenting information on the nature of an interruption task to support timesharing and attention management. Participants performed a visually demanding simulated Air Traffic Control (ATC) task involving Data Link communication. At times, an interruption task was introduced, which consisted of counting subsets of signals that were presented in visual, auditory, or tactile form. Half of the subjects automatically received information on the modality and urgency of these pending interruption tasks whereas the other participants had the option to request this information. Within-subject variables in this study included ATC-related workload and the frequency and priority of interruption tasks. High-priority tasks had to be performed immediately whereas low-priority tasks could be delayed for up to two minutes. The results show that information about the nature of pending tasks supported participants in scheduling and timesharing more effectively. They were able to avoid intramodal interference and scanning costs associated with performing the ATC task concurrently with a visual interruption task. Crossmodal interference was lowest for auditory interruption tasks. Overall, these findings illustrate the benefits of multimodal information presentation and more informative interruption signals.

Redundancy Gains in Communication Tasks: A Comparison of Auditory, Visual, and Redundant Auditory-Visual Information Presentation on NextGen Flight Decks

The redundant presentation of information in more than one sensory channel has traditionally been assumed to benefit performance. However, a recent meta-analysis suggests that redundancy gains may depend on task type and a number of moderator variables. The present study examined the effectiveness of visual, auditory, and redundant auditory-visual information presentation under high and low workload in the context of a mid-fidelity NextGen flight simulation with experienced airline pilots as participants. Overall, for two types of communication tasks – data link and ATIS – faster responses were observed for redundant displays, compared to vision and audition alone. No significant benefit of redundancy was found for accuracy due to a ceiling effect and workload did not mediate redundancy effects. The findings from this research add to the knowledge base in multimodal and redundant information processing and can inform modality choices in the design of displays for complex, data-rich domains.

Multimodal Information Presentation in Support of NextGen Operations

ABSTRACT Objective: This study examined the effectiveness of visual, auditory, tactile, and redundant auditory-visual information presentation in the context of a medium-fidelity ‘Next Generation Air Transportation System’ (NextGen) flight simulation. Background: Data overload, especially in the visual channel, and associated breakdowns in monitoring represent a major challenge in aviation. These problems are expected to worsen with NextGen, which will require pilots to manage increased amounts of data and adopt new responsibilities. The introduction of multimodal interfaces (interfaces that distribute information across multiple sensory channels) has been proposed as a means to offload the overburdened visual channel and thus address data overload. Method: Experienced commercial airline pilots completed 2 scenarios using a medium-fidelity flight simulator. For each scenario, NextGen tasks and events were presented either using technology that is currently available (visual and auditory displays) or technology proposed as part of NextGen design concepts (i.e., tactile and redundant displays). Performance was measured based on response time and accuracy. Results: Faster responses were observed for redundant displays, compared to either vision or audition alone. No significant benefit of redundancy was found for accuracy and workload did not mediate redundancy effects. For traffic events, there were faster response times with tactile displays, but higher response accuracy with auditory displays. Conclusion: The findings from this research add to the knowledge base in multimodal information processing and can inform the design of displays for NextGen operations.

Converging on Error Management: A Review of Current Findings and Future Needs

For many years, the focus of research in the area of human error was the prevention of erroneous actions and assessments through training and design. However, errors are a fundamental aspect of human adaptation in complex systems and will never be eliminated completely. More recent approaches to error stress the need to minimize its negative consequences through support for error management, i.e., the detection, explanation, and recovery from erroneous actions. For the most part, these efforts have examined the first step in this sequence — error detection. However, there continue to be gaps in our understanding of how operators explain and recover from errors. This paper reviews the current state of error management research and methods. Findings are presented from an ongoing converging operations approach to error management research. Finally, promising areas in which to make further advances in this field are identified.

Error Management on Modern Flight Decks: How Pilots Explain and Recover from Unintended Actions and Outcomes

Errors are a fundamental aspect of human adaptation in complex systems. Efforts to reduce and/or eliminate errors through training and design have met with only limited success. Therefore, it is critical to invest also in effective support for error management (the detection, explanation, and recovery from errors) to mitigate their negative consequences. To date, most research on error management has focused on the first step in this process – error detection. Instead, this paper presents a subset of results from a recent simulator study that examined the role of error explanation and the nature and effectiveness of error recovery strategies. Process tracing was used to analyze error management activities of pilots on highly automated flight decks. Our findings suggest that incomplete mental models of automation function limit error explanation and often lead to recovery strategies that rely on either “resetting” the system or serendipitous error evasion prior to negative consequences, or reverting to lower levels of automation when immediate corrective actions become necessary. These findings will be discussed in terms of their implications for training and design.

Modeling Error Recovery in Dynamic Collaborative Domains

For many years, the focus of research in the area of human error was the prevention of erroneous actions and assessments through training and design. However, errors can never be eliminated completely. Therefore, the goal of more recent efforts is to minimize their negative consequences through support for error management, i.e., the detection, explanation, and recovery from erroneous actions. For the most part, these efforts have examined the first step in this sequence - error detection. In contrast, little is known about how operators explain and recover from errors. This is true especially for dynamic collaborative environments such as aviation. In this paper, we present findings from a survey and an incident report analysis that suggest the need for adapting the current model of error recovery. Specifically, we emphasize the importance of considering constraints imposed by specific domains in order to predict and explain the predominance and success of certain recovery strategies.