Patricia May Ververs

Costs and Benefits of Head-Up Display Use: A Meta-Analytic Approach:

Costs and benefits associated with head-up display (HUD) use were assessed by analyzing data obtained from studies in the aviation domain. Eighteen studies investigating the use of HUDs, head-down displays, and conformal displays were included in analyses of flight path tracking and event detection performance. HUD and conformality benefits consisted of faster detection responses to changes in symbology and presentation of traffic, as well as increased flight path tracking accuracy. Further investigation into the HUD data revealed a heterogeneous component, suggesting that factors other than display location were contributing to pilot performance. One factor modulating performance was the relative expectancy of the event. HUD use was advantageous if an event in the environment was expected. However, when an event was unexpected, detection was degraded with the presence of HUD instrumentation. This finding reveals a potential cost due to HUD use when confronted with anomalous situations.

Pathway HUDs: Are they viable?

We describe two experiments that examine 3D pathway displays in a head-up location for aircraft landing and taxi. We address both guidance performance and pilot strategies in dividing, focusing, and allocating attention between flight path information and event monitoring. In Experiment 1 the 3D pathway head-up display (HUD) was compared with a conventional 2D HUD. The former was found to produce better guidance, with few costs to event detection. Some evidence was provided that attentional tunneling of the pathway HUD inhibits the detection of unexpected traffic events. In Experiment 2, the pathway display was compared in a head-up versus a head-down location. Excellent guidance was achieved in both locations. A slight HUD cost for vertical tracking in the air was offset by a HUD benefit for event detection and for lateral tracking during taxi (i.e., on the ground). The results of both experiments are interpreted within the framework of object- and space-based theories of visual attention and point to the conclusion that pathway HUDs combine the independent advantages of pathways and HUDs, particularly during ground operations. Actual or potential applications include understanding the costs and benefits of positioning a 3D pathway display

Considering Etiquette in the Design of an Adaptive System

In this article, the authors empirically assess the costs and benefits of designing an adaptive system to follow social conventions regarding the appropriateness of interruptions. Interruption management is one area within the larger topic of automation etiquette. The authors tested these concepts in an outdoor environment using the Communications Scheduler, a wearable adaptive system that classifies users’ cognitive state via brain and heart sensors and adapts its interactions. Designed to help dismounted soldiers, it manages communications in much the same way as a good administrative assistant. Depending on a combination of message priority, user workload, and system state, it decides whether to interrupt the user’s current tasks. The system supports decision makers in two innovative ways: It reliably measures a mobile user’s cognitive workload to adapt its behavior, and it implements rules of etiquette adapted from human-human interactions to improve human-computer interactions. Results indicate costs and benefits to both interrupting and refraining from interrupting. When users were overloaded, primary task performance was improved by managing interruptions. However, overall situation awareness on secondary tasks suffered. This work empirically quantifies costs and benefits of “appropriate” interruption behaviors, demonstrating the value of designing adaptive agents that follow social conventions for interactions with humans.

Rapid image analysis using neural signals

The problem of extracting information from large collections of imagery is a challenge with few good solutions. Computers typically cannot interpret imagery as effectively as humans can, and manual analysis tools are slow. The research reported here explores the feasibility of speeding up manual image analysis by tapping into split second perceptual judgments using electroencephalograph sensors. Experimental results show that a combination of neurophysiological signals and overt physical responses--detected while a user views imagery in high speed bursts of approximately 10 images per second--provide a basis for detecting targets within large image sets. Results show an approximately six-fold, statistically significant, reduction in the time required to detect targets at high accuracy levels compared to conventional broad-area image analysis.

Costs and Benefits of Head up Displays: An Attention Perspective and a Meta Analysis

This paper reports a meta analysis of all studies located in the literature that have compared head up versus head down display of equivalent information, as these displays support both tracking (e.g., flight path control) and discrete event detection. The data clearly indicate a HUD advantage for most tasks, except tracking during cruise flight and event detection during final approach. The latter HUD cost however is observed only when events to be detected are entirely unexpected, reflecting a form of cognitive tunneling. The meta-analysis also reveals an advantage for conformal over non-conformal HUD imagery.

Supporting Real-time Cognitive State Classification on a Mobile Individual

The effectiveness of neurophysiologically triggered adaptive systems hinges on reliable and effective signal processing and cognitive state classification. Although this presents a difficult technical challenge in any context, these concerns are particularly pronounced in a system designed for mobile contexts. This paper describes a neurophysiologically derived cognitive state classification approach designed for ambulatory task contexts. We highlight signal processing and classification components that render the electroencephalogram (EEG) -based cognitive state estimation system robust to noise. Field assessments show classification performance that exceeds 70% for all participants in a context that many have regarded as intractable for cognitive state classification using EEG.

Neurophysiologically driven image triage: a pilot study

Effective analysis of complex imagery is a vital aspect of important domains such as intelligence image analysis. As technological developments lower the cost of gathering and storing imagery, the cost of searching through large image sets for important information has been growing substantially. This paper demonstrates the feasibility of using neurophysiological signals associated with early perceptual processing to identify critical information within large image sets efficiently. Brain signals called evoked response potentials, detected in conjunction with rapid serial presentation of images, show promise as a human computer interaction modality for screening high volumes of imagery accurately and efficiently.

A joint human-automation cognitive system to support rapid decision-making in hostile environments

Honeywell has designed a joint human-computer cognitive system to support rapid decision making demands of dismounted soldiers. In highly networked environments the sheer magnitude of communication amid multiple tasks could overwhelm individual soldiers. Key cognitive bottlenecks constrain information flow and the performance of decision-making, especially under stress. The adaptive decision-support system mitigates non-optimal human performance via automation when the system detects a breakdown in the humans cognitive state. The humans cognitive state is assessed in real-time via a suite of neuro-physiological and physiological sensors. Adaptive mitigation strategies can include task management, optimizing information presentation via modality management, task sharing, and task loading. Mitigations are designed with consideration for both the costs and benefits of intermittent augmentation. The paper describes the system development and evolution, explorations of usable cognitive mitigation strategies, and four evaluations that show adaptive automaton can effectively, mitigate human decision-making performance at extremes (overload and underload) of workload.

Closing the Loop of an Adaptive System with Cognitive State

This paper describes an adaptive system that “closes the loop” by utilizing a real-time, directly sensed measure of cognitive state of the human operator. The Honeywell Augmented Cognition team has developed a Closed Loop Integrated Prototype (CLIP) of a Communications Scheduler, for application to the U.S. Armys Future Force Warrior (FFW) program. It is expected that in a highly networked environment the sheer magnitude of communication traffic could overwhelm the individual soldier. The CLIP exploits real-time neurophysiological and physiological measurements of the human operator in order to create a cognitive state profile, which is used to augment the work environment to improve human-automation joint performance. An experiment showed that the Communications Scheduler enabled higher situation awareness and message comprehension in high workload conditions. Based solely on cognitive state, the system inferred a subjects message comprehension and repeated unattended messages in the majority of cases, without yielding an unacceptably high false alarm rate.

Neuro-Physiologically-Driven Adaptive Automation to Improve Decision Making Under Stress

The advent of netted communications and a wide array of battlefield sensors is enabling real-time information streaming and asset management. However, the burden of information management is placed solely on the receiver of the information. Honeywell Laboratories developed a Communications Scheduler (CoS), an adaptive information management system for the dismounted Soldier, driven by an assessment of the individuals current cognitive capacity to process incoming information, in order to improve decision making under high task load conditions. An evaluation was conducted to demonstrate whether cognitive capacity to perform under differing task loads could be detected using neuro-physiological sensors and then if the adaptive automation would appropriately regulate information flow. Results revealed an improvement in primary task performance, no degradation in concurrent secondary tasks, and lower subjective workload ratings while performing cognitively challenging working memory tasks with the CoS, although a slight loss in situation awareness of lower priority message was found. The appropriate allocation of cognitive resources is key to managing multiple tasks, focusing on the most important ones, and maintaining overall situation awareness.