Florian Fortmann

Supporting situation awareness with peripheral feedback on monitoring behavior

Maintaining situation awareness during supervisory control of automated systems is a mental task, which requires the human operator to perform adequate monitoring behavior. But research has shown that this requirement is often violated, e.g., due to distraction and fatigue. To overcome this problem, we envision an adaptive, peripheral light display conveying feedback on the adequacy of the monitoring behavior of the human operator. This paper presents the results of a focus group, which has been conducted in order to specify (1) which type of information regarding the adequacy of monitoring behavior should be conveyed by the peripheral feedback display and (2) how the information should be encoded in light.

A-PiMod: A New Approach to Solving Human Factors Problems with Automation

The objective of this paper is to present a new adaptive automation concept which (1) addresses the still open human factors problems with automation from a team centred perspective and (2), as part of this, offers a new ‘team’ centred approach to solving these problems. In so doing, this paper poses questions about what it means to work in a team, what kind of expertise a third crew member (i.e. automation) offers, and how team members might share information about their state, intentions and actions. In elucidating this new automation concept, this paper introduces new role/work practice concepts for pilots, and a potential roadmap for adaptive automation and single crew operations.

Developing a feedback system to augment monitoring performance of aircraft pilots

As a supervisor of a highly automated technical system, a human operator is the “ultima ratio” in abnormal situations that cannot be handled by the automation. Therefore, a human operator must adequately monitor the automated system throughout the entire operation. Unfortunately, humans tend to fall prey to a couple of demons leading to inadequate monitoring behavior, such as boredom, attentional tunneling or perservation. As a consequence, the human might lose situation awareness and be unable to detect and handle an abnormal situation in the given amount time. In this paper, we present a feedback system to augment monitoring performance on the part of an aircraft pilot. The feedback system has been developed under the umbrella of the European research and development project A-PiMod, where novel team-centered concepts for pilot-automation interaction are investigated.

Expert-based design and evaluation of an ambient light display to improve monitoring performance during multi-UAV supervisory control

In complex supervisory control settings, inadequate monitoring behavior is a substantial source of human error and a risk factor threatening human life and the environment. In this paper, we present a problem-oriented approach to support a commander supervising multiple highly-automated unmanned aerial vehicles (UAV) to perform adequate monitoring behavior. Our approach utilizes an ambient light display (ALD) to continuously externalize the commanders monitoring performance using ambient visual cues in the peripheral field. We designed and evaluated our prototype with experts. Our results indicate that our prototype can support adequate monitoring behavior of a commander supervising multiple UAVs, and does not affect workload.

An intelligent SA-adaptive interface to aid supervisory control of a UAV swarm

Building and maintaining Situation Awareness (SA) during supervisory control of a swarm of highly automated Unmanned Aerial Vehicles (UAV) is a complex and visually demanding process, which is often affected by weak scanning behaviors of the human operators. As a consequence severe errors, such as incorrect decisions, are likely to occur. This paper presents an Intelligent SA-Adaptive Interface (ISAAI) for supervisory control of a UAV swarm, which aims at optimizing the scanning behavior of a human operator with regard to his/her actual SA-needs. Therefore, the ISAAI integrates an eye movement based technique to assess human operator SA during mission execution and a technique for guiding visual attention to relevant but unattended information.

Adaptive Automation and the Third Pilot: Managing Teamwork and Workload in an Airline Cockpit

The objective of this paper is to present a new adaptive automation concept which offers an innovative ‘team’ centred approach to solving human factors/workload management problems. The A-PiMod concept/approach is defined by the concept of partnership – specifically, the “Third Pilot” and the crew and automation are in charge together. We are proposing partnership as opposed to dynamic changes in control function where changes can be controlled autonomously by the system. In support of this, a new multimodal concept is proposed which supports improved assessment of crew state/workload (i.e. information inputs re crew activity/interactions provides a means to communicate with the crew in relation to crew state and decision support, and allows for flexible crew/cockpit interaction).

Development and Evaluation of an Assistant System to Aid Monitoring Behavior during Multi-UAV Supervisory Control: Experiences from the D3CoS Project

The core function a human operator in charge of a supervisory control task is responsible for is monitoring. However, research has shown that the correct execution of this function is often violated. The consequences can be disastrous for human life and the environment. Within the framework of the European project D3CoS, we developed an assistant system to aid the monitoring behavior of a human operator in charge of supervisory control of highly automated unmanned aerial vehicles. The idea behind the assistant system was to continuously invoke visual cues on the display used to supervise the mission in order to guide the operators visual attention towards information demanding attention. Two studies were performed to evaluate the system along different target measures, such as situation awareness, workload, user acceptance and market potential. Overall, the results show that the system has positive effects on many target measures but not on all of them. Further research is needed to improve the system functions.

Aircraft Pilot Intention Recognition for Advanced Cockpit Assistance Systems

Present aircraft are highly automated systems. In general, automation improved aviation safety significantly. However, automation exhibits itself in many forms of adverse behaviors related to human factors problems. A major finding is that insufficient support of partnership between the pilot crew and the aircraft automation can result in conflicting intentions. The European project A-PiMod Applying Pilot Models for Safer Aircraft addresses issues of conventional automation in the aviation domain. The overall objective of the project is to foster pilot crew-automation partnership on the basis of a novel architecture for cooperative automation. An essential part of the architecture is an intention recognition module. The intention recognition module employs a Hidden Markov Model HMM to infer the most probable current intention of the human pilots. The HMM is trained and evaluated with data containing interactions of human pilots with the aircraft cockpit systems. The data was obtained during experiments with human pilots in a flight simulator.