Carmen Bruder

How to measure monitoring performance of pilots and air traffic controllers

In prior research on the future of aviation it was established that operators will have to work with highly automated systems. Increasing automation will require operators monitoring appropriately (OMA). OMA are expected to demonstrate the use of distinctly different monitoring phases (orientation, anticipation, detection, and recheck). Within these phases, they must grasp in time the relevant information that would enable them to take control should automation fail. The presented study aims at finding appropriate measurements for the identification of OMA on the basis of eye tracking. In order to do this, a normative model of adequate monitoring behavior was designed including the definition of areas of interest. We tested 90 participants who had to monitor a dynamic automatic process, and then take control. In order to decide on suitable eye tracking parameters it was asked which parameters are significantly related to manual control performance. The results show that the suitability of parameters depends on the specific phase of the monitoring process. Gaze durations allow for differentiating between high and low performing subjects during orientation phases. In contrast, relative fixation counts are suitable for predicting monitoring performance during detection phases. In general, the results support the assumption that eye tracking parameters are appropriate for identifying OMA.

Adaptive training interfaces for less-experienced, elderly users of electronic devices

A great number of complex electronic devices are now part of our everyday lives. While many of us learn to handle these products by trial and error; others, especially older users with little experience in using electronic devices, need support. In order to allow the user maximum flexibility in terms of learning time and location, a training programme is presented which is implemented as part of the software embedded in the product itself. Particular focus is placed on the effect of adaptive training on learning. In this study, the training versions differed in their ability to adjust their complexity to the users experience (adaptive user interface complexity) and their capability to support the learner by prompting them during the learning process (adaptive training advice). The results show that the adjustment of complexity had a positive effect on users’ experience: elderly users who trained with an adaptive interface were more successful in learning to use a mobile phone. Adaptive training advice, however, was found to have no significant effects on learners’ success and reduced their self-efficacy. This work offers guidelines on how to design integrated training applications for electronic devices that successfully help elderly users with little prior experience.

Future Ability Requirements for Human Operators in Aviation

The present study addresses the optimal fit between technical innovations in aviation and aircraft operators. Because of the increase in computerization, an accurate and efficient monitoring of the automation poses a key challenge to future operators. As the German Aerospace Centers Department of Aviation and Space Psychology is responsible for personnel selection of pilots and air traffic controllers, our objective for the selection of future personnel is to distinguish good monitoring operators from bad operators. In order to identify good monitoring behavior we developed a simulation tool that represents tasks of pilots and controllers within a dynamic air traffic flow. Participants have either to monitor the automatic process or to control the dynamic traffic manually. Monitoring behavior is measured by recording eye movement parameters. The identification of accurate monitoring behavior enables us to adapt selection profiles to future ability requirements.

Differences in Monitoring between Experts and Novices

In the future of aviation, operators will have to work with highly automated systems. This necessitates operators monitoring appropriately (OMA). A theoretical model for adequate monitoring behavior was developed and eye movement parameters were defined to identify OMA. The present eye-tracking study focused on differences in monitoring behavior between experts (experienced pilots and air traffic controllers) and job applicants. Results from 21 experts and 33 applicants are reported.

Eye-tracking measurements and their link to a normative model of monitoring behaviour

Increasing automation necessitates operators monitoring appropriately (OMA) and raises the question of how to identify them in future selections. A normative model was developed providing criteria for the identification of OMA. According to this model, the monitoring process comprises distinct monitoring phases (orientation, anticipation, detection and recheck) in which attention should be focused on relevant areas. The current study tests the normative model on the basis of eye tracking. The eye-tracking data revealed increased concentration on relevant areas during the orientation and anticipation phase in comparison to the other phases. For the assessment of monitoring behaviour in the context of personnel selection, this implies that the anticipation and orientation phases should be considered separately as they appear to be more important in the context of monitoring than the other phases. Practitioner Summary: A normative model was developed for the assessment of monitoring behaviour. Using the eye-tracking method, this model was tested with applicants for an Air Traffic Controller training programme. The results are relevant for the future selection of human operators, who will have to monitor highly automated systems.

On the Development of a Monitoring Test for the Selection of Aviation Operators

In future ATM systems aviation operators will have to work with highly automated systems. According to prior research operational monitoring will become a prominent aspect of task performance [1]. Thus, operators monitoring appropriately will be needed, being able to detect system errors in time and to take over control if automation fails. To monitor appropriately, different monitoring phases orientation, anticipation, detection, and recheck have to be performed by the subjects. Within these phases, the relevant information should be gathered to enable taking over full control, once automation should fail. Based on this model of appropriate monitoring behavior [2, 3], the present study focuses on the development of a monitoring test for the selection of future operators in the field of aviation.

Making Multi-team Systems More Adaptable by Enhancing Transactive Memory System Structures – The Case of CDM in APOC

The DLR project ‘Inter Team Collaboration’ (ITC) aims to provide systems engineers with tools and human factors concepts that allow a systemic access to the social side of socio-technical systems. A main design question for implementing Collaborative Decision Making (CDM) in APOC is how to induce collaborative decision making in a dynamic environment of ATM to make it more adaptive and resilient. Our main assumption is that the establishment of a Transactive-Memory System (TMS) is the basic predisposition for a successful implementation of intensive CDM. A TMS reflects linkages across MTS boundaries. Assumedly, its emergence is a function of social structures (like motives), but also of communication structures. The MTS is conceptualized as a nonlinear dynamical system (NDS), where CDM is conceived as an attractor to system-behavior. Recurrence analyses on behavioral data assessed within Human-in-the-Loop-experiments will be applied to identify MTS transition phases in reaction to perturbations.

To look and (not) see: predicting the detection of automation failures based on the eye movements of human operators

In order to detect automation failures in time, operators are required to monitor automated systems efficiently. The present study analyzed eye movements to predict whether or not subjects would detect an automation failure. Eye movements were recorded whilst subjects were monitoring an automated system where automation failures sometimes occur. The data imply that the eye movements of human operators effectively differ between operators who detect the automation failure and those who miss it. The findings are discussed in the context of personnel selection and incident reporting as used in air traffic control (ATC).