Mickaël Causse

Failure to Detect Critical Auditory Alerts in the Cockpit Evidence for Inattentional Deafness

Objective: The aim of this study was to test whether inattentional deafness to critical alarms would be observed in a simulated cockpit. Background: The inability of pilots to detect unexpected changes in their auditory environment (e.g., alarms) is a major safety problem in aeronautics. In aviation, the lack of response to alarms is usually not attributed to attentional limitations, but rather to pilots choosing to ignore such warnings due to decision biases, hearing issues, or conscious risk taking. Method: Twenty-eight general aviation pilots performed two landings in a flight simulator. In one scenario an auditory alert was triggered alone, whereas in the other the auditory alert occurred while the pilots dealt with a critical windshear. Results: In the windshear scenario, 11 pilots (39.3%) did not report or react appropriately to the alarm whereas all the pilots perceived the auditory warning in the no-windshear scenario. Also, of those pilots who were first exposed to the no-windshear scenario and detected the alarm, only three suffered from inattentional deafness in the subsequent windshear scenario. Conclusion: These findings establish inattentional deafness as a cognitive phenomenon that is critical for air safety. Pre-exposure to a critical event triggering an auditory alarm can enhance alarm detection when a similar event is encountered subsequently. Application: Case-based learning is a solution to mitigate auditory alarm misperception.

Monitoring cognitive and emotional processes through pupil and cardiac response during dynamic versus logical task.

The paper deals with the links between physiological measurements and cognitive and emotional functioning. As long as the operator is a key agent in charge of complex systems, the definition of metrics able to predict his performance is a great challenge. The measurement of the physiological state is a very promising way but a very acute comprehension is required; in particular few studies compare autonomous nervous system reactivity according to specific cognitive processes during task performance and task related psychological stress is often ignored. We compared physiological parameters recorded on 24 healthy subjects facing two neuropsychological tasks: a dynamic task that require problem solving in a world that continually evolves over time and a logical task representative of cognitive processes performed by operators facing everyday problem solving. Results showed that the mean pupil diameter change was higher during the dynamic task; conversely, the heart rate was more elevated during the logical task. Finally, the systolic blood pressure seemed to be strongly sensitive to psychological stress. A better taking into account of the precise influence of a given cognitive activity and both workload and related task-induced psychological stress during task performance is a promising way to better monitor operators in complex working situations to detect mental overload or pejorative stress factor of error.

Why a Comprehensive Understanding of Mental Workload through the Measurement of Neurovascular Coupling Is a Key Issue for Neuroergonomics

Raja Parasuraman, the father of Neuroergonomics (the crossroads of Ergonomics and Neuroscience, Figure 1) has opened the doors to new discoveries and techniques for advancing understanding of human behavior with the underlying brain mechanisms (Parasuraman, 1998). As of his death in 2015, a precise and objective definition of the concept of mental workload (MWL) had still not yet been formulated. In this opinion piece, we posit that MWL is associated through the measurement of neurovascular coupling (NVC); innovative neuroimaging methods is now capable of measuring such a phenomenon; all while highlighting Parasuramans many contributions to this field.

Mitigation of Conflicts with Automation: Use of Cognitive Countermeasures

Objective: The aim of this study was to empirically assess the efficacy of cognitive countermeasures based on the technique of information removal to enhance human operator attentional disengagement abilities when facing attentional tunneling. Background: Lessons learned from human factors studies suggest that conflict with automation leads to the degradation of operators’ performance by promoting excessive focusing on a single task to the detriment of the supervision of other critical parameters. Method: An experimental setup composed of a real unmanned ground vehicle and a ground station was developed to test the efficiency of the cognitive countermeasures. The scenario (with and without countermeasure) involved an authority conflict between the participants and the robot induced by a battery failure. The effects of the conflict and, in particular, the impact of cognitive countermeasures on the participants’ cognition and arousal were assessed through heart rate measurement and eye tracking techniques. Results: In the control group (i.e., no countermeasure), 8 out of 12 participants experienced attentional tunneling when facing the conflict, leading them to neglect the visual alarms displayed that would have helped them to understand the evolution of the tactical situation. Participants in the countermeasure group showed lower heart rates and enhanced attentional abilities, and 10 out of 11 participants made appropriate decisions. Conclusions: The use of cognitive countermeasures appeared to be an efficient means to mitigate excessive focus issues in the unmanned ground vehicle environment. Applications: The principle of cognitive countermeasures can be applied to a large domain of applications involving human operators interacting with critical systems.

Formal Detection of Attentional Tunneling in Human Operator–Automation Interactions

The allocation of visual attention is a key factor for the humans when operating complex systems under time pressure with multiple information sources. In some situations, attentional tunneling is likely to appear and leads to excessive focus and poor decision making. In this study, we propose a formal approach to detect the occurrence of such an attentional impairment that is based on machine learning techniques. An experiment was conducted to provoke attentional tunneling during which psycho-physiological and oculomotor data from 23 participants were collected. Data from 18 participants were used to train an adaptive neuro-fuzzy inference system (ANFIS). From a machine learning point of view, the classification performance of the trained ANFIS proved the validity of this approach. Furthermore, the resulting classification rules were consistent with the attentional tunneling literature. Finally, the classifier was robust to detect attentional tunneling when performing over test data from four participants.

Neural and psychophysiological correlates of human performance under stress and high mental workload.

In our anxiogenic and stressful world, the maintenance of an optimal cognitive performance is a constant challenge. It is particularly true in complex working environments (e.g. flight deck, air traffic control tower), where individuals have sometimes to cope with a high mental workload and stressful situations. Several models (i.e. processing efficiency theory, cognitive-energetical framework) have attempted to provide a conceptual basis on how human performance is modulated by high workload and stress/anxiety. These models predict that stress can reduce human cognitive efficiency, even in the absence of a visible impact on the task performance. Performance may be protected under stress thanks to compensatory effort, but only at the expense of a cognitive cost. Yet, the psychophysiological cost of this regulation remains unclear. We designed two experiments involving pupil diameter, cardiovascular and prefrontal oxygenation measurements. Participants performed the Toulouse N-back Task that intensively engaged both working memory and mental calculation processes under the threat (or not) of unpredictable aversive sounds. The results revealed that higher task difficulty (higher n level) degraded the performance and induced an increased tonic pupil diameter, heart rate and activity in the lateral prefrontal cortex, and a decreased phasic pupil response and heart rate variability. Importantly, the condition of stress did not impact the performance, but at the expense of a psychophysiological cost as demonstrated by lower phasic pupil response, and greater heart rate and prefrontal activity. Prefrontal cortex seems to be a central region for mitigating the influence of stress because it subserves crucial functions (e.g. inhibition, working memory) that can promote the engagement of coping strategies. Overall, findings confirmed the psychophysiological cost of both mental effort and stress. Stress likely triggered increased motivation and the recruitment of additional cognitive resources that minimize its aversive effects on task performance (effectiveness), but these compensatory efforts consumed resources that caused a loss of cognitive efficiency (ratio between performance effectiveness and mental effort).

P300 Event-Related Potential as an Indicator of Inattentional Deafness?

An analysis of airplane accidents reveals that pilots sometimes purely fail to react to critical auditory alerts. This inability of an auditory stimulus to reach consciousness has been coined under the term of inattentional deafness. Recent data from literature tends to show that tasks involving high cognitive load consume most of the attentional capacities, leaving little or none remaining for processing any unexpected information. In addition, there is a growing body of evidence for a shared attentional capacity between vision and hearing. In this context, the abundant information in modern cockpits is likely to produce inattentional deafness. We investigated this hypothesis by combining electroencephalographic (EEG) measurements with an ecological aviation task performed under contextual variation of the cognitive load (high or low), including an alarm detection task. Two different audio tones were played: standard tones and deviant tones. Participants were instructed to ignore standard tones and to report deviant tones using a response pad. More than 31% of the deviant tones were not detected in the high load condition. Analysis of the EEG measurements showed a drastic diminution of the auditory P300 amplitude concomitant with this behavioral effect, whereas the N100 component was not affected. We suggest that these behavioral and electrophysiological results provide new insights on explaining the trend of pilots’ failure to react to critical auditory information. Relevant applications concern prevention of alarms omission, mental workload measurements and enhanced warning designs.

Reward and Uncertainty Favor Risky Decision-Making in Pilots: Evidence from Cardiovascular and Oculometric Measurements

In this paper we examined plan continuation error (PCE), a well known error made by pilots consisting in continuing the flight plan despite adverse meteorological conditions. Our hypothesis is that a large range of strong negative emotional consequences, including those induced by economic pressure, are associated with the decision to revise the flight plan and favor PCE. We investigated the economic hypothesis with a simplified landing task (reproduction of a real aircraft instrument) in which uncertainty and reward were manipulated. Heart rate (HR), heart rate variability (HRV) and eye tracking measurements were performed to get objective clues both on the cognitive and emotional state of the volunteers. Results showed that volunteers made more risky decisions under the influence of the financial incentive, in particular when uncertainty was high. Psychophysiological examination showed that HR increased and total HRV decreased in response to the cognitive load generated by the task. In addition, HR also increased in response to the financially motivated condition. Eventually, fixation times increased when uncertainty was high, confirming the difficulty in obtaining/interpreting information from the instrument in this condition. These results support the assumption that risky-decision making observed in pilots can be, at least partially, explained by a shift from cold to hot (emotional) decision-making in response to economic constraints and uncertainty.

Frequency analysis of a task-evoked pupillary response: Luminance-independent measure of mental effort

Pupil diameter is a widely-studied cognitive load measure, which, despite its convenience for non-intrusive operator state monitoring in complex environments, is still not available for in situ measurements because of numerous methodological limitations. The most important of these limitations is the influence of pupillary light reflex. Hence, there is the need of providing a pupil-based cognitive load measure that is independent of light conditions. In this paper, we present a promising technique of pupillary signal analysis resulting in luminance-independent measure of mental effort that could be used in real-time without a priori on luminous conditions. Twenty-two participants performed a short-term memory task under different screen luminance conditions. Our results showed that the amplitude of pupillary dilation due to load on memory was luminance-dependent with higher amplitude corresponding to lower-luminance condition. Furthermore, our experimentation showed that load on memory and luminance factors express themselves differently according to frequency. Therefore, as our statistical analysis revealed, the ratio between low (0-1.6 Hz) and high frequency (1.6-4 Hz) bands (LF/HF ratio) of power spectral densities of pupillary signal is sensitive to the cognitive load but not to luminance. Our results are promising for the measurement of load on memory in ecological settings.

Effects of the audiovisual conflict on auditory early processes.

Auditory alarm misperception is one of the critical events that lead aircraft pilots to an erroneous flying decision. The rarity of these alarms associated with their possible unreliability may play a role in this misperception. In order to investigate this hypothesis, we manipulated both audiovisual conflict and sound rarity in a simplified landing task. Behavioral data and event related potentials (ERPs) of thirteen healthy participants were analyzed. We found that the presentation of a rare auditory signal (i.e., an alarm), incongruent with visual information, led to a smaller amplitude of the auditory N100 (i.e., less negative) compared to the condition in which both signals were congruent. Moreover, the incongruity between the visual information and the rare sound did not significantly affect reaction times, suggesting that the rare sound was neglected. We propose that the lower N100 amplitude reflects an early visual-to-auditory gating that depends on the rarity of the sound. In complex aircraft environments, this early effect might be partly responsible for auditory alarm insensitivity. Our results provide a new basis for future aeronautic studies and the development of countermeasures.