Bruno Berberian

Automation technology and sense of control: a window on human agency

Previous studies have shown that the perceived times of voluntary actions and their effects are perceived as shifted towards each other, so that the interval between action and outcome seems shortened. This has been referred to as ‘intentional binding’ (IB). However, the generality of this effect remains unclear. Here we demonstrate that Intentional Binding also occurs in complex control situations. Using an aircraft supervision task with different autopilot settings, our results first indicated a strong relation between measures of IB and different levels of system automation. Second, measures of IB were related to explicit agency judgement in this applied setting. We discuss the implications for the underlying mechanisms, and for sense of agency in automated environments.

The out-of-the-loop Brain: A neuroergonomic approach of the human automation interaction

Abstract The world surrounding us has become increasingly technological. Nowadays, the influence of automation is perceived in each aspect of everyday life. If automation makes some aspects of life easier, faster and safer, empirical data also suggests that it could have negative performance and safety consequences regarding human operators, a set of difficulties called the “out-of-the-loop” (OOTL) performance problem. However, after decades of research, this phenomenon remains difficult to grasp and counter. In this paper, we propose a neuroergonomics approach to treat this phenomenon. We first describe how automation impacts human operators. Then, we present the current knowledge relative to this OOTL phenomenon. Finally, we describe how recent insights in neurosciences can help characterize, quantify and compensate this phenomenon.

Data transmission latency and sense of control

Latency has been identified as a major bottleneck for usability of human-system interaction devices. However, the theoretical basis of the effect of latency on action control mechanisms remains weak. In this study, we aimed to investigate the cognitive implications of latency for Human-Computer Interaction. We proposed models of agency (i.e., mechanism underlying the feeling of control) as a possible interpretative framework on the nature of the transformation induced by latency. In a series of 3 experiments, we propose to tackle this problem by (1) characterizing the effects (performance and agency) of transmission delays on UAS camera control, and (2) designing and evaluating HMI solutions to mitigate these effects with regard to the agency principle. Our results showed that (1) latency decreases sense of agency and human performance, (2) models of agency could provide HMI solution for latency compensation. Interests of agentive experience accounts for better system design are discussed.

Action, observation et imagerie mentale d'une action : Peut-on apprendre implicitement dans tous les cas ?

This experiment is aimed to study if an implicit learning can take place when participants practice, observe or mentally imagine regular displacements of a target in a Serial Reaction Time task (SRT). The results indicate that participants who really practice the task, and participants who realized motor imagery and also participants who made observation, learned the sequence, contrary to the participants who only made visual imagery. Indeed, times on target of these participants decrease significantly when a new sequence is introduced, which is not the case for the participants of the visual imagery condition. However, the results obtained with the recognition test do not enable us to definitively conclude about the nature of the learning (implicit vs. explicit). Finally, this study highlights that the motor imagery condition gives performance similar to those obtained in practice and in observation conditions, which represents an interesting contribution in the field of the psychology of the sport and cognitive psychology.

Dynamics of visuo-spatial remembering: a study of information structuring in memory

We studied the process by which learning a pattern of motor activity reaches a steady-state characterized by a reduction in fluctuations. The stimuli consisted of eight visually presented dots that appeared sequentially. In a 20-trial learning phase, participants reproduced the positions of the eight dots after each presentation. Next, they reproduced the pattern 40 times without renewed presentation. In one condition, spatial distances between the dots were proportional to the intervals between their appearances; in the other they were not proportional. We analyzed how the reproduction stabilized at the configuration and dot levels. In proportional as well as non-proportional conditions, stabilization occurs at different time scales for the configuration and dot levels. The stabilization rate differed between proportional and non-proportional conditions. These results are discussed in the framework of dynamical systems.

My Brain Is Out of the Loop: A Neuroergonomic Approach of OOTL Phenomenon

The world surrounding us has become increasingly technological. Nowadays, the influence of automation is perceived in each aspect of everyday life and not only in the world of industry. Automation certainly makes some aspects of life easier, faster and safer. Nonetheless, empirical data suggests that traditional automation has many negative performance and safety consequences. Particularly, in cases of automatic equipment failure, human supervisors seemed effectively helpless to diagnose the situation, determine the appropriate solution and retake control, a set of difficulties called the “out-of-the-loop” (OOL) performance problem. Because automation is not powerful enough to handle all abnormalities, this difficulty in “takeover” is a central problem in automation design.

Toward a Model for Effective Human-Automation Interaction: The Mediated Agency

In our increasingly technological world, automation largely improved some aspects of our life. Nonetheless, automation can also have negative consequences. Indeed, operators seem often helpless to takeover an automated system in case of failure. This “out-of-the-loop” problem occurs when operator is unable to understand the intentions and to predict the outcome of actions of the system, causing a decrease of control. The following article illustrates how the psychological approach of agency can help (1) to better understand this OOTL performance problem and (2) to propose design principles to improve human machine interaction in case of system automation.

Evaluation of performance monitoring ERPs through difficulty manipulation in a response-feedback paradigm

Performance monitoring is an amply studied function, since it is of major importance in carrying out actions in our everyday life. No consensus has been reached on the functional role and the relationship between each event-related potential (ERP) characterizing this function. In this study, we used a modified version of the flanker task, measuring the impact of task difficulty on the amplitudes of response-locked and feedback-locked performance monitoring ERPs in a single trial. We observed a functional differentiation between fronto-central (ERN/CRN and FRN) and centro-parietal (Pe/Pc and P300) components: the former seem to be only sensitive to accuracy, whereas the latter seem to be mainly modulated by task difficulty. The use of a surface Laplacian transformation, estimating current source density, on our data also supported an effect of difficulty on centro-parietal response-locked and feedback-locked ERPs. This technique allowed the spatial resolution to be improved and provided clarity, associated with the difficulty manipulation, on the activity of response-locked and feedback-locked performance monitoring ERPs.

Out of the Loop, in Your Bubble: Mind Wandering Is Independent From Automation Reliability, but Influences Task Engagement

This study examined the influence of automation reliability on task-unrelated mind wandering (MW) frequency and the impact of MW on task engagement. Automated environment features make it particularly prone to increase MW frequency. Through mechanisms like complacency or agency, automating a task could increase MW frequency for the operator. For safety-critical industries, the lower perception and degraded stimuli processing associated with MW, summarized by the term “decoupling hypothesis,” are particularly concerning. Sixteen participants supervised an autopilot avoiding obstacles with two levels of reliability. Each condition lasted 45 min. We recorded thoughts as either pertaining to being focused, task-related MW or task-unrelated MW. We also recorded perceived mental demand, trust regarding the autopilot and oculometric measures. Based on questionnaire results, our protocol succeeded in inducing more mental demand and lower trust when the automation was unreliable. Attentional states were not correlated, nor did it influence trust in the system reliability. On the contrary, mental demand ratings and pupil diameter were lower during both task-related and task-unrelated MW, compared to those during the focus attentional state. This shows that perceptual decoupling also affects the engagement of operators in automated environments, which may dramatically lower their ability to supervise automation efficiently. This research informs human-automation designers to consider operator engagement when creating automated systems.

Agency modulates interactions with automation technologies

Abstract The increasing presence of automation between operators and automated systems tends to disrupt operators from action outcomes, leading them to leave the control loop. The theoretical framework of agency suggests that priming the operator about the system’s upcoming behaviour could help restore an appropriate sense of control and increase user acceptance of what the system is doing. In a series of two experiments, we test whether providing information about what the system is about to do next leads to an increase in the level of user acceptance, concomitant with an increase in control and performance. Using an aircraft supervision task, we demonstrated the benefit of prime messages regarding system acceptance and performance. Taken together, our results indicate that the principles proposed by this framework could be used to improve human–machine interaction and maintain a high level of sense of control in supervisory tasks. Practitioner Summary: The out-of-the-loop performance problem is a major potential consequence of automation, leaving operators helpless to takeover automation in case of failure. Using an aircraft supervision task, the following article illustrates how the psychological approach of agency can help improving human–system interactions by designing more acceptable and more controllable automated interfaces