Mark W. Scerbo

Evaluation of an adaptive automation system using three EEG indices with a visual tracking task

A system was evaluated for use in adaptive automation using two experiments with electroencephalogram (EEG) indices based on the beta, alpha, and theta bandwidths. Subjects performed a compensatory tracking task while their EEG was recorded and converted to one of three engagement indices: beta/(alpha + theta), beta/alpha, or 1/alpha. In experiment one, the tracking task was switched between manual and automatic modes depending on whether the subjects engagement index was increasing or decreasing under a positive or negative feedback condition. Subjects were run for three consecutive 16-min trials. In experiment two, the task was switched depending on whether the absolute level of the engagement index for the subject was above or below baseline levels. It was hypothesized that negative feedback would produce more switches between manual and automatic modes, and that the beta/(alpha + theta) index would be most effective. The results confirmed these hypotheses. Tracking performance was better under negative feedback in both experiments; also, the use of absolute levels of engagement in experiment two resulted in better performance. There were no systematic changes in these effects over three 16-min trials. The implications for the use of such systems for adaptive automation are discussed.

A Closed-Loop System for Examining Psychophysiological Measures for Adaptive Task Allocation

A closed-loop system was evaluated for its efficacy in using psychophysiological indexes to moderate workload. Participants were asked to perform either 1 or 3 tasks from the Multiattribute Task Battery and complete the NASA Task Load Index after each trial. An electroencephalogram (EEG) was sampled continuously while they performed the tasks, and an EEG index (beta/alpha plus theta) was derived. The system made allocation decisions as a function of the level of operator engagement based on the value of the EEG index. The results of the study demonstrated that it was possible to moderate an operators level of engagement through a closed-loop system driven by the operators own EEG. In addition, the system had a significant impact on behavioral, subjective, and psychophysiological correlates of workload as task load increased. The theoretical and practical implications of these results for adaptive automation are discussed.

Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool.

Introduction: Increased workload during task performance may increase fatigue and facilitate errors. The National Aeronautics and Space Administration-Task Load Index (NASA-TLX) is a previously validated tool for workload self-assessment. We assessed the relationship of workload and performance during simulator training on a complex laparoscopic task. Methods: NASA-TLX workload data from three separate trials were analyzed. All participants were novices (n = 28), followed the same curriculum on the fundamentals of laparoscopic surgery suturing model, and were tested in the animal operating room (OR) on a Nissen fundoplication model after training. Performance and workload scores were recorded at baseline, after proficiency achievement, and during the test. Performance, NASA-TLX scores, and inadvertent injuries during the test were analyzed and compared. Results: Workload scores declined during training and mirrored performance changes. NASA-TLX scores correlated significantly with performance scores (r = −0.5, P < 0.001). Participants with higher workload scores caused more inadvertent injuries to adjacent structures in the OR (r = 0.38, P < 0.05). Increased mental and physical workload scores at baseline correlated with higher workload scores in the OR (r = 0.52–0.82; P < 0.05) and more inadvertent injuries (r = 0.52, P < 0.01). Conclusions: Increased workload is associated with inferior task performance and higher likelihood of errors. The NASA-TLX questionnaire accurately reflects workload changes during simulator training and may identify individuals more likely to experience high workload and more prone to errors during skill transfer to the clinical environment.

Effects of a psychophysiological system for adaptive automation on performance, workload, and the event-related potential P300 component

The present study examined the effects of an electroencephalographic- (EEG-) based system for adaptive automation on tracking performance and workload. In addition, event-related potentials (ERPs) to a secondary task were derived to determine whether they would provide an additional degree of workload specificity. Participants were run in an adaptive automation condition, in which the system switched between manual and automatic task modes based on the value of each individuals own EEG engagement index; a yoked control condition; or another control group, in which task mode switches followed a random pattern. Adaptive automation improved performance and resulted in lower levels of workload. Further, the P300 component of the ERP paralleled the sensitivity to task demands of the performance and subjective measures across conditions. These results indicate that it is possible to improve performance with a psychophysiological adaptive automation system and that ERPs may provide an alternative means for distinguishing among levels of cognitive task demand in such systems. Actual or potential applications of this research include improved methods for assessing operator workload and performance.

Simulator training to automaticity leads to improved skill transfer compared with traditional proficiency-based training: a randomized controlled trial.

Objective:We hypothesized that novices will perform better in the operating room after simulator training to automaticity compared with traditional proficiency based training (current standard training paradigm). Background:Simulator-acquired skill translates to the operating room, but the skill transfer is incomplete. Secondary task metrics reflect the ability of trainees to multitask (automaticity) and may improve performance assessment on simulators and skill transfer by indicating when learning is complete. Methods:Novices (N = 30) were enrolled in an IRB-approved, blinded, randomized, controlled trial. Participants were randomized into an intervention (n = 20) and a control (n = 10) group. The intervention group practiced on the FLS suturing task until they achieved expert levels of time and errors (proficiency), were tested on a live porcine fundoplication model, continued simulator training until they achieved expert levels on a visual spatial secondary task (automaticity) and were retested on the operating room (OR) model. The control group participated only during testing sessions. Performance scores were compared within and between groups during testing sessions. Results:Intervention group participants achieved proficiency after 54 ± 14 and automaticity after additional 109 ± 57 repetitions. Participants achieved better scores in the OR after automaticity training [345 (range, 0–537)] compared with after proficiency-based training [220 (range, 0–452; P < 0.001]. Conclusions:Simulator training to automaticity takes more time but is superior to proficiency-based training, as it leads to improved skill acquisition and transfer. Secondary task metrics that reflect trainee automaticity should be implemented during simulator training to improve learning and skill transfer.

Automation-induced complacency for monitoring highly reliable systems: the role of task complexity, system experience, and operator trust

The increase in quantity and complexity of advanced automated systems has generated new concerns surrounding automation-induced complacency, or the difficulties operators have monitoring the status of automated systems. The present investigation consists of two studies that assessed the impact of system reliability, monitoring complexity, operator trust, and system experience on automation-induced complacency. In both studies, participants operated a manually controlled flight task while monitoring several simulated aircraft displays for failures. The ability of operators to detect a single automation failure over three experimental sessions was also assessed. Results indicated that realistic levels of system reliability severely impaired an operators ability to monitor effectively. Further, as system experience increased, operator monitoring performance declined. The results also indicated that the complexity of the monitoring task heavily influenced operator monitoring, with poorer performance associated with more cognitively demanding tasks. Finally, results from both studies indicated that operator trust increased and monitoring performance decreased as a function of increasing system reliability. These results suggest that for highly reliable systems, increasing task complexity and extensive experience may severely impair an operators ability to monitor for unanticipated system states.

Effects of Instruction Type and Boredom Proneness in Vigilance: Implications for Boredom and Workload

The present study examined the effects of instruction type and boredom proneness (BP) on vigilance performance, workload, and boredom. Subjects completed the Boredom Proneness Scale and were assigned to high and low groups based on their scores. They then monitored a VDT for critical signals. Half the subjects were instructed to detect “critical” flickers (detection emphasis), and the remaining subjects were instructed to relax but to respond to any flickers observed (relaxation emphasis). Subjects also provided pre- and postvigil ratings of workload, stress, and boredom. A performance decrement was observed for all conditions. Low-BP subjects outperformed high-BP subjects and reported less boredom. Thus the results from the present study provide evidence for the long-sought, elusive link between trait boredom and performance in vigilance. In addition, subjects who received relaxation-emphasis instructions reported lower workload, frustration, and stress for the vigil than did those receiving detection-emphasis instructions. These results are discussed in terms of a recent dynamic model of stress as it relates to sustained attention.

A brain-based system for adaptive automation

Adaptive automation refers to technology that can change its mode of operation dynamically. Further, both the technology and the operator can initiate changes in the level or mode of automation. One of the important issues surrounding this technology concerns the method for initiating changes in the state of automation. The present paper considers the potential of using brain activity to drive an adaptive automation system. Relevant research on EEG is presented followed by a review of several experiments in which EEG is used to trigger changes among system modes in an adaptive automation system. The system moderates operator task load based upon an index derived from a ratio of EEG power bands. The research shows that it may be feasible to build an adaptive automation system and use this index of brain activity to drive the system. The paper concludes with a discussion of several issues that still need to be addressed before this approach can move beyond the laboratory.

Comparison of a brain-based adaptive system and a manual adaptable system for invoking automation

Objective: Two experiments are presented examining adaptive and adaptable methods for invoking automation. Background: Empirical investigations of adaptive automation have focused on methods used to invoke automation or on automation-related performance implications. However, no research has addressed whether performance benefits associated with brain-based systems exceed those in which users have control over task allocations. Method: Participants performed monitoring and resource management tasks as well as a tracking task that shifted between automatic and manual modes. In the first experiment, participants worked with an adaptive system that used their electroencephalographic signals to switch the tracking task between automatic and manual modes. Participants were also divided between high-and low-reliability conditions for the system-monitoring task as well as high- and low-complacency potential. For the second experiment, participants operated an adaptable system that gave them manual control over task allocations. Results: Results indicated increased situation awareness (SA) of gauge instrument settings for individuals high in complacency potential using the adaptive system. In addition, participants who had control over automation performed more poorly on the resource management task and reported higher levels of workload. A comparison between systems also revealed enhanced SA of gauge instrument settings and decreased workload in the adaptive condition. Conclusion: The present results suggest that brain-based adaptive automation systems may enhance perceptual level SA while reducing mental workload relative to systems requiring user-initiated control. Application: Potential applications include automated systems for which operator monitoring performance and high-workload conditions are of concern.

Evaluation of a Psychophysiologically Controlled Adaptive Automation System, Using Performance on a Tracking Task

Three experiments were conducted to evaluate the performance of a psychophysiologically controlled adaptive automation system. Subjects were asked to perform a compensatory tracking task while their electroencephalogram (EEG) was recorded and an engagement index was derived from the EEG, using the alpha, beta, and theta bandwidths: β/(α + θ) and β/θ. In Experiment I, EEG was recorded from three different sites: frontal, parietal, and temporal. Although tracking performance did not differ as a function of site, the number of task mode allocations was greater under a negative feedback contingency than under a positive feedback contingency. This effect was seen primarily from frontal sites. Experiments II and III evaluated the adaptive automation system, using extended runs under positive and negative feedback with either a slope (Experiment II) or absolute (Experiment III) criterion used to drive the system. Using either criterion, performance was found to be significantly better under negative feedback. Future evaluation and use of psychophysiologically controlled adaptive automation systems are discussed.