Ben Willems

Optical brain monitoring for operator training and mental workload assessment

An accurate measure of mental workload in human operators is a critical element of monitoring and adaptive aiding systems that are designed to improve the efficiency and safety of human-machine systems during critical tasks. Functional near infrared (fNIR) spectroscopy is a field-deployable non-invasive optical brain monitoring technology that provides a measure of cerebral hemodynamics within the prefrontal cortex in response to sensory, motor, or cognitive activation. In this paper, we provide evidence from two studies that fNIR can be used in ecologically valid environments to assess the: 1) mental workload of operators performing standardized (n-back) and complex cognitive tasks (air traffic control--ATC), and 2) development of expertise during practice of complex cognitive and visuomotor tasks (piloting unmanned air vehicles--UAV). Results indicate that fNIR measures are sensitive to mental task load and practice level, and provide evidence of the fNIR deployment in the field for its ability to monitor hemodynamic changes that are associated with relative cognitive workload changes of operators. The methods reported here provide guidance for the development of strategic requirements necessary for the design of complex human-machine interface systems and assist with assessments of human operator performance criteria.

Cognitive Workload and Learning Assessment During the Implementation of a Next-Generation Air Traffic Control Technology Using Functional Near-Infrared Spectroscopy

Neuroimaging technologies, such as functional near-infrared spectroscopy (fNIR), could provide performance metrics directly from brain-based measures to assess safety and performance of operators in high-risk fields. In this paper, we objectively and subjectively examine the cognitive workload of air traffic control specialists utilizing a next-generation conflict resolution advisory. Credible differences were observed between continuously increasing workload levels that were induced by increasing the number of aircraft under control. In higher aircraft counts, a possible saturation in brain activity was realized in the fNIR data. A learning effect was also analyzed across a three-day/nine-session training period. The difference between Day 1 and Day 2 was credible, while there was a noncredible difference between Day 2 and Day 3. The results presented in this paper indicate some advantages in objective measures of cognitive workload assessment with fNIR cortical imaging over the subjective workload assessment keypad.

Using Brain Activity to Predict Task Performance and Operator Efficiency

The efficiency and safety of many complex human-machine systems are closely related to the cognitive workload and situational awareness of their human operators. In this study, we utilized functional near infrared (fNIR) spectroscopy to monitor anterior prefrontal cortex activation of experienced operators during a standard working memory and attention task, the n-back. Results indicated that task efficiency can be estimated using operator’s fNIR and behavioral measures together. Moreover, fNIR measures had more predictive power than behavioral measures for estimating operator’s future task performance in higher difficulty conditions.

Human Performance Assessment Study in Aviation Using Functional Near Infrared Spectroscopy

Functional near infrared (fNIR) spectroscopy is a field-deployable optical neuroimaging technology that provides a measure of the prefrontal cortex’s cerebral hemodynamics in response to the completion of sensory, motor, or cognitive tasks. Technologies such as fNIR could provide additional performance metrics directly from brain-based measures to assess safety and performance of operators in high-risk fields. This paper reports a case study utilizing a continuous wave fNIR technology deployed in a real-time air traffic control (ATC) setting to evaluate the cognitive workload of certified professional controllers (CPCs) during the deployment of one of the Federal Aviation Administration’s (FAA’s) Next Generation (NextGen) technologies.

Human-in-the-Loop Simulation Experiment of Integrated Arrival/Departure Control Services For NextGen Operational Improvement

The National Airspace System suffers from a reduction in airport throughput or even closure when weather or traffic volume constrains arrival or departure gates to and from the Terminal Radar Approach Control (TRACON) facilities. To alleviate this, the Federal Aviation Administration introduced the Integrated Arrival and Departure Control Services (IADCS) concept that would extend terminal separations and procedures to the adjacent Air Route Traffic Control Center (ARTCC) sectors for more flexible traffic. It proposes the use of resectorization, bidirectional gates, bidirectional routes, and Air Traffic Control assigned routes. We evaluated them in the human-in-the-loop high-fidelity experiment and collected objective and subjective data. Our results clearly showed that all IADCS procedures except the vertically separated/bidirectional gate procedure were more effective than the Baseline condition that controllers currently use in the field. We conjecture the vertically separated/bidirectional gate procedure requires more complex perceptual and cognitive processes than the other procedures.

Is Functional Near Infrared Spectroscopy (fNIRS) Appropriate for your Research

Functional near infrared spectroscopy (fNIRS) is an emerging neuroimaging technique that has found home in various human factors and ergonomics applications. Why fNIRS? Is it better than EEG or fMRI? Is it an appropriate neuroimaging technique for my research/application? What are the methodological considerations for fNIRS analyses? This panel discussion is aimed at answering these questions, among others, when panelists from varied human factors and ergonomics applications discuss how they employ fNIRS in their investigations.

Human Factors Evaluation of Conflict Resolution Advisories for Air Traffic Control in the En Route Domain

In this paper, we report results of a human-in-the-loop simulation experiment that evaluated how Conflict Resolution Advisories (CRA) affected en route air traffic controllers’ performance. Twelve current en route Certified Professional Controllers from Air Route Traffic Control Centers (ARTCCs) participated in the experiment. Results showed that controllers used CRA menus significantly more often than Baseline menus. They also spent more time interacting with the CRA menus than with the Baseline menus. Most of the participants’ subjective ratings favored the CRA, but they also pointed out a few features to be improved.

Accuracy Assessment of Air Traffic Conflict Probe Prototype for Operational Evaluations

This study is part of the Separation Management research program, whose goals include improving the FAA’s operational Conflict Probe function. Conflict Probe alerts air traffic controllers to conflicts, or situations where aircraft will be too close to each other. The present study is one link in a chain of research efforts. We used the results of a meta-analysis of Human Factors literature on automation accuracy (Rein, Masalonis, Messina, & Willems, in press) in conjunction with FAA mathematical studies on the accuracy of the current Conflict Probe prototype (Crowell, Fabian, Young, Musialek, & Paglione, 2011; Crowell & Young, 2012) and determined the acceptability of the prototype’s conflict detection performance. The present results will feed upcoming operational research including a human-in-the loop (HITL) simulation in which the prototype will be used, by helping establish whether the prototype was “good enough” to improve joint human-automation system performance. In addition, the present analysis enhances the methodology for determining the accuracy of the operational Conflict Probe, although for this paper we did not evaluate or report on operational data. We obtained data from the aforementioned FAA mathematical analysis, which reported the prototype’s performance on some of the standard SDT metrics. We further analyzed their data to generate values on a wider set of accuracy metrics, and compared the results to the findings of Rein et al. regarding how accurate automation “should” be, as well as considering the results from an operational/face validity perspective. We focused on reliability, a measure of overall percent correct by the automation, which has been used in past multi-experiment analyses of automation accuracy (Wickens & Dixon, 2007), and which Rein et al. found to have a relationship to system performance. With a “best case” estimate of Conflict Probe reliability, its performance far exceeds that needed to improve system performance. However, the estimate may have been too liberal from an operational perspective, because the input data included many correct rejections where the proximity of the aircraft was well beyond the distances defining a conflict. In such cases, the automation’s failure to alert would have been technically correct, but not useful to the controller, who would know without any automated assistance that no conflict was present. We conclude that the current Conflict Probe prototype is suitable for conducting the HITL research, but that additional scenario evaluation research should be run to determine for what kinds of conflicts and near-conflicts the automation can complement rather than duplicate controller skill. This scenario evaluation research, and related follow-up mathematical analysis, will answer the question “how accurate is Conflict Probe?” The HITL will answer “how accurate does Conflict Probe need to be?” These answers will be evaluated in conjunction with each other to improve the operational Conflict Probe.

Evaluation of the impact of data communications on controller workload using the functional near infra red imaging technique

• FAA William J. Hughes Technical Center • NextGen • Data Communications • En Route Automation Modernization System • En Route Data Communications Experiment • Workload Assessment Keypad • Functional Near Infra Red Spectography • Workload Results • Discussion

Prototyping an En Route Data Communications human computer interface

• FAA William J. Hughes Technical Center • NextGen • Data Communications • En Route Automation Modernization System • En Route Data Communications Experiment • Prototyping Approach • Discussion Discussion • prototype to create a functional environment to evaluate requirements • design decisions are always a compromise of many alternatives • balance requirements of individual technologies, but integrate them at the user interface