Julian Abich

The Psychometrics of Mental Workload Multiple Measures Are Sensitive but Divergent

Objective: A study was run to test the sensitivity of multiple workload indices to the differing cognitive demands of four military monitoring task scenarios and to investigate relationships between indices. Background: Various psychophysiological indices of mental workload exhibit sensitivity to task factors. However, the psychometric properties of multiple indices, including the extent to which they intercorrelate, have not been adequately investigated. Method: One hundred fifty participants performed in four task scenarios based on a simulation of unmanned ground vehicle operation. Scenarios required threat detection and/or change detection. Both single- and dual-task scenarios were used. Workload metrics for each scenario were derived from the electroencephalogram (EEG), electrocardiogram, transcranial Doppler sonography, functional near infrared, and eye tracking. Subjective workload was also assessed. Results: Several metrics showed sensitivity to the differing demands of the four scenarios. Eye fixation duration and the Task Load Index metric derived from EEG were diagnostic of single-versus dual-task performance. Several other metrics differentiated the two single tasks but were less effective in differentiating single- from dual-task performance. Psychometric analyses confirmed the reliability of individual metrics but failed to identify any general workload factor. An analysis of difference scores between low- and high-workload conditions suggested an effort factor defined by heart rate variability and frontal cortex oxygenation. Conclusions: General workload is not well defined psychometrically, although various individual metrics may satisfy conventional criteria for workload assessment. Application: Practitioners should exercise caution in using multiple metrics that may not correspond well, especially at the level of the individual operator.

Establishing Workload Manipulations Utilizing a Simulated Environment

Research seeking to improve the measurement of workload requires the use of established task load manipulations to impose varying levels of demand on human operators. The present study sought to establish task load manipulations for research utilizing realistically complex task environments that elicit distinct levels of workload (i.e. low, medium, and high). A repeated measures design was used to test the effects of various demand manipulations on performance and subjective workload ratings using the NASA-Task Load Index (TLX) and Instantaneous Self-Assessment technique (ISA). This experiment successfully identified task demand manipulations that can be used to investigate operator workload within realistically complex environments. Results revealed that the event rate manipulations had the most consistent impact on performance and subjective workload ratings in both tasks, with each eliciting distinct levels of workload.

Field Assessment of Multimodal Communication for Dismounted Human-Robot Teams

A field assessment of multimodal communication (MMC) was conducted as part of a program integration demonstration to support and enable bi-directional communication between a dismounted Soldier and a robot teammate. In particular, the assessment was focused on utilizing auditory and visual/gesture based communications. The task involved commanding a robot using semantically-based MMC. Initial participant data indicates a positive experience with the multimodal interface (MMI) prototype. The results of the experiment inform recommendations for multimodal designers regarding perceived usability and functionality of the currently implemented MMI.

Psychophysiological Metrics for Workload are Demand-Sensitive but Multifactorial

Various psychophysiological indices of mental workload exhibit sensitivity to task demand factors, but the psychometrics of indices has been neglected. In particular, the extent to which different metrics converge on a common latent factor is unclear. In the present study, 150 participants performed in four task scenarios based on a simulation of unmanned vehicle operation. Scenarios required threat detection and/or change detection. Both single- and dual-task scenarios were used. Workload metrics were derived from the electroencephalogram (EEG), electrocardiogram (ECG), transcranial Doppler sonography (TCD), functional Near Infra-Red (fNIR) and eyetracking. Subjective workload was also assessed. Several metrics were appropriately sensitive to the differing levels of task load presented by the four scenarios. However, factor analysis identified multiple factors, each of which was associated with a single response system only, with no general factor. Caution should be used in assessing workload in the individual operator.

Individual Differences in UGV Operation: A Comparison of Subjective and Psychophysiological Predictors

Unmanned systems operations are complex, cognitively demanding tasks that elicit highly variable workload. The ability to predict performance and workload within these complex tasks can provide a powerful tool for practitioners regarding fit-for-duty verification. Further, monitoring workload aids in diagnostic assessment of factors that impact performance. The goal for this analysis was to examine the quality of cross-task averages of both baseline and concurrent psychophysiological and subjective measures to predict task performance and perceived workload. At a theoretical level, these findings suggest the need for a multivariate conceptualization of processing ‘resources’, encompassing both implicit and explicit responses. At a practical level, both subjective and psychophysiological measures may be necessary for optimizing performance prediction, at least for certain tasks.

Image quality assessment using the SSIM and the just noticeable difference paradigm

The structural similarity index (SSIM) has been shown to be a superior objective image quality metric. A web-based pilot experiment was conducted with the goal of quantifying, through the use of a sample of human participants, a trend in SSIM values showing when the human visual system can begin to perceive distortions applied to reference images. The just noticeable difference paradigm was used to determine the point at which at least 50% of participants were unable to discern between compressed and uncompressed grayscale images. For four images, this point was at an SSIM value of 96, while for two images it was at 92, for an average of 95. These results suggest that, despite the wide differences in the type of image used, the point at which a human observer cannot determine that compression has been used hovers around an SSIM value of 95.

Investigating Workload Measures for Adaptive Training Systems

Adaptive training systems have the potential to be tremendously beneficial for both trainees and trainers. The greatest challenge in creating adaptive training systems is the development of methods capable of reliably and unobtrusively monitoring the trainee’s cognitive state throughout the training process. Previous research suggests that eye tracking methods may be capable of supporting this requirement. The present study sought to evaluate the use of eye tracking methods across multiple tasks to identify potential limitations for its future implementation in adaptive training systems. The results suggest that, although eye tracking is capable of detecting fluctuations in operator workload related to the performance of tasks requiring focal vision, it is insensitive to workload fluctuations associated with tasks that can be performed using peripheral vision. This limitation must be considered by developers intending to use this technology within adaptive systems.

Workload Is Multidimensional, Not Unitary: What Now?

It is commonly assumed that workload is a unitary construct, but recent data suggest that there are multiple subjective and objective facets of workload that are only weakly intercorrelated. This article reviews the implications of treating workload as multivariate. Examples from several simulated task environments show that high subjective workload is compatible with a variety of patterns of multivariate psychophysiological response. Better understanding of the cognitive neuroscience of the different components of workload, including stress components, is required. At a practical level, neither subjective workload measures nor single physiological responses are adequate for evaluating task demands, building predictive models of human performance, and driving augmented cognition applications. Multivariate algorithms that accommodate the variability of cognitive and affective responses to demanding tasks are needed.

Impact of three task demand factors on simulated unmanned system intelligence, surveillance, and reconnaissance operations

Abstract The present study investigated how three task demand factors influenced performance, subjective workload and stress of novice intelligence, surveillance, and reconnaissance operators within a simulation of an unmanned ground vehicle. Manipulations were task type, dual-tasking and event rate. Participants were required to discriminate human targets within a street scene from a direct video feed (threat detection [TD] task) and detect changes in symbols presented in a map display (change detection [CD] task). Dual-tasking elevated workload and distress, and impaired performance for both tasks. However, with increasing event rate, CD task deteriorated, but TD improved. Thus, standard workload models provide a better guide to evaluating the demands of abstract symbols than to processing realistic human characters. Assessment of stress and workload may be especially important in the design and evaluation of systems in which human character critical signals must be detected in video images. Practitioner Summary: This experiment assessed subjective workload and stress during threat and CD tasks performed alone and in combination. Results indicated an increase in event rate led to significant improvements in performance during TD, but decrements during CD, yet both had associated increases in workload and engagement.

Technological evaluation of gesture and speech interfaces for enabling dismounted soldier-robot dialogue

With increasing necessity for intuitive Soldier-robot communication in military operations and advancements in interactive technologies, autonomous robots have transitioned from assistance tools to functional and operational teammates able to service an array of military operations. Despite improvements in gesture and speech recognition technologies, their effectiveness in supporting Soldier-robot communication is still uncertain. The purpose of the present study was to evaluate the performance of gesture and speech interface technologies to facilitate Soldier-robot communication during a spatial-navigation task with an autonomous robot. Gesture and speech semantically based spatial-navigation commands leveraged existing lexicons for visual and verbal communication from the U.S Army field manual for visual signaling and a previously established Squad Level Vocabulary (SLV). Speech commands were recorded by a Lapel microphone and Microsoft Kinect, and classified by commercial off-the-shelf automatic speech recognition (ASR) software. Visual signals were captured and classified using a custom wireless gesture glove and software. Participants in the experiment commanded a robot to complete a simulated ISR mission in a scaled down urban scenario by delivering a sequence of gesture and speech commands, both individually and simultaneously, to the robot. Performance and reliability of gesture and speech hardware interfaces and recognition tools were analyzed and reported. Analysis of experimental results demonstrated the employed gesture technology has significant potential for enabling bidirectional Soldier-robot team dialogue based on the high classification accuracy and minimal training required to perform gesture commands.