Santosh Mathan

Trial-by-Trial Variations in Subjective Attentional State are Reflected in Ongoing Prestimulus EEG Alpha Oscillations

Parieto-occipital electroencephalogram (EEG) alpha power and subjective reports of attentional state are both associated with visual attention and awareness, but little is currently known about the relationship between these two measures. Here, we bring together these two literatures to explore the relationship between alpha activity and participants’ introspective judgments of attentional state as each varied from trial-to-trial during performance of a visual detection task. We collected participants’ subjective ratings of perceptual decision confidence and attentional state on continuous scales on each trial of a rapid serial visual presentation detection task while recording EEG. We found that confidence and attentional state ratings were largely uncorrelated with each other, but both were strongly associated with task performance and post-stimulus decision-related EEG activity. Crucially, attentional state ratings were also negatively associated with prestimulus EEG alpha power. Attesting to the robustness of this association, we were able to classify attentional state ratings via prestimulus alpha power on a single-trial basis. Moreover, when we repeated these analyses after smoothing the time series of attentional state ratings and alpha power with increasingly large sliding windows, both the correlations and classification performance improved considerably, with the peaks occurring at a sliding window size of approximately 7 min worth of trials. Our results therefore suggest that slow fluctuations in attentional state in the order of minutes are reflected in spontaneous alpha power. Since these subjective attentional state ratings were associated with objective measures of both behavior and neural activity, we suggest that they provide a simple and effective estimate of task engagement that could prove useful in operational settings that require human operators to maintain a sustained focus of visual attention.

Salient EEG Channel Selection in Brain Computer Interfaces by Mutual Information Maximization

Modern brain computer interface (BCI) applications use information obtained from the users electroencephalogram (EEG) to estimate the mental states. Selecting an optimal subset of the EEG channels instead of using all of them is especially important for ambulatory EEG where the user is mobile due to reduced data communication and computational load requirements. In addition, elimination of irrelevant sensors improves the robustness of the classification system by reducing dimensionality. In this paper, we propose a filter approach for EEG channel selection using mutual information (MI) maximization. This method ranks the EEG channels, such that the MI between the selected sensors and class labels is maximized. This selection criterion is known to reduce classification error. We employ a computationally efficient approach for MI estimation and EEG channel ranking. This approach is illustrated on EEG data recorded from three subjects performing two mental tasks. Experiment results show that the proposed approach works well and the position of the selected channels using the proposed method is consistent with the expected cortical areas for the mental tasks

Evaluating and improving adaptive educational systems with learning curves

Personalised environments such as adaptive educational systems can be evaluated and compared using performance curves. Such summative studies are useful for determining whether or not new modifications enhance or degrade performance. Performance curves also have the potential to be utilised in formative studies that can shape adaptive model design at a much finer level of granularity. We describe the use of learning curves for evaluating personalised educational systems and outline some of the potential pitfalls and how they may be overcome. We then describe three studies in which we demonstrate how learning curves can be used to drive changes in the user model. First, we show how using learning curves for subsets of the domain model can yield insight into the appropriateness of the model’s structure. In the second study we use this method to experiment with model granularity. Finally, we use learning curves to analyse a large volume of user data to explore the feasibility of using them as a reliable method for fine-tuning a system’s model. The results of these experiments demonstrate the successful use of performance curves in formative studies of adaptive educational systems.

A framework for rapid visual image search using single-trial brain evoked responses

We report the design and performance of a brain computer interface for single-trial detection of viewed images based on human dynamic brain response signatures in 32-channel electroencephalography (EEG) acquired during a rapid serial visual presentation. The system explores the feasibility of speeding up image analysis by tapping into split-second perceptual judgments of humans. We present an incremental learning system with less memory storage and computational cost for single-trial event-related potential (ERP) detection, which is trained using cross-session data. We demonstrate the efficacy of the method on the task of target image detection. We apply linear and nonlinear support vector machines (SVMs) and a linear logistic classifier (LLC) for single-trial ERP detection using data collected from image analysts and naive subjects. For our data the detection performance of the nonlinear SVM is better than the linear SVM and the LLC. We also show that our ERP-based target detection system is five-fold faster than the traditional image viewing paradigm.

Channel selection and feature projection for cognitive load estimation using ambulatory EEG

We present an ambulatory cognitive state classification system to assess the subjects mental load based on EEG measurements. The ambulatory cognitive state estimator is utilized in the context of a real-time augmented cognition (AugCog) system that aims to enhance the cognitive performance of a human user through computer-mediated assistance based on assessments of cognitive states using physiological signals including, but not limited to, EEG. This paper focuses particularly on the offline channel selection and feature projection phases of the design and aims to present mutual-information-based techniques that use a simple sample estimator for this quantity. Analyses conducted on data collected from 3 subjects performing 2 tasks (n-back/Larson) at 2 difficulty levels (low/high) demonstrate that the proposed mutual-information-based dimensionality reduction scheme can achieve up to 94% cognitive load estimation accuracy.

EEG indices of reward motivation and target detectability in a rapid visual detection task

A large corpus of data has demonstrated the sensitivity of behavioral and neural measures to variation in the availability of reward. The present study aimed to extend this work by exploring reward motivation in an RSVP task using complex satellite imagery. We found that reward motivation significantly influenced neural activity both in the preparatory period and in response to target images. Pre-stimulus alpha activity and, to a lesser degree, P3 and CNV amplitude were found to be significantly predictive of reward condition on single trials. Target-locked P3 amplitude was modulated both by reward condition and by variation in target detectability inherent to our task. We further quantified this exogenous influence, showing that P3 differences reflected single-trial variation in P3 amplitude for different targets. These findings provide theoretical insight into the neural indices of reward in an RSVP task, and have important applications in the field of satellite imagery analysis.

Considering Etiquette in the Design of an Adaptive System

In this article, the authors empirically assess the costs and benefits of designing an adaptive system to follow social conventions regarding the appropriateness of interruptions. Interruption management is one area within the larger topic of automation etiquette. The authors tested these concepts in an outdoor environment using the Communications Scheduler, a wearable adaptive system that classifies users’ cognitive state via brain and heart sensors and adapts its interactions. Designed to help dismounted soldiers, it manages communications in much the same way as a good administrative assistant. Depending on a combination of message priority, user workload, and system state, it decides whether to interrupt the user’s current tasks. The system supports decision makers in two innovative ways: It reliably measures a mobile user’s cognitive workload to adapt its behavior, and it implements rules of etiquette adapted from human-human interactions to improve human-computer interactions. Results indicate costs and benefits to both interrupting and refraining from interrupting. When users were overloaded, primary task performance was improved by managing interruptions. However, overall situation awareness on secondary tasks suffered. This work empirically quantifies costs and benefits of “appropriate” interruption behaviors, demonstrating the value of designing adaptive agents that follow social conventions for interactions with humans.

Rapid image analysis using neural signals

The problem of extracting information from large collections of imagery is a challenge with few good solutions. Computers typically cannot interpret imagery as effectively as humans can, and manual analysis tools are slow. The research reported here explores the feasibility of speeding up manual image analysis by tapping into split second perceptual judgments using electroencephalograph sensors. Experimental results show that a combination of neurophysiological signals and overt physical responses--detected while a user views imagery in high speed bursts of approximately 10 images per second--provide a basis for detecting targets within large image sets. Results show an approximately six-fold, statistically significant, reduction in the time required to detect targets at high accuracy levels compared to conventional broad-area image analysis.

Large-scale image database triage via EEG evoked responses

This paper describes an approach for target image search using human brain signals generated by perceptual processes in the brain. The human brain generates event related potentials (ERPs) in response to critical events, such as interesting/novel visual stimuli in the form of a target image. In this paper, we describe experiments involving six professional image analysts and summarize the ERP detection performance as they search for targets within a large image database. We develop a disjoint windowing scheme for data preprocessing to discard irrelevant and redundant information from the raw data to get clean training data. We apply support vector machines to detect ERPs and conduct 10-fold cross validation for parameter regularization. The results demonstrate that the ERP pattern recognition can provide reliable inference for image triage.

Cognitive State Estimation Based on EEG for Augmented Cognition

Augmented cognition is an emerging concept that aims to enhance user performance and cognitive capabilities on the basis of adaptive assistance. An integral part of such systems is the automatic assessment of the instantaneous cognitive state of the user. This paper describes an automatic cognitive state estimation methodology based on the use of EEG measurements with ambulatory users. The required robustness in this context is achieved through the use of a mutual information based dimensionality reduction approach in conjunction with a committee of classifiers, and median filter outlier rejection element. We present classification results associated with cognitive tasks performed in mobile and stationary modalities