Tony Johnson

Estimating direction in brain-behavior interactions: Proactive and reactive brain states in driving

Abstract Conventional neuroimaging analyses have ascribed function to particular brain regions, exploiting the power of the subtraction technique in fMRI and event‐related potential analyses in EEG. Moving beyond this convention, many researchers have begun exploring network‐based neurodynamics and coordination between brain regions as a function of behavioral parameters or environmental statistics; however, most approaches average evoked activity across the experimental session to study task‐dependent networks. Here, we examined on‐going oscillatory activity as measured with EEG and use a methodology to estimate directionality in brain‐behavior interactions. After source reconstruction, activity within specific frequency bands (delta: 2–3 Hz; theta: 4–7 Hz; alpha: 8–12 Hz; beta: 13–25 Hz) in a priori regions of interest was linked to continuous behavioral measurements, and we used a predictive filtering scheme to estimate the asymmetry between brain‐to‐behavior and behavior‐to‐brain prediction using a variant of Granger causality. We applied this approach to a simulated driving task and examined directed relationships between brain activity and continuous driving performance (steering behavior or vehicle heading error). Our results indicated that two neuro‐behavioral states may be explored with this methodology: a Proactive brain state that actively plans the response to the sensory information and is characterized by delta‐beta activity, and a Reactive brain state that processes incoming information and reacts to environmental statistics primarily within the alpha band. Graphical abstract Figure. No Caption available. HighlightsTraditional neuroscience studies investigate localized task‐evoked responsesOur approach examines continuous tracking of brain‐behavior interactions in oscillatory activityBrain leads behavior in a Proactive state, while brain follows behavior in a Reactive stateReactive states are largely carried by alpha activity in regions sensitive to environmental statisticsProactive states rely more on a diffuse delta‐beta network, particularly when linked with steering behavior

Classifying high-noise EEG in complex environments for brain-computer interaction technologies

Future technologies such as Brain-Computer Interaction Technologies (BCIT) or affective Brain Computer Interfaces (aBCI) will need to function in an environment with higher noise and complexity than seen in traditional laboratory settings, and while individuals perform concurrent tasks. In this paper, we describe preliminary results from an experiment in a complex virtual environment. For analysis, we classify between a subject hearing and reacting to an audio stimulus that is addressed to them, and the same subject hearing an irrelevant audio stimulus. We performed two offline classifications, one using BCILab [1], the other using LibSVM [2]. Distinct classifiers were trained for each individual in order to improve individual classifier performance [3]. The highest classification performance results were obtained using individual frequency bands as features and classifying with an SVM classifier with an RBF kernel, resulting in mean classification performance of 0.67, with individual classifier results ranging from 0.60 to 0.79.

A bag-of-words model for task-load prediction from EEG in complex environments

Neurotechnologies based on electroencephalography (EEG) and other physiological measures to improve task performance in complex environments will require tools and analysis methods that can account for increased environmental noise and task complexity compared to traditional neuroscience laboratory experiments. We propose a bag-of-words (BoW) model to address the difficulties associated with realistic applications in complex environments. In this paper, our proof-of-concept results show that a BoW classifier can discriminate two task-relevant states (high versus low task-load) while an individual performs a simulated security patrol mission with complex, concurrent tasking. Classifier performance is largely consistent across six simulation missions for a given participant, but performance decreases when trying to predict between two individuals. Overall, these initial results suggest that this BoW approach holds promise for detecting task-relevant states in real-world settings.

CTAGGER: Semi-structured community tagging for annotation and data-mining in event-rich contexts

Analysis of dynamic brain imaging data from EEG, MEG or fMRI requires a common temporal context to enable meta-analysis and data mining across experiments. However, there is no standardized method of annotating events, even from laboratory experiments in controlled settings, and the event-rich environments of real-world brain imaging present a still greater annotation challenge. We have developed a MATLAB toolbox called CTAGGER to enable a user-friendly, semi-structured and expandable strategy for event annotation in dynamic brain imaging and other time-series. To facilitate common labeling and comparison across data collections and laboratories, an individuals annotation can be collected in a common community database to encourage annotation reuse. The tools allow the creation of multiple event overlays to facilitate the reuse and combination of brain imaging data for multiple analyses.

Experimentation and evaluation of threat detection and local area awareness using advanced computational technologies in a simulated military environment

Tomorrows military systems will require novel methods for assessing Soldier performance and situational awareness (SA) in mobile operations involving mixed-initiative systems. Although new methods may augment Soldier assessments, they may also reduce Soldier performance as a function of demand on workload, requiring concurrent performance of mission and assessment tasks. The present paper describes a unique approach that supports assessment in environments approximating the operational context within which future systems will be deployed. A complex distributed system was required to emulate the operational environment. Separate computational and visualization systems provided an environment representative of the military operational context, including a 3D urban environment with dynamic human entities. Semi-autonomous driving was achieved with a simulated autonomous mobility system and SA was assessed through digital reports. A military crew station mounted on a 6-DOF motion simulator was used to create the physical environment. Cognitive state evaluation was enabled using physiological monitoring. Analyses indicated individual differences in temporal and accuracy components when identifying key features of potential threats; i.e., comparing Soldiers and insurgents with non-insurgent civilians. The assessment approach provided a natural, operationally-relevant means of assessing needs of future secure mobility systems and detecting key factors affecting Soldier-system performance as foci for future development.

Integration of an intelligent systems behavior simulator and a scalable soldier-machine interface

As the Armys Future Combat Systems (FCS) introduce emerging technologies and new force structures to the battlefield, soldiers will increasingly face new challenges in workload management. The next generation warfighter will be responsible for effectively managing robotic assets in addition to performing other missions. Studies of future battlefield operational scenarios involving the use of automation, including the specification of existing and proposed technologies, will provide significant insight into potential problem areas regarding soldier workload. The US Army Tank Automotive Research, Development, and Engineering Center (TARDEC) is currently executing an Army technology objective program to analyze and evaluate the effect of automated technologies and their associated control devices with respect to soldier workload. The Human-Robotic Interface (HRI) Intelligent Systems Behavior Simulator (ISBS) is a human performance measurement simulation system that allows modelers to develop constructive simulations of military scenarios with various deployments of interface technologies in order to evaluate operator effectiveness. One such interface is TARDECs Scalable Soldier-Machine Interface (SMI). The scalable SMI provides a configurable machine interface application that is capable of adapting to several hardware platforms by recognizing the physical space limitations of the display device. This paper describes the integration of the ISBS and Scalable SMI applications, which will ultimately benefit both systems. The ISBS will be able to use the Scalable SMI to visualize the behaviors of virtual soldiers performing HRI tasks, such as route planning, and the scalable SMI will benefit from stimuli provided by the ISBS simulation environment. The paper describes the background of each system and details of the system integration approach.

Translating Driving Research from Simulation to Interstate Driving with Realistic Traffic and Passenger Interactions

In this driving study, participants were assigned to a driver-passenger dyad and performed two drives along Interstate-95 in normal traffic conditions. During the driving session, the driver had to safely navigate the route while listening and discussing news stories that were relayed by the passenger. The driver then performed a set of memory tasks to evaluate how well they retained information from the discussion in a multitask context. We report preliminary analyses that examined subjective factors which may influence success in social communication, including trait and state similarity derived from questionnaires as well as physiological synchrony from implicit state measurements derived from brain activity data. Although this dataset is still in collection, these initial findings suggest potential metrics that capture the contextual complexity in naturalistic, multitask environments, providing a rich opportunity to study how successful communication reflects shared social and emotional experiences.