Adam D. Gerson

Cortically coupled computer vision for rapid image search

We describe a real-time electroencephalography (EEG)-based brain-computer interface system for triaging imagery presented using rapid serial visual presentation. A target image in a sequence of nontarget distractor images elicits in the EEG a stereotypical spatiotemporal response, which can be detected. A pattern classifier uses this response to reprioritize the image sequence, placing detected targets in the front of an image stack. We use single-trial analysis based on linear discrimination to recover spatial components that reflect differences in EEG activity evoked by target versus nontarget images. We find an optimal set of spatial weights for 59 EEG sensors within a sliding 50-ms time window. Using this simple classifier allows us to process EEG in real time. The detection accuracy across five subjects is on average 92%, i.e., in a sequence of 2500 images, resorting images based on detector output results in 92% of target images being moved from a random position in the sequence to one of the first 250 images (first 10% of the sequence). The approach leverages the highly robust and invariant object recognition capabilities of the human visual system, using single-trial EEG analysis to efficiently detect neural signatures correlated with the recognition event.

Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring

We describe a brain-computer interface (BCI) system, which uses a set of adaptive linear preprocessing and classification algorithms for single-trial detection of error related negativity (ERN). We use the detected ERN as an estimate of a subjects perceived error during an alternative forced choice visual discrimination task. The detected ERN is used to correct subject errors. Our initial results show average improvement in subject performance of 21% when errors are automatically corrected via the BCI. We are currently investigating the generalization of the overall approach to other tasks and stimulus paradigms.

Cortical Origins of Response Time Variability During Rapid Discrimination of Visual Objects

In this paper, we use single-trial analysis of electroencephalography (EEG) to ascertain the cortical origins of response time variability in a rapid serial visual presentation (RSVP) task. We extract spatial components that maximally discriminate between target and distractor stimulus conditions over specific time windows between stimulus onset and the time of a motor response. We then compute the peak latency of this differential activity on a trial-by-trial basis, and correlate this with response time. We find, for our nine participants, that the majority of the latency is introduced by component activity which begins far-frontally 200 ms prior to the response and proceeds to become parietally distributed near the time of response. This activity is consistent with the hypothesis that cortical networks involved in generating the late positive complexes may be the origins of the observed response time variability in rapid discrimination of visual objects.

A data analysis competition to evaluate machine learning algorithms for use in brain-computer interfaces

We present three datasets that were used to conduct an open competition for evaluating the performance of various machine-learning algorithms used in brain-computer interfaces. The datasets were collected for tasks that included: 1) detecting explicit left/right (L/R) button press; 2) predicting imagined L/R button press; and 3) vertical cursor control. A total of ten entries were submitted to the competition, with winning results reported for two of the three datasets.

High-throughput image search via single-trial event detection in a rapid serial visual presentation task

We describe a method, using linear discrimination, for detecting single-trial EEG signatures of object recognition events in a rapid serial visual presentation (RSVP) task. We record EEG using a high spatial density array (87 electrodes) during the rapid presentation (50-200 msec per image) of natural images. Subjects were instructed to release a button when they recognized a target image (an image with a person/people). Trials consisted of 100 images each, with a 50% chance of a single target being in a trial. Subject EEG was analyzed on a single-trial basis with an optimal spatial linear discriminator learned at multiple time windows after the presentation of an image. Linear discrimination enables the estimation of a forward model and thus allows for an approximate localization of the discriminating activity. Results show multiple loci for discriminating activity (e.g. motor and visual). Using these detected EEG signatures, we show that in many cases we can detect targets more accurately than the overt response (button release) and that such signatures can be used to prioritize images for high-throughput search.

A System for Single-trial Analysis of Simultaneously Acquired EEG and fMRI

In this paper we describe a system for simultaneously acquiring EEG and fMRI and evaluate it in terms of discriminating, single-trial, task-related neural components in the EEG. Using an auditory oddball stimulus paradigm, we acquire EEG data both inside and outside a 1.5T MR scanner and compare both power spectra and single-trial discrimination performance for both conditions. We find that EEG activity acquired inside the MR scanner during echo planer image acquisition is of high enough quality to enable single-trial discrimination performance that is 95 % of that acquired outside the scanner. We conclude that EEG acquired simultaneously with fMRI is of high enough fidelity to permit single-trial analysis.

Spatial signatures of visual object recognition events learned from single-trial analysis of EEG

In this paper we use linear discrimination for learning EEG signatures of object recognition events in a rapid serial visual presentation (RSVP) task. We record EEG using a high spatial density array (63 electrodes) during the rapid presentation (50-200 msec per image) of natural images. Each trial consists of 100 images, with a 50% chance of a single target being in a trial. Subjects are instructed to press a left mouse button at the end of the trial if they detected a target image, otherwise they are instructed to press the right button. Subject EEG was analyzed on a single-trial basis with an optimal spatial linear discriminator learned at multiple time windows after the presentation of an image. Analysis of discrimination results indicated a periodic fluctuation (time-localized oscillation) in A/sub z/ performance. Analysis of the EEG using the discrimination components learned at the peaks of the A/sub z/ fluctuations indicate 1) the presence of a positive evoked response, followed in time by a negative evoked response in strongly overlapping areas and 2) a component which is not correlated with the discriminator learned during the time-localized fluctuation. Results suggest that multiple signatures, varying over time, may exist for discriminating between target and distractor trials.

Comparison of supervised and unsupervised linear methods for recovering task-relevant activity in EEG

In this paper we compare three linear methods, independent component analysis (ICA), common spatial patterns (CSP), and linear discrimination (LD) for recovering task relevant neural activity from high spatial density electroencephalography (EEG). Each linear method uses a different objective function to recover underlying source components by exploiting statistical structure across a large number of sensors. We test these methods using a dual-task event-related paradigm. While engaged in a primary task, subjects must detect infrequent changes in the visual display, which would be expected to evoke several well-known event-related potentials (ERPs), including the N2 and P3. We find that though each method utilizes a different objective function, they in fact yield similar components. We note that one advantage of the LD approach is that the recovered component is easily interpretable, namely it represents the component within a given time window which is most discriminating for the task, given a spatial integration of the sensors. Both ICA and CSP return multiple components, of which the most discriminating component may not be the first. Thus, for these methods, visual inspection or additional processing is required to determine the significance of these components for the task.