Jason Omedes

Using frequency-domain features for the generalization of EEG error-related potentials among different tasks

EEG brain-computer interfaces (BCI) require a calibration phase prior to the on-line control of the device, which is a difficulty for the practical development of this technology as it is user-, session- and task-specific. The large body of research in BCIs based on event-related potentials (ERP) use temporal features, which have demonstrated to be stable for each user along time, but do not generalize well among tasks different from the calibration task. This paper explores the use of low frequency features to improve the generalization capabilities of the BCIs using error-potentials. The results show that there exists a stable pattern in the frequency domain that allows a classifier to generalize among the tasks. Furthermore, the study also shows that it is possible to combine temporal and frequency features to obtain the best of both domains.

Omnidirectional Vision for Indoor Spatial Layout Recovery

1 Abstract— In this work, we study the problem of recovering the spatial layout of a scene from a collection of lines extracted from a single indoor image. Equivalent methods for conventional cameras have been proposed in the literature, but not much work has been done about this topic using omnidirectional vision, particulary powerful to obtain the spatial layout due to its wide field of view. As the geometry of omnidirectional and conventional images is different, most of the proposed methods for standard cameras do not work and new algorithms with specific considerations are required. We first propose a new method for vanishing points (VPs) estimation and line classification for omnidirectional images. Our main contribution is a new approach for spatial layout recovery based on these extracted lines and vanishing points, combined with a set of geometrical constraints, which allow us to detect floor-wall boundaries regardless of the number of walls. In our proposal, we first make a 4 walls room hypothesis and subsequently we expand this room in order to find the best fitting. We demonstrate how we can find the floor-wall boundary of the interior of a building, even when this boundary is partially occluded by objects and show several examples of these interpretations.

Spatial layout recovery from a single omnidirectional image and its matching-free sequential propagation

The goal of this work is to recover the spatial layout of indoor environments from omnidirectional images assuming a Manhattan world structure. We propose a new method for scene structure recovery from a single image. This method is based on the line extraction for omnidirectional images, line classification, and vanishing points estimation combined with a new hierarchical expansion procedure for detecting floor and wall boundaries. Each single omnidirectional image independently provides a useful hypothesis of the 3D scene structure. In order to enhance the robustness and accuracy of this single image-based hypothesis, we extend this estimation with a new homography-based procedure applied to the various hypotheses obtained along the sequence of consecutive images. A key point in this contribution is the use of geometrical constraints for computing the homographies from a single line of the floor. The homography parametrization proposed allows the design of a matching-free method for spatial layout propagation along a sequence of images. Experimental results show single image layout recovery performance and the improvement obtained with the propagation of the hypothesis through the image sequence.

Exploiting Task Constraints for Self-Calibrated Brain-Machine Interface Control Using Error-Related Potentials

This paper presents a new approach for self-calibration BCI for reaching tasks using error-related potentials. The proposed method exploits task constraints to simultaneously calibrate the decoder and control the device, by using a robust likelihood function and an ad-hoc planner to cope with the large uncertainty resulting from the unknown task and decoder. The method has been evaluated in closed-loop online experiments with 8 users using a previously proposed BCI protocol for reaching tasks over a grid. The results show that it is possible to have a usable BCI control from the beginning of the experiment without any prior calibration. Furthermore, comparisons with simulations and previous results obtained using standard calibration hint that both the quality of recorded signals and the performance of the system were comparable to those obtained with a standard calibration approach.

Analysis and asynchronous detection of gradually unfolding errors during monitoring tasks

Human studies on cognitive control processes rely on tasks involving sudden-onset stimuli, which allow the analysis of these neural imprints to be time-locked and relative to the stimuli onset. Human perceptual decisions, however, comprise continuous processes where evidence accumulates until reaching a boundary. Surpassing the boundary leads to a decision where measured brain responses are associated to an internal, unknown onset. The lack of this onset for gradual stimuli hinders both the analyses of brain activity and the training of detectors. This paper studies electroencephalographic (EEG)-measurable signatures of human processing for sudden and gradual cognitive processes represented as a trajectory mismatch under a monitoring task. Time-locked potentials and brain-source analysis of the EEG of sudden mismatches revealed the typical components of event-related potentials and the involvement of brain structures related to cognitive control processing. For gradual mismatch events, time-locked analyses did not show any discernible EEG scalp pattern, despite related brain areas being, to a lesser extent, activated. However, and thanks to the use of non-linear pattern recognition algorithms, it is possible to train an asynchronous detector on sudden events and use it to detect gradual mismatches, as well as obtaining an estimate of their unknown onset. Post-hoc time-locked scalp and brain-source analyses revealed that the EEG patterns of detected gradual mismatches originated in brain areas related to cognitive control processing. This indicates that gradual events induce latency in the evaluation process but that similar brain mechanisms are present in sudden and gradual mismatch events. Furthermore, the proposed asynchronous detection model widens the scope of applications of brain-machine interfaces to other gradual processes.

Shared control of a robot using EEG-based feedback signals

In the last years there has been an increasing interest on using human feedback during robot operation to incorporate non-expert human expertise while learning complex tasks. Most work has considered reinforcement learning frameworks were human feedback, provided through multiple modalities (speech, graphical interfaces, gestures) is converted into a reward. This paper explores a different communication channel: cognitive EEG brain signals related to the perception of errors by humans. In particular, we consider error potentials (ErrP), voltage deflections appearing when a user perceives an error, either committed by herself or by an external machine, thus encoding binary information about how a robot is performing a task. Based on this potential, we propose an algorithm based on policy matching for inverse reinforcement learning to infer the user goal from brain signals. We present two cases of study involving a target reaching task in a grid world and using a real mobile robot, respectively. For discrete worlds, the results show that the robot is able to infer and reach the target using only error potentials as feedback elicited from human observation. Finally, promising preliminary results were obtained for continuous states and actions in real scenarios.

Asynchronous detection of error potentials

Recent developments in brain-machine interfaces (BMIs) have proposed the use of errorrelated potentials as cognitive signal that can provide feedback to control devices or to teach them how to solve a task. Due to the nature of these signals, all the proposed error-based BMIs use discrete tasks to classify a signal as correct or incorrect under the assumption that the response is time-locked to a known event. However, during the continuous operation of a robotic device, the occurrence of an error is not known a priori and thus it is required to be constantly classifying. Here, we present an experimental protocol that allows to train a decoder and detect errors in single trial using a sliding window.

Brain connectivity in continuous error tasks

Error-related potentials (ErrP) have been recently incorporated in brain-machine interfaces (BMIs) due to its ability to adapt and correct both the output of the BMI or the behavior of the machine. Most of these applications rely on synchronous tasks with different users evaluations associated to correct and wrong events. Asynchronous detection during the continuous evaluation of the task, however, has to cope with background noise and an increased number of misdetections common in event-related potential detection. This paper studies a different characteristic that may carry additional information to be exploited by asynchronous ErrP detectors: brain connectivity coherence patterns appearing while the user monitors the continuous operation of a device. The results obtained with five subject revealed the presence of an error potential in an asynchronous reaching task an showed an increase in the coherency within the theta band.

Factors that affect error potentials during a grasping task: toward a hybrid natural movement decoding BCI

OBJECTIVE In this manuscript, we consider factors that may affect the design of a hybrid brain-computer interface (BCI). We combine neural correlates of natural movements and interaction error-related potentials (ErrP) to perform a 3D reaching task, focusing on the impact that such factors have on the evoked ErrP signatures and in their classification. APPROACH Users attempted to control a 3D virtual interface that simulated their own hand, to reach and grasp two different objects. Three factors of interest were modulated during the experimentation: (1) execution speed of the grasping, (2) type of grasping and (3) mental strategy (motor imagery or real motion) used to produce motor commands. Thirteen healthy subjects carried out the protocol. The peaks and latencies of the ErrP were analyzed for the different factors as well as the classification performance. MAIN RESULTS ErrP are evoked for erroneous commands decoded from neural correlates of natural movements. The analysis of variance (ANOVA) analyses revealed that latency and magnitude of the most characteristic ErrP peaks were significantly influenced by the speed at which the grasping was executed, but not the type of grasp. This resulted in an greater accuracy of single-trial decoding of errors for fast movements (75.65%) compared to slow ones (68.99%). SIGNIFICANCE Understanding the effects of combining paradigms is a first step to design hybrid BCI that optimize decoding accuracy and can be deployed in motor substitution and neuro-rehabilitation applications.

Asynchronous Decoding of Error Potentials during the Monitoring of a Reaching Task

Brain-machine interfaces (BMIs) have demonstrated how they can be used for reaching tasks with both invasive and non-invasive signal recording methods. Despite the constant improvements in this field, there still exist diverse factors to overcome before achieving a natural control. In particular, the high variability of the brain signals often leads to the incorrect decoding of the subject intentions, producing unreliable behaviors in the controlled device. A possible solution to this problem would be that of correcting this erroneous decoding using a feedback signal from the user. In this work, we evaluate the possibility of decoding neural signals associated to performance monitoring (EEG-recorded error-related potentials) during a reaching task. Compared to previous works where these error potentials were recorded under scenarios with discrete movements performed by the cursor, under real conditions the cursor is moving continuously and thus the system is required to asynchronously detect any possible error. To this end, we simulated two different erroneous events during the monitoring of a reaching task: errors at the beginning of the movement, and errors happening in the middle of the trajectory being executed. Through the analysis of the recorded EEG of three subjects, we demonstrate the existence of neural correlates for the two types of elicited error potentials, and we are able to asynchronously detect them with high accuracies.