Sirko Straube

pySPACE—a signal processing and classification environment in Python

In neuroscience large amounts of data are recorded to provide insights into cerebral information processing and function. The successful extraction of the relevant signals becomes more and more challenging due to increasing complexities in acquisition techniques and questions addressed. Here, automated signal processing and machine learning tools can help to process the data, e.g., to separate signal and noise. With the presented software pySPACE (http://pyspace.github.io/pyspace), signal processing algorithms can be compared and applied automatically on time series data, either with the aim of finding a suitable preprocessing, or of training supervised algorithms to classify the data. pySPACE originally has been built to process multi-sensor windowed time series data, like event-related potentials from the electroencephalogram (EEG). The software provides automated data handling, distributed processing, modular build-up of signal processing chains and tools for visualization and performance evaluation. Included in the software are various algorithms like temporal and spatial filters, feature generation and selection, classification algorithms, and evaluation schemes. Further, interfaces to other signal processing tools are provided and, since pySPACE is a modular framework, it can be extended with new algorithms according to individual needs. In the presented work, the structural hierarchies are described. It is illustrated how users and developers can interface the software and execute offline and online modes. Configuration of pySPACE is realized with the YAML format, so that programming skills are not mandatory for usage. The concept of pySPACE is to have one comprehensive tool that can be used to perform complete signal processing and classification tasks. It further allows to define own algorithms, or to integrate and use already existing libraries.

How to evaluate an agent's behavior to infrequent events?—Reliable performance estimation insensitive to class distribution

In everyday life, humans and animals often have to base decisions on infrequent relevant stimuli with respect to frequent irrelevant ones. When research in neuroscience mimics this situation, the effect of this imbalance in stimulus classes on performance evaluation has to be considered. This is most obvious for the often used overall accuracy, because the proportion of correct responses is governed by the more frequent class. This imbalance problem has been widely debated across disciplines and out of the discussed treatments this review focusses on performance estimation. For this, a more universal view is taken: an agent performing a classification task. Commonly used performance measures are characterized when used with imbalanced classes. Metrics like Accuracy, F-Measure, Matthews Correlation Coefficient, and Mutual Information are affected by imbalance, while other metrics do not have this drawback, like AUC, d-prime, Balanced Accuracy, Weighted Accuracy and G-Mean. It is pointed out that one is not restricted to this group of metrics, but the sensitivity to the class ratio has to be kept in mind for a proper choice. Selecting an appropriate metric is critical to avoid drawing misled conclusions.

On the Applicability of Brain Reading for Predictive Human-Machine Interfaces in Robotics

The ability of todays robots to autonomously support humans in their daily activities is still limited. To improve this, predictive human-machine interfaces (HMIs) can be applied to better support future interaction between human and machine. To infer upcoming context-based behavior relevant brain states of the human have to be detected. This is achieved by brain reading (BR), a passive approach for single trial EEG analysis that makes use of supervised machine learning (ML) methods. In this work we propose that BR is able to detect concrete states of the interacting human. To support this, we show that BR detects patterns in the electroencephalogram (EEG) that can be related to event-related activity in the EEG like the P300, which are indicators of concrete states or brain processes like target recognition processes. Further, we improve the robustness and applicability of BR in application-oriented scenarios by identifying and combining most relevant training data for single trial classification and by applying classifier transfer. We show that training and testing, i.e., application of the classifier, can be carried out on different classes, if the samples of both classes miss a relevant pattern. Classifier transfer is important for the usage of BR in application scenarios, where only small amounts of training examples are available. Finally, we demonstrate a dual BR application in an experimental setup that requires similar behavior as performed during the teleoperation of a robotic arm. Here, target recognition processes and movement preparation processes are detected simultaneously. In summary, our findings contribute to the development of robust and stable predictive HMIs that enable the simultaneous support of different interaction behaviors.

An Adaptive Spatial Filter for User-Independent Single Trial Detection of Event-Related Potentials

Goal: Current brain-computer interfaces (BCIs) are usually based on various, often supervised, signal processing methods. The disadvantage of supervised methods is the requirement to calibrate them with recently acquired subject-specific training data. Here, we present a novel algorithm for dimensionality reduction (spatial filter), that is ideally suited for single-trial detection of event-related potentials (ERPs) and can be adapted online to a new subject to minimize or avoid calibration time. Methods: The algorithm is based on the well-known xDAWN filter, but uses generalized eigendecomposition to allow an incremental training by recursive least squares (RLS) updates of the filter coefficients. We analyze the effectiveness of the spatial filter in different transfer scenarios and combinations with adaptive classifiers. Results: The results show that it can compensate changes due to switching between different users, and therefore allows to reuse training data that has been previously recorded from other subjects. Conclusions: The presented approach allows to reduce or completely avoid a calibration phase and to instantly use the BCI system with only a minor decrease of performance. Significance: The novel filter can adapt a precomputed spatial filter to a new subject and make a BCI system user independent.

Electrophysiological correlates of figure-ground segregation directly reflect perceptual saliency.

In a figure identification task, we investigated the influence of different visual cue configurations (spatial frequency, orientation or a combination of both) on the human EEG. Combining psychophysics with ERP and time-frequency analysis, we show that the neural response at about 200ms reflects perceptual saliency rather than physical cue contrast. Increasing saliency caused (i) a negative shift of the posterior P2 coinciding with a power decrease in the posterior theta-band and (ii) an amplitude and latency increase of the posterior P3. We demonstrate that visual cues interact for a percept that is non-linearly related to the physical figure-ground properties.

The electrophysiological correlate of saliency: evidence from a figure-detection task.

Although figure-ground segregation in a natural environment usually relies on multiple cues, we experience a coherent figure without usually noticing the individual single cues. It is still unclear how various cues interact to achieve this unified percept and whether this interaction depends on task demands. Studies investigating the effect of cue combination on the human EEG are still lacking. In the present study, we combined psychophysics, ERP and time-frequency analysis to investigate the interaction of orientation and spatial frequency as visual cues in a figure detection task. The figure was embedded in a matrix of Gabor elements, and we systematically varied figure saliency by changing the underlying cue configuration. We found a strong correlation between the posterior P2 amplitude and the perceived saliency of the figure: the P2 amplitude decreased with increasing saliency. Analogously, the power of the theta-band decreased for more salient figures. At longer latencies, the posterior P3 component was modulated in amplitude and latency, possibly reflecting increased decision confidence at higher saliencies. In conclusion, when the cue composition (e.g. one or two cues) or cue strength is changed in a figure detection task, first differences in the electrophysiological response reflect the perceived saliency and not directly the underlying cue configuration.

Human Force Discrimination during Active Arm Motion for Force Feedback Design

The goal of this study was to analyze the human ability of external force discrimination while actively moving the arm. With the approach presented here, we give an overview for the whole arm of the just-noticeable differences (JNDs) for controlled movements separately executed for the wrist, elbow, and shoulder joints. The work was originally motivated in the design phase of the actuation system of a wearable exoskeleton, which is used in a teleoperation scenario where force feedback should be provided to the subject. The amount of this force feedback has to be calibrated according to the human force discrimination abilities. In the experiments presented here, 10 subjects performed a series of movements facing an opposing force from a commercial haptic interface. Force changes had to be detected in a two-alternative forced choice task. For each of the three joints tested, perceptual thresholds were measured as absolute thresholds (no reference force) and three JNDs corresponding to three reference forces chosen. For this, we used the outcome of the QUEST procedure after 70 trials. Using these four measurements we computed the Weber fraction. Our results demonstrate that different Weber fractions can be measured with respect to the joint. These were 0.11, 0.13, and 0.08 for wrist, elbow, and shoulder, respectively. It is discussed that force perception may be affected by the number of muscles involved and the reproducibility of the movement itself. The minimum perceivable force, on average, was 0.04 N for all three joints.

Online movement prediction in a robotic application scenario

Current movement prediction systems based on electroencephalography were mainly developed and evaluated in highly controlled scenarios, in which subjects concentrate only on the desired task with as few as possible disturbing sources present. However, it has not been addressed sufficiently how the suggested methods perform in more complex and uncontrolled environments. In this work we predict arm movements online in a robotic teleoperation scenario and present a completely online running methodology. The system is evaluated on ten sessions from three subjects. Evaluation criteria are the overall classification performance and the success in predicting an upcoming movement in the application. Our results confirm that it is possible to predict movements in less restricted applications motivating the transfer of these methods to real world applications.

Balanced Relative Margin Machine - The missing piece between FDA and SVM classification

We suggest a new method called BRMM by modifying the existing RMM.The new method generalizes other well-known methods (SVM, RFDA, SVR).We prove and visualize the connections to the other methods.We suggest a sparse BRMM and prove an upper bound on the number of features. In this theoretical work we approach the class of relative margin classification algorithms from the mathematical programming perspective. In particular, we propose a Balanced Relative Margin Machine (BRMM) and then extend it by a 1-norm regularization. We show that this new classifier concept connects Support Vector Machines (SVM) with Fishers Discriminant Analysis (FDA) by the insertion of a range parameter. It is also strongly connected to the Support Vector Regression. Using this BRMM it is now possible to optimize the classifier type instead of choosing it beforehand. We verify our findings empirically by means of simulated and benchmark data.

Visual detection and identification are not the same: Evidence from psychophysics and fMRI

Sometimes object detection as opposed to identification is sufficient to initiate the appropriate action. To explore the neural origin of behavioural differences between the two tasks, we combine psychophysical measurements and fMRI, specifically contrasting shape detection versus identification of a figure. This figure consisted of Gabor elements being oriented differently from those in the background. We equalized performance levels for detection and identification by adjusting orientation differences accordingly for each observer. Hence, stimulus saliency was constant for both tasks allowing a differentiation between the activations specific for detection versus identification processes. Identification yielded higher psychophysical thresholds, slower reaction times and increased hemodynamic activations in the lateral-occipital complex (LOC) and an adjacent area in the collateral sulcus (CoS). Additional analysis using cortex-based alignment revealed four voxel-clusters differentially activated by the tasks, situated in the inferior parietal lobe, the precuneus, the anterior cingulum and the medial frontal gyrus. Our results indicate partly separated cortical mechanisms for object detection and identification.