Yonghong Huang

A framework for rapid visual image search using single-trial brain evoked responses

We report the design and performance of a brain computer interface for single-trial detection of viewed images based on human dynamic brain response signatures in 32-channel electroencephalography (EEG) acquired during a rapid serial visual presentation. The system explores the feasibility of speeding up image analysis by tapping into split-second perceptual judgments of humans. We present an incremental learning system with less memory storage and computational cost for single-trial event-related potential (ERP) detection, which is trained using cross-session data. We demonstrate the efficacy of the method on the task of target image detection. We apply linear and nonlinear support vector machines (SVMs) and a linear logistic classifier (LLC) for single-trial ERP detection using data collected from image analysts and naive subjects. For our data the detection performance of the nonlinear SVM is better than the linear SVM and the LLC. We also show that our ERP-based target detection system is five-fold faster than the traditional image viewing paradigm.

The 9th annual MLSP competition: New methods for acoustic classification of multiple simultaneous bird species in a noisy environment

Birds have been widely used as biological indicators for ecological research. They respond quickly to environmental changes and can be used to infer about other organisms (e.g., insects they feed on). Traditional methods for collecting data about birds involves costly human effort. A promising alternative is acoustic monitoring. There are many advantages to recording audio of birds compared to human surveys, including increased temporal and spatial resolution and extent, applicability in remote sites, reduced observer bias, and potentially lower cost. However, it is an open problem for signal processing and machine learning to reliably identify bird sounds in real-world audio data collected in an acoustic monitoring scenario. Some of the major challenges include multiple simultaneously vocalizing birds, other sources of non-bird sound (e.g., buzzing insects), and background noise like wind, rain, and motor vehicles.

Rapid image analysis using neural signals

The problem of extracting information from large collections of imagery is a challenge with few good solutions. Computers typically cannot interpret imagery as effectively as humans can, and manual analysis tools are slow. The research reported here explores the feasibility of speeding up manual image analysis by tapping into split second perceptual judgments using electroencephalograph sensors. Experimental results show that a combination of neurophysiological signals and overt physical responses--detected while a user views imagery in high speed bursts of approximately 10 images per second--provide a basis for detecting targets within large image sets. Results show an approximately six-fold, statistically significant, reduction in the time required to detect targets at high accuracy levels compared to conventional broad-area image analysis.

Large-scale image database triage via EEG evoked responses

This paper describes an approach for target image search using human brain signals generated by perceptual processes in the brain. The human brain generates event related potentials (ERPs) in response to critical events, such as interesting/novel visual stimuli in the form of a target image. In this paper, we describe experiments involving six professional image analysts and summarize the ERP detection performance as they search for targets within a large image database. We develop a disjoint windowing scheme for data preprocessing to discard irrelevant and redundant information from the raw data to get clean training data. We apply support vector machines to detect ERPs and conduct 10-fold cross validation for parameter regularization. The results demonstrate that the ERP pattern recognition can provide reliable inference for image triage.

A reproducing kernel Hilbert space framework for pairwise time series distances

A good distance measure for time series needs to properly incorporate the temporal structure, and should be applicable to sequences with unequal lengths. In this paper, we propose a distance measure as a principled solution to the two requirements. Unlike the conventional feature vector representation, our approach represents each time series with a summarizing smooth curve in a reproducing kernel Hilbert space (RKHS), and therefore translate the distance between time series into distances between curves. Moreover we propose to learn the kernel of this RKHS from a population of time series with discrete observations using Gaussian process-based non-parametric mixed-effect models. Experiments on two vastly different real-world problems show that the proposed distance measure leads to improved classification accuracy over the conventional distance measures.

A Fusion Approach for Image Triage using Single Trial ERP Detection

This paper addresses the problem of conducting visual target search on a large set of images. We use electroencephalography to detect targets and apply a fusion approach combining neurophysiologic signals and overt physical responses to achieve high target detection accuracy. We conducted an experimental evaluation of the method using trained human experts to find target objects in broad area satellite images. Based on the fusion results, we applied spatial target likelihood maps to present the estimated target locations in the images. The results demonstrate the efficacy of the method on multiple subjects

Mixed effects models for EEG evoked response detection

Human brain signals associated with perceptual processes have been shown to be useful for visual target image search. For the purpose of online training, we develop a novel mixed effects evoked response detector, which is capable of combining individual random effects and population fixed effects, for the analysis of neural signatures associated with targets. To avoid numerical problems in high dimensional matrix computations, we develop equivalent dimension reduced expressions for the mixed models. We construct the mixed effects evoked response model using principal component analysis to provide bases for the population model and linear discriminant analysis (LDA) to provide bases for the individual models. In addition, the LDA is adopted for Elecroencephalography channel dimensionality reduction. Data collected at different time and experimental conditions from two subjects performing image search tasks are utilized to assess the quality of the models. We also compare the proposed model with the support vector machine (SVM). The results demonstrate that the mixed models approach the SVM and provide reliable inference on cross session evaluation for the single-trial evoked response detection.

Boosting linear logistic regression for single trial ERP detection in rapid serial visual presentation tasks.

In this paper, we employ the AdaBoost algorithm to the linear logistic regression model to detect encephalography (EEG) signatures, called evoked response potentials of visual recognition events in a single trial. In the experiments, a large amount of images were displayed at a very high presentation rate, named rapid serial visual presentation. The EEG was recorded using 32 electrodes during the rapid image presentation. Subjects were instructed to click the mouse when they recognize a target image. The results demonstrated that the boosting method improves the detection performance compared with the base classifier by approximately 3% as measured by area under the ROC curve

Estimating Mutual Information Using Gaussian Mixture Model for Feature Ranking and Selection

Feature selection is a critical step for pattern recognition and many other applications. Typically, feature selection strategies can be categorized into wrapper and filter approaches. Filter approach has attracted much attention because of its flexibility and computational efficiency. Previously, we have developed an ICA-MI framework for feature selection, in which the mutual information (MI) between features and class labels was used as the criterion. However, since this method depends on the linearity assumption, it is not applicable for an arbitrary distribution. In this paper, exploiting the fact that Gaussian mixture model (GMM) is generally a suitable tool for estimating probability densities, we propose GMM-MI method for feature ranking and selection. We will discuss the details of GMM-MI algorithm and demonstrate the experimental results. We will also compare the GMM-MI method with the ICA-MI method in terms of performance and computational efficiency.

A hybrid generative/discriminative method for EEG evoked potential detection

We propose a new method for the detection of evoked potentials that combines a generative model and a discriminative classifier. The method is a variant of the support vector machine (SVM), which uses the Fisher kernel. The kernel function is derived from a generative statistical model known as mixed effects model (MEM). Instead of arbitrarily selecting the Gaussian kernel for the SVM, we exploit the Fisher kernel derived from the MEM for the SVM. The strength of this approach is that it combines the rich information encoded in the generative model, the MEM, with the discriminative power of the SVM algorithm. Our results show that the new method of combining the two complementary approaches - the generative model (MEM) and the discriminative model (SVM) via the Fisher kernel - achieves substantial improvement over the generative model (MEM) and provides better performance than the discriminative model (Gaussian kernel SVM) on the detection of evoked potentials in neural signals.