Kenneth E. Hild

Blind source separation using Renyi's mutual information

A blind source separation algorithm is proposed that is based on minimizing Renyis mutual information by means of nonparametric probability density function (PDF) estimation. The two-stage process consists of spatial whitening and a series of Givens rotations and produces a cost function consisting only of marginal entropies. This formulation avoids the problems of PDF inaccuracy due to truncation of series expansion and the estimation of joint PDFs in high-dimensional spaces given the typical paucity of data. Simulations illustrate the superior efficiency, in terms of data length, of the proposed method compared to fast independent component analysis (FastICA), Comons (1994) minimum mutual information, and Bell and Sejnowskis (1995) Infomax.

Feature extraction using information-theoretic learning

A classification system typically consists of both a feature extractor (preprocessor) and a classifier. These two components can be trained either independently or simultaneously. The former option has an implementation advantage since the extractor need only be trained once for use with any classifier, whereas the latter has an advantage since it can be used to minimize classification error directly. Certain criteria, such as minimum classification error, are better suited for simultaneous training, whereas other criteria, such as mutual information, are amenable for training the feature extractor either independently or simultaneously. Herein, an information-theoretic criterion is introduced and is evaluated for training the extractor independently of the classifier. The proposed method uses nonparametric estimation of Renyis entropy to train the extractor by maximizing an approximation of the mutual information between the class labels and the output of the feature extractor. The evaluations show that the proposed method, even though it uses independent training, performs at least as well as three feature extraction methods that train the extractor and classifier simultaneously

Online entropy manipulation: stochastic information gradient

Entropy has found significant applications in numerous signal processing problems including independent components analysis and blind deconvolution. In general, entropy estimators require O(N/sup 2/) operations, N being the number of samples. For practical online entropy manipulation, it is desirable to determine a stochastic gradient for entropy, which has O(N) complexity. In this paper, we propose a stochastic Shannons entropy estimator. We determine the corresponding stochastic gradient and investigate its performance. The proposed stochastic gradient for Shannons entropy can be used in online adaptation problems where the optimization of an entropy-based cost function is necessary.

Stochastic blind equalization based on PDF fitting using Parzen estimator

This work presents a new blind equalization approach that aims to force the probability density function (pdf) at the equalizer output to match the known constellation pdf. Quadratic distance between pdfs is used as the cost function to be minimized. The proposed method relies on the Parzen window method to estimate the data pdf and is implemented by a stochastic gradient descent algorithm. The kernel size of the Parzen estimator allows a dual mode switch or a soft switch between blind and decision-directed equalization. The proposed method converges faster than the constant modulus algorithm (CMA) working at the symbol rate, with a similar computational burden, and reduces the residual error of the CMA in multilevel modulations at the same time. A comparison with the most common blind techniques is presented.

Blind source separation using Renyi's α-marginal entropies

Abstract We have recently suggested the minimization of a nonparametric estimator of Renyis mutual information as a criterion for blind source separation. Using a two-stage topology, consisting of spatial whitening and a series of Givens rotations, the cost function reduces to the sum of marginal entropies, just like in the Shannons entropy case. Since we use a Parzen window density estimator and eliminate the joint entropy by employing an orthonormal demixing matrix, the problems of probability density function inaccuracy due to truncation of series expansion and the estimation of joint pdfs in high-dimensional spaces (given the typical paucity of data) are avoided, respectively. In our previous formulation, the algorithm was restricted to Renyis second-order entropy and Gaussian kernels for the Parzen window estimator. The present work extends the previous results by formulating a new estimation methodology for Renyis entropy, which allows the designer to choose any order of entropy and any suitable kernel function. Simulations illustrate that the proposed method compares favorably to Hyvarinens FastICA, Bell and Sejnowskis Infomax and Commons minimum of mutual information.

RSVP keyboard: An EEG based typing interface

Humans need communication. The desire to communicate remains one of the primary issues for people with locked-in syndrome (LIS). While many assistive and augmentative communication systems that use various physiological signals are available commercially, the need is not satisfactorily met. Brain interfaces, in particular, those that utilize event related potentials (ERP) in electroencephalography (EEG) to detect the intent of a person noninvasively, are emerging as a promising communication interface to meet this need where existing options are insufficient. Existing brain interfaces for typing use many repetitions of the visual stimuli in order to increase accuracy at the cost of speed. However, speed is also crucial and is an integral portion of peer-to-peer communication; a message that is not delivered timely often looses its importance. Consequently, we utilize rapid serial visual presentation (RSVP) in conjunction with language models in order to assist letter selection during the brain-typing process with the final goal of developing a system that achieves high accuracy and speed simultaneously. This paper presents initial results from the RSVP Keyboard system that is under development. These initial results on healthy and locked-in subjects show that single-trial or few-trial accurate letter selection may be possible with the RSVP Keyboard paradigm.

A novel adaptive beamformer for MEG source reconstruction effective when large background brain activities exist

This paper proposes a novel prewhitening eigenspace beamformer suitable for magnetoencephalogram (MEG) source reconstruction when large background brain activities exist. The prerequisite for the method is that control-state measurements, which contain only the contributions from the background interference, be available, and that the covariance matrix of the background interference can be obtained from such control-state measurements. The proposed method then uses this interference covariance matrix to remove the influence of the interference in the reconstruction obtained from the target measurements. A numerical example, as well as applications to two types of MEG data, demonstrates the effectiveness of the proposed method

Adaptive blind deconvolution of linear channels using Renyi's entropy with Parzen window estimation

Blind deconvolution of linear channels is a fundamental signal processing problem that has immediate extensions to multiple-channel applications. In this paper, we investigate the suitability of a class of Parzen-window-based entropy estimates, namely Renyis entropy, as a criterion for blind deconvolution of linear channels. Comparisons between maximum and minimum entropy approaches, as well as the effect of entropy order, equalizer length, sample size, and measurement noise on performance, will be investigated through Monte Carlo simulations. The results indicate that this nonparametric entropy estimation approach outperforms the standard Bell-Sejnowski and normalized kurtosis algorithms in blind deconvolution. In addition, the solutions using Shannons entropy were not optimal either for super- or sub-Gaussian source densities.

A framework for rapid visual image search using single-trial brain evoked responses

We report the design and performance of a brain computer interface for single-trial detection of viewed images based on human dynamic brain response signatures in 32-channel electroencephalography (EEG) acquired during a rapid serial visual presentation. The system explores the feasibility of speeding up image analysis by tapping into split-second perceptual judgments of humans. We present an incremental learning system with less memory storage and computational cost for single-trial event-related potential (ERP) detection, which is trained using cross-session data. We demonstrate the efficacy of the method on the task of target image detection. We apply linear and nonlinear support vector machines (SVMs) and a linear logistic classifier (LLC) for single-trial ERP detection using data collected from image analysts and naive subjects. For our data the detection performance of the nonlinear SVM is better than the linear SVM and the LLC. We also show that our ERP-based target detection system is five-fold faster than the traditional image viewing paradigm.

An analysis of entropy estimators for blind source separation

An extensive analysis of a non-parametric, information-theoretic method for instantaneous blind source separation (BSS) is presented. As a result a modified stochastic information gradient estimator is proposed to reduce the computational complexity and to allow the separation of sub-Gaussian sources. Interestingly, the modification enables the method to simultaneously exploit spatial and spectral diversity of the sources. Consequently, the new algorithm is able to separate i.i.d, sources, which requires higher-order spatial statistics, and it is also able to separate temporally correlated Gaussian sources, which requires temporal statistics. Three reasons are given why Renyis entropy estimators for Information-Theoretic Learning (ITL), on which the proposed method is based, is to be preferred over Shannons entropy estimators for ITL. Also contained herein is an extensive comparison of the proposed method with JADE, Infomax, Comons MI, FastICA, and a non-parametric, information-theoretic method that is based on Shannons entropy. Performance comparisons are shown as a function of the data length, source kurtosis, number of sources, and stationarity/correlation of the sources.