Xi-n Li

Complex Independent Component Analysis by Entropy Bound Minimization

We first present a new (differential) entropy estimator for complex random variables by approximating the entropy estimate using a numerically computed maximum entropy bound. The associated maximum entropy distributions belong to the class of weighted linear combinations and elliptical distributions, and together, they provide a rich array of bivariate distributions for density matching. Next, we introduce a new complex independent component analysis (ICA) algorithm, complex ICA by entropy-bound minimization (complex ICA-EBM), using this new entropy estimator and a line search optimization procedure. We present simulation results to demonstrate the superior separation performance and computational efficiency of complex ICA-EBM in separation of complex sources that come from a wide range of bivariate distributions.

Independent Component Analysis by Entropy Bound Minimization

A novel (differential) entropy estimator is introduced where the maximum entropy bound is used to approximate the entropy given the observations, and is computed using a numerical procedure thus resulting in accurate estimates for the entropy. We show that such an estimator exists for a wide class of measuring functions, and provide a number of design examples to demonstrate its flexible nature. We then derive a novel independent component analysis (ICA) algorithm that uses the entropy estimate thus obtained, ICA by entropy bound minimization (ICA-EBM). The algorithm adopts a line search procedure, and initially uses updates that constrain the demixing matrix to be orthogonal for robust performance. We demonstrate the superior performance of ICA-EBM and its ability to match sources that come from a wide range of distributions using simulated and real-world data.

Joint blind source separation by generalized joint diagonalization of cumulant matrices

In this paper, we show that the joint blind source separation (JBSS) problem can be solved by jointly diagonalizing cumulant matrices of any order higher than one, including the correlation matrices and the fourth-order cumulant matrices. We introduce an efficient iterative generalized joint diagonalization algorithm such that a series of orthogonal procrustes problems are solved. We present simulation results to show that the new algorithms can reliably solve the permutation ambiguity in JBSS and that they offer superior performance compared with existing multiset canonical correlation analysis (MCCA) and independent vector analysis (IVA) approaches. Experiment on real-world data for separation of fetal heartbeat in electrocardiogram (ECG) data demonstrates a new application of JBSS, and the success of the new algorithms for a real-world problem.

Blind spatiotemporal separation of second and/or higher-order correlated sources by entropy rate minimization

We propose a new entropy rate estimator for a second and/or higher-order correlated source by modeling it as the output of a linear filter, which can be mixed-phase, driven by Gaussian or non-Gaussian noise. Based on this estimator, we develop a new spatiotemporal blind source separation (BSS) algorithm, full BSS (FBSS), by minimizing the entropy rate of separated sources. FBSS provides more flexibilities in exploiting the temporal structures of sources than the state-of-the-art spatiotemporal BSS algorithms, which use AR models, and thus imply minimum-phase property of sources.

Noncircular Principal Component Analysis and Its Application to Model Selection

One of the most commonly used data analysis tools, principal component analysis (PCA), since is based on variance maximization, assumes a circular model, and hence cannot account for the potential noncircularity of complex data. In this paper, we introduce noncircular PCA (ncPCA), which extends the traditional PCA to the case where there can be both circular and noncircular Gaussian signals in the subspace. We study the properties of ncPCA, introduce an efficient algorithm for its computation, and demonstrate its application to model selection, i.e., the detection of both the signal subspace order and the number of circular and noncircular signals. We present numerical results to demonstrate the advantages of ncPCA over regular PCA when there are noncircular signals in the subspace. At the same time, we note that since a noncircular model has more degrees of freedom than a circular one, there are cases where a circular model might be preferred even though the underlying problem is noncircular. In particular, we show that a circular model is preferred when the signal-to-noise ratio (SNR) is low, number of samples is small, or the degree of noncircularity of the signals is low. Hence, ncPCA inherently provides guidance as to when to take noncircularity into account.

High-Resolution Multiple Wideband and Nonstationary Source Localization With Unknown Number of Sources

In this paper, a new algorithm for high-resolution multiple wideband and nonstationary source localization using a sensor array is proposed. The received signals of the sensor array are first converted into the time-frequency domain via short-time Fourier transform (STFT) and we find that a set of short-time power spectrum matrices at different time instants have the joint diagonalization structure in each frequency bin. Based on such joint diagonalization structure, a novel cost function is designed and a new spatial spectrum for direction-of-arrival (DOA) estimation at hand is derived. Compared to the maximum-likelihood (ML) method with high computational complexity, the proposed algorithm obtains the DOA estimates via one-dimensional (1-D) search instead of multidimensional search. Therefore its computational complexity is much lower than the ML method. Unlike the subspace-based high-resolution DOA estimation techniques, it is not necessary to determine the number of sources in advance for the proposed algorithm. Moreover, the proposed method is robust to the effects of reverberation caused by multipath reflections. Hence it is suitable for multiple acoustic source localization in a reverberant room. The results of numerical simulations and experiments in a real room with a moderate reverberation are provided to demonstrate the good performance of the proposed approach.

Nonorthogonal independent vector analysis using multivariate Gaussian model

We consider the problem of joint blind source separation of multiple datasets and introduce an effective solution to the problem. We pose the problem in an independent vector analysis (IVA) framework utilizing the multivariate Gaussian source vector distribution. We provide a new general IVA implementation using a decoupled nonorthogonal optimization algorithm and establish the connection between the new approach and another approach using second-order statistics, multiset canonical correlation analysis. Experimental results are given to demonstrate the success of the new algorithm in achieving reliable source separation for both Gaussian and non-Gaussian sources.

Blind Source Separation by Entropy Rate Minimization

By assuming latent sources are statistically independent, independent component analysis separates underlying sources from a given linear mixture. Since in many applications, latent sources are both non-Gaussian and have sample dependence, it is desirable to exploit both properties jointly. In this paper, we use mutual information rate to construct a general framework for analysis and derivation of algorithms that take both properties into account. We discuss two types of source models for entropy rate estimation-a Markovian and an invertible filter model-and give the general independent component analysis cost function, update rule, and performance analysis based on these. We also introduce four algorithms based on these two models, and show that their performance can approach the Cramér-Rao lower bound. In addition, we demonstrate that the algorithms with flexible models exhibit very desirable performance for “natural” data.

An effective decoupling method for matrix optimization and its application to the ICA problem

Matrix optimization of cost functions is a common problem. Construction of methods that enable each row or column to be individually optimized, i.e., decoupled, are desirable for a number of reasons. With proper decoupling, the convergence characteristics such as local stability can be improved. Decoupling can enable density matching in applications such as independent component analysis (ICA). Lastly, efficient Newton algorithms become tractable after decoupling. The most common method for decoupling rows is to reduce the optimization space to orthogonal matrices. Such restrictions can degrade performance. We present a decoupling procedure that uses standard vector optimization procedures while still admitting nonorthogonal solutions. We utilize the decoupling procedure to develop a new decoupled ICA algorithm that uses Newton optimization enabling superior performance when the sample size is limited.

Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems

It is well known that there exist two fundamental limitations in the linear discriminant analysis (LDA). One is that it cannot be applied when the within-class scatter matrix is singular, which is caused by the undersampled problem. The other is that it lacks the capability to capture the nonlinearly clustered structure of the data due to its linear nature. In this paper, a new kernel-based nonlinear discriminant analysis algorithm using minimum squared errors criterion (KDA-MSE) is proposed to solve these two problems. After mapping the original data into a higher-dimensional feature space using kernel function, the MSE criterion is used as the discriminant rule and the corresponding dimension reducing transformation is derived. Since the MSE solution does not require the scatter matrices to be nonsingular, the proposed KDA-MSE algorithm is applicable to the undersampled problem. Moreover, the new KDA-MSE algorithm can be applied to multiclass problem, whereas the existing MSE-based kernel discriminant methods are limited to handle twoclass data only. Extensive experiments, including object recognition and face recognition on three benchmark databases, are performed and the results demonstrate that our algorithm is competitive in comparison with other kernel-based discriminant techniques in terms of recognition accuracy.