Dongxin Xu

Learning from Examples with Information Theoretic Criteria

This paper discusses a framework for learning based on information theoretic criteria. A novel algorithm based on Renyis quadratic entropy is used to train, directly from a data set, linear or nonlinear mappers for entropy maximization or minimization. We provide an intriguing analogy between the computation and an information potential measuring the interactions among the data samples. We also propose two approximations to the Kulback-Leibler divergence based on quadratic distances (Cauchy-Schwartz inequality and Euclidean distance). These distances can still be computed using the information potential. We test the newly proposed distances in blind source separation (unsupervised learning) and in feature extraction for classification (supervised learning). In blind source separation our algorithm is capable of separating instantaneously mixed sources, and for classification the performance of our classifier is comparable to the support vector machines (SVMs).

Synthetic aperture radar automatic target recognition with three strategies of learning and representation

We describe a new architecture for synthetic aperture radar (SAR) automatic target recognition (ATR) based on the premise that the pose of the target is estimated within a high degree of precision. The advantage of our classifier design is that the input space complexity is decreased with the pose information, which enables fewer features to classify targets with a higher degree of accuracy. Moreover, the training of the classifier can be done discriminantly, which also improves performance and decreases the complexity of the classifier. Three strategies of learning and representation to build the pattern space and discriminant functions are compared: Vapniks support vector machine (SVM), a newly developed quadratic mutual information (QMI) cost function for neural networks, and a principal component analysis extended recently with multiresolution (PCA-M). Experimental results obtained in the MSTAR database show that the performance of our classifiers is better than that of standard template matching in the same dataset. We also rate the quality of the classifiers for detection using confusers, and show significant improvement in rejection.

Novel semi-blind ICI equalization algorithm for wireless OFDM systems

Intercarrier interference is deemed as one of the crucial problems in the wireless orthogonal frequency division multiplexing (OFDM) systems. The conventional ICI mitigation schemes involve the frequency-domain channel estimation or the additional coding, both of which require the spectral overhead and hence lead to the significant throughput reduction. Besides, the OFDM receivers using the ICI estimation rely on a large-dimensional matrix inverter with high computational complexity especially for many subcarriers such as digital video broadcasting (DVB) systems and wireless metropolitan-area networks (WMAN). To the best of our knowledge, no semi-blind ICI equalization has been addressed in the existing literature. Thus, in this paper, we propose a novel semi-blind ICI equalization scheme using the joint multiple matrix diagonalization (JMMD) algorithm to greatly reduce the intercarrier interference in OFDM. However, the well-known phase and permutation indeterminacies emerge in all blind equalization schemes. Hence we also design a few OFDM pilot blocks and propose an iterative identification method to determine the corresponding phase and permutation variants in our semi-blind scheme. Our semi-blind ICI equalization algorithm integrating the JMMD with the additional pilot-based iterative identification is very promising for the future high-throughput OFDM systems. Through Monte Carlo simulations, the QPSK-OFDM system with our proposed semi-blind ICI equalizer can achieve significantly better performance with symbol error rate reduction in several orders-of-magnitude. For the 16QAM-OFDM system, our scheme can also improve the performance over the plain OFDM system to some extent.

Pose estimation in SAR using an information theoretic criterion

In this paper we formulate pose estimation statistically and show that pose can be estimated from a low dimensional feature space obtained by maximizing the mutual information between the aspect angle and the output of a nonlinear mapper. We use the Havrda-Charvat definition of entropy to implement a nonparametric estimator based on the Parzen window method. Results in the MSTAR data set are presented and show the performance of the methodology.

Pilot-free dynamic phase and amplitude estimations for wireless ICI self-cancellation coded OFDM systems

OFDM has the advantage over the conventional single-carrier modulation schemes in the presence of frequency-selective fadings. Nevertheless, intercarrier-interference (ICI) due to Doppler frequency drift, phase offset, local oscillator frequency drift, and sampling clock offset will be a severe problem in the wireless OFDM systems. Previous ICI self-cancellation coding schemes can greatly reduce the ICI, but they are very sensitive to the phase ambiguity, which is due to the composite effect of the phase offset, the multipath fading and the local oscillator frequency drift. In this paper, a novel receiver which combines the current ICI self-cancellation coding techniques with a new pilot-free joint phase/amplitude estimation and symbol detection scheme is proposed. Based on the energy modulation or the irregular symbol constellation, our new technique does not have any requirement of pilot symbols and it can operate on all kinds of phase error ranges. The proposed scheme is promising in comparison with other existing methods at different noise levels through OFDM simulations.

Learning from examples with quadratic mutual information

Discusses an algorithm to train nonlinear mappers with information theoretic criteria (entropy or mutual information) directly from a training set. The method is based on a Parzen window estimator and uses Renyis quadratic definition of entropy and a distance measure based on the Cauchy-Schwartz inequality. We apply the algorithm to the difficult problem of vehicle pose estimation in synthetic aperture radar (SAR) with very good results.

Generalized eigendecomposition with an on-line local algorithm

This article presents a novel, on-line, local learning algorithm to obtain generalized eigenvalues and their corresponding eigenvectors in descending order with a linear adaptive filter. The filter is composed of a set of forward linear projections constrained by lateral inhibitions. Equilibrium points of the adaptation process and their stability are briefly analyzed. Simulations and experimental comparisons are given to verify the validity and effectiveness of the proposed algorithm.

Generalized Oja's rule for linear discriminant analysis with Fisher criterion

Online learning rules for both principal component analysis (PCA) and linear discriminant analysis (LDA) with Fisher criterion are analyzed under the same framework, and a generalized Ojas rule for both is derived. For the LDA problem, the relationship between the Fisher criterion and the criterion of minimizing mean square error (MSE) is discussed. The experiments show that the convergence speed of the generalized Ojas rule as an adaptive method for the Fisher criterion is much faster than that of gradient descent method for the MSE criterion.

Blind equalization of communication sequences based on optimization of cumulant criteria

Blind equalization draws a lot of attention. Several statistical objective functions such as kurtosis and constant modulus were based on the noise-free model and hence their ISI cancellers were sensitive to noise. In this paper, a unifying adaptive blind equalization method is proposed, which can be robust to noise than the current cumulant-based adaptive methods. Our new objective functions can be applied for wired or wireless i.i.d. communication symbols with noise. The simulation shows that our new method outperforms the kurtosis-based method when the background noise exists.

Multi channel HMM

In speech recognition, the speech signal is usually represented in multidimensions but the hidden Markov model (HMM) is one-dimensional. A multichannel HMM (MC-HMM) is proposed as a more robust modeling method for multi-channel signals. Weighting among channels can be incorporated into the model in an uniform way, i.e. both model parameters and weighting coefficients can be estimated by the efficient Baum-Welch training procedure. Moreover, it can be shown that weighting among channels is exactly equivalent to relaxing the probability constraints. Therefore, for the weighting, no extra parameter is actually needed, and consequently no extra memory and computational costs are required. The preliminary experiment results on word spotting show that MC-HMM is better than the standard HMM.