Kwokleung Chan

Variational Bayesian learning of ICA with missing data

Missing data are common in real-world data sets and are a problem for many estimation techniques. We have developed a variational Bayesian method to perform independent component analysis (ICA) on high-dimensional data containing missing entries. Missing data are handled naturally in the Bayesian framework by integrating the generative density model. Modeling the distributions of the independent sources with mixture of gaussians allows sources to be estimated with different kurtosis and skewness. Unlike the maximum likelihood approach, the variational Bayesian method automatically determines the dimensionality of the data and yields an accurate density model for the observed data without overfitting problems. The technique is also extended to the clusters of ICA and supervised classification framework.

Speech feature analysis using variational Bayesian PCA

In most hidden Markov model-based automatic speech recognition systems, one of the fundamental questions is to determine the intrinsic speech feature dimensionality and the number of clusters used on the Gaussian mixture model. We analyzed mel-frequency band energies using a variational Bayesian principal component analysis method to estimate the feature dimensionality as well as the number of Gaussian mixtures by learning a maximum lower bound of the evidence instead of maximizing the likelihood function as used in conventional speech recognition systems. In analyzing the Texas Instruments/Massachusetts Institute of Technology (TIMIT) speech database, our method revealed the intrinsic structures of vowels and consonants. The usefulness of this method is demonstrated in the superior classification performance for the most difficult phonemes /b/, /d/, and /g/.

Application of variational Bayesian PCA for speech feature extraction

In a standard mel-frequency cepstral coefficient-based speech recognizer, it is common to use the same feature dimension and the number of Gaussian mixtures for all subunits. We proposed to use different transformations and different number of mixtures for each subunit. We obtained the transformations from mel-frequency band energies by using the variational Bayesian principal component analysis (PCA) method. In the method, hyperparameters of the Gaussian mixtures and the number of mixtures are automatically learned through maximization of a lower bound of the evidence instead of the likelihood in the conventional maximum likelihood paradigm. Analyzing the TIMIT speech data, we revealed intrinsic structures of vowels and consonants. We demonstrated the userfulness of the method for speech recognition by performing phoneme classification of /b/, /d/ and /g/ phonemes.