Alain Biem

An application of discriminative feature extraction to filter-bank-based speech recognition

A pattern recognizer is usually a modular system which consists of a feature extractor module and a classifier module. Traditionally, these two modules have been designed separately, which may not result in an optimal recognition accuracy. To alleviate this fundamental problem, the authors have developed a design method, named discriminative feature extraction (DFE), that enables one to design the overall recognizer, i.e., both the feature extractor and the classifier, in a manner consistent with the objective of minimizing recognition errors. This paper investigates the application of this method to designing a speech recognizer that consists of a filter-hank feature extractor and a multi-prototype distance classifier. Carefully investigated experiments demonstrate that DFE achieves the design of a better recognizer and provides an innovative recognition-oriented analysis of the filter-bank, as an alternative to conventional analysis based on psychoacoustic expertise or heuristics.

Pattern recognition using discriminative feature extraction

We propose a new design method, called discriminative feature extraction for practical modular pattern recognizers. A key concept of discriminative feature extraction is the design of an overall recognizer in a manner consistent with recognition error minimization. The utility of the method is demonstrated in a Japanese vowel recognition task.

Feature extraction based on minimum classification error/generalized probabilistic descent method

A novel approach to pattern recognition which comprehensively optimizes both a feature extraction process and a classification process is introduced. Assuming that the best features for recognition are the ones that yield the lowest classification error rate over unknown data, an overall recognizer, consisting of a feature extractor module and a classifier module, is trained using the minimum classification error (MCE)/generalized probabilistic descent (GPD) method. Although the proposed discriminative feature extraction approach is a direct and simple extension of MCE/GPD, it is a significant departure from conventional approaches, providing a comprehensive basis for the entire system design. Experimental results are presented for the simple example of optimally designing a cepstrum representation for vowel recognition. The results clearly demonstrate the effectiveness of the proposed method.<<ETX>>

Filter bank design based on discriminative feature extraction

A filter bank model, which achieves minimum error, is investigated in this paper. A bank-of-filter feature extractor module is comprehensively optimized with the classifiers parameters for minimization of the errors occurring at the back-end classifier. The method has been applied to readjusting Mel-scale and Bark-scale based filter banks for the Japanese vowel recognition task, the framework being provided by the minimum classification error (MCE)/generalised probabilistic descent method (GPD). The results show suggestive phenomena underlying the accuracy of the proposed approach.<<ETX>>

Discriminative feature extraction for speech recognition

A novel approach to pattern recognition, called discriminative feature extraction (DFE) is introduced as a way to interactively handle the input data with a given classifier. The entire recognizer, consisting of the feature extractor as well as the classifier, is trained with the minimum classification error generalised probabilistic descent learning algorithm. Both the philosophy and implementation examples of this approach are described. DFE realizes a significant departure from conventional approaches, providing a comprehensive base for the entire system design. By way of example, an automatic scaling process is described, and experimental results for designing a cepstrum representation for vowel recognition are presented.<<ETX>>

Cepstrum-based filter-bank design using discriminative feature extraction training at various levels

This paper investigates the realization of optimal filter bank-based cepstral parameters. The framework is the discriminative feature extraction method (DFE) which iteratively estimates the filter-bank parameters according to the errors that the system makes. Various parameters of the filter-bank, such as center frequency, bandwidth, and gain are optimized using a string-level optimization and a frame-level optimization scheme. Application to vowel and noisy telephone speech recognition tasks shows that the DFE method realizes a more robust classifier by appropriate feature extraction.

HMM topology optimization for handwriting recognition

This paper addresses the problem of hidden Markov model (HMM) topology estimation in the context of on-line handwriting recognition. HMM have been widely used in applications related to speech and handwriting recognition with great success. One major drawback with these approaches, however, is that the techniques that they use for estimating the topology of the models (number of states, connectivity between the states and the number of Gaussians), are usually heuristically derived, without optimal certainty. This paper addresses this problem, by comparing a couple of commonly used heuristically derived methods to an approach that uses the Bayesian information criterion (BIC) for computing the optimal topology. Experimental results on discretely written letters show that using BIC gives comparable results to heuristic approaches with a model that has nearly 10% fewer parameters.

Discriminative training for large vocabulary telephone-based name recognition

This paper describes progress on a commercial application of the MECS recognition system to the task of recognizing Japanese family names spoken by customers into the answering machines of a large marketing/human resource company. The task is thus speaker-independent, open vocabulary, and is characterized by large variation in caller speaking styles, telephone types and acoustic environments. Our results show that context-independent hidden Markov models trained discriminatively with the minimum classification error criterion are a practical alternative to context-dependent models based on phonetic decision trees, yielding better performance with a much smaller number of parameters. On this difficult task we have obtained 59% correct family name recognition. A phoneme-based confidence measure enables us to obtain 85% correct name recognition for accepted utterances, at an overall utterance acceptance rate of 15%.

Discriminative training of tied mixture density HMMs for online handwritten digit recognition

This paper describes and evaluates the maximum mutual information criterion (MMI) for online unconstrained-style handwritten digit recognition based on hidden Markov models (HMMs). The study focuses on determining the best MMI optimization scheme and the HMM parameters that exhibit the most discriminative capabilities in the context of tied mixture density hidden Markov models (TDHMMs), where all HMM states share a pool of Gaussians. The experimental results show that the second-order optimization scheme is the most efficient and that although means and covariance matrix are shared by all models, they contribute the most to discrimination.

Discriminative feature extraction application to filter bank design

This paper investigates the design of a filter bank model by the discriminative feature extraction method (DFE). A filter bank-based feature extractor is optimized with the classifiers parameters for the minimization of the errors occurring at the back-end classification process. The framework of minimum classification error/generalized probabilistic descent method (MCE/GPD) is used as the basis for optimization. The method is first tested in a vowel recognition task. Analysis of the process shows how DFE extracts those parts of the spectrum that are relevant to discrimination. Then the method is applied to a multi-speaker word recognition system intended to act as telephone directory assistance operator.