John Goddard Close

Post-training discriminative pruning for RBMs

One of the major challenges in the area of artificial neural networks is the identification of a suitable architecture for a specific problem. Choosing an unsuitable topology can exponentially increase the training cost, and even hinder network convergence. On the other hand, recent research indicates that larger or deeper nets can map the problem features into a more appropriate space, and thereby improve the classification process, thus leading to an apparent dichotomy. In this regard, it is interesting to inquire whether independent measures, such as mutual information, could provide a clue to finding the most discriminative neurons in a network. In the present work, we explore this question in the context of Restricted Boltzmann machines, by employing different measures to realize post-training pruning. The neurons which are determined by each measure to be the most discriminative, are combined and a classifier is applied to the ensuing network to determine its usefulness. We find that two measures in particular seem to be good indicators of the most discriminative neurons, producing savings of generally more than 50% of the neurons, while maintaining an acceptable error rate. Further, it is borne out that starting with a larger network architecture and then pruning is more advantageous than using a smaller network to begin with. Finally, a quantitative index is introduced which can provide information on choosing a suitable pruned network.

A Speaker Verification System Using SVM over a Spanish Corpus

This paper presents a description of the principal aspects employed in the development of a speaker verification system based on a Spanish corpus. The main goal is to obtain classification results and behavior using Support Vector Machines (SVM) as the classifier technique. The most relevant aspects involved in developing a Spanish corpus are given. For the front end processing a novel method to suppress silences between words is proposed and successfully applied. The validation to the complete system is made using randomly selected feature vectors and vectors from continuous sequences of the voice signal. Additionally, Gaussian Mixtures Models (GMM) and Artificial Neural Networks (ANN) are also used as classifiers to compare and validate the results.

String kernels for the classification of speech data

Support Vector Machines (SVM) have been applied to a wide variety of classification problems with excellent results. This has to do with their provable generalization ability derived from Statistical Learning Theory. Recently specialized kernels, such as the Fisher kernel and string kernels, have been introduced in an attempt to apply the same SVM framework to sequential data. Notable results have been obtained on classification tasks related to biosequences and text documents showing that the specialized kernels may provide a viable and interesting alternative to other classifiers, such as those using Hidden Markov Models. String kernels are particularly attractive because of their conceptual simplicity and they also furnish insight into the task of sequential data classification. In the present paper string kernels are applied to a new application area, that of automatic speech recognition. In particular, different string kernels are tested on the task of phoneme recognition and the results obtained are...

A descriptive analysis between syllables in English and Spanish

In recent years different attempts have been made to incorporate temporal information longer than the phoneme into automatic speech recognizers (ASR) for English. The reason for these approaches is related to limitations which arise with existing systems based on phonemes, such as the degradation in performance of ASRs under noisy conditions and variations in pronunciation due to phoneme omission. It is conjectured that humans naturally use longer time periods, corresponding, for example, to syllables, to perceptually integerate information. In the case of Spanish, little seems to have been done in this direction for ASRs. In the present paper, a preliminary comparison is made between syllables in Spanish and English with a view to their factibility in an ASR for Spanish. In particular, a descriptive statistical analysis is conducted with a Spanish speech database to derive the most common structures of the syllables and the most common monosyllables. This is contrasted with previously found results in English. Spectrograms are also used to illustrate pertinent characteristics of Spanish. These results suggest that syllables may indeed provide useful information for an ASR in Spanish and could provide greater success than their counterparts in English. [Work supported by CONACYT under Project 31929‐A.] (To be presented in Spanish.)