Jan Anguita

Audio, video and multimodal person identification in a smart room

In this paper, we address the modality integration issue on the example of a smart room environment aiming at enabling person identification by combining acoustic features and 2D face images. First we introduce the monomodal audio and video identification techniques and then we present the use of combined input speech and face images for person identification. The various sensory modalities, speech and faces, are processed both individually and jointly. Its shown that the multimodal approach results in improved performance in the identification of the participants.

On the use of genuine-impostor statistical information for score fusion in multimodal biometrics/sur l’usage de l’information statistioue client-imposteur pour la fusion des scores en biométrie multimodale

Matching score level fusion techniques in multimodal person verification conventionally use global score statistics in the normalization and fusion stages. In this paper, novel normalization and fusion methods are presented to take advantage of the separate statistics of the monomodal scores in order to reduce the genuine and impostor pdf lobe overlapping and improve the verification rate. Joint mean normalization is an affine transformation that normalizes the mean of the monomodal biometrics scores separately for the genuine and impostor individuals. Histogram equalization is used to align the statistical distribution of the monomodal scores and make the whole separate statistics comparable. The presented weighting fusion methods have been designed to minimize the variances of the separate multimodal statistics and reduce the multimodal pdf lobe overlapping. The results obtained in speech and face scores fusion upon polycost and xm2vts databases show that the proposed techniques provide better results than the conventional methods.RésuméLes techniques de fusion au niveau des degrés de pertinence dans la vérification multimodale de personnes utilisent conventionnellement des statistiques globales de perticence pour les étapes de normalisation et de fusion. Dans le présent article, de nouvelles méthodes de normalisation et de fusion sont présentées pour profiter des statistiques séparées des pertinences monomodales en vue de réduire la superposition des densités de probabilité de client et d’imposteur et d’améliorer le taux de vérification. La normalisation conjointe de la moyenne est une transformation affine qui normalise la moyenne des qualifications biométriques monomodales séparément pour les individus client et imposteur. L’égalisation de l’histogramme est utilisée pour aligner la distribution statistique des pertinences monomodales et peut rendre comparables les statistiques complètes séparées. Les présentes méthodes de fusion avec pondération on été conçues de façon à minimiser les variances des statistiques multimodales séparées et réduire la superposition des densités de probabilité multimodales. Les résultats obtenus dans la fusion de pertinences pour voix et visage avec les bases de données polycost et xm2vts démontrent que la normalisation proposée et les techniques de fusion produisent de meilleurs résultats que les méthodes conventionnelles.

Detection of confusable words in automatic speech recognition

A new method to detect words that are likely to be confused by speech recognition systems is presented in this letter. A new dissimilarity measure between two words is calculated in two steps. First, the phonetic transcriptions of the words are aligned using only phonetic information. Two kinds of alignments are used: either with or without insertions and deletions. Second, the dissimilarity measure is calculated on the basis of the resulting alignment and acoustic information obtained from the hidden Markov models of the phones. In a classical false acceptance/false rejection framework, the equal error rate was measured to be less than 5%.

Improved Jacobian Adaptation for Robust Speaker Verification

Jacobian Adaptation (JA) has been successfully used in Automatic Speech Recognition (ASR) systems to adapt the acoustic models from the training to the testing noise conditions. In this work we present an improvement of JA for speaker verification, where a specific training noise reference is estimated for each speaker model. The new proposal, which will be referred to as Model-dependent Noise Reference Jacobian Adaptation (MNRJA), has consistently outperformed JA in our speaker verification experiments.

Jacobian Adaptation with Continuous Noise Estimation for Real Speaker Verification Applications

Jacobian adaptation (JA) of the acoustic models is a fast adaptation technique that has been successfully used in both speech and speaker recognition. This technique adapts the acoustic models on the basis of the difference between the testing and the training noise conditions. For this reason, a noise reference of both the training and the testing phase is needed. In previous works, the noise conditions have been commonly supposed to be known or estimated from the first part of the signal. In this work we propose to obtain the noise references by using continuous noise estimation methods, which are more appropriate for real applications and can deal with non-stationary noises. Several noise estimation methods are compared: recursive averaging (RA), minimum statistics (MS) and minima controlled recursive averaging (MCRA). The obtained results show that these techniques are effective for JA