Christophe Cerisara

On noise masking for automatic missing data speech recognition: A survey and discussion

Automatic speech recognition (ASR) has reached very high levels of performance in controlled situations. However, the performance degrades significantly when environmental noise occurs during the recognition process. Nowadays, the major challenge is to reach a good robustness to adverse conditions, so that automatic speech recognizers can be used in real situations. Missing data theory is a very attractive and promising approach. Unlike other denoising methods, missing data recognition does not match the whole data with the acoustic models, but instead considers part of the signal as missing, i.e. corrupted by noise. While speech recognition with missing data can be handled efficiently by methods such as data imputation or marginalization, accurately identifying missing parts (also called masks) remains a very challenging task. This paper reviews the main approaches that have been proposed to address this problem. The objective of this study is to identify the mask estimation methods that have been proposed so far, and to open this domain up to other related research, which could be adapted to overcome this difficult challenge. In order to restrict the range of methods, only the techniques using a single microphone are considered.

A recombination model for multi-band speech recognition

We describe a continuous speech recognition system that uses the multi-band paradigm. This principle is based on the recombination of several independent sub-recognizers, each one assigned to a specific frequency band. The major issue of such systems consists of deciding at which time the recombination must be done. Our algorithm lets each band be totally independent from the others, and uses the different solutions to resegment the initial sentence. Finally, the bands are synchronously merged together, according to this new segmentation. The whole system is too complex to be entirely described here, and, in this paper, we concentrate on the synchronous recombination part, which is achieved by a classifier. The system has been tested in clean and noisy environments, and proved to be especially robust to noise.

Multi-band automatic speech recognition

This paper presents a new architecture for automatic speech recognition systems which is characterized by the division of the spectral domain of the speech signal into several independent frequency bands. This model is based on the psycho-acoustic work of Fletcher (1953) who proposed a similar principle for the human auditory system. Jont B. Allen published a paper in 1994 in which he summarized the work of Fletcher and also proposed to adapt the multi-band paradigm to automatic speech recognition (ASR) (Allen, 1994). Many researchers have then studied this principle and built such ASR systems. The goal of this paper is to analyse some of the most important issues in the design of a multi-band ASR system in order to determine which architecture it should have in which environment. Two other major problems are then considered: how to train multi-band systems and how to use them for continuous ASR.

Automatic Dialog Acts Recognition Based on Sentence Structure

This paper deals with automatic dialog acts (DAs) recognition in Czech. Our work focuses on two applications: a multimodal reservation system and an animated talking head for hearing-impaired people. In that context, we consider the following DAs: statements, orders, investigation questions and other questions. The main goal of this paper is to propose, implement and evaluate new approaches to automatic DAs recognition based on sentence structure and prosody. Our system is tested on a Czech corpus that simulates a task of train tickets reservation. With lexical-only information, the classification accuracy is 91%. We proposed two methods to include sentence structure information, which respectively give 94% and 95%. When prosodic information is further considered, the recognition accuracy reaches 96%

α-Jacobian environmental adaptation

The robustness of automatic speech recognition systems to noise is still a problem, especially for small footprint systems. This paper addresses the problem of noise robustness using model compensation methods. Such algorithms are already available, but their complexity is usually high. An often-referenced method for achieving noise robustness is parallel model combination (PMC). Several algorithms have been proposed to develop more computationally efficient methods than PMC. For example, Jacobian adaptation approximates PMC with a linear transformation function in the cepstral domain. However, the Jacobian approximation is valid only for test environments that are close to the training conditions whereas, in real test conditions, the mismatch between the test and training environments is usually large. In this paper, we propose two methods, respectively called static and dynamic α-Jacobian adaptation (or α-JAC), to compute new linear approximations of PMC for realistic test environments. We further extend both algorithms to compensate for additive and convolutional noise and we derive the corresponding non-linear algorithm that is approximated. All these algorithms are experimentally compared in important mismatch conditions. As compared to Jacobian adaptation, improvements are observed with both static and dynamic α-Jacobian adaptation.

Environmental adaptation based on first order approximation

We propose an algorithm that compensates for both additive and convolutional noise. The goal of this method is to achieve an efficient environmental adaptation to realistic environments both in terms of computation time and memory. The algorithm described in this paper is an extension of an additive noise adaptation algorithm. Experimental results are given on a realistic database recorded in a car. This database is further filtered by a low pass filter to combine additive and channel noise. The proposed adaptation algorithm reduces the error rate by 75 % on this database, when compared to our baseline system without environmental adaptation.

Automatic discovery of topics and acoustic morphemes from speech

This work deals with automatic lexical acquisition and topic discovery from a speech stream. The proposed algorithm builds a lexicon enriched with topic information in three steps: transcription of an audio stream into phone sequences with a speaker- and task-independent phone recogniser, automatic lexical acquisition based on approximate string matching, and hierarchical topic clustering of the lexical entries based on a knowledge-poor co-occurrence approach. The resulting semantic lexicon is then used to automatically cluster the incoming speech stream into topics. The main advantages of this algorithm are its very low computational requirements and its independence to pre-defined linguistic resources, which makes it easy to port to new languages and to adapt to new tasks. It is evaluated both qualitatively and quantitatively on two corpora and on two tasks related to topic clustering. The results of these evaluations are encouraging and outline future directions of research for the proposed algorithm, such as building automatic orthographic labels of the lexical items.

Automatic dialogue act recognition with syntactic features

This work studies the usefulness of syntactic information in the context of automatic dialogue act recognition in Czech. Several pieces of evidence are presented in this work that support our claim that syntax might bring valuable information for dialogue act recognition. In particular, a parallel is drawn with the related domain of automatic punctuation generation and a set of syntactic features derived from a deep parse tree is further proposed and successfully used in a Czech dialogue act recognition system based on conditional random fields. We finally discuss the possible reasons why so few works have exploited this type of information before and propose future research directions to further progress in this area.

Sentence Structure for Dialog Act Recognition in Czech

This paper deals with automatic dialog acts (Das) recognition in Czech based on sentence structure. We consider the following DAs: statements, orders, yes/no questions and other questions. In our previous works, we have proposed, implemented and evaluated new approaches to automatic DAs recognition based on sentence structure and prosody. The word sequences were manually transcribed. The main goal of this paper is to evaluate the performances of our approaches when these word sequences are unknown and estimated from a speech recognizer. Our system is tested on a Czech corpus that simulates a task of train tickets reservation. When manual transcription is used, classification accuracy without and with sentence structure models is 91 %, 94 % and 95 %. The recognition accuracy reaches 96 % with prosodic combination. When word sequences are estimated from a speech recognizer, the classification score is 88 % without and 91 % and 92 % with sentence structure models. The combination with prosody gives 93 % of accuracy

Confidence Measures for Semi-Automatic Labeling of Dialog Acts

This paper deals with semi-supervised classifier training for automatic dialog acts (DAs) recognition. In our previous works, we have designed a dialog act recognition system for reservation applications in the Czech language. In this work, we propose to retrain this system on another corpus, for another task (broadcast news speech), in a different language (French) and with another set of dialog acts. This is realized using a semi-supervised approach based on the expectation-maximization (EM) algorithm. We show that, in the proposed experimental setup, the use of confidence measures to filter out incorrectly recognized dialog acts is required to improve the results. Two confidence measures are thus proposed and evaluated on the French broadcast news corpus. Experimental results confirm the interest of this approach for the task of training automatic dialog act classifiers.