Franz Gerl

Self-learning speaker identification for enhanced speech recognition

A novel approach for joint speaker identification and speech recognition is presented in this article. Unsupervised speaker tracking and automatic adaptation of the human-computer interface is achieved by the interaction of speaker identification, speech recognition and speaker adaptation for a limited number of recurring users. Together with a technique for efficient information retrieval a compact modeling of speech and speaker characteristics is presented. Applying speaker specific profiles allows speech recognition to take individual speech characteristics into consideration to achieve higher recognition rates. Speaker profiles are initialized and continuously adapted by a balanced strategy of short-term and long-term speaker adaptation combined with robust speaker identification. Different users can be tracked by the resulting self-learning speech controlled system. Only a very short enrollment of each speaker is required. Subsequent utterances are used for unsupervised adaptation resulting in continuously improved speech recognition rates. Additionally, the detection of unknown speakers is examined under the objective to avoid the requirement to train new speaker profiles explicitly. The speech controlled system presented here is suitable for in-car applications, e.g. speech controlled navigation, hands-free telephony or infotainment systems, on embedded devices. Results are presented for a subset of the SPEECON database. The results validate the benefit of the speaker adaptation scheme and the unified modeling in terms of speaker identification and speech recognition rates.

Simultaneous speech recognition and speaker identification

In this paper we present a self-learning speech controlled system comprising speech recognition, speaker identification and speaker adaptation for a small number of users, e.g. five recurring speakers. A compact representation of speech and speaker characteristics is discussed. It is combined with a technique for efficient information retrieval to capture individual speech characteristics allowing robust speaker identification with limited training data. Speech recognition is enhanced by applying speaker specific profiles which are incrementally adapted. However, the computational load and memory consumption are essential design parameters for an embedded system. Such a personalization of human-computer interfaces represents an important research issue. In this paper in-car applications such as speech controlled navigation, hands-free telephony or infotainment systems are investigated. Results for a subset of the SPEECON database are presented. They validate the benefit of the unified modeling of speech and speaker characteristics.

Evaluation of two approaches for speaker specific speech recognition

In this paper we examine two approaches for the automatic personalization of speech controlled systems. Speech recognition may be significantly improved by continuous speaker adaptation if the speaker can be reliably tracked. We evaluate two approaches for speaker identification suitable to identify 5-10 recurring users even in adverse environments. Only a very limited amount of speaker specific data can be used for training. A standard speaker identification approach is extended by speaker specific speech recognition. Multiple recognitions of speaker identity and spoken text are avoided to reduce latencies and computational complexity. In comparison, the speech recognizer itself is used to decode spoken phrases and to identify the current speaker in a single step. The latter approach is advantageous for applications which have to be performed on embedded devices, e.g. speech controlled navigation in automobiles. Both approaches were evaluated on a subset of the SPEECON database which represents realistic command and control scenarios for in-car applications.

Adaptive systems for unsupervised speaker tracking and speech recognition

Speech recognition offers an intuitive and convenient interface to control technical devices. Improvements achieved through ongoing research activities enable the user to handle increasingly complex tasks via speech. For special applications, e.g. dictation, highly sophisticated techniques have been developed to yield high recognition accuracy. Many use cases, however, are characterized by changing conditions such as different speakers or time-variant environments. A manifold of approaches has been published to handle the problem of changes in the acoustic environment or speaker specific voice characteristics by adapting the statistical models of a speech recognizer and speaker tracking. Combining speaker adaptation and speaker tracking may be advantageous, because it allows a system to adapt to more than one user at the same time. The performance of speech controlled systems may be continuously improved over time. In this article we review some techniques and systems for unsupervised speaker tracking which may be combined with speech recognition. We discuss a unified view on speaker identification and speech recognition embedded in a self-learning system. The latter adapts individually to its main users without requiring additional interventions of the user such as an enrollment. Robustness is continuously improved by progressive speaker adaptation. We analyze our evaluation results for a realistic in-car application to validate the evolution of the system in terms of speech recognition accuracy and identification rate.

Detection of unknown speakers in an unsupervised speech controlled system

In this paper we investigate the capability of our self-learning speech controlled system comprising speech recognition, speaker identification and speaker adaptation to detect unknown users. Our goal is to enhance automated speech controlled systems by an unsupervised personalization of the human-computer interface. New users should be allowed to use a speech controlled device without the need to identify themselves or to undergo a time-consumptive enrollment. Instead, the system should detect new users during the operation of the device. New speaker profiles should be initialized and incrementally adjusted without any additional intervention of the user. Such a personalization of humancomputer interfaces represents an important research issue. Exemplarily, in-car applications such as speech controlled navigation, hands-free telephony or infotainment systems are investigated. Results for detecting unknown speakers are presented for a subset of the SPEECON database.

Evolution of an adaptive unsupervised speech controlled system

In this paper we present a self-learning speech controlled system comprising speech recognition, speaker identification and speaker adaptation. Our goal is the automatic personalization of speech controlled devices for some five recurring speakers without the requirement of a speaker specific training. We present a novel approach to detect unknown speakers based on a unified speech and speaker model. New users are detected in an unsupervised manner based on only a few utterances. New speaker profiles are initialized without any additional intervention of the user. Each profile is continuously adapted by tracking the speaker identity on successive utterances to enhance future speech recognition and speaker identification. Experiments on the evolution of such a system were carried out on a subset of the SPEECON database. The results show that in the long run the system produces adaptation profiles which give improvements in speech recognition rate only slightly lower than supervised or closed-set systems.

Combined Speaker Adaptation

The first component of the target system is a flexible yet robust speaker adaptation to provide speaker specific statistical models. In this chapter a balanced adaptation strategy is motivated. New speaker profiles can be initialized and existing profiles can be continuously adapted in an unsupervised way. The algorithm is suitable for short-term adaptation based on a few utterances and long-term adaptation when sufficient training data is available. A first experiment has been conducted to investigate the improvement of a speech recognizer’s performance when speakers are optimally tracked so that the corresponding speaker models can be continuously adapted.

Unsupervised Speech Controlled System with Long-Term Adaptation

Chapters 2 and 3 provided a general overview of problems related to the scope of this book. The fundamentals and existing solutions as known from literature were discussed in detail.

Combining Self-Learning Speaker Identification and Speech Recognition

Chapter 2 introduced the fundamentals about speaker change detection, speaker identification, speech recognition and speaker adaptation. The basic strategies relevant for the problem of this book were explained.