Plamen J. Prodanov

Reliability-based decision fusion in multimodal biometric verification systems

We present a methodology of reliability estimation in the multimodal biometric verification scenario. Reliability estimation has shown to be an efficient and accurate way of predicting and correcting erroneous classification decisions in both unimodal (speech, face, online signature) and multimodal (speech and face) systems. While the initial research results indicate the high potential of the proposed methodology, the performance of the reliability estimation in a multimodal setting has not been sufficiently studied or evaluated. In this paper, we demonstrate the advantages of using the unimodal reliability information in order to perform an efficient biometric fusion of two modalities. We further show the presented method to be superior to state-of-the-art multimodal decision-level fusion schemes. The experimental evaluation presented in this paper is based on the popular benchmarking bimodal BANCA database.

Speaker Verification with Confidence and Reliability Measures

In pattern recognition, the need to quantify the quality of a classifiers output has gained importance in the past years. Speaker verification is no exception. This paper presents a probabilistic reliability framework incorporating signal-domain information into the confidence estimation and contrasts this method with classical approaches to estimating the confidence in a given speaker verification classifier output. We show that the method proposed can deal with adverse acoustic conditions for a wide range of signal-to-noise ratios, does not depend on a Gaussian assumption for impostor and client score distributions, and presents benefits in terms of scalability and interpretability of the measure. We contrast reliability and confidence approaches, and evaluate performance on a degraded version of the 295-users XM2VTS database

Bayesian networks based multi-modality fusion for error handling in human–robot dialogues under noisy conditions

Abstract In this paper, we introduce probabilistic model based architecture for error handling in human–robot spoken dialogue systems under adverse audio conditions. In this architecture, a Bayesian network framework is used for interpretation of multi-modal signals in the spoken dialogue between a tour-guide robot and visitors in mass exhibition conditions. In particular, we report on experiments interpreting speech and laser scanner signals in the dialogue management system of the autonomous tour-guide robot RoboX, successfully deployed at the Swiss National Exhibition (Expo.02). A correct interpretation of a user’s (visitor’s) goal or intention at each dialogue state is a key issue for successful voice-enabled communication between tour-guide robots and visitors. To infer the visitors’ goal under the uncertainty intrinsic to these two modalities, we introduce Bayesian networks for combining noisy speech recognition with data from a laser scanner, which are independent of acoustic noise. Experiments with real-world data, collected during the operation of RoboX at Expo.02 demonstrate the effectiveness of the approach in adverse environment. The proposed architecture makes it possible to model error-handling processes in spoken dialogue systems, which include complex combination of different multi-modal information sources in cases where such information is available.

Confidence and Reliability Measures in Speaker Verification

Speaker verification is a biometric identity verification technique whose performance can be severely degraded by the presence of noise. Using a coherent notation, we reformulate and review several methods which have been proposed to quantify the uncertainty in verification results, some with a view to coping with the effects of mismatched training-testing environments. We also include a recently proposed method, which is firmly rooted in a probabilistic approach and interpretation, and explicitly measures signal quality before assigning a reliability value to the speaker verification classifiers decision. We evaluate the performance of the confidence and reliability measures over a noisy 251-users database, showing that taking into account signal-domain quality can lead to better accuracy in prediction of classifier errors. We discuss possible strategies for using the measures in a speaker verification system, balancing acquisition duration and verification error rate.

On developing a voice-enabled interface for interactive tour-guide robots

This paper considers design methodologies in order to develop voice-enabled interfaces for tour-guide robots deployed at the Robotics Exposition of the Swiss National Exhibition (Expo.02). Human-robot voice communication presents new challenges for design of fully autonomous mobile robots, in that interactivity must be robot-initiated in conversation and within a dynamic adverse environment. We approached these general problems for a voice-enabled interface, tailored to limited computational resources of one on-board processor, when integrating smart speech signal acquisition, automatic speech recognition and synthesis, as well as a dialogue system into the multi-modal, multi-sensor interface for the expo tour-guide robot. We also focus on particular issues that needed to be addressed in voice-based interaction when planning specific tasks and research experiments for Expo.02, where tour-guide robots had to interact with hundreds of thousands of visitors over 6 months, 7 days a week, 10 h per day.