Luca Cristoforetti

Speaker localization in CHIL lectures: evaluation criteria and results

This work addresses the problem of automatic speaker localization and tracking in a real lecture scenario. Evaluation criteria recently adopted under CHIL and NIST benchmarking are outlined. Two speaker localization systems are described, which are based on the use of Generalized Cross Correlation Phase Transform analysis and Global Coherence Field. Benchmarking results, obtained on a set of 13 lectures, showed an average RMS error of about 30 cm in the speaker localization.

WOZ acoustic data collection for interactive TV

This paper describes a multichannel acoustic data collection recorded under the European DICIT project, during Wizard of Oz (WOZ) experiments carried out at FAU and FBK-irst laboratories. The application of interest in DICIT is a distant-talking interface for control of interactive TV working in a typical living room, with many interfering devices. The objective of the experiments was to collect a database supporting efficient development and tuning of acoustic processing algorithms for signal enhancement. In DICIT, techniques for sound source localization, multichannel acoustic echo cancellation, blind source separation, speech activity detection, speaker identification and verification as well as beamforming are combined to achieve a maximum possible reduction of the user speech impairments typical of distant-talking interfaces. The collected database permitted to simulate at preliminary stage a realistic scenario and to tailor the involved algorithms to the observed user behaviors. In order to match the project requirements, the WOZ experiments were recorded in three languages: English, German and Italian. Besides the user inputs, the database also contains non-speech related acoustic events, room impulse response measurements and video data, the latter used to compute three-dimensional positions of each subject. Sessions were manually transcribed and segmented at word level, introducing also specific labels for acoustic events.

The DIRHA-ENGLISH corpus and related tasks for distant-speech recognition in domestic environments

This paper introduces the contents and the possible usage of the DIRHA-ENGLISH multi-microphone corpus, recently realized under the EC DIRHA project. The reference scenario is a domestic environment equipped with a large number of microphones and microphone arrays distributed in space. The corpus is composed of both real and simulated material, and it includes 12 US and 12 UK English native speakers. Each speaker uttered different sets of phonetically-rich sentences, newspaper articles, conversational speech, keywords, and commands. From this material, a large set of 1-minute sequences was generated, which also includes typical domestic background noise as well as inter/intra-room reverberation effects. Dev and test sets were derived, which represent a very precious material for different studies on multi-microphone speech processing and distant-speech recognition. Various tasks and corresponding Kaldi recipes have already been developed. The paper reports a first set of baseline results obtained using different techniques, including Deep Neural Networks (DNN), aligned with the state-of-the-art at international level.

Use of parallel recognizers for robust in-car speech interaction

This paper refers to an activity under way at the speech recognition technology level for the development of a hands-free dialogue interaction system in the car environment. The use of a set of HMM recognizers, running in parallel, is being investigated in order to ensure low complexity, modularity, fast response, and to allow a real-time reconfiguration of the language models and grammars according to the policy indicated by natural language understanding and dialogue manager modules. A corpus of spontaneous speech interactions was collected using the Wizard-of-Oz method in a real driving situation with a microphone placed far from the driver. The use of parallel recognition units, each specialized on a given geographical domain, was explored using the resulting real corpus. Experiments show the advantage of selecting the recognized sentence according to the maximum likelihood among the active units when compared to the use of a single language model based on a very large vocabulary.

The SCENIC project: Environment-aware sound sensing and rendering

Abstract The SCENIC project is aimed at making the environment become an integral part of the acoustic system. The goal is to boost the performance of arrays of speakers and microphones and, in some cases, to enable applications that would not be possible otherwise. This paper describes how this can be achieved.

Real-time prototype for multiple source tracking through generalized state coherence transform and particle filtering

This real-time demo implements a novel method for tracking bidi-mensional TDOAs of several concurrent acoustic sources using a small number of microphones. A batch on-line frequency-domain ICA is applied to the mixture recorded by an array consisting of three or four microphones. The estimated mixing matrices are used to compute an approximated instantaneous kernel density of the time-delay propagation through the Generalized State Coherence Transform (GSCT). Tracking of the propagation parameters of each source is performed through multiple disjoint particle filters which estimate the posterior kernel density of the bidimensional time-delays based on the instantaneous GSCT function. The tracked 2D TDOAs are then used to drive a Blind Source Separation (BSS) algorithm that extracts the sound source selected by the user. The demo will be run in presence of up to five competitive sources.

Use of Multiple Speech Recognition Units in an In-car Assistance System

This chapter presents an advanced dialogue system based on in-car hands-free voice interaction, conceived for obtaining driving assistance and for accessing tourist information while driving. Part of the related activities aimed at developing this “Virtual Intelligent Codriver” are being conducted under the European VICO project. The architecture of the dialogue system is here presented, with a description of its main modules: Front-end Speech Processing, Recognition Engine, Natural Language Understanding, Dialogue Manager and Car Wide Web. The use of a set of HMM recognizers, running in parallel, is being investigated within this project in order to ensure low complexity, modularity, fast response, and to allow a real-time reconfiguration of the language models and grammars according to the dialogue context. A corpus of spontaneous speech interactions was collected at ITC-irst using the Wizard-of-Oz method in a real driving situation. Multiple recognition units specialized on geographical subdomains and simpler language models were experimented using the resulting corpus. This investigation shows that, in presence of large lists of names (e.g. cities, streets, hotels), the choice of the output with maximum likelihood among the active units, although a simple approach, provides better results than the use of a single comprehensive language model.