François Grondin

The ManyEars open framework

ManyEars is an open framework for microphone array-based audio processing. It consists of a sound source localization, tracking and separation system that can provide an enhanced speaker signal for improved speech and sound recognition in real-world settings. ManyEars software framework is composed of a portable and modular C library, along with a graphical user interface for tuning the parameters and for real-time monitoring. This paper presents the integration of the ManyEars Library with Willow Garage’s Robot Operating System. To facilitate the use of ManyEars on various robotic platforms, the paper also introduces the customized microphone board and sound card distributed as an open hardware solution for implementation of robotic audition systems.

WISS, a speaker identification system for mobile robots

This paper presents WISS, a speaker identification system for mobile robots integrated to ManyEars, a sound source localization, tracking and separation system. Speaker identification consists in recognizing an individual among a group of known speakers. For mobile robots, performing speaker identification in presence of noise that changes over time is one important challenge. To deal with this issue, WISS uses Parallel Model Combination (PMC) and masks to update in real-time the speaker models (obtained in clean conditions) to both additive and convolutive noises. The results show that the weighted rate of good speaker identifications is 96% on average for a Signal-to-Noise Ratio (SNR) of 16 dB, whereas it only decreases to 84% when the SNR drops to 2 dB.

Contact-Free Respiration Rate Monitoring Using a Pan–Tilt Thermal Camera for Stationary Bike Telerehabilitation Sessions

One typical remote consultation envisioned for in-home telerehabilitation involves having the patient exercise on a stationary bike. Making sure that the patient is breathing well while pedaling is of primary concern for the remote clinician. One key requirement for in-home telerehabilitation is to make the system as simple as possible for the patients, avoiding, for instance, to have them wear sensors and devices. This paper presents a contact-free respiration rate monitoring system measuring temperature variations between inspired and expired air in the mouth-nose region using thermal imaging. The thermal camera is installed on a pan-tilt unit and coupled to a tracking algorithm, allowing the system to keep track of the mouth-nose region as the patient exercises. Results demonstrate that the system works in real time even when the patient moves or rotates its head while exercising. Recommendations are also made to minimize limitations of the system, such as the presence of people in the background or when the patient is talking, for its eventual use in in-home telerehabilitation sessions.

Time difference of arrival estimation based on binary frequency mask for sound source localization on mobile robots

Localization of sound sources in adverse environments is an important challenge in robot audition. The target sound source is often corrupted by coherent broadband noise, which introduces localization ambiguities as noise is often mistaken as the target source. To discriminate the time difference of arrival (TDOA) parameters of the target source and noise, this paper presents a binary mask for weighted generalized cross-correlation with phase transform (GCC-PHAT). Simulation and experiments on a mobile robot suggest that the proposed technique improves TDOA discrimination. It also brings the additional benefit of modulating the computing load requirement according to voice activity.

Enhancing a beam+ telepresence robot for remote home care applications

The aging population is putting increasing pressure on health care systems in many developed countries, and maintaining quality of care while controlling costs becomes a major issue that needs to be addressed. With platforms now available at 2,000

Noise mask for TDOA sound source localization of speech on mobile robots in noisy environments

US, telepresence robots are one potential solution to provide remote care services to elders living in their homes. However, they need improved capabilities to make them more than simple “Skype on wheels” devices. To make telepresence robots suitable for remote home care applications, they must offer enhanced and robust functionalities such as autonomous navigation, artificial audition and vital sign monitoring. Therefore, we integrated such capabilities on a beam+ platform using a robot control architecture, demonstrating the feasibility of adding these capabilities on a commercial robot.

Multimodal biometric identification system for mobile robots combining human metrology to face recognition and speaker identification

Sound source localization is an important challenge for mobile robots operating in real life settings. Sound sources of interest, such as speech, are often corrupted by broadband coherent noise sound source(s) that are non-stationary during transitions between steady-state segments. The interfering noise introduces localization ambiguities leading to the localization of invalid sound sources. Masks to reduce such interferences perform well under stationary noise, but the performance degrades as localization of invalid sound sources generated by noise appear and disappear suddenly during transitions between steady-state. This paper presents a new mask based on speech non-stationarity to discriminate between the time difference of arrival (TDOA) of speech source and noise transition. Simulations and experiments on a mobile robot suggest that the proposed technique improve TDOA discrimination and reduces significantly localization of invalid sound sources caused by noise.

Integration framework for speech processing with live visualization interfaces

Recognizing a person from a distance is important to establish meaningful social interaction and to provide additional cues regarding the situations experienced by a robot. To do so, face recognition and speaker identification are biometrics commonly used, with identification performance that are influenced by the distance between the person and the robot. This paper presents a system that combines these biometrics with human metrology (HM) to increase identification performance and range. HM measures are derived from 2D silhouettes extracted online using a dynamic background subtraction approach, processing in parallel 45 front features and 24 side features in 400 ms compared to 38 front and 22 side features extracted in sequence in 30 sec by using the approach presented by Lin and Wang [1]. By having each modality identify a set of up to five possible candidates, results suggest that combining modalities provide better performance compared to what each individual modality provides, from a wider range of distances.

Lookupia: An Intelligent Real Estate Search Engine for Finding Houses Optimally Geolocated to Reach Points of Interest

Audition is a rich source of spatial, identity, linguistic and paralinguistic information. Processing all this information requires acquisition, processing and interpretation of sound sources, which are instantaneous, invisible and noisy signals. This can lead to different responses by the system in relation to the information perceived. This paper presents our first implementation of an integration framework for speech processing. Acquisition includes sound capture, sound source localization, tracking, separation and enhancement, and voice activity detection. Processing involves speech and emotion recognition. Interpretation consists of translating speech utterances into commands that can influence interaction through dialogue management and speech synthesis. The paper also describes two visualization interfaces, inspired by comic strips, to represent live vocal interactions in real life environments. These interfaces are used to demonstrate how the framework performs in live interactions and its use in a usability study.

Robust speech/non-speech discrimination based on pitch estimation for mobile robots

Lookupia is an intelligent real estate search engine for finding houses optimally geolocated to reach points of interest. It uses data from OpenStreetMap and most of public transit corporations that publish transit schedules in the General Transit Feed Specification (GTFS) format.