Antonius Kalker

On the quality assessment of sound signals

This paper constitutes an introduction to the field of quality evaluation of sound (speech and audio) signals. The need for such an assessment is inherent to modern communications: VoIP, mobile phone, or teleconference systems require meaningful measures of performance, which may ultimately assure good service or profitable business. A brief survey on subjective and objective evaluation methods is provided. Recent developments as well as new topics to be investigated are also addressed. Experiments are conducted to illustrate how to validate quality assessment methods.

Visual conditioning for augmented-reality-assisted video conferencing

Typical video conferencing scenarios bring together individuals from disparate environments. Unless one commits to expensive tele-presence rooms, conferences involving many individuals result in a cacophony of visuals and backgrounds. Ideally one would like to separate participant visuals from their respective environments and render them over visually pleasing backgrounds that enhance immersion for all. Yet available image/video segmentation techniques are limited and result in significant artifacts even with recently popular commodity depth sensors. In this paper we present a technique that accomplishes robust and visually pleasing rendering of segmented participants over adaptively-designed virtual backgrounds. Our method works by determining virtual backgrounds that match and highlight participant visuals and uses directional textures to hide segmentation artifacts due to noisy segmentation boundaries, missing regions, etc. Taking advantage of simple computations and look-up-tables, our work leads to fast, real-time implementations that can run on mobile and other computationally-limited platforms.

Video Based VAD Using Adaptive Color Information

This paper presents a new approach for Voice Activity Detection (VAD) in videoconferencing applications based only on video information. In the proposed approach, a face detection scheme is applied to locate the face of the participant, and anthropometric measures are used to identify one patch below each eye, as well as a neighborhood around the mouth. The patches below the eyes are used to train a skin color model suited for that specific person, which is then used to identify non-skin pixels in the mouth neighborhood. The number of non-skin pixels is taken as an estimate of the mouth openness, and its evolution across time is explored for VAD.

Phase-Domain Statistical Analysis for Audio Source Localization

We consider the problem of estimating the time-difference-of-arrival (TDOA) for audio source localization in noisy environments, defining a framework for statistical analysis in the phase domain which enables more reliable estimates. This is motivated by the fact that with complex sources, noise, and interfering signals, different frequency bands have significantly different signal-to-noise ratios, creating non-uniform distributions of errors in the phase measurements. Through a new method for analysis of the variations of the phase in frequency windows, we first estimate the signal-to-noise ratio for frequency, and then use it in a maximum-likelihood estimation of the time difference of arrival. We show that this corresponds to a generalization of the Phase Transform method (PHAT), and provides a theoretical justification of why it works so well. Numerical results show how the proposed technique compares favorably with PHAT.