Patrick Kechichian

Model-based speech enhancement using a bone-conducted signal

Codebook-based single-microphone noise suppressors, which exploit prior knowledge about speech and noise statistics, provide better performance in nonstationary noise. However, as the enhancement involves a joint optimization over speech and noise codebooks, this results in high computational complexity. A codebook-based method is proposed that uses a reference signal observed by a bone-conduction microphone, and a mapping between air- and bone-conduction codebook entries generated during an offline training phase. A smaller subset of air-conducted speech codebook entries that accurately models the clean speech signal is selected using this reference signal. Experiments support the expected improvement in performance at low computational complexity.

Effects of noise on arousal in a speech communication setting

Speech communication in natural environments is often impaired by varying levels of ambient noise. Such noise can reduce speech intelligibility and make conversations more effortful, consequently causing an increase in arousal, frustration or stress in the partaking speakers. This contribution investigates the effects of background noise on arousal in a speech communication setting using collaborative tasks and examines the measurability of such detrimental effects through physiological signals heart rate variability and skin conductance. We focus on the differences in responses between the various noise conditions aiming to establish the sensitivity of the employed physiological measures. Furthermore, self-reported mental effort scores are used to assess the dependency of subjective mental effort requirements on background noise for the examined communication setting. Our results indicate that while mental effort scores show a significant positive relationship with background noise level, skin conductance and heart rate variability features, which are commonly employed for arousal state detection, may be inappropriate and lack sensitivity to distinguish communication settings differing solely by the level of background noise.

Robustness improvement of ultrasound-based sensor systems for speech communication

In recent years, auxiliary sensors have been employed to improve the robustness of emerging hands-free speech communication systems based on air-conduction microphones, especially in low signal-to-noise-ratio environments. One such sensor, based on ultrasound, captures articulatory movement information during speech production and has been used in a voice activity detector and also shown to improve the performance of speech recognizers. However, studies thus far have tested such sensors in ideal scenarios where only relevant articulatory information was assumed to be present. Therefore, in this paper the robustness of such sensors in realistic scenarios is investigated. Challenges arising from non-articulatory movements and other environmental influences captured by ultrasound sensors are discussed and strategies for their detection presented. Finally, the proposed strategies are evaluated in an ultrasound-based voice activity detector.

Utility of auxiliary sensor data for speech enhancement

In recent years, data from various auxiliary acoustic and nonacoustic sensors have been used for enhancing noisy speech. These include bone-conduction microphones, surface electromyographic sensors, ultrasonic imaging of facial movements, etc. The signal from such sensors is correlated with the speech signal to varying degrees, and unlike microphone data, is typically not affected by acoustic background noise, making its use attractive for speech enhancement. In this paper, we discuss the measurement of the utility of such data from an information-theoretic perspective, and quantify the information that is shared between clean speech and the auxiliary signal, which is not present in the observed noisy speech signal. The measure is applied to simultaneously recorded air-and bone-conducted speech data.