Kihyeon Kim

Extension of two-channel transfer function based generalized sidelobe canceller for dealing with both background and point-source noise

This paper describes an algorithm to suppress non-stationary noise as well as stationary noise in a speech enhancement system that employs a two-channel generalized sidelobe canceller (GSC). Our approach builds on recent advances in GSC design involving a transfer function ratio (TFR). The proposed system has four stages. The first stage estimates a new TFR along the acoustic paths from the non-stationary noise source to the microphones and the power of the stationary noise components. Second, the estimated power of the stationary noise components is used to execute spectral subtraction (SS) with respect to the input signals. Thirdly, the optimal gain is estimated for speech enhancement on the primary channel. In the final stage, an adaptive filter reduces the residual correlated noise components of the signal. These algorithmic improvements consistently give a better performance than a transfer function based GSC (TF-GSC) alone or a GSC with SS post-filtering under various noise conditions while slightly increasing the computational complexity.

Dual-Channel Speech Enhancement Using a Perceptual Filterbank for Hands-Free Communication

We investigate a dual-channel speech enhancement method using perceptual adaptive noise suppressor, which improves perceptual quality of speech in automobile environment for hands-free communication. In particular, the perceptual adaptive noise suppressor, which is composed of a Mel-based perceptual filterbank, an adaptive filter, and a speech modification block, estimates the envelope of the desired speech by suppressing the nonspeech components. Experiments indicate that the proposed scheme shows 8.06 dB of NR improvement and 0.70 of PESQ score improvement compared to the Transfer Function Generalized Sidelobe Canceller structure alone.

Design of Audio-Visual Interface for Aiding Driver’s Voice Commands in Automotive Environment

This chapter describes an information-modeling and integration of an embedded audio-visual speech recognition system, aimed at improving speech recognition under adverse automobile noisy environment. In particular, we employ lip-reading as an added feature for enhanced speech recognition. Lip motion feature is extracted by active shape models and the corresponding hidden Markov models are constructed for lip-reading . For realizing efficient hidden Markov models, tied-mixture technique is introduced for both visual and acoustical information. It makes the model structure simple and small while maintaining suitable recognition performance. In decoding process, the audio-visual information is integrated into the state output probabilities of hidden Markov model as multistream features . Each stream is weighted according to the signal-to-noise ratio so that the visual information becomes more dominant under adverse noisy environment of an automobile. Representative experimental results demonstrate that the audio-visual speech recognition system achieves promising performance in adverse noisy condition, making it suitable for embedded devices.