Wei-Han Liu

Robust speaker's location detection in a vehicle environment using GMM models

Human-computer interaction (HCI) using speech communication is becoming increasingly important, especially in driving where safety is the primary concern. Knowing the speakers location (i.e., speaker localization) not only improves the enhancement results of a corrupted signal, but also provides assistance to speaker identification. Since conventional speech localization algorithms suffer from the uncertainties of environmental complexity and noise, as well as from the microphone mismatch problem, they are frequently not robust in practice. Without a high reliability, the acceptance of speech-based HCI would never be realized. This work presents a novel speakers location detection method and demonstrates high accuracy within a vehicle cabinet using a single linear microphone array. The proposed approach utilize Gaussian mixture models (GMM) to model the distributions of the phase differences among the microphones caused by the complex characteristic of room acoustic and microphone mismatch. The model can be applied both in near-field and far-field situations in a noisy environment. The individual Gaussian component of a GMM represents some general location-dependent but content and speaker-independent phase difference distributions. Moreover, the scheme performs well not only in nonline-of-sight cases, but also when the speakers are aligned toward the microphone array but at difference distances from it. This strong performance can be achieved by exploiting the fact that the phase difference distributions at different locations are distinguishable in the environment of a car. The experimental results also show that the proposed method outperforms the conventional multiple signal classification method (MUSIC) technique at various SNRs.

Speaker attention system for mobile robots using microphone array and face tracking

This paper presents a real-time human-robot interface system (HRIS), which processes both speech and vision information to improve the quality of communication between human and an autonomous mobile robot. The HRIS contains a real-time speech attention system and a real-time face tracking system. In the speech attention system, a microphone-array voice acquisition system has been developed to estimate the direction of speaker and purify the speakers speech signal in a noisy environment. The developed face tracking system aims to track the speakers face under illumination variation and react to the face motion. The proposed HRIS can provide a robot with the abilities of finding a speakers direction, tracking the speakers face, moving its body to the speaker, focusing its attention to the speaker who is talking to it, and purifying the speakers speech. The experimental results show that the HRIS not only purifies speech signal with a significant performance, but also tracks a face under illumination variation in real-time

A robust adaptive speech enhancement system for vehicular applications

This work proposes and implements a novel and robust adaptive speech enhancement system, which contains both time domain and frequency domain beamformers using Hinfin filtering approach to provide a clean and undisturbed speech waveform and improve the speech recognition rate in vehicle environments. A microphone array data acquisition hardware is also designed and implemented for the proposed speech enhancement system. Mutually matched microphones are needed for traditional multidimensional noise reduction methods, but this requirement is not practical for consumer applications from the cost standpoint. To overcome this issue, the proposed system adapts the mismatch dynamics to maintain the theoretical performance allowing unmatched microphones to be used in an array. Furthermore, to achieve a high speech recognition performance, the speech recognizer is usually required to be retrained for different vehicle environments due to different noise characteristics and channel effects. The proposed system using the Hinfin filtering approach, which makes no assumptions about noise and disturbance, is robust to the modeling error in a channel recovery process. Consequently, the real vehicular experimental results show that the proposed frequency domain beamformer provides a satisfactory speech recognition performance without the need to retrain the speech recognizer

A Robust Speech Enhancement System for Vehicular Applications Using H/spl infin/ Adaptive Filtering

This work proposes a novel and robust adaptive speech enhancement system, which contains both time-domain and frequency-domain beamformers using Hinfin filtering approach in vehicle environments. A corresponding microphone array data acquisition hardware is also designed and implemented. Traditionally, mutually matched microphones are needed, but this requirement is not practical. To conquer this issue, the proposed system adapts the mismatch dynamics to allow unmatched microphones to be used in an array. Furthermore, to achieve a satisfactory speech recognition performance, the speech recognizer is usually required to be retrained for different vehicle environments due to different noise characteristics and channel effects. The channel effect usually causes the modeling error in a channel recovery process because of the long channel response. The proposed system using the Hinfin filtering approach, which makes no assumptions about noise and disturbance, is robust to the modeling error. Consequently, the proposed frequency-domain beamformer provides a satisfactory performance without the need to retrain the speech.

Processing of speech signals using a microphone array for intelligent robots

Abstract For intelligent robots to interact with people, an efficient human-robot communication interface is very important (e.g. voice command). However, recognizing voice command or speech represents only part of speech communication. The physics of speech signals includes other information, such as speaker direction. Secondly, a basic element of processing the speech signal is recognition at the acoustic level. However, the performance of recognition depends greatly on the reception. In a noisy environment, the success rate can be very poor. As a result, prior to speech recognition, it is important to process the speech signals to extract the needed content while rejecting others (such as background noise). This paper presents a speech purification system for robots to improve the signal-to-noise ratio of reception and an algorithm with a multidirection calibration beamformer.

Indoor sound field feature matching for robot's location and orientation detection

In this work, an indoor sound field feature matching method is proposed and is applied to detect a mobile robots location and orientation. The sound field feature, captured from a sound source to a pair of microphones, contains the dynamic of the propagation path. Because of the complexity of indoor environment, the features from different path can be distinguished using appropriate models. Gaussian mixture models are utilized in this paper to characterize the phase difference and magnitude ratio distributions between the microphone pair in consecutive data frames. The application provides an alternative thinking compared with traditional methods such as direction of arrival (DOA) using propagation delay. They usually suffer from reverberation, non-line-of-sight and microphone mismatch problems. The experimental results show the method not only has a high recognition rate for robots location and orientation, but also is robust against environmental noise.

Gaussian mixture-sound field landmark model for robot localization applications

This investigation proposes a robust robot localization system. The system contains a novel Gaussian mixture-sound field landmark model (GM-SFLM) and can localize the robot accurately in noisy environments. Moreover, the proposed method depends nothing on the geometry relation between source locations and two microphones; it is able to cover both near-field and far-field problems. With this proposed GM-SFLM, we can localize robot in 2-dimensional indoor environments. Furthermore, we realize the GM-SFLM into a quadruped robot system composed of an eRobot and a robot agent by using embedded Ethernet technology. The experiment demonstrates that when the robot is completely non-line-of-sight, this system still provides high detection accuracy. Additionally, the proposed method has advantages of high accuracy, low-cost, easy to implement and environmental adaptation.

Robust speaker's location estimation in a vehicle environment using GMM models

In this work, a robust speakers location estimation method in a vehicle environment is presented. This method applies Gaussian mixture models (GMM) to the phase information obtained from a microphone array. The individual Gaussian component of a GMM represents some general location-dependent phase difference distribution between two microphones. These distributions are effective in modeling the speakers location. The relation between geometry of microphone array and frequency band is taken into consideration to avoid aliasing problems. The proposed approach provides an accurate estimation even in near-field, noisy and complex vehicle environment. Moreover, it performs well not only in non-line-of-sight cases, but also in the conditions that the speakers are aligned in a direction to the microphone array with difference distances. Experiments are conducted in a mini-van vehicle and the results show that the proposed method outperform the popular technique multiple signal classification method (MUSIC) in different SNR cases.

Fast communication: An investigation of time delay estimation in room acoustic environment using magnitude ratio

This paper investigates the relation between the nonstationary sound source and the frequency domain magnitude ratio of two microphones based on short-term frequency analysis. The fluctuation level of nonstationary sound sources is modeled by the exponent of polynomials from the concept of moving pole model. According to this model, the sufficient condition for utilizing the fluctuation level and magnitude ratio to estimate the time delay between two microphones is suggested. Simulation results are presented to show the performance of the suggested method.

A Robust Statistical-based Speaker's Location Detection Algorithm in a Vehicular Environment

This work proposes a threshold adaptation method to detect speakers location in a vehicular environment. The method is robust to unmodeled sound source locations in a noisy environment and uses a single linear microphone array. The proposed approach is an improvement over previous work which adopts Gaussian mixture models (GMMs) to model location-dependent but content and speaker independent acoustic characteristics of sound sources. Experimental results show that this scheme can overcome the far-filed and near-filed problem with the same architecture and perform well in both line-of-sight and non-line-of-sight cases