Hong Kook Kim

A name recognition based call-and-come service for home robots

In this paper, we propose and implement an efficient robot name registration and recognition system in order to provide a call-and-come service for home robots. The service is designed to enable a robot to come to a user when its name is called correctly by the user. Therefore, techniques such as voice detection, robust speech recognition and detection of the location of the use are required. For efficient robot name registration by voice, the proposed method first restricts the search space for name registration by using monophone-based acoustic models. Then, the registration of robot names is completed using triphone-based acoustic models in the restricted search space. In order to provide a reliable service, the parameters for the robot name verification are calculated to reduce the acceptance rate of false calls. In addition, acoustic models are adapted by using a distance speech database to improve the performance of distance speech recognition. Moreover, the location of the user is estimated by using a microphone array. The experimental results for the registration and recognition of robot names show that the average word error rate (WER) of speech recognition is 1.7% in home environments, which is an acceptable WER for a real call- and-come service.

Immersive modeling system (IMMS) for personal electronic products using a multi-modal interface

In developing new personal electronic products, the development time has shortened to a few months due to high competition in the market. Now it has become very hard to meet the time to market by evaluating products by making physical prototypes. To overcome this problem, we propose an immersive modeling system (IMMS) that enables us to interact with a digital product model in the augmented virtual environment using a multi-modal interface. The IMMS allows the user to evaluate the product model realistically using visual, auditory, and tactile/force senses. The architecture and main modules of the system are described in detail. The integration problems encountered during the development of the test bed are discussed. Application examples to personal electronic products are also included.

Non-negative matrix factorization based noise reduction for noise robust automatic speech recognition

In this paper, we propose a noise reduction method based on non-negative matrix factorization (NMF) for noise-robust automatic speech recognition (ASR). Most noise reduction methods applied to ASR front-ends have been developed for suppressing background noise that is assumed to be stationary rather than non-stationary. Instead, the proposed method attenuates non-target noise by a hybrid approach that combines a Wiener filtering and an NMF technique. This is motivated by the fact that Wiener filtering and NMF are suitable for reduction of stationary and non-stationary noise, respectively. It is shown from ASR experiments that an ASR system employing the proposed approach improves the average word error rate by 11.9%, 22.4%, and 5.2%, compared to systems employing the two-stage mel-warped Wiener filter, the minimum mean square error log-spectral amplitude estimator, and NMF with a Wiener post-filter, respectively.

Probabilistic spectral gain modification applied to beamformer-based noise reduction in a car environment

In this paper, we propose a new noise reduction technique for improving the performance of conventional beamformers used in cars. To this end, the probabilistic discrimination on target-directional signal presence is used as a parameter for the spectral gain modification (SGM) of a beamformer output. The direction of arrival (DOA)-based target-to-non-target-directional signal ratio (TNR) is first estimated by using spatial cues such as phase differences from multiple microphone signals. Next, the estimated TNR is utilized to estimate the target-directional signal presence probabilities (TDSPPs) that include global and local terms. The performance of the proposed SGM is evaluated by the degree of noise reduction, average and segmental signal-to-noise ratio (SNR), as well as perceptual evaluation of speech quality (PESQ) scores under car noise conditions whose SNR varies from -5 to 20 dB. As a result, it is shown that the proposed SGM significantly improves the target-directional signal enhancing performance against conventional beamformers, i.e., delay-and-sum beamformer (DSB), super directive beamformer (SDB) and generalized sidelobe canceller (GSC), for all input SNRs.

Mechanical noise suppression based on non-negative matrix factorization and multi-band spectral subtraction for digital cameras

This paper proposes a new non-stationary noise suppression method to reduce the mechanical noise generated when audio signals are recorded with a digital camera. The proposed method first utilizes a non-negative matrix factorization (NMF) technique to estimate the noise spectrum of the mechanical noise from the noisy audio spectrum. After that, the mechanical noise contaminated in the audio signal is suppressed by multi-band spectral subtraction. In particular, the NMF technique estimates the noise spectrum in a frame-wise manner in order for the proposed method to operate in real-time. The performance of the proposed mechanical noise suppression method is evaluated in terms of log-spectral distortion, cepstral distortion, subjective quality, and computational complexity. In addition, it is compared with the performance of conventional methods. It is shown from the evaluation that the proposed method provides lower log-spectral and cepstral distortions with better subjective preference than conventional methods. Moreover, the complexity of the proposed method is low enough to be implemented on a commercially available digital camera.

Crosstalk Cancellation for Spatial Sound Reproduction in Portable Devices with Stereo Loudspeakers

To reproduce spatial sound through stereo loudspeakers in a portable device environment, it is important to properly design a crosstalk cancellation algorithm that cancels out acoustical crosstalk signals. In other words, the difference between the direct path of head-related transfer functions (HRTFs) and the crosstalk path of HRTFs is very small at certain frequencies, and this causes an excessive boost of frequencies when designing a crosstalk cancellation filter. To mitigate this problem, we propose a crosstalk cancellation filter design method that allows for the selective attenuation of unwanted peaks in the spectrum by constraining the magnitude of the difference between the direct and crosstalk path. The performance of the proposed method is evaluated by subjective source localization and objective tests. It is shown from the tests that the proposed method can provide improved spatial sound effects with very closely spaced stereo loudspeakers.

Hybrid probabilistic adaptation mode controller for generalized sidelobe canceller-based target-directional speech enhancement

In this paper, we propose a new adaptation mode controller (AMC) for a generalized sidelobe canceller (GSC) having prior knowledge of the direction-of-arrival (DOA) of a desired signal source. To this end, the DOA-based a posteriori target-to-non-target-directional signal ratio (TNR) is first estimated from spatial cues such as phase differences among multi-microphone signals. Next, the estimated TNR is utilized to estimate the target-directional-signal absence probability (TSAP) and presence probability (TSPP), which include global and local terms. The probabilities are then applied to control parameters of adaptive filters via AMC. The performance evaluation of target-directional speech by the proposed approach is carried out by the perceptual evaluation of speech quality (PESQ) scores, noise reduction (NR), and average signal-to-noise ratio (SNR) under car noise conditions with various SNRs from −5 to 20 dB. It is shown from the experiments that the proposed approach provides better results than the conventional AMC.

A smart background music mixing algorithm for portable digital imaging devices

In this paper, we propose a smart background music (BGM) mixing algorithm for portable digital imaging devices to enable users to enjoy video content with BGM. The proposed algorithm automatically adjusts the BGM output energy based on the activity and energy of foreground audio (FGA) contained in a video file. To this end, the proposed algorithm classifies each segment of FGA as speech, non-speech, or a mixed signal. After that, it estimates a scale factor for mixing FGA and BGM according to the signal classification result and the energy of FGA. In addition, a fade-in and fade-out process is incorporated in the proposed algorithm in order to improve the perceptual quality of output audio at the boundaries where signal classification is changed. In order to demonstrate the effectiveness of the proposed algorithm, we implement it on a portable digital imaging device in real time and compare the users preference of the proposed algorithm with those of conventional algorithms that mixes FGA with BGM based on voice activity detection or a predefined fixed scale factor. It is shown from the experiments that the proposed algorithm is pretty much preferred by around 79%, compared to the conventional algorithms.

Hybrid probabilistic adaptation mode controller for generalized sidelobe cancellers applied to multi-microphone speech enhancement

In this paper, we propose a new adaptation mode controller (AMC) for a generalized sidelobe canceller (GSC)-based speech enhancement system. Here, a likelihood ratio for target speech presence was first estimated and then utilized to estimate both the local target speech presence probability (SPP) and global SPP. Next, the estimated SPPs were applied to the design of an AMC that controlled the parameters of adaptive filters for an adaptive blocking matrix (ABM) and noise canceller (NC). In particular, the combination of local and global SPPs was applied to the AMC in the ABM, whereas only global SPPs were used for the NC. Finally, a multiple-microphone speech enhancement system was constructed on the basis of a GSC having the proposed AMC. The performance of the speech enhancement system was subsequently evaluated in terms of the perceptual evaluation of speech quality (PESQ) and the cepstral distortion (CD) for car noise conditions. It was shown from this evaluation that a speech enhancement system using the proposed AMC method provided better performance than conventional AMC methods using power ratios between the target and non-target directional signals, the inter-channel normalized cross-correlation, and the local SPPs only.

Nonnegative Matrix Factorization Based Adaptive Noise Sensing over Wireless Sensor Networks

An adaptive noise sensing method is proposed to improve the speech sensing performance of speech-based applications operated over wireless sensor networks. The proposed method is based on nonnegative matrix factorization (NMF), which consists of adaptive noise sensing and noise reduction. In other words, adaptive noise sensing is performed by adapting a priori noise basis matrix of the NMF, which is estimated from the noise signal, resulting in an adapted noise basis matrix. Subsequently, the adapted noise basis matrix is used for the NMF decomposition of noisy speech into clean speech and background noise. The estimated clean speech signal is then applied to a front-end of the speech-based applications. The performance of the proposed NMF-based noise sensing and reduction method is first evaluated by measuring the source to distortion ratio (SDR), the source to interferences ratio (SIR), and the source to artifacts ratio (SAR). In addition, the proposed method is applied to an automatic speech recognition (ASR) system, which is a typical speech-based application, and then the average word error rate (WER) of the ASR is compared with that employing either a Wiener filter, or a conventional NMF-based noise reduction method using only a priori noise basis matrix.