Yong-Joo Chung

Classification of Continuous Heart Sound Signals Using the Ergodic Hidden Markov Model

Recently, hidden Markov models (HMMs) have been found to be very effective in classifying heart sound signals. For the classification based on the HMM, the continuous cyclic heart sound signal needs to be manually segmented to obtain isolated cycles of the signal. However, the manual segmentation will be practically inadequate in real environments. Although, there have been some research efforts for the automatic segmentation, the segmentation errors seem to be inevitable and will result in performance degradation in the classification. To solve the problem of the segmentation, we propose to use the ergodic HMM for the classification of the continuous heart sound signal. In the classification experiments, the proposed method performed successfully with an accuracy of about 99(%) requiring no segmentation information.

A classification approach for the heart sound signals using hidden markov models

Stethoscopic auscultation is still one of the primary tools for the diagnosis of heart diseases due to its easy accessibility and relatively low cost. Recently, many research efforts have been done on the automatic classification of heart sound signals for supporting clinicians to make better heart sound diagnosis. Conventionally, automatic classification methods of the heart sound signals have been usually based on artificial neural networks (ANNs). But, in this paper, we propose to use hidden Markov models (HMMs) as the classification tool for the heart sound signal. In the experiments classifying 10 different kinds of heart sound signals, the proposed method has shown quite successful results compared with ANNs achieving average classification rate about 99%.

Compensation of SNR and noise type mismatch using an environmental sniffing based speech recognition solution

Multiple-model based speech recognition (MMSR) has been shown to be quite successful in noisy speech recognition. Since it employs multiple hidden Markov model (HMM) sets that correspond to various noise types and signal-to-noise ratio (SNR) values, the selected acoustic model can be closely matched with the test noisy speech, which leads to improved performance when compared with other state-of-the-art speech recognition systems that employ a single HMM set. However, as the number of HMM sets is usually limited due to practical considerations as well as effective model selection, acoustic mismatch can still be a problem in MMSR. In this study, we proposed methods to improve recognition performance by mitigating the mismatch in SNR and noise type for an MMSR solution. For the SNR mismatch, an optimal SNR mapping between the test noisy speech and the HMM was determined by experimental investigation. Improved performance was demonstrated by employing the SNR mapping instead of using the estimated SNR of the test noisy speech directly. We also proposed a novel method to reduce the effect of noise type mismatch by compensating the test noisy speech in the log-spectrum domain. We first derive the relation between the log-spectrum vectors in the test and training noisy speech. Since the relation is a non-linear function of the speech and noise parameters, the statistical information regarding the testing log-spectrum vectors was obtained by approximation using vector Taylor series (VTS) algorithm. Finally, the minimum mean square error estimation of the training log-spectrum vectors was used to reduce the mismatch between the training and test noisy speech. By employing the proposed methods in the MMSR framework, relative word error rate reduction of 18.7% and 21.3% was achieved on the Aurora 2 task when compared to a conventional MMSR and multi-condition training (MTR) method, respectively.

Vector Taylor series based model adaptation using noisy speech trained hidden Markov models

Novel relation between training and test noisy speech in the cepstrum domain is derived.Noisy speech trained HMM is adapted to the test speech using the noise corruption model.Better performance is observed compared with the state-of-the-art model adaptation methods.Easy to implement by adapting directly the conventional ML trained HMM. Conventionally, in vector Taylor series (VTS) based compensation for noise-robust speech recognition, hidden Markov models (HMMs) are usually trained with clean speech. However, it is known that better performance is generally obtained by training the HMM with noisy speech rather than clean speech. From this viewpoint, we propose a novel VTS-based HMM adaptation method for the noisy speech trained HMM. We derive a mathematical relation between the training and test noisy speech in the cepstrum-domain using VTS and the mean and covariance of the noisy speech trained HMM are adapted to the test noisy speech in an iterative expectation-maximization (EM) algorithm. In the experimental results on the Aurora 2 database, we could obtain about 10-25% relative improvements in word error rates (WERs) over multi-condition training (MTR) method depending on speech front-ends and the HMM complexity.

Using Kullback-Leibler Distance in Determining the Classes for the Heart Sound Signal Classification

Many research efforts have been done on the automatic classification of heart sound signals to support clinicians in heart sound diagnosis. Recently, hidden Markov models (HMMs) have been used quite successfully in the automatic classification of the heart sound signal. However, in the classification using HMMs, there are so many heart sound signal types that it is not reasonable to assign a new class to each of them. In this paper, rather than constructing an HMM for each signal type, we propose to build an HMM for a set of acoustically-similar signal types. To define the classes, we use the KL (Kullback-Leibler) distance between different signal types to determine if they should belong to the same class. From the classification experiments on the heart sound data consisting of 25 different types of signals, the proposed method proved to be quite efficient in determining the optimal set of classes.

Speech Recognition based on Environment Adaptation using SNR Mapping

Multiple-model based speech recognition framework (MMSR) has been known to be very successful in speech recognition. Since it uses multiple hidden Markov modes (HMMs) that corresponds to various noise types and signal-to-noise ratio (SNR) values, the selected acoustic model can have a close match with the test noisy speech. However, since the number of HMM sets is limited in practical use, the acoustic mismatch still remains as a problem. In this study, we experimentally determined the optimal SNR mapping between the test noisy speech and the HMM set to mitigate the mismatch between them. Improved performance was obtained by employing the SNR mapping instead of using the estimated SNR from the test noisy speech. When we applied the proposed method to the MMSR, the experimental results on the Aurora 2 database show that the relative word error rate reduction of 6.3% and 9.4% was achieved compared to a conventional MMSR and multi-condition training (MTR), respectively.

Speech feature compensation in multiple model based speech recognition system using vts-based environmental parameter estimation

Multiple-model based speech recognition (MMSR) has been shown to be quite successful in noisy speech recognition. In this study, we propose a method to improve recognition performance by mitigating the mismatch in noise/channel type for an MMSR solution. We propose a novel method to reduce the effect of noise and channel mismatch by compensating the test noisy speech in the log-spectrum domain. We derive the relation between the log-spectrum vectors in the test and training noisy speech by using vector Taylor series (VTS) algorithm. Based on it, minimum mean square error estimation of the training log-spectrum vectors is obtained from the test noisy vectors by iteratively estimating environmental parameters. The estimated training vectors are used for recognition to reduce the noise and channel mismatch. We could find that the proposed method achieved WER reduction based on the Aurora2 task by +18.7% compared with a conventional MMSR method.

Performance improvement in multiple-model speech recognizer under noisy environments

Multiple-model speech recognizer has been shown to be quite successful in noisy speech recognition. However, its performance has usually been tested using the general speech front-ends which do not incorporate any noise adaptive algorithms. For the accurate evaluation of the effectiveness of the multiple-model frame in noisy speech recognition, we used the state-of-the-art front-ends and compared its performance with the well-known multi-style training method. In addition, we improved the multiple-model speech recognizer by employing N-best reference HMMs for interpolation and using multiple SNR levels for training each of the reference HMM.

Data-Driven Jacobian Adaptation in a Multi-model Structure for Noisy Speech Recognition

We propose a data-driven approach for the Jacobian adaptation (JA) to make it more robust against the noisy environments in speech recognition. The reference hidden Markov model (HMM) in the JA is trained directly with the noisy speech for improved acoustic modeling instead of using the model composition methods like the parallel model combination (PMC). This is made possible by estimating the Jacobian matrices and other statistical information for the adaptation using the Baum-Welch algorithm during the training. The adaptation algorithm has shown to give improved robustness especially when used in a multi-model structure. From the speech recognition experiments based on HMMs, we could find the proposed adaptation method gives better recognition results compared with conventional HMM parameter compensation methods and the multi-model approach could be a viable solution in the noisy speech recognition.