Changliang Liu

An SVM-Based Mandarin Pronunciation Quality Assessment System

This paper presents our Mandarin pronunciation quality assessment system for the examination of Putonghua Shuiping Kaoshi (PSK) and investigates a novel Support Vector Machine (SVM) based method to improve its assessment accuracy. Firstly, an selective speaker adaptation module is introduced, in which we select well pronounced speech from results of the first-pass automatic pronunciation scoring as the adaptation data, and adopt Maximum Likelihood Linear Regression to update the acoustic model (AM). Then, compared with the traditional triphone based AM, the monophone based AM is studied. Finally, we propose a new method of incorporating all kinds of posterior probabilities using SVM classifier. Experimental results show that the average correlation coefficient between machine and human scores is improved from 83.72% to 85.48%. It suggests that the two methods of selective speaker adaptation and multi-model combination using SVM are very effective to improve the accuracy of pronunciation quality assessment.

Combining Phoneme Loop Posteriori with Decoding Posteriori as Confidence Measure for Speech Recognition in E-service

This paper presents our confidence measure system for speech recognition to integrate with e-Service to make Human-Computer Interaction more convenient. In order to make the system more robust for practical usage, the confidence measure is optimized to improve its performance as well as speed, compared with traditional state based confidence measure. First, the decoding likelihood of the best path is normalized with all the survival paths to form the onepass-based posteriori. After decoding, when recognition result is available as well as the phoneme level division point, the phoneme loop posteriori based confidence is calculated. Different models are compared for speed and performance. Then they are combined to form the final confidence for the judgement. Experiments are designed, and the proposed confidence measure get a relative improvement of 20%, 19.33% for equal error rate and 37.19%, 35.17% of false acceptance rate for out-of-vocabulary set on the development sets and the test sets, with no loss of false rejection rate for in-vocabulary set.

An Effective CALL System for Strongly Accented Mandarin Speech

In this paper, we investigate some specific acoustic problems of the computer assisted language learning (CALL) system by modifying the acoustic model and feature under the speech recognition framework. At first, in order to alleviate the distortion of channel and speaker, speaker-dependent Cepstrum Mean Normalization (Speaker CMN) is adopted, by which the average correlation coefficient (ACC) between human and machine scores is improved from 78.00% to 84.14%. Then, Heteroscedastic Linear Discriminate Analysis (HLDA) is applied to enhance the discrimination ability of acoustic model, which successfully increases ACC from 84.14% to 84.62%. Additionally, HLDA can lessen the great human-machine scoring difference of speeches that have very good or too bad quality, and so leads to an increase of the correctly-rank rate from 85.59% to 90.99%. Finally, we use the technology of Maximum a Posteriori (MAP) to tune the acoustic model to match the strongly accented testing speech. As the result, ACC is improved from 84.62% to 86.57%.

Application of LVCSR to the Detection of Chinese Mandarin Reading Miscues

For a reading tutor, the most important task is to detect the reading miscues such as insertions, omissions, etc. This paper constructed a Chinese reading miscues detection system based on technologies of general large vocabulary continuous speech recognition and proposed two methods to improve the performance of the detection. The first is to align the reference to the confusion network resulted from the recognition instead of the 1-best result to find out the reading miscues. And the second is using the knowledge of reference to regulate the decoding by weighting the n-gram language model probability, if it exists in the reference. The experiments on a Chinese Mandarin reading corpus proved the effectiveness of these two modifications. The detection MDerr and FArate are depressed 50.1% and 70.8% totally by these two methods.

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

In this paper we present our investigation into improving the performance of our computer-assisted language learning (CALL) system through exploiting the acoustic model and features within the speech recognition framework. First, to alleviate channel distortion, speaker-dependent cepstrum mean normalization (CMN) is adopted and the average correlation coefficient (average CC) between machine and expert scores is improved from 78.00% to 84.14%. Second, heteroscedastic linear discriminant analysis (HLDA) is adopted to enhance the discriminability of the acoustic model, which successfully increases the average CC from 84.14% to 84.62%. Additionally, HLDA causes the scoring accuracy to be more stable at various pronunciation proficiency levels, and thus leads to an increase in the speaker correct-rank rate from 85.59% to 90.99%. Finally, we use maximum a posteriori (MAP) estimation to tune the acoustic model to fit strongly accented test speech. As a result, the average CC is improved from 84.62% to 86.57%. These three novel techniques improve the accuracy of evaluating pronunciation quality.

Some acoustic improvements for pronunciation quality assessment for strongly accented mandarin speech

This paper presents our recent study in resolving some specific acoustic problems of the computer assisted language learning (CALL) system by modifying the acoustic model (AM) and feature under ASR framework. Firstly, speaker dependent cepstrum mean normalization (Speaker CMN) is adopted to alleviate the distortion of channel, with which the average human-machine scoring correlation coefficient (ACC) is improved from 78.00% to 84.14%. Heteroscedastic linear discriminate analysis (HLDA) is then applied to enhance the discrimination ability of AM, which successfully increases ACC from 84.14% to 84.62%. Additionally, HLDA can lessen the great human-machine scoring difference of those speeches that have very good or too bad pronunciation quality, and so lead to an increase of the correctly-rank rate (CRR) from 85.59% to 90.99%. Finally, we use maximum a posteriori (MAP) to tune AM to match the strong accented test speech. As the result, ACC is improved from 84.62% to 86.57%.

An Mandarin Pronunciation Quality Assessment System Using Two Kinds of Acoustic Models

This paper presents our Mandarin pronunciation quality assessment system for the examination of Putonghua Shuiping Kaoshi (PSK) and investigates some measures to improve the assessment accuracy. In this paper, a selective speaker adaptation method is studied. In the adaptation module, we select well pronounced speech as the adaptation data, and adopt Maximum Likelihood Linear Regression (MLLR) to update the speaker-independent (SI) acoustic model. Besides the triphone based acoustic model, the monophone based acoustic model is also applied to our system. Further improvements are obtained by combining posterior probabilities computed with triphone and monophone based acoustic models using Support Vector Machine (SVM) to assess the goodness of pronunciations. The experiment results show that the average correlation coefficient (ACC) between machine and the human scores achieves 0.8549, almost equivalent to ACC between different experts. The improved system achieves usable performance in actual applications.

Forward optimal measures for automatic mispronunciation detection

Pronunciation measure computation is a vital part of Computer Assisted Pronunciation Training (CAPT) system. This paper conducts some research on pronunciation measures based on the two popular measures - Log posterior probability (LPP) and Goodness of Pronunciation (GOP). A modified GOP - AGOP is proposed which directly uses the segmentation information of forced alignment instead of free phone recognizer (FPR) when computing the denominator of GOP to avoid the effect of inaccuracy of FPR. The context dependent acoustic models is investigated in mispronunciation detection. It is found that Tri-phone AM has better performance in mispronunciation detection of continuous speech. This paper also proposes a fast algorithm of pronunciation measure - FAGOP which uses the maximization instead of summation to calculate the denominator of AGOP approximately and applies Viterbi algorithm with some effective pruning strategy to reduce the computation perplexity. It achieves much better efficiency while barely impairing the detection presicion.

Dynamic Multiple Pronunciation Incorporation in a Refined Search Space for Reading Miscue Detection

Error prediction is important for detecting reading miscues by a reading tutor. In order to incorporate the error prediction into the decoder of a conventional speech recognizer, this paper proposes an algorithm of Dynamic Multiple Pronunciation Incorporation (DMPI). It solves the confliction between the coverage of errors and the perplexity increase of search space. A multiple pronunciation model (MPM) is developed to model the misreading errors. The pronunciation variants referred to in current reference are extracted from MPM and added to the search space of the recognizer – a refined state network before recognizing. The original state network is redeveloped to reserve some redundant fan-in and fan-out nodes which make the merging of the original state network and the additional state network very easy. The experiment result proved effectiveness of this algorithm. The EER is decreased by about 9.5%.

An LVCSR Based Reading Miscue Detection System with Knowledge of Reference and Error Patterns Dynamically Incorporated

This paper constructs a reading miscue detection system based on conventional Large Vocabulary Continuous Speech Recognition (LVCSR) framework. In order to incorporate the knowledge of reference (what the reader ought to read) and some error patterns into the decoding process, two methods are proposed: Dynamic Interpolation of Language Model (DILM) and Dynamic Multiple Pronunciation Incorporation (DMPI). DILM dynamically interpolates the general language model based on analysis of reference and so restricts the active paths of decoding not too far away from the reference. It makes the recognition more accurate, which further improve the detection performance. DMPI dynamically adds some pronunciation variations into the search space to predict reading substitutions. It solves the con¿iction between the coverage of error predictions and the perplexity the search space. The experimental results show that the proposed two methods can totally decrease EER by 14% relatively totally, from 46.4% to 39.8%.