Jeon Gue Park

GenieTutor: A Computer-Assisted Second-Language Learning System Based on Spoken Language Understanding

This paper introduces a computer-assisted second-language learning system using spoken language understanding. The system consists of automatic speech recognition, semantic/grammar correction evaluation, and tutoring module. The speech recognition is optimized for non-natives as well as natives for educational purpose and smooth interaction. Semantic/grammar correction evaluation evaluates whether the non-native learners utterance is appropriate semantically and is correct grammatically. Tutoring module decides to go to the next turn or ask the learner to try again, and also provides a turn-by-turn corrective feedback using evaluation results. We constructed English learning service consisting of three stages such as Pronunciation Clinic, Think&Talk and Look&Talk using the system.

Fast speech recognition to access a very large list of items on embedded devices

In this paper, we propose a fast decoding algorithm to recognize a very large number of item names on a resource-limited embedded device. The proposed algorithm is based on a multi-pass search scheme. The algorithm is composed of a two-stage HMM-based coarse match and a detailed match. The two-stage HMM-based coarse match is aimed at rapidly selecting a small set of candidates that are assumed to contain a correct hypothesis with high probability, and the detailed match re-ranks the candidates by performing acoustic rescoring. The proposed algorithm is implemented on an in-car navigation system with a 32-bit fixed-point processor operating at 620 MHz. The experimental result shows that the proposed method runs at maximum speed 1. 74 times real-time on the embedded device while minimizing the degradation of the recognition accuracy for a 220 K Korean Point-of-Interest (POI) recognition domain.

A Case study of Edutainment Robot: Applying Voice Question Answering to Intelligent Robot

This paper presents a case study of edutainment robot, which is an intelligent robot for educational use with a voice-QA model applied. The emphatic functions of our robot are: analyzing spoken question from a student, finding an appropriate answer in Korean encyclopedia, and then serving the answer with speech synthesis. We develop the ESTk, which is an Automatic Speech Recognition (ASR) system based on Finite State Network (FSN) for processing Korean spoken questions. For answer extraction, we utilize machine learning techniques and pattern extraction method. With our live-update interaction method, our robot can be extended with new knowledge in real-time. By conducting a quiz game, we show a possibility of our robot as an edutainment robot.

Deep neural network using trainable activation functions.

This paper proposes trainable activation functions for deep neural network (DNN). A DNN is a feed-forward neural network composed of more than one hidden nonlinear layer. It is characterized by a set of weight matrices, bias vectors, and a nonlinear activation function. In model parameter training, weight matrices and bias vectors are updated using an error back-propagation algorithm but activation functions is not. It is just fixed empirically. Many rectifier-type nonlinear functions have been proposed as activation functions, but the best nonlinear functions for any given task domain remain unknown. In order to address the issue, we propose a trainable activation function. In the proposed approach, conventional nonlinear activation functions were approximated for a Taylor series, and the coefficients were retrained simultaneously with other parameters. The effectiveness of the proposed approach was evaluated for MNIST handwritten digit recognition domain.

Deep neural network based acoustic model parameter reduction using manifold regularized low rank matrix factorization

In this paper, we propose a deep neural network (DNN) model parameter reduction based on manifold regularized low rank matrix factorization to reduce the computational complexity of acoustic model for low resource embedded devices. One of the most common DNN model parameter reduction techniques is truncated singular value decomposition (TSVD). TSVD reduces the number of parameters by approximating a target matrix with a low rank one in terms of minimizing the Euclidean norm. In this work, we questioned whether the Euclidean norm is appropriate as objective function to factorize DNN matrices because DNN is known to learn nonlinear manifold of acoustic features. Therefore, in order to exploit the manifold structure for robust parameter reduction, we propose manifold regularized matrix factorization approach. The proposed method was evaluated on TIMIT phone recognition domain.

Phonetic state relation graph regularized deep neural network for robust acoustic model

In this paper, we propose a phonetic state relation graph regularized Deep Neural Network (DNN) for a robust acoustic model. A DNN-based acoustic model is trained in terms of minimizing a cost function that is usually penalized by regularizations. Regularization generally reflects prior knowledge that plays a role in constraining the model parameter space. In DNN-based acoustic models, various regularizations have been proposed to improve robustness. However, most approaches do not handle speech generation knowledge even if this process is the most fundamental prior. For example, l1 and l2-norm regularizations are equivalent to set Gaussian prior and Laplacian prior to model parameters respectively. This means that any speech signal specific knowledge is not used for regularization. Manifold-based regularization exploits the local linear structure of observed acoustic features, which are simply realization of the speech generation process. Therefore, to incorporate prior knowledge of speech generation into regularization, we propose a phonetic state relation graph based approach. This method was evaluated on the TIMIT phone recognition domain. The results showed that it reduced the phone error rate from 20.8% to 20.3% under the same conditions.

I-vector based utterance verification for large-vocabulary speech recognition system

This paper proposes a new Utterance Verification (UV) algorithm based on i-vector. Phone segments are extracted and concatenated from the training data, which are used to train the Universal Background Model (UBM) and the Total Variability (TV) matrix, and then, i-vector is extracted from the enrollment and evaluation data using UBM and TV matrix. We compare two Confidence Measures (CMs), cosine distance scoring and Support Vector Machine (SVM). To compensate the channel effect, we use two channel compensation methods, Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN). The decision is made by the word-level CM by combining the phone-level CMs. Experiments are conducted in the Korean isolated word recognition domain. Experimental results show that SVM is superior to cosine distance scoring. Best performance is achieved when SVM is used without any channel compensation method.

Implementation of CNN in the view of mini-batch DNN training for efficient second order optimization

This paper describes some implementation schemes of CNN in view of mini-batch DNN training for efficient second order optimization. This uses same procedure updating parameters of DNN to train parameters of CNN by simply arranging an input image as a sequence of local patches, which is actually equivalent with mini-batch DNN training. Through this conversion, second order optimization providing higher performance can be simply conducted to train the parameters of CNN. In both results of image recognition on MNIST DB and syllable automatic speech recognition, our proposed scheme for CNN implementation shows better performance than one based on DNN.

A useful feature-engineering approach for a LVCSR system based on CD-DNN-HMM algorithm

In this paper, we propose a useful feature-engineering approach for Context-Dependent Deep-Neural-Network Hidden-Markov-Model (CD-DNN-HMM) based Large-Vocabulary-Continuous-Speech-Recognition (LVCSR) systems. The speech recognition performance of a LVCSR system is improved from two feature-engineering perspectives. The first performance improvement is achieved by adopting the intra/inter-frame feature subsets when the Gaussian-Mixture-Model (GMM) HMMs for the HMM state-level alignment are built. And the second performance gain is then followed with the additional features augmenting the front-end of the DNN. We evaluate the effectiveness of our feature-engineering approach under a series of Korean speech recognition tasks (isolated single-syllable recognition with a medium-sized speech corpus and conversational speech recognition with a large-sized database) using the Kaldi speech recognition toolkit. The results show that the proposed feature-engineering approach outperforms the traditional Mel Frequency Cepstral Coefficient (MFCCs) GMM + Mel-frequency filter-bank output DNN method.

Speech Recognition Based Pronunciation Evaluation Using Pronunciation Variations and Anti-models for Non-native Language Learners

This paper proposes a speech recognition based automatic pronunciation evaluation method using pronunciation variations and anti-models for non-native language learners. To this end, the proposed pronunciation evaluation method consists of (a) speech recognition step and (b) pronunciation analysis step. As a first step, a Viterbi decoding algorithm is performed with a multiple pronunciation dictionary for non-native language learners, which is generated in an indirect data-driven method. As a result, the phoneme sequence, log-likelihoods of the acoustic models and anti-models and the duration of each phoneme are obtained for an input speech. As a second step, each recognized phoneme is evaluated using the speech recognition results and the reference phoneme sequence. For the automatic pronunciation evaluation experiments, we select English as a target language and Korean speakers as non-native language learners. Moreover, it is shown from the experiments that the proposed method achieves the average value between a false rejection rate (FRR) and a false alarm rate (FAR) as 32.4%, which outperforms an anti-model based method or a pronunciation variant based method.