Ken-ichi Iso

Speaker clustering using vector quantization and spectral clustering

We present a speaker clustering method for conversational speech recordings that contain short utterances from multiple speakers. The proposed method represents a speech segment with a vector of VQ code frequencies and uses a cosine between two vectors as their similarity measure. The clustering is performed by a spectral clustering algorithm with cluster number estimation based on an eigen structure of the similarity matrix. We conducted experiments on five test sets with different utterance length distributions to compare the proposed method with the conventional approach based on a hierarchical agglomerative clustering using BIC stopping criterion. The results show that the proposed method significantly outperforms the conventional one in speaker diarization error rate and purity metrics.

Speaker adaptation of deep neural networks using a hierarchy of output layers

Deep neural networks (DNN) used for acoustic modeling in speech recognition often have a very large number of output units corresponding to context dependent (CD) triphone HMM states. The amount of data available for speaker adaptation is often limited so a large majority of these CD states may not be observed during adaptation. In this case, the posterior probabilities of unseen CD states are only pushed towards zero during DNN speaker adaptation and the ability to predict these states can be degraded relative to the speaker independent network. We address this problem by appending an additional output layer which maps the original set of DNN output classes to a smaller set of phonetic classes (e.g. monophones) thereby reducing the occurrences of unseen states in the adaptation data. Adaptation proceeds by backpropagation of errors from the new output layer, which is disregarded at recognition time when posterior probabilities over the original set of CD states are used. We demonstrate the benefits of this approach over adapting the network with the original set of CD states using experiments on a Japanese voice search task and obtain 5.03% relative reduction in character error rate with approximately 60 seconds of adaptation data.

Robust DNN-Based VAD Augmented with Phone Entropy Based Rejection of Background Speech.

We propose a DNN-based voice activity detector augmented by entropy based frame rejection. DNN-based VAD classifies a frame into speech or non-speech and achieves significantly higher VAD performance compared to conventional statistical model-based VAD. We observed that many of the remaining errors are false alarms caused by background human speech, such as TV / radio or surrounding peoples’ conversations. In order to reject such background speech frames, we introduce an entropy-based confidence measure using the phone posterior probability output by a DNN-based acoustic model. Compared to the target speaker’s voice background speech tends to have relatively unclear pronunciation or is contaminated by other types of noises so its entropy becomes larger than audio signals with only the target speaker’s voice. Combining DNN-based VAD and the entropy criterion, we reject speech frames classified by the DNN-based VAD as having an entropy larger than a threshold value. We have evaluated the proposed approach and confirmed greater than 10% reduction in Sentence Error Rate.