Mike Schuster

Statistical parametric speech synthesis using deep neural networks

Conventional approaches to statistical parametric speech synthesis typically use decision tree-clustered context-dependent hidden Markov models (HMMs) to represent probability densities of speech parameters given texts. Speech parameters are generated from the probability densities to maximize their output probabilities, then a speech waveform is reconstructed from the generated parameters. This approach is reasonably effective but has a couple of limitations, e.g. decision trees are inefficient to model complex context dependencies. This paper examines an alternative scheme that is based on a deep neural network (DNN). The relationship between input texts and their acoustic realizations is modeled by a DNN. The use of the DNN can address some limitations of the conventional approach. Experimental results show that the DNN-based systems outperformed the HMM-based systems with similar numbers of parameters.

Deep Learning for Acoustic Modeling in Parametric Speech Generation: A systematic review of existing techniques and future trends

Hidden Markov models (HMMs) and Gaussian mixture models (GMMs) are the two most common types of acoustic models used in statistical parametric approaches for generating low-level speech waveforms from high-level symbolic inputs via intermediate acoustic feature sequences. However, these models have their limitations in representing complex, nonlinear relationships between the speech generation inputs and the acoustic features. Inspired by the intrinsically hierarchical process of human speech production and by the successful application of deep neural networks (DNNs) to automatic speech recognition (ASR), deep learning techniques have also been applied successfully to speech generation, as reported in recent literature. This article systematically reviews these emerging speech generation approaches, with the dual goal of helping readers gain a better understanding of the existing techniques as well as stimulating new work in the burgeoning area of deep learning for parametric speech generation.

Deploying GOOG-411: Early lessons in data, measurement, and testing

We describe our early experience building and optimizing GOOG-411, a fully automated, voice-enabled, business finder. We show how taking an iterative approach to system development allows us to optimize the various components of the system, thereby progressively improving user-facing metrics. We show the contributions of different data sources to recognition accuracy. For business listing language models, we see a nearly linear performance increase with the logarithm of the amount of training data. To date, we have improved our correct accept rate by 25% absolute, and increased our transfer rate by 35% absolute.

Japanese and Korean voice search

This paper describes challenges and solutions for building a successful voice search system as applied to Japanese and Korean at Google. We describe the techniques used to deal with an infinite vocabulary, how modeling completely in the written domain for language model and dictionary can avoid some system complexity, and how we built dictionaries, language and acoustic models in this framework. We show how to deal with the difficulty of scoring results for multiple script languages because of ambiguities. The development of voice search for these languages led to a significant simplification of the original process to build a system for any new language which in in parts became our default process for internationalization of voice search.

Speech recognition for mobile devices at Google

We briefly describe here some of the content of a talk to be given at the conference.

Efficient generation of high-order context dependent weighted finite state transducers for speech recognition

This paper describes an algorithm for efficient building of weighted finite state transducers for speech recognition when high-order context-dependent models of order K>3 (triphones) with tied states are used. We show how an algorithm to build a part of the needed composed transducers directly from the decision trees in combination with an improved compilation process can lead to much faster, simpler and more memory-efficient compilation. In our case, it also resulted in substantially smaller final networks. With the described algorithm, it is simple to use high-order full cross-word models with little overhead directly within a one-pass time-synchronous search, which we test comparing resulting final network sizes, recognition rates and speed on a large, spontaneous Japanese speech database. Using the proposed algorithm, it is possible to do real-time recognition using full crossword quinphones with a large acoustic model in about 125 MB of memory at about 9% search error.

Phoneme boundary estimation using bidirectional recurrent neural networks and its applications

This paper describes a phoneme boundary estimation method based on bidirectional recurrent neural networks (BRNNs). Experimental results showed that the proposed method could estimate segment boundaries significantly better than an HMM or a multilayer perceptron-based method. Furthermore, we incorporated the BRNN-based segment boundary estimator into the HMM-based and segment model-based recognition systems. As a result, we confirmed that (1) BRNN outputs were effective for improving the recognition rate and reducing computational time in an HMM-based recognition system and (2) segment lattices obtained by the proposed methods dramatically reduce the computational complexity of segment model-based recognition.

Construction of weighted finite state transducers for very wide context-dependent acoustic models

A previous paper by the authors described an algorithm for efficient construction of weighted finite state transducers for speech recognition when high-order context-dependent models of order K > 3 (triphones) with tied state observation distributions are used, and showed practical application of the algorithm up to K = 5 (quinphones). In this paper we give additional details of the improved implementation and analyze the algorithms practical runtime requirements and memory footprint for context-orders up to K = 13 (+/-6 phones context) when building fully cross-word capable WFSTs for large vocabulary speech recognition tasks. We show that for typical systems it is possible to use any practical context-order K les 13 without having to fear an exponential explosion of the search space, since the necessary state ID to phone transducer (resembling a phone-loop observing all possible K-phone constraints) can be built in a few minutes at most. The paper also gives some implementation details of how we efficiently collect context statistics and build phonetic decision trees for very wide context-dependent acoustic models