Harvey Lloyd-Thomas

An integrated grammar/bigram language model using path scores

This paper describes a language model in which context-free grammar rules are integrated into an n-gram framework, complementing it instead of attempting to replace it. This releases the grammar from the aim of parsing sentences overall (which is often undesirable as well as unrealistic), enabling it to be employed selectively in modelling phrases that are identifiable within a flow of speech. Algorithms for model training and for sentence scoring and interpretation are described. All are based on the principle of summing over paths that span the sentence, but implementation is node-based for efficiency. Perplexity results for this system (using a hierarchy of grammars from empty to full-coverage) are compared with those for n-gram models, and the system is used for re-scoring N-best sentence lists for a speaker-independent recogniser.

A comparison of model estimation techniques for speaker verification

In this paper we address the problem of building speaker dependent hidden Markov models for a speaker verification system. A number of model building techniques are described and the comparative performance of a system using models built using each of these techniques is presented. Mean estimated models, models where the means of the HMMs are estimated using segmental K means but where the variances are taken from speaker independent models, out performed other techniques such as Baum-Welch re-estimation for training times of 120 s, 60 s and 15 s. Mean estimated models were also built with varying numbers of components in the state mixture distributions and a performance gain was again observed. The incorporation of transitional features into the system had degraded performance when the Baum-Welch algorithm was used for model estimation. However the inclusion of delta and delta-delta cepstra into the system using mean estimated models now gave a significant improvement in performance. Taken together these changes halved the equal error rate of the system from 15.7% to 7.8%.

A robust language model incorporating a substring parser and extended n-grams

Describes a language model for speech recognition which incorporates a substring parser (to take advantage of syntactic structure covered by a context-free grammar) and extended bigrams (to take advantage of remote dependencies between words). The use of extended bigrams significantly reduces the perplexity and a distribution clustering algorithm alleviates the additional storage cost. The substring parser is the foundation for training and scoring procedures based on paths at all levels through the syntactic structures, with subtrees linked by bigrams. The word bigram score is therefore absorbed into a grammar framework, consolidating the two kinds of language model, and again a significant reduction in perplexity is observed. The aim is an integrated, robust language model that is adaptive to the speaker.<<ETX>>

Training and Application of Integrated Grammar/Bigram Language Models

This paper discusses a robust language model consisting of context-free grammar rules and symbol bigrams, integrated into a single framework. The aim is to remove the sharp grammatical/ungrammatical distinction by exploiting whatever grammar structure is present in every sentence, and hence to achieve continuity of scoring across the language. Both training and scoring are based on a similar principle: summing over paths that span the sentence. In addition to finding the overall score, a procedure for finding the best interpretation is described. Efficiency is maximised by the use of node-based (rather than path-based) procedures.