William M. Fisher

Multi-site data collection and evaluation in spoken language understanding

The Air Travel Information System (ATIS) domain serves as the common task for DARPA spoken language system research and development. The approaches and results possible in this rapidly growing area are structured by available corpora, annotations of that data, and evaluation methods. Coordination of this crucial infrastructure is the charter of the Multi-Site ATIS Data COllection Working group (MADCOW). We focus here on selection of training and test data, evaluation of language understanding, and the continuing search for evaluation methods that will correlate well with expected performance of the technology in applications.

An acoustic‐phonetic data base

DARPA has sponsored the design and collection of a large speech data base. Six hundred and thirty speakers read ten sentences each. Two sentences were constant for all speakers; the remaining eight sentences were selected from a set of 450 designed at MIT and 1890 selected at TI from text sources. The set of sentences is phonetically rich, balanced, and deep. Although all recordings were made in Dallas, we sampled as many varieties of American English as possible. Selection of volunteer speakers was based on their childhood locality to give a balanced representation of geographical origins. The subject population is adult; 70% male; young (63% in their twenties); well educated (78% with bachelors degree); and predominantly white (96%). Recordings were made in a noise‐reducing sound booth using a Sennheiser headset microphone and digitized at 20 kHz. A natural reading style was encouraged. The recordings are complete, and time‐registered phonetic transcriptions are being added to the 6300 speech files at ...

A statistical text-to-phone function using ngrams and rules

Adopting concepts from statistical language modeling and rule-based transformations can lead to effective and efficient text-to-phone (TTP) functions. We present here the methods and results of one such effort, resulting in a relatively compact and fast set of TTP rules that achieves 94.5% segmental phonemic accuracy.

Better alignment procedures for speech recognition evaluation

In evaluating speech recognition, an alignment of reference symbols with hypothesized symbols is the basis of other measures. The authors report on advances made at the National Institute of Standards and Technology on algorithms for alignment. They empirically justify phonological alignment, which minimizes differences in phonological features. A novel technique for identifying splits and merges is briefly described.<<ETX>>

DARPA ATIS test results June 1990

The first Spoken Language System tests to be conducted in the DARPA Air Travel Information System (ATIS) domain took place during the period June 15 - 20, 1989. This paper presents a brief description of the test protocol, comparator software used for scoring results at NIST, test material selection process, and preliminary tabulation of the scored results for seven SLS systems from five sites: BBN, CMU, MIT/LCS, SRI and Unisys. One system, designated cmu-spi(r) in this paper, made use of digitized speech as input (.wav files), and generated CAS-format answers. Other systems made use of SNOR transcriptions (.snr files) as input.