Nanette Veilleux

Parse scoring with prosodic information: an analysis/synthesis approach

Abstract Prosody is used by human listeners to disambiguate spoken language and, in particular, the relative size and location of prosodic phrase boundaries provides an important cue for resolving syntactic ambiguity. Therefore, automatically detected prosodic phrase boundaries should provide information useful in speech understanding for choosing among several candidate parses. Here, we propose two scoring algorithms to rank candidate parses, both based on an analysis/synthesis approach that compares the recognized prosodic phrase structure (analysis) with the predicted structure (synthesis) for each candidate parse. The two scoring algorithms, one rule-based and one using a probabilistic model, yield similar overall results when evaluated in experiments with a corpus of ambiguous sentences read by FM radio announcers. To decouple the performance of the analysis and synthesis components, we have used the scoring algorithms with hand-labeled breaks, which results in disambiguation performance comparable to the performance of human subjects in perceptual experiments. Performance degrades somewhat using automatically recognized breaks.

Markov modeling of prosodic phrase structure

A simple computational model for predicting phrase boundaries from text is described. The hierarchical structure of the model is based on linguistic theory, and the model itself is probabilistic. The probabilistic component is important for several reasons: it can capture the fact that the same sequence of words can be produced with a variety of prosodic contours (some more likely than others), it allows for automatic training of the model, and it provides a mechanism for combining a prosodic component with other knowledge sources. Results are presented for the model, trained and evaluated on a database of FM radio news.<<ETX>>

Probabilistic parse scoring with prosodic information

Prosodic patterns provide important cues for resolving syntactic ambiguity, and can be used to improve the accuracy of automatic speech understanding. With this goal, the authors propose a method of scoring syntactic parses in terms of observed prosodic cues, which can be used in ranking sentence hypotheses and associated parses. Specifically, the score is the probability of a hypothesized word sequence and associated syntactic parse given acoustic features, based on acoustic and language (prosody/syntax) models that represent probabilities in terms of abstract prosodic labels. Experimental results on a corpus of ambiguous sentence pairs indicate that the algorithm achieve ambiguity resolution performance close to that of human listeners.<<ETX>>

Turning points, tonal targets, and the English L- phrase accent

This study of the alignment of L- in the H* L- H% contour of American English finds the strongest predictor of the location of the retracted phrase-accent “elbow” to be the location of the accent-related F0 peak, rather than one of a set of metrical “attractors” investigated. A strong correlation between peak height and elbow sharpness suggests that the elbow may arise from interaction between peak height and perception-based constraints on the implementation of postnuclear deaccenting: F0 must become low immediately after the Nuclear-Pitch-Accented syllable, to prevent the percept of a pitch accent on a following accentable syllable. Relativised Area Under the Curve (R-AUC), a global measure for characterising F0 contours, distinguishes this H* L- from contrasting contours (e.g., H* L*) more reliably than a turning point-based approach focused on the location/scaling of an F0 elbow. The R-AUC measure is sensitive to perceptually significant, but difficult-to-quantify, changes in contour shape, while avoiding the pitfalls of purely shape-based theories.

Science fiction in computer science education

The use of science fiction (SF) to engage students in computer science learning is becoming more popular [1-6]. There is ample material available to help both undergraduate and graduate students make connections between technical content and human experience, from Star Trek to The Hitchhikers Guide to the Galaxy to 2001: A Space Odyssey to I, Robot and many others. Fiction can be included in technical courses or used to draw students into the field in introductory classes. The panelists, who represent a range of schools, perspectives and classes, will present brief overviews (5-8 minutes) of how they have used science fiction to engage students in technical topics as well as ethical and societal issues related to computing. After the overviews, there will be plenty of time for discussion of examples used within the community and ways to make connections between science fiction and particular classes or topics. We will be gathering additional examples from the discussion and making them available online.

A methodology for analyzing prosody

Prosody consists of that information in speech which can be suprasegmental, i.e., operate over units larger than a single segment. Prosodic contours may span a word, phrase, or larger units. The goal of this study is to understand the mapping between discrete, abstract units (e.g., boundary tones, pitch accents) and their observed continuously varying acoustic correlates (e.g., duration and F0). Since prosody synthesis and analysis have traditionally been a challenging problem, the initial focus of this study has been restricted to the FM radio news broadcasting style of speech. This pilot study indicates that in this speech style, prosodic units appear to be more strongly and more regularly marked than in conversational styles. Time‐ and pitch‐scale modification is employed using a sinusoidal model to change the duration and F0 contours of natural utterances. This technique allows synthesis of an utterance with a new prosodic contour, taking the input parameters from a linguistic model of prosody, or fro...

Probabilistic parse scoring based on prosodic phrasing

The relative size and location of prosodic phrase boundaries provides an important cue for resolving syntactic ambiguity. In previous work, we have introduced an analysis/synthesis formalism for scoring parses in terms of the similarity between prosodic patterns recognized from a given utterance and synthesized for the hypothesized parse. This paper describes a new approach to the synthesis problem, as well as an alternative scoring criterion. Specifically, a decision tree is designed to predict prosodic phrase structure for a given syntactic parse, and the tree is used to compute a parse score, which now is the probability of the recognized break sequence. Unlike the rule-based synthesis algorithm used in the previous work, the decision tree can be automatically trained and can therefore be designed specifically for different speaking styles or task domains. In experiments with a corpus of ambiguous sentences spoken by FM radio announcers, we have achieved disambiguation performance similar to the rule-based algorithm, which is close to the performance of human subjects in perceptual experiments using the scoring algorithm with hand labeled breaks.

On the nature of perceptual differences between accentual peaks and plateaux

It is probably not much of an overstatement to say that the theoretical mainstream in intonational phonology today takes the historical debate between level-based and configuration-based approaches to intonation to be resolved in such a way that explicit reference to contour shape no longer has any place in phonological representations. Part of what has made this an attractive position is the high degree of within-category variability one encounters in the shapes of phonological entities such as pitch accents in phonetic realization. It is well-known, for example, that a given pitch accent category (e.g., English L+H*, as shown in Figure 1) can be realized either as a sharp peak (as on the left), or instead can linger for a time near the F0 maximum, creating what is often referred to as a plateau-shaped configuration (as on the right). This distinction is not known to form the basis of a phonological contrast in any intonation system, nor is it clear whether the distinction is categorical, or even whether it is conditioned by contextual factors.

Alternatives to f0 turning points in American English intonation.

Since the inception of the autosegmental‐metrical approach to intonation (Bruce 1977, Pierrehumbert 1980, Ladd 1996), the location and scaling of f0 turning points have been used to characterize phonologically distinct f0 contours in various languages, including American English. This approach is undermined, however, by the difficulty listeners experience in perceiving differences in turning point location. Numerous studies have demonstrated either listener insensitivity to changes in turning point location or the capacity for other aspects of contour “shape” to override turning‐point alignment for contour identification (Chen 2003, D’Imperio 2000, Niebuhr 2008). Even labelers with access to visual representations of the f0 encounter similar challenges. By contrast, a family of related measurements using area under the f0 curve to quantify differences in contour shape appear more robust. For example, a measure of the synchronization of the center of gravity of the accentual rise with the boundaries of the...

A range of intonation patterns produced in an elicitation task

A production task was designed to elicit a specific intonation contour, characterized by an exaggerated rise‐fall‐rise in f0 (L+H*L−H% in ToBI terms), conveying an attitude of ‘‘dismayed surprise.’’ Speakers were given models, a short practice period, and a preceding context for each utterance. A targetlike rise‐fall‐rise contour (including H*L−H% variants) was successfully elicited for each of the contexts, but in less than half the utterances: Of 1134 utterances, 631 (56%) were judged by a panel of 3 experienced ToBI labelers as different from the intended contour, e.g., L*H−H%. Of these, 261 were produced with contours perceptually similar to the target, with a sustained flat or minimally rising f0 after the fall from the pitch accent, or with the rise occurring early in the next phrase. Alignment characteristics and individual speaker data may determine whether these are one contour with a continuum of realizations, or separate contours. These results (a) suggest that speakers have more than one way o...