Antje Schweitzer

Can intonation contours be lexicalised? Implications for discourse meanings

We propose that words and short phrases in English can also be stored with intonation contours they frequently occur with, along with their discourse meanings . We present a corpus study investigating intonational collocations, i .e . pairings of lexical phrases and accent contours, in a corpus of conversational speech, Switchboard . We developed a novel method to identify similar accent contours, using automatic pitch parameterisation (Möhler and Conkie, 1998) and clustering . We found that intonational collocations are widespread, accounting for up to 34% of all tokens, and 76% of frequent lexical types in our data . This shows prima facie evidence that word-contour pairs can be stored . A qualitative analysis of frequent intonational collocations showed they are used with very specific discourse meanings, consistent with our claim that they are stored . For each, lower frequency collocates with related discourse meanings could be identified, consistent with the exemplar-theoretic prediction that these discourse meanings can spread by analogy . Finally, we present the results of a perception experiment showing that frequent words collocated with a particular accent type, and lower frequency words related to the frequent collocates, are judged to sound more natural than low frequency, unrelated collocates of that accent type . This is consistent with the claim that frequency-based collocation is part of grammar, and affects language expectations (cf . Bybee and Eddington, 2006) .

Frequency Matters: Pitch Accents and Information Status

This paper presents the results of a series of experiments which examine the impact of two information status categories (given and new) and frequency of occurrence on pitch accent realisations. More specifically the experiments explore within-type similarity of pitch accent productions and the effect information status and frequency of occurrence have on these productions. The results indicate a significant influence of both pitch accent type and information status category on the degree of within-type variability, in line with exemplartheoretic expectations.

Multimodal Speech Synthesis

Speech output generation in the SmartKom system is realized by a corpus-based unit selection strategy that preserves many properties of the human voice. When the system’s avatar “Smartakus” is present on the screen, the synthetic speech signal is temporally synchronized with Smartakus visible speech gestures and prosodically adjusted to his pointing gestures to enhance multimodal communication. The unit selection voice was formally evaluated and found to be very well accepted and reasonably intelligible in SmartKom- specific scenarios.

Gesture‐speech interaction in the SmartKom project

The goal of the SmartKom project is to create an intuitive multi‐modal dialog system. Information is presented on the display by an artificial lifelike character, combining the output modalities speech, mimics, and gesture. Multi‐modality poses new requirements for speech synthesis. Speech may be accompanied by gestures, adding three new aspects: First, synchronization of lip movements; second, temporal alignment of speech and gesture; and third, effects of gestures on prosody. This paper focuses on the last two points. Building on de Ruiter’s (2000) Sketch Model, speech is synthesized independently from the temporal structure of the accompanying gesture. Preparation phase and retraction phase of gestures can be adjusted to align the relevant speech material with the stroke phase. Concerning prosody, results of a pilot study on intonation of deictic elements show that speech material accompanied by gestures is more likely to be prosodically prominent. Gestures in the SmartKom system are always triggered by linguistic properties of the accompanying speech. Most of the gestures are pointing gestures, which are triggered by deictic elements. Such linguistic information is integrated into the concept input to the synthesis module and is taken into account during prosody generation.

Exemplar Dynamics in Phonetic Convergence of Speech Rate.

We motivate and test an exemplar-theoretic view of phonetic convergence, in which convergence effects arise because exemplars just perceived in a conversation are stored in a speaker’s memory, and used subsequently in speech production. Most exemplar models assume that production targets are established using stored exemplars, taking into account their frequencyand recency-influenced level of activation. Thus, convergence effects are expected to arise because the exemplars just perceived from a partner have a comparably high activation. However, in the case of frequent exemplars, this effect should be countered by the high frequency of already stored, older exemplars. We test this assumption by examining speech rate convergence in spontaneous speech by female German speakers. We fit two linear mixed models, calculating speech rate on the basis of either infrequent, or frequent, syllables, and predict a speaker’s speech rate in a phrase by the partner’s speech rate in the preceding phrase. As anticipated, we find a significant main effect indicating convergence only for the infrequent syllables. We also find an unexpected significant interaction of the partner’s speech rate and the speaker’s assessment of the partner in terms of likeability, indicating divergence, but again, only for the infrequent case.

Prosodic and segmental convergence in spontaneous German conversations.

Phonetic convergence is the process of adapting one’s speech to that of an interlocutor and is hence seen as a dynamic process causing continuous updates in multiple speech parameters. Current methods of measuring phonetic convergence are often impressionistic or perceptual, or have only been tested in limited scenarios. The aim of this ongoing project is to investigate convergence in German in a natural open scenario and to develop computational objective methods of accounting for it. A variety of phonetic parameters with respect to their relevance for convergence will be examined. On the prosodic level, one goal is to examine phonological features such as the prosodic inventory or distribution of prosodic events as well as the detailed phonetic implementation of prosodic events. The latter can be measured by the PaIntE F0 parametrization method. Convergence effects have been shown to be very subtle; thus large amounts of data are required in order to obtain statistically valid and robust results. Develo...

Exploring prosodic and conversational context factors in pitch perception

Listeners use pitch information to contextualize and interpret what they hear in conversation, but contextualization requires a frame of reference in terms of both acoustic information and conversational structure. We investigate how different acoustic features such as fundamental frequency (F0), intensity, and duration, as well as different contexts for listening to speech (i.e. in isolation versus adjacent to another conversational turn) relate to listeners’ perception of pitch in speech. Following a perception experiment in which listeners gave pitch ratings for individual turns or pairs of turns drawn from a corpus, we explore the relationship of prosodic features to listeners’ judgments. While whole-turn F0 appears to be most relevant to judgment of turns in isolation, pitch and intensity in the region of the transition are prioritized in the turn-comparative judgments.

Cross-Gender and Cross-Dialect Tone Recognition for Vietnamese.

We investigate tone recognition in Vietnamese across gender and dialects. In addition to well-known parameters such as single fundamental frequency (F0) values and energy features, we explore the impact of harmonicity on recognition accuracy, as well as that of the PaIntE parameters, which quantify the shape of the F0 contour over complete syllables instead of providing more local single values. Using these new features for tone recognition in the GlobalPhone database, we observe significant improvements of approx. 1% in recognition accuracy when adding harmonicity, and of another approx. 4% when adding the PaIntE parameters. Furthermore, we analyze the influence of gender and dialect on recognition accuracy. The results show that it is easier to recognize tones for female than for male speakers, and easier for the Northern dialect than for the Southern dialect. Moreover, we achieve reasonable results testing models across gender, while the performance drops strongly when testing across dialects.

A 3D computer game for testing perception of acoustic detail in speech

This paper presents an experimental framework for acoustic perception studies, including an example application which tests attention to fine phonetic detail in natural speech. Traditional perception experiments in speech research do not employ natural testing scenarios, and they are notorious for being perceived as boring and repetitive by test subjects. This contribution presents a solution to the problem, by employing a computer game in which perception of the acoustic aspect to be tested is crucial for success in the game. The game is designed as a first person shooter. The player moves around within a virtual 3D environment and reacts to agents which belong to two different categories, each of which is associated with one response key. The two categories are initially distinguished by visual and acoustic cues, and the acoustic cues are designed to differ only in the perceptual aspect to be tested. Gradually, visual cues are removed. Thus, players with better perception of the aspect score higher in t...