Agustín Gravano

High Frequency Word Entrainment in Spoken Dialogue

Cognitive theories of dialogue hold that entrainment, the automatic alignment between dialogue partners at many levels of linguistic representation, is key to facilitating both production and comprehension in dialogue. In this paper we examine novel types of entrainment in two corpora---Switchboard and the Columbia Games corpus. We examine entrainment in use of high-frequency words (the most common words in the corpus), and its association with dialogue naturalness and flow, as well as with task success. Our results show that such entrainment is predictive of the perceived naturalness of dialogues and is significantly correlated with task success; in overall interaction flow, higher degrees of entrainment are associated with more overlaps and fewer interruptions.

Restoring punctuation and capitalization in transcribed speech

Adding punctuation and capitalization greatly improves the readability of automatic speech transcripts. We discuss an approach for performing both tasks in a single pass using a purely text-based n-gram language model. We study the effect on performance of varying the n-gram order (from n = 3 to n = 6) and the amount of training data (from 58 million to 55 billion tokens). Our results show that using larger training data sets consistently improves performance, while increasing the n-gram order does not help nearly as much.

Backchannel-Inviting Cues in Task-Oriented Dialogue

We examine BACKCHANNEL-INVITING CUES — distinct prosodic, acoustic and lexical events in the speaker’s speech that tend to precede a short response produced by the interlocutor to convey continued attention —in theColumbiaGames Corpus, a large corpus of task-oriented dialogues. We show that the likelihood of occurrence of a backchannel increases quadratically with the number of cues conjointly displayed by the speaker. Our results are important for improving the coordination of conversational turns in interactive voice-response systems, so that systems can produce backchannels in appropriate places, and so that they can elicit backchannels from users in expected places. Index Terms: dialogue, prosody, turn-taking, backchannels.

The Prosody of Backchannels in American English

We examine prosodic and contextual factors characterizing the backchannel function of single affirmative words. Data is drawn from collaborative task-oriented dialogues between speakers of Standard American English. Despite high lexical variability, backchannels are prosodically well defined: they have higher pitch and intensity and greater pitch slope than affirmative words expressing other pragmatic functions. Additionally, we identify phrase-final rising pitch as a salient trigger for backchanneling.

Entrainment in Speech Preceding Backchannels.

In conversation, when speech is followed by a backchannel, evidence of continued engagement by ones dialogue partner, that speech displays a combination of cues that appear to signal to ones interlocutor that a backchannel is appropriate. We term these cues back-channel-preceding cues (BPC)s, and examine the Columbia Games Corpus for evidence of entrainment on such cues. Entrainment, the phenomenon of dialogue partners becoming more similar to each other, is widely believed to be crucial to conversation quality and success. Our results show that speaking partners entrain on BPCs; that is, they tend to use similar sets of BPCs; this similarity increases over the course of a dialogue; and this similarity is associated with measures of dialogue coordination and task success.

Turn-Yielding Cues in Task-Oriented Dialogue

We examine a number of objective, automatically computable TURN-YIELDING CUES --- distinct prosodic, acoustic and syntactic events in a speakers speech that tend to precede a smooth turn exchange --- in the Columbia Games Corpus, a large corpus of task-oriented dialogues. We show that the likelihood of occurrence of a turn-taking attempt from the interlocutor increases linearly with the number of cues conjointly displayed by the speaker. Our results are important for improving the coordination of speaking turns in interactive voice-response systems, so that systems can correctly estimate when the user is willing to yield the conversational floor, and so that they can produce their own turn-yielding cues appropriately.

On the role of context and prosody in the interpretation of 'okay'

We examine the effect of contextual and acoustic cues in the disambiguation of three discourse-pragmatic functions of the word okay. Results of a perception study show that contextual cues are stronger predictors of discourse function than acoustic cues. However, acoustic features capturing the pitch excursion at the right edge of okay feature prominently in disambiguation, whether other contextual cues are present or not.

Classification of Discourse Functions of Affirmative Words in Spoken Dialogue

Wepresentresultsofaseriesofmachinelearningexperi, mentsthataddresstheclassificationofthediscoursefunction� ofsingleaffirmativecuewordssuchas� alright,�okayand� mm-hminaspokendialoguecorpus.�Wesuggestthatasimple� discourse/sententialdistinctionisnotsufficientforsuch� wordsandproposetwoadditionalclassificationsub,tasks:� identifying�(a)�whethersuchwordsconveyacknowledgment� oragreement,�and�(b)�whethertheycuethebeginningorend� ofadiscoursesegment.�Wealsostudytheclassificationof� eachindividualwordintoitsmostcommondiscoursefunc, tions.� Weshowthatmodelsbasedoncontextualfeatures� extractedfromthetime,alignedtranscriptsapproachtheerror� rateoftrainedhumanaligners.� Index Terms:�cuewords,�discoursemarkers,�spokendialogue� systems.�

Very short utterances in conversation

Faced with the difficulties of finding an operationalized definition of backchannels, we have previously proposed an intermediate, auxiliary unit ‐ the very short utterance (VSU) ‐ which is defined operationally and is automatically extractable from recorded or ongoing dialogues. Here, we extend that work in the following ways: (1) we test the extent to which the VSU/NONVSU distinction corresponds to backchannels/non-backchannels in a different data set that is manually annotated for backchannels ‐ the Columbia Games Corpus; (2) we examine to the extent to which VSUS capture other short utterances with a vocabulary similar to backchannels; (3) we propose a VSU method for better managing turn-taking and barge-ins in spoken dialogue systems based on detection of backchannels; and (4) we attempt to detect backchannels with better precision by training a backchannel classifier using durations and inter-speaker relative loudness differences as features. The results show that VSUS indeed c apture a large proportion of backchannels ‐ large enough that VSUs can be used to improve spoken dialogue system turntaking; and that building a reliable backchannel classifier working in real time is feasible.

The Effect of Contour Type and Epistemic Modality on the Assessment of Speaker Certainty

In an empirical study participants were asked to rate the perceived degree of certainty of utterances that contained either the modal would or main verb be (e.g. That would be me vs. That’s me), and which were also variously produced with one of three intonational contours (downstepped, declarative, and yes-noquestion). We found that both downstepped contour and epistemic would made a signicant and independent contribution to the assessment of speaker certainty. That is, participants rated utterances with the downstepped contour as most certain, followed by those with the declarative contour, while the yes-noquestion contour was perceived as highly uncertain. Similarly, participants rated speakers’ responses with epistemic would as signicantly more certain than those without it.