Daniel Luzzati

Multimodal data collection of human-robot humorous interactions in the Joker project

Thanks to a remarkably great ability to show amusement and engagement, laughter is one of the most important social markers in human interactions. Laughing together can actually help to set up a positive atmosphere and favors the creation of new relationships. This paper presents a data collection of social interaction dialogs involving humor between a human participant and a robot. In this work, interaction scenarios have been designed in order to study social markers such as laughter. They have been implemented within two automatic systems developed in the Joker project: a social dialog system using paralinguistic cues and a task-based dialog system using linguistic content. One of the major contributions of this work is to provide a context to study human laughter produced during a human-robot interaction. The collected data will be used to build a generic intelligent user interface which provides a multimodal dialog system with social communication skills including humor and other informal socially oriented behaviors. This system will emphasize the fusion of verbal and non-verbal channels for emotional and social behavior perception, interaction and generation capabilities.

ALORS: a skimming parser for spontaneous speech processing

Abstract This paper describes a parser for processing spontaneous spoken utterances which deals with both dislocation and “noise” phenomena. The parser is built upon a timetable corpus collected in an information center of the SNCF (French railway company). With spontaneous speech we consider that the only relevant problems are those that have been found in a corpus. Our corpus has been recorded in a human/machine situation and in a human/human situation in order to observe how the linguistic behavior provoked by the machine, changes automatic processing. As this work deals with cognitive levels, ALORS processes a keyboard input. Using a skimming strategy, it takes into account some aspects of speech recognition, and we evaluate its compatibility with word spotting speech recognition.