Dafydd Gibbon

Handbook of Multimodal and Spoken Dialogue Systems

Human language is said to come in two forms: there are spoken languages that languages need to be considered as essentially heterogeneous systems that An International Handbook. A multimodal and multifocal dialogue corpus. dialogue systems and offers ideas on how to continue along the path GIBBON D., MERTINS I., MOORE R. (eds), Handbook of Multimodal and Spoken. A Review and Meta-Analysis of Multimodal Affect Detection Systems human-computer interaction handbook: fundamentals, evolving technologies and emerging Predicting emotion in spoken dialogue from multiple knowledge sources.

Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation

Editorial Preface. 1. Representation and annotation of dialogue. 2. Audio-visual and multimodel speech-based systems. 3. Consumer off-the-shelf (COTS) product and service evaluation. 4. Terminology for spoken language systems. 5. Reference materials. Bibliographical references. List of abbreviations. Index. CD-ROM disclaimer.

Lexicon Development for Speech and Language Processing

Preface. Computational Lexicography D. Gibbon. Constraint-Based Lexica G. Bouma, et al. Phonology-Based Lexical Knowledge Representation L. Cahill, et al. Inductive Lexica W. Daelemans, G. Durieux. Recognizing Lexical Patterns in Text G. Grefenstette, et al. Speech Databases C. Draxler. The Use of Lexica in Text-To-Speech Systems S. Quazza, H. v.d. Heuvel. The Use of Lexica in Automatic Speech Recognition M. Adda-Decker, L. Lamel. Morphology in the Mental Lexicon: A Computational Model for Visual Word Recognition R.H. Baayen, et al. Author Information.

Audio-visual and multimodal speech-based systems

Communication between humans uses many modalities. We communicate not only via verbal language, but also through our use of intonation, gaze, hand gestures, body gestures, and facial expressions. Using these modalities, we can add, modify, and substitute information in spoken conversations. Complementary use of several modalities in human-to-human communication ensures high accuracy, and only few communication problems occur. When communication problems do occur, conversation partners can easily recover, using the redundancy and complementarity of modalities. The goal of research on multimodal systems is to investigate how human—computer interaction can benefit from multiple modalities in similar ways.

Incremental Multimodal Feedback for Conversational Agents

Just like humans, conversational computer systems should not listen silently to their input and then respond. Instead, they should enforce the speaker-listener link by attending actively and giving feedback on an utterance while perceiving it. Most existing systems produce direct feedback responses to decisive (e.g. prosodic) cues. We present a framework that conceives of feedback as a more complex system, resulting from the interplay of conventionalized responses to eliciting speaker events and the multimodal behavior that signals how internal states of the listener evolve. A model for producing such incremental feedback, based on multi-layered processes for perceiving, understanding, and evaluating input, is described.

Towards an unrestricted domain TTS system for African tone languages

In this paper we discuss the procedural problems, issues and challenges involved in developing a generic speech synthesizer for African tone languages. We base our development methodology on the “MultiSyn” unit-selection approach, supported by Festival Text-To-Speech (TTS) Toolkit for Ibibio, a Lower Cross subgroup of the (New) Benue-Congo language family widely spoken in the southeastern region of Nigeria. We present in a chronological order, the several levels of infrastructural and linguistic problems as well as challenges identified in the Local Language Speech Technology Initiative (LLSTI) during the development process (from the corpus preparation and refinement stage to the integration and synthesis stage). We provide solutions to most of these challenges and point to possible outlook for further refinement. The evaluation of the initial prototype shows that the synthesis system will be useful to non-literate communities and a wide spectrum of applications.

Idiomaticity and functional variation: A case study of international amateur radio talk

The linguistic domain of idiomaticity poses many problems for the study of language form, use, and variation. With selected aspects of idiomaticity as a starting point, I will attempt in this paper to develop a description of the use of idioms as a segment of a more general theory of language use. Evidence for this approach is drawn from international amateur radio talk (IART) in English.

Speech Lexica and Consistent Multilingual Vocabularies

This contribution describes the theoretical foundations and lexical engineering procedures used in developing a common, consistent, linguistically and formally well-defined lexical database for all components of the Verbmobil speech-to-speech translation system.

Context and Variation in Two-way Radio Discourse.

International amateur radio talk (IART) is described as a ‘clear case’ of the significance of the channel as a restrictive factor in discourse, and sample texts of IART are analyzed. The situation of IART and its subvarieties and neighboring varieties within a formal and functional ‘variety space’ is examined within a proposed theory of the use of speech. Language use is regarded as defined by a hierarchically organized set of constitutive factors, and purpose‐oriented aspects of language use are included by positing an ongoing adaptive process, the ‘functional cycle’. Three constitutive systems are defined: register, genre, and style. The claim is made that ‘language use is language variation’: The assignment of specific features to the factors of use defines a functional variety, and the continuous checking and logging of indexical values defines the ongoing adaptation of speech to context.

Spoken language system and corpus design

This handbook provides easy access to current practice and requirements in the main spoken language technologies.