Jens Allwood

On the Semantics and Pragmatics of Linguistic Feedback

This paper is an exploration in the semantics and pragmatics of linguistic feedback, i.e., linguistic mechanisms which enable the participants in spoken interaction to exchange information about basic communicative functions, such as contact, perception, understanding, and attitudinal reactions to the communicated content. Special attention is given to the type of reaction conveyed by feedback utterances, the communicative status of the information conveyed (i. e., the level of awareness and intentionality of the communicating sender), and the context sensitivity of feedback expressions. With regard to context sensitivity, which is one of the most characteristic features of feedback expressions, the discussion focuses on the way in which the type of speech act (mood), the factual polarity and the information status of the preceding utterance influence the interpretation of feedback utterances. The different content dimensions are exemplified by data from recorded dialogues and by data given through linguistic intuition. Finally, two different ways of formalizing the analysis are examined, one using attribute-value matrices and one based on the theory of situation semantics.

The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena

This paper deals with a multimodal annotation scheme dedicated to the study of gestures in interpersonal communication, with particular regard to the role played by multimodal expressions for feedback, turn management and sequencing. The scheme has been developed under the framework of the MUMIN network and tested on the analysis of multimodal behaviour in short video clips in Swedish, Finnish and Danish. The preliminary results obtained in these studies show that the reliability of the categories defined in the scheme is acceptable, and that the scheme as a whole constitutes a versatile analysis tool for the study of multimodal communication behaviour.

BODILY COMMUNICATION DIMENSIONS OF EXPRESSION AND CONTENT

Bodily communication perceived visually or through the tactile senses has a central place in human communication. It is probably basic both from an ontogenetic and a phylogenetic perspective, being connected with archaic levels in our brains such as the limbic system and the autonomous neural system. It is interesting from a biological, psychological and social point of view and given recent developments in ICT (Information and Communication Technology). It is also becoming more and more interesting from a technological point of view.

Speech Management—on the Non-written Life of Speech

This paper introduces the concept of speech management (SM), which refers to processes whereby a speaker manages his or her linguistic contributions to a communicative interaction, and which involves phenomena which have previously been studied under such rubrics as “planning”, “editing”, “(self-)repair”, etc. It is argued that SM phenomena exhibit considerable systematicity and regularity and must be considered part of the linguistic system. Furthermore, it is argued that SM phenomena must be related not only to such intraindividual factors as planning and memory, but also to interactional factors such as turntaking and feedback, and to informational content. Structural and functional taxonomies are presented together with a formal description of complex types of SM. The structural types are exemplified with data from a corpus of SM phenomena.

Cooperation, dialogue and ethics

This paper describes some of the basic cooperative mechanisms of dialogue. Ideal cooperation is seen as consisting of four features (cognitive consideration, joint purpose, ethical consideration and trust), which can also to some extent be seen as requirements building on each other. Weaker concepts such as “coordination” and “collaboration” have only some of these features or have them to lesser degrees. We point out the central role of ethics and trust in cooperation, and contrast the result with popular AI accounts of collaboration. Dialogue is also seen as associated with social activities, in which certain obligations and rights are connected with particular roles. Dialogue is seen to progress through the written, vocal or gestural contributions made by participants. Each of the contributions has associated with it both expressive and evocative functions, as well as specific obligations for participants. These functions are dependent on the surface form of a contribution, the activity and the local context, for their interpretation. We illustrate the perspective by analysing dialogue extracts from three different activity types (a travel dialogue, a quarrel and a dialogue with a computer system). Finally, we consider what kind of information is shared in dialogue, and the ways in which dialogue participants manifest this sharing to each other through linguistic and other communicative behaviour. The paper concludes with a comparison to other accounts of dialogue and prospects for integration of these ideas within dialogue systems.

Annotations and tools for an activity based Spoken Language Corpus

The paper contains a description of the Spoken Language Corpus of Swedish at the Department of Linguistics, Goteborg University (GSLC), and a summary of the various types of analysis and tools that have been developed for work on this corpus. Work on the corpus was started in the late 1970:s. It is incrementally growing and presently consists of 1.3 million words from about 25 different social activities. The corpus was initiated to meet a growing interest in naturalistic spoken language data. It is based on the fact that spoken language varies considerably in different social activities with regard to pronunciation, vocabulary, grammar and communicative functions. The goal of the corpus is to include spoken language from as many social activities as possible to get a more complete understanding of the role of language and communication in human social life. This type of spoken language corpus is still fairly unique even for English, since many spoken language corpora (certainly for Swedish) have been collected for special purposes, like speech recognition, phonetics, dialectal variation or interaction with a computerized dialog system in a very narrow domain, e.g. (Map Task (Isard and Carletta (1995), TRAINS (Heeman and Allen 1994), Waxholm (Blomerg et al. 1993). Compared to English corpora, the Goteborg corpus is most similar to the Wellington Corpus of Spoken New Zealand English (Holmes, Vine and Johnson 1998), but also has traits in common with the BNC, the London/Lund corpus (Svartvik 1990) and the Danish BySoc corpus (Gregersen 1991, Henrichsen 1997).

The analysis of embodied communicative feedback in multimodal corpora: a prerequisite for behavior simulation

Communicative feedback refers to unobtrusive (usually short) vocal or bodily expressions whereby a recipient of information can inform a contributor of information about whether he/she is able and willing to communicate, perceive the information, and understand the information. This paper provides a theory for embodied communicative feedback, describing the different dimensions and features involved. It also provides a corpus analysis part, describing a first data coding and analysis method geared to find the features postulated by the theory. The corpus analysis part describes different methods and statistical procedures and discusses their applicability and the possible insights gained with these methods.

On the Distinctions between Semantics and Pragmatics

The term pragmatics was proposed by Morris in 1938 as a tribute to C. S. Peirces philosophy of pragmatism, to designate the study of signs and their relationship to interpreters. In 1946 Morris changed this slightly to make pragmatics the study of the origin, use and effect of signs. One of the main differences between the two versions is that in the second version the term use also includes the production of signs. The term semantics in 1938 was used to designate the more abstract study of the relationship between signs and the objects they signify (leaving out the interpreter). In 1946 this was changed to the study of signification in all modes of signifying. The second version again widens the definition given in the 1938-version. Syntax (or syntactics (Morris (1938)) in both the 1938 and 1946 version was to designate the even more abstract study of the relationship between signs without taking either their signification, origin, use or effect into account.

Comparing Human-to-Human and Human-to-AEA Communication in Service Encounters

An increasing number of companies are introducing artificial agents as self-service tools on their websites, often motivated by the need to provide cost-efficient interaction solutions. These agents are designed to help customers and clients to conduct their business on the website. Their role on commercial websites is often to act as online sales/shopping assistants with the hope of replacing some of the interactions between customers and sales staff, thus supplementing or replacing human-to-human communication. However, research on artificial agents and comparisons with human-to-human communication, in particular, is still scarce. The purpose of this article is to explore the similarities and differences in communication between an artificial agent and customers compared with face-to-face communication between human service providers and customers. The method employed is a qualitative comparison of face-to-face human service provision in a travel agency setting and logs of interactions between customers and an artificial agent on an airline company website. The analysis is based on the theory of “activity-based communication analysis” and makes use of a framework of specific communication features provided by this theory. The article demonstrates a number of deficiencies in communication between artificial embodied agents and humans, suggesting that artificial embodied agents still lack many of the desirable communicative aspects of human-to-human service encounters.

Towards multimodal spoken language corpora: TransTool and SyncTool

This paper argues for the usefulness of multimodal spoken language corpora and specifies components of a platform for the creation, maintenance and exploitation of such corpora. Two of the components, which have already been implemented as prototypes, are described in more detail: TransTool and SyncTool. TransTool is a transcription editor meant to facilitate and partially automate the task of a human transcriber, while SyncTool is a tool for aligning the resulting transcriptions with a digitized audio and video recording in order to allow synchronized presentation of different representations (e.g., text, audio, video, acoustic analysis). Finally, a brief comparison is made between these tools and other programs developed for similar purposes.