Gregor Bertrand

Companion-Technology: Towards User- and Situation-Adaptive Functionality of Technical Systems

The properties of multimodality, individuality, adaptability, availability, cooperativeness and trustworthiness are at the focus of the investigation of Companion Systems. In this article, we describe the involved key components of such a system and the way they interact with each other. Along with the article comes a video, in which we demonstrate a fully functional prototypical implementation and explain the involved scientific contributions in a simplified manner. The realized technology considers the entire situation of the user and the environment in current and past states. The gained knowledge reflects the context of use and serves as basis for decision-making in the presented adaptive system.

Context Models for Adaptive Dialogs and Multimodal Interaction

This article presents a context adaptive approach for multimodal interaction for the use in cognitive technical systems, so called companion systems. A system architecture is presented and we clarify where context awareness occurs on different levels with a layered context model. The focus is on the topics of dialog management, multimodal fusion, and multimodal fission, as the main participants in interaction. An implemented prototype is presented, yielding some concrete instances of the described context models and the adaption to them.

GEEDI - Guards for Emotional and Explanatory DIalogues

In this paper, we describe the development of a dialogue model that integrates emotional dialogue strategies and explanations in a simple hence powerful way. As intelligent environments make inroads into the market, the need for user-friendly interaction with these systems grows. Pro-active reaction to user knowledge and emotions is one of the key points in user-friendly adaption of dialogue systems and therefore one of the main topics of research. As intelligent environments grow in complexity and field of application, the knowledge requirements for the user grow as well. Therefore it is vitally important to impart knowledge and information in an emotionally sensitive and user-aware way. In our dialog model we consider the natural structure of a nontrivial dialogue as a structure divided into several goals. These goals are protected by so called guards which represent preconditions which have to be fulfilled in order to tackle the related goal.

CALIGRAPHI - Creation of Adaptive Dialogues Using a Graphical Interface

In the research area of spoken language dialogue systems there are many ways for modeling dialogues. The dialog models particular structure depends on the algorithm used to interpret it. In most cases a dialogues model is quite difficult to understand and to create. We present a novel technique for modeling dialogues based on ready to use open source tools in an easy and understandable way. Making use of our approach a dialogue designer (unfamiliar with the internals of the dialogue manager) can simply develop even complex and adaptive dialogues. The dialogues are then ready to be interpreted by the dialogue management in order to integrate them seamlessly into the spoken language dialogue system.

Adaptive Explanation Architecture for Maintaining Human-Computer Trust

One of the most important challenges in the field of human-computer interaction is maintaining and enhancing the willingness of a user to interact with a technical system. This cooperativeness provides a solid basis for a real dialogue between user and technical system. The goals and tasks of an intelligent technical system seem unrealistic without it. In particular intelligent technical systems, which are continually assisting the user in his everyday life, degenerate without this willingness for dialogue to helpers for quite simple tasks and cannot fulfil their original purpose as intelligent assistants for complex tasks. Trust has shown to be an important factor influencing the frequency and kind of usage. If the user does not understand system actions or instructions, the trust of the user in the system will decrease and this can lead to a reduced frequency or in the worst case to a total cease of usage. Therefore, the intelligibility of a technical system should be upheld. This paper is concerned with how the intelligibility of an intelligent technical system can be upheld by providing explanations to the user. Providing explanations may prevent or at least decrease the loss of trust. However, trust is a complex construct consisting of different bases. We show why and how these bases of trust should be treated by giving individual kinds of explanations.

Adapting Dialogue to User Emotion - A Wizard-of-Oz study for adaptation strategies

In Spoken Language Dialogue Systems (SLDS) there is a growing trend to make the system act more human-like to render the dialogue more agreeable for the user. One of the methods to achieve these system qualities is to take into account the way that the user is feeling and to react appropriately. However, it is not clear what appropriate means in this context. What is the system to do when sensing a specific emotion? Is the appropriate reaction user-dependent? Situation-dependent? Context-dependent? Is the appropriate reaction dependent on the complex process of neurotransmitters circulating in the brain system of the user at the moment of the reaction? In order to address some of the questions above we conduct a Wizard-of- Oz study based on the findings of a preliminary study. We are collecting data about different kinds of users that are put in a cognitively demanding situation. We try to find out relations between different types of users and different types of system strategies that address their emotional state.

Individual emotional profiles in Wizard-of-Oz-experiments

One of the most important and difficult fields in research of assistive environment technology is the recognition of emotional and motivational users states. Emotion studies in the past show, that there are only a few universal interindividual valid psychobiological profiles states, which are stable associated with a users emotional state. In this approach we look for intraindividual valid psychobiological patterns of emotions and motivations. In order to predict such states separate for different subjects we introduced a calibration procedure for each of the 20 subjects. We expect higher emotion recognition rates than preceding studies focusing on universal patterns on sample data. First results will be presented at the conference.

What Do You Want to Do Next? Providing the User with More Freedom in Adaptive Spoken Dialogue Systems

Spoken dialogue is a suitable form for user interaction in intelligent environments because of its low demands to presentation hardware (e.g. microphone and speaker set) and its low cognitive demand. That means that the user can still do something else while communicating with a spoken dialogue system (SDS). In this paper, we investigate the demands to modern SDSs in intelligent environments and how to provide the user with more freedom by using constraint programming techniques for dialogue management. For this we present an architecture which allows for adaptivity and freedom in SDSs.