Birgit Lugrin

Context-Aware Automated Analysis and Annotation of Social Human--Agent Interactions

The outcome of interpersonal interactions depends not only on the contents that we communicate verbally, but also on nonverbal social signals. Because a lack of social skills is a common problem for a significant number of people, serious games and other training environments have recently become the focus of research. In this work, we present NovA (Nonverbal behavior Analyzer), a system that analyzes and facilitates the interpretation of social signals automatically in a bidirectional interaction with a conversational agent. It records data of interactions, detects relevant social cues, and creates descriptive statistics for the recorded data with respect to the agents behavior and the context of the situation. This enhances the possibilities for researchers to automatically label corpora of human--agent interactions and to give users feedback on strengths and weaknesses of their social behavior.

Games are Better than Books: In-Situ Comparison of an Interactive Job Interview Game with Conventional Training

Technology-enhanced learning environments are designed to help users practise social skills. In this paper, we present and evaluate a virtual job interview training game which has been adapted to the special requirements of young people with low chances on the job market. The evaluation spanned three days, during which we compared the technology-enhanced training with a traditional learning method usually practised in schools, i.e. reading a job interview guide. The results are promising as professional career counsellors rated the pupils who trained with the system significantly better than those who learned with the traditional method.

Modeling and Evaluating a Bayesian Network of Culture-Dependent Behaviors

Anthropomorphic user interfaces such as virtual agents or humanoid robots aim on simulating believable human behavior. As human behavior is influenced by diversifying factors such as cultural background, research in anthropomorphic user interfaces considers culture background for their behavioral models as well. This paper presents a hybrid approach of creating a culture-specific model of non-verbal behaviors for simulated dialogs based on both: theoretical knowledge and empirical data. Therefore, the structure and variables of a Bayesian network are designed based on models and theories from the social sciences, while its parameters are learned from a video corpus of German and Japanese conversations in first time meeting scenarios. To validate the model a 10-fold-cross-validation has been conducted, suggesting that with the model culture-specific behavior can automatically be generated for some of the investigated behavioral aspects.

Visualization Support for Comparing Energy Consumption Data

Providing effective feedback can empower users to change their behaviour and take the necessary actions to reduce their energy consumption. The types of feedback that allow comparison of energy usage seem to be particularly valuable. This paper introduces the time-stack visualization, which has been designed to support comparisons of individual and collective energy usage data. It also describes a user study conducted to compare the effectiveness of time-stack against a similar visualization called time-pie. The results show that although the two visualizations are generally comparable in their effectiveness, users rate time-stack more favourably.

There Once Was a Robot Storyteller: Measuring the Effects of Emotion and Non-verbal Behaviour

Social Robots bear great potential to tell stories to human listeners. Through their embodiment and ability to display non-verbal and emotional behaviour, additional modalities can be used to transport the user into the story compared to traditional media such as books or audio books. Based on theoretical knowledge about the design of social robot storytellers and the analysis of human storytellers, we designed the behaviour of an emotional social robot storyteller and compared it to a neutral version of the robotic storyteller and the voice of a human storyteller represented by an audio book. Results suggest that the emotional robot is able to transport the participants equally well as the traditional audio book, while the neutral robot performed worse. We therefore claim, that emotional behaviour in the domain of robotic storytelling should be carefully designed to support the story rather then preventing a good transportation.

Investigating Politeness Strategies and Their Persuasiveness for a Robotic Elderly Assistant

This work is targeted towards the development of a Robotic Elderly Assistant REA system that provides assistance in the form of recommendations to support single-living elderly people in their domestic environment. To avoid potential face threats the REA should be as polite as possible whilst keeping a certain persuasiveness to promote its recommendations. This paper investigates different verbalizations of the REAs recommendations regarding their perceived politeness as well as their persuasiveness. We present the results of a laboratory study with younger adults and a user study with the inhabitants of a retirement home. Results suggest that the different politeness strategies reflected different levels of politeness in both studies, while their perceived persuasiveness needs further investigation in the domain of elderly care.

Combining a Data-Driven and a Theory-Based Approach to Generate Culture-Dependent Behaviours for Virtual Characters

To incorporate culture into intelligent systems, there are two approaches that are commonly proposed. Theory-based approaches that build computational models based on cultural theories to predict culture-dependent behaviours, and data-driven approaches that rely on multimodal recordings of existing cultures. Based on our former work, we present a hybrid approach of integrating culture into a Bayesian Network that aims at predicting culture-dependent non-verbal behaviours for a given conversation. While the model is structured based on cultural theories and theoretical knowledge on their influence on prototypical behaviour, the parameters of the model are learned from a multimodal corpus recorded in the German and Japanese cultures. The model is validated in two ways: With a cross-fold validation we estimate the power of the network by predicting behaviours for parts of the recorded data that were not used to train the network. Secondly we performed a perception study with virtual characters whose behaviour is driven by the calculations of the network and are rated by members of the German and Japanese cultures. With this chapter, we aim at giving guidance for other culture-specific generation approaches by providing a hybrid methodology to build culture-specific computational models as well as potential approaches for their evaluation.

In the Face of Emotion: A Behavioral Study on Emotions Towards a Robot Using the Facial Action Coding System

Since the arrival of social robots, emotions have become a vital topic for the field of Human-Robot Interaction (HRI). We experimentally investigated the multilevel phenomenon of emotions with a multi-method approach. By using the Facial Action Coding System (FACS) we analyzed the facial expressions of 33 participants watching an anthropomorphic robot (Reeti) in a friendly interaction or being tortured. The results show that human emotional reactions towards an anthropomorphic robot can be captured via facial expressions as well as via self-report measurements. With this and future studies, we aim to provide information on spontaneous facial expressions towards a robot that could serve as guidance for automatic approaches.

Female Robots as Role-Models? - The Influence of Robot Gender and Learning Materials on Learning Success.

Social robots are likely to become a part of everybody’s future. One of their major areas of application lies in the domain of education. Especially for female learners, female teachers can act as role models in what learners might perceive as stereo-typical male learning domains. The present contribution investigates whether the gender of a social robot and learning materials that were either designed stereotypically male or stereotypically female, influence the learning success of female learners. A user study revealed that female students gained more knowledge when learning with a female robot using stereotypical male materials. We were thus taking a first step towards the possibility that social robots could serve as a tool to counteract social believes and minimize stereotypes.

A Pilot Study on Adaptive Gesture Use in Interaction with Non-native Listeners

Given that human speakers adapt their communicative behavior towards non-native listeners -- a phenomenon known as foreigner talk or teacher talk -- the communicative behavior of an interactive, intelligent virtual agent (IIVA) should also to be adaptive towards the needs of non-native listeners. To investigate the question whether it makes sense to distinguish diferent degrees of language proficiency in non-native addressees when designing communicative behavior skills for IIVAs in mixed-cultural settings, we present first results from a pilot study that is meant to prepare a comprehensive corpus collection. Native speakers of German were asked to explain given German terms to non-native speakers of either low or intermediate language proficiency in German. Results showed significant differences in gesture frequency and also in the types and size of gestures being used, depending on the language proficiency of the non-native listener.