Kerstin Limbrecht

Multimodal emotion classification in naturalistic user behavior

The design of intelligent personalized interactive systems, having knowledge about the users state, his desires, needs and wishes, currently poses a great challenge to computer scientists. In this study we propose an information fusion approach combining acoustic, and biophysiological data, comprising multiple sensors, to classify emotional states. For this purpose a multimodal corpus has been created, where subjects undergo a controlled emotion eliciting experiment, passing several octants of the valence arousal dominance space. The temporal and decision level fusion of the multiple modalities outperforms the single modality classifiers and shows promising results.

Similarities and differences of emotions in human–machine and human–human interactions: what kind of emotions are relevant for future companion systems?

Cognitive-technical intelligence is envisioned to be constantly available and capable of adapting to the users emotions. However, the question is: what specific emotions should be reliably recognised by intelligent systems? Hence, in this study, we have attempted to identify similarities and differences of emotions between human–human (HHI) and human–machine interactions (HMI). We focused on what emotions in the experienced scenarios of HMI are retroactively reflected as compared with HHI. The sample consisted of N = 145 participants, who were divided into two groups. Positive and negative scenario descriptions of HMI and HHI were given by the first and second groups, respectively. Subsequently, the participants evaluated their respective scenarios with the help of 94 adjectives relating to emotions. The correlations between the occurrences of emotions in the HMI versus HHI were very high. The results do not support the statement that only a few emotions in HMI are relevant. Practitioner Summary: This study sought to identify the relevant emotions in different technical domains their companion systems tend to use. Overall, the 20 essential emotions found as highly relevant for HMI were as follows: (i) positive, i.e. satisfied, pleased, happy, relieved, pleasant, well, serene, optimistic, confident and self-confident and (ii) negative, i.e. annoyed, aggravated, impatient, angry, unsatisfied, displeased, irritable, frustrated, enraged and tense.

Where have they gone? - a discussion on the balancing act of female doctors between work and family.

Goals: The increasing number of vacant positions for doctors increasingly puts the issue of reconciling work and family into the spotlight in companies, hospitals and universities, as increased expectations of a better work-life balance are seen as one of the reasons for these vacancies. Highly qualified professionals are trained, but not available to the labour market. The aim is to summarise what difficulties doctors who want to have a family and their potential employers must face. Methods: The following articles show the current state of research and potential starting points for an optimisation of the medical profession from a family-friendly perspective and intend to stimulate debate. Results: Some basic steps towards better work-life balance have already been taken, such as the provision of childcare places and the increasing availability of more flexible working patterns. But it seems that these measures, since they have been implemented neither sufficiently nor universally, do not suffice to secure the next generation of staff. Especially women in leadership positions are still rare to find. Conclusions: Both male and female doctors want better quality of life by achieving a better work-life balance. The expansion of family-friendly services is seen as a necessary step to allow female doctors to successfully combine work and family.

Mapping discrete emotions into the dimensional space: An empirical approach

A critical task in Affective Computing is the reliable assessment of emotional states. The two most prominent approaches to classify emotions are categorical concepts of discrete emotions (e.g. OCC) and dimensional models typically using the pleasure - arousal - dominance space (PAD). In current research and applications, however, there is little overlap between these two concepts. A mapping of discrete categories into the dimensional space would offer new possibilities to model the emotional states of users and artificial agents, though. We hence let N=70 healthy subjects place the labels of discrete OCC emotions into PAD space according to their subjective knowledge with a simple visual tool. There was a high inter-subject consistency regarding the positioning of OCC emotions for the dimension of pleasure. However, arousal and dominance ratings showed considerably greater variance. We conclude that global and reliable mappings of OCC emotions into the PAD space can best be provided for the pleasure dimension. The exact positioning of discrete emotions regarding arousal and dominance can only be gained by individual calibration of a given user in a strict within-subject approach.

The Influence of Naturalness, Attractiveness and Intensity on Facial Emotion Recognition

Understanding the determinants facial emotion recognition is still one of the main topics in emotion research. Mimic expressions are not only a representation of feelings caused by emotions, but are also an important communication channel in social interaction. Research of the last 20 years could show that emotion recognition abilities differ within and between individuals. Moreover, every basic emotion seems to be processed differently in the human brain. Underlying processes are still not clear. In this study, the common practice of presenting pictures of faces showing basic emotions to participants was used in a more methodological sense. Possible determining factors in facial emotion recognition, like naturalness and intensity of the expressed emotion and the attractiveness of the photographed person, were analyzed.

Emotion Identification and Modelling on the Basis of Paired Physiological Data Features for Companion Systems

Abstract A technical companion system should be able to detect its users emotion and model the users emotional state in order to react to it accordingly. We have developed a novel method to determine a users most significant emotional change in the two emotion dimensions of pleasure and arousal on the basis of paired data features of physiological data when comparing two events. An experiment was set up where participants first viewed blocked IAPS picture presentations and then took part in a mental training wizard-of-oz scenario. Six meaningful features from four physiological channels of the IAPS picture presentation data - containing two electromyography channels ( corrugator supercilii and zygomaticus major ), skin conductance and peripheral blood volume - were extracted. Three pairs of features were found to contain valuable information about emotional changes when comparing two situations with different emotional contents. The method was then tested on a new blocked IAPS dataset and on the wizard-of-oz interaction scenario dataset to verify its performance. In 75% of the subjects, the detected emotion matched one of the two induced emotions. This information could be used by future companion technologies for modelling of the users current emotional state in the two-dimensional emotion space of pleasure and arousal.