Viktor Kessler

On Annotation and Evaluation of Multi-modal Corpora in Affective Human-Computer Interaction

In this paper, we discuss the topic of affective human-computer interaction from a data driven viewpoint. This comprises the collection of respective databases with emotional contents, feasible annotation procedures and software tools that are able to conduct a suitable labeling process. A further issue that is discussed in this paper is the evaluation of the results that are computed using statistical classifiers. Based on this we propose to use fuzzy memberships in order to model affective user state and endorse respective fuzzy performance measures.

Continuous Multimodal Human Affect Estimation using Echo State Networks

A continuous multimodal human affect recognition for both arousal and valence dimensions in a non-acted spontaneous scenario is investigated in this paper. Different regression models based on Random Forests and Echo State Networks are evaluated and compared in terms of robustness and accuracy. Moreover, an extension of Echo State Networks to a bi-directional model is introduced to improve the regression accuracy. A hybrid method using Random Forests, Echo State Networks and linear regression fusion is developed and applied on the test subset of the AVEC16 challenge. Finally, the label shift and prediction delay is discussed and an annotator specific regression model, as well as fusion architecture, is proposed for future work.

Audio-Visual Recognition of Pain Intensity

In this work, a multi-modal pain intensity recognition system based on both audio and video channels is presented. The system is assessed on a newly recorded dataset consisting of several individuals, each subjected to 3 gradually increasing levels of painful heat stimuli under controlled conditions. The assessment of the dataset consists of the extraction of a multitude of features from each modality, followed by an evaluation of the discriminative power of each extracted feature set. Finally, several fusion architectures, involving early and late fusion, are assessed. The temporal availability of the audio channel is taken in consideration during the assessment of the fusion architectures.

The SenseEmotion Database: A Multimodal Database for the Development and Systematic Validation of an Automatic Pain- and Emotion-Recognition System

In our modern industrial society the group of the older (generation 65+) is constantly growing. Many subjects of this group are severely affected by their health and are suffering from disability and pain. The problem with chronic illness and pain is that it lowers the patient’s quality of life, and therefore accurate pain assessment is needed to facilitate effective pain management and treatment. In the future, automatic pain monitoring may enable health care professionals to assess and manage pain in a more and more objective way. To this end, the goal of our SenseEmotion project is to develop automatic pain- and emotion-recognition systems for successful assessment and effective personalized management of pain, particularly for the generation 65+. In this paper the recently created SenseEmotion Database for pain- vs. emotion-recognition is presented. Data of 45 healthy subjects is collected to this database. For each subject approximately 30 min of multimodal sensory data has been recorded. For a comprehensive understanding of pain and affect three rather different modalities of data are included in this study: biopotentials, camera images of the facial region, and, for the first time, audio signals. Heat stimulation is applied to elicit pain, and affective image stimuli accompanied by sound stimuli are used for the elicitation of emotional states.

Selecting Features from Foreign Classes.

Supervised learning algorithms restrict the training of classification models to the classes of interest. Other related classes are typically neglected in this process and are not involved in the final decision rule. Nevertheless, the analysis of these foreign samples and their labels might provide additional information on the classes of interest. By revealing common patterns in foreign classification tasks it might lead to the identification of structures suitable for the original classes. This principle is used in the field of transfer learning. In this work, we investigate the use of foreign classes for the feature selection process of binary classifiers. While the final classification model is trained according to the traditional supervised learning scheme, its feature signature is designed for separating a pair of foreign classes. We systematically analyse these classifiers in \(10 \times 10\) cross-validation experiments on microarray datasets with multiple diagnostic classes. For each evaluated classification model, we observed foreign feature combinations that outperformed at least 90% of those feature sets designed for the original diagnostic classes on at least 88.9% of all datasets.

3D Network exploration and visualisation for lifespan data

BackgroundThe Ageing Factor Database AgeFactDB contains a large number of lifespan observations for ageing-related factors like genes, chemical compounds, and other factors such as dietary restriction in different organisms. These data provide quantitative information on the effect of ageing factors from genetic interventions or manipulations of lifespan. Analysis strategies beyond common static database queries are highly desirable for the inspection of complex relationships between AgeFactDB data sets. 3D visualisation can be extremely valuable for advanced data exploration.ResultsDifferent types of networks and visualisation strategies are proposed, ranging from basic networks of individual ageing factors for a single species to complex multi-species networks. The augmentation of lifespan observation networks by annotation nodes, like gene ontology terms, is shown to facilitate and speed up data analysis. We developed a new Javascript 3D network viewer JANet that provides the proposed visualisation strategies and has a customised interface for AgeFactDB data. It enables the analysis of gene lists in combination with AgeFactDB data and the interactive visualisation of the results.ConclusionInteractive 3D network visualisation allows to supplement complex database queries by a visually guided exploration process. The JANet interface allows gaining deeper insights into lifespan data patterns not accessible by common database queries alone. These concepts can be utilised in many other research fields.

Visual Confusion Recognition in Movement Patterns from Walking Path and Motion Energy

For elderly people healthcare in ambient living environments, recognizing confusion states in an automatic and non-contact manner is essential. In this work we provide a visual approach to confusion recognition consisting of behavior monitoring and movement pattern analysis. To collect data for evaluation, we created a dataset from a search experiment. After extracting and analyzing the movement patterns, we achieved a recognition rate of \(89.6\%\) when cross-validating over different subjects and \(88.9\%\) when testing on a new set of samples. To our knowledge, we are the first to investigate confusion recognition using visual information. Our work shows that the mental confusion can be effectively recognized based on the movement pattern.

Machine Learning Driven Heart Rate Detection with Camera Photoplethysmography in Time Domain

Measuring bio signals such as the heart rate in non medical applications is gaining an increasing importance. With camera based photoplethysmography (PPG) it is possible to measure the heart rate remotely with built in webcams of every tablet and laptop. Recent research with machine learning based methods showed great success compared to signal processing based methods. In this paper, we use k-nearest neighbor (kNN) and multilayer perceptron (MLP) with an alternative representation of the input vector. Estimating the quality of peaks with a Gaussian distribution could further improve the detection. Overall we could improve the root mean square error (RMSE) from 23.97 to 8.62.

A Reinforcement Learning Algorithm to Train a Tetris Playing Agent

In this paper we investigate reinforcement learning approaches for the popular computer game Tetris. User-defined reward functions have been applied to TD(0) learning based on e-greedy strategies in the standard Tetris scenario. The numerical experiments show that reinforcement learning can significantly outperform agents utilizing fixed policies.