Natascha Esau

Real-Time Facial Expression Recognition Using a Fuzzy Emotion Model

This paper presents the fuzzy video based emotion recognition system VISBER, that allows to analyze facial expressions in video sequences. In order to process images in real-time a tracking mechanism is used for face localization. The fuzzy classification itself analyzes the deformation of a face separately in each image. In contrast to most existing approaches, also blended emotions with varying intensities as proposed by psychologists can be handled. For this purpose we propose a fuzzy emotion model which is generally applicable for also for other emotion recognition solutions. Furthermore, VISBER supports the automatic adaptation to the characteristics of individual human faces by a short training phase that can be done before the emotion recognition starts.

Fuzzy emotion recognition in natural speech dialogue

This paper describes the realization of a natural speech dialogue for the robot head MEXI with focus on its emotion recognition. Specific for MEXI is that it can recognize emotions from natural speech and also produce natural speech output with emotional prosody. For recognizing emotions from the prosody of natural speech we use a fuzzy rule based approach. Since MEXI often communicates with well known persons but also with unknown humans, for instance at exhibitions, we realized a speaker-dependent mode as well as a speaker-independent mode in the prosody based emotion recognition. A key point of our approach is that it automatically selects the most significant features from a set of twenty analyzed features based on a training data base of speech samples. This is important according to our results, since the set of significant features differs considerably between the distinguished emotions. With our approach we reached average recognition rates of 84% in speaker-dependent mode and 60% in speaker-independent mode.

Prosody based emotion recognition for MEXI

This paper describes the emotion recognition from natural speech as realized for the robot head MEXI. We use a fuzzy logic approach for analysis of prosody in natural speech. Since MEXI often communicates with well known persons but also with unknown humans, for instance at exhibitions, we realized a speaker dependent mode as well as a speaker independent mode in our prosody based emotion recognition. A key point of our approach is that it automatically selects the most significant features from a set of twenty analyzed features based on a training database of speech samples. This is important according to our results, since the set of significant features differs considerably between the distinguished emotions. With our approach we reach average recognition rates of 84% in speaker dependent mode and 60% in speaker independent mode.

Feature based Face Localization and Recognition on Mobile Devices

Due to increasing miniaturization and decreasing prizes of cameras more and more mobile devices like PDAs or smartphones are equipped with a camera. Due to this fact, mobile face recognition will gain popularity in identifying persons e.g. in order to prevent unauthorized use or access to data and equipment. In this paper, the feature based face localization and recognition system FaceScry is presented. In spite of the limited resources available on mobile devices, FaceScry is able to localize an arbitrary number of faces with different sizes in images taken under varying illumination conditions in real-time. Also face recognition is size invariant due to the selected set of features, which mainly consists of angles and cross ratios. Since it stores reference face data for recognition as feature vectors and not as huge image data, FaceScry also allows for keeping a reasonable personal face data base for recognizing a set of persons on the smartphone

Integration of emotional reactions on human facial expressions into the robot head MEXI

Emotion recognition and adequate reactions are a crucial part of human communication and hence should also be considered for interactions between humans and robots. In this paper we present the robot head MEXI which is able to recognize emotions of its human counterpart from a video sequence using a fuzzy rule based approach. It reacts on these perceptions in an emotional way. Therefor MEXI maintains an internal state made up of (artificial) emotions and drives. This internal state is used to evaluate its perceptions and action alternatives and controls its behavior on the basis of this evaluation. This is a major difference between MEXI and usual goal based agents that rely on a world model to control and plan their actions. For MEXI the behavior based programming paradigm originally developed by Arkin for robot navigation was extended to support a multidimensional control architecture based on emotions and drives.

Approximation of Environment Models for an Air Gap Adjustment System in a Hybrid Planning Context

In mechatronic systems a lot of components above the controller level are needed for the development towards self-optimizing systems. Among them a hybrid planning architecture integrating discrete and continuous domains is of major importance to support the permanent determination of system objectives and their implementation during the course of action, which defines the principle of self-optimizing mechatronic systems. Such a novel hybrid planning architecture is outlined in this paper. In order to plan efficiently, environment models are needed for predicting future system behaviors. In this paper we propose a fuzzy logic based approach to environment modeling and apply it in a railway-bound domain within the context of an air gap adjustment system for a dual-fed linear motor powering a wheeled train.

Emotional Communication with the Robot Head MEXI

This paper presents the robot head MEXI which is able to communicate to humans in an emotional way. MEXI recognizes emotions of its human counterpart from the prososdy of his or her natural speech using a fuzzy rule based approach. MEXI reacts on its perceptions by showing artificial emotions in its facial expressions and in the prosody of its synthesized natural speech. MEXI does not rely on a world model to control and plan its actions like usual goal based agents. Instead MEXI uses its internal state consisting of emotions and drives to evaluate its perceptions and action alternatives and controls its behavior on the basis of this evaluation. For MEXI, the behavior based programming paradigm originally developed by Arkin for robot navigation was extended to support a multidimensional control architecture based on emotions and drives

Hierarchical hybrid planning for a self-optimizing active suspension system

This paper presents a hierarchical hyprid planning approach developed for realizing self-optimizing mechatronic systems. The hybrid planning approach is based on a discrete plan and a forecast of continuous system behavior generated by simulation during runtime. This does not only allow an online adaptation of a previously generated plan with regard to the actual system state as well as the current environmental conditions, even the planning objectives may be adapted to newly arising needs during system operation. The planner exploits the hierarchy already present in the system model by considering hierarchical parameterizations of the system as a discrete dimension of choice. The parameterizations are proposed by a novel hierarchical multiobjective optimization, which calculates Pareto points in a bottom up fashion, taking into account the constraints imposed by lower levels of the system hierarchy on the higher ones. Evaluations of the hierarchical hybrid planner in the context of an innovative railbound transport system show that it outperforms its non-hierarchical predecessor.

Real-Time Camera Tracking for Mobile Devices: The VisiTrack System

This paper presents VisiTrack, a novel approach for video based incremental tracking in real-time. The major objectives in the development of VisiTrack was to design or select algorithms that are well suited for embedded real-time computation. We had a special focus on latency reduction and storage minimization since the algorithms should run on mobile devices like PDAs with the appropriate extension, i.e. mainly a camera, in real-time. The image analysis, camera localization and feature position approximation of VisiTrack are explained in detail. The CV-SDF model, an extension of Synchronous Dataflow graphs (SDF), supporting the principles of linear processing and fine-grained pipelining was defined and applied for the design of all VisiTrack modules in order to fulfill real-time constraints and reduce system latency. Furthermore the camera localization and position approximation include mechanisms for minimization of errors that may arise for instance due to measurement inaccuracies. Current applications of VisiTrack in the augmented reality domain and robotic self localization show its good performance. However VisiTrack is not limited to these application domains.

Emotional Robot Competence and Its Use in Robot Behavior Control

Emotional competence plays a crucial role in human communication and hence has also gained increasing attention for the design of interaction processes between humans and robots. Like humans, emotionally intelligent robots should be capable of coping with the emotions of their human counterpart as well as with their own artificial emotions, which requires some key competencies. Robots need the abilities to recognize and to understand human emotions in a certain situation, they have to be able to react adequately in order to regulate their own emotions as well as the emotions of their human counterpart, and they have to express their own emotions in an adequate way. In this paper we elaborate the concepts of emotional competence and show how artificial emotions and drives can be integrated into a robotic system to realize emotionally competent and proactive behavior. For this purpose we propose a fuzzy emotion model which is used as basis for human emotion recognition and for representing the static aspects of a robot’s emotions. Subsequently, a dynamic model for artificial robotic emotions and drives that allows for adequate control of robotic behavior is described. Furthermore, the application of our concepts in the emotionally competent robot head MEXI is presented.