Spiros Ioannou

Robust feature detection for facial expression recognition

This paper presents a robust and adaptable facial feature extraction system used for facial expression recognition in human-computer interaction (HCI) environments. Such environments are usually uncontrolled in terms of lighting and color quality, as well as human expressivity and movement; as a result, using a single feature extraction technique may fail in some parts of a video sequence, while performing well in others. The proposed system is based on a multicue feature extraction and fusion technique, which provides MPEG-4-compatible features assorted with a confidence measure. This confidence measure is used to pinpoint cases where detection of individual features may be wrong and reduce their contribution to the training phase or their importance in deducing the observed facial expression, while the fusion process ensures that the final result regarding the features will be based on the extraction technique that performed better given the particular lighting or color conditions. Real data and results are presented, involving both extreme and intermediate expression/emotional states, obtained within the sensitive artificial listener HCI environment that was generated in the framework of related European projects.

An intelligent system for facial emotion recognition

An intelligent emotion recognition system, interweaving psychological findings about emotion representation with analysis and evaluation of facial expressions has been generated and its performance has been investigated with experimental real data. Additionally, a fuzzy rule based system has been created for classifying facial expressions to the six archetypal emotion categories. The continuous 2-D emotion space was then examined and a pool of known and novel classification and clustering techniques have been applied to our data obtaining high rates in classification and clustering into quadrants of the emotion representation space.

Adaptive on-line neural network retraining for real life multimodal emotion recognition

Emotions play a major role in human-to-human communication enabling people to express themselves beyond the verbal domain. In recent years, important advances have been made in unimodal speech and video emotion analysis where facial expression information and prosodic audio features are treated independently. The need however to combine the two modalities in a naturalistic context, where adaptation to specific human characteristics and expressivity is required, and where single modalities alone cannot provide satisfactory evidence, is clear. Appropriate neural network classifiers are proposed for multimodal emotion analysis in this paper, in an adaptive framework, which is able to activate retraining of each modality, whenever deterioration of the respective performance is detected. Results are presented based on the IST HUMAINE NoE naturalistic database; both facial expression information and prosodic audio features are extracted from the same data and feature-based emotion analysis is performed through the proposed adaptive neural network methodology.

An intelligent scheme for facial expression recognition

This paper addresses the problem of emotion recognition in faces through an intelligent neuro-fuzzy system, which is capable of analysing facial features extracted following the MPEG-4 standard, associating these features to symbolic fuzzy predicates, and reasoning on the latter, so as to classify facial images according to the underlying emotional states. Results are presented which illustrate the capability of the developed system to analyse and recognise facial expressions in human computer interaction applications.

Confidence-Based Fusion of Multiple Feature Cues for Facial Expression Recognition

Since facial expressions are a key modality in human communication, the automated analysis of facial images for the estimation of the displayed expression is essential in the design of intuitive and accessible human computer interaction systems. In most existing rule-based expression recognition approaches, analysis is semiautomatic or requires high quality video. In this paper we propose a feature extraction system which combines analysis from multiple channels based on their confidence, to result in better facial feature boundary detection. The facial features are then used for expression estimation. The proposed approach has been implemented as an extension to an existing expression analysis system in the framework of the IST ERMIS project

Dealing with feature uncertainty in facial expression recognition

Since facial expressions are a key modality in human communication, the automated analysis of facial images for the estimation of the displayed expression is central in the design of intuitive and human friendly human computer interaction systems. In existing approaches, over-formalised description of knowledge concerning the human face and human expressions, as well as failures of the image and video processing components, often lead to misclassification. In this paper, we propose the utilisation of extended fuzzy rules for the more flexible description of knowledge, and the consideration of uncertainty and lack of confidence in the process of feature extraction from image and video. The two are combined using a flexible possibilistic rule evaluation structure, leading to more robust overall operation. The proposed approach has been implemented as an extension to an existing expression analysis system and conclusions from comparative study have been drawn.

Combination of multiple extraction algorithms in the detection of facial features

Automated analysis of facial images for the estimation of the displayed expression is essential in the design of intuitive and accessible human computer interaction systems. In existing rule-based expression recognition approaches, different feature extraction techniques have been tested that allow for the automatic detection of feature points, providing the required input for a rule based expression analysis; each one of these techniques outperforms others under specific constraints. In this paper we propose a feature extraction system which combines analysis from multiple channels based on their confidence, to result in better, error resilient facial feature boundary detection. The proposed approach has been implemented as an extension to an existing expression analysis system in the framework of the 1ST ERMIS project.

Adaptive Rule-Based Facial Expression Recognition

This paper addresses the problem of emotion recognition in faces through an intelligent neuro- fuzzy system, which is capable of analysing facial features extracted following the MPEG-4 standard and classifying facial images according to the underlying emotional states, following rules derived from expression profiles. Results are presented which illustrate the capability of the developed system to analyse and recognise facial expressions in man-machine interaction applications.

Intelligent Facial Analysis and Expression Recognition

Since facial expressions are a key modality in human communication, the automated analysis of facial images and video for the estimation of the displayed expression is central in the design of intuitive and human friendly computer interaction systems. In this paper we present an intelligent feature extraction system which combines analysis from multiple channels based on their confidence, to result in better, error resilient facial feature boundary detection. Neural networks are a key component of the system. Issues such as uncertainty and lack of confidence in the process of feature extraction are considered during the expression analysis and recognition. Various results are presented which illustrate the performance of the method.

Possibilistic Rule Evaluation: A case study in facial expression analysis

Fuzzy rule systems are an important element in the arsenal of automated control, as they are able to process the input provided by sensors and provide their output in very small times. Lately, there has been augmented interest in utilizing fuzzy rule systems in a wider range of applications, due to the intuitive way in which they represent and utilize knowledge. In these contexts, input is not always readily available from a sensor but often provided as the questionable output of another expert system or even totally missing. In this paper we propose a novel approach to rule evaluation that is able to operate under such uncertain conditions and evaluate it by applying it to the case of facial expression analysis. Our approach has a possibilistic, rather that probabilistic, flavor.