Anisha Halder

General and Interval Type-2 Fuzzy Face-Space Approach to Emotion Recognition

Facial expressions of a person representing similar emotion are not always unique. Naturally, the facial features of a subject taken from different instances of the same emotion have wide variations. In the presence of two or more facial features, the variation of the attributes together makes the emotion recognition problem more complicated. This variation is the main source of uncertainty in the emotion recognition problem, which has been addressed here in two steps using type-2 fuzzy sets. First a type-2 fuzzy face space is constructed with the background knowledge of facial features of different subjects for different emotions. Second, the emotion of an unknown facial expression is determined based on the consensus of the measured facial features with the fuzzy face space. Both interval and general type-2 fuzzy sets (GT2FS) have been used separately to model the fuzzy face space. The interval type-2 fuzzy set (IT2FS) involves primary membership functions for m facial features obtained from n-subjects, each having l-instances of facial expressions for a given emotion. The GT2FS in addition to employing the primary membership functions mentioned above also involves the secondary memberships for individual primary membership curve, which has been obtained here by formulating and solving an optimization problem. The optimization problem here attempts to minimize the difference between two decoded signals: the first one being the type-1 defuzzification of the average primary membership functions obtained from the n-subjects, while the second one refers to the type-2 defuzzified signal for a given primary membership function with secondary memberships as unknown. The uncertainty management policy adopted using GT2FS has resulted in a classification accuracy of 98.333% in comparison to 91.667% obtained by its interval type-2 counterpart. A small improvement (approximately 2.5%) in classification accuracy by IT2FS has been attained by pre-processing measurements using the well-known interval approach.

Interval type-2 fuzzy model for emotion recognition from facial expression

The paper proposes a new approach to emotion recognition from facial expression of a subject by constructing an Interval type-2 fuzzy model. An interval type-2 fuzzy face-space is first constructed with the background knowledge of facial features of different subjects for different emotions. The fuzzy face-space thus created comprises primary membership distributions for m facial features, obtained from n subjects, each having

A support vector machine classifier of emotion from voice and facial expression data

\textit{l}

Reducing uncertainty in interval type-2 fuzzy sets for qualitative improvement in emotion recognition from facial expressions

-instances of facial expression for a given emotion. Second, the emotion of an unknown facial expression is determined based on the consensus of the measured facial features with the fuzzy face-space.The classification accuracy of the proposed method is as high as 88.66 %.

Uncertainty management in type-2 fuzzy face-space for emotion recognition

The paper provides a novel approach to emotion recognition from facial expression and voice of subjects. The subjects are asked to manifest their emotional exposure in both facial expression and voice, while uttering a given sentence. Facial features including mouth-opening, eye-opening, eyebrow-constriction, and voice features including, first three formants: F1, F2, and F3, and respective powers at those formants, and pitch are extracted for 7 different emotional expressions of each subject. A linear Support Vector Machine classifier is used to classify the extracted feature vectors into different emotion classes. Sensitivity of the classifier to Gaussian noise is studied, and experimental results confirm that the recognition accuracy of emotion up to a level of 95% is maintained, even when the mean and standard deviation of noise are as high as 5% and 20% respectively over the individual features. A further analysis to identify the importance of individual features reveals that mouthopening and eye-opening are primary features, in absence of which classification accuracy falls off by a large margin of more than 22%.

Automated emotion recognition employing a novel modified binary quantum-behaved gravitational search algorithm with differential mutation

The essence of the paper is to reduce uncertainty in interval type-2 fuzzy sets, and demonstrate the merit of uncertainty reduction in pattern classification problem. The area under the footprint of uncertainty has been used as the measure of uncertainty. A mathematical approach to reduce the area under the footprint of uncertainty has been proposed. Experiments have been designed to compare the relative performance of the classical interval type-2 fuzzy sets with its revised counterpart in emotion recognition from facial expression. Statistical tests performed favor the proposed results of uncertainty reduction. The proposed uncertainty reduction scheme helps in saving approximately 6% gain in classification accuracy with respect to one published work when applied to emotion recognition problem.

Correlation between stimulated emotion extracted from EEG and its manifestation on facial expression

Manifestation of a given emotion on facial expression is not always unique, as the facial attributes in different instances of similar emotional experiences may vary widely. When a number of facial attributes are used to recognize the emotion of a subject, the variation of individual attributes together makes the problem more complicated. This variation is the main source of uncertainty in the emotion recognition problem, which has been addressed here in two steps using type-2 fuzzy sets. First a type-2 fuzzy face-space is constructed with the background knowledge of facial features of different subjects for different emotions. Second, the emotion of the unknown subject is determined based on the consensus of the measured facial features with the fuzzy face-space. The face-space comprises both primary and secondary membership distributions. The primary membership distributions here have been constructed based on the highest frequency of occurrence of the individual attributes. Naturally, the membership values of an attribute at all except the point of highest frequency of occurrence suffer from inaccuracy, which has been taken care of by secondary memberships. An algorithm for the evaluation of the secondary membership distribution from its type-2 primary counterpart has been proposed. The uncertainty management policy adopted using general type-2 fuzzy set has a classification accuracy of 96.67% in comparison to 88.67% obtained by interval type-2 counterpart only.

Application of general type-2 fuzzy set in emotion recognition from facial expression

The present paper proposes a supervised learning based automated human facial emotion recognition strategy with a feature selection scheme employing a novel variation of the gravitational search algorithm GSA. The initial feature set is generated from the facial images by using the 2-D discrete cosine transform DCT and then the proposed modified binary quantum GSA with differential mutation MBQGSA-DM is utilized to select a sub-set of features with high discriminative power. This is achieved by minimising the cost function formulated as the ratio of the within class and interclass distances. The overall system performs its final classification task based on selected feature inputs, utilising a back propagation based artificial neural network ANN. Extensive experimental evaluations are carried out utilising a standard, benchmark emotion database, that is, Japanese Female Facial Expresssion JAFFE database and the results clearly indicate that the proposed method outperforms several existing techniques, already known in literature for solving similar problems. Further validation has also been carried out on a facial expression database developed at Jadavpur University, Kolkata, India and the results obtained further strengthen the notion of superiority of the proposed method.

Facial Action Point Based Emotion Recognition by Principal Component Analysis

Determining correlation between aroused emotion and its manifestation on facial expression, voice, gesture and posture have interesting applications in psychotherapy. A set of audiovisual stimulus, selected by a group of experts, is used to excite emotion of the subjects. EEG and facial expression of the subjects excited by the selected audio-visual stimulus are collected, and the nonlinear-correlation from EEG to facial expression, and vice-versa is obtained by employing feed-forward neural network trained with back-propagation algorithm. Experiments undertaken reveals that the trained network can reproduce the correlated EEG-facial expression trained instances with 100 % accuracy, and is also able to predict facial expression (EEG) from unknown EEG (facial expression) of the same subject with an accuracy of around 95.2%.

Multi-robot Box-Pushing Using Differential Evolution Algorithm for Multiobjective Optimization

This paper proposes a new technique for emotion recognition of an unknown subject using General Type-2 Fuzzy sets (GT2FS). The proposed technique includes two steps- first, a type-2 fuzzy face-space is created with the background knowledge of facial features of different subjects containing different emotions. Second, the emotion of an unknown facial expression is determined based on the consensus of the measured facial features with the fuzzy face-space. The GT2FS has been used here to model the fuzzy face space. The general type-2 fuzzy involves both primary and secondary membership distributions which have been obtained here by formulating and solving an optimization problem. The optimization problem here attempts to minimize the difference between two decoded signals: the first one being the type-1 defuzzification of the average primary membership distributions obtained from n-subjects, while the second one refers to the type-2 defuzzified signal for a given primary distribution with secondary memberships as unknown. The uncertainty management policy adopted using general type-2 fuzzy set has resulted in a classification accuracy of 96.67%.