Kamlesh Mistry

A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition

This paper proposes a facial expression recognition system using evolutionary particle swarm optimization (PSO)-based feature optimization. The system first employs modified local binary patterns, which conduct horizontal and vertical neighborhood pixel comparison, to generate a discriminative initial facial representation. Then, a PSO variant embedded with the concept of a micro genetic algorithm (mGA), called mGA-embedded PSO, is proposed to perform feature optimization. It incorporates a nonreplaceable memory, a small-population secondary swarm, a new velocity updating strategy, a subdimension-based in-depth local facial feature search, and a cooperation of local exploitation and global exploration search mechanism to mitigate the premature convergence problem of conventional PSO. Multiple classifiers are used for recognizing seven facial expressions. Based on a comprehensive study using within- and cross-domain images from the extended Cohn Kanade and MMI benchmark databases, respectively, the empirical results indicate that our proposed system outperforms other state-of-the-art PSO variants, conventional PSO, classical GA, and other related facial expression recognition models reported in the literature by a significant margin.

Intelligent facial emotion recognition using moth-firefly optimization

A descriptor combining LBP, LGBP and LBPV is proposed for feature extraction.Moth-firefly optimization is proposed for feature selection.It mitigates premature convergence of FA and MFO algorithms.Simulated Annealing is also used to further improve the most promising solution.It outperforms other optimization and facial expression recognition methods. In this research, we propose a facial expression recognition system with a variant of evolutionary firefly algorithm for feature optimization. First of all, a modified Local Binary Pattern descriptor is proposed to produce an initial discriminative face representation. A variant of the firefly algorithm is proposed to perform feature optimization. The proposed evolutionary firefly algorithm exploits the spiral search behaviour of moths and attractiveness search actions of fireflies to mitigate premature convergence of the Levy-flight firefly algorithm (LFA) and the moth-flame optimization (MFO) algorithm. Specifically, it employs the logarithmic spiral search capability of the moths to increase local exploitation of the fireflies, whereas in comparison with the flames in MFO, the fireflies not only represent the best solutions identified by the moths but also act as the search agents guided by the attractiveness function to increase global exploration. Simulated Annealing embedded with Levy flights is also used to increase exploitation of the most promising solution. Diverse single and ensemble classifiers are implemented for the recognition of seven expressions. Evaluated with frontal-view images extracted from CK+, JAFFE, and MMI, and 45-degree multi-view and 90-degree side-view images from BU-3DFE and MMI, respectively, our system achieves a superior performance, and outperforms other state-of-the-art feature optimization methods and related facial expression recognition models by a significant margin.

Intelligent facial emotion recognition using a layered encoding cascade optimization model

A layered cascade optimization model is developed for facial emotion recognition.Two layered cascade-based evolutionary algorithms are proposed for feature selection.They focus on within-class and between-class variations for feature optimization.Both a neural network and an adaptive ensemble classifier are employed for expression recognition.Superior performance is shown in both frontal and 90? side-view expression recognition. In this research, we propose a facial expression recognition system with a layered encoding cascade optimization model. Since generating an effective facial representation is a vital step to the success of facial emotion recognition, a modified Local Gabor Binary Pattern operator is first employed to derive a refined initial face representation and we then propose two evolutionary algorithms for feature optimization including (i) direct similarity and (ii) Pareto-based feature selection, under the layered cascade model. The direct similarity feature selection considers characteristics within the same emotion category that give the minimum within-class variation while the Pareto-based feature optimization focuses on features that best represent each expression category and at the same time provide the most distinctions to other expressions. Both a neural network and an ensemble classifier with weighted majority vote are implemented for the recognition of seven expressions based on the selected optimized features. The ensemble model also automatically updates itself with the most recent concepts in the data. Evaluated with the Cohn-Kanade database, our system achieves the best accuracies when the ensemble classifier is applied, and outperforms other research reported in the literature with 96.8% for direct similarity based optimization and 97.4% for the Pareto-based feature selection. Cross-database evaluation with frontal images from the MMI database has also been conducted to further prove system efficiency where it achieves 97.5% for Pareto-based approach and 90.7% for direct similarity-based feature selection and outperforms related research for MMI. When evaluated with 90? side-view images extracted from the videos of the MMI database, the system achieves superior performances with >80% accuracies for both optimization algorithms. Experiments with other weighting and meta-learning combination methods for the construction of ensembles are also explored with our proposed ensemble showing great adpativity to new test data stream for cross-database evaluation. In future work, we aim to incorporate other filtering techniques and evolutionary algorithms into the optimization models to further enhance the recognition performance.

Adaptive facial point detection and emotion recognition for a humanoid robot

We propose a robust landmark detector to deal with pose variation and occlusions.SVRs and NNs are respectively used to estimate intensities of 18 selected AUs.Fuzzy c-means clustering is employed to detect seven basic and compound emotions.Our unsupervised facial point detector outperforms other supervised models.The overall development is integrated with a modern humanoid robot platform. Automatic perception of facial expressions with scaling differences, pose variations and occlusions would greatly enhance natural human robot interaction. This research proposes unsupervised automatic facial point detection integrated with regression-based intensity estimation for facial action units (AUs) and emotion clustering to deal with such challenges. The proposed facial point detector is able to detect 54 facial points in images of faces with occlusions, pose variations and scaling differences using Gabor filtering, BRISK (Binary Robust Invariant Scalable Keypoints), an Iterative Closest Point (ICP) algorithm and fuzzy c-means (FCM) clustering. Especially, in order to effectively deal with images with occlusions, ICP is first applied to generate neutral landmarks for the occluded facial elements. Then FCM is used to further reason the shape of the occluded facial region by taking the prior knowledge of the non-occluded facial elements into account. Post landmark correlation processing is subsequently applied to derive the best fitting geometry for the occluded facial element to further adjust the neutral landmarks generated by ICP and reconstruct the occluded facial region. We then conduct AU intensity estimation respectively using support vector regression and neural networks for 18 selected AUs. FCM is also subsequently employed to recognize seven basic emotions as well as neutral expressions. It also shows great potential to deal with compound and newly arrived novel emotion class detection. The overall system is integrated with a humanoid robot and enables it to deal with challenging real-life facial emotion recognition tasks.

Feature selection using firefly optimization for classification and regression models

Abstract In this research, we propose a variant of the Firefly Algorithm (FA) for discriminative feature selection in classification and regression models for supporting decision making processes using data-based learning methods. The FA variant employs Simulated Annealing (SA)-enhanced local and global promising solutions, chaotic-accelerated attractiveness parameters and diversion mechanisms of weak solutions to escape from the local optimum trap and mitigate the premature convergence problem in the original FA algorithm. A total of 29 classification and 11 regression benchmark data sets have been used to evaluate the efficiency of the proposed FA model. It shows statistically significant improvements over other state-of-the-art FA variants and classical search methods for diverse feature selection problems. In short, the proposed FA variant offers an effective method to identify optimal feature subsets in classification and regression models for supporting data-based decision making processes.

Intelligent Appearance and shape based facial emotion recognition for a humanoid robot

In this paper, we present an intelligent facial emotion recognition system with real-time face tracking for a humanoid robot. The system is able to detect facial actions and emotions from images with up to 60 degrees of pose variations. We employ the Active Appearance Model to perform real-time face tracking and extract both texture and geometric representations of images. A POSIT algorithm is also used to identify head rotations. The extracted texture and shape features are employed to detect 18 facial actions and seven basic emotions. The overall system is integrated with a humanoid robot platform to further extend its vision APIs. The system is proved to be able to deal with challenging facial emotion recognition tasks with various pose variations.

Facial expression recongition using firefly-based feature optimization

Automatic facial expression recognition plays an important role in various application domains such as medical imaging, surveillance and human-robot interaction. This research proposes a novel facial expression recognition system with modified Local Gabor Binary Patterns (LGBP) for feature extraction and a firefly algorithm (FA) variant for feature optimization. First of all, in order to deal with illumination changes, scaling differences and rotation variations, we propose an extended overlap LGBP to extract initial discriminative facial features. Then a modified FA is proposed to reduce the dimensionality of the extracted facial features. This FA variant employs Gaussian, Cauchy and Levy distributions to further mutate the best solution identified by the FA to increase exploration in the search space to avoid premature convergence. The overall system is evaluated using three facial expression databases (i.e. CK+, MMI, and JAFFE). The proposed system outperforms other heuristic search algorithms such as Genetic Algorithm and Particle Swarm Optimization and other existing state-of-the-art facial expression recognition research, significantly.

Intelligent facial expression recognition with adaptive feature extraction for a humanoid robot

Automatic facial expression recognition plays an important role in agent-based interface development and datadriven animation. This paper presents an intelligent facial action and emotion recognition system for a humanoid robot. Motivated by the Facial Action Coding System, this research focuses on the recognition of seven basic emotions and 18 Action Units (AU). Since effective facial representations of original face images are vital for automatic facial emotion recognition, this research implements a novel shape and appearance feature extraction method, which integrates an Independent Active Appearance Model (AAM) with a rotation-invariant feature point detector, BRISK (Binary Robust Invariant Scalable Keypoints). In comparison to AAM with a traditional inverse compositional fitting, our model with BRISK fitting is with less computational cost and is capable of dealing with feature extraction from images of faces with rotations and scaling differences without prior training required. Subsequently shape and appearancebased neural network AU analyzers are used to respectively detect 18 AUs. Emotions are then decoded from the derived AUs using a neural network emotion recognizer. The system is integrated with a modern humanoid robot platform. Evaluation results indicate its high accuracy for AU and emotion recognition. It is also among the top performers on the extended Cohn-Kanade (CK+) database in comparison to other existing state-of-the-art applications.

An Enhanced Intelligent Agent with Image Description Generation

In this paper, we present an Embodied Conversational Agent (ECA) enriched with automatic image understanding, using vision data derived from state-of-the-art machine learning techniques for the advancement of autonomous interaction with the elderly or infirm. The agent is developed to conduct health and emotion well-being monitoring for the elderly. It is not only able to conduct question-answering via speech-based interaction, but also able to provide analysis of the user’s surroundings, company, emotional states, hazards and fall actions via visual data using deep learning techniques. The agent is accessible from a web browser and can be communicated with via voice means, with a webcam required for the visual analysis functionality. The system has been evaluated with diverse real-life images to prove its efficiency.