Rizwan Ahmed Khan

Framework for reliable, real-time facial expression recognition for low resolution images

Automatic recognition of facial expressions is a challenging problem specially for low spatial resolution facial images. It has many potential applications in human-computer interactions, social robots, deceit detection, interactive video and behavior monitoring. In this study we present a novel framework that can recognize facial expressions very efficiently and with high accuracy even for very low resolution facial images. The proposed framework is memory and time efficient as it extracts texture features in a pyramidal fashion only from the perceptual salient regions of the face. We tested the framework on different databases, which includes Cohn-Kanade (CK+) posed facial expression database, spontaneous expressions of MMI facial expression database and FG-NET facial expressions and emotions database (FEED) and obtained very good results. Moreover, our proposed framework exceeds state-of-the-art methods for expression recognition on low resolution images.

Human vision inspired framework for facial expressions recognition

We present a novel human vision inspired framework that can recognize facial expressions very efficiently and accurately. We propose to computationally process small, salient region of the face to extract features as it happens in human vision. To determine which facial region(s) is perceptually salient for a particular expression, we conducted a psycho-visual experimental study with an eye-tracker. A novel feature space conducive for recognition task is proposed, which is created by extracting Pyramid Histogram of Orientation Gradients features only from the salient facial regions. By processing only salient regions, proposed framework achieved two goals: (a) reduction in computational time for feature extraction (b) reduction in feature vector dimensionality. The proposed framework achieved automatic expression recognition accuracy of 95.3% on extended Cohn-Kanade (CK+) facial expression database for six universal facial expressions.

Pain detection through shape and appearance features

In this paper we are proposing a novel computer vision system that can recognize expression of pain in videos by analyzing facial features. Usually pain is reported and recorded manually and thus carry lot of subjectivity. Manual monitoring of pain makes difficult for the medical practitioners to respond quickly in critical situations. Thus, it is desirable to design such a system that can automate this task. With our proposed model pain monitoring can be done in real-time without any human intervention. We propose to extract shape information using pyramid histogram of orientation gradients (PHOG) and appearance information using pyramid local binary pattern (PLBP) in order to get discriminative representation of face. We tested our proposed model on UNBC-McMaster Shoulder Pain Expression Archive Database and recorded results that exceeds state-of-the-art.

Exploring human visual system: Study to aid the development of automatic facial expression recognition framework

This paper focus on understanding human visual system when it decodes or recognizes facial expressions. Results presented can be exploited by the computer vision research community for the development of robust descriptor based on human visual system for facial expressions recognition. We have conducted psycho-visual experimental study to find which facial region is perceptually more attractive or salient for a particular expression. Eye movements of 15 observers were recorded with an eye-tracker in free viewing conditions as they watch a collection of 54 videos selected from Cohn-Kanade facial expression database, showing six universal facial expressions. The results of the study shows that for some facial expressions only one facial region is perceptually more attractive than others. Other cases shows the attractiveness of two to three facial regions. This paper also proposes a novel framework for automatic recognition of expressions which is based on psycho-visual study.

Visual attention: Effects of blur

The detection of salient regions in images is of great interest for a lot of computer vision applications as adaptive content delivery, smart resizing and auto-cropping, content based image retrieval or visually impaired people assistance. In this paper we focus on the effect of blurriness on human visual attention when observers see images with no prior knowledge. We investigate the hypothesis that sharp objects tend to capture attention irrespective of intensity, color or contrast. Eye movements of 17 subjects were recorded with an eye-tracker in free viewing conditions. Observers were asked to watch a collection of 122 color and grayscale images selected according to criteria driven by basic features of visual perception. The results of the experimental study clearly demonstrate the influence of the sharp/blur aspect of an image part on its saliency. These results indicate that blur information might be integrated in models of attention to efficiently improve the extraction of salient regions.

Facial expression recognition using entropy and brightness features

This paper proposes a novel framework for universal facial expression recognition. The framework is based on two sets of features extracted from the face image: entropy and brightness. First, saliency maps are obtained by state-of-the-art saliency detection algorithm i.e. “frequency-tuned salient region detection”. Then only localized salient facial regions from saliency maps are processed to extract entropy and brightness features. To validate the performance of saliency detection algorithm against human visual system, we have performed a visual experiment. Eye movements of 15 subjects were recorded with an eye-tracker in free viewing conditions as they watch a collection of 54 videos selected from Cohn-Kanade facial expression database. Results of the visual experiment provided the evidence that obtained saliency maps conforms well with human fixations data. Finally, evidence of the proposed frameworks performance is exhibited through satisfactory classification results on Cohn-Kanade database.

Automatic Affect Analysis: From Children to Adults

This article presents novel and robust framework for automatic recognition of facial expressions for children. The proposed framework also achieved results better than state of the art methods for stimuli containing adult faces. The proposed framework extract features only from perceptual salient facial regions as it gets its inspiration from human visual system. In this study we are proposing novel shape descriptor, facial landmark points triangles ratio (LPTR). The framework was first tested on the “Dartmouth database of children’s faces” which contains photographs of children between 6 and 16 years of age and achieved promising results. Later we tested proposed framework on Cohn-Kanade (CK+) posed facial expression database (adult faces) and obtained results that exceeds state of the art.

Saliency-based framework for facial expression recognition

This article proposes a novel framework for the recognition of six universal facial expressions. The framework is based on three sets of features extracted from a face image: entropy, brightness, and local binary pattern. First, saliency maps are obtained using the state-of-the-art saliency detection algorithm “frequency-tuned salient region detection”. The idea is to use saliency maps to determine appropriate weights or values for the extracted features (i.e., brightness and entropy). We have performed a visual experiment to validate the performance of the saliency detection algorithm against the human visual system. Eye movements of 15 subjects were recorded using an eye-tracker in free-viewing conditions while they watched a collection of 54 videos selected from the Cohn-Kanade facial expression database. The results of the visual experiment demonstrated that the obtained saliency maps are consistent with the data on human fixations. Finally, the performance of the proposed framework is demonstrated via satisfactory classification results achieved with the Cohn-Kanade database, FG-NET FEED database, and Dartmouth database of children’s faces.

Body expression recognition from animated 3D skeleton

We present a novel and generic framework for the recognition of body expressions using human postures. Motivated by the state of the art from the domain of psychology, our approach recognizes expression by analyzing sequence of pose. Features proposed in this article are computationally simple and intuitive to understand. They are based on visual cues and provide in-depth understanding of body postures required to recognize body expressions. We have evaluated our approach on different databases with heterogeneous movements and body expressions. Our recognition results exceeds state of the art for some database and for others we obtain results at par with state of the art.

Toward an efficient body expression recognition based on the synthesis of a neutral movement

We present a novel framework for the recognition of body expressions using human postures. Proposed system is based on analyzing the spectral difference between an expressive and a neutral animation. Second problem that has been addressed in this paper is formalization of neutral animation. Formalization of neutral animation has not been tackled before and it can be very useful for the domain of synthesis of animation, recognition of expressions, etc. In this article, we proposed a cost function to synthesize a neutral motion from expressive motion. The cost function formalizes a neutral motion by computing the distance and by combining it with acceleration of each body joints during a motion. We have evaluated our approach on several databases with heterogeneous movements and body expressions. Our body expression recognition results exceeds state of the art on evaluated databases.