Deepak Ghimire

Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines

Facial expressions are widely used in the behavioral interpretation of emotions, cognitive science, and social interactions. In this paper, we present a novel method for fully automatic facial expression recognition in facial image sequences. As the facial expression evolves over time facial landmarks are automatically tracked in consecutive video frames, using displacements based on elastic bunch graph matching displacement estimation. Feature vectors from individual landmarks, as well as pairs of landmarks tracking results are extracted, and normalized, with respect to the first frame in the sequence. The prototypical expression sequence for each class of facial expression is formed, by taking the median of the landmark tracking results from the training facial expression sequences. Multi-class AdaBoost with dynamic time warping similarity distance between the feature vector of input facial expression and prototypical facial expression, is used as a weak classifier to select the subset of discriminative feature vectors. Finally, two methods for facial expression recognition are presented, either by using multi-class AdaBoost with dynamic time warping, or by using support vector machine on the boosted feature vectors. The results on the Cohn-Kanade (CK+) facial expression database show a recognition accuracy of 95.17% and 97.35% using multi-class AdaBoost and support vector machines, respectively.

Nonlinear transfer function-based local approach for color image enhancement

The main objective of image enhancement is to improve some characteristic of an image to make it visually better one. This paper proposes a method for enhancing the color images based on nonlinear transfer function and pixel neighborhood by preserving details. In the proposed method, the image enhancement is applied only on the V (luminance value) component of the HSV color image and H and S component are kept unchanged to prevent the degradation of color balance between HSV components. The V channel is enhanced in two steps. First the V component image is divided into smaller overlapping blocks and for each pixel inside the block the luminance enhancement is carried out using nonlinear transfer function. In the second step, each pixel is further enhanced for the adjustment of the image contrast depending upon the center pixel value and its neighborhood pixel values. Finally, original H and S component image and enhanced V component image are converted back to RGB image. The subjective and objective performance evaluation shows that the proposed enhancement method yields better results without changing image original color in comparison with the conventional methods.

A Robust Face Detection Method Based on Skin Color and Edges

In this paper we propose a method to detect human faces in color images. Many existing systems use a window-based classifier that scans the entire image for the presence of the human face and such systems suffers from scale variation, pose variation, illumination changes, etc. Here, we propose a lighting insensitive face detection method based upon the edge and skin tone information of the input color image. First, image enhancement is performed, especially if the image is acquired from an unconstrained illumination condition. Next, skin segmentation in YCbCr and RGB space is conducted. The result of skin segmentation is refined using the skin tone percentage index method. The edges of the input image are combined with the skin tone image to separate all non- face regions from candidate faces. Candidate verification using primitive shape features of the face is applied to decide which of the candidate regions corresponds to a face. The advantage of the proposed method is that it can detect faces that are of different sizes, in different poses, and that are making different expressions under unconstrained illumination conditions

Extreme Learning Machine Ensemble Using Bagging for Facial Expression Recognition

Abstract —An extreme learning machine (ELM) is a recently proposed learning algorithm for a single-layer feed forward neural network. In this paper we studied the ensemble of ELM by using a bagging algorithm for facial expression recognition (FER). Facial expression analysis is widely used in the behavior interpretation of emotions, for cognitive science, and social interactions. This paper presents a method for FER based on the histogram of orientation gradient (HOG) features using an ELM ensemble. First, the HOG features were extracted from the face image by dividing it into a number of small cells. A bagging algorithm was then used to construct many different bags of training data and each of them was trained by using separate ELMs. To recognize the expression of the input face image, HOG features were fed to each trained ELM and the results were combined by using a majority voting scheme. The ELM ensemble using bagging improves the generalized capability of the network significantly. The two available datasets (JAFFE and CK+) of facial expressions were used to evaluate the performance of the proposed classification system. Even the performance of individual ELM was smaller and the ELM ensemble using a bagging algorithm improved the recognition performance significantly.

Facial expression recognition based on local region specific features and support vector machines

Facial expressions are one of the most powerful, natural and immediate means for human being to communicate their emotions and intensions. Recognition of facial expression has many applications including human-computer interaction, cognitive science, human emotion analysis, personality development etc. In this paper, we propose a new method for the recognition of facial expressions from single image frame that uses combination of appearance and geometric features with support vector machines classification. In general, appearance features for the recognition of facial expressions are computed by dividing face region into regular grid (holistic representation). But, in this paper we extracted region specific appearance features by dividing the whole face region into domain specific local regions. Geometric features are also extracted from corresponding domain specific regions. In addition, important local regions are determined by using incremental search approach which results in the reduction of feature dimension and improvement in recognition accuracy. The results of facial expressions recognition using features from domain specific regions are also compared with the results obtained using holistic representation. The performance of the proposed facial expression recognition system has been validated on publicly available extended Cohn-Kanade (CK+) facial expression data sets.

Recognition of facial expressions based on salient geometric features and support vector machines

Facial expressions convey nonverbal cues which play an important role in interpersonal relations, and are widely used in behavior interpretation of emotions, cognitive science, and social interactions. In this paper we analyze different ways of representing geometric feature and present a fully automatic facial expression recognition (FER) system using salient geometric features. In geometric feature-based FER approach, the first important step is to initialize and track dense set of facial points as the expression evolves over time in consecutive frames. In the proposed system, facial points are initialized using elastic bunch graph matching (EBGM) algorithm and tracking is performed using Kanade-Lucas-Tomaci (KLT) tracker. We extract geometric features from point, line and triangle composed of tracking results of facial points. The most discriminative line and triangle features are extracted using feature selective multi-class AdaBoost with the help of extreme learning machine (ELM) classification. Finally the geometric features for FER are extracted from the boosted line, and triangles composed of facial points. The recognition accuracy using features from point, line and triangle are analyzed independently. The performance of the proposed FER system is evaluated on three different data sets: namely CK+, MMI and MUG facial expression data sets.

A novel approach of automatic music genre classification based on timbrai texture and rhythmic content features

Music genre classification is an essential component for the music information retrieval system. There are two important components to be considered for better genre classification, which are audio feature extraction and classifier. This paper incorporates two different kinds of features for genre classification, timbrai texture and rhythmic content features. Timbrai texture contains the Mel-frequency Cepstral Coefficient (MFCC) with other several spectral features. Before choosing a timbrai feature we explore which feature plays an insignificant role on genre discrimination. This facilitates the reduction of feature dimension. For the timbrai features up to the 4-th order central moments and the covariance components of mutual features are considered to improve the overall classification result. For the rhythmic content the features extracted from beat histogram are selected. In the paper Extreme Learning Machine (ELM) with bagging is used as the classifier for classifying the genres. Based on the proposed feature sets and classifier, experiment is performed with well-known datasets: GTZAN with ten different music genres. The proposed method acquires better classification accuracy compared to the existing methodologies.

Color Image Enhancement in HSV Space Using Nonlinear Transfer Function and Neighborhood Dependent Approach with Preserving Details

This paper proposes a method for enhancing the color images based on nonlinear transfer function and pixel neighborhood by preserving details. In the proposed method, an input RGB color image is converted into an HSV color image. The image enhancement is applied only on the V (luminance value) component of the color image, because change in the H and S component could change the color balance between HSV components. The V component is enhanced in two steps. At first the V channel is divided into smaller blocks and in each block dynamic range compression is carried out using nonlinear transfer function. In the second step each pixels in each block are further enhanced for the adjustment of the image contrast depending upon the centre pixel and its neighborhood. The aim behind dividing the image into blocks is to preserve the details. The original H and S component image and enhanced V component image are converted back to the RGB image. The experimental results show that the proposed method yields better performance by preserving details and no change in color in comparison with other methods.

Automatic music genre classification using timbral texture and rhythmic content features

Music genre classification is a vital component for the music information retrieval system. There are two important components to be considered for better genre classification, which are audio feature extraction and classifier. This paper incorporates two different kinds of features for genre classification, timbral texture and rhythmic content features. Timbral texture contains the Mel-frequency Cepstral Coefficient (MFCC) with other several spectral features. Before choosing a timbral feature we explore which feature contributes a less significant role on genre discrimination. This facilitates the reduction of feature dimension. For the timbral features up to the 4-th order central moments and the covariance components of mutual features are considered to improve the overall classification result. For the rhythmic content the features extracted from beat histogram are selected. In the paper Extreme Learning Machine (ELM) with bagging is used as the classifier for classifying the genres. Based on the proposed feature sets and classifier, experiments are performed with two well-known datasets: GTZAN and the ISMIR2004 databases with ten and six different music genres, respectively. The proposed method acquires better and competitive classification accuracy compared to the existing approaches for both data sets.

Evaluation of different audio features for musical genre classification

Musical genre classification is an important issue for the music information retrieval system. There are two essential components for music genre classification, which are audio features and classifier. This paper considers various kinds of the features for genre classification related with dynamics, rhythm, spectral, and tonal characteristics of music. In the paper up to the 4th order central moments for different features are considered to evaluate the overall classification accuracy. In addition, Extreme Learning Machine (ELM) with bagging is introduced and compared with well-known Support Vector Machines (SVM) in terms of the overall classification accuracy. Based on the aforementioned features sets and ELM classifier, experiments are performed with well-known datasets: GTZAN with ten different musical genres. Through the experiments we found that some type of features is more important to others and the two classifiers provide comparable results for genre classification.