Gaoyun An

An illumination normalization model for face recognition under varied lighting conditions

In this paper, a novel illumination normalization model is proposed for the pre-processing of face recognition under varied lighting conditions. The novel model could compensate all the illumination effects in face samples, like the diffuse reflection, specular reflection, attached shadow and cast shadow. Firstly, it uses the TV_L^1 model to get the low-frequency part of face image, and adopts the self-quotient model to normalize the diffuse reflection and attached shadow. Then it generates the illumination invariant small-scale part of face sample. Secondly, TV_L^2 model is used to get the noiseless large-scale part of face sample. All kinds of illumination effects in the large-scale part are further removed by the region-based histogram equalization. Thirdly, two parts are fused to generate the illumination invariant face sample. The result of our model contains multi-scaled image information, and all illumination effects in face samples are compensated. Finally, high-order statistical relationships among variables of samples are extracted for classifier. Experimental results on some large scale face databases prove that the processed image by our model could largely improve the recognition performances of conventional methods under low-level lighting conditions.

3D SMoSIFT: three-dimensional sparse motion scale invariant feature transform for activity recognition from RGB-D videos

Abstract. Human activity recognition based on RGB-D data has received more attention in recent years. We propose a spatiotemporal feature named three-dimensional (3D) sparse motion scale-invariant feature transform (SIFT) from RGB-D data for activity recognition. First, we build pyramids as scale space for each RGB and depth frame, and then use Shi-Tomasi corner detector and sparse optical flow to quickly detect and track robust keypoints around the motion pattern in the scale space. Subsequently, local patches around keypoints, which are extracted from RGB-D data, are used to build 3D gradient and motion spaces. Then SIFT-like descriptors are calculated on both 3D spaces, respectively. The proposed feature is invariant to scale, transition, and partial occlusions. More importantly, the running time of the proposed feature is fast so that it is well-suited for real-time applications. We have evaluated the proposed feature under a bag of words model on three public RGB-D datasets: one-shot learning Chalearn Gesture Dataset, Cornell Activity Dataset-60, and MSR Daily Activity 3D dataset. Experimental results show that the proposed feature outperforms other spatiotemporal features and are comparative to other state-of-the-art approaches, even though there is only one training sample for each class.

Facial expression recognition using sparse local Fisher discriminant analysis

In this paper, a novel sparse learning method, called sparse local Fisher discriminant analysis (SLFDA) is proposed for facial expression recognition. The SLFDA method is derived from the original local Fisher discriminant analysis (LFDA) and exploits its sparse property. Because the null space of the local mixture scatter matrix of LFDA has no discriminant information, we find the solutions of LFDA in the range space of the local mixture scatter matrix. The sparse solution is obtained by finding the minimum 1-norm solution from the LFDA solutions. This problem is then formulated as an 1-minimization problem and solved by linearized Bregman iteration, which guarantees convergence and is easily implemented. The proposed SLFDA can deal with multi-modal problems as well as LFDA; in addition, it has more discriminant power than LFDA because the non-zero elements in the basis images are selected from the most important factors or regions. Experiments on several benchmark databases are performed to test and evaluate the proposed algorithm. The results show the effectiveness of SLFDA.

Independent Gabor Analysis of Discriminant Features Fusion for Face Recognition

A discriminant feature fusion model is proposed for face recognition with large variations of pose, expression, lighting, etc. Discriminant features are extracted by the wavelet transform-based method from two source images. One source image is a holistic gray value image and the other is an illumination invariant geometric image. Face sample is reconstructed by the adaptive fused discriminant feature. Then a bank of Gabor filters is built to extract Gabor representations of the reconstructed samples. Finally higher-order statistical relationships among variables of samples are extracted for classifier. According to experiments, the model outperforms conventional algorithms under complex conditions (large variations of lighting, expression, accessory, etc.).

Fully automatic 3D facial expression recognition using polytypic multi-block local binary patterns

3D facial expression recognition has been greatly promoted for overcoming the inherent drawbacks of 2D facial expression recognition and has achieved superior recognition accuracy to the 2D. In this paper, a novel holistic, full-automatic approach for 3D facial expression recognition is proposed. First, 3D face models are represented in 2D-image-like structure which makes it possible to take advantage of the wealth of 2D methods to analyze 3D models. Then an enhanced facial representation, namely polytypic multi-block local binary patterns (P-MLBP), is proposed. The P-MLBP involves both the feature-based irregular divisions to depict the facial expressions accurately and the fusion of depth and texture information of 3D models to enhance the facial feature. Based on the BU-3DFE database, three kinds of classifiers are employed to conduct 3D facial expression recognition for evaluation. Their experimental results outperform the state of the art and show the effectiveness of P-MLBP for 3D facial expression recognition. Therefore, the proposed strategy is validated for 3D facial expression recognition; and its simplicity opens a promising direction for fully automatic 3D facial expression recognition. 2D-image-like structure is proposed to represent 3D models.2D-image-like structure builds the bridge between 3D models and 2D methods.Irregular divisions and data fusion are utilized in P-MLBP to enhance facial feature.Proposed P-MLBP represented facial feature achieves preferable recognition results.The proposed approach is effective to automatic 3D facial expression recognition.

Semi-supervised sparse feature selection based on multi-view Laplacian regularization

Semi-supervised sparse feature selection, which can exploit the large number unlabeled data and small number labeled data simultaneously, has placed an important role in web image annotation. However, most of the semi-supervised sparse feature selection methods are developed for single-view data and these methods cannot naturally deal with the multi-view data, though it has shown that leveraging information contained in multiple views can dramatically improve the feature selection performance. Recently, multi-view learning has obtained much research attention because it can reveal and leverage the correlated and complementary information between different views. So in this paper, we apply multi-view learning into semi-supervised sparse feature selection and propose a semi-supervised sparse feature selection method based on multi-view Laplacian regularization, namely, multi-view Laplacian sparse feature selection (MLSFS).11MLSFS: Multi-view Laplacian Sparse Feature Selection. MLSFS utilizes multi-view Laplacian regularization to boost semi-supervised sparse feature selection performance. A simple iterative method is proposed to solve the objective function of MLSFS. We apply MLSFS algorithm into image annotation task and conduct experiments on two web image datasets. The experimental results show that the proposed MLSFS outperforms the state-of-art single-view sparse feature selection methods. Display Omitted Multi-view Laplacian sparse feature selection (MLSFS) algorithm is proposed.Multi-view learning is utilized to exploit the complementation of different views features.A effective iterative algorithm is introduced to optimize the objective function.The convergence of the algorithm is proven.Experiments demonstrate MLSFS has good performance of feature selection.

Tensor rank one differential graph preserving analysis for facial expression recognition

This paper presents a new dimensionality reduction algorithm for multi-dimensional data based on the tensor rank-one decomposition and graph preserving criterion. Through finding proper rank-one tensors, the algorithm effectively enhances the pairwise inter-class margins and meanwhile preserves the intra-class local manifold structure. In the algorithm, a novel marginal neighboring graph is devised to describe the pairwise inter-class boundaries, and a differential formed objective function is adopted to ensure convergence. Furthermore, the algorithm has less computation in comparison with the vector representation based and the tensor-to-tensor projection based algorithms. The experiments for the basic facial expressions recognition show its effectiveness, especially when it is followed by a neural network classifier. Display Omitted Highlights? A new dimensionality reduction algorithm is proposed based on tensor rank-one analysis and graph preserving. ? A novel marginal neighboring graph is devised to describe the pairwise inter-class boundaries. ? The algorithm is verified convergence during an iterative process. ? The algorithm consumes relatively less time than the vector representation and the tensor-to-tensor projection algorithms. ? The algorithm is effective for the facial expression recognition especially when followed by a neural network classifier.

Independent Gabor Analysis of Multiscale Total Variation-Based Quotient Image

A new algorithm for independent Gabor analysis of multiscale total variation-based quotient image is proposed and applied to face recognition with only one sample per subject here. With our preproposed multiscale TV-based quotient image (TVQI) model, the large-scale and small-scale features are firstly fused to produce the most expressive lighting invariant face. Then a bank of Gabor filters is built to extract lighting invariant Gabor face representations with specified scales and orientations. Last, an information maximization algorithm is adopted to extract higher-order statistical relationships among variables of samples for classifier. According to the experiments on the large-scale CAS-PEAL face database, our approach could outperform Gabor-based ICA, Gabor-based KPCA, and TVQI when they face most outliers (lighting, expression, masking, etc.).

Three-dimensional face recognition under expression variation

In this paper, we introduce a fully automatic framework for 3D face recognition under expression variation. For 3D data preprocessing, an improved nose detection method is presented. The small pose is corrected at the same time. A new facial expression processing method which is based on sparse representation is proposed subsequently. As a result, this framework enhances the recognition rate because facial expression is the biggest obstacle for 3D face recognition. Then, the facial representation, which is based on the dual-tree complex wavelet transform (DT-CWT), is extracted from depth images. It contains the facial information and six subregions’ information. Recognition is achieved by linear discriminant analysis (LDA) and nearest neighbor classifier. We have performed different experiments on the Face Recognition Grand Challenge database and Bosphorus database. It achieves the verification rate of 98.86% on the all vs. all experiment at 0.1% false acceptance rate (FAR) in the Face Recognition Grand Challenge (FRGC) and 95.03% verification rate on nearly frontal faces with expression changes and occlusions in the Bosphorus database.

Action Recognition Using Local Consistent Group Sparse Coding with Spatio-Temporal Structure

This paper presents a novel and efficient framework for human action recognition through integrating the local consistent group sparse representation with spatio-temporal structure of each video sequence. We firstly propose a sparse encoding scheme named local consistent group sparse coding (LCGSC) to generate the sparse representation of each video sequence. The novel encoding scheme takes global structural information of features belonging to one group into consideration as well as the local correlations between similar features. In order to incorporate the spatio-temporal structures, an average location (AL) model is proposed to describe the distribution of each visual word along the spatio-temporal coordinates on the basis of the obtained sparse codes. Eventually, each video sequence is jointly represented by the sparse representation and the spatio-temporal layouts which fully model its motion, appearance and spatio-temporal information. Our framework is computationally efficient and achieves comparable performance on the challenging datasets with state-of-the-art methods.