Cunjian Chen

Automatic facial makeup detection with application in face recognition

Facial makeup has the ability to alter the appearance of a person. Such an alteration can degrade the accuracy of automated face recognition systems, as well as that of meth-ods estimating age and beauty from faces. In this work, we design a method to automatically detect the presence of makeup in face images. The proposed algorithm extracts a feature vector that captures the shape, texture and color characteristics of the input face, and employs a classifier to determine the presence or absence of makeup. Besides extracting features from the entire face, the algorithm also considers portions of the face pertaining to the left eye, right eye, and mouth. Experiments on two datasets consisting of 151 subjects (600 images) and 125 subjects (154 images), respectively, suggest that makeup detection rates of up to 93.5% (at a false positive rate of 1%) can be obtained using the proposed approach. Further, an adaptive pre-processing scheme that exploits knowledge of the presence or absence of facial makeup to improve the matching accuracy of a face matcher is presented.

Soft Biometrics for Surveillance: An Overview

Abstract Biometrics is the science of automatically recognizing people based on physical or behavioral characteristics such as face, fingerprint, iris, hand, voice, gait, and signature. More recently, the use of soft biometric traits has been proposed to improve the performance of traditional biometric systems and allow identification based on human descriptions. Soft biometric traits include characteristics such as height, weight, body geometry, scars, marks, and tattoos (SMT), gender, etc. These traits offer several advantages over traditional biometric techniques. Soft biometric traits can be typically described using human understandable labels and measurements, allowing for retrieval and recognition solely based on verbal descriptions. Unlike many primary biometric traits, soft biometrics can be obtained at a distance without subject cooperation and from low quality video footage, making them ideal for use in surveillance applications. This chapter will introduce the current state of the art in the emerging field of soft biometrics.

Can facial cosmetics affect the matching accuracy of face recognition systems

The matching performance of automated face recognition has significantly improved over the past decade. At the same time several challenges remain that significantly affect the deployment of such systems in security applications. In this work, we study the impact of a commonly used face altering technique that has received limited attention in the biometric literature, viz., non-permanent facial makeup. Towards understanding its impact, we first assemble two databases containing face images of subjects, before and after applying makeup. We present experimental results on both databases that reveal the effect of makeup on automated face recognition and suggest that this simple alteration can indeed compromise the accuracy of a biometric system. While these are early results, our findings clearly indicate the need for a better understanding of this face altering scheme and the importance of designing algorithms that can successfully overcome the obstacle imposed by the application of facial makeup.

A Study on Using Mid-Wave Infrared Images for Face Recognition

The problem of face identication in the Mid-Wave InfraRed (MWIR) spectrum is studied in order to understand the performance of intra-spectral (MWIR to MWIR) and cross-spectral (visible to MWIR) matching. The contributions of this work are two-fold. First, a database of 50 subjects is assembled and used to illustrate the challenges associated with the problem. Second, a set of experiments is performed in order to demonstrate the possibility of MWIR intra-spectral and cross-spectral matching. Experiments show that images captured in the MWIR band can be eciently matched to MWIR images using existing techniques (originally not designed to address such a problem). These results are comparable to the baseline results, i.e., when comparing visible to visible face images. Experiments also show that cross-spectral matching (the heterogeneous problem, where gallery and probe sets have face images acquired in dierent spectral bands) is a very challenging problem. In order to perform cross-spectral matching, we use multiple texture descriptors and demonstrate that fusing these descriptors improves recognition performance. Experiments on a small database, suggests that the problem of cross-spectral matching requires further investigation.

Evaluation of gender classification methods on thermal and near-infrared face images

Automatic gender classification based on face images is receiving increased attention in the biometrics community. Most gender classification systems have been evaluated only on face images captured in the visible spectrum. In this work, the possibility of deducing gender from face images obtained in the near-infrared (NIR) and thermal (THM) spectra is established. It is observed that the use of local binary pattern histogram (LBPH) features along with discriminative classifiers results in reasonable gender classification accuracy in both the NIR and THM spectra. Further, the performance of human subjects in classifying thermal face images is studied. Experiments suggest that machine-learning methods are better suited than humans for gender classification from face images in the thermal spectrum.

Can facial metrology predict gender

We investigate the question of whether facial metrology can be exploited for reliable gender prediction. A new method based solely on metrological information from facial landmarks is developed. Here, metrological features are defined in terms of specially normalized angle and distance measures and computed based on given landmarks on facial images. The performance of the proposed metrology-based method is compared with that of a state-of-the-art appearance-based method for gender classification. Results are reported on two standard face databases, namely, MUCT and XM2VTS containing 276 and 295 images, respectively. The performance of the metrology-based approach was slightly lower than that of the appearance-based method by only about 3.8% for the MUCT database and about 5.7% for the XM2VTS database.

An ensemble of patch-based subspaces for makeup-robust face recognition

We propose a face-matching framework to address the problem of makeup.An ensemble of subspaces is used for face representation and matching.Each subspace pertains to a set of randomly selected patches.Multiple texture descriptors are used to describe a patch.Combination of sparse and collaborative classifiers is used in these subspaces.Large number of experiments are conducted to convey the efficacy of the scheme. Recent research has demonstrated the negative impact of makeup on automated face recognition. In this work, we introduce a patch-based ensemble learning method, which uses multiple subspaces generated by sampling patches from before-makeup and after-makeup face images, to address this problem. In the proposed scheme, each face image is tessellated into patches and each patch is represented by a set of feature descriptors, viz., Local Gradient Gabor Pattern (LGGP), Histogram of Gabor Ordinal Ratio Measures (HGORM) and Densely Sampled Local Binary Pattern (DS-LBP). Then, an improved Random Subspace Linear Discriminant Analysis (SRS-LDA) method is used to perform ensemble learning by sampling patches and constructing multiple common subspaces between before-makeup and after-makeup facial images. Finally, Collaborative-based and Sparse-based Representation Classifiers are used to compare feature vectors in this subspace and the resulting scores are combined via the sum-rule. The proposed face matching algorithm is evaluated on the YMU makeup dataset and is shown to achieve very good results. It outperforms other methods designed specifically for the makeup problem.

Predicting gender and weight from human metrology using a copula model

We investigate the use of human metrology for the prediction of certain soft biometrics, viz. gender and weight. In particular, we consider geometric measurements from the head, and those from the remaining parts of the human body, and analyze their potential in predicting gender and weight. For gender prediction, the proposed model results in a 0.7% misclassification rate using both body and head information, 1.0% using only body information, and 12.2% using only head information on the CAESAR 1D database consisting of 2,369 subjects. For weight prediction, the proposed model gives 0.01 mean absolute error (in the range 0 to 1) using both body and head information, 0.01 using only body information, and 0.07 using only measurements from the head. This leads to the observation that human body metrology contains enough information for reliable prediction of gender and weight. Furthermore, we investigate the efficacy of the model in practical applications, where metrology data may be missing or severely contaminated by various sources of noises. The proposed copula-based technique is observed to reduce the impact of noise on prediction performance.

Can gender be predicted from near-infrared face images?

Gender classification based on facial images has received increased attention in the computer vision literature. Previous work on this topic has focused on images acquired in the visible spectrum (VIS). We explore the possibility of predicting gender from face images acquired in the near-infrared spectrum (NIR). In this regard, we address the following two questions: (a) Can gender be predicted from NIR face images; and (b) Can a gender predictor learned using VIS images operate successfully on NIR images and vice-versa? Our experimental results suggest that NIR face images do have some discriminatory information pertaining to gender, although the degree of discrimination is noticeably lower than that of VIS images. Further, the use of an illumination normalization routine may be essential for facilitating cross-spectral gender prediction.

Local Gradient Gabor Pattern (LGGP) with Applications in Face Recognition, Cross-spectral Matching and Soft Biometrics

Researchers in face recognition have been using Gabor filters for image representation due to their robustness to complex variations in expression and illumination. Numerous methods have been proposed to model the output of filter responses by employing either local or global descriptors. In this work, we propose a novel but simple approach for encoding Gradient information on Gabor-transformed images to represent the face, which can be used for identity, gender and ethnicity assessment. Extensive experiments on the standard face benchmark FERET (Visible versus Visible), as well as the heterogeneous face dataset HFB (Near-infrared versus Visible), suggest that the matching performance due to the proposed descriptor is comparable against state-of-the-art descriptor-based approaches in face recognition applications. Furthermore, the same feature set is used in the framework of a Collaborative Representation Classification (CRC) scheme for deducing soft biometric traits such as gender and ethnicity from face images in the AR, Morph and CAS-PEAL databases.