Nese Alyuz

Bosphorus Database for 3D Face Analysis

A new 3D face database that includes a rich set of expressions, systematic variation of poses and different types of occlusions is presented in this paper. This database is unique from three aspects: i) the facial expressions are composed of judiciously selected subset of Action Units as well as the six basic emotions, and many actors/actresses are incorporated to obtain more realistic expression data; ii) a rich set of head pose variations are available; and iii) different types of face occlusions are included. Hence, this new database can be a very valuable resource for development and evaluation of algorithms on face recognition under adverse conditions and facial expression analysis as well as for facial expression synthesis.

A 3D Face Recognition System for Expression and Occlusion Invariance

Facial expression variations and occlusions complicate the task of identifying persons from their 3D facial scans. We propose a new 3D face registration and recognition method based on local facial regions that is able to provide better accuracy in the presence of expression variations and facial occlusions. Proposed fast and flexible alignment method uses average regional models (ARMs), where local correspondences are inferred by the iterative closest point (ICP) algorithm. Dissimilarity scores obtained from local regional matchers are fused to robustly identify probe subjects. In this work, a multi-expression 3D face database, Bosphorus 3D face database, that contains significant amount of different expression types and realistic facial occlusion is used for identification experiments. The experimental results on this challenging database demonstrate that the proposed system improves the performance of the standard ICP-based holistic approach (71.39%) by obtaining 95.87% identification rate in the case of expression variations. When facial occlusions are present, the performance gain is even better. Identification rate improves from 47.05% to 94.12%.

3D Face Recognition Benchmarks on the Bosphorus Database with Focus on Facial Expressions

This paper presents an evaluation of several 3D face recognizers on the Bosphorus database which was gathered for studies on expression and pose invariant face analysis. We provide identification results of three 3D face recognition algorithms, namely generic face template based ICP approach, one-to-all ICP approach, and depth image-based Principal Component Analysis (PCA) method. All of these techniques treat faces globally and are usually accepted as baseline approaches. In addition, 2D texture classifiers are also incorporated in a fusion setting. Experimental results reveal that even though global shape classifiers achieve almost perfect identification in neutral-to-neutral comparisons, they are sub-optimal under extreme expression variations. We show that it is possible to boost the identification accuracy by focusing on the rigid facial regions and by fusing complementary information coming from shape and texture modalities.

3-D Face Recognition Under Occlusion Using Masked Projection

With advances in sensor technology, the three-dimensional (3-D) face has become an emerging biometric modality, preferred especially in high security applications. However, dealing with occlusions covering the facial surface is a great challenge, which should be handled to enable applicability to fully automatic security systems. In this paper, we propose a fully automatic 3-D face recognition system which is robust to occlusions. We basically consider two problems: 1) occlusion handling for surface registration, and 2) missing data handling for classification based on subspace analysis techniques. For the alignment problem, we employ an adaptively-selected-model-based registration scheme, where a face model is selected for an occluded face such that only the valid nonoccluded patches are utilized. After registering to the model, occlusions are detected and removed. In the classification stage, a masking strategy, which we call masked projection, is proposed to enable the use of subspace analysis techniques with incomplete data. Furthermore, a regional scheme suitable for occlusion handling is incorporated in classification to improve the overall results. Experimental results on two databases with realistic facial occlusions, namely, the Bosphorus and the UMB-DB, are reported. Experimental results confirm that registration based on the adaptively selected model together with the masked subspace analysis classification offer an occlusion robust face recognition system.

Registration of three-dimensional face scans with average face models

The accuracy of a three-dimensional (3-D) face recogni- tion system depends on a correct registration that aligns the facial surfaces and makes a comparison possible. The best results ob- tained so far use a costly one-to-all registration approach, which requires the registration of each facial surface to all faces in the gallery. We explore the approach of registering the new facial sur- face to an average face model (AFM), which automatically estab- lishes correspondence to the preregistered gallery faces. We pro- pose a new algorithm for constructing an AFM and show that it works better than a recent approach. We inspect thin-plate spline and iterative closest-point-based registration schemes under manual or automatic landmark detection prior to registration. Ex- tending the single-AFM approach, we consider employing category- specific alternative AFMs for registration and evaluate the effect on subsequent classification. We perform simulations with multiple AFMs that correspond to different clusters in the face shape space and compare these with gender- and morphology-based groupings. We show that the automatic clustering approach separates the faces into gender and morphology groups, consistent with the other race effect reported in the psychology literature. Last, we describe and analyze a regular resampling method, that significantly in- creases the accuracy of registration.

Robust 3D face recognition in the presence of realistic occlusions

Facial occlusions pose significant problems for automatic face recognition systems. In this work, we propose a novel occlusion-resistant three-dimensional (3D) facial identification system. We show that, under extreme occlusions due to hair, hands, and eyeglasses, typical 3D face recognition systems exhibit poor performance. In order to deal with occlusions, our proposed system employs occlusion-resistant registration, occlusion detection, and regional classifiers. A two-step registration module first detects the nose region on the curvedness-weighted convex shape index map, and then performs fine alignment using nose-based Iterative Closest Point (ICP) algorithm. Occluded areas are determined automatically via a generic face model. After non-facial parts introduced by occlusions are removed, a variant of Gappy Principal Component Analysis (Gappy PCA) is used to restore the full face from occlusion-free facial surfaces. Experimental results obtained on realistically occluded facial images from the Bosphorus 3D face database shows that, with the use of score-level fusion of regional Linear Discriminant Analysis (LDA) classifiers, the proposed method improves rank-1 identification accuracy significantly: from 76.12% to 94.23%.

3D Face Recognition: Technology and Applications

Abstract 3D face recognition has received a lot of attention in the last decade, leading to improved sensors and algorithms that promise to enable large-scale deployment of biometric systems that rely on this modality. This chapter discusses advances in 3D face recognition with respect to current research and technology trends, together with its open challenges. Five real-world scenarios are described for application of 3D face biometrics. Then we provide a comparative overview of the currently available commercial sensors, and point out to research databases acquired with each technology. The algorithmic aspects of 3D face recognition are broadly covered; we inspect automatic landmarking and automatic registration as sine qua non parts of a complete 3D facial biometric system. We summarize major coordinated actions in evaluating 3D face recognition algorithms, and conclude with a case study on a recent and challenging database.

Regional registration and curvature descriptors for expression resistant 3D face recognition

Registration plays a vital role in 3D face recognition. In the registration phase, some fiducial points are needed. In this paper, a method is proposed to automatically localize landmark points. In addition, the performance of registration is affected by deformations caused by expression variations. It is asserted that regional registration and recognition is resistant to facial deformations. Finally, we propose to use translation and rotation invariant curvature descriptors as surface features to deal with erronous registration. In experiments conducted on the Bosphorus 3D face database, which contains significant expression variations, the performance of the proposed system was satisfactory.

Component-based registration with curvature descriptors for expression insensitive 3d face recognition

Deformations caused by facial expression variations complicate the task of 3D face registration which is vital for successful 3D face recognition systems. In this work, we propose to use a hierarchical component-based face registration technique capable of handling the difficulties caused by non-rigid nature of faces. Local components independently registered by the iterative closest point (ICP) algorithm provides a fast registration with the use of a generic face model and does not suffer from non-rigidity of human facial surface. Invariance of the proposed approach is further increased by utilizing curvature-based 3D surface descriptors. Identification experiments conducted on the multi-expression Bosphorus database reveal that the accuracy of the classical ICP-based approach can be significantly increased under extreme expression variations.

Alternative face models for 3D face registration

3D has become an important modality for face biometrics. The accuracy of a 3D face recognition system depends on a correct registration that aligns the facial surfaces and makes a comparison possible. The best results obtained so far use a one-to-all registration approach, which means each new facial surface is registered to all faces in the gallery, at a great computational cost. We explore the approach of registering the new facial surface to an average face model (AFM), which automatically establishes correspondence to the pre-registered gallery faces. Going one step further, we propose that using a couple of well-selected AFMs can trade-off computation time with accuracy. Drawing on cognitive justifications, we propose to employ category-specific alternative average face models for registration, which is shown to increase the accuracy of the subsequent recognition. We inspect thin-plate spline (TPS) and iterative closest point (ICP) based registration schemes under realistic assumptions on manual or automatic landmark detection prior to registration. We evaluate several approaches for the coarse initialization of ICP. We propose a new algorithm for constructing an AFM, and show that it works better than a recent approach. Finally, we perform simulations with multiple AFMs that correspond to different clusters in the face shape space and compare these with gender and morphology based groupings. We report our results on the FRGC 3D face database.