Yuhang Wu

Pose-robust face signature for multi-view face recognition

Despite the great progress achieved in unconstrained face recognition, pose variations still remain a challenging and unsolved practical issue. We propose a novel framework for multi-view face recognition based on extracting and matching pose-robust face signatures from 2D images. Specifically, we propose an efficient method for monocular 3D face reconstruction, which is used to lift the 2D facial appearance to a canonical texture space and estimate the self-occlusion. On the lifted facial texture we then extract various local features, which are further enhanced by the occlusion encodings computed on the self-occlusion mask, resulting in a pose-robust face signature, a novel feature representation of the original 2D facial image. Extensive experiments on two public datasets demonstrate that our method not only simplifies the matching of multi-view 2D facial images by circumventing the requirement for pose-adaptive classifiers, but also achieves superior performance.

Rendering or normalization? An analysis of the 3D-aided pose-invariant face recognition

In spite of recent progress achieved in near-frontal face recognition, the problem of pose variations prevalent in 2D facial images captured in the wild still remains a challenging and unsolved issue. Among existing approaches of pose-invariant face recognition, 3D-aided methods have been demonstrated effective and promising. In this paper, we present an extensive evaluation of two widely adopted frameworks of 3D-aided face recognition in order to compare the state-of-the-art, identify remaining issues, and offer suggestions for future research. Specifically, we compare the pose normalization and the pose synthesis (rendering) based methods in an empirical manner. The database (UHDB31) that we use covers 21 well-controlled pose variations, half of which show a combination of yaw and pitch. Through the experiments, we present the advantages and disadvantages of these two methods to provide solid data for future research in 3D-aided pose-invariant face recognition.

Benchmarking 3D Pose Estimation for Face Recognition

3D-Model-Aided 2D face recognition (MaFR) has attracted a lot of attention in recent years. By registering a 3D model, facial textures of the gallery and the probe can be lifted and aligned in a common space, thus alleviating the challenge of pose variations. One obstacle preventing accurate registration is the 3D-2D pose estimation, which is easily affected by landmarks. In this work, we present the performance that state-of-the-art pose estimation algorithms could reach using state-of-the-art automatic landmark localization methods. We generated an application-specific dataset with more than 59,000 synthetic face images and ground truth camera pose and landmarks, covering 45 poses and six illumination conditions. Our experiments compared four recently proposed pose estimation algorithms using 2D landmarks detected by two automatic methods. Our results highlight one near-real-time landmark detection method and a highly accurate pose estimation algorithm, which would potentially boost the 3D-Model-Aided 2D face recognition performance.

Robust 3D Face Shape Reconstruction from Single Images via Two-Fold Coupled Structure Learning and Off-the-Shelf Landmark Detectors.

In this paper, we propose a robust method for monocular face shape reconstruction (MFSR) using a sparse set of facial landmarks that are detected by most of the off-theshelf landmark detectors. Different from the classical shape-from-shading framework, we formulate the MFSR problem as a Two-Fold Coupled Structure Learning (2FCSL) process, which consists of learning a regression between two subspaces spanned by 3D sparse landmarks and 2D sparse landmarks, and a coupled dictionary learned on 3D sparse and dense shape using K-SVD. To handle variations in face pose, we explicitly incorporate pose estimation in our method. Extensive experiments on both synthetic and real data from two challenging datasets using manual and automatic landmarks indicate that our method achieves promising performance and is robust to pose variations and landmark localization noise.

GoDP: Globally Optimized Dual Pathway deep network architecture for facial landmark localization in-the-wild

Abstract Facial landmark localization is a fundamental module for pose-invariant face recognition. The most common approach for facial landmark detection is cascaded regression, which is composed of two steps: feature extraction and facial shape regression. Recent methods employ deep convolutional networks to extract robust features for each step, while the whole system could be regarded as a deep cascaded regression architecture. In this work, instead of employing a deep regression network, a Globally Optimized Dual-Pathway (GoDP) deep architecture is proposed to identify the target pixels through solving a cascaded pixel labeling problem without resorting to high-level inference models or complex stacked architecture. The proposed end-to-end system relies on distance-aware soft-max functions and dual-pathway proposal-refinement architecture. Results show that it outperforms the state-of-the-art cascaded regression-based methods on multiple in-the-wild face alignment databases. The model achieves 1.84 normalized mean error (NME) on the AFLW database [1], which outperforms 3DDFA [2] by 61.8%. Experiments on face identification demonstrate that GoDP, coupled with DPM-headhunter [3], is able to improve rank-1 identification rate by 44.2% compare to Dlib [4] toolbox on a challenging database.

Towards fitting a 3D dense facial model to a 2D image: A landmark-free approach

Head pose estimation helps to align a 3D face model to a 2D image, which is critical to research requiring dense 2D-to-2D or 3D-to-2D correspondence. Traditional pose estimation relies strongly on the accuracy of landmarks, so it is sensitive to missing or incorrect landmarks. In this paper, we propose a landmark-free approach to estimate the pose projection matrix. The method can be used to estimate this matrix in unconstrained scenarios and we demonstrate its effectiveness through multiple head pose estimation experiments.

Annotated face model-based alignment: a robust landmark-free pose estimation approach for 3D model registration

Registering a 3D facial model onto a 2D image is important for constructing pixel-wise correspondences between different facial images. The registration is based on a 3

Evaluation of a 3D-aided pose invariant 2D face recognition system

\times