Xiaozheng Zhang

Face recognition across pose: A review

One of the major challenges encountered by current face recognition techniques lies in the difficulties of handling varying poses, i.e., recognition of faces in arbitrary in-depth rotations. The face image differences caused by rotations are often larger than the inter-person differences used in distinguishing identities. Face recognition across pose, on the other hand, has great potentials in many applications dealing with uncooperative subjects, in which the full power of face recognition being a passive biometric technique can be implemented and utilised. Extensive efforts have been put into the research toward pose-invariant face recognition in recent years and many prominent approaches have been proposed. However, several issues in face recognition across pose still remain open, such as lack of understanding about subspaces of pose variant images, problem intractability in 3D face modelling, complex face surface reflection mechanism, etc. This paper provides a critical survey of researches on image-based face recognition across pose. The existing techniques are comprehensively reviewed and discussed. They are classified into different categories according to their methodologies in handling pose variations. Their strategies, advantages/disadvantages and performances are elaborated. By generalising different tactics in handling pose variations and evaluating their performances, several promising directions for future research have been suggested.

Recognizing Rotated Faces From Frontal and Side Views: An Approach Toward Effective Use of Mugshot Databases

Mug shot photography has been used to identify criminals by the police for more than a century. However, the common scenario of face recognition using frontal and side-view mug shots as gallery remains largely uninvestigated in computerized face recognition across pose. This paper presents a novel appearance-based approach using frontal and sideface images to handle pose variations in face recognition, which has great potential in forensic and security applications involving police mugshot databases. Virtual views in different poses are generated in two steps: 1) shape modelling and 2) texture synthesis. In the shape modelling step, a multilevel variation minimization approach is applied to generate personalized 3-D face shapes. In the texture synthesis step, face surface properties are analyzed and virtual views in arbitrary viewing conditions are rendered, taking diffuse and specular reflections into account. Appearance-based face recognition is performed with the augmentation of synthesized virtual views covering possible viewing angles to recognize probe views in arbitrary conditions. The encouraging experimental results demonstrated that the proposed approach by using frontal and side-view images is a feasible and effective solution to recognizing rotated faces, which can lead to a better and practical use of existing forensic databases in computerized human face-recognition applications.

Automatic Texture Synthesis for Face Recognition from Single Views

One possible solution for pose- and illumination-invariant face recognition is to employ appearance-based approaches, which rely greatly on correct facial textures. However, existing facial texture analysis algorithms are suboptimal, because they usually neglect specular reflections and require numerous training images for virtual view synthesis. This paper presents a novel texture synthesis approach from a single frontal view for face recognition. Using a generic 3D face shape, facial textures are analyzed with consideration of all of the ambient, diffuse, and specular reflections. Virtual views are synthesized under different poses and illuminations. The proposed approach was evaluated using the CMU-PIE face database. Encouraging results show that the proposed approach improves face recognition performances across pose and illumination variations

Heterogeneous Specular and Diffuse 3-D Surface Approximation for Face Recognition Across Pose

This paper proposes a novel heterogeneous specular and diffuse (HSD) 3-D surface approximation which considers spatial variability of specular and diffuse reflections in face modelling and recognition. Traditional 3-D face modelling and recognition methods constrain human faces with either the Lambertian assumption or the homogeneity assumption, resulting in suboptimal shape and texture models. The proposed HSD approach allows both specular and diffuse reflectance coefficients to vary spatially to better accommodate surface properties of real human faces. From a small number of face images of a person under different lighting conditions, 3-D shape and surface reflectivity property are estimated using a localized stochastic optimization method. The resultant personalized 3-D face model is used to render novel gallery views under different poses for recognition across pose. The proposed approach is evaluated on both synthetic and real face datasets and benchmarked against the state-of-the-art approaches. Experimental results demonstrated that it can achieve a higher level of performances in modelling accuracy, algorithm reliability, and recognition accuracy, which suggests that face modelling and recognition beyond the Lambertian and homogeneity assumptions is a feasible and better solution towards pose-invariant face recognition.

Multilevel Quadratic Variation Minimization for 3D Face Modeling and Virtual View Synthesis

One of the key remaining problems in face recognition is that of handling the variability in appearance due to changes in pose. One strategy is to synthesize virtual face views from real views. In this paper, a novel 3D face shape-modeling algorithm, Multilevel Quadratic Variation Minimization (MQVM), is proposed. Our method makes sole use of two orthogonal real views of a face, i.e., the frontal and profile views. By applying quadratic variation minimization iteratively in a coarse-to-fine hierarchy of control lattices, the MQVM algorithm can generate C²-smooth 3D face surfaces. Then realistic virtual face views can be synthesized by rotating the 3D models. The algorithm works properly on sparse constraint points and large images. It is much more efficient than single-level quadratic variation minimization. The modeling results suggest the validity of the MQVM algorithm for 3D face modeling and 2D face view synthesis under different poses.

Polygonal Approximation Using Integer Particle Swarm Optimization

Polygonal approximation is an effective yet challenging digital curve representation for image analysis, pattern recognition and computer vision. This paper proposes a novel approach, integer particle swarm optimization (iPSO), for polygonal approximation. When compared to the traditional binary version of particle swarm optimization (bPSO), the new iPSO directly uses an integer vector to represent the candidate solution and provides a more efficient and convenient means for solution processing. The velocity and position updating mechanisms in iPSO not only have clear physical meaning, but also guarantee the optimality of the solutions. The method is suitable for polygonal approximation which could otherwise be an intractable optimization problem. The proposed method has been tested on commonly used synthesized shapes and lake contours extracted from the maps of four famous lakes in the world. The experimental results show that the proposed iPSO has better solution quality and computational efficiency than the bPSO-based methods and better solution quality than the other state-of-the-art methods.

A comparative evaluation of Average Face on holistic and local face recognition approaches

This study focuses on a recent paper ldquo100% Accuracy in Automatic Face Recognitionrdquo published on Science, in which an ldquoAverage Facerdquo is proposed and claimed to be capable of dramatically improving performance of a face recognition system. To reveal its working mechanism, we perform the averaging process using pose-varied synthetic images generated from 3D face database and conduct a comparative study to observe its effectiveness on holistic and local face recognition approaches. Two representative methods, i.e. eigenface and local binary pattern (LBP) are employed to perform the experiments. It is interesting to find from our experiments that the performance of the ldquoAverage Facerdquo is not independent of the face recognition approaches. Although face averaging increases the recognition accuracy of eigenface method, it impairs the performance of LBP method.

Primitive-based 3D structure inference from a single 2D image for insect modeling: Towards an electronic field guide for insect identification

3D insect models are useful to overcome viewing angle variations and self-occlusions in computer-assisted insect taxonomy for electronic field guides. The acquisition of 3D information is, however, unreliable due to the flexibility and small size of the insect bodies. This paper explores how to infer 3D insect models from a single 2D insect image, which will assist both insect description and identification. The 3D structure of the insect body is modeled from two geometric primitives, generalized cylinders and deformable ellipsoids. The primitives are fitted and warped based on both edge and medial axis constraints of the 2D image. Individualized 3D models are then built to approximate the insect structure. The proposed approach results in seemingly useful 3D insect models capable of representing the major morphological characteristics for a variety of insects with different body types. This method could be a helpful assistance for computer-assisted insect taxonomy and insect identification by entomologists and the public.

Parametric manifold of an object under different viewing directions

The appearance of a 3D object depends on both the viewing directions and illumination conditions. It is proven that all n-pixel images of a convex object with Lambertian surface under variable lighting from infinity form a convex polyhedral cone (called illumination cone) in n-dimensional space. This paper tries to answer the other half of the question: What is the set of images of an object under all viewing directions? A novel image representation is proposed, which transforms any n-pixel image of a 3D object to a vector in a 2n-dimensional pose space. In such a pose space, we prove that the transformed images of a 3D object under all viewing directions form a parametric manifold in a 6-dimensional linear subspace. With in-depth rotations along a single axis in particular, this manifold is an ellipse. Furthermore, we show that this parametric pose manifold of a convex object can be estimated from a few images in different poses and used to predict objects appearances under unseen viewing directions. These results immediately suggest a number of approaches to object recognition, scene detection, and 3D modelling. Experiments on both synthetic data and real images were reported, which demonstrates the validity of the proposed representation.

Colour Adjustment and Specular Removal for Non-uniform Shape from Shading

Surface colour changes and specular reflections are two major problems in 3D modelling using shape-from-shading (SFS). This paper proposes to pre-process the input image for a typical SFS algorithm, so that the resultant image has no colour changes and specular reflection. First, a chromaticity-based specular reflection removal algorithm is applied to achieve a pure diffuse (Lambertian) reflected image. Then, a novel chromaticity-based colour adjustment approach is proposed to generate an image without surface colour changes. The standard SFS algorithms can then be applied successfully onto the processed images to produce plausible 3D models. In experiments, the proposed approach was tested on standard SFS datasets with complex surface colours. The experimental results show it’s promising to facilitate SFS algorithms to handle SFS problems with more complex surface properties and illumination conditions.