Volker Blanz

A morphable model for the synthesis of 3D faces

In this paper, a new technique for modeling textured 3D faces is introduced. 3D faces can either be generated automatically from one or more photographs, or modeled directly through an intuitive user interface. Users are assisted in two key problems of computer aided face modeling. First, new face images or new 3D face models can be registered automatically by computing dense one-to-one correspondence to an internal face model. Second, the approach regulates the naturalness of modeled faces avoiding faces with an “unlikely” appearance. Starting from an example set of 3D face models, we derive a morphable face model by transforming the shape and texture of the examples into a vector space representation. New faces and expressions can be modeled by forming linear combinations of the prototypes. Shape and texture constraints derived from the statistics of our example faces are used to guide manual modeling or automated matching algorithms. We show 3D face reconstructions from single images and their applications for photo-realistic image manipulations. We also demonstrate face manipulations according to complex parameters such as gender, fullness of a face or its distinctiveness.

Face recognition based on fitting a 3D morphable model

This paper presents a method for face recognition across variations in pose, ranging from frontal to profile views, and across a wide range of illuminations, including cast shadows and specular reflections. To account for these variations, the algorithm simulates the process of image formation in 3D space, using computer graphics, and it estimates 3D shape and texture of faces from single images. The estimate is achieved by fitting a statistical, morphable model of 3D faces to images. The model is learned from a set of textured 3D scans of heads. We describe the construction of the morphable model, an algorithm to fit the model to images, and a framework for face identification. In this framework, faces are represented by model parameters for 3D shape and texture. We present results obtained with 4,488 images from the publicly available CMU-PIE database and 1,940 images from the FERET database.

Reanimating Faces in Images and Video

This paper presents a method for photo‐realistic animation that can be applied to any face shown in a single imageor a video. The technique does not require example data of the persons mouth movements, and the image to beanimated is not restricted in pose or illumination. Video reanimation allows for head rotations and speech in theoriginal sequence, but neither of these motions is required.

Component-Based Face Recognition with 3D Morphable Models

We present a system for pose and illumination invariant face recognition that combines two recent advances in the computer vision field: 3D morphable models and component-based recognition. A 3D morphable model is used to compute 3D face models from three input images of each subject in the training database. The 3D models are rendered under varying pose and illumination conditions to build a large set of synthetic images. These images are then used for training a component-based face recognition system. The face recognition module is preceded by a fast hierarchical face detector resulting in a system that can detect and identify faces in video images at about 4 Hz. The system achieved a recognition rate of 88% on a database of 2000 real images of ten people, which is significantly better than a comparable global face recognition system. The results clearly show the potential of the combination of morphable models and component-based recognition towards pose and illumination invariant face recognition.

Face identification across different poses and illuminations with a 3D morphable model

We present a novel approach for recognizing faces in images taken from different directions and under different illumination. The method is based on a 3D morphable face model that encodes shape and texture in terms of model parameters, and an algorithm that recovers these parameters from a single image of a face. For face identification, we use the shape and texture parameters of the model that are separated from imaging parameters, such as pose and illumination. In addition to the identity, the system provides a measure of confidence. We report experimental results for more than 4000 images from the publicly available CMU-PIE database.

Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions

This paper presents a novel algorithm aiming at analysis and identification of faces viewed from different poses and illumination conditions. Face analysis from a single image is performed by recovering the shape and textures parameters of a 3D Morphable Model in an analysis-by-synthesis fashion. The shape parameters are computed from a shape error estimated by optical flow and the texture parameters are obtained from a texture error. The algorithm uses linear equations to recover the shape and texture parameters irrespective of pose and lighting conditions of the face image. Identification experiments are reported on more than 5000 images from the publicly available CMU-PIE database which includes faces viewed from 13 different poses and under 22 different illuminations. Extensive identification results are available on our web page for future comparison with novel algorithms.

Face recognition based on frontal views generated from non-frontal images

This paper presents a method for face recognition across large changes in viewpoint. Our method is based on a morphable model of 3D faces that represents face-specific information extracted from a dataset of 3D scans. For non-frontal face recognition in 2D still images, the morphable model can be incorporated in two different approaches: in the first, it serves as a preprocessing step by estimating the 3D shape of novel faces from the non-frontal input images, and generating frontal views of the reconstructed faces at a standard illumination using 3D computer graphics. The transformed images are then fed into state-of-the-art face recognition systems that are optimized for frontal views. This method was shown to be extremely effective in the Face Recognition Vendor Test FRVT 2002. In the process of estimating the 3D shape of a face from an image, a set of model coefficients are estimated. In the second method, face recognition is performed directly from these coefficients. In this paper we explain the algorithm used to preprocess the images in FRVT 2002, present additional FRVT 2002 results, and compare these results to recognition from the model coefficients.

What object attributes determine canonical views

We investigated preferred or canonical views for familiar and three-dimensional nonsense objects using computer-graphics psychophysics. We assessed the canonical views for objects by allowing participants to actively rotate realistically shaded three-dimensional models in realtime. Objects were viewed on a Silicon Graphics workstation and manipulated in virtual space with a three-degree-of-freedom input device. In the first experiment, participants adjusted each object to the viewpoint from which they would take a photograph if they planned to use the object to illustrate a brochure. In the second experiment, participants mentally imaged each object on the basis of the name and then adjusted the object to the viewpoint from which they imagined it. In both experiments, there was a large degree of consistency across participants in terms of the preferred view for a given object. Our results provide new insights on the geometrical, experiential, and functional attributes that determine canonical views under ecological conditions.

Comparison of View-Based Object Recognition Algorithms Using Realistic 3D Models

Two view-based object recognition algorithms are compared: (1) a heuristic algorithm based on oriented filters, and (2) a support vector learning machine trained on low-resolution images of the objects. Classification performance is assessed using a high number of images generated by a computer graphics system under precisely controlled conditions. Training- and test-images show a set of 25 realistic three-dimensional models of chairs from viewing directions spread over the upper half of the viewing sphere. The percentage of correct identification of all 25 objects is measured.

A statistical method for robust 3D surface reconstruction from sparse data

General information about a class of objects, such as human faces or teeth, can help to solve the otherwise ill-posed problem of reconstructing a complete surface from sparse 3D feature points or 2D projections of points. We present a technique that uses a vector space representation of shape (3D morphable model) to infer missing vertex coordinates. Regularization derived from a statistical approach makes the system stable and robust with respect to noise by computing the optimal tradeoff between fitting quality and plausibility. We present a direct, noniterative algorithm to calculate this optimum efficiently, and a method for simultaneously compensating unknown rigid transformations. The system is applied and evaluated in two different fields: (1) reconstruction of 3D faces at unknown orientations from 2D feature points at interactive rates, and (2) restoration of missing surface regions of teeth for CAD-CAM production of dental inlays and other medical applications.