Gary A. Atkinson

Recovery of surface orientation from diffuse polarization

When unpolarized light is reflected from a smooth dielectric surface, it becomes partially polarized. This is due to the orientation of dipoles induced in the reflecting medium and applies to both specular and diffuse reflection. This paper is concerned with exploiting polarization by surface reflection, using images of smooth dielectric objects, to recover surface normals and, hence, height. This paper presents the underlying physics of polarization by reflection, starting with the Fresnel equations. These equations are used to interpret images taken with a linear polarizer and digital camera, revealing the shape of the objects. Experimental results are presented that illustrate that the technique is accurate near object limbs, as the theory predicts, with less precise, but still useful, results elsewhere. A detailed analysis of the accuracy of the technique for a variety of materials is presented. A method for estimating refractive indices using a laser and linear polarizer is also given.

3D face reconstructions from photometric stereo using near infrared and visible light

This paper seeks to advance the state-of-the-art in 3D face capture and processing via novel Photometric Stereo (PS) hardware and algorithms. The first contribution is a new high-speed 3D data capture system, which is capable of acquiring four raw images in approximately 20ms. The results presented in this paper demonstrate the feasibility of deploying the device in commercial settings. We show how the device can operate with either visible light or near infrared (NIR) light. The NIR light sources offer the advantages of being less intrusive and more covert than most existing face recognition methods allow. Furthermore, our experiments show that the accuracy of the reconstructions is also better using NIR light. The paper also presents a modified four-source PS algorithm which enhances the surface normal estimates by assigning a likelihood measure for each pixel being in a shadowed region. This likelihood measure is determined by the discrepancies between measured pixel brightnesses and expected values. Where the likelihood of shadow is high, then one light source is omitted from the computation for that pixel, otherwise a weighted combination of pixels is used to determine the surface normal. This means that the precise shadow boundary is not required by our method. The results section of the paper provides a detailed analysis of the methods presented and a comparison to ground truth. We also analyse the reflectance properties of a small number of skin samples to test the validity of the Lambertian model and point towards potential improvements to our method using the Oren-Nayar model.

Shape Estimation Using Polarization and Shading from Two Views

This paper presents a novel method for 3D surface reconstruction that uses polarization and shading information from two views. The method relies on the polarization data acquired using a standard digital camera and a linear polarizer. Fresnel theory is used to process the raw images and to obtain initial estimates of surface normals, assuming that the reflection type is diffuse. Based on this idea, the paper presents two novel contributions to the problem of surface reconstruction. The first is a technique to enhance the surface normal estimates by incorporating shading information into the method. This is done using robust statistics to estimate how the measured pixel brightnesses depend on the surface orientation. This gives an estimate of the object material reflectance function, which is used to refine the estimates of the surface normals. The second contribution is to use the refined estimates to establish correspondence between two views of an object. To do this, surface patches are extracted from each view, which are then aligned by minimising an energy functional based on the surface normal estimates and local topographic properties. The optimum alignment parameters for different patch pairs are then used to establish stereo correspondence. This process results in an unambiguous field of surface normals, which can be integrated to recover the surface depth. Our technique is most suited to smooth nonmetallic surfaces. It complements existing stereo algorithms since it does not require salient surface features to obtain correspondences. An extensive set of experiments, yielding reconstructed objects and reflectance functions, are presented and compared to ground truth.

Multi-view surface reconstruction using polarization

A new technique for surface reconstruction is developed that uses polarization information from two views. One common problem arising from many multiple view techniques is that of finding correspondences between pixels on each image. In the new method, these correspondences are found by exploiting the spontaneous polarization of light caused by reflection to recover surface normals. These normals are then used to recover surface height. The similarity between reconstructed surface regions determines whether or not a pair of points correspond to each other. The technique is thus able to overcome the convex/concave ambiguity found in many single view techniques. Because the technique relies on smooth surface regions to detect correspondences, rather than feature detection, it is applicable to objects normally inaccessible to stereo vision. Also due to this fact, it is possible to remove noise without causing oversmoothing problems.

Face Recognition and Verification Using Photometric Stereo: The Photoface Database and a Comprehensive Evaluation

This paper presents a new database suitable for both 2-D and 3-D face recognition based on photometric stereo (PS): the Photoface database. The database was collected using a custom-made four-source PS device designed to enable data capture with minimal interaction necessary from the subjects. The device, which automatically detects the presence of a subject using ultrasound, was placed at the entrance to a busy workplace and captured 1839 sessions of face images with natural pose and expression. This meant that the acquired data is more realistic for everyday use than existing databases and is, therefore, an invaluable test bed for state-of-the-art recognition algorithms. The paper also presents experiments of various face recognition and verification algorithms using the albedo, surface normals, and recovered depth maps. Finally, we have conducted experiments in order to demonstrate how different methods in the pipeline of PS (i.e., normal field computation and depth map reconstruction) affect recognition and verification performance. These experiments help to 1) demonstrate the usefulness of PS, and our device in particular, for minimal-interaction face recognition, and 2) highlight the optimal reconstruction and recognition algorithms for use with natural-expression PS data. The database can be downloaded from http://www.uwe.ac.uk/research/Photoface.

The Photoface database

In this paper we present a new database suitable for both 2D and 3D face recognition based on photometric stereo, the so-called Photoface database. The Photoface database was collected using a custom-made four-source photometric stereo device that could be easily deployed in commercial settings. Unlike other publicly available databases the level of cooperation between subjects and the capture mechanism was minimal. The proposed device may also be used, to capture 3D expressive faces. Apart from the description of the device and the Photoface database, we present experiments from baseline face recognition and verification algorithms using albedo, normals and the recovered depth maps. Finally, we have conducted experiments in order to demonstrate how different methods in the pipeline of photometric stereo (i.e. normal field computation and depth map reconstruction methods) affect recognition/verification performance.

Two-dimensional BRDF estimation from polarisation

This paper presents a novel technique for reflectance function (BRDF) estimation, which uses polarisation information and photometric stereo. The first stage of the technique is standard and involves the acquisition of polarisation information (angle and degree of polarisation) using a linear polariser and a digital camera. This yields a field of ambiguous surface normal estimates for an arbitrarily shaped object. A photometric stereo algorithm is then used with three different light source directions to disambiguate the surface normals. Next, the proposed algorithm constructs a 3D histogram of the surface normals and pixel brightnesses. A surface, representing the BRDF, is then fitted to the histogram data using simulated annealing optimisation. The result is a set of Cartesian triples that relate the surface normals to the observed pixel brightnesses. Unlike most previous techniques for BRDF estimation, the technique is image-based and does not require sophisticated equipment or intrusive light sources. Although the technique is restricted to smooth and slightly rough dielectric objects, no prior knowledge about the surface geometry is assumed.

Face recognition in 2D and 2.5D using ridgelets and photometric stereo

A new technique for face recognition - Ridgefaces - is presented. The method combines the well-known Fisherface method with the ridgelet transform and high-speed Photometric Stereo (PS). The paper first derives ridgelet projections for 2D/2.5D face images before the Fisherface approach is used to reduce the dimensionality and increase the spread of the resulting feature vectors. The ridgelet transform is attractive because it is efficient at extracting highly discriminating low-frequency directional features. Best recognition is obtained when Ridgefaces is performed on surface normals acquired from PS, although good results are also found using standard 2D images and PS-derived albedo maps.

Surface reconstruction using polarization and photometric stereo

This paper presents a novel shape recovery technique that combines photometric stereo with polarization information. First, a set of ambiguous surface normals are estimated from polarization data. This is achieved using Fresnel theory to interpret the polarization patterns of light reflected from dielectric surfaces. The process is repeated using three different known light source positions. Photometric stereo is then used to disambiguate the surface normals. The relative pixel brightnesses for the different light source positions reveal the correct surface orientations. Finally, the resulting unambiguous surface normal estimates are integrated to recover a depth map. The technique is tested on various objects of different materials. The paper also demonstrates how the depth estimates can be enhanced by applying methods suggested in earlier work.

A robust multi-scale integration method to obtain the depth from gradient maps

Highlights? A critical survey and classification of gradient integration methods. ? A new multi-scale integrator to recover the depth map from a gradient map. ? Use of an input weight map to identify missing data. ? A weight-sensitive Poisson formulation of the integration problem. ? Empirical comparison with state-of-art methods. We describe a robust method for the recovery of the depth map (or height map) from a gradient map (or normal map) of a scene, such as would be obtained by photometric stereo or interferometry. Our method allows for uncertain or missing samples, which are often present in experimentally measured gradient maps, and also for sharp discontinuities in the scenes depth, e.g. along object silhouette edges. By using a multi-scale approach, our integration algorithm achieves linear time and memory costs. A key feature of our method is the allowance for a given weight map that flags unreliable or missing gradient samples. We also describe several integration methods from the literature that are commonly used for this task. Based on theoretical analysis and tests with various synthetic and measured gradient maps, we argue that our algorithm is as accurate as the best existing methods, handling incomplete data and discontinuities, and is more efficient in time and memory usage, especially for large gradient maps.