Mark F. Hansen

3D face reconstructions from photometric stereo using near infrared and visible light

This paper seeks to advance the state-of-the-art in 3D face capture and processing via novel Photometric Stereo (PS) hardware and algorithms. The first contribution is a new high-speed 3D data capture system, which is capable of acquiring four raw images in approximately 20ms. The results presented in this paper demonstrate the feasibility of deploying the device in commercial settings. We show how the device can operate with either visible light or near infrared (NIR) light. The NIR light sources offer the advantages of being less intrusive and more covert than most existing face recognition methods allow. Furthermore, our experiments show that the accuracy of the reconstructions is also better using NIR light. The paper also presents a modified four-source PS algorithm which enhances the surface normal estimates by assigning a likelihood measure for each pixel being in a shadowed region. This likelihood measure is determined by the discrepancies between measured pixel brightnesses and expected values. Where the likelihood of shadow is high, then one light source is omitted from the computation for that pixel, otherwise a weighted combination of pixels is used to determine the surface normal. This means that the precise shadow boundary is not required by our method. The results section of the paper provides a detailed analysis of the methods presented and a comparison to ground truth. We also analyse the reflectance properties of a small number of skin samples to test the validity of the Lambertian model and point towards potential improvements to our method using the Oren-Nayar model.

Face Recognition and Verification Using Photometric Stereo: The Photoface Database and a Comprehensive Evaluation

This paper presents a new database suitable for both 2-D and 3-D face recognition based on photometric stereo (PS): the Photoface database. The database was collected using a custom-made four-source PS device designed to enable data capture with minimal interaction necessary from the subjects. The device, which automatically detects the presence of a subject using ultrasound, was placed at the entrance to a busy workplace and captured 1839 sessions of face images with natural pose and expression. This meant that the acquired data is more realistic for everyday use than existing databases and is, therefore, an invaluable test bed for state-of-the-art recognition algorithms. The paper also presents experiments of various face recognition and verification algorithms using the albedo, surface normals, and recovered depth maps. Finally, we have conducted experiments in order to demonstrate how different methods in the pipeline of PS (i.e., normal field computation and depth map reconstruction) affect recognition and verification performance. These experiments help to 1) demonstrate the usefulness of PS, and our device in particular, for minimal-interaction face recognition, and 2) highlight the optimal reconstruction and recognition algorithms for use with natural-expression PS data. The database can be downloaded from http://www.uwe.ac.uk/research/Photoface.

Twins 3D face recognition challenge

Existing 3D face recognition algorithms have achieved high enough performances against public datasets like FRGC v2, that it is difficult to achieve further significant increases in recognition performance. However, the 3D TEC dataset is a more challenging dataset which consists of 3D scans of 107 pairs of twins that were acquired in a single session, with each subject having a scan of a neutral expression and a smiling expression. The combination of factors related to the facial similarity of identical twins and the variation in facial expression makes this a challenging dataset. We conduct experiments using state of the art face recognition algorithms and present the results. Our results indicate that 3D face recognition of identical twins in the presence of varying facial expressions is far from a solved problem, but that good performance is possible.

The Photoface database

In this paper we present a new database suitable for both 2D and 3D face recognition based on photometric stereo, the so-called Photoface database. The Photoface database was collected using a custom-made four-source photometric stereo device that could be easily deployed in commercial settings. Unlike other publicly available databases the level of cooperation between subjects and the capture mechanism was minimal. The proposed device may also be used, to capture 3D expressive faces. Apart from the description of the device and the Photoface database, we present experiments from baseline face recognition and verification algorithms using albedo, normals and the recovered depth maps. Finally, we have conducted experiments in order to demonstrate how different methods in the pipeline of photometric stereo (i.e. normal field computation and depth map reconstruction methods) affect recognition/verification performance.

Biologically inspired 3D face recognition from surface normals

A major consideration in state-of-the-art face recognition systems is the amount of data that is required to represent a face. Even a small (64×64) photograph of a face has 212 dimensions in which a face may sit. When large (>1 MB) photographs of faces are used, this represents a very large (and practically intractable) space and ways of reducing dimensionality without losing discriminatory information are needed for storing data for recognition. The eigenface technique, which is based upon Principal Components Analysis (PCA), is a well established dimension reduction method in face recognition research but does not have any biological basis. Humans excel at familiar face recognition and this paper attempts to show that modelling a biologically plausible process is a valid alternative approach to using eigenfaces for dimension reduction. Using a biologically inspired method to extract the certain facial discriminatory information which mirrors some of the idiosyncrasies of the human visual system, we show that recognition rates remain high despite 90% of the raw data being discarded.

Towards on-farm pig face recognition using convolutional neural networks

Abstract Identification of individual livestock such as pigs and cows has become a pressing issue in recent years as intensification practices continue to be adopted and precise objective measurements are required (e.g. weight). Current best practice involves the use of RFID tags which are time-consuming for the farmer and distressing for the animal to fit. To overcome this, non-invasive biometrics are proposed by using the face of the animal. We test this in a farm environment, on 10 individual pigs using three techniques adopted from the human face recognition literature: Fisherfaces, the VGG-Face pre-trained face convolutional neural network (CNN) model and our own CNN model that we train using an artificially augmented data set. Our results show that accurate individual pig recognition is possible with accuracy rates of 96.7% on 1553 images. Class Activated Mapping using Grad-CAM is used to show the regions that our network uses to discriminate between pigs.

Baseline face recognition using photometric stereo data

Abstract This research is motivated by the need for face recognition in uncontrolled environments. In other words, we are interested in face recognition arrangements whereby the users do not need to interact with the recognition technology. The contribution of this paper is to perform a range of recognition experiments on face image data as people casually enter a building, without any instructions about expression. Specifically, we capture four images per session in rapid succession (all within 20 ms). The four images are synchronised to different light sources to enable photometric stereo processing to estimate albedo images, surface normals and depth maps. Additional capture sessions then take place over periods of many weeks. Our recognition experiments are on each of the three modalities as well as a fusion technique for the albedo and depth. Using a variety of photometric stereo methods, surface integration methods (to recover depth) and recognition algorithms such as principal component analysis and nonnegative matrix factorisation, we acquire a maximum recognition rate of 86% for 96 subjects.

Long-range concealed object detection through active covert illumination

A common surveillance problem is the automatic detection of objects concealed under clothing and the identification of those carrying them. As many 2D methods rely on texture information, the application of patterned clothing can be used to camouflage features that may provide a clue as to the shape of the object hidden beneath. Photometric stereo (PS) is a 3D surface reconstruction technique utilising several images of an object, lit from multiple directions, a particular advantage of which is that it reliably separates textural elements, such as printed patterns, from physical shape offering many possibilities for concealed object detection. The success of such a technique is primarily dependent on the ability to artificially illuminate the subject considerably more brightly than the ambient lighting. At night, this is readily plausible; and longer wavelength, near-infrared (nIR) lighting allows us to capture the images covertly. However in daytime, sunlight can prevent sufficient illumination of the subject to calculate the surface image, especially at long range. Certain wavelengths of light are attenuated by airborne moisture considerably more than others. By using a wavelength of light that is heavily attenuated by the atmosphere, in combination with a narrow bandpass filter, we show that it is possible to provide sufficient lighting contrast to perform PS over much longer distances than in previous work. We examine the 940nm wavelength, which falls within one of these spectral regions and evaluate sensor technology equipped with a “black silicon” CMOS, offering extreme light sensitivity, against cameras using traditional silicon sensors, with application to long distance surface reconstruction using PS. Having shown that we can produce reconstructions of considerably better quality than those from traditional cameras, we present several methods for the reliable detection of concealed objects and recognition of faces, using the high level of surface detail that PS can provide.

Photometric stereo for three-dimensional leaf venation extraction

Highlights • An accurate and robust leaf venation extraction method is proposed.• The proposed 3D imaging system can recover illumination-independent and high-resolution surface normal features.• The proposed venation extraction algorithm employs local shape measures by fusing shape index and curvedness features.• The algorithm can determine venation polarity – whether veins are raised above or recessed into a leaf.• The proposed method can overcome undesirable variations commonly found in real-world environments.

Automated monitoring of dairy cow body condition, mobility and weight using a Single 3D Video Capture Device

Highlights • 3D imaging for concurrently monitoring cow body condition, lameness and weight.• Novel rolling ball software tool is proposed for body condition assessment.• Original moving spine segmentation/modelling approach in 3D postulated.• Real-world performance that is comparable or better than manual scoring.• Limitations of conventional scoring discussed and a learning approach introduced.