Melvyn L. Smith

Object surface recovery using a multi-light photometric stereo technique for non-Lambertian surfaces subject to shadows and specularities

This paper presents a new multi-light source photometric stereo system for reconstructing images of various characteristics of non-Lambertian rough surfaces with widely varying texture and specularity. Compared to the traditional three-light photometric stereo method, extra lights are employed using a hierarchical selection strategy to eliminate the effects of shadows and specularities, and to make the system more robust. We also show that six lights is the minimum needed in order to apply photometric stereo to the entire visible surface of any convex object. Experiments on synthetic and real scenes demonstrate that the proposed method can extract surface reflectance and orientation effectively, even in the presence of strong shadows and highlights. Hence, the method offers advantages in the recovery of dichromatic surfaces possessing rough texture or deeply relieved topographic features, with applications in reverse engineering and industrial surface inspection. Experimental results are presented in the paper.

3D face reconstructions from photometric stereo using near infrared and visible light

This paper seeks to advance the state-of-the-art in 3D face capture and processing via novel Photometric Stereo (PS) hardware and algorithms. The first contribution is a new high-speed 3D data capture system, which is capable of acquiring four raw images in approximately 20ms. The results presented in this paper demonstrate the feasibility of deploying the device in commercial settings. We show how the device can operate with either visible light or near infrared (NIR) light. The NIR light sources offer the advantages of being less intrusive and more covert than most existing face recognition methods allow. Furthermore, our experiments show that the accuracy of the reconstructions is also better using NIR light. The paper also presents a modified four-source PS algorithm which enhances the surface normal estimates by assigning a likelihood measure for each pixel being in a shadowed region. This likelihood measure is determined by the discrepancies between measured pixel brightnesses and expected values. Where the likelihood of shadow is high, then one light source is omitted from the computation for that pixel, otherwise a weighted combination of pixels is used to determine the surface normal. This means that the precise shadow boundary is not required by our method. The results section of the paper provides a detailed analysis of the methods presented and a comparison to ground truth. We also analyse the reflectance properties of a small number of skin samples to test the validity of the Lambertian model and point towards potential improvements to our method using the Oren-Nayar model.

The analysis of surface texture using photometric stereo acquisition and gradient space domain mapping

Abstract A novel machine vision based inspection technique was developed for the analysis of three-dimensional (3D) surface textural patterns. Previous machine vision based methods of texture analysis have in general analysed projected two-dimensional (2D) textural patterns. This paper presents an innovative approach for the direct analysis of 3D texture topography. The method employed has particular application for the evaluation of textures frequently encountered during numerous manufacturing and finishing processes. A 3D surface topographic description is acquired using a photometric stereo technique. A method of gradient space domain mapping and surface reconstruction are used to characterise textural form, shape and regularity, and to quantify any observed deviation in an idealised form in relation to a CAD based prototypical texture model. Unlike conventional viewer centred methods for the assessment of projected texture, the technique offers a 3D object centred analysis, uses a fixed lighting configuration, is largely insensitive to a variation in object pose, and is rapid in operation. These aspects are of considerable advantage in terms of practical application. The technique was applied to real texture samples, and a selection of experimental results are presented in this paper.

Automated inspection of textured ceramic tiles

Abstract An innovative object-centred approach for the visual inspection of textured ceramic tiles has been developed for the acquisition of product quality control data. A demonstrator system has been constructed for the analysis of complex surfaces, surfaces which possess detailed three-dimensional topographic features, which are concealed by coincident regular or random chromatic patterns. The technique is able to isolate mixed topographic and chromatic surface features in order to allow for subsequent separate analysis. Surface topography is captured as an array of perturbed surface normals, known as a surface bump map, while the surface reflectance is isolated as a separate albedo image. Synthetically rendered images of the bump map description allow surface topography to be viewed off-line under user defined virtual illumination conditions, and in isolation from the surface albedo. Topographic features defined by the bump map description may be subject to analysis in terms of perturbation thresholding, or integrated to recover their three-dimensional relief. The technique has application for the quality inspection of a wide range of manufactured and processed surfaces, uses a low-cost fixed lighting configuration, requires no initial training, is largely pose invariant, and has been tested on a representative range of real tile samples. Experimental results are presented in the paper.

Gradient space analysis of surface defects using a photometric stereo derived bump map

An innovative method has been developed for the quantification of the surface quality of Lambertian component materials, while in the presence of a complex coincident albedo pattern. The technique uses a generic lighting configuration and is insensitive to component object pose. This is of considerable advantage for the relaxation of environmental structuring during on-line component inspection. A two-stage inspection strategy is proposed, consisting of an initial global surface assessment, together with the subsequent classification of segmented surface defect features. The technique has particular application for the detection of characteristic surface faults, such as indentations, protrusions or missing surface topology, typical of a range of manufacturing processes, including injection moulding, machining and stone polishing. The technique has been validated using a variety of flat and curved specimen components. Experimental results are presented in the paper.

Reflectance of human skin using colour photometric stereo: with particular application to pigmented lesion analysis

Background/purpose: The optical appearance of human skin is highly dependent on the interaction between the illumination (type and position), observer position and the skin surface structure. Different currently available photographic techniques record different aspects of this appearance, each providing its own incomplete description. This limits their usefulness, especially for pigmented skin lesion diagnosis. In this paper a new, easy to use, low‐cost photographic method is described,which aims to generate an efficiently encoded yet reasonably complete representation of skin appearance.

Face Recognition and Verification Using Photometric Stereo: The Photoface Database and a Comprehensive Evaluation

This paper presents a new database suitable for both 2-D and 3-D face recognition based on photometric stereo (PS): the Photoface database. The database was collected using a custom-made four-source PS device designed to enable data capture with minimal interaction necessary from the subjects. The device, which automatically detects the presence of a subject using ultrasound, was placed at the entrance to a busy workplace and captured 1839 sessions of face images with natural pose and expression. This meant that the acquired data is more realistic for everyday use than existing databases and is, therefore, an invaluable test bed for state-of-the-art recognition algorithms. The paper also presents experiments of various face recognition and verification algorithms using the albedo, surface normals, and recovered depth maps. Finally, we have conducted experiments in order to demonstrate how different methods in the pipeline of PS (i.e., normal field computation and depth map reconstruction) affect recognition and verification performance. These experiments help to 1) demonstrate the usefulness of PS, and our device in particular, for minimal-interaction face recognition, and 2) highlight the optimal reconstruction and recognition algorithms for use with natural-expression PS data. The database can be downloaded from http://www.uwe.ac.uk/research/Photoface.

Dynamic photometric stereo-a new technique for moving surface analysis

This paper describes a novel approach for two- and three-dimensional surface data capture from moving surfaces based upon an evolution of the existing photometric stereo (PS) technique. Limitations in current methods are described, together with the potential benefits of applying PS and the particular need for a new dynamic form of the method. Important issues relating to conventional idealised PS model assumptions are considered in the context of realising useful practical application, including in particular the modelling of real illuminates. Several possible techniques for achieving dynamic PS are considered and a new technique termed narrow infrared photometric stereo (NIRPS) introduced. New potential application areas range from the continuous inspection of fast moving surfaces typically encountered in numerous industrial processes to three-dimensional surface topographic texture acquisition in the field using portable hand-held technology. A selection of experimental results is presented in the paper.

Dynamic photometric stereo for on line quality control of ceramic tiles

The rapid and automated detection of manufacturing flaws is becoming increasingly important in order to maintain competitive advantage in many production environments. In the case of natural and ornamental materials, the presence of both surface colouration and surface topography is often such that manual inspection, along with many conventional imaging techniques, fails to isolate physical or structural defects in the presence of complex and random patterns. In this paper the concepts of photometric stereo are adapted and extended for application in manufacturing environments. A case study on the high speed inspection of ceramic tiles is presented for the analysis of surfaces at production line rates of up to 30 m/min. This new technique, for the first time, demonstrates a genuine and commercially attractive potential for the practical automated quality control of complex surfaces. A commercial system, based on this research, is currently being developed.

Overview of passive and active vision techniques for hand-held 3D data acquistion

The digitization of the 3D shape of real objects is a rapidly expanding discipline, with a wide variety of applications, including shape acquisition, inspection, reverse engineering, gauging and robot navigation. Developments in computer product design techniques, automated production, and the need for close manufacturing tolerances will be facts of life for the foreseeable future. A growing need exists for fast, accurate, portable, non-contact 3D sensors. However, in order for 3D scanning to become more commonplace, new methods are needed for easily, quickly and robustly acquiring accurate full geometric models of complex objects using low cost technology. In this paper, a brief survey is presented of current scanning technologies available for acquiring range data. An overview is provided of current 3D-shape acquisition using both active and passive vision techniques. Each technique is explained in terms of its configuration, principle of operation, and the inherent advantages and limitations. A separate section then focuses on the implications of scannerless scanning for hand held technology, after which the current status of 3D acquisition using handheld technology, together with related issues concerning implementation, is considered more fully. Finally, conclusions for further developments in handheld devices are discussed. This paper may be of particular benefit to new comers in this field.