Philip L. Worthington

New constraints on data-closeness and needle map consistency for shape-from-shading

This paper makes two contributions to the problem of needle-map recovery using shape-from-shading. First, we provide a geometric update procedure which allows the image irradiance equation to be satisfied as a hard constraint. This not only improves the data closeness of the recovered needle-map, but also removes the necessity for extensive parameter tuning. Second, we exploit the improved ease of control of the new shape-from-shading process to investigate various types of needle-map consistency constraint. The first set of constraints are based on needle-map smoothness. The second avenue of investigation is to use curvature information to impose topographic constraints. Third, we explore ways in which the needle-map is recovered so as to be consistent with the image gradient field. In each case we explore a variety of robust error measures and consistency weighting schemes that can be used to impose the desired constraints on the recovered needle-map. We provide an experimental assessment of the new shape-from-shading framework on both real world images and synthetic images with known ground truth surface normals. The main conclusion drawn from our analysis is that the data-closeness constraint improves the efficiency of shape-from-shading and that both the topographic and gradient consistency constraints improve the fidelity of the recovered needle-map.

Needle map recovery using robust regularizers

This article describes how robust error-kernels can be used as smoothness priors in recovering shape-from-shading (SFS). Conventionally, the smoothness error is added to the data-closeness (or brightness-error) as a quadratic regularizer. This leads to over-smoothing of the recovered needle-map or surface, and the loss of important detail provided by surface discontinuities. To solve this problem, we investigate the use of robust regularizers to reduce the smoothening by treating rapid changes in surface orientation as outliers in the calculation of the smoothness error. In particular, we study an existing continuous approximation to the Tukey bi-weight as a robust regularizer, and introduce a novel regularizer of the form log cosh h , which approximates the Huber estimator. The latter regularizer has a sigmoidal derivative and offers a compromise between premature outlier rejection and smoothening. Experiments on synthetic and real-world data reveal that this robust regularizer enhances needle-map recovery, without sacrificing robustness to noise or becoming over-sensitive to numerical instabilities. q 1999 Elsevier Science B.V. All rights reserved.

Object recognition using shape-from-shading

Investigates whether surface topography information extracted from intensity images using a shape-from-shading (SFS) algorithm can be used for the purposes of 3D object recognition. We consider how curvature and shape-index information delivered by this algorithm can be used to recognize objects based on their surface topography. We explore two contrasting object recognition strategies. The first of these is based on a low-level attribute summary and uses histograms of curvature and orientation measurements. The second approach is based on the structural arrangement of constant shape-index maximal patches and their associated region attributes. We show that region curvedness and a string ordering of the regions according to size provides recognition accuracy of about 96 percent. By polling various recognition schemes, including a graph matching method, we show that a recognition rate of 98-99 percent is achievable.

Enhanced Canny edge detection using curvature consistency

Edges are often considered as primary image artifacts for extraction by low-level processing techniques, and the starting point for many computer vision techniques. As a result, reliable edge detection has long been a research goal. This paper describes initial investigations into recovering reliable edges using curvature models. Essentially, we modify Cannys edge detector using a curvature consistency process to adjust the gradient direction estimates prior to finding the zero crossings in those directions.

3D surface topography from intensity images

The paper demonstrates how a new shape from shading scheme can be used to extract topographic information from 2D intensity imagery. The shape-from-shading scheme has two novel ingredients. Firstly, it uses a geometric update procedure which allows the image irradiance equation to be satisfied as a hard constraint. This not only improves the data closeness of the recovered needle-map, but also removes the necessity for extensive parameter tuning. Secondly, we use curvature information to impose topographic constraints on the recovered needle-map. The topographic information is captured using the shape index of J.J. Koenderink and A.J. Van Doorn (1995) and consistency is imposed using a robust error function. We show that the new shape-from-shading scheme leads to a meaningful topographic labelling of 3D surface structures.

Estimating facial pose using shape-from-shading

This paper reports the application of a recently developed shape-from-shading technique to estimate facial pose. The shape-from-shading algorithm uses a new geometric technique for solving the image irradiance equation together with curvature consistency constraints. Orientation histograms extracted from the the needle-maps delivered by the new shape-from-shading algorithm are used to estimate facial pose. We present a simple model of how the histogram bin-contents transform under rotation of the head. The parameters of this model are the head pose angles. We estimate pose by searching for the rotation angles which maximise the correlation between transformed histograms. A sensitivity analysis reveals that the methods can deliver pose estimates that are accurate within a few degrees.

Surface topography using shape-from-shading

Abstract This paper demonstrates how a recently reported shape-from-shading scheme can be used to extract topographic information from 2D intensity imagery (Worthington and Hancock, IEEE Trans. Pattern Anal. Mach. Intell. 21 (1999) 1250–1267). The shape-from-shading scheme has two important features which enhance the recovered surface description. Firstly, it uses a geometric update procedure which allows the image irradiance equation to be satisfied as a hard constraint. This not only improves the data closeness of the recovered needle map, but also removes the necessity for extensive parameter tuning. Secondly, we use curvature information to impose topographic constraints on the recovered needle map. The topographic information is captured using the shape index of Koenderink and van Doorn (Image Vision Comput. 10 (1992) 557–565) and consistency is imposed using a robust error function. We show that the new shape-from-shading scheme leads to a meaningful topographic labelling of 3D surface structures. Moreover, the resulting topographic information proves to be useful in a simple histogram-based object recognition scheme.

Appearance-based object recognition using shape-from-shading

This paper investigates the use of shape-from-shading for object recognition. The local surface orientation information recovered using shape-from-shading is shown to provide useful input to an appearance-based object recognition scheme. We consider two representations which may be recovered from shading information-the needle-map, and the local curvature shape-index-and their relative performance for object recognition. Specifically, we use a histogram-comparison technique, and focus upon the relative stability of the representations to small changes of viewpoint. We demonstrate that the needle-map representation allows the view-sphere to be spanned using a significantly smaller number of characteristic views than using either the raw images or the shape index.

Novel View Synthesis using Needle-Map Correspondence.

Interest in view interpolation and novel view synthesis is growing. In this paper we show how dense correspondence can be found between needle-maps generated using shape-from-shading, which in turn can be used to generate new needle-maps. From these we can produce novel intermediate views, and also estimates of how each intermediate view would look under different lighting conditions. The approach offers the prospect of creating large sets of realistic views of a scene under different viewing and lighting conditions from a small number of original images.

Histogram-based object recognition using shape-from-shading

This paper presents an experimental study of the problem of object recognition using surface attributes delivered by shape-from-shading. We focus our attention on histogram-based representations. Several surface attributes are considered. These include slant and tilt angles, mean and Gaussian curvature, and, shape-index and curvedness. Our study reveals that recognition rates of 98% are achievable under controlled lighting conditions. When there is significant illumination variation then recognition rates of 80% are achievable.