Robert J. Woodham

Photometric Method For Determining Surface Orientation From Multiple Images

A novel technique called photometric stereo is introduced. The idea of photometric stereo is to vary the direction of incident illumination between successive images, while holding the viewing direction constant. It is shown that this provides sufficient information to determine surface orientation at each image point. Since the imaging geometry is not changed, the correspondence between image points is known a priori. The technique is photometric because it uses the radiance values recorded at a single image location, in successive views, rather than the relative positions of displaced features. Photometric stereo is used in computer-based image understanding. It can be applied in two ways. First, it is a general technique for deter-mining surface orientation at each image point. Second, it is a technique for determining object points that have a particular surface orientation. These applications are illustrated using synthesized examples.

Analysing images of curved surfaces

A reflectance map makes the relationship between image intensity and surface orientation explicit. Trade-offs between image intensity and surface orientation emerge which cannot be resolved locally in a single view. Existing methods for determining surface orientation from a single view embody assumptions about surface curvature. The Hessian matrix is introduced to represent surface curvature. Properties of surface curvature are expressed as properties of the Hessian matrix. For several classes of surface, image analysis simplifies. This result has already been established for planar surfaces forming trihedral corners. Similar simplification is demonstrated for developable surfaces and for the subclass of surfaces known as generalized cones. These studies help to delineate shape information that can be determined from geometric measurements at object boundaries and shape information that can be determined from intensity measurements over sections of smoothly curved surface. A novel technique called photometric stereo is discussed. The idea of stereo is to obtain multiple images in order to determine the underlying scene precisely. In photometric stereo, the viewing direction is constant. Multiple images are obtained by varying the incident illumination. It is shown that this provides sufficient information to determine surface orientation at each image point.

Gradient and curvature from the photometric-stereo method, including local confidence estimation

The photometric-stereo method is one technique for three-dimensional shape determination that has been implemented in a variety of experimental settings and that has produced consistently good results. The idea is to use intensity values recorded from multiple images obtained from the same viewpoint but under different conditions of illumination. The resulting radiometric constraint makes it possible to obtain local estimates of both surface orientation and surface curvature without requiring either global smoothness assumptions or prior image segmentation. Photometric stereo is moved one step closer to practical possibility by a description of an experimental setting in which surface gradient estimation is achieved on full-frame video data at near-video-frame rates (i.e., 15 Hz). The implementation uses commercially available hardware. Reflectance is modeled empirically with measurements obtained from a calibration sphere. Estimation of the gradient (p, q) requires only simple table lookup. Curvature estimation additionally uses the reflectance map R(p, q). The required lookup table and reflectance maps are derived during calibration. Because reflectance is modeled empirically, no prior physical model of the reflectance characteristics of the objects to be analyzed is assumed. At the same time, if a good physical model is available, it can be retrofitted to the method for implementation purposes. Photometric stereo is subject to error in the presence of cast shadows and interreflection. No purely local technique can succeed because these phenomena are inherently nonlocal. Nevertheless, it is demonstrated that one can exploit the redundancy in three-light-source photometric stereo to detect locally, in most cases, the presence of cast shadows and interreflection. Detection is facilitated by the explicit inclusion of a local confidence estimate in the lookup table used for gradient estimation.

ACME, A Telerobotic Active Measurement Facility

We are developing a robotic measurement facility which makes it very easy to build “reality-based” models, i.e., computational models of existing, physical objects based on actual measurements. These include not only models of shape, but also reflectance, contact forces, and sound. Such realistic models are crucial in many applications, including telerobotics, virtual reality, computer-assisted medicine, computer animation, computer games, and training simulators.

Acquisition of Elastic Models for Interactive Simulation

We present method and implementation to acquire deformable models of elastic objects. The method is based on the Greens functions matrix representation of an elastic solid. In this paper, we present a robust estimation technique for this Greens functions matrix. Robustness is achieved by regularization and a fitting technique which we describe here in detail. The underlying data for estimation are acquired with a robotic measurement system. We describe the UBC Active Measurement System (ACME) as it relates to the deformable model acquisition. In particular, we characterize the ability of ACME to record the global deformation of an object based on our previously presented range-flow technique. We provide results for two elastic objects: a plush toy and a medical physical soft-tissue wrist model.

An Analytic Method for Radiometric Correction of Satellite Multispectral Scanner Data

The problem of radiometric correction of multispectral scanner data is posed as the problem of determining an intrinsic reflectance factor characteristic of the surface material being imaged and invariant to topography, position of the sun, atmosphere, and position of the viewer. A scene radiance equation for remote sensing is derived based on an idealized physical model of image formation. The scene radiance equation is more complex for rugged terrain than for flat terrain since it must model slope-, aspect-, and elevation-dependent effects. Scene radiance is determined by the bidirectional reflectance distribution function (BRDF) of the surface material and the distribution of light sources. The sun is treated as a collimated source and the sky is treated as a uniform hemispherical source. The atmosphere is treated as an optically thin, horizontally uniform layer. The limits of this approach are reviewed using results obtained with Landsat MSS images and a digital terrain model (DTM) of a test site near St. Mary Lake, British Columbia, Canada. New results, based on regression analysis, are described for the St. Mary Lake site. Previous work is extended to take advantage of explicit forest cover data and to consider numeric models of sky radiance. The calculation of sky irradiance now takes occlusion by adjacent terrain into account. The results for St. Mary Lake suggest that the cosine of the incident solar angle and elevation are the two most important correction terms. Skylight and inter-reflection from adjacent terrain, however, also are significant.

3D Pose from Motion for Cross-View Action Recognition via Non-linear Circulant Temporal Encoding

We describe a new approach to transfer knowledge across views for action recognition by using examples from a large collection of unlabelled mocap data. We achieve this by directly matching purely motion based features from videos to mocap. Our approach recovers 3D pose sequences without performing any body part tracking. We use these matches to generate multiple motion projections and thus add view invariance to our action recognition model. We also introduce a closed form solution for approximate non-linear Circulant Temporal Encoding (nCTE), which allows us to efficiently perform the matches in the frequency domain. We test our approach on the challenging unsupervised modality of the IXMAS dataset, and use publicly available motion capture data for matching. Without any additional annotation effort, we are able to significantly outperform the current state of the art.

Determining surface curvature with photometric stereo

A method is described to compute dense representations of the intrinsic curvature at each point on a visible surface, based on photometric stereo. The idea of photometric stereo is to use the intensity values recorded from multiple images obtained from the same viewpoint but under different conditions of illumination. Previously, photometric stereo has been used to obtain local estimates of surface orientation. Here, an extension to photometric stereo is described in which the spatial derivatives of the intensity values are used to determine the principal curvatures and associated directions, at each point on a visible surface. The result shows that it is possible to obtain reliable local estimates of intrinsic surface curvature. The method is demonstrated using images of two pottery vases, one of which is used for calibration purposes.<<ETX>>

Principal components analysis and neural network implementation of photometric stereo

An implementation of photometric stereo is described in which all directions of illumination are close to the viewing direction. This has practical importance but creates a numerical problem that is ill-conditioned. Ill-conditioning is dealt with in two ways. First, many more than the theoretical minimum number of required images are acquired. Second, principal components analysis (PCA) is used as a linear preprocessing technique to extract a reduced dimensionality subspace to use as input. Overall, the approach is empirical. The ability of a radial basis function (RBF) neural network to do non-parametric functional approximation is exploited. One network maps image irradiance to surface normal. A second network maps surface normal to image irradiance. The two networks are trained using samples from a calibration sphere. Comparison between the actual input and the inversely predicted input is used as a confidence estimate. Results on real data are demonstrated

Using Digital Terrain Data To Model Image Formation In Remote Sensing

Computer-based image analysis requires explicit models of the image-forming process in order to deal with the effects of variations in viewing direction, incident illumination, surface slope and surface material. A fixed illumination, surface material and imaging geometry is incorporated into a single model, called a reflectance map, that allows observed brightness to be written as a function of surface orientation. The reflectance map is used to generate synthetic images from digital terrain models. Synthetic images are used to predict properties of real images. This technique is illustrated using Landsat imagery. Accurate shadow regions are determined from a digital terrain model by calculating which surface elements are visible from the light source. Once shadows are determined, the effect of sky illumination and atmospheric haze is estimated.