Sébastien Roy

A maximum-flow formulation of the N-camera stereo correspondence problem

This paper describes a new algorithm for solving the N-camera stereo correspondence problem by transforming it into a maximum-flow problem. Once solved, the minimum-cut associated to the maximum-flow yields a disparity surface for the whole image at once. This global approach to stereo analysis provides a more accurate and coherent depth map than the traditional line-by-line stereo. Moreover, the optimality of the depth surface is guaranteed and can be shown to be a generalization of the dynamic programming approach that is widely used in standard stereo. Results show improved depth estimation as well as better handling of depth discontinuities. While the worst case running time is O(n/sup 2/d/sup 2/log(nd)), the observed average running time is O(n/sup 1.2/ d/sup 1.3/) for an image size of n pixels and depth resolution d.

Stereo Without Epipolar Lines: A Maximum-Flow Formulation

This paper describes a new algorithm for solving the stereo correspondence problem with a global 2-d optimization by transforming it into a maximum-flow problem in a graph. This transformation effectively removes explicit use of epipolar geometry, thus allowing direct use of multiple cameras with arbitrary geometries. The maximum-flow, solved both efficiently and globally, yields a minimum-cut that corresponds to a disparity surface for the whole image at once. This global and efficient approach to stereo analysis allows the reconstruction to proceed in an arbitrary volume of space and provides a more accurate and coherent depth map than the traditional stereo algorithms. In particular, smoothness is applied uniformly instead of only along epipolar lines, while the global optimality of the depth surface is guaranteed. Results show improved depth estimation as well as better handling of depth discontinuities. While the worst case running time is O(n1.5 d1.5 log(nd)), the observed average running time is O(n1.2 d1.3) for an image size of n pixels and depth resolution d.

Dynamic histogram warping of image pairs for constant image brightness

The constant image brightness (CIB) assumption assumes that the intensities of corresponding points in two images are equal. This assumption is central to much of computer vision. However, surprisingly little work has been performed to support this assumption, despite the fact the many of algorithms are very sensitive to deviations from CIB. An examination of the images contained in the SRI JISCT stereo database revealed that the constant image brightness assumption is indeed often false. Moreover, the simple additive/multiplicative models of the form I/sub L/=/spl beta/I/sub R/+/spl alpha/ do not adequately represent the observed deviations. A comprehensive physical model of the observed deviations is difficult to develop. However, many potential sources of deviations can be represented by a nonlinear monotonically increasing relationship between intensities. Under these conditions, we believe that an expansion/contraction matching of the intensity histograms represents the best method to both measure the degree of validity of the CIB assumption and correct for it. Dynamic histogram warping (DHW) is closely related to histogram specification. It is shown that histogram specification introduces artifacts that do not occur with dynamic histogram warping. Experimental results show that image histograms are closely matched after DHW, especially when both histograms are modified simultaneously. DHW is also capable of removing simple constant additive and multiplicative biases without derivative operations, thereby avoiding amplification of high frequency noise. DHW can improve the estimates from stereo and optical flow estimators.

Cylindrical rectification to minimize epipolar distortion

We propose anew rectification method for aligning epipolar lines of a pair of stereo images taken under any camera geometry. It effectively remaps both images onto the surface of a cylinder instead of a plane, which is used in common rectification methods. For a large set of camera motions, remapping to a plane has the drawback of creating rectified images that are potentially infinitely large and presents a loss of pixel information along epipolar lines. In contrast, cylindrical rectification guarantees that the rectified images are bounded for all possible camera motions and minimizes the loss of pixel information along epipolar line. The processes (e.g., stereo matching, etc.) subsequently applied to the rectified images are thus more accurate and general since they can accommodate any camera geometry.

Motion without structure

We propose a new paradigm, motion without structure, for determining the ego-motion between two frames. It is best suited for cases where reliable feature point correspondence is difficult, or for cases where the expected camera motion is large. The problem is posed as a five-dimensional search over the space of possible motions during which the structural information present in the two views is neither implicitly or explicitly used or estimated. To accomplish this search, a cost function is devised that measures the relative likelihood of each hypothesized motion. This cost function is invariant to the structure present in the scene. An analysis of the global scene statistics present in an image, together with the geometry of epipolar misalignment, suggests a measure based on the sum of squared differences between pixels in the first image and their corresponding epipolar line segments in the second image. The measure relies on a simple statistical characteristic of neighboring image intensity levels. Specifically, that the variance of intensity differences between two arbitrary points in an image is a monotonically increasing symmetrical function of the distance between the two points. This assumption is almost always true, though the size of the neighborhood over which the monotonic dependency holds varies from image to image. This range determines the maximum permissible motion between two frames, which can be quite large. Experiments with both outdoor scenes and an indoor calibrated sequence achieve very good accuracy (less then 1 pixel image displacement error) and robustness to noise.

LightSphere: fast lighting compensation for matching a 2D image to a 3D model

We describe a fast object recognition method that identifies 2D color image queries among a set of 3D models. It is fast enough for searching a very large database. The main application is face recognition, for which we report very good accuracy over a wide range of pose and lighting conditions. We make weaker assumptions about both lighting and reflectance than are usual. We avoid finding eigenvectors or solving systems of equations. Instead, we use the query to estimate a specialization of the BRDF to the fixed lighting and pose of the query. In a single image pass, we compute a lookup table for re-rendering, which represents expectation values for the action of the light via the BRDF. This yields a similarity measure of the consistency between model and query under the regularity assumptions. We report recognition results on a data set of 42 3D face models and 1764 query images, comprising 7 poses and 6 lighting conditions. The recognition accuracy is indistinguishable from much slower methods, which make stronger assumptions about the BRDF and lighting.