Davi Geiger

Dynamic programming for detecting, tracking, and matching deformable contours

The problem of segmenting an image into separate regions and tracking them over time is one of the most significant problems in vision. Terzopoulos et al. (1987) proposed an approach to detect the contour regions of complex shapes, assuming a user selected initial contour not very far from the desired solution. We propose to further explore the information provided by the users selected points and apply an optimal method to detect contours which allows a segmentation of the image. The method is based on dynamic programming (DP), and applies to a wide variety of shapes. It is exact and not iterative. We also consider a multiscale approach capable of speeding up the algorithm by a factor of 20, although at the expense of losing the guaranteed optimality characteristic. The problem of tracking and matching these contours is addressed. For tracking, the final contour obtained at one frame is sampled and used as initial points for the next frame. Then, the same DP process is applied. For matching, a novel strategy is proposed where the solution is a smooth displacement field in which unmatched regions are allowed while cross vectors are not. The algorithm is again based on DP and the optimal solution is guaranteed. We have demonstrated the algorithms on natural objects in a large spectrum of applications, including interactive segmentation and automatic tracking of the regions of interest in medical images. >

Parallel and deterministic algorithms from MRFs: surface reconstruction

Deterministic approximations to Markov random field (MRF) models are derived. One of the models is shown to give in a natural way the graduated nonconvexity (GNC) algorithm proposed by A. Blake and A. Zisserman (1987). This model can be applied to smooth a field preserving its discontinuities. A class of more complex models is then proposed in order to deal with a variety of vision problems. All the theoretical results are obtained in the framework of statistical mechanics and mean field techniques. A parallel, iterative algorithm to solve the deterministic equations of the two models is presented, together with some experiments on synthetic and real images. >

Occlusions and binocular stereo

Binocular stereo is the process of obtaining depth information from a pair of cameras. In the past, stereo algorithms have had problems at occlusions and have tended to fail there (though sometimes post-processing has been added to mitigate the worst effects). We show that, on the contrary, occlusions can help stereo computation by providing cues for depth discontinuities.We describe a theory for stereo based on the Bayesian approach, using adaptive windows and a prior weak smoothness constraint, which incorporates occlusion. Our model assumes that a disparity discontinuity, along the epipolar line, in one eyealways corresponds to an occluded region in the other eye thus, leading to anocclusion constraint. This constraint restricts the space of possible disparity values, thereby simplifying the computations. An estimation of the disparity at occluded features is also discussed in light of psychophysical experiments. Using dynamic programming we can find the optimal solution to our system and the experimental results are good and support the assumptions made by the model.

Determining the similarity of deformable shapes

Determining the similarity of two shapes is a significant task in both machine and human vision systems that must recognize or classify objects. The exact properties of human shape similarity judgements are not well understood yet, and this task is particularly difficult in domains where the shapes are not related by rigid transformation. In this paper we identify a number of possibly desirable properties of a shape similarity method, and determine the extent to which these properties can be captured by approaches that compare local properties of the contours of the shapes, through elastic matching. Special attention is devoted to objects that possess articulations, i.e. articulated parts. Elastic matching evaluates the similarity of two shapes as the sum of local deformations needed to change one shape into another. We show that similarities of part structure can be captured by such an approach, without the explicit computation of part structure. This may be of importance, since although parts appear to play a significant role in visual recognition, it is difficult to stably determine part structure. We also show novel results about how one can evaluate smooth and polyhedral shapes with the same method. Finally, we describe shape similarity effects that cannot be handled by current approaches.

A common framework for image segmentation

We attempt to unify several approaches to image segmentation in early vision under a common framework. The Bayesian approach is very attractive since: (i) it enables the assumptions used to be explicitly stated in the probability distributions, and (ii) it can be extended to deal with most other problems in early vision. Here, we consider the Markov random field formalism, a special case of the Bayesian approach, in which the probability distributions are specified by an energy function.We show that: (i) our discrete formulations for the energy function is closely related to the continuous formulation; (ii) by using the mean field (MF) theory approach, introduced by Geiger and Girosi [1991], several previous attempts to solve these energy functions are effectively equivalent; (iii) by varying the parameters of the energy functions we can obtain connections to nonlinear diffusion and minimal description length approaches to image segmentation; and (iv) simple modifications to the energy can give a direct relation to robust statistics or can encourage hysteresis and nonmaximum suppression.

Occlusions, Discontinuities, and Epipolar Lines in Stereo

Binocular stereo is the process of obtaining depth information from a pair of left and right views of a scene. We present a new approach to compute the disparity map by solving a global optimization problem that models occlusions, discontinuities, and epipolar-line interactions.

Approximate tree matching and shape similarity

We present a framework for 2D shape contour (silhouette) comparison that can account for stretchings, occlusions and region information. Topological changes due to the original 3D scenarios and articulations are also addressed. To compare the degree of similarity between any two shapes, our approach is to represent each shape contour with a free tree structure derived from a shape axis (SA) model, which we have recently proposed. We then use a tree matching scheme to find the best approximate match and the matching cost. To deal with articulations, stretchings and occlusions, three local tree matching operations, merge, cut, and merge-and-cut, are introduced to yield optimally approximate matches, which can accommodate not only one-to-one but many-to-many mappings. The optimization process gives guaranteed globally optimal match efficiently. Experimental results on a variety of shape contours are provided.

Junctions: detection, classification, and reconstruction

Junctions are important features for image analysis and form a critical aspect of image understanding tasks such as object recognition. We present a unified approach to detecting, classifying, and reconstructing junctions in images. Our main contribution is a modeling of the junction which is complex enough to handle all these issues and yet simple enough to admit an effective dynamic programming solution. We use a template deformation framework along with a gradient criterium to detect radial partitions of the template. We use the minimum description length principle to obtain the optimal number of partitions that best describes the junction. The Kona detector presented by Parida et al. (1997) is an implementation of this model. We demonstrate the stability and robustness of the detector by analyzing its behavior in the presence of noise, using synthetic/controlled apparatus. We also present a qualitative study of its behavior on real images.

Representation and self-similarity of shapes

Representing shapes in a compact and informative form is a significant problem for vision systems that must recognize or classify objects. We describe a compact representation model for two-dimensional (2D) shapes by investigating their self-similarities and constructing their shape axis trees (SA-trees). Our approach can be formulated as a variational one (or, equivalently, as MAP estimation of a Markov random field). We start with a 2D shape, its boundary contour, and two different parameterizations for the contour (one parameterization is oriented counterclockwise and the other clockwise). To measure its self-similarity, the two parameterizations are matched to derive the best set of one-to-one point-to-point correspondences along the contour. The cost functional used in the matching may vary and is determined by the adopted self-similarity criteria, e.g., cocircularity, distance variation, parallelism, and region homogeneity. The loci of middle points of the pairing contour points yield the shape axis and they can be grouped into a unique free tree structure, the SA-tree. By implicitly encoding the (local and global) shape information into an SA-tree, a variety of vision tasks, e.g., shape recognition, comparison, and retrieval, can be performed in a more robust and efficient way via various tree-based algorithms. A dynamic programming algorithm gives the optimal solution in O(N/sup 1/), where N is the size of the contour.

Autonomous robot calibration for Hand-eye coordination

Autonomous robot calibration is defined as the process of determining a robots model by using only its internal sen sors. It is shown that autonomous calibration of a manip ulator and stereo camera system is possible. The pro posed autonomous calibration algorithm may obtain the manipulator kinematic parameters, external kinematic camera parameters, and internal camera parameters. To do this, only joint angle readings and camera image plane data are used. A condition for the identifiability of the manipulator/camera parameters is derived. The method is a generalization of a recently developed scheme for self- calibrating a manipulator by forming it into a mobile closed-loop kinematic chain.