Radu Horaud

Stereo correspondence through feature grouping and maximal cliques

The authors propose a method for solving the stereo correspondence problem. The method consists of extracting local image structures and matching similar such structures between two images. Linear edge segments are extracted from both the left and right images. Each segment is characterized by its position and orientation in the image as well as its relationships with the nearby segments. A relational graph is thus built from each image. For each segment in one image as set of potential assignments is represented as a set of nodes in a correspondence graph. Arcs in the graph represent compatible assignments established on the basis of segment relationships. Stereo matching becomes equivalent to searching for sets of mutually compatible nodes in this graph. Sets are found by looking for maximal cliques. The maximal clique best suited to represent a stereo correspondence is selected using a benefit function. Numerous results obtained with this method are shown. >

Surface feature detection and description with applications to mesh matching

In this paper we revisit local feature detectors/descriptors developed for 2D images and extend them to the more general framework of scalar fields defined on 2D manifolds. We provide methods and tools to detect and describe features on surfaces equiped with scalar functions, such as photometric information. This is motivated by the growing need for matching and tracking photometric surfaces over temporal sequences, due to recent advancements in multiple camera 3D reconstruction. We propose a 3D feature detector (MeshDOG) and a 3D feature descriptor (MeshHOG) for uniformly triangulated meshes, invariant to changes in rotation, translation, and scale. The descriptor is able to capture the local geometric and/or photometric properties in a succinct fashion. Moreover, the method is defined generically for any scalar function, e.g., local curvature. Results with matching rigid and non-rigid meshes demonstrate the interest of the proposed framework.

Figure-ground discrimination: a combinatorial optimization approach

The figure-ground discrimination problem is considered from a combinatorial optimization perspective. A mathematical model encoding the figure-ground discrimination problem that makes explicit a definition of shape based on cocircularity, smoothness, proximity, and contrast is presented. This model consists of building a cost function on the basis of image element interactions. This cost function fits the constraints of an interacting spin system that, in turn, is a well suited physical model that solves hard combinatorial optimization problems. Two combinatorial optimization methods for solving the figure-ground problem, namely mean field annealing, which combines mean field approximation theory and annealing, and microcanonical annealing, are discussed. Mean field annealing may be viewed as a deterministic approximation of stochastic methods such as simulated annealing. The theoretical bases of these methods are described, and the computational models are derived. The efficiencies of mean field annealing, simulated annealing, and microcanonical annealing algorithms are compared. Within the framework of such a comparison, the figure-ground problem may be viewed as a benchmark. >

An analytic solution for the perspective 4-point problem

The perspective n-point (PnP) problem is the problem of finding the position and orientation of a camera with respect to a scene object from n correspondence points. The authors propose an analytic solution for the perspective 4-point problem. The solution is found by replacing the four points with a pencil of three lines and by exploring the geometric constraints available with the perspective camera model. The P4P problem is cast into the problem of solving a biquadratic polynomial equation in one unknown.<<ETX>>

New Methods for Matching 3-D Objects with Single Perspective Views

In this paper we analyze the ability of a computer vision system to derive properties of the three-dimensional (3-D) physical world from viewing two-dimensional (2-D) images. We present a new approach which consists of a model-based interpretation of a single perspective image. Image linear features and linear feature sets are backprojected onto the 3-D space and geometric models are then used for selecting possible solutions. The paper treats two situations: 1) interpretation of scenes resulting from a simple geometric structure (orthogonality) in which case we seek to determine the orientation of this structure relatively to the viewer (three rotations) and 2) recognition of moderately complex objects whose shapes (geometrical and topological properties) are provided in advance. The recognition technique is limited to objects containing, among others, straight edges and planar faces. In the first case the computation can be carried out by a parallel algorithm which selects the solution that has received the largest number of votes (accumulation space). In the second case an object is uniquely assigned to a set of image features through a search strategy. As a by-product, the spatial position and orientation (six degrees of freedom) of each recognized object is determined as well. The method is valid over a wide range of perspective images and it does not require perfect low-level image segmentation. It has been successfully implemented for recognizing a class of industrial parts.

Articulated shape matching using Laplacian eigenfunctions and unsupervised point registration

Matching articulated shapes represented by voxel-sets reduces to maximal sub-graph isomorphism when each set is described by a weighted graph. Spectral graph theory can be used to map these graphs onto lower dimensional spaces and match shapes by aligning their embeddings in virtue of their invariance to change of pose. Classical graph isomorphism schemes relying on the ordering of the eigenvalues to align the eigenspaces fail when handling large data-sets or noisy data. We derive a new formulation that finds the best alignment between two congruent K-dimensional sets of points by selecting the best subset of eigenfunctions of the Laplacian matrix. The selection is done by matching eigenfunction signatures built with histograms, and the retained set provides a smart initialization for the alignment problem with a considerable impact on the overall performance. Dense shape matching casted into graph matching reduces then, to point registration of embeddings under orthogonal transformations; the registration is solved using the framework of unsupervised clustering and the EM algorithm. Maximal subset matching of non identical shapes is handled by defining an appropriate outlier class. Experimental results on challenging examples show how the algorithm naturally treats changes of topology, shape variations and different sampling densities.

Simultaneous robot-world and hand-eye calibration

Zhuang et al. (1994) proposed a method that allows simultaneous computation of the rigid transformations from world frame to robot base frame and from hand frame to camera frame. Their method attempts to solve a homogeneous matrix equation of the for, AX=ZB. They use quaternions to derive explicit linear solution for X and Z. In this paper, we present two new solutions that attempt to solve the homogeneous matrix equation mentioned above: 1) a closed-form method which uses quaternion algebra and a positive quadratic error function associated with this representation; 2) a method based on nonlinear constrained minimization and which simultaneously solves for rotations and translations. These results may be useful to other problems that can be formulated in the same mathematical form. We perform a sensitivity analysis for both our two methods and the linear method developed by Zhuang et al. This analysis allows the comparison of the three methods. In the light of this comparison, the nonlinear optimization method, which solves for rotations and translations simultaneously, seems to be the most stable one with respect to noise and to measurement errors.

Visually guided object grasping

We present a visual serving approach to the problem of object grasping and more generally, to the problem of aligning an end-effector with an object. First, we extend the method proposed by Espiau et al. (1992) to the case of a camera which is not mounted onto the robot being controlled, and we stress the importance of the real-time estimation of the image Jacobian. Next, we show how to represent a grasp or more generally, an alignment between two solids in 3D projective space using an uncalibrated stereo rig. Such a 3D projective representation is view-invariant in the sense that it can be easily mapped into an image set-point without any knowledge about the camera parameters. Finally, we perform an analysis of the performances of the visual servoing algorithm and of the grasping precision that can be expected from this type of approach.

Euclidean shape and motion from multiple perspective views by affine iterations

We describe a method for solving the Euclidean reconstruction problem with a perspective camera model by incrementally performing Euclidean reconstruction with either a weak or a paraperspective camera model. With respect to other methods that compute shape and motion from a sequence of images with a calibrated camera, this method converges in a few iterations, is computationally efficient, and solves for the sign (reversal) ambiguity. We give a detailed account of the method, analyze its convergence, and test it with both synthetic and real data.

Skeletal Quads: Human Action Recognition Using Joint Quadruples

Recent advances on human motion analysis have made the extraction of human skeleton structure feasible, even from single depth images. This structure has been proven quite informative for discriminating actions in a recognition scenario. In this context, we propose a local skeleton descriptor that encodes the relative position of joint quadruples. Such a coding implies a similarity normalisation transform that leads to a compact (6D) view-invariant skeletal feature, referred to as skeletal quad. Further, the use of a Fisher kernel representation is suggested to describe the skeletal quads contained in a (sub)action. A Gaussian mixture model is learnt from training data, so that the generation of any set of quads is encoded by its Fisher vector. Finally, a multi-level representation of Fisher vectors leads to an action description that roughly carries the order of sub-action within each action sequence. Efficient classification is here achieved by linear SVMs. The proposed action representation is tested on widely used datasets, MSRAction3D and HDM05. The experimental evaluation shows that the proposed method outperforms state-of-the-art algorithms that rely only on joints, while it competes with methods that combine joints with extra cues.