Selim Benhimane

Homography-based 2D Visual Tracking and Servoing

The objective of this paper is to propose a new homography-based approach to image-based visual tracking and servoing. The visual tracking algorithm proposed in the paper is based on a new efficient second-order minimization method. Theoretical analysis and comparative experiments with other tracking approaches show that the proposed method has a higher convergence rate than standard first-order minimization techniques. Therefore, it is well adapted to real-time robotic applications. The output of the visual tracking is a homography linking the current and the reference image of a planar target. Using the homography, a task function isomorphic to the camera pose has been designed. A new image-based control law is proposed which does not need any measure of the 3D structure of the observed target (e.g. the normal to the plane). The theoretical proof of the existence of the isomorphism between the task function and the camera pose and the theoretical proof of the stability of the control law are provided. The experimental results, obtained with a 6 d.o.f. robot, show the advantages of the proposed method with respect to the existing approaches.

Real-time image-based tracking of planes using efficient second-order minimization

The tracking algorithm presented in this paper is based on minimizing the sum-of-squared-difference between a given template and the current image. Theoretically, amongst all standard minimization algorithms, the Newton method has the highest local convergence rate since it is based on a second-order Taylor series of the sum-of-squared-differences. However, the Newton method is time consuming since it needs the computation of the Hessian. In addition, if the Hessian is not positive definite, convergence problems can occur. That is why several methods use an approximation of the Hessian. The price to pay is the loss of the high convergence rate. The aim of this paper is to propose a tracking algorithm based on a second-order minimization method which does not need to compute the Hessian.

A dataset and evaluation methodology for template-based tracking algorithms

Unlike dense stereo, optical flow or multi-view stereo, template-based tracking lacks benchmark datasets allowing a fair comparison between state-of-the-art algorithms. Until now, in order to evaluate objectively and quantitatively the performance and the robustness of template-based tracking algorithms, mainly synthetically generated image sequences were used. The evaluation is therefore often intrinsically biased.

Vision-based Control for Car Platooning using Homography Decomposition

In this paper, we present a complete system for car platooning using visual tracking. The visual tracking is achieved by directly estimating the projective transformation (in our case a homography) between a selected reference template attached to the leading vehicle and the corresponding area in the current image. The relative position and orientation of the servoed car with regard to the leading one is computed by decomposing the homography. The control objective is stated in terms of path following task in order to cope with the non-holonomic constraints of the vehicles.

Efficient visual hull computation for real-time 3D reconstruction using CUDA

In this paper we present two efficient GPU-based visual hull computation algorithms. We compare them in terms of performance using image sets of varying size and different voxel resolutions. In addition, we present a real-time 3D reconstruction system which uses the proposed GPU-based reconstruction method to achieve real-time performance (30 fps) using 16 cameras and 4 PCs.

Efficient Homography-Based Tracking and 3-D Reconstruction for Single-Viewpoint Sensors

This paper addresses the problem of motion estimation and 3-D reconstruction through visual tracking with a single-viewpoint sensor and, in particular, how to generalize tracking to calibrated omnidirectional cameras. We analyze different minimization approaches for the intensity-based cost function (sum of squared differences). In particular, we propose novel variants of the efficient second-order minimization (ESM) with better computational complexities and compare these algorithms with the inverse composition (IC) and the hyperplane approximation (HA). Issues regarding the use of the IC and HA for 3-D tracking are discussed. We show that even though an iteration of ESM is computationally more expensive than an iteration of IC, the faster convergence rate makes it globally faster. The tracking algorithm was validated by using an omnidirectional sensor mounted on a mobile robot.

Homography-based 2D visual servoing

The objective of this paper is to propose a new homography-based approach to image-based visual servoing. The visual servoing method does not need any measure of the 3D structure of the observed target. Only visual information measured from the reference and the current image are needed to compute the task function (isomorphic to the camera pose) and the control law to be applied to the robot. The control law is designed in order to make the task function converge to zero. We provide the theoretical proof of the existence of the isomorphism between the task function and the camera pose and the theoretical proof of the local stability of the control law. The experimental results, obtained with a 6 d.o.f. robot, show the advantages of the proposed method with respect to the existing approaches

Gravity-aware handheld Augmented Reality

This paper investigates how different stages in handheld Augmented Reality (AR) applications can benefit from knowing the direction of the gravity measured with inertial sensors. It presents approaches to improve the description and matching of feature points, detection and tracking of planar templates, and the visual quality of the rendering of virtual 3D objects by incorporating the gravity vector. In handheld AR, both the camera and the display are located in the users hand and therefore can be freely moved. The pose of the camera is generally determined with respect to piecewise planar objects that have a known static orientation with respect to gravity. In the presence of (close to) vertical surfaces, we show how gravity-aligned feature descriptors (GAFD) improve the initialization of tracking algorithms relying on feature point descriptor-based approaches in terms of quality and performance. For (close to) horizontal surfaces, we propose to use the gravity vector to rectify the camera image and detect and describe features in the rectified image. The resulting gravity-rectified feature descriptors (GREFD) provide an improved precision-recall characteristic and enable faster initialization, in particular under steep viewing angles. Gravity-rectified camera images also allow for real-time 6 DoF pose estimation using an edge-based object detection algorithm handling only 4 DoF similarity transforms. Finally, the rendering of virtual 3D objects can be made more realistic and plausible by taking into account the orientation of the gravitational force in addition to the relative pose between the handheld device and a real object.

Online learning of patch perspective rectification for efficient object detection

For a large class of applications, there is time to train the system. In this paper, we propose a learning-based approach to patch perspective rectification, and show that it is both faster and more reliable than state-of-the-art ad hoc affine region detection methods. Our method performs in three steps. First, a classifier provides for every keypoint not only its identity, but also a first estimate of its transformation. This estimate allows carrying out, in the second step, an accurate perspective rectification using linear predictors. We show that both the classifier and the linear predictors can be trained online, which makes the approach convenient. The last step is a fast verification - made possible by the accurate perspective rectification - of the patch identity and its sub-pixel precision position estimation. We test our approach on real-time 3D object detection and tracking applications. We show that we can use the estimated perspective rectifications to determine the object pose and as a result, we need much fewer correspondences to obtain a precise pose estimation.

A unified approach to visual tracking and servoing

Abstract In this paper, we present a generic and flexible system for vision-based robot control. The system integrates visual tracking and visual servoing approaches in a unifying framework. In this framework, the generality is obtained using a template matching algorithm based on an efficient second-order minimization. Contrarily to feature-based visual servoing schemes, we avoid the design of feature-dependent visual tracking algorithms. By integrating the visual tracking process with the visual servoing techniques, we can easily deal with constrained tasks. This reduces the computation cost and improves the precision of the system. The experimental results prove the efficiency of the unified system in real conditions.