Vincent Gay-Bellile

Coarse-to-fine low-rank structure-from-motion

We address the problem of deformable shape and motion recovery from point correspondences in multiple perspective images. We use the low-rank shape model, i.e. the 3D shape is represented as a linear combination of unknown shape bases. We propose a new way of looking at the low-rank shape model. Instead of considering it as a whole, we assume a coarse-to-fine ordering of the deformation modes, which can be seen as a model prior. This has several advantages. First, the high level of ambiguity of the original low-rank shape model is drastically reduced since the shape bases can not anymore be arbitrarily re-combined. Second, this allows us to propose a coarse-to-fine reconstruction algorithm which starts by computing the mean shape and iteratively adds deformation modes. It directly gives the sought after metric model, thereby avoiding the difficult upgrading step required by most of the other methods. Third, this makes it possible to automatically select the number of deformation modes as the reconstruction algorithm proceeds. We propose to incorporate two other priors, accounting for temporal and spatial smoothness, which are shown to improve the quality of the recovered model parameters. The proposed model and reconstruction algorithm are successfully demonstrated on several videos and are shown to outperform the previously proposed algorithms.

NonLinear refinement of structure from motion reconstruction by taking advantage of a partial knowledge of the environment

We address the challenging issue of camera localization in a partially known environment, i.e. for which a geometric 3D model that covers only a part of the observed scene is available. When this scene is static, both known and unknown parts of the environment provide constraints on the camera motion. This paper proposes a nonlinear refinement process of an initial SfM reconstruction that takes advantage of these two types of constraints. Compare to those that exploit only the model constraints i.e. the known part of the scene, including the unknown part of the environment in the optimization process yields a faster, more accurate and robust refinement. It also presents a much larger convergence basin. This paper will demonstrate these statements on varied synthetic and real sequences for both 3D object tracking and outdoor localization applications.

Direct Estimation of Non-Rigid Registrations with Image-Based Self-Occlusion Reasoning

The registration problem for images of a deforming surface has been well studied. External occlusions are usually well-handled. In 2D image-based registration, self- occlusions are more challenging. Consequently, the surface is usually assumed to be only slightly self-occluding. This paper is about image-based non-rigid registration with self-occlusion reasoning. A specific framework explicitly modeling self-occlusions is proposed. It is combined with an intensity-based, i.e. direct, data term for registration. Self-occlusions are detected as shrinking areas in the 2D warp. Experimental results on several challenging datasets show that our approach successfully registers images with self-occlusions while effectively detecting the occluded regions.

Feature-Driven Direct Non-Rigid Image Registration

The direct registration problem for images of a deforming surface has been well studied. Parametric flexible warps based, for instance, on the Free-Form Deformation or a Radial Basis Function such as the Thin-Plate Spline, are often estimated using additive Gauss-Newton-like algorithms. The recently proposed compositional framework has been shown to be more efficient, but cannot be directly applied to such non-groupwise warps.Our main contribution in this paper is the Feature-Driven framework. It makes possible the use of compositional algorithms for most parametric warps such as those above mentioned. Two algorithms are proposed to demonstrate the relevance of our Feature-Driven framework: the Feature-Driven Inverse Compositional and the Feature-Driven Learning-based algorithms. As another contribution, a detailed derivation of the Feature-Driven warp parameterization is given for the Thin-Plate Spline and the Free-Form Deformation. We experimentally show that these two types of warps have a similar representational power. Experimental results show that our Feature-Driven registration algorithms are more efficient in terms of computational cost, without loss of accuracy, compared to existing methods.

An interactive Augmented Reality system: A prototype for industrial maintenance training applications

In this paper, we present an innovative Augmented Reality prototype designed for industrial education and training applications. The system uses an Optical See-Through HMD integrating a calibrated camera and a laser pointer to interactively augment an industrial object with virtual sequences designed to train a user for specific maintenance tasks. The training leverages user interactions by simply pointing on a specific object component. The architecture of our prototype involves two main vision-based modules : camera localization and user-interaction handling. The first module includes markerless trackers for camera localization, which can deal with partial occlusions and specular reflections on the metallic object surfaces. In the second module, we developed fast image processing methods for red laser dot tracking. By combining these processing elements, the proposed system is able to interactively augment in real time an industrial object making the learning process more interesting and intuitive.

Deformable Surface Augmentation in Spite of Self-Occlusions

The augmentation problem for images of a deforming surface has been studied since recently. The surface is usually assumed not to be self-occluding. Two dimensional deformation estimation in the presence of self-occlusions is very challenging. This paper proposes a specific framework explicitly modeling self-occlusions for augmented reality applications. The basic idea is to detect self-occlusions as warp shrinkage areas. Deformations are initially estimated via direct non-rigid image registration. Temporal smoothness is then used to refine the warps and the image are augmented. Experimental results on several challenging datasets show that our approach convincingly augments self-occluded surfaces. Associated videos are available on the first authors Web homepage.

Vehicle 6-DoF localization based on SLAM constrained by GPS and digital elevation model information

Vehicle geo-localization based on monocular visual Simultaneous Localization And Mapping (SLAM) remains a challenging issue mainly due to the accumulation errors and scale factor drift. To tackle these limitations, a common solution is to introduce geo-referenced information into the visual SLAM algorithm. In this paper, we propose two different bundle adjustment processes that merge both GPS measurements and “Digital Elevation Model” (DEM) data. Proposed solutions are devoted to ensure an accurate and robust geo-localization in both rural and urban environment. Experiments on synthetic and large scale real sequences show that, in addition to the real-time (i.e. about 30 Hz) performances, we obtain an accurate 6DoF localization.

Non-parametric depth calibration of a TOF camera

Time-of-Flight (TOF) cameras measure, in real-time, the distance between the camera and objects in the scene. This opens new perspectives in different application fields: 3D reconstruction, Augmented Reality, video-surveillance, etc. However, like any sensor, TOF cameras have limitations related to their technology. One of them is distance distortion. In this paper, we present a new depth calibration method (estimation of distance distortion) for TOF cameras. Our approach has several advantages. First, it is based on a non-parametric model, contrary to most of the other methods. Second, it models under the same formalism the distortion variation according to the distance and the pixel position in the image. This improves calibration accuracy even at the image boundaries which are typically more distorted than the image center. A comparison with two state of the art parametric methods is presented.

Generic edgelet-based tracking of 3D objects in real-time

This paper addresses the challenging issue of real-time camera localization relative to any object that have texture or not, sharp edges or occluding contours. 3D contour points, dynamically extracted from a CAD model by Analysis-by-Synthesis on the graphics hardware, are combined with a keyframe-based SLAM algorithm to estimate camera poses. Our tracking solution is accurate, robust to sudden motions and to occlusions, as demonstrated on synthetic and real data. This solution is also easy to deploy since it only uses an RGB camera and a CAD model of the object of interest, requires no manual intervention on this model and runs on a consumer tablet at a frequency of 40Hz on a HD video-stream. Videos are available as supplemental material.

Joint Reconstruction and Registration of a Deformable Planar Surface Observed by a 3D Sensor

We address the problem of reconstruction and registration of a deforming 3D surface observed by some 3D sensor giving a cloud of 3D points at each time instant. This problem is difficult since the basic data term does not provide enough constraints. We bring two main contributions. First, we examine a set of data and penalty terms that make the problem well-posed. The most important terms we introduce are the non- extensibility penalty and the attraction to boundary shape. Second, we show how the error function combining all these terms can be efficiently minimized with the Levenberg-Marquardt algorithm and sparse matrices. We report convincing results for challenging datasets coming from different kinds of 3D sensors. The algorithm is robust to missing and erroneous data points, and to spurious boundary detection.