Steve Bourgeois

Towards geographical referencing of monocular SLAM reconstruction using 3D city models: Application to real-time accurate vision-based localization

In the past few years, lots of works were achieved on Simultaneous Localization and Mapping (SLAM). It is now possible to follow in real time the trajectory of a moving camera in an unknown environment. However, current SLAM methods are still prone to drift errors, which prevent their use in large-scale applications. In this paper, we propose a solution to reduce those errors a posteriori. Our solution is based on a postprocessing algorithm that exploits additional geometric constraints, relative to the environment, to correct both the reconstructed geometry and the camera trajectory. These geometric constraints are obtained through a coarse 3D modelisation of the environment, similar to those provided by GIS database. First, we propose an original articulated transformation model in order to roughly align the SLAM reconstruction with this 3D model through a non-rigid ICP step. Then, to refine the reconstruction, we introduce a new bundle adjustment cost function that includes, in a single term, the usual 3D point/ID observation consistency constraint as well as the geometric constraints provided by the 3D model. Results on large-scale synthetic and real sequences show that our method successfully improves SLAM reconstructions. Besides, experiments prove that the resulting reconstruction is accurate enough to be directly used for global relocalization applications.

NonLinear refinement of structure from motion reconstruction by taking advantage of a partial knowledge of the environment

We address the challenging issue of camera localization in a partially known environment, i.e. for which a geometric 3D model that covers only a part of the observed scene is available. When this scene is static, both known and unknown parts of the environment provide constraints on the camera motion. This paper proposes a nonlinear refinement process of an initial SfM reconstruction that takes advantage of these two types of constraints. Compare to those that exploit only the model constraints i.e. the known part of the scene, including the unknown part of the environment in the optimization process yields a faster, more accurate and robust refinement. It also presents a much larger convergence basin. This paper will demonstrate these statements on varied synthetic and real sequences for both 3D object tracking and outdoor localization applications.

Real-time vehicle global localisation with a single camera in dense urban areas: Exploitation of coarse 3D city models

In this system paper, we propose a real-time car localisation process in dense urban areas by using a single perspective camera and a priori on the environment. To tackle this problem, it is necessary to solve two well-known monocular SLAM limitations: scale factor drift and error accumulation. The proposed idea is to combine a monocular SLAM process based on bundle adjustment with simple knowledge, i.e. the position and orientation of the camera with regard to the road and a coarse 3D model of the environment, as those provided by GIS database. First, we show that, thanks to specific SLAM-based constraints, the road homography can be expressed only with respect to the scale factor parameter. This allows the scale factor to be robustly and frequently estimated. Then, we propose to use the global information brought by 3D city models in order to correct the monocular SLAM error accumulation. Even with coarse 3D models, turnings give enough geometrical constraints to allow fitting the reconstructed 3D point cloud with the 3D model. Experiments on large-scale sequences (several kilometres) show that the entire process permits the real-time localisation of a car in city centre, even in real traffic condition.

An interactive Augmented Reality system: A prototype for industrial maintenance training applications

In this paper, we present an innovative Augmented Reality prototype designed for industrial education and training applications. The system uses an Optical See-Through HMD integrating a calibrated camera and a laser pointer to interactively augment an industrial object with virtual sequences designed to train a user for specific maintenance tasks. The training leverages user interactions by simply pointing on a specific object component. The architecture of our prototype involves two main vision-based modules : camera localization and user-interaction handling. The first module includes markerless trackers for camera localization, which can deal with partial occlusions and specular reflections on the metallic object surfaces. In the second module, we developed fast image processing methods for red laser dot tracking. By combining these processing elements, the proposed system is able to interactively augment in real time an industrial object making the learning process more interesting and intuitive.

Vehicle 6-DoF localization based on SLAM constrained by GPS and digital elevation model information

Vehicle geo-localization based on monocular visual Simultaneous Localization And Mapping (SLAM) remains a challenging issue mainly due to the accumulation errors and scale factor drift. To tackle these limitations, a common solution is to introduce geo-referenced information into the visual SLAM algorithm. In this paper, we propose two different bundle adjustment processes that merge both GPS measurements and “Digital Elevation Model” (DEM) data. Proposed solutions are devoted to ensure an accurate and robust geo-localization in both rural and urban environment. Experiments on synthetic and large scale real sequences show that, in addition to the real-time (i.e. about 30 Hz) performances, we obtain an accurate 6DoF localization.

Vision-Based Differential GPS: Improving VSLAM / GPS Fusion in Urban Environment with 3D Building Models

We improve in this paper the localization accuracy of visual SLAM (VSLAM) / GPS fusion in dense urban area by using 3D building models provided by Geographic Information System (GIS). GPS inaccuracies are corrected by comparison of the reconstruction resulting from the VSLAM / GPS fusion with 3D building models. These corrected GPS data are thereafter re-injected in the fusion process. Experimental results demonstrate the accuracy improvements achieved through our proposed solution.

Generic edgelet-based tracking of 3D objects in real-time

This paper addresses the challenging issue of real-time camera localization relative to any object that have texture or not, sharp edges or occluding contours. 3D contour points, dynamically extracted from a CAD model by Analysis-by-Synthesis on the graphics hardware, are combined with a keyframe-based SLAM algorithm to estimate camera poses. Our tracking solution is accurate, robust to sudden motions and to occlusions, as demonstrated on synthetic and real data. This solution is also easy to deploy since it only uses an RGB camera and a CAD model of the object of interest, requires no manual intervention on this model and runs on a consumer tablet at a frequency of 40Hz on a HD video-stream. Videos are available as supplemental material.

Noise modelling in time-of-flight sensors with application to depth noise removal and uncertainty estimation in three-dimensional measurement

Time-of-flight (TOF) sensors provide real-time depth information at high frame-rates. One issue with TOF sensors is the usual high level of noise (i.e. the depth measures repeatability within a static setting). However, until now, TOF sensors’ noise has not been well studied. The authors show that the commonly agreed hypothesis that noise depends only on the amplitude information is not valid in practice. They empirically establish that the noise follows a signal-dependent Gaussian distribution and varies according to pixel position, depth and integration time. They thus consider all these factors to model noise in two new noise models. Both models are evaluated, compared and used in the two following applications: depth noise removal by depth filtering and uncertainty (repeatability) estimation in three-dimensional measurement.

Fast and automatic city-scale environment modeling for an accurate 6DOF vehicle localization

To provide high quality augmented reality service in a car navigation system, accurate 6DoF localization is required. To ensure such accuracy, most of current vision-based solutions rely on an off-line large scale modeling of the environment. Nevertheless, while existing solutions require expensive equipments and/or a prohibitive computation time, we propose in this paper a complete framework that automatically builds an accurate city scale database using only a standard camera, a GPS and Geographic Information System (GIS). As illustrated in the experiments, only few minutes are required to model large scale environments. The resulting databases can then be used during a localization algorithm for high quality Augmented Reality experiences.

Augmented reality in large environments: Application to aided navigation in urban context

This paper addresses the challenging issue of vision-based localization in urban context. It briefly describes our contributions in large environments modeling and accurate camera localization. The efficiency of the resulting system is illustrated through Augmented Reality results on large trajectory of several hundred meters.