Bertrand Douillard

On the segmentation of 3D LIDAR point clouds

This paper presents a set of segmentation methods for various types of 3D point clouds. Segmentation of dense 3D data (e.g. Riegl scans) is optimised via a simple yet efficient voxelisation of the space. Prior ground extraction is empirically shown to significantly improve segmentation performance. Segmentation of sparse 3D data (e.g. Velodyne scans) is addressed using ground models of non-constant resolution either providing a continuous probabilistic surface or a terrain mesh built from the structure of a range image, both representations providing close to real-time performance. All the algorithms are tested on several hand labeled data sets using two novel metrics for segmentation evaluation.

Classification and Semantic Mapping of Urban Environments

In this paper we address the problem of classifying objects in urban environments based on laser and vision data. We propose a framework based on Conditional Random Fields (CRFs), a flexible modeling tool allowing spatial and temporal correlations between laser returns to be represented. Visual features extracted from color imagery as well as shape features extracted from 2D laser scans are integrated in the estimation process. The paper contains the following novel developments: (1) a probabilistic formulation for the problem of exploiting spatial and temporal dependencies to improve classification; (2) three methods for classification in 2D semantic maps; (3) a novel semi-supervised learning algorithm to train CRFs from partially labeled data; (4) the combination of local classifiers with CRFs to perform feature selection on high-dimensional feature vectors. The system is extensively evaluated on two different datasets acquired in two different cities with different sensors. An accuracy of 91% is achieved on a seven-class problem. The classifier is also applied to the generation of a 3 km long semantic map.

Laser and Vision Based Outdoor Object Mapping

Generating rich representations of environments can significantly improve the autonomy of mobile robotics. In this paper we introduce a novel approach to building object-type maps of outdoor environments. Our approach uses conditional random fields (CRF) to jointly classify laser returns in a 2D scan map into seven object types (car, wall, tree trunk, foliage, person, grass, and other). The spatial connectivity of the CRF model is determined via Delaunay triangulation of the laser map. Our model incorporates laser shape features, visual appearance features, structural information extracted from clusters of laser returns, and visual object detectors trained on image data sets available on the internet. The parameters of the CRF are trained from partially labeled laser and camera data collected by a car moving through an urban environment. Our approach achieves 91% accuracy in classifying objects observed along a 3 kilometer trajectory.

A Pipeline for the Segmentation and Classification of 3D Point Clouds

This paper presents algorithms for fast segmentation of 3D point clouds and subsequent classification of the obtained 3D segments. The method jointly determines the ground surface and segments individual objects in 3D, including overhanging structures. When compared to six other terrain modelling techniques, this approach has minimal error between the sensed data and the representation; and is fast (processing a Velodyne scan in approximately 2 seconds). Applications include improved alignment of successive scans by enabling operations in sections (Velodyne scans are aligned 7% sharper compared to an approach using raw points) and more informed decision-making (paths move around overhangs). The use of segmentation to aid classification through 3D features, such as the Spin Image or the Spherical Harmonic Descriptor, is discussed and experimentally compared. Moreover, the segmentation facilitates a novel approach to 3D classification that bypasses feature extraction and directly compares 3D shapes via the ICP algorithm. This technique is shown to achieve accuracy on par with the best feature based classifier (92.1%) while being significantly faster and allowing a clearer understanding of the classifier’s behaviour.

Scan segments matching for pairwise 3D alignment

This paper presents a method for pairwise 3D alignment which solves data association by matching scan segments across scans. Generating accurate segment associations allows to run a modified version of the Iterative Closest Point (ICP) algorithm where the search for point-to-point correspondences is constrained to associated segments. The novelty of the proposed approach is in the segment matching process which takes into account the proximity of segments, their shape, and the consistency of their relative locations in each scan. Scan segmentation is here assumed to be given (recent studies provide various alternatives [10], [19]). The method is tested on seven sequences of Velodyne scans acquired in urban environments. Unlike various other standard versions of ICP, which fail to recover correct alignment when the displacement between scans increases, the proposed method is shown to be robust to displacements of several meters. In addition, it is shown to lead to savings in computational times which are potentially critical in real-time applications.

Hybrid elevation maps: 3D surface models for segmentation

This paper presents an algorithm for segmenting 3D point clouds. It extends terrain elevation models by incorporating two types of representations: (1) ground representations based on averaging the height in the point cloud, (2) object models based on a voxelisation of the point cloud. The approach is deployed on Riegl data (dense 3D laser data) acquired in a campus type of environment and compared against six other terrain models. Amongst elevation models, it is shown to provide the best fit to the data as well as being unique in the sense that it jointly performs ground extraction, overhang representation and 3D segmentation. We experimentally demonstrate that the resulting model is also applicable to path planning.

A 3D laser and vision based classifier

This paper presents a method for modelling semantic content in scenes, in order to facilitate urban driving. More specifically, it presents a 3D classifier based on Velodyne data and monocular color imagery. The system contains two main components: a ground model and an object model. The ground model is a novel extension of elevation maps using Conditional Random Fields. It allows estimation of ground type (grass vs. asphalt) in addition to modelling the geometry of the scene. The object model involves two segmentation procedures. The first is a novel extension of elevation maps to a hierarchical clustering algorithm. The second is a new algorithm for defining regions of interest in images, which reasons jointly in the 3D Cartesian frame and the image plane. These two procedures provide a segmentation of the objects in the 3D laser data and in the images. Based on the resulting segmentation, object classification is implemented using a rule based system to combine binary deterministic and probabilistic features. The overall 3D classifier is tested on logs acquired by the MIT Urban Grand Challenge 2007 vehicle. The classifier achieves an accuracy of 89% on a set of 500 scenes involving 16 classes. The proposed approach is evaluated against seven other standard classification algorithms, and is shown to produce superior performance.

A spatio-temporal probabilistic model for multi-sensor object recognition

This paper presents a general framework for multi-sensor object recognition through a discriminative probabilistic approach modelling spatial and temporal correlations. The algorithm is developed in the context of Conditional Random Fields (CRFs) trained with virtual evidence boosting. The resulting system is able to integrate arbitrary sensor information and incorporate features extracted from the data. The spatial relationships captured by are further integrated into a smoothing algorithm to improve recognition over time. We demonstrate the benefits of modelling spatial and temporal relationships for the problem of detecting cars using laser and vision data in outdoor environments.

Visual ground segmentation by radar supervision

Abstract Imaging sensors are being increasingly used in autonomous vehicle applications for scene understanding. This paper presents a method that combines radar and monocular vision for ground modeling and scene segmentation by a mobile robot operating in outdoor environments. The proposed system features two main phases: a radar-supervised training phase and a visual classification phase. The training stage relies on radar measurements to drive the selection of ground patches in the camera images, and learn online the visual appearance of the ground. In the classification stage, the visual model of the ground can be used to perform high level tasks such as image segmentation and terrain classification, as well as to solve radar ambiguities. This method leads to the following main advantages: (a) self-supervised training of the visual classifier across the portion of the environment where radar overlaps with the camera field of view. This avoids time-consuming manual labeling and enables on-line implementation; (b) the ground model can be continuously updated during the operation of the vehicle, thus making feasible the use of the system in long range and long duration applications. This paper details the algorithms and presents experimental tests conducted in the field using an unmanned vehicle.

A Spatio-Temporal Probabilistic Model for Multi-Sensor Multi-Class Object Recognition

This paper presents a general probabilistic framework for multi-sensor multi-class object recognition based on Conditional Random Fields (CRFs) trained with virtual evidence boosting. The learnt representation models spatial and temporal relationships and is able to integrate arbitrary sensor information by automatically extracting features from data. We demonstrate the benefits of modelling spatial and temporal relationships for the problem of detecting seven classes of objects using laser and vision data in outdoor environments. Additionally, we show how this framework can be used with partially labeled data, thereby significantly reducing the burden of manual data annotation.