Vincent Fremont

Extrinsic calibration between a multi-layer lidar and a camera

In this paper, we present a novel approach for solving the extrinsic calibration between a camera and a multi-layer laser range finder. Our approach is oriented for intelligent vehicle applications, where the separation distance between sensors frames are frequently very important. For this purpose, we use a circle-based calibration object because its geometry allows us to obtain not only an accurate estimation pose by taking advantage of the 3D multi-layer laser range finder perception but also a simultaneous estimation of the pose in the camera frame and the camera intrinsic parameters. These advantages simplify the calibration task in outdoor environments. The method determines the relative position of the sensors by estimating sets of corresponded features and by solving the classical absolute orientation problem. The proposed method is evaluated by using different synthetics environments and real data. An error propagation analysis is made in order to estimate the calibration accuracy and the confidence intervals. Finally, we present a laser data projection into images to validate the consistency of the results.

UAV altitude estimation by mixed stereoscopic vision

Altitude is one of the most important parameters to be known for an Unmanned Aerial Vehicle (UAV) especially during critical maneuvers such as landing or steady flight. In this paper, we present mixed stereoscopic vision system made of a fish-eye camera and a perspective camera for altitude estimation. Contrary to classical stereoscopic systems based on feature matching, we propose a plane sweeping approach in order to estimate the altitude and consequently to detect the ground plane. Since there exists a homography between the two views and the sensor being calibrated and the attitude estimated by the fish-eye camera, the algorithm consists then in searching the altitude which verifies this homography. We show that this approach is robust and accurate, and a CPU implementation allows a real time estimation. Experimental results on real sequences of a small UAV demonstrate the effectiveness of the approach.

An experiment of a 3D real-time robust visual odometry for intelligent vehicles

Vision systems are nowadays very promising for many on-board vehicles perception functionalities, like obstacles detection/recognition and ego-localization. In this paper, we present a 3D visual odometric method that uses a stereo-vision system to estimate the 3D ego-motion of a vehicle in outdoor road conditions. In order to run in real-time, the studied technique is sparse meaning that it makes use of feature points that are tracked during several frames. A robust scheme is also employed to reject outliers that are detected on moving objects of the environment. Moreover, efforts have been spent on the realtime implementation of the method. In this article, we describe the key stages of the method: features extraction and tracking, quadrifocal constraints, optimization solver and robustification. Real experiments are reported to compare the performance of this approach with GPS data and 2D-wheel-based odometry.

Lane marking aided vehicle localization

A localization system that exploits L1-GPS estimates, vehicle data, and features from a video camera as well as lane markings embedded in digital navigation maps is presented. A sensitivity analysis of the detected lane markings is proposed in order to quantify both the lateral and longitudinal errors caused by 2D-world hypothesis violation. From this, a camera observation model for vehicle localization is proposed. The paper presents also a method to build a map of the lane markings in a first stage. The solver is based on dynamical Kalman filtering with a two-stage map-matching process which is described in details. This is a software-based solution using existing automotive components. Experimental results in urban conditions demonstrate an significant increase in the positioning quality.

Color-Based Road Detection and Its Evaluation on the KITTI Road Benchmark

Road detection is one of the key issues of scene understanding for Advanced Driving Assistance Systems (ADAS). Recent approaches has addressed this issue through the use of different kinds of sensors, features and algorithms. KITTI-ROAD benchmark has provided an open-access dataset and standard evaluation mean for road area detection. In this paper, we propose an improved road detection algorithm that provides a pixel-level confidence map. The proposed approach is inspired from our former work based on road feature extraction using illuminant intrinsic image and plane extraction from v-disparity map segmentation. In the former research, detection results of road area are represented by binary map. The novelty of this improved algorithm is to introduce likelihood theory to build a confidence map of road detection. Such a strategy copes better with ambiguous environments, compared to a simple binary map. Evaluations and comparisons of both, binary map and confidence map, have been done using the KITTI-ROAD benchmark.

Automatic Camera-Based Microscope Calibration for a Telemicromanipulation System Using a Virtual Pattern

In the context of virtualized-reality-based telemicromanipulation, this paper presents a visual calibration technique for an optical microscope coupled to a charge-coupled device (CCD) camera. The accuracy and flexibility of the proposed automatic virtual calibration method, based on parallel single-plane properties, are outlined. In contrast to standard approaches, a 3-D virtual calibration pattern is constructed using the micromanipulator tip with subpixel-order localization in the image frame. The proposed procedure leads to a linear system whose solution provides directly both the intrinsic and extrinsic parameters of the geometrical model. Computer simulations and real data have been used to test the proposed technique, and promising results have been obtained. Based on the proposed calibration techniques, a 3-D virtual microenvironment of the workspace is reconstructed through the real-time imaging of two perpendicular optical microscopes. Our method provides a flexible, easy-to-use technical alternative to the classical techniques used in micromanipulation systems.

Exploiting fully convolutional neural networks for fast road detection

Road detection is a crucial task in autonomous navigation systems. It is responsible for delimiting the road area and hence the free and valid space for maneuvers. In this paper, we consider the visual road detection problem where, given an image, the objective is to classify every of its pixels into road or non-road. We address this task by proposing a convolutional neural network architecture. We are especially interested in a model that takes advantage of a large contextual window while maintaining a fast inference. We achieve this by using a Network-in-Network (NiN) architecture and by converting the model into a fully convolutional network after training. Experiments have been conducted to evaluate the effects of different contextual window sizes (the amount of contextual information) and also to evaluate the NiN aspect of the proposed architecture. Finally, we evaluated our approach using the KITTI road detection benchmark achieving results in line with other state-of-the-art methods while maintaining real-time inference. The benchmark results also reveal that the inference time of our approach is unique at this level of accuracy, being two orders of magnitude faster than other methods with similar performance.

Turntable-based 3D object reconstruction

In this paper, we present a system that can acquire graphical models from real objects. Given an image sequence of a complex shape object placed on a turntable, the presented algorithm generates automatically the 3D model. In contrast to previous approaches, the technique described here is only based on conies properties and uses the spatiotemporal aspect of the sequence of images. From the projective properties of the conies and using the camera calibration parameters the Euclidean 3D coordinates of a point are obtained from the geometric locus of the image points trajectories. An algorithm has been implemented to compute the 3D reconstruction automatically. Examples on both synthetic and real image sequences are presented

Flexible Microscope Calibration using Virtual Pattern for 3-D Telemicromanipulation

In the context of virtualized reality based telemicromanipulation, we present in this paper a visual calibration technique for optical microscope coupled with a CCD camera. In contrast to previous approaches, a virtual calibration pattern is constructed using the micromanipulator with a sub-pixel localization in the image. We also present a new camera calibration algorithm based on Parallel Single-Plane properties. The proposed procedure leads to a linear system from which the solution gives directly both intrinsic and extrinsic parameters of the geometrical model. Both computer simulation and real data have been used to test the proposed technique, and very good results have been obtained. Compared with classical techniques, our method provides an alternative technical solution, easy to use and flexible in the context of micromanipulation and virtual reality.

Multi-modal object detection and localization for high integrity driving assistance

Much work is currently devoted to increasing the reliability, completeness and precision of the data used by driving assistance systems, particularly in urban environments. Urban environments represent a particular challenge for the task of perception, since they are complex, dynamic and completely variable. This article examines a multi-modal perception approach for enhancing vehicle localization and the tracking of dynamic objects in a world-centric map. 3D ego-localization is achieved by merging stereo vision perception data and proprioceptive information from vehicle sensors. Mobile objects are detected using a multi-layer lidar that is simultaneously used to identify a zone of interest to reduce the complexity of the perception process. Object localization and tracking is then performed in a fixed frame which simplifies analysis and understanding of the scene. Finally, tracked objects are confirmed by vision using 3D dense reconstruction in focused regions of interest. Only confirmed objects can generate an alarm or an action on the vehicle. This is crucial to reduce false alarms that affect the trust that the driver places in the driving assistance system. Synchronization issues between the sensing modalities are solved using predictive filtering. Real experimental results are reported so that the performance of the multi-modal system may be evaluated.