Romuald Aufrère

A model-driven approach for real-time road recognition

Abstract. This article describes a method designed to detect and track road edges starting from images provided by an on-board monocular monochromic camera. Its implementation on specific hardware is also presented in the framework of the VELAC project. The method is based on four modules: (1) detection of the road edges in the image by a model-driven algorithm, which uses a statistical model of the lane sides which manages the occlusions or imperfections of the road marking – this model is initialized by an off-line training step; (2) localization of the vehicle in the lane in which it is travelling; (3) tracking to define a new search space of road edges for the next image; and (4) management of the lane numbers to determine the lane in which the vehicle is travelling. The algorithm is implemented in order to validate the method in a real-time context. Results obtained on marked and unmarked road images show the robustness and precision of the method.

Accurate road following and reconstruction by computer vision

This paper presents a method designed to track and to recover the three-dimensional (3-D) shape of a road by computer vision. The method is based first upon an accurate detection algorithm which provides a reliable estimation of the roadside in the image. This algorithm works by recursive updating of a statistical model of the lane obtained by an off-line training phase. Once the sides have been located, a reconstruction algorithm computes the vehicle location on its lane, the 3-D shape of the road, and gives both the sides location and their confidence interval for the next image. The detection algorithm then looks for the roadside in this interval in order to limit the computational times, which are about 30-150 ms on a HP workstation.

Localization of a Group of Communicating Vehicles by State Exchange

This paper considers the problem of cooperative localization of an heterogeneous group of road vehicles. Each vehicle can be equipped with proprioceptive and exteroceptive sensors enabling it to localize itself in its environment and also to localize (but not to identify) the other members of the group. Localization information can be exchanged between the vehicles through a wireless communication device. Every member of the group maintains (if possible) an estimation of the state of its environment and transmits it (if possible) to its neighbors. The global state of the environment is obtained by fusing the environment states of the vehicles. This fusion is based on extended Kalman filter where the poses of the detected vehicles are represented by a single system. The proposed approach takes into account the sensor constraints such as data unavailability and delays

A fast and robust vision based road following algorithm

This article presents a fast and robust method designed to detect and track a road lane from images provided by an on-board monocular monochromatic camera. The detection method is based upon a model driven algorithms. It uses a statistical model of the lane which permits to manage the occlusions or imperfections of road marking. This model is obtained by an off-line training phase. The detections are made in optimal interest zones deduced from the model. The tracking process permits to locate the vehicle on its lane and gives the confidence interval of the roadside for the next image. The method has been applied both on marked and unmarked roads images. The results obtained on image sequences of real road scenes show the robustness and precision of the proposed approach.

Cooperative Multi-Vehicle Localization

This paper considers the problem of cooperative localization of an heterogeneous group of road vehicles. Each vehicle is equipped with proprioceptive and exteroceptive sensors enabling it to localize itself in its environment and also to identify and localize the other members of the group. Localization information can be exchanged between the vehicles through a wireless communication device. Every member of the group maintains an estimation of the state of its environment and transmits it to its neighbors. The global state of the environment is obtained by fusing the environment states of the vehicles. This fusion is based on an extended Kalman filter where the group is represented by a single system which describes the state of every member. The proposed approach take into account sensor constraints as data unavailability and delays

Recovering the 3D shape of a road by on-board monocular vision

Deals with a method designed to recover the 3D geometry of a road from an image sequence provided by an on-board monocular monochromatic camera. It only requires the road edges to be detected in the image. The reconstruction process is able to compute (1) the 3D coordinates of the road axis points, (2) the vehicles position on its lane and (3) the prediction of the road edge localization in the next images of the sequence which is very helpful for the detection phase. It also computes the confidence intervals associated with the 3D parameters. The description of the method is followed by the presentation of its most significant results.

Road detection and vehicles tracking by vision for an on-board ACC system in the VELAC vehicle

The paper presents a method designed to detect and track vehicles on highway in a safety improvement purpose. The goal of this kind of system is to regulate the speed of a vehicle so as to respect safety distances relative to vehicles ahead. The method is exclusively based on monocular computer vision and uses two algorithms. The first one is able to locate the lane borders in the image, and to deduce the 3D shape of the road axis. The second algorithm detects, tracks and computes the 3D location of vehicles ahead by using fixed lights embedded on these vehicles. By combining the results of the two algorithms, a fusion step permits us to know were the most dangerous vehicle is, according to its position, speed and circulation lane. The method has been implemented on our experimental vehicle VELAC and the whole system operates in real time conditions.

Accurate vision based road tracker

Vision based road trackers are about to be integrated in current vehicles mainly for security purposes in response to sleepiness problems for example. Nevertheless, such systems must be acceptable for drivers and must have a good reliability both in terms of roadsides recognition and of vehicle location estimation. Such a system must therefore be able to run in spite of difficult situations (due to occlusions, traffic, bad weather conditions, etc). Furthermore, the accuracy of the 3D vehicle estimation must be sufficient in order to feed subsequent warning systems. The system we have designed is able to recognize with reliability the lane sides in the current image and uses a 3D/4D modelling which provides both a good recognition as well as a very accurate 3D parameters estimation (vehicle location, steer angle, road curvatures, etc). The paper focuses mainly on this 3D original estimation stage and presents our recent developments (distance between vehicle and each road side for lane tracking application, analysis distances increasing, vertical road curvature estimation). The algorithm behaviour is then presented in simulated and real situations as well in order to prove the reliability of the approach.

Road Detection and Vehicle Tracking by Vision for Adaptive Cruise Control

This article deals first with a process designed to detect the circulation lane of a vehicle by onboard monocular vision. This detection process is based on a recursive updating of a statistical model of the lane obtained by a training phase. Once the lane has been located, a reconstruction algorithm computes the vehicle location on its lane and the three-dimensional shape of the road. Thereafter, the authors seek to detect and track vehicles situated in front of their vehicle and equipped with specific visual markers in order to achieve an accurate determination of their location and speed. By combining these various data, the most dangerous obstacle can be identified. Each of these three processes is described in detail, and significant examples are provided.

Real-Time Monocular SLAM With Low Memory Requirements

The localization of a vehicle in an unknown environment is often solved using simultaneous localization and mapping (SLAM) techniques. Many methods have been developed, each requiring a different amount of landmarks (map size), and thus of memory, to work efficiently. Similarly, the required computational time is quite variable from one approach to another. In this paper, we focus on a monocular SLAM problem and propose a new method called MSLAM, which is based on an extended Kalman filter (EKF). The aim is to provide a solution that has low memory and processing time requirements and that can achieve good localization results while benefiting from EKF advantages (i.e., direct access to the covariance matrix, no conversion required for the measures or the state, etc.). To do so, a minimal Cartesian representation (three parameters for three dimensions) is used. However, linearization errors are likely to happen with such a representation. New methods allowing to avoid or hugely decrease the impact of the linearization failures are presented. The first contribution proposed here computes a proper projection of a 3-D uncertainty in the image plane, allowing to track landmarks during longer periods of time. A corrective factor of the Kalman gain is also introduced. It allows to detect wrong updates and correct them, thus reducing the impact of the linearization on the whole system. Our approach is compared with a classic SLAM implementation over different data sets and conditions to illustrate the efficiency of the proposed contributions. The quality of the map built is tested by using it with another vehicle for localization purposes. Finally, a public data set presenting a long trajectory (1.3 km) is also used in order to compare MSLAM with a state-of-the-art monocular EKF-SLAM algorithm, both in terms of accuracy and computational needs.