Stefan Müller-Schneiders

A Decision Fusion and Reasoning Module for a Traffic Sign Recognition System

A novel approach for a decision fusion and reasoning system for vision-based traffic sign recognition is presented. This module consists of several steps. In the first stage, a track-based Bayesian fusion scheme is used to fuse the classification results from each frame to obtain a fusion result for each track to decide whether a sign is present, as well as to determine the sign type. In order to determine the sign type, the temporal fusion scheme has been combined with a decision tree. In the second stage, the system combines and fuses probable identical objects which help to further reduce failures in the recognition process. The decision is based on the fusion results, as well as a position cue. Finally, a reasoning module is used to decide which of the passed signs should be shown to the driver. In addition to these modules, a general evaluation method for multi-class tracking systems is shown. While some failures are observed from the evaluation on object level, the additional post processing steps improve the system in such a way that the finally presented signs are almost always correct on the test set.

A novel approach to lane detection and tracking

In this paper, a new robust approach for camera based lane recognition is presented. The tracking filter and the detection interact such that the tracking filter is used to place a region of interest for a detection of lane segments in various distances, and each successful detection is used to update the lane geometry in the tracking filter. A novel and time efficient detection algorithm is used to detect the position and the slope of the lane segments. To be able to cope with sudden changes in the curvature, two interacting Extended Kalman filters are used to select the bandwidth for the filter. To our best knowledge this algorithm has not been applied to vision based lane tracking before. First results indicate that our approach is robust and real time capable with an average execution time of below 3 ms on a 3Ghz standard PC.

A novel multi-lane detection and tracking system

In this paper a novel spline-based multi-lane detection and tracking system is proposed. Reliable lane detection and tracking is an important component of lane departure warning systems, lane keeping support systems or lane change assistance systems. The major novelty of the proposed approach is the usage of the so-called Catmull-Rom spline in combination with the extended Kalman filter tracking. The new spline-based model enables an accurate and flexible modeling of the lane markings. At the same time the application of the extended Kalman filter contributes significantly to the system robustness and stability. There is no assumption about the parallelism or the shapes of the lane markings in our method. The number of lane markings is also not restrained, instead each lane marking is separately modeled and tracked. The system runs on a standard PC in real time (i.e. 30 fps) with WVGA image resolution (752 × 480). The test vehicle has been driven on the roads with challenging scenarios, like worn out lane markings, construction sites, narrow corners, exits and entries of the highways, etc., and good performance has been demonstrated. The quantitative evaluation has been performed using manually annotated video sequences.

Time to contact estimation using interest points

This paper presents a novel approach to obtain reliable and robust time-to-contact estimates from a monocular moving camera observing various obstacles. The algorithm utilizes interest points to measure the relative scale change of an obstacle and applies robust estimation techniques to combine the different measurements into one of three possible motion models. These include a model with constant distance, with constant velocity and with constant acceleration. An interacting multiple model framework is used to select the appropriate model and finally to estimate the time-to-contact with the observed obstacle. The algorithm presented is evaluated utilizing a large set of recorded video sequences with radar ground truth. Due to its field of application the entire algorithm is designed to use as little computation time as possible and is thus realtime capable.

3D Traffic Sign Tracking Using a Particle Filter

In recent years, there was much activity in the development of camera based active safety systems to aid and to support the driver of a car. One application for such a system is the detection and classification of traffic signs. An important aspect of such a system is the tracking of traffic signs. We present a novel algorithm to track traffic signs in 3D using a single monochrome camera. The algorithm allows to use the constraint that the observed movement on the image plane is entirely caused by the host car movement, which is partially known from internal sensors. The usage of the sensor information improves the tracking process and allows a robust rejection of false positive detections. We also present a way to incorporate a shape cue directly from the image plane into the tracking process. First tests show good results in practice and indicate, that this kind of tracking makes a very valuable addition to a traffic sign detection system.

Efficient and robust extrinsic camera calibration procedure for Lane Departure Warning

Intelligent Driver Assistance Systems, such as Lane Departure Warning, extract 3D information of the road geometry from a camera. Therefore, the transformation between the image and the ground plane has to be determined with a very high accuracy. Conventional calibration methods are usually a compromise between the accuracy and a preferably small effort for the calibration set-up. In this paper, we present an efficient and robust method for an accurate estimation of the extrinsic parameters based on minimizing an error function. The idea is to avoid the difficult and time-consuming measurement of marker positions in the 3D world coordinate system which is fixed with respect to the vehicle. A pattern of circles is placed on the ground plane in front of the car. For our approach, it is only necessary to measure the relative distances between the centers of the circles to each other. A nonlinear-optimization algorithm minimizes the squared difference between the distances of the backprojected circles segmented in the images on the ground plane and of the measurement in the real world.

A novel region of interest selection approach for traffic sign recognition based on 3D modelling

Most existing vision-based traffic sign recognition (TSR) systems rely for the initial shape detection on a processing step that assumes possible positions and sizes for traffic signs to be known a priori. Usually the better the expected position and size can be estimated, the better do these shape detection algorithms perform concerning accuracy and processing time. This paper presents a novel region of interest concept that is based on the known real world positions of traffic signs during the different driving situations. The vehicle surrounding along the predicted host path and the expected positions of traffic signs are modelled in 3D. A projection to the image plane results in a region of interest for traffic signs. This projected region of interest reduces the search-space for initial candidate detection. A reduced search-space leads to a lower number of false positives and reduced computational costs.

Performance evaluation of a real time traffic sign recognition system

Traffic sign recognition has been a very active research topic for many years now. However, during this time of intensive research, no common evaluation methodology has been established. This paper explains in detail, how we evaluated our real time video-based traffic sign recognition system and thus may serve as a building block towards establishing a commonly accepted evaluation methodology. The proposed evaluation methods are taken from the visual surveillance research community, which was very active in evaluation techniques during the recent years.

ELA - an exit lane assistant for adaptive cruise control and navigation systems

This paper presents a new application, the Exit Lane Assistant (ELA), based on a novel and robust vision-based classifier of lane boundary types. Using the knowledge that the exit lane is separated by a special lane boundary type from the other lanes, the intention of leaving the motorway can be recognized by classifying the type of the crossed lane boundaries. Therefore, lane markings are detected at predefined vertical coordinates in the image, so-called scanlines. The detection results, lane marking detected or not detected, are saved into a one-dimensional time-series for each scanline and lane boundary. Based on a Fourier analysis of the set of time series, features are extracted and compared with the theoretic values for the different boundary types using a nearest-neighbor classifier. The lane boundary type is determined fusing the classification results of each scanline based on their confidence. The exit lane ist finally recognized by a rule-based fusion with a digital map.

A novel approach for the online initial calibration of extrinsic parameters for a car-mounted camera

This paper presents a novel approach for an online initial camera calibration to estimate the extrinsic parameters for vision-based intelligent driver assistance systems. The method uses the periodicity of dashed lane markings and velocity information to determine the extrinsic camera parameters: height, pitch and roll angle. A lane marking detector is utilized to convert the images of road scenes into a set of one-dimensional time series. Thereby, the lane marking detector samples the markings at predefined vertical coordinates in the image, so-called scanlines. Based on a correlation analysis and velocity information, the spatial shift between the scanlines is determined. Thus, the distances along the longitudinal lane markings are measured in the coordinate system of the vehicle independently of camera mounting parameters. The Gauss-Newton algorithm is implemented to minimize the squared error between these estimated distances and the distances obtained by the backprojection to a ground plane using the parameter dependent pinhole camera model. Finally, the approach is evaluated using synthetic and real data with promising results.