Christoph Stiller

Vision meets robotics: The KITTI dataset

We present a novel dataset captured from a VW station wagon for use in mobile robotics and autonomous driving research. In total, we recorded 6 hours of traffic scenarios at 10–100 Hz using a variety of sensor modalities such as high-resolution color and grayscale stereo cameras, a Velodyne 3D laser scanner and a high-precision GPS/IMU inertial navigation system. The scenarios are diverse, capturing real-world traffic situations, and range from freeways over rural areas to inner-city scenes with many static and dynamic objects. Our data is calibrated, synchronized and timestamped, and we provide the rectified and raw image sequences. Our dataset also contains object labels in the form of 3D tracklets, and we provide online benchmarks for stereo, optical flow, object detection and other tasks. This paper describes our recording platform, the data format and the utilities that we provide.

Estimating motion in image sequences

We have reviewed the estimation of 2D motion from time-varying images, paying particular attention to the underlying models, estimation criteria, and optimization strategies. Several parametric and nonparametric models for the representation of motion vector fields and motion trajectory fields have been discussed. For a given region of support, these models determine the dimensionality of the estimation problem as well as the amount of data that has to be interpreted or transmitted thereafter. Also, the interdependence of motion and image data has been addressed. We have shown that even ideal constraints may not provide a well-defined estimation criterion. Therefore, the data term of an estimation criterion is usually supplemented with a smoothness term that can be expressed explicitly or implicitly via a constraining motion model. We have paid particular attention to the statistical criteria based on Markov random fields. Because the optimization of an estimation criterion typically involves a large number of unknowns, we have presented several fast search strategies.

Making Bertha Drive?An Autonomous Journey on a Historic Route

125 years after Bertha Benz completed the first overland journey in automotive history, the Mercedes Benz S-Class S 500 INTELLIGENT DRIVE followed the same route from Mannheim to Pforzheim, Germany, in fully autonomous manner. The autonomous vehicle was equipped with close-to-production sensor hardware and relied solely on vision and radar sensors in combination with accurate digital maps to obtain a comprehensive understanding of complex traffic situations. The historic Bertha Benz Memorial Route is particularly challenging for autonomous driving. The course taken by the autonomous vehicle had a length of 103 km and covered rural roads, 23 small villages and major cities (e.g. downtown Mannheim and Heidelberg). The route posed a large variety of difficult traffic scenarios including intersections with and without traffic lights, roundabouts, and narrow passages with oncoming traffic. This paper gives an overview of the autonomous vehicle and presents details on vision and radar-based perception, digital road maps and video-based self-localization, as well as motion planning in complex urban scenarios.

Three Decades of Driver Assistance Systems: Review and Future Perspectives

This contribution provides a review of fundamental goals, development and future perspectives of driver assistance systems. Mobility is a fundamental desire of mankind. Virtually any society strives for safe and efficient mobility at low ecological and economic costs. Nevertheless, its technical implementation significantly differs among societies, depending on their culture and their degree of industrialization. A potential evolutionary roadmap for driver assistance systems is discussed. Emerging from systems based on proprioceptive sensors, such as ABS or ESC, we review the progress incented by the use of exteroceptive sensors such as radar, video, or lidar. While the ultimate goal of automated and cooperative traffic still remains a vision of the future, intermediate steps towards that aim can be realized through systems that mitigate or avoid collisions in selected driving situations. Research extends the state-of-the-art in automated driving in urban traffic and in cooperative driving, the latter addressing communication and collaboration between different vehicles, as well as cooperative vehicle operation by its driver and its machine intelligence. These steps are considered important for the interim period, until reliable unsupervised automated driving for all conceivable traffic situations becomes available. The prospective evolution of driver assistance systems will be stimulated by several technological, societal and market trends. The paper closes with a view on current research fields.

3D Traffic Scene Understanding From Movable Platforms

In this paper, we present a novel probabilistic generative model for multi-object traffic scene understanding from movable platforms which reasons jointly about the 3D scene layout as well as the location and orientation of objects in the scene. In particular, the scene topology, geometry, and traffic activities are inferred from short video sequences. Inspired by the impressive driving capabilities of humans, our model does not rely on GPS, lidar, or map knowledge. Instead, it takes advantage of a diverse set of visual cues in the form of vehicle tracklets, vanishing points, semantic scene labels, scene flow, and occupancy grids. For each of these cues, we propose likelihood functions that are integrated into a probabilistic generative model. We learn all model parameters from training data using contrastive divergence. Experiments conducted on videos of 113 representative intersections show that our approach successfully infers the correct layout in a variety of very challenging scenarios. To evaluate the importance of each feature cue, experiments using different feature combinations are conducted. Furthermore, we show how by employing context derived from the proposed method we are able to improve over the state-of-the-art in terms of object detection and object orientation estimation in challenging and cluttered urban environments.

Segmentation of 3D lidar data in non-flat urban environments using a local convexity criterion

Present object detection methods working on 3D range data are so far either optimized for unstructured offroad environments or flat urban environments. We present a fast algorithm able to deal with tremendous amounts of 3D Lidar measurements. It uses a graph-based approach to segment ground and objects from 3D lidar scans using a novel unified, generic criterion based on local convexity measures. Experiments show good results in urban environments including smoothly bended road surfaces.

Systems for Safety and Autonomous Behavior in Cars: The DARPA Grand Challenge Experience

In this paper, we review technologies for autonomous ground vehicles and their present capabilities in research and in the automotive market. We outline technology requirements for enhanced functions and for infrastructure development. Since the recent Grand Challenge competition is a major force to advance technology in this field, we specifically refer to our experiences in developing a participating vehicle. We present a multisensor platform that has been proven in an off-road environment. It combines different sensing modalities that inherently yield uncertain information. Finite-state machines are formulated to generate rule-based autonomous behavior that enables fully autonomous off-road driving. Overall, the intent of the paper is to evaluate approaches and technologies used in the two Grand Challenges as they contribute to the needs of autonomous cars on the road

Team AnnieWAY's Entry to the 2011 Grand Cooperative Driving Challenge

In this paper, we present the concepts and methods developed for the autonomous vehicle known as AnnieWAY, which is our winning entry to the 2011 Grand Cooperative Driving Challenge. We describe algorithms for sensor fusion, vehicle-to-vehicle communication, and cooperative control. Furthermore, we analyze the performance of the proposed methods and compare them with those of competing teams. We close with our results from the competition and lessons learned.

Object-based estimation of dense motion fields

Motion estimation belongs to key techniques in image sequence processing. Segmentation of the motion fields such that, ideally, each independently moving object uniquely corresponds to one region, is one of the essential elements in object-based image processing. This paper is concerned with unsupervised simultaneous estimation of dense motion fields and their segmentations. It is based on a stochastic model relating image intensities to motion information. Based on the analysis of natural images, a region-based model of motion-compensated prediction error is proposed. In each region the error is modeled by a white stationary generalized Gaussian random process. The motion field and its segmentation are themselves modeled by a compound Gibbs/Markov random field accounting for statistical bindings in spatial direction and along the direction of motion trajectories. The a posteriori distribution of the motion field for a given image sequence is formulated as an objective function, such that its maximization results in the MAP estimate. A deterministic multiscale relaxation technique with regular structure is employed for optimization of the objective function. Simulation results are in a good agreement with human perception for both the motion fields and their segmentations.

Trajectory planning for Bertha — A local, continuous method

In this paper, we present the strategy for trajectory planning that was used on-board the vehicle that completed the 103 km of the Bertha-Benz-Memorial-Route fully autonomously. We suggest a local, continuous method that is derived from a variational formulation. The solution trajectory is the constrained extremum of an objective function that is designed to express dynamic feasibility and comfort. Static and dynamic obstacle constraints are incorporated in the form of polygons. The constraints are carefully designed to ensure that the solution converges to a single, global optimum.