Michael Aeberhard

Experience, Results and Lessons Learned from Automated Driving on Germany's Highways

The BMW Group Research and Technology has been testing automated vehicles on Germanys highways since Spring 2011. Since then, thousands of kilometers have been driven on the highways around Munich, Germany. Throughout this project, fundamental technologies, such as environment perception, localization, driving strategy and vehicle control, were developed in order to safely operate prototype automated vehicles in real traffic with speeds up to 130 km/h. The goal of this project was to learn what technologies are necessary for automated driving. This paper presents the architecture and algorithms developed during this project, results from real driving scenarios, the lessons learned throughout the project and a quick introduction into the latest developments for improving the system.

Track-to-Track Fusion With Asynchronous Sensors Using Information Matrix Fusion for Surround Environment Perception

Driver-assistance systems and automated driving applications in the future will require reliable and flexible surround environment perception. Sensor data fusion is typically used to increase reliability and the observable field of view. In this paper, a novel approach to track-to-track fusion in a high-level sensor data fusion architecture for automotive surround environment perception using information matrix fusion (IMF) is presented. It is shown that IMF produces the same good accuracy in state estimation as a low-level centralized Kalman filter, which is widely known to be the most accurate method of fusion. Additionally, as opposed to state-of-the-art track-to-track fusion algorithms, the presented approach guarantees a globally maintained track over time as an object passes in and out of the field of view of several sensors, as required in surround environment perception. As opposed to the often-used cascaded Kalman filter for track-to-track fusion, it is shown that the IMF algorithm has a smaller error and maintains consistency in the state estimation. The proposed approach using IMF is compared with other track-to-track fusion algorithms in simulation and is shown to perform well using real sensor data in a prototype vehicle with a 12-sensor configuration for surround environment perception in highly automated driving applications.

A prediction-based reactive driving strategy for highly automated driving function on freeways

Highly automated driving on freeways requires a complex artificial intelligence that makes optimal decisions based on the current measurements and information. The architecture of the decision-making process, hereinafter referred to as driving strategy, should allow diversity in decision-making for various traffic situations and modular expandability of the overall intelligence. Besides a reactive response to changes in the dynamic environment, a deliberative component should also be considered to incorporate the future evolution of the environment. This paper presents a novel driving strategy that meets the above requirements. The complex driving task is discretized by organization into a finite set of “behavioral strategies” through the developed “decision network”. The decision-making process itself is realized by a nonlinear model predictive approach which is solved using combinatorial optimization formulation. Lastly, the capability of the proposed approach is demonstrated in two freeway situations.

Object existence probability fusion using dempster-shafer theory in a high-level sensor data fusion architecture

Future driver assistance systems need to be more robust and reliable because these systems will react to increasingly complex situations. This requires increased performance in environment perception sensors and algorithms for detecting other relevant traffic participants and obstacles. An objects existence probability has proven to be a useful measure for determining the quality of an object. This paper presents a novel method for the fusion of the existence probability based on Dempster-Shafer evidence theory in the framework of a highlevel sensor data fusion architecture. The proposed method is able to take into consideration sensor reliability in the fusion process. The existence probability fusion algorithm is evaluated for redundant and partially overlapping sensor configurations.

A Combined Model- and Learning-Based Framework for Interaction-Aware Maneuver Prediction

This paper presents a novel online-capable interaction-aware intention and maneuver prediction framework for dynamic environments. The main contribution is the combination of model-based interaction-aware intention estimation with maneuver-based motion prediction based on supervised learning. The advantages of this framework are twofold. On one hand, expert knowledge in the form of heuristics is integrated, which simplifies the modeling of the interaction. On the other hand, the difficulties associated with the scalability and data sparsity of the algorithm due to the so-called curse of dimensionality can be reduced, as a reduced feature space is sufficient for supervised learning. The proposed algorithm can be used for highly automated driving or as a prediction module for advanced driver assistance systems without the need of intervehicle communication. At the start of the algorithm, the motion intention of each driver in a traffic scene is predicted in an iterative manner using the game-theoretic idea of stochastic multiagent simulation. This approach provides an interpretation of what other drivers intend to do and how they interact with surrounding traffic. By incorporating this information into a Bayesian network classifier, the developed framework achieves a significant improvement in terms of reliable prediction time and precision compared with other state-of-the-art approaches. By means of experimental results in real traffic on highways, the validity of the proposed concept and its online capability is demonstrated. Furthermore, its performance is quantitatively evaluated using appropriate statistical measures.

Track-to-track fusion with asynchronous sensors and out-of-sequence tracks using information matrix fusion for advanced driver assistance systems

Future advanced driver assistance systems will contain multiple sensors that are used for several applications, such as highly automated driving on freeways. The problem is that the sensors are usually asynchronous and their data possibly out-of-sequence, making fusion of the sensor data non-trivial. This paper presents a novel approach to track-to-track fusion for automotive applications with asynchronous and out-of-sequence sensors using information matrix fusion. This approach solves the problem of correlation between sensor data due to the common process noise and common track history, which eliminates the need to replace the global track estimate with the fused local estimate at each fusion cycle. The information matrix fusion approach is evaluated in simulation and its performance demonstrated using real sensor data on a test vehicle designed for highly automated driving on freeways.

A generic driving strategy for urban environments

Autonomous driving in urban environments depends on the ability to interpret the current situation and to react accordingly. This means to continuously make decisions for certain comfort-optimized maneuvers under the constraints of traffic rules and feasibility. This work presents a novel, longitudinal driving strategy formulated as a discrete planning problem. Instead of designing an algorithm for a single one of various potential subproblems, two interfaces are presented, called static and dynamic events, that are capable of representing any situation along the chosen lane of the autonomous vehicle. This allows fast, analytic calculation of Inevitable Collision States which are used as heuristic to realize a guided A* search. Instead of being limited to a small, finite set of maneuvers as rule-based driving strategies like state machines are, the algorithm selects the optimal of an infinite number of possible, implicit maneuvers. The presented algorithm has a worst-case runtime of 80 ms for a planning horizon of 13 seconds and is therefore capable of running online. The approach is evaluated on a simulator in a complex city scenario and on a prototype vehicle on the test track.

A Game-Theoretic Approach to Replanning-Aware Interactive Scene Prediction and Planning

This paper presents a novel cooperative-driving prediction and planning framework for dynamic environments based on the methods of game theory. The proposed algorithm can be used for highly automated driving on highways or as a sophisticated prediction module for advanced driver-assistance systems with no need for intervehicle communication. The main contribution of this paper is a model-based interaction-aware motion prediction of all vehicles in a scene. In contrast to other state-of-the-art approaches, the system also models the replanning capabilities of all drivers. With that, the driving strategy is able to capture complex interactions between vehicles, thus planning maneuver sequences over longer time horizons. It also enables an accurate prediction of traffic for the next immediate time step. The prediction model is supported by an interpretation of what other drivers intend to do, how they interact with traffic, and the ongoing observation. As part of the prediction loop, the proposed planning strategy incorporates the expected reactions of all traffic participants, offering cooperative and robust driving decisions. By means of experimental results under simulated highway scenarios, the validity of the proposed concept and its real-time capability is demonstrated.

Online Active Calibration for a Multi-LRF System

Multi-LRF(Laser Range Finder) systems have been broadly utilized in sensor fusion for automobile. In order to convert multiple LRF data into a unified coordinate system, we have to obtain the rigid transformation among multi-LRF. In this paper, we propose a new algorithm for online extrinsic calibration of multi-LRFs by observing a planar checkerboard pattern and solving for transformation between the views of a planar checkerboard from a camera and multi-LRF. Existing LRF calibration is achieved by freely moving a checkerboard pattern and conducting much offline optimization. Compared with traditional algorithm, the advantages of our approach are twofold. Firstly, adopting the noise of images and LRF depth readings, we can exactly calculate the exact position and pose of the checkerboard that can largely reduce the transformation error. Secondly, the complete calibration process is online, which means the exact position and pose of the checkerboard can be obtained in real-time and manipulated by robotic arm. In the end, our calibration approach is validated through real experiments that show the superiority with respect to the state-of-art methods.

Please take over! An analysis and strategy for a driver take over request during autonomous driving

During autonomous driving, in particular conditional or highly automated driving, a critical part of the system is the driver take over request. Little focus has been given to this important aspect in an automated driving journey. A driver take over request, or TOR, can happen for various reasons and under varying circumstances. Once a TOR occurs, as defined in conditional or highly automated driving, the driver has a finite amount of time in order to take over manual control of the vehicle before the automated driving system deactivates. This paper presents a detailed analysis of why a TOR can occur, how the automated driving system should react during the TOR phase and what should happen at the end of a TOR in order to realize a safe and comfortable TOR for the driver. Various driving strategies during a TOR are presented and evaluated for a single-lane highway scenario.