Stefan Wonneberger

Toward automated driving in cities using close-to-market sensors: An overview of the V-Charge Project

Future requirements for drastic reduction of CO2 production and energy consumption will lead to significant changes in the way we see mobility in the years to come. However, the automotive industry has identified significant barriers to the adoption of electric vehicles, including reduced driving range and greatly increased refueling times. Automated cars have the potential to reduce the environmental impact of driving, and increase the safety of motor vehicle travel. The current state-of-the-art in vehicle automation requires a suite of expensive sensors. While the cost of these sensors is decreasing, integrating them into electric cars will increase the price and represent another barrier to adoption. The V-Charge Project, funded by the European Commission, seeks to address these problems simultaneously by developing an electric automated car, outfitted with close-to-market sensors, which is able to automate valet parking and recharging for integration into a future transportation system. The final goal is the demonstration of a fully operational system including automated navigation and parking. This paper presents an overview of the V-Charge system, from the platform setup to the mapping, perception, and planning sub-systems.

Exploration of FPGA-based dense block matching for motion estimation and stereo vision on a single chip

Camera-based systems in series vehicles have gained in importance in the past several years, which is documented, for example, by the introduction of front-view cameras and applications such as traffic sign or lane detection by all major car manufacturers. Besides a pure or enhanced visualization of the vehicles environment, camera systems have also been extensively used for the design and implementation of complex driver assistance functions in diverse research scenarios, as they offer the possibility to extract both depth and motion information of static and moving objects. However, the evolution of existing computation-intensive vision applications from research vehicles toward series integration is currently a challenging task, which is due to the absence of highperformance computer architectures that adhere to the existing strict power and cost constraints. This paper addresses this challenge and explores FPGA-based dense block matching, which enables the calculation of depth information and motion estimation on shared hardware resources, regarding its applicability in intelligent vehicles. This includes the introduction of design scalability in time and space, thereby supporting customized application implementations and multiple camera setups. The presented modular concept also enables enhancements with pre- and post-processing features, which can be utilized to refine the obtained matching results. Its usability has been evaluated in diverse application scenarios and reaches high-performance image processing results of up to 740 GOPS at an acceptable energy level of 11 Watts, rendering it a suitable candidate for future series vehicles.

Automated valet parking and charging for e-mobility

Automated valet parking services provide great potential to increase the attractiveness of electric vehicles by mitigating their two main current deficiencies: reduced driving ranges and prolonged refueling times. The European research project V-Charge aims at providing this service on designated parking lots using close-to-market sensors only. For this purpose the project developed a prototype capable of performing fully automated navigation in mixed traffic on designated parking lots and GPS-denied parking garages with cameras and ultrasonic sensors only. This paper summarizes the work of the project, comprising advances in network communication and parking space scheduling, multi-camera calibration, semantic mapping concepts, visual localization and motion planning. The project pushed visual localization, environment perception and automated parking to centimetre precision. The developed infrastructure-based camera calibration and semi-supervised semantic mapping concepts greatly reduce maintenance efforts. Results are presented from extensive month-long field tests.

Efficient 3D triangulation in hardware for dense structure-from-motion in low-speed automotive scenarios

With the introduction of surround view cameras in modern vehicles and the possibility of calculating dense motion fields in real-time from a moving camera a detailed 3D reconstruction of the static environment is possible (structure-from-motion). Beside the necessity of a motion field between two image frames the task of triangulating those individual 2D point matches to 3D points in the world becomes non real-time on modern CPUs when to be repeated for all image points. In this work we evaluate different approaches to the 3D triangulation optimization problem in a typical structure-from-motion processing chain for an efficient implementation in hardware. An architecture for solving this problem using linear triangulation with an inhomogeneous solution to the equation system is proposed. We evaluate our implementation using FPGAs against a software-implementation with synthetic datasets and from low-speed parking area scenes for numerical accuracy and real-time capabilities. In addition the proposed fixed-point arithmetic implementation is compared against an implementation using floating-point units.

Collision mitigation for crossing traffic in urban scenarios

Current collision mitigation systems focus on rear end collisions. To address the full spectrum of real world accidents, these systems will have to be enhanced to cover more traffic situations. Vehicle to vehicle accidents in crossing traffic situations make up around 25% of accidents in Germany. This paper discusses the requirements and differences compared to rear-end collisions. Presented here is an action concept that takes into account how the impact configuration is changed by breaking the host (impacting) vehicle. Based on this concept the requirements for the detection of crossing traffic were derived. These requirements were met by developing a video system based on a monocular wide field of view camera. It is further shown how this action concept and sensor were integrated into a demonstrator vehicle and evaluated in full scale testing.

Parallel feature extraction and heterogeneous object-detection for multi-camera driver assistance systems

We present a flexible architecture for image-based feature detection and object classification on an FPGA. This architecture is tailored to the requirements of future driver assistance systems, which will make it necessary to detect a wide range of different object types in multi-camera systems requiring highly efficient hardware. In contrast to other designs, which typically address a specific object type or only accelerate early processing steps, the proposed pipeline offers different operation modes to switch resources for either detection or classification speed. In addition, the architecture can incorporate heterogeneous processors for different feature types. The design is tailored to support any object detection system using weak features and cascaded classifiers. For evaluation, a classic Viola Jones Detector is implemented being fully compatible with OpenCV.