J. Marius Zöllner

Design of an automotive traffic sign recognition system targeting a multi-core SoC implementation

This paper describes the design of an automotive traffic sign recognition application. All stages of the design process, starting on system-level with an abstract, pure functional model down to final hardware/software implementations on an FPGA, are shown. The proposed design flow tackles existing bottlenecks of todays system-level design processes, following an early model-based performance evaluation and analysis strategy, which takes into account hardware, software and real-time operating system aspects. The experiments with the traffic sign recognition application show, that the developed mechanisms are able to identify appropriate system configurations and to provide a seamless link into the underlying implementation flows.

Visual state estimation of traffic lights using hidden Markov models

The comprehension of dynamic objects in the environment is a major concern of prospective assistance systems. Among the relevant dynamic objects are not only road users, but also parts of the traffic infrastructure: Traffic lights switch between different light colors to manage traffic at intersections. We propose a camera-based approach to incorporate the visual information of traffic lights. Assistance systems can use it to realize comfort, fuel economy and safety functions. We focus on the classification and state estimation using support vector machines and hidden Markov models. Our system is able to distinguish different types of traffic lights - even blinking lights - in real-time.

A Situation context aware Dempster-Shafer fusion of digital maps and a road sign recognition system

Speed limit information systems solely based on one modality can hardly overcome their respective intrinsic disadvantages: Digital maps lack support for short-term changes brought by variable message signs and road works, while camera based systems cannot recognize implicit speed limits and may fail in adverse lighting scenarios. In this work we show a fusion approach that is able to overcome these limitations. It is specifically tailored to our task by adapting sensor reliability based on the perceived situation context. This enables the camera based system to easily outvote the digital map in a construction site or the digital map to veto against uncertain camera recognition results during nighttime. The proposed fusion approach was implemented and evaluated on a qualitative base showing very promising results: The situation context aware fusion is able to deduce the correct effective speed limit even when one of the sensors fails. Moreover, it reduces conflicts encountered between the sources compared to a not situation context aware fusion.

Anticipatory driving assistance for energy efficient driving

Due to the rising ecological awareness as well as the scarcity of fuel, associated with the rising price of petrol, the demand for economically ideal working vehicles rose. Car manufacturers put a lot of effort in improving the car body, the engine, or the power train of cars. However, the driver can save fuel and money by learning a more efficient driving style, too. Assistance systems can help drivers to learn a more economic driving style and drive anticipating. In this paper an Anticipatory Energy Saving Assistant (ANESA1) is presented. It is helping the driver to drive anticipating and, thus, driving more energy efficient. For this purpose, ANESA is giving the driver precisely timed commands regarding the velocity restrictions upcoming. By approaching a velocity restriction freewheeling, the energy loss of braking is avoided. The advices of ANESA are based on the freewheeling characteristic of the car, given the current speed and the elevation profile of the track ahead, and the upcoming velocity restrictions, which can be road signs or sharp turns. Evaluations in a Driving Simulator showed, that drivers saved 12.97% of energy in average using ANESA by the more usage of 2.53% of time. The tests showed as well, that using ANESA drivers saved 8.06% more energy than trying to save energy by their own abilities.

Driver head pose and gaze estimation based on multi-template ICP 3-D point cloud alignment

Head movements, combined with the line of gaze, play a fundamental role in predicting the drivers actions and in inferring his intention. However, a gaze tracking system for automotive applications needs to satisfy high demands: It must not disturb the driver in his freedom of movements, it must cover large and fast head turns in yaw and pitch, be resistant to changing illumination conditions, be fast enough to recognize fast mirror checks, which are performed almost exclusively through eye rather than head movements, and be accurate and reliable enough to derive high quality information for driver assistance systems relying on their output. In this work a multi-template, ICP-based gaze tracking system is introduced. The system determines the head pose and subsequently estimates the drivers line of gaze by analyzing the angles of the eyes. Due to a fast search of correspondences, and switching between point-to-point and point-to-plane alignment, real-time performance and high accuracy can be achieved. The system is compared with other state of the art head pose estimation systems based on a publicly available benchmark database, where a classification rate of 92% at a tolerance of 10 degrees in yaw could be achieved. We further show in the experiments section, that head rotations up to 4 radians per seconds can be handled. Taking the angles of the eyes into account, rather than the head pose only, the drivers line of sight could be successfully mapped to particular regions of interest.

Probabilistic driving style determination by means of a situation based analysis of the vehicle data

Today, driver assistance systems assist the driver in manifold ways. Their acceptance and usefulness can be highly increased by adapting them to the needs and the personality of the driver. In this work the driving style of the driver is determined by means of rating the drivers actions in commonly occurring traffic situations. Therefore, the vehicle data is evaluated and a probabilistic affiliation of the driver being aggressive, anxious, economical, keen, or sedate is made. The situations are chosen to be day-to-day traffic situations, for instance approaching a village, stopping on a stop sign, or passing through a tight bend. Based on the determined driving style, future driving assistance systems can be personalized to the individual driver and, thus, get more valuable. As a showcase, we adjust our Anticipatory Energy Saving Assistant (ANESA) to the drivers character, which is giving driving hints how to save energy in tight curves. By personalizing ANESA more credence is gained, resulting in extra savings of energy as we show in the experiments made.

Functional system architectures towards fully automated driving

The functional system architecture of an automated vehicle plays a crucial role in the performance of the vehicle. When considered as a backbone, it does not only transmit information between distinct layers, but rather serves as a feedback mechanism coordinating the degradation between them and thereby regulates the behavior of the system against failures. Hence, the design of robust functional architectures is essential to cope with the uncertainties of the world. This paper summarizes existing system architectures and investigates them regarding their robustness against measurement inaccuracies, failures, and unexpected evolution of traffic situations. After illustrating their strengths and deficiencies, we derive the requirements and propose a structure for future, robust system architectures.

Traffic intersection situation description ontology for advanced driver assistance

This work provides an approach to create a generic situation description for advanced driver assistance systems using logic reasoning on a traffic situation knowledge base. It contains multiple objects of different type such as vehicles and infrastructure elements like roads, lanes, intersections, traffic signs, traffic lights and relations among them. Logic inference is performed to check and extend the situation description and interpret the situation e. g. by reasoning about traffic rules. The capabilities of our ontological situation description approach are shown at the example of complex intersections with several roads, lanes, vehicles and different combinations of traffic signs and traffic lights. Real-time issues are discussed thereon.

Recognition and tracking of temporary lanes in motorway construction sites

We propose a system for detecting a construction site on motorways, and subsequently track its individual lanes using kalman filters. Beacons and yellow lane markers are extracted from color images and provide indicators for a construction site situation. Yellow lane markers are used in this case for tracking the temporary lanes at road works. When leaving the construction site, again white road markers are extracted to track regular motorway lanes.

Obstacle detection with a Photonic Mixing Device-camera in autonomous vehicles

In autonomous vehicles as well as in modern driver assistance systems, obstacle detection shows to be the most important task to be achieved. This paper presents a collision avoidance system, based on a modern Time-Of-Flight camera. These cameras allow a 3D perception of the environment, in which obstacles can be detected, independent of special features. Thus, the system is capable of all kinds of objects, including pedestrians as well as bicycles or vehicles. The used Photonic Mixing Device (PMD) camera has a measurement range of up to 50 m. The system is integrated into an autonomous vehicle, on which detected obstacles are investigated in detail. The vehicle steering commands are then generated by a behaviour network, depending on the presence of obstacles in the driving lane.