Tilo Schwarz

Precise timestamping and temporal synchronization in multi-sensor fusion

This paper presents a new approach to exact timestamping of asynchronous measurements in a multi-sensor setup. In order to improve the performance of a single sensor, driver assistance systems use several different sensors which have different latencies that usually cannot be measured directly. These unknown latencies pose a problem in data association and temporal synchronization. Consequently, a method to estimate or incorporate the latencies is needed in all sensor fusion algorithms in order to derive the real time of a measurement. In this paper a method is described to compensate the sensor latencies even if they cannot be observed directly.

Track level fusion algorithms for automotive safety applications

Fusion of information from different sensor systems is vital for automotive safety systems. In a typical automotive sensor fusion setup the fusion can be a measurement fusion or a track level fusion in a centralized fusion center. Track level fusion is desired due to communication, computation and organizational constraints. Track level fusion algorithms have to deal with different correlation issues among the sensor systems and the fusion center that maintains the global track. This correlation problem leads to a degraded fusion estimate which is not desirable in a safety and time critical application. This paper presents an overview of track level fusion algorithms to fuse two homogenous sensor systems and evaluates their performance.

Temporal synchronization in multi-sensor fusion for future driver assistance systems

A new approach to precise timestamping and temporal synchronization in a multi-sensor setup is presented. Modern driver assistance systems use an increasing amount of different sensors that do not provide the exact timestamps of the measurements, nor the period of time between two measurements. This paper describes a method to determine these timestamps up to millisecond accuracy. Possible drifts of the internal sensor clocks are taken into account. In addition, a frequent problem are lost measurements that lead to large errors in the timestamping. These framedrops are detected reliably and handled adequately via the method at hand. Finally a method to evaluate the performance of the whole timestamping procedure is described and real-world results are presented.

Influence of the Sensor Local Track Covariance on the Track-to-Track Sensor Fusion

The information fusion of the processed sensory tracks is carried out using track-to-track fusion algorithms. The performance analysis of a selected track-to-track fusion algorithms under different sensory track covariance configurations are carried out in this paper. This is the first paper that does the study on the influence of sensory track covariance on the performance of three important algorithms for track-to-track fusion. A simulation setup with known system parameters and an optimal centralized measurement fuser based on the Kalman estimator as the benchmark is used to numerically evaluate the different algorithms with different sensory track covariance configurations. The results of this experiment shows that sensory track covariance plays an important role in achieving a consistent fused estimate in a track-to-track fusion problem. It is difficult to obtain this vital information at the fusion center in a real world system due to certain practical limitations. It is necessary to compensate this loss of information by estimating the respective sensors local track covariance. Some practical solutions based on the available information at the fusion center, which could be used to carry out this compensation is proposed in this paper.

Object management strategy for an unified high level automotive sensor fusion framework

An important requirement in autonomous driving for many complex scenarios is to correctly detect static and dynamic targets under various states of motion. The possibility of fulfilling this requirement depends upon the availability of different sensor data to the sensor fusion module. This paper uses data from sensors with built-in tracking modules and our objective is to make the resultant of two different sensor fusion modules that use the same sensor tracked data, to be statistically relevant based on the respective operational requirements despite this commmon prior set-up. In our case, we have two sensor fusion modules. One sensor fusion module deals with dynamic targets with well-defined object representation and other module deals only with static targets of undefined shapes. The authors have developed different concepts to manage the relevancy of the deliverables of the two modules. A novel approach based on multi-hypothesis tracking is presented. The results are evaluated using simulation and as well as with real world sensor data with reference ground truth target data.

Track to Track Fusion Incorporating Out of Sequence Track Based on Information Matrix Fusion

This paper presents the problem of information fusion in a multi-sensor setup of asynchronous sensors with different latencies. This leads to the problem of tracks that have arbitrary arrival time at the fusion center. A solution for the integration of tracks that are temporally out of order is proposed. The proposed algorithm is quite suitable for the trackto-track fusion requirements. This solution avoids the complex calculation involved in negating the effect of process noise that influences the estimation accuracy in a track-to-track fusion and also the correlated process noise problem that arises during the integration of out of order track with the global track. Monte Carlo simulations are carried out with different sensor characteristics to study the performance of the algorithm. The result of the algorithm is compared with an optimal filtering benchmark.

Combi-Tor: Track-to-Track Association Framework for Automotive Sensor Fusion

The data association algorithm plays the vital role of forming an appropriate and valid set of tracks from the available tracks at the fusion center, which are delivered by different sensors local tracking systems. The architecture of the data association module has to be designed taking into account the fusion strategy of the sensor fusion system, the granularity and the quality of the data provided by the sensors. The current generation environment perception sensors used for automotive sensor fusion are capable of providing estimated kinematic and as well as non-kinematic information on the observed targets. This paper focuses on integrating the kinematic and non-kinematic information in a track-to-track association (T2TA) procedure. A scalable framework called Combi-Tor is introduced here that is designed to calculate the association decision using likelihood ratio tests based on the available kinematic and non-kinematic information on the targets, which are tracked and classified by different sensors. The calculation of the association decision includes the uncertainty in the sensors local tracking and classification modules. The required sufficient statistical derivations are discussed. The performance of this T2TA framework and the traditional T2TA scheme considering only the kinematic information are evaluated using Monte-Carlo simulation. The initial results obtained using the real world sensor data is presented.