Stefan Franz

Sensor fusion to enable next generation low cost Night Vision systems

The next generation of automotive Night Vision Enhancement systems offers automatic pedestrian recognition with a performance beyond current Night Vision systems at a lower cost. This will allow high market penetration, covering the luxury as well as compact car segments. Improved performance can be achieved by fusing a Far Infrared (FIR) sensor with a Near Infrared (NIR) sensor. However, fusing with todays FIR systems will be too costly to get a high market penetration. The main cost drivers of the FIR system are its resolution and its sensitivity. Sensor cost is largely determined by sensor die size. Fewer and smaller pixels will reduce die size but also resolution and sensitivity. Sensitivity limits are mainly determined by inclement weather performance. Sensitivity requirements should be matched to the possibilities of low cost FIR optics, especially implications of molding of highly complex optical surfaces. As a FIR sensor specified for fusion can have lower resolution as well as lower sensitivity, fusing FIR and NIR can solve performance and cost problems. To allow compensation of FIR-sensor degradation on the pedestrian detection capabilities, a fusion approach called MultiSensorBoosting is presented that produces a classifier holding highly discriminative sub-pixel features from both sensors at once. The algorithm is applied on data with different resolution and on data obtained from cameras with varying optics to incorporate various sensor sensitivities. As it is not feasible to record representative data with all different sensor configurations, transformation routines on existing high resolution data recorded with high sensitivity cameras are investigated in order to determine the effects of lower resolution and lower sensitivity to the overall detection performance. This paper also gives an overview of the first results showing that a reduction of FIR sensor resolution can be compensated using fusion techniques and a reduction of sensitivity can be compensated.

Assessment of image sensor performance with statistical perception performance analysis

The performance of perceptive systems depends on a large number of factors. The practical problem during development is, that this dependency is very often not explicitly known. In this contribution we address this problem and present an approach to evaluate perception performance, as a function of e.g. quality of the sensor data. The approach is to use standardized quality metrics for imaging sensors, and to relate them to the observed performance of the environment perception. During our experiments, several imaging setups were analyzed. The output of each setup is processed offline to track down performance differences with respect to the quality of sensor data. We show how and to what extend the measurement of the Modulation Transfer Function (MTF) using standardized tests can be applied to evaluate the performance of imaging systems. The influence of the MTF on the signal-to-noise ratio can be used to evaluate the performance on a recognition task. We assess the measured performance by processing the data of different, simultaneously recorded imaging setups for the task of lane recognition.

Analysis and assessment of far infrared sensor performance parameters and their impact on pedestrian detection

In this contribution four different far infrared sensor setups with different optical configurations are evaluated based on their performance for pedestrian detection. The focus of the measurements is on the impact of resolution and sensitivity on the detection performance.

Evaluating sensor effects on perception performance

Performance of perceptive systems depends on the quality of the input data. In this contribution, an approach to evaluate perception performance as a function of quality of the sensor data is presented. Standardized quality metrics support the imaging sensors performance measurement. Several imaging setups are analyzed with real world experiments. The output of each setup is processed offline to track down performance differences with respect to the quality of sensor data. An adapted measurement is calculated to measure the sensor performance with respect to the data quality for the involved perceptive components. The measured performance is assessed by processing the data of different simultaneously recorded imaging setups for the task of feature extraction of road lanes.

Performance evaluation of FIR sensor systems applied to pedestrian detection

Besides resolution, an important performance parameter of a FIR camera is the sensitivity. It depends on the sensitivity of the detector array itself and the characteristics of the optic. The effects of the optic are considerably driven by the f-number, with high values resulting in decreased sensitivity, but providing the possibility for simple lens design and cheaper production costs. In this contribution 4 different sensor setups with different optics are evaluated for their impact on the performance of trained pedestrian classifiers. To overcome the expensive and time consuming process of ground truth generation for multiple sensors, an approach for reusing available high sensitivity reference data is presented. Classifiers are trained on specially transformed reference data with characteristics of sensors with degraded sensitivity. For the evaluation of the classifiers, data of real world road scenarios is collected simultaneously with the target sensors mounted in parallel in a test vehicle, following a detailed script for recording a pedestrian scene test catalogue. This allows for a direct analysis and comparison of the different sensors and their impact on the detection performance.

Characterization and adjustment of high performance objectives for DUV applications

Aside from steppers also inspection systems in the semiconductor industry as well as in micro material processing require DUV imaging optics with very high optical requirements. A test and adjustments set-up based on the Shack-Hartmann wave front sensor for objectives and telescopes is presented. It allows primarily to characterize the image quality of systems under test for both finite as well as infinite object and image distances. From the wave front the modulation transfer function, point spread function or encircled energy data can be derived. Also, other data such as magnifications, focal lengths and even distortion with micrometer accuracy can be obtained with the test bench. The test system consists of a spherical waves generator, the sensor including adapting optics and the mechanical motion system. It is highly motorized and all essential functions are computer controlled. The available wavelengths currently range from NIR to 193nm.