Ralf Haeusler

Ensemble Learning for Confidence Measures in Stereo Vision

With the aim to improve accuracy of stereo confidence measures, we apply the random decision forest framework to a large set of diverse stereo confidence measures. Learning and testing sets were drawn from the recently introduced KITTI dataset, which currently poses higher challenges to stereo solvers than other benchmarks with ground truth for stereo evaluation. We experiment with semi global matching stereo (SGM) and a census data term, which is the best performing real-time capable stereo method known to date. On KITTI images, SGM still produces a significant amount of error. We obtain consistently improved area under curve values of sparsification measures in comparison to best performing single stereo confidence measures where numbers of stereo errors are large. More specifically, our method performs best in all but one out of 194 frames of the KITTI dataset.

Performance of Correspondence Algorithms in Vision-Based Driver Assistance Using an Online Image Sequence Database

This paper discusses options for testing correspondence algorithms in stereo or motion analysis that are designed or considered for vision-based driver assistance. It introduces a globally available database, with a main focus on testing on video sequences of real-world data. We suggest the classification of recorded video data into situations defined by a cooccurrence of some events in recorded traffic scenes. About 100-400 stereo frames (or 4-16 s of recording) are considered a basic sequence, which will be identified with one particular situation. Future testing is expected to be on data that report on hours of driving, and multiple hours of long video data may be segmented into basic sequences and classified into situations. This paper prepares for this expected development. This paper uses three different evaluation approaches (prediction error, synthesized sequences, and labeled sequences) for demonstrating ideas, difficulties, and possible ways in this future field of extensive performance tests in vision-based driver assistance, particularly for cases where the ground truth is not available. This paper shows that the complexity of real-world data does not support the identification of general rankings of correspondence techniques on sets of basic sequences that show different situations. It is suggested that correspondence techniques should adaptively be chosen in real time using some type of statistical situation classifiers.

Benchmarking stereo data (not the matching algorithms)

Current research in stereo image analysis focuses on improving matching algorithms in terms of accuracy, computational costs, and robustness towards real-time applicability for complex image data and 3D scenes. Interestingly, performance testing takes place for a huge number of algorithms, but, typically, on very small sets of image data only. Even worse, there is little reasoning whether data as commonly applied is actually suitable to prove robustness or even correctness of a particular algorithm. We argue for the need of testing stereo algorithms on a much broader variety of image data then done so far by proposing a simple measure for putting image stereo data of different quality into relation to each other. Potential applications include purpose-directed decisions for the selection of image stereo data for testing the applicability of matching techniques under particular situations, or for realtime estimation of stereo performance (without any need for providing ground truth) in cases where techniques should be selected depending on the given situation.

Mosaicing of microscope images based on SURF

In this paper, we describe the design of a mosaicing technique for images from a microscope system with automatically controlled object stage and image capture unit. Due to the limited field of view in microscope imagery, larger objects are split up into many adjacent, but slightly overlapping frames. In many fields, such as medicine or biology, it is vastly beneficial that these image patches are recomposed to a single (panoramic) image. We propose a feature matching and registration method based on SURF (Speeded-Up Robust Features). This method is most accurate for microscopy images, which usually have repetitive, blob-like structures. Further steps in our algorithm are estimation of transformation parameters for image warping and blending for elimination of color and luminance differences between images. For feature matching, we propose a new method of dividing descriptor windows. This increases matching speed considerably. The experimental results provided demonstrate the performance of our method.

Synthesizing Real World Stereo Challenges

Synthetic datasets for correspondence algorithm benchmarking recently gained more and more interest. The primary aim in its creation commonly has been to achieve highest possible realism for human observers which is regularly assumed to be the most important design target. But datasets must look realistic to the algorithm, not to the human observer. Therefore, we challenge the realism hypothesis in favor of posing specific, isolated and non-photorealistic problems to algorithms. There are three benefits: (i) Images can be created in large numbers at low cost. This addresses the currently largest problem in ground truth generation. (ii) We can combinatorially iterate through the design space to explore situations of highest relevance to the application. With increasing robustness of future stereo algorithms, datasets can be modified to increase matching challenges gradually. (iii) By isolating the core problems of stereo methods we can focus on each of them in turn. Our aim is not to produce a new dataset. Instead, we contribute with a new perspective on synthetic vision benchmark generation and show encouraging examples to validate our ideas. We believe that the potential of using synthetic data for evaluation in computer vision has not yet been fully utilized. Our first experiments demonstrate it is worthwhile to setup purpose designed datasets, as typical stereo failure can readily be reproduced, and thereby be better understood. Datasets are made available online [1].

Advance in vision-based driver assistance

Vision-based driver assistance is an active safety measure currently under development in various car companies and research institutes worldwide. The paper informs about related activities at The University of Auckland, focussing on stereo vision, performance evaluation, provided test data, and currently developed components.

Monocular 3D Reconstruction of Objects Based on Cylindrical Panoramas

This paper discusses ways of using a single panoramic image (captured by a rotating sensor-line camera having very-high spatial resolution) for the geometric shape recovery of a shown object. The objective is to create a sparse polyhedral model, only allowing a few interactive user inputs for a given single panoramic image. The study was motivated by the general question whether a single panoramic image projection allows some kind of 3D shape recovery, possibly benefitting from available monocular approaches for standard (say, pinhole-type) camera models.

Evaluation of stereo confidence measures on synthetic and recorded image data

We comparatively discuss a set of confidence measures for stereo analysis by testing them on semi-global matching (SGM) cost functions. The aim is a prediction of (potentially) erroneous areas in calculated disparity maps. The evaluation is done by using the sparsification technique which provides more information than commonly used RMS or NCC measures. We also present an approach for combining different confidence measures. This allows us to perform a quantisation of confidence estimates in terms of disparity errors.

Bilateral Filtering of 3D Point Clouds for Refined 3D Roadside Reconstructions

Stereo vision systems mounted on mobile platforms such as a road vehicles facilitate quick and inexpensive 3D reconstructions. However, reconstructed point clouds are noisy and appear unappealing in visualizations if not using shape priors or other specific ways of improvement. This paper describes a generic method to enhance the appearance of reconstructed meshes using bilateral filtering to smooth point clouds while preserving edge features and other small details. We demonstrate on synthetic and measured data that the proposed filtering process is beneficial for visualizations of noisy 3D data, and also for the preservation of the basic geometry of a real-world scene.

Current work in the .enpeda.. project

The environment perception and driver assistance (.enpeda..) project searches for solutions for vision-based driver assistance systems (DAS) which are currently starting to be active safety components of cars (e.g., lane departure warning, blind spot supervision). We review current projects in .enpeda.. in the international context of developments in this field.