Simon Hermann

Iterative semi-global matching for robust driver assistance systems

Semi-global matching (SGM) is a technique of choice for dense stereo estimation in current industrial driver-assistance systems due to its real-time processing capability and its convincing performance. In this paper we introduce iSGM as a new cost integration concept for semi-global matching. In iSGM, accumulated costs are iteratively evaluated and intermediate disparity results serve as input to generate semi-global distance maps. This novel data structure supports fast analysis of spatial disparity information and allows for reliable search space reduction in consecutive cost accumulation. As a consequence horizontal costs are stabilized which improves the robustness of the matching result. We demonstrate the superiority of this iterative integration concept against a standard configuration of semi-global matching and compare our results to current state-of-the-art methods on the KITTI Vision Benchmark Suite.

A Comparative Study on 2D Curvature Estimators

Curvature is a frequently used property in two-dimensional (2D) shape analysis, directly or for derived features such as corners or convex and concave arcs. This paper presents curvature estimators which follow approaches in differential geometry. Digital-straight segment approximation (as known from digital geometry) is used in those estimators. Results of multigrid experiments are evaluated leading to a comparative performance analysis of several curvature estimators

Inclusion of a Second-Order Prior into Semi-Global Matching

Todays stereo vision algorithms and computing technology allow real-time 3D data analysis, for example for driver assistance systems. A recently developed Semi-Global Matching (SGM) approach by H. Hirschmuller became a popular choice due to performance and robustness. This paper evaluates different parameter settings for SGM, and its main contribution consists in suggesting to include a second order prior into the smoothness term of the energy function. It also proposes and tests a new cost function for SGM. Furthermore, some preprocessing (edge images) proved to be of great value for improving SGM stereo results on real-world sequences, as previously already shown by S. Guan and R. Klette for belief propagation. There is also a performance gain for engineered stereo data (e.g.) as currently used on the Middlebury stereo website. However, the fact that results are not as impressive as on the .enpeda.. sequences indicates that optimizing for engineered data does not neccessarily improve real world stereo data analysis.

Hierarchical scan-line dynamic programming for optical flow using semi-global matching

Dense and robust optical flow estimation is still a major challenge in low-level computer vision. In recent years, mainly variational methods contributed to the progress in this field. One reason for their success is their suitability to be embedded into hierarchical schemes, which makes them capable of handling large pixel displacements. Matching-based regularization techniques, like dynamic programming or belief propagation concepts, can also lead to accurate optical flow fields. However, results are limited to short- or mid-scale optical flow vectors, because these techniques are usually not combined with coarse-to-fine strategies. This paper introduces fSGM, a novel algorithm that is based on scan-line dynamic programming. It uses the cost integration strategy of semi-global matching, a concept well known in the area of stereo matching. The major novelty of fSGM is that it embeds the scan-line dynamic programming approach into a hierarchical scheme, which allows it to handle large pixel displacements with an accuracy comparable to variational methods. We prove the exceptional performance of fSGM by comparing it to current state-of-the-art methods on the KITTI Vision Benchmark Suite.

Illumination invariant cost functions in semi-global matching

The paper evaluates three categories of similarity measures: ordering-based (census), gradient-based, and illumination-based cost functions. The performance of those functions is evaluated especially with respect to illumination changes using two different sets of data, also including real world driving sequences of hundreds of stereo frames with strong illumination differences. The overall result is that there are cost functions in all three categories that can perform well on a quantitative and qualitative level. This leads to the assumption that those cost functions are in fact closely related at a qualitative level, and we provide our explanation.

The gradient - A powerful and robust cost function for stereo matching

Using gradient information for a pixel-based cost function for stereo matching has lacked adequate attention in the literature. This paper provides experimental evidence to show that the gradient as a data descriptor outperforms other pixel-based functions such as absolute differences and the Birchfield and Tomasi cost functions. The cost functions are tested against stereo image datasets where ground truth data is available. Furthermore, analysing the effect of the cost functions when exposure and illumination settings are different between the left and right camera is analysed. Not only has the performance of the cost functions been analysed, but also analysis into “why” one cost function is better than another. The analysis tests the global and spacial optimality of the cost function, showing that the gradient information returns stronger minima than the other two. These results are aimed at future research towards the design of a new smoothness prior that also depends on the characteristics of the employed cost function. This paper shows that the gradient is a simple, yet powerful, data descriptor that shows robustness to illumination and exposure differences, but is often overlooked by the stereo community.

A comparative study of stereo-matching algorithms for road-modeling in the presence of windscreen wipers

In this study we examine three road-modeling methods, which we evaluate on seven stereo matching algorithms. The road-modeling methods we consider are a B-spline modeling technique based on region-growing and two versions of the popular v-disparity approach. The used stereo algorithms are variations or different parameterizations of belief propagation, graph cut and semi-global matching.

Evaluation of a new coarse-to-fine strategy for fast semi-global stereo matching

The paper considers semi-global stereo matching in the context of vision-based driver assistance systems. The need for real-time performance in this field requires a design change of the originally proposed method to run on current hardware. This paper proposes such a new design; the novel strategy first generates a disparity map from half-resolution input images. The result is then used as prior to restrict the disparity search space for full-resolution computation. This approach is compared to an SGM strategy as employed currently in a state-of-the-art real-time FPGA solution. Furthermore, trinocular stereo evaluation is performed on ten real-world traffic sequences with a total of 4,000 trinocular frames. An extension to the original evaluation methodology is proposed to resolve ambiguities and to incorporate disparity density in a statistically meaningful way. Evaluation results indicate that the novel SGM method is up to 40% faster when compared to the previous strategy. It returns denser disparity maps, and is also more accurate on evaluated traffic scenes.

Mid-level segmentation and segment tracking for long-range stereo analysis

This paper presents a novel way of combining dense stereo and motion analysis for the purpose of mid-level scene segmentation and object tracking. The input is video data that addresses long-range stereo analysis, as typical when recording traffic scenes from a mobile platform. The task is to identify shapes of traffic-relevant objects without aiming at object classification at the considered stage. We analyse disparity dynamics in recorded scenes for solving this task. Statistical shape models are generated over subsequent frames. Shape correspondences are established by using a similarity measure based on set theory. The motion of detected shapes (frame to frame) is compensated by using a dense motion field as produced by a real-time optical flow algorithm. Experimental results show the quality of the proposed method which is fairly simple to implement.

Real-World stereo-analysis evaluation

Evaluation of stereo-analysis algorithms is usually done by analysing the performance of stereo matchers on data sets with available ground truth. The trade-off between precise results, obtained with this sort of evaluation, and the limited amount (in both, quantity and diversity) of data sets, needs to be considered if the algorithms are required to analyse real-world environments. This chapter discusses a technique to objectively evaluate the performance of stereo-analysis algorithms using real-world image sequences. The lack of ground truth is tackled by incorporating an extra camera into a multi-view stereo camera system. The relatively simple hardware set-up of the proposed technique can easily be reproduced for specific applications.