Tobi Vaudrey

Efficient Dense Scene Flow from Sparse or Dense Stereo Data

This paper presents a technique for estimating the three-dimensional velocity vector field that describes the motion of each visible scene point (scene flow). The technique presented uses two consecutive image pairs from a stereo sequence. The main contribution is to decouple the position and velocity estimation steps, and to estimate dense velocities using a variational approach. We enforce the scene flow to yield consistent displacement vectors in the left and right images. The decoupling strategy has two main advantages: Firstly, we are independent in choosing a disparity estimation technique, which can yield either sparse or dense correspondences, and secondly, we can achieve frame rates of 5 fps on standard consumer hardware. The approach provides dense velocity estimates with accurate results at distances up to 50 meters.

Stereoscopic Scene Flow Computation for 3D Motion Understanding

Building upon recent developments in optical flow and stereo matching estimation, we propose a variational framework for the estimation of stereoscopic scene flow, i.e., the motion of points in the three-dimensional world from stereo image sequences. The proposed algorithm takes into account image pairs from two consecutive times and computes both depth and a 3D motion vector associated with each point in the image. In contrast to previous works, we partially decouple the depth estimation from the motion estimation, which has many practical advantages. The variational formulation is quite flexible and can handle both sparse or dense disparity maps. The proposed method is very efficient; with the depth map being computed on an FPGA, and the scene flow computed on the GPU, the proposed algorithm runs at frame rates of 20 frames per second on QVGA images (320×240 pixels). Furthermore, we present solutions to two important problems in scene flow estimation: violations of intensity consistency between input images, and the uncertainty measures for the scene flow result.

Differences between stereo and motion behaviour on synthetic and real-world stereo sequences

Performance evaluation of stereo or motion analysis techniques is commonly done either on synthetic data where the ground truth can be calculated from ray-tracing principals, or on engineered data where ground truth is easy to estimate. Furthermore, these scenes are usually only shown in a very short sequence of images. This paper shows why synthetic scenes may not be the only testing criteria by giving evidence of conflicting results of disparity and optical flow estimation for real-world and synthetic testing. The data dealt with in this paper are images taken from a moving vehicle. Each real-world sequence contains 250 image pairs or more. Synthetic driver assistance scenes (with ground truth) are 100 or more image pairs. Particular emphasis is paid to the estimation and evaluation of scene flow on the synthetic stereo sequences. All image data used in this paper is made publicly available at http: //www.mi.auckland.ac.nz/EISATS.

Performance of Correspondence Algorithms in Vision-Based Driver Assistance Using an Online Image Sequence Database

This paper discusses options for testing correspondence algorithms in stereo or motion analysis that are designed or considered for vision-based driver assistance. It introduces a globally available database, with a main focus on testing on video sequences of real-world data. We suggest the classification of recorded video data into situations defined by a cooccurrence of some events in recorded traffic scenes. About 100-400 stereo frames (or 4-16 s of recording) are considered a basic sequence, which will be identified with one particular situation. Future testing is expected to be on data that report on hours of driving, and multiple hours of long video data may be segmented into basic sequences and classified into situations. This paper prepares for this expected development. This paper uses three different evaluation approaches (prediction error, synthesized sequences, and labeled sequences) for demonstrating ideas, difficulties, and possible ways in this future field of extensive performance tests in vision-based driver assistance, particularly for cases where the ground truth is not available. This paper shows that the complexity of real-world data does not support the identification of general rankings of correspondence techniques on sets of basic sequences that show different situations. It is suggested that correspondence techniques should adaptively be chosen in real time using some type of statistical situation classifiers.

Moving Object Segmentation Using Optical Flow and Depth Information

This paper discusses the detection of moving objects (being a crucial part of driver assistance systems) using monocular or stereoscopic computer vision. In both cases, object detection is based on motion analysis of individually tracked image points (optical flow), providing a motion metric which corresponds to the likelihood that the tracked point is moving. Based on this metric, points are segmented into objects by employing a globally optimal graph-cut algorithm. Both approaches are comparatively evaluated using real-world vehicle image sequences.

New Lane Model and Distance Transform for Lane Detection and Tracking

Particle filtering of boundary points is a robust way to estimate lanes. This paper introduces a new lane model in correspondence to this particle filter-based approach, which is flexible to detect all kinds of lanes. A modified version of an Euclidean distance transform is applied to an edge map of a road image from a birds-eye view to provide information for boundary point detection. An efficient lane tracking method is also discussed. The use of this distance transform exploits useful information in lane detection situations, and greatly facilitates the initialization of the particle filter, as well as lane tracking. Finally, the paper validates the algorithm with experimental evidence for lane detection and tracking.

Robustness evaluation of stereo algorithms on long stereo sequences

This paper presents an approach to test stereo algorithms against long stereo sequences (say, 100+ image pairs). Stereo sequences of this length have not been quantitatively evaluated in the past, even though they are the input data of a vision-based driver assistance system. Using stereo sequences allows one to exploit the temporal information, which is, in general, not well used currently. The presented approach focuses on evaluating the robustness of algorithms against differing noise parameters (Gaussian noise, brightness differences, and blurring).

Low-Level Image Processing for Lane Detection and Tracking

Lane detection and tracking is a significant component of vision-based driver assistance systems (DAS). Low-level image processing is the first step in such a component. This paper suggests three useful techniques for low-level image processing in lane detection situations: bird’s-eye view mapping, a specialized edge detection method, and the distance transform. The first two techniques have been widely used in DAS, while the distance transform is a method newly exploited in DAS, that can provide useful information in lane detection situations. This paper recalls two methods to generate a bird’s-eye image from the original input image, it also compares edge detectors. A modified version of the Euclidean distance transform called real orientation distance transform (RODT) is proposed. Finally, the paper discusses experiments on lane detection and tracking using these technologies.

A Study on Stereo and Motion Data Accuracy for a Moving Platform

Stereo and motion analysis are potential techniques for providing information for control or assistance systems in various robotics or driver assistance applications. This paper evaluates the performance of several stereo and motion algorithms over a long synthetic sequence (100 stereo pairs). Such an evaluation of low-level computer vision algorithms is necessary, as moving platforms are being used for image analysis in a wide area of applications. In this paper algorithms are evaluated with respect to robustness by modifying the test sequence with various types of realistic noise. The novelty of this paper is comparing top performing algorithms on a long sequence of images, taken from a moving platform.

Integrating disparity images by incorporating disparity rate

Intelligent vehicle systems need to distinguish which objects are moving and which are static. A static concrete wall lying in the path of a vehicle should be treated differently than a truck moving in front of the vehicle. This paper proposes a new algorithm that addresses this problem, by providing dense dynamic depth information, while coping with real-time constraints. The algorithm models disparity and disparity rate pixel-wise for an entire image. This model is integrated over time and tracked by means of many pixel-wise Kalman filters. This provides better depth estimation results over time, and also provides speed information at each pixel without using optical flow. This simple approach leads to good experimental results for real stereo sequences, by showing an improvement over previous methods.