Michel Goncalves Almeida Antunes

A Global Approach for the Detection of Vanishing Points and Mutually Orthogonal Vanishing Directions

This article presents a new global approach for detecting vanishing points and groups of mutually orthogonal vanishing directions using lines detected in images of man-made environments. These two multi-model fitting problems are respectively cast as Uncapacited Facility Location (UFL) and Hierarchical Facility Location (HFL) instances that are efficiently solved using a message passing inference algorithm. We also propose new functions for measuring the consistency between an edge and a putative vanishing point, and for computing the vanishing point defined by a subset of edges. Extensive experiments in both synthetic and real images show that our algorithms outperform the state-of-the-art methods while keeping computation tractable. In addition, we show for the first time results in simultaneously detecting multiple Manhattan-world configurations that can either share one vanishing direction (Atlanta world) or be completely independent.

Plane-based Odometry using an RGB-D Camera

Odometry consists in using data from a moving sensor to estimate change in position over time. It is a crucial step for several applications in robotics and computer vision. This paper presents a novel approach for estimating the relative motion between successive RGB-D frames that uses plane-primitives instead of point features. The planes in the scene are extracted and the motion estimation is cast as a plane-to-plane registration problem with a closed-form solution. Point features are only extracted in the cases where the plane surface configuration is insufficient to determine motion with no ambiguity. The initial estimate is refined in a photo-geometric optimization step that takes full advantage of the plane detection and simultaneous availability of depth and visual appearance cues. Extensive experiments show that our plane-based approach is as accurate as state-of-the-art point-based approaches when the camera displacement is small, and significantly outperforms them in case of wide-baseline and/or dynamic foreground.

Piecewise-Planar StereoScan:Structure and Motion from Plane Primitives

This article describes a pipeline that receives as input a sequence of images acquired by a calibrated stereo rig and outputs the camera motion and a Piecewise-Planar Reconstruction (PPR) of the scene. It firstly detects the 3D planes viewed by each stereo pair from semi-dense depth estimation. This is followed by estimating the pose between consecutive views using a new closed-form minimal algorithm that relies in point correspondences only when plane correspondences are insufficient to fully constrain the motion. Finally, the camera motion and the PPR are jointly refined, alternating between discrete optimization for generating plane hypotheses and continuous bundle adjustment. The approach differs from previous works in PPR by determining the poses from plane-primitives, by jointly estimating motion and piecewise-planar structure, and by operating sequentially, being suitable for applications of SLAM and visual odometry. Experiments are carried in challenging wide-baseline datasets where conventional point-based SfM usually fails.

Line-sweep: Cross-ratio for wide-baseline matching and 3D reconstruction

We propose a simple and useful idea based on cross-ratio constraint for wide-baseline matching and 3D reconstruction. Most existing methods exploit feature points and planes from images. Lines have always been considered notorious for both matching and reconstruction due to the lack of good line descriptors. We propose a method to generate and match new points using virtual lines constructed using pairs of keypoints, which are obtained using standard feature point detectors. We use cross-ratio constraints to obtain an initial set of new point matches, which are subsequently used to obtain line correspondences. We develop a method that works for both calibrated and uncalibrated camera configurations. We show compelling line-matching and large-scale 3D reconstruction.

SymStereo: Stereo Matching using Induced Symmetry

Stereo methods always require a matching function for assessing the likelihood of two pixels being in correspondence. Such functions, commonly referred as matching costs, measure the photo-similarity (or dissimilarity) between image regions centered in putative matches. This article proposes a new family of stereo cost functions that measure symmetry instead of photo-similarity for associating pixels across views. We start by observing that, given two stereo views and an arbitrary virtual plane passing in-between the cameras, it is possible to render image signals that are either symmetric or anti-symmetric with respect to the contour where the virtual plane meets the scene. The fact is investigated in detail and used as cornerstone to develop a new stereo framework that relies in symmetry cues for solving the data association problem. Extensive experiments in dense stereo show that our symmetry-based cost functions compare favorably against the best performing photo-similarity matching costs. In addition, we investigate the possibility of accomplishing Stereo Rangefinding that consists in using passive stereo to exclusively recover depth along a pre-defined scan plane. Thorough experiments provide evidence that stereo from induced symmetry is specially well suited for this purpose.

Can stereo vision replace a Laser Rangefinder

Many robotic systems combine cameras with Laser Rangefinders (LRF) for simultaneously achieving multi-purpose visual sensing and accurate depth recovery. Employing a single sensor modality for accomplishing both goals is an appealing proposition because it enables substantial savings in equipment, and tends to decrease the overall complexity of the system. This article explores the possibility of replacing LRF by passive stereo vision for reconstructing the scene along a 2D scan plane. We present a new stereo algorithm that is specifically tailored for the purpose. The algorithm recovers the depth along the scan plane using a symmetry-based matching cost (SymStereo), and refines the raw estimates by applying dynamic programming, followed by a Markov Random Field (MRF) that decides if the reconstructed contour is a line or not. We report for the first time comparative experiments between Stereo Rangefinding (SRF) and LRF. The results are encouraging by showing that SRF can be a plausible alternative to LRF in several application scenarios. Moreover, since SRF also enables independent depth estimates along multiple scan planes with arbitrary orientation, being the only constraint that the scan plane intersects the stereo baseline, it is an important benefit that can be decisive for many robotic applications.

Plane Surface Detection and Reconstruction using Induced Stereo Symmetry

We propose an algorithm for the detection and reconstruction of plane surfaces using a new stereo approach dubbed SymStereo. SymStereo relies in symmetry analysis for recovering the 3D curve where a virtual cut plane intersects the scene structure. The result is a profile cut that resembles the one that would be obtained by a Laser Range Finder (LRF). The article shows that the framework is particularly well suited for piecewiseplanar reconstruction using only a pair of calibrated views. Since the intersection of two planes is always a line, the 3D space is sampled by a discrete set of virtual planes and the line segments in the profile cuts are extracted. The plane surfaces are determined by grouping co-planar lines using a straightforward RANSAC procedure in the dual Plücker space. We test the algorithm in estimating the relative pose of the stereo rig with respect to planes with different textures. The results are highly accurate and, more importantly, the approach succeeds in situations where current stereo methods fail due to low and/or repetitive texture. We also report experiments in wide-baseline stereo images of complex scenes with multiple planes partially occluded by non-planar objects.

Stereo estimation of depth along virtual cut planes

Stereo vision is broadly employed in robotics and intelligent vehicles for recovering the 3D structure of the environment. The scene depth is typically estimated by triangulation after associating pixels between views using a dense stereo matching approach. In the last few years, the image resolution has steadily increased due to the advances in camera technology. Unfortunately, achieving real-time stereo using large size images is difficult because of the computational cost of dense matching. An obvious solution is to re-sample the acquired input images, but this implies decreasing the accuracy of depth estimates. We propose an alternative that consists in performing the stereo reconstruction of the contour C where a pre-defined virtual cut plane intersects the scene. This approach enables a trade-off between runtime and 3D model resolution that does not interfere with depth accuracy. The profile cuts C are independently recovered using the SymStereo framework that has been recently introduced in [1]. It is proved through comparative experiments that SymStereo is particularly well suited for recovering depth along virtual cut planes, outperforming state-of-the-art stereo cost functions both in terms of accuracy and runtime.

Using the GPU for Fast Symmetry-Based Dense Stereo Matching in High Resolution Images

SymStereo is a new algorithm used for stereo estimation. Instead of measuring photo-similarity, it proposes novel cost functions that measure symmetry for evaluating the likelihood of two pixels being a match. In this work we propose a parallel approach of the LogN matching cost variant of SymStereo capable of processing pairs of images in real-time for depth estimation. The power of the graphics processing units utilized allows exploring more efficiently the bank of log-Gabor wavelets developed to analyze symmetry, in the spectral domain. We analyze tradeoffs and propose different parameter-izations of the signal processing algorithm to accommodate image size, dimension of the filter bank, number of wavelets and also the number of disparities that controls the space density of the estimation, and still process up to 53 frames per second (fps) for images with size 288 × 384 and up to 3 fps for 768 × 1024 images.

Efficient Stereo Matching Using Histogram Aggregation with Multiple Slant Hypotheses

This paper presents an enhancement to the recent framework of histogram aggregation [1], that enables to improve the matching accuracy while preserving a low computational complexity. The original algorithm uses a fronto-parallel support window for cost aggregation, which leads to inaccurate results in the presence of significant surface slant. We address the problem by considering a pre-defined set of discrete orientation hypotheses for the aggregation window. It is shown that a single orientation hypothesis in the Disparity Space Image is usually representative of a large interval of possible 3D slants, and that handling slant in the disparity space has the advantage of avoiding visibility issues. We also propose a fast recognition scheme in the Disparity Space Image volume for selecting the most likely orientation hypothesis for aggregation. The experiments clearly prove the effectiveness of the approach.