Bogdan Matei

A general method for Errors-in-Variables problems in computer vision

The Errors-in-Variables (EIV) model from statistics is often employed in computer vision though only rarely under this name. In an EIV model all the measurements are corrupted by noise while the a priori information is captured with a nonlinear constraint among the true (unknown) values of these measurements. To estimate the model parameters and the uncorrupted data, the constraint can be linearized, i.e., embedded in a higher dimensional space. We show that linearization introduces data-dependent (heteroscedastic) noise and propose an iterative procedure, the heteroscedastic EIV (HEIV) estimator to obtain consistent estimates in the most general, multivariate case. Analytical expressions for the covariances of the parameter estimates and corrected data points, a generic method for the enforcement of ancillary constraints arising from the underlying geometry are also given. The HEIV estimator minimizes the first order approximation of the geometric distances between the measurements and the true data points, and thus can be a substitute for the widely used Levenberg-Marquardt based direct solution of the original nonlinear problem. The HEIV estimator has however the advantage of a weaker dependence on the initial solution and a faster convergence. In comparison to Kanatanis renormalization paradigm (an earlier solution of the same problem) the HEIV estimator has more solid theoretical foundations which translate into better numerical behavior We show that the HEIV estimator can provide an accurate solution to most 3D vision estimation tasks, and illustrate its performance through two case studies: calibration and the estimation of the fundamental matrix.

Building segmentation for densely built urban regions using aerial LIDAR data

We present a novel building segmentation system for densely built areas, containing thousands of buildings per square kilometer. We employ solely sparse LIDAR (Light/Laser Detection Ranging) 3D data, captured from an aerial platform, with resolution less than one point per square meter. The goal of our work is to create segmented and delineated buildings as well as structures on top of buildings without requiring scanning for the sides of buildings. Building segmentation is a critical component in many applications such as 3D visualization, robot navigation and cartography. LIDAR has emerged in recent years as a more robust alternative to 2D imagery because it acquires 3D structure directly, without the shortcomings of stereo in un- textured regions and at depth discontinuities. Our main technical contributions in this paper are: (i) a ground segmentation algorithm which can handle both rural regions, and heavily urbanized areas, where the ground is 20% or less of the data, (ii) a building segmentation technique, which is robust to buildings in close proximity to each other, sparse measurements and nearby structured vegetation clutter, and (Hi) an algorithm for estimating the orientation of a boundary contour of a building, based on minimizing the number of vertices in a rectilinear approximation to the building outline, which can cope with significant quantization noise in the outline measurements. We have applied the proposed building segmentation system to several urban regions with areas of hundreds of square kilometers each, obtaining average segmentation speeds of less than three minutes per km2 on a standard Pentium processor. Extensive qualitative results obtained by overlaying the 3D segmented regions onto 2D imagery indicate accurate performance of our system.

Rapid object indexing using locality sensitive hashing and joint 3D-signature space estimation

We propose a new method for rapid 3D object indexing that combines feature-based methods with coarse alignment-based matching techniques. Our approach achieves a sublinear complexity on the number of models, maintaining at the same time a high degree of performance for real 3D sensed data that is acquired in largely uncontrolled settings. The key component of our method is to first index surface descriptors computed at salient locations from the scene into the whole model database using the locality sensitive hashing (LSH), a probabilistic approximate nearest neighbor method. Progressively complex geometric constraints are subsequently enforced to further prune the initial candidates and eliminate false correspondences due to inaccuracies in the surface descriptors and the errors of the LSH algorithm. The indexed models are selected based on the MAP rule using posterior probability of the models estimated in the joint 3D-signature space. Experiments with real 3D data employing a large database of vehicles, most of them very similar in shape, containing 1,000,000 features from more than 365 models demonstrate a high degree of performance in the presence of occlusion and obscuration, unmodeled vehicle interiors and part articulations, with an average processing time between 50 and 100 seconds per query

Estimation of Nonlinear Errors-in-Variables Models for Computer Vision Applications

In an errors-in-variables (EIV) model, all the measurements are corrupted by noise. The class of EIV models with constraints separable into the product of two nonlinear functions, one solely in the variables and one solely in the parameters, is general enough to represent most computer vision problems. We show that the estimation of such nonlinear EIV models can be reduced to iteratively estimating a linear model having point dependent, i.e., heteroscedastic, noise process. Particular cases of the proposed heteroscedastic errors-in-variables (HEIV) estimator are related to other techniques described in the vision literature: the Sampson method, renormalization, and the fundamental numerical scheme. In a wide variety of tasks, the HEIV estimator exhibits the same, or superior, performance as these techniques and has a weaker dependence on the quality of the initial solution than the Levenberg-Marquardt method, the standard approach toward estimating nonlinear models

Video georegistration: algorithm and quantitative evaluation

An algorithm is presented for video georegistration, with a particular concern for aerial video, i.e., video captured from an airborne platform. The algorithms input is a video stream with telemetry (camera model specification sufficient to define an initial estimate of the view) and geodetically calibrated reference imagery (coaligned digital orthoimage and elevation map). The output is a spatial registration of the video to the reference so that it inherits the available geodetic coordinates. The video is processed in a continuous fashion to yield a corresponding stream of georegistered results. Quantitative results of evaluating the developed approach with real world aerial video also are presented. The results suggest that the developed approach may provide valuable input to the analysis and interpretation of aerial video.

Optimal rigid motion estimation and performance evaluation with bootstrap

A new method for 3D rigid motion estimation is derived under the most general assumption that the measurements are corrupted by inhomogeneous and anisotropic, i.e., heteroscedastic noise. This is the case, for example, when the motion of a calibrated stereo-head is to be determined from image pairs. Linearization in the quaternion space transforms the problem into a multivariate, heteroscedastic errors-in-variables (HEIV) regression, from which the rotation and translation estimates are obtained simultaneously. The significant performance improvement is illustrated, for real data, by comparison with the results of quaternion, subspace and renormalization based approaches described in the literature. Extensive use as made of bootstrap, an advanced numerical tool from statistics, both to estimate the covariances of the 3D data points and to obtain confidence regions for the rotation and translation estimates. Bootstrap enables an accurate recovery of these information using only the two image pairs serving as input.

Shapeme histogram projection and matching for partial object recognition

Histograms of shape signature or prototypical shapes, called shapemes, have been used effectively in previous work for 2D/3D shape matching and recognition. We extend the idea of shapeme histogram to recognize partially observed query objects from a database of complete model objects. We propose representing each model object as a collection of shapeme histograms and match the query histogram to this representation in two steps: 1) compute a constrained projection of the query histogram onto the subspace spanned by all the shapeme histograms of the model and 2) compute a match measure between the query histogram and the projection. The first step is formulated as a constrained optimization problem that is solved by a sampling algorithm. The second step is formulated under a Bayesian framework, where an implicit feature selection process is conducted to improve the discrimination capability of shapeme histograms. Results of matching partially viewed range objects with a 243 model database demonstrate better performance than the original shapeme histogram matching algorithm and other approaches.

Linear model hashing and batch RANSAC for rapid and accurate object recognition

This paper proposes a joint feature-based model indexing and geometric constraint based alignment pipeline for efficient and accurate recognition of 3D objects from a large model database. Traditional approaches either first prune the model database using indexing without geometric alignment or directly perform recognition based alignment. The indexing based pruning methods without geometric constraints can miss the correct models under imperfections such as noise, clutter and obscurations. Alignment based verification methods have to linearly verify each model in the database and hence do not scale up. The proposed techniques use spin images as semi-local shape descriptors and locality-sensitive hashing (LSH) to index into a joint spin image database for all the models. The indexed models represented in the pruned set are further pruned using progressively complex geometric constraints. A simple geometric configuration of multiple spin images, for instance a doublet, is first used to check for geometric consistency. Subsequently, full Euclidean geometric constraints are applied using RANSAC-based techniques on the pruned spin images and the models to verify specific object identity. As a result, the combined indexing and geometric alignment based pipeline is able to focus on matching the most promising models, and generate far less pose hypotheses while maintaining the same level of performance as the sequential alignment based recognition. Furthermore, compared to geometric indexing techniques like geometric hashing, the construction time and storage complexity for the proposed technique remains linear in the number of features rather than higher order polynomial. Experiments on a 56 3D model database show promising results.

Reduction of bias in maximum likelihood ellipse fitting

An improved maximum likelihood estimator for ellipse fitting based on the heteroscedastic errors-in-variables (HEIV) regression algorithm is proposed. The technique significantly reduces the bias of the parameter estimates present in the direct least squares method, while it is numerically more robust than renormalization, and requires less computations than minimizing the geometric distance with the Levenberg-Marquardt optimization procedure. The HEIV algorithm also provides closed-form expressions for the covariances of the ellipse parameters and corrected data points. The quality of the different solutions is assessed by defining confidence regions in the input domain, either analytically or by bootstrap. The latter approach is exclusively data driven and it is used whenever the expression of the covariance for the estimates is not available.

A real-time pedestrian detection system based on structure and appearance classification

We present a real-time pedestrian detection system based on structure and appearance classification. We discuss several novel ideas that contribute to having low-false alarms and high detection rates, while at the same time achieving computational efficiency: (i) At the front end of our system we employ stereo to detect pedestrians in 3D range maps using template matching with a representative 3D shape model, and to classify other background objects in the scene such as buildings, trees and poles. The structure classification efficiently labels substantial amount of non-relevant image regions and guides the further computationally expensive process to focus on relatively small image parts; (ii)We improve the appearance-based classifiers based on HoG descriptors by performing template matching with 2D human shape contour fragments that results in improved localization and accuracy; (iii) We build a suite of classifiers tuned to specific distance ranges for optimized system performance. Our method is evaluated on publicly available datasets and is shown to match or exceed the performance of leading pedestrian detectors in terms of accuracy as well as achieving real-time computation (10 Hz), which makes it adequate for in-vehicle navigation platform.