Daniel Barath

Novel Methods for Estimating Surface Normals from Affine Transformations

The aim of this paper is to describe different estimation techniques in order to deal with point-wise surface normal estimation from calibrated stereo configuration. We show here that the knowledge of the affine transformation between two projections is sufficient for computing the normal vector unequivocally. The formula which describes the relationship among the cameras, normal vectors and affine transformations is general, since it works for every kind of cameras, not only for the pin-hole one. However, as it is proved in this study, the normal estimation can optimally be solved for the perspective camera. Other non-optimal solutions are also proposed for the problem. The methods are tested both on synthesized data and real-world images. The source codes of the discussed algorithms are available on the web.

A Minimal Solution for Two-View Focal-Length Estimation Using Two Affine Correspondences

A minimal solution using two affine correspondences is presented to estimate the common focal length and the fundamental matrix between two semi-calibrated cameras – known intrinsic parameters except a common focal length. To the best of our knowledge, this problem is unsolved. The proposed approach extends point correspondence-based techniques with linear constraints derived from local affine transformations. The obtained multivariate polynomial system is efficiently solved by the hidden-variable technique. Observing the geometry of local affinities, we introduce novel conditions eliminating invalid roots. To select the best one out of the remaining candidates, a root selection technique is proposed outperforming the recent ones especially in case of high-level noise. The proposed 2-point algorithm is validated on both synthetic data and 104 publicly available real image pairs. A Matlab implementation of the proposed solution is included in the paper.

Multi-H: Efficient Recovery of Tangent Planes in Stereo Images

Multi-H – an efficient method for the recovery of the tangent planes of a set of point correspondences satisfying the epipolar constraint is proposed. The problem is formulated as a search for a labeling minimizing an energy that includes a data and spatial regularization terms. The number of planes is controlled by a combination of MeanShift [6] and α-expansion [3]. Experiments on the fountain-P11 3D dataset show that Multi-H provides highly accurate tangent plane estimates. It also outperforms all state-of-the-art techniques for multihomography estimation on the publicly available AdelaideRMF dataset. Since Multi-H achieves nearly error-free performance, we introduce and make public a more challenging dataset for multi-plane fitting evaluation.

Novel Ways to Estimate Homography from Local Affine Transformations

State-of-the-art 3D reconstruction methods usually apply point correspondences in order to compute the 3D geometry of objects represented by dense point clouds. However, objects with relatively large and flat surfaces can be most accurately reconstructed if the homographies between the corresponding patches are known. Here we show how the homography between patches on a stereo image pair can be estimated. We discuss that these proposed estimators are more accurate than the widely used point correspondence-based techniques because the latter ones only consider the last column (the translation) of the affine transformations, whereas the new algorithms use all the affine parameters. Moreover, we prove that affine-invariance is equivalent to perspectiveinvariance in the case of known epipolar geometry. Three homography estimators are proposed. The first one calculates the homography if at least two point correspondences and the related affine transformations are known. The second one computes the homography from only one point pair, if the epipolar geometry is estimated beforehand. These methods are solved by linearization of the original equations, and the refinements can be carried out by numerical optimization. Finally, a hybrid homography estimator is proposed that uses both point correspondences and photo-consistency between the patches. The presented methods have been quantitatively validated on synthesized tests. We also show that the proposed methods are applicable to realworld images as well, and they perform better than the state-of-the-art point correspondence-based techniques.

Accurate Closed-form Estimation of Local Affine Transformations Consistent with the Epipolar Geometry

For a pair of images satisfying the epipolar constraint, a method for accurate estimation of local affine transformations is proposed. The method returns the local affine transformation consistent with the epipolar geometry that is closest in the least squares sense to the initial estimate provided by an affine-covariant detector. The minimized L2 norm of the affine matrix elements is found in closed-form. We show that the used norm has an intuitive geometric interpretation. The method, with negligible computational requirements, is validated on publicly available benchmarking datasets and on synthetic data. The accuracy of the local affine transformations is improved for all detectors and all image pairs. Implicitly, precision of the tested feature detectors was compared. The Hessian-Affine detector combined with ASIFT view synthesis was the most accurate.

Optimal Surface Normal from Affine Transformation

This paper deals with surface normal estimation from calibrated stereo images. We show here how the affine transformation between two projections defines the surface normal of a 3D planar patch. We give a formula that describes the relationship of surface normals, camera projections, and affine transformations. This formula is general since it works for every kind of cameras. We propose novel methods for estimating the normal of a surface patch if the affine transformation is known between two perspective images. We show here that the normal vector can be optimally estimated if the projective depth of the patch is known. Other non-optimal methods are also introduced for the problem. The proposed methods are tested both on synthesized data and images of real-world 3D objects.

A theory of point-wise homography estimation

Estimating homography using only one affine correspondence.The proposed theory makes multi-homography estimation less ambiguous.Stochastic sampling can be omitted from robust homography estimation.Affine-covariant detectors are compared w.r.t. quality of estimated homographies.Equivalence of affine and perspective-invariances for known epipolar geometry. We propose a method, called HAF, to estimate planar homography from an affine correspondence satisfying the epipolar constraint in an image pair. An affine correspondence consists of a point pair and the related local affine transformation mapping the pixels infinitely close to the point locations from the first to the second images. As a minimal solver, it estimates the homography from only one correspondence, however, it is generalized for the over-determined case as well. The required local affinities are obtained by affine-covariant feature detectors accurately. As a side-effect of the tests, the state-of-the-art affine-covariant detectors are compared to each other w.r.t. the accuracy of the estimated point-wise homographies. The proposed method is validated both on the publicly available AdelaideRMF dataset and in a synthetic testing environment.

A Novel Technique for Point-wise Surface Normal Estimation

Nowadays multi-view stereo reconstruction algorithms can achieve impressive results using many views of the scene. Our primary objective is to robustly extract more information about the underlying surface from fewer images. We present a method for point-wise surface normal and tangent plane estimation in stereo case to reconstruct real-world scenes. The proposed algorithm works for general camera model, however, we choose the pinhole-camera in order to demonstrate its efficiency. The presented method uses particle swarm optimization under geometric and epipolar constraints in order to achieve suitable speed and quality. An oriented point cloud is generated using a single point correspondence for each oriented 3D point and a cost function based on photo-consistency. It can straightforwardly be extended to multi-view reconstruction. Our method is validated in both synthesized and real tests. The proposed algorithm is compared to one of the state-of-the-art patch-based multi-view reconstruction algorithms.

Approximate Epipolar Geometry from Six Rotation Invariant Correspondences.

We propose a method for estimating an approximate fundamental matrix from six rotation invariant feature correspondences exploiting their rotation components, e.g. provided by SIFT or ORB detectors. The cameras are not calibrated. First, a linear sub-space is calculated from the point coordinates, then the rotations are used assuming orthographic projection. It is demonstrated that combining the proposed method with Graph-cut RANSAC makes it superior to the state-of-the-art in terms of accuracy for tasks requiring a strict time limit. These tasks are practically the ones which need to be done real time. We tested the method on 203 publicly available real image pairs.

Efficient energy-based topological outlier rejection

Abstract An approach is proposed for outlier rejection from a set of 2D point correspondences which does not require any underlying models, e.g. fundamental matrix. The solution is obtained by minimizing an energy originated from the neighborhood-graphs in both images using a grab-cut-like algorithm: iterated graph-cut and re-fitting. The method is validated on publicly available datasets, it is real time for most of the problems and achieves more accurate results than RANSAC and its state-of-the-art variants in terms of outlier rejection ratio. It is applicable to scenes where a single fundamental matrix is not estimable, e.g. non-rigid or degenerate ones.