Pascal Monasse

Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion

Multi-view structure from motion (SfM) estimates the position and orientation of pictures in a common 3D coordinate frame. When views are treated incrementally, this external calibration can be subject to drift, contrary to global methods that distribute residual errors evenly. We propose a new global calibration approach based on the fusion of relative motions between image pairs. We improve an existing method for robustly computing global rotations. We present an efficient a contrario trifocal tensor estimation method, from which stable and precise translation directions can be extracted. We also define an efficient translation registration method that recovers accurate camera positions. These components are combined into an original SfM pipeline. Our experiments show that, on most datasets, it outperforms in accuracy other existing incremental and global pipelines. It also achieves strikingly good running times: it is about 20 times faster than the other global method we could compare to, and as fast as the best incremental method. More importantly, it features better scalability properties.

OpenMVG: Open Multiple View Geometry

The OpenMVG C++ library provides a vast collection of multiple-view geometry tools and algorithms to spread the usage of computer vision and structure-from-motion techniques. Close to the state-of-the-art in its domain, it provides an easy access to common tools used in 3D reconstruction from images. Following the credo “Keep it simple, keep it maintainable” the library is designed as a modular collection of algorithms, libraries and binaries that can be used independently or as bricks to build larger systems. Thanks to its strict test driven development, the library is packaged with unit-test code samples that make the library easy to learn, modify and use. Since its first release in 2013 under the MPL2 license, OpenMVG has gathered an active community of users and contributors from many fields, spanning hobbyists, students, computer vision experts, and industry members.

A Precision Analysis of Camera Distortion Models

This paper addresses the question of identifying the right camera direct or inverse distortion model, permitting a high subpixel precision to fit to real camera distortion. Five classic camera distortion models are reviewed and their precision is compared for direct or inverse distortion. By definition, the three radially symmetric models can only model a distortion radially symmetric around some distortion center. They can be extended to deal with non-radially symmetric distortions by adding tangential distortion components, but still may be too simple for very accurate modeling of real cameras. The polynomial and the rational models instead miss a physical or optical interpretation, but can cope equally with radially and non-radially symmetric distortions. Indeed, they do not require the evaluation of a distortion center. When requiring high precisions, we found that the distortion modeling must also be evaluated primarily as a numerical problem. Indeed, all models except the polynomial involve a non-linear minimization, which increases the numerical risk. The estimation of a polynomial distortion model leads instead to a linear problem, which is secure and much faster. We concluded by extensive numerical experiments that, although high degree polynomials were required to reach a high precision of 1/100 pixels, such polynomials were easily estimated and produced a precise distortion modeling without overfitting. Our conclusion is validated by three independent experimental setups: the models were compared first on the lens distortion database of the Lensfun library by their distortion simulation and inversion power; second by fitting real camera distortions estimated by a non parametric algorithm; and finally by the absolute correction measurement provided by the photographs of tightly stretched strings, warranting a high straightness.

Multiscale line segment detector for robust and accurate SfM

We propose a multiscale extension of a well-known line segment detector, LSD. We show that its multiscale nature makes it much less prone to over-segmentation, more robust to low contrast and less sensitive to noise, while keeping the parameterless advantage of LSD and still being fast. Moreover, we show that in scenes with little or no feature points, but where it is however possible to perform structure from motion from matched line segments, the accuracy is significantly improved. This provides an objective and automatic quantitative assessment of our detector that goes much beyond the usual qualitative visual inspection found in the literature.

Camera Matrix Calibration Using Circular Control Points and Separate Correction of the Geometric Distortion Field

We achieve a precise camera calibration using circular control points by, first, separation of the lens distortion parameters from other camera parameters and computation of the distortion field in advance by using a calibration harp. Second, in order to compensate for perspective bias, which is prone to occur when using a circled pattern, we incorporate conic affine transformation into the minimization error when estimating the homography, and leave all the other calibration steps as they are used in the literature. Such an error function allows to compensate for the perspective bias. Combined with precise key point detection, the approach is shown to be more stable than current state-of-the-art global calibration method.

A Critical Review of the Trifocal Tensor Estimation.

We explore the advantages offered by the trifocal tensor in the pose estimation of a triplet of cameras as opposed to computing the relative poses pair by pair with the fundamental matrix. Theoretically, the trinilearities characterize uniquely three corresponding image points in a tighter way than the three epipolar equations and this translates in an increasing accuracy. However, we show that this initial improvement is not enough to have a remarkable impact on the pose estimation after bundle adjustment, and the use of the fundamental matrix with image triplets remains relevant.

The Multiscale Line Segment Detector

We propose a multiscale extension of a well-known line segment detector, LSD. We show that its multiscale nature makes it much less susceptible to over-segmentation and more robust to low contrast and less sensitive to noise, while keeping the parameter-less advantage of LSD and still being fast. We also present here a dense gradient filter that disregards regions in which lines are likely to be irrelevant. As it reduces line mismatches, this filter improves the robustness of the application to structure-from-motion. It also yields a faster detection.

The IPOL Demo System: A Scalable Architecture of Microservices for Reproducible Research

We identified design problems related to the architecture, ergonomy, and performance in the previous version of the Image Processing on Line (IPOL) demonstration system. In order to correct them we moved to an architecture of microservices and performed many refactorings. This article first describes the state of the art in Reproducible Research platforms and explains IPOL in that context. The specific problems which were found are discussed, along with the solutions implemented in the new demo system, and the changes in its architecture with respect to the previous system. Finally, we expose the challenges of the system in the short term.