Peter F. Sturm

Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction

In this paper sequences of camera motions that lead to inherent ambiguities in uncalibrated Euclidean reconstruction or self-calibration are studied. Our main contribution is a complete, detailed classification of these critical motion sequences (CMS). The practically important classes are identified and their degrees of ambiguity are derived. We also discuss some practical issues, especially concerning the reduction of the ambiguity of a reconstruction.

Gradient Response Maps for Real-Time Detection of Textureless Objects

We present a method for real-time 3D object instance detection that does not require a time-consuming training stage, and can handle untextured objects. At its core, our approach is a novel image representation for template matching designed to be robust to small image transformations. This robustness is based on spread image gradient orientations and allows us to test only a small subset of all possible pixel locations when parsing the image, and to represent a 3D object with a limited set of templates. In addition, we demonstrate that if a dense depth sensor is available we can extend our approach for an even better performance also taking 3D surface normal orientations into account. We show how to take advantage of the architecture of modern computers to build an efficient but very discriminant representation of the input images that can be used to consider thousands of templates in real time. We demonstrate in many experiments on real data that our method is much faster and more robust with respect to background clutter than current state-of-the-art methods.

Structure-from-motion using lines: Representation, triangulation, and bundle adjustment

We address the problem of camera motion and 3D structure reconstruction from line correspondences across multiple views, from initialization to final bundle adjustment. One of the main difficulties when dealing with line features is their algebraic representation. First, we consider the triangulation problem. Based on Plucker coordinates to represent the 3D lines, we propose a maximum likelihood algorithm, relying on linearizing the Plucker constraint and on a Plucker correction procedure, computing the closest Plucker coordinates to a given 6-vector. Second, we consider the bundle adjustment problem, which is essentially a nonlinear optimization process on camera motion and 3D line parameters. Previous overparameterizations of 3D lines induce gauge freedoms and/or internal consistency constraints. We propose the orthonormal representation, which allows handy nonlinear optimization of 3D lines using the minimum four parameters with an unconstrained optimization engine. We compare our algorithms to existing ones on simulated and real data. Results show that our triangulation algorithm outperforms standard linear and bias-corrected quasi-linear algorithms, and that bundle adjustment using our orthonormal representation yields results similar to the standard maximum likelihood trifocal tensor algorithm, while being usable for any number of views.

Algorithms for plane-based pose estimation

We present several methods for the estimation of relative pose between planes and cameras, based on projections of sets of coplanar features in images. While such methods exist for simple cases, especially one plane seen in one or several views, the aim of this paper is to propose solutions for multi-plane multi-view situations, possibly with little overlap. We propose a factorization-based method for the general case of n planes seen in m views. A mechanism for computing missing data, i.e. when one or several of the planes are not visible in one or several of the images, is described. Experimental results for real images are shown.

Bayesian 3D modeling from images using multiple depth maps

This paper addresses the problem of reconstructing the geometry and color of a Lambertian scene, given some fully calibrated images acquired with wide baselines. In order to completely model the input data, we propose to represent the scene as a set of colored depth maps, one per input image. We formulate the problem as a Bayesian MAP problem which leads to an energy minimization method. Hidden visibility variables are used to deal with occlusion, reflections and outliers. The main contributions of this work are: a prior for the visibility variables that treats the geometric occlusions; and a prior for the multiple depth maps model that smoothes and merges the depth maps while enabling discontinuities. Real world examples showing the efficiency and limitations of the approach are presented.

Critical Motion Sequences for the Self-Calibration of Cameras and Stereo Systems with Variable Focal Length

We consider the self-calibration problem for a moving camera whose intrinsic parameters are known, except the focal length, which may vary freely across different views. The conditions under which the determination of the focal length’s values for an image sequence is not possible, are derived. These depend only on the camera’s motions. We give a complete catalogue of the so-called critical motion sequences. This is then used to derive the critical motion sequences for stereo systems with variable focal lengths.

Camera calibration and 3D reconstruction from single images using parallelepipeds

In this paper parallelepipeds and their use in camera calibration and 3D reconstruction processes are studied. Parallelepipeds naturally characterize rigidity constraints present in a scene, such as parallelism and orthogonality. A subclass of parallelepipeds-the cuboids-has been frequently used over the past to partially calibrate cameras. However, the full potential of parallelepipeds, in camera calibration as well as in scene reconstruction, has never been clearly established. We propose a new framework for the use of parallelepipeds which is based on an extensive study of this potential. In particular, we exhibit the complete duality that exists between the intrinsic metric characteristics of a parallelepiped and the intrinsic parameters of a camera. Our framework allows to fully exploit parallelepipeds and thus overcomes several limitations of calibration approaches based on cuboids. To illustrate this framework, we present an original and very efficient interactive method for 3D reconstruction from single images. This method allows to quickly build a scene model from a single uncalibrated image.

The minimum clinically important difference in Scoliosis Research Society-22 Appearance, Activity, And Pain domains after surgical correction of adolescent idiopathic scoliosis.

Study Design. Longitudinal cohort. Objective. To determine the minimum clinically important difference (MCID) of the Scoliosis Research Society (SRS)-22 Appearance, Activity, and Pain domains in patients with adolescent idiopathic scoliosis undergoing surgical correction of their spinal deformity. Summary of Background Data. The MCID, a threshold of improvement that is clinically relevant to the individual patient, is increasingly used to evaluate treatment effectiveness. MCID values for the SRS-22 domains have not been determined. Methods. Patients with adolescent idiopathic scoliosis who underwent surgical correction and had completed SRS-22 before operation and the SRS-30 and Scoliosis Appearance Questionnaire (SAQ) at 1 year after operation from a multicenter database for pediatric scoliosis were identified. The SAQ is a modification of the Walter Reed Visual Assessment Scale and is used to assess the patients perception of their spinal deformity. Paired sample t tests were used to compare preoperative and 1-year postoperative scores. Spearman correlations were used to evaluate associations between domain scores and summed responses to anchors for Appearance, Activity, and Pain. MCID values for the SRS-22 domains were determined using receiver operating characteristic curve analysis, with summed responses to anchor questions 23 to 30 of the SRS-30 and items 26 and 32 of the SAQ. Results. There were 735 women and 152 men with a mean age of 14.3 years and a mean Cobb angle of 53°. There was a statistically significant difference between paired preoperative and 1-year SRS domain scores. Analysis of variance showed a statistically significant difference between the summed responses to the anchors. The MCID was 0.20 for the Pain domain (area under the curve [AUC] = 0.723), 0.08 for Activity (AUC = 0.648), and 0.98 for Appearance (AUC = 0.629). The MCID for activity was less than the standard error of measurement. Conclusion. The MCID for the Pain domain was 0.20 and 0.98 for Appearance. Because these patients were generally in good health, a minimal though significant change in activity was observed, such that the calculated MCID was within the measurement error. As expected, the largest and most important change was in the Appearance domain. Future studies are needed to determine the MCID for the mental domain and the total SRS score and to further validate the MCID values in this study.

Self-Calibration of a 1D Projective Camera and Its Application to the Self-Calibration of a 2D Projective Camera

We introduce the concept of self-calibration of a 1D projective camera from point correspondences, and describe a method for uniquely determining the two internal parameters of a 1D camera based on the trifocal tensor of three 1D images. The method requires the estimation of the trifocal tensor which can be achieved linearly with no approximation unlike the trifocal tensor of 2D images, and solving for the roots of a cubic polynomial in one variable. Interestingly enough, we prove that a 2D camera undergoing a planar motion reduces to a 1D camera. From this observation, we deduce a new method for self-calibrating a 2D camera using planar motions.

Using geometric constraints through parallelepipeds for calibration and 3D modeling

This paper concerns the incorporation of geometric information in camera calibration and 3D modeling. Using geometric constraints enables more stable results and allows us to perform tasks with fewer images. Our approach is motivated and developed within a framework of semi-automatic 3D modeling, where the user defines geometric primitives and constraints between them. In this paper, first a duality that exists between the shape parameters of a parallelepiped and the intrinsic parameters of a camera is described. Then, a factorization-based algorithm exploiting this relation is developed. Using images of parallelepipeds, it allows us to simultaneously calibrate cameras, recover shapes of parallelepipeds, and estimate the relative pose of all entities. Besides geometric constraints expressed via parallelepipeds, our approach simultaneously takes into account the usual self-calibration constraints on cameras. The proposed algorithm is completed by a study of the singular cases of the calibration method. A complete method for the reconstruction of scene primitives that are not modeled by parallelepipeds is also briefly described. The proposed methods are validated by various experiments with real and simulated data, for single-view as well as multiview cases.