João Pedro Barreto

Geometric properties of central catadioptric line images and their application in calibration

In central catadioptric systems lines in a scene are projected to conic curves in the image. This work studies the geometry of the central catadioptric projection of lines and its use in calibration. It is shown that the conic curves where the lines are mapped possess several projective invariant properties. From these properties, it follows that any central catadioptric system can be fully calibrated from an image of three or more lines. The image of the absolute conic, the relative pose between the camera and the mirror, and the shape of the reflective surface can be recovered using a geometric construction based on the conic loci where the lines are projected. This result is valid for any central catadioptric system and generalizes previous results for paracatadioptric sensors. Moreover, it is proven that systems with a hyperbolic/elliptical mirror can be calibrated from the image of two lines. If both the shape and the pose of the mirror are known, then two line images are enough to determine the image of the absolute conic encoding the cameras intrinsic parameters. The sensitivity to errors is evaluated and the approach is used to calibrate a real camera.

Issues on the geometry of central catadioptric image formation

An imaging system with a single effective viewpoint is called a central projection system. The conventional perspective camera is an example of a central projection system. Systems using mirrors to enhance the field of view while keeping a unique center of projection are also examples of central projection systems. Perspective image formation can be described by a linear model with well known properties. In general central catadioptric imaging, the mapping between points in the world and in the image is highly nonlinear. The paper establishes a general model for central catadioptric image formation made up of three functions: a linear function mapping the world into an oriented projective plane, a nonlinear transformation between two oriented projective planes, and a collineation in the plane. The model is used to study issues in the projection of lines. The equations and geometric properties of general catadioptric imaging of lines are derived. The application of the results in auto-calibration of central catadioptric systems and reconstruction are discussed. A method to calibrate the system using three line images is presented.

Geometric Properties of Central Catadioptric Line Images

It is highly desirable that an imaging system has a single effective viewpoint. Central catadioptric systems are imaging systems that use mirrors to enhance the field of view while keeping a unique center of projection. A general model for central catadioptric image formation has already been established. The present paper exploits this model to study the catadioptric projection of lines. The equations and geometric properties of general catadioptric line imaging are derived. We show that it is possible to determine the position of both the effective viewpoint and the absolute conic in the catadioptric image plane from the images of three lines. It is also proved that it is possible to identify the type of catadioptric system and the position of the line at infinity without further information. A methodology for central catadioptric system calibration is proposed. Reconstruction aspects are discussed. Experimental results are presented. All the results presented are original and completely new.

A Minimal Solution for the Extrinsic Calibration of a Camera and a Laser-Rangefinder

This paper presents a new algorithm for the extrinsic calibration of a perspective camera and an invisible 2D laser-rangefinder (LRF). The calibration is achieved by freely moving a checkerboard pattern in order to obtain plane poses in camera coordinates and depth readings in the LRF reference frame. The problem of estimating the rigid displacement between the two sensors is formulated as one of registering a set of planes and lines in the 3D space. It is proven for the first time that the alignment of three plane-line correspondences has at most eight solutions that can be determined by solving a standard p3p problem and a linear system of equations. This leads to a minimal closed-form solution for the extrinsic calibration that can be used as hypothesis generator in a RANSAC paradigm. Our calibration approach is validated through simulation and real experiments that show the superiority with respect to the current state-of-the-art method requiring a minimum of five input planes.

Erratum to “A New Solution for Camera Calibration and Real-Time Image Distortion Correction in Medical Endoscopy–Initial Technical Evaluation” [Mar 12 634-644]

Medical endoscopy is used in a wide variety of diagnostic and surgical procedures. These procedures are renowned for the difficulty of orienting the camera and instruments inside the human body cavities. The small size of the lens causes radial distortion of the image, which hinders the navigation process and leads to errors in depth perception and object morphology. This article presents a complete software-based system to calibrate and correct the radial distortion in clinical endoscopy in real time. Our system can be used with any type of medical endoscopic technology, including oblique-viewing endoscopes and HD image acquisition. The initial camera calibration is performed in an unsupervised manner from a single checkerboard pattern image. For oblique-viewing endoscopes the changes in calibration during operation are handled by a new adaptive camera projection model and an algorithm that infer the rotation of the probe lens using only image information. The workload is distributed across the CPU and GPU through an optimized CPU+GPU hybrid solution. This enables real-time performance, even for HD video inputs. The system is evaluated for different technical aspects, including accuracy of modeling and calibration, overall robustness, and runtime profile. The contributions are highly relevant for applications in computer-aided surgery and image-guided intervention such as improved visualization by image warping, 3-D modeling, and visual SLAM.

A unifying geometric representation for central projection systems

In this paper, we study projection systems with a single effective viewpoint, including combinations of mirrors and lenses (catadioptric) as well as just lenses with or without radial distortion (dioptric systems). First, we extend a well-known unifying model for central catadioptric systems to incorporate a class of dioptric systems with radial distortion. Second, we provide a new representation for the image plane of central systems. This representation is the lifting through a Veronese map of the original image plane to the 5D projective space. We study how a collineation in the original image plane can be transferred to a collineation in the lifted space, and we prove that in the case of central parabolic systems and cameras with lens distortion the locus of the lifted points representing projections of world lines is a plane. The similarities between paracatadioptric systems and lens with radial distortion are emphasized by extending to the latter algorithms initially established for the former.

Fundamental matrix for cameras with radial distortion

When deploying a heterogeneous camera network or when we use cheap zoom cameras like in cell-phones, it is not practical, if not impossible to off-line calibrate the radial distortion of each camera using reference objects. It is rather desirable to have an automatic procedure without strong assumptions about the scene. In this paper, we present a new algorithm for estimating the epipolar geometry of two views where the two views can be radially distorted with different distortion factors. It is the first algorithm in the literature solving the case of different distortion in the left and right view linearly and without assuming the existence of lines in the scene. Points in the projective plane are lifted to a quadric in three-dimensional projective space. A radial distortion of the projective plane results to a matrix transformation in the space of lifted coordinates. The new epipolar constraint depends linearly on a 4 /spl times/ 4 radial fundamental matrix which has 9 degrees of freedom. A complete algorithm is presented and tested on real imagery.

Visual Servoing/Tracking Using Central Catadioptric Images

Visual control of robot motion may benefit from enhanced camera field of view. With traditional cameras the available fields of view are only enough to view a region around the observed object (for eye-in-hand systems) or around the end-effector (for independent-eye systems). Central catadioptric systems have larger fields of view thus allowing the entire robot AND the surrounding objects to be imaged with a unique camera. Therefore, the whole robot’s articulated mechanism can be observed and its joints can be tracked and controlled simultaneously. This results in a new visual robot control concept where tracking and control are embedded together. Key to the understanding of both servoing and tracking is the central catadioptric Jacobian matrix linking the robot’s joint velocities to image observations. In spite of a more complex projection matrix associated with catadioptric sensors, we study the catadioptric Jacobian matrix and we show that it does not introduce any additional singularity with respect to the traditional pinhole camera model. Experiments showing a rigid body being tracked with a catadioptric camera are described.

sRD-SIFT: Keypoint Detection and Matching in Images With Radial Distortion

Keypoint detection and matching is of fundamental importance for many applications in computer and robot vision. The association of points across different views is problematic because image features can undergo significant changes in appearance. Unfortunately, state-of-the-art methods, like the scale-invariant feature transform (SIFT), are not resilient to the radial distortion that often arises in images acquired by cameras with microlenses and/or wide field-of-view. This paper proposes modifications to the SIFT algorithm that substantially improve the repeatability of detection and effectiveness of matching under radial distortion, while preserving the original invariance to scale and rotation. The scale-space representation of the image is obtained using adaptive filtering that compensates the local distortion, and the keypoint description is carried after implicit image gradient correction. Unlike competing methods, our approach avoids image resampling (the processing is carried out in the original image plane), it does not require accurate camera calibration (an approximate modeling of the distortion is sufficient), and it adds minimal computational overhead. Extensive experiments show the advantages of our method in establishing point correspondence across images with radial distortion.

Automatic Camera Calibration Applied to Medical Endoscopy

The paper proposes a new calibration algorithm for cameras with lens distortion, that uses a single image of a planar chessboard pattern acquired in general position. The radial distortion is modeled using the first order division model, and the method provides a closed form estimation of the intrinsic parameters and distortion coefficient. The experimental evaluation shows that the calibration accuracy is comparable to state-of-the-art algorithms requiring multiple input images. We believe that our approach is particularly well suited for the the calibration of medical endoscopes in computer aided surgery. Since the lens is mounted on the camera before each usage in the OR, the calibration procedure must be performed by the clinical practitioner with minimum effort. We solve this problem by proposing a fully automatic procedure that requires no human intervention other than acquiring a single calibration image.