Faraz M. Mirzaei

A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation

Vision-aided inertial navigation systems (V-INSs) can provide precise state estimates for the 3-D motion of a vehicle when no external references (e.g., GPS) are available. This is achieved by combining inertial measurements from an inertial measurement unit (IMU) with visual observations from a camera under the assumption that the rigid transformation between the two sensors is known. Errors in the IMU-camera extrinsic calibration process cause biases that reduce the estimation accuracy and can even lead to divergence of any estimator processing the measurements from both sensors. In this paper, we present an extended Kalman filter for precisely determining the unknown transformation between a camera and an IMU. Contrary to previous approaches, we explicitly account for the time correlation of the IMU measurements and provide a figure of merit (covariance) for the estimated transformation. The proposed method does not require any special hardware (such as spin table or 3-D laser scanner) except a calibration target. Furthermore, we employ the observability rank criterion based on Lie derivatives and prove that the nonlinear system describing the IMU-camera calibration process is observable. Simulation and experimental results are presented that validate the proposed method and quantify its accuracy.

Optimal estimation of vanishing points in a Manhattan world

In this paper, we present an analytical method for computing the globally optimal estimates of orthogonal vanishing points in a “Manhattan world” with a calibrated camera. We formulate this as constrained least-squares problem whose optimality conditions form a multivariate polynomial system. We solve this system analytically to compute all the critical points of the least-squares cost function, and hence the global minimum, i.e., the globally optimal estimate for the orthogonal vanishing points. The same optimal estimator is used in conjunction with RANSAC to generate orthogonal-vanishing-point hypotheses (from triplets of lines) and thus classify lines into parallel and mutually orthogonal groups. The proposed method is validated experimentally on the York Urban Database.

3D LIDAR-camera intrinsic and extrinsic calibration: Identifiability and analytical least-squares-based initialization

In this paper we address the problem of estimating the intrinsic parameters of a 3D LIDAR while at the same time computing its extrinsic calibration with respect to a rigidly connected camera. Existing approaches to solve this nonlinear estimation problem are based on iterative minimization of nonlinear cost functions. In such cases, the accuracy of the resulting solution hinges on the availability of a precise initial estimate, which is often not available. In order to address this issue, we divide the problem into two least-squares sub-problems, and analytically solve each one to determine a precise initial estimate for the unknown parameters. We further increase the accuracy of these initial estimates by iteratively minimizing a batch nonlinear least-squares cost function. In addition, we provide the minimal identifiability conditions, under which it is possible to accurately estimate the unknown parameters. Experimental results consisting of photorealistic 3D reconstruction of indoor and outdoor scenes, as well as standard metrics of the calibration errors, are used to assess the validity of our approach.

Globally optimal pose estimation from line correspondences

Correspondences between 2D lines in an image and 3D lines in the surrounding environment can be exploited to determine the cameras position and attitude (pose). In this paper, we introduce a novel approach to estimate the cameras pose by directly solving the corresponding least-squares problem algebraically. Specifically, the optimality conditions of the least-squares problem form a system of polynomial equations, which we efficiently solve through the eigendecomposition of a so-called multiplication matrix. Contrary to existing methods, the proposed algorithm (i) is guaranteed to find the globally optimal estimate in the least-squares sense, (ii) does not require initialization, and (iii) has computational cost only linear in the number of measurements. The superior performance of the proposed algorithm compared to previous approaches is demonstrated through extensive simulations and experiments.

1|A Kalman filter-based algorithm for IMU-camera calibration

Vision-aided Inertial Navigation Systems (V-INS) can provide precise state estimates for the 3D motion of a vehicle when no external references (e.g., GPS) are available. This is achieved by combining inertial measurements from an IMU with visual observations from a camera under the assumption that the rigid transformation between the two sensors is known. Errors in the IMU-camera calibration process causes biases that reduce the accuracy of the estimation process and can even lead to divergence. In this paper, we present a Kalman filter-based algorithm for precisely determining the unknown transformation between a camera and an IMU. Contrary to previous approaches, we explicitly account for the time correlations of the IMU measurements and provide a figure of merit (covariance) for the estimated transformation. The proposed method does not require any special hardware (such as spin table or 3D laser scanner) except a calibration target. Simulation and experimental results are presented that validate the proposed method and quantify its accuracy.

A Laser-Aided Inertial Navigation System (L-INS) for human localization in unknown indoor environments

This paper presents a novel 3D indoor Laser-aided Inertial Navigation System (L-INS) for the visually impaired. An Extended Kalman Filter (EKF) fuses information from an Inertial Measurement Unit (IMU) and a 2D laser scanner, to concurrently estimate the six degree-of-freedom (d.o.f.) position and orientation (pose) of the person and a 3D map of the environment. The IMU measurements are integrated to obtain pose estimates, which are subsequently corrected using line-to-plane correspondences between linear segments in the laser-scan data and orthogonal structural planes of the building. Exploiting the orthogonal building planes ensures fast and efficient initialization and estimation of the map features while providing human-interpretable layout of the environment. The L-INS is experimentally validated by a person traversing a multistory building, and the results demonstrate the reliability and accuracy of the proposed method for indoor localization and mapping.

A 3D pose estimator for the visually impaired

This paper presents an indoor localization system for the visually impaired. The basis of our system is an Extended Kalman Filter (EKF) for six degree-of-freedom (d.o.f.) position and orientation (pose) estimation. The sensing platform consists of an Inertial Measurement Unit (IMU) and a 2D laser scanner. The IMU measurements are integrated to obtain pose estimates which are subsequently corrected using line-to-plane correspondences between linear segments in the lasers-can data and known 3D structural planes of the building. Furthermore, we utilize Lie derivatives to show that the system is observable when at least three planes are detected by the laser scanner. Experimental results are presented that demonstrate the reliability of the proposed method for accurate and real-time indoor localization.

An analytical least-squares solution to the odometer-camera extrinsic calibration problem

In order to fuse camera and odometer measurements, we first need to estimate their relative transformation through the so-called odometer-camera extrinsic calibration. In this paper, we present a two-step analytical least-squares solution for the extrinsic odometer-camera calibration that (i) is not iterative and finds the least-squares optimal solution without any initialization, and (ii) does not require any special hardware or the presence of known landmarks in the scene. Specifically, in the first step, we estimate a subset of the 3D relative rotation parameters by analytically minimizing a least-squares cost function. We then back-substitute these estimates in the measurement constraints, and determine the rest of the 3D transformation parameters by analytically minimizing a second least-squares cost function. Simulation and experimental results are presented that validate the efficiency and accuracy of the proposed algorithm.

Analytical Least-Squares Solution for 3D Lidar-Camera Calibration

This paper addresses the problem of estimating the intrinsic parameters of the 3D Velodyne lidar while at the same time computing its extrinsic calibration with respect to a rigidly connected camera. Existing approaches to solve this nonlinear estimation problem are based on iterative minimization of nonlinear cost functions. In such cases, the accuracy of the resulting solution hinges on the availability of a precise initial estimate, which is often not available. In order to address this issue, we divide the problem into two least-squares sub-problems, and analytically solve each one to determine a precise initial estimate for the unknown parameters. We further increase the accuracy of these initial estimates by iteratively minimizing a batch nonlinear least-squares cost function. In addition, we provide the minimal observability conditions, under which, it is possible to accurately estimate the unknown parameters. Experimental results consisting of photorealistic 3D reconstruction of indoor and outdoor scenes are used to assess the validity of our approach.