Hsiang-Jen Chien

Geometric calibration of a multi-layer LiDAR system and image sensors using plane-based implicit laser parameters for textured 3-D depth reconstruction

The paper proposes the calibration of a LiDAR-camera system that consists of a multi-layer laser rangefinder device and a pair of video cameras. The method calibrates the intrinsic laser parameters and the extrinsic parameters of the integrated LiDAR-camera system. Using a linear form, the dimensionality of the calibration parameter space is reduced in the plane-based least square model. The optimal laser intrinsic parameters can be determined during the optimization of the extrinsic parameters, without being explicitly modeled. However, due to limited FOV of the cameras, the reduced model may lead to a solution that cannot be generalized to the working space. Hence, we use additional scene planes to improve the determination of intrinsic laser parameters. Overall performance is improved if calibration targets can be accurately estimated from the cameras. Results indicate a reduction of 50% in the flatness error is achievable and running time of the process is also decreased.

When to use what feature? SIFT, SURF, ORB, or A-KAZE features for monocular visual odometry

Image feature-based ego-motion estimation has been dominating the development of visual odometry (VO) visual simultaneously localisation and mapping (V-SLAM) and structure-from-motion (SfM) for several years. The detection extraction or representation of image features play crucial roles when solving camera pose estimation problems in terms of accuracy and computational cost. In this paper we review three popular classes of image features namely SIFT SURF and ORB as well as the recently proposed A-KAZE features. These image features are evaluated using the KITTI benchmark dataset to conclude about reasons for deciding about the selection of a particular feature when implementing monocular visual odometry.

Improved Visual Odometry based on Transitivity Error in Disparity Space: A Third-eye Approach

Accurate estimation of ego-motion heavily relies on correct point correspondences in the context of visual odometry. In order to ensure a metric reconstruction of camera motion, we can refer to the 3D structure of the scene. In this paper we present an indicator for evaluating the accuracy of stereo-based 3D point measurements as well as for filtering out low-confidence correspondences for ego-motion estimation. In a typical binocular system, the left and right images are matched to produce a disparity map. For a trinocular system, however, the map can be derived indirectly via disparity maps of both cameras with respect to the third camera. The difference between an explicitly matched disparity map and its indirect construction defines a transitivity error in disparity space (TED). We evaluate the effectiveness of TED from different perspectives, using a trinocular vehicle-mounted vision system. Results presented in 3D Euclidean space, or in 2D images show improvements of more than 7.5%, indicating that, by taking TED into account, more consistency is ensured for ego-motion estimation.

Visual odometry driven online calibration for monocular LiDAR-camera systems

Recently LiDAR-camera systems have rapidly emerged in many applications. The integration of laser range-finding technologies into existing vision systems enables a more comprehensive understanding of 3D structure of the environment. The advantage, however, relies on a good geometrical calibration between the LiDAR and the image sensors. In this paper we consider visual odometry, a discipline in computer vision and robotics, in the context of recently emerging online sensory calibration studies. By embedding the online calibration problem into a LiDAR-monocular visual odometry technique, the temporal change of extrinsic parameters can be tracked and compensated effectively.

Multi-frame Feature Integration for Multi-camera Visual Odometry

State-of-the-art ego-motion estimation approaches in the context of visual odometry VO rely either on Kalman filters or bundle adjustment. Recently proposed multi-frame feature integration MFIi¾?[1] techniques aim at finding a compromise between accuracy and computation efficiency. In this paper we generalise an MFI algorithm towards the full use of multi-camera-based visual odometry for achieving more consistent ego-motion estimation in a parallel scalable manner. A series of experiments indicated that the generalised integration technique contributes to an improvement of above 70i¾?% over our direct VO implementation, and further improved the monocular MFI technique by more than 20i¾?%.

Multi-objective Visual Odometry

Visual odometry (VO) has been extensively studied in the last decade. Despite a variety of implementation details, the proposed approaches share the same principle - a minimisation of a carefully chosen energy function. In this paper we review four commonly adopted energy models including perspective, epipolar, rigid, and photometric alignments, and propose a novel VO technique that unifies multiple objectives for outlier rejection and egomotion estimation to outperform mono-objective egomotion estimation. The experiments show an improvement above 50% is achievable by trading off 15% additional computational cost.

Regularised Energy Model for Robust Monocular Ego-motion Estimation.

For two decades, ego-motion estimation is an actively developing topic in computer vision and robotics. The principle of existing motion estimation techniques relies on the minimisation of an energy function based on re-projection errors. In this paper we augment such an energy function by introducing an epipolar-geometryderived regularisation term. The experiments prove that, by taking soft constraints into account, a more reliable motion estimation is achieved. It also shows that the implementation presented in this paper is able to achieve a remarkable accuracy comparative to the stereo vision approaches, with an overall drift maintained under 2% over hundreds of metres.

Improved Stixel Estimation Based on Transitivity Analysis in Disparity Space

We present a novel method for stixel construction using a calibrated collinear trinocular vision system. Our method takes three conjugate stereo images at the same time to measure the consistency of disparity values by means of the transitivity error in disparity space. Unlike previous stixel estimation methods that are built based on a single disparity map, our proposed method introduces a multi-map fusion technique to obtain more robust stixel calculations. We also apply a polynomial curve fitting approach to detect an accurate road manifold, using the v-disparity space which is built based on a confidence map, which further supports accurate stixel calculation. Comparing the depth information from the extracted stixels (using stixel maps) with depth measurements obtained from a highly accurate LiDAR range sensor, we evaluate the accuracy of the proposed method. Experimental results indicate a significant improvement of 13.6% in the accuracy of stixel detection compared to conventional binocular vision.

Visual Odometry in Dynamic Environments with Geometric Multi-layer Optimisation

This paper presents a novel approach for optimising visual odometry results in a dynamic outdoor environment. Egomotion estimation is still considered to be one of the more difficult tasks in computer vision because of its continued computation pipeline: every phase of visual odometry can be a source of noise or errors, and influence future results. Also, tracking features in a dynamic environment is very challenging. Since feature tracking can only match two features in integer coordinates, there will be a data loss at sub-pixel level. In this paper we introduce a weighting scheme that measures the geometric relations between different layers: We divide tracked features into three groups based on geometric constrains; each group is recognised as being a “layer”. Each layer has a weight which depends on the distribution of the grouped features on the 2D image and the actual position in 3D scene coordinates. This geometric multi-layer approach can effectively remove all the dynamic features in the scene, and provide more reliable feature tracking results. Moreover, we propose a 3-state Kalman filter optimisation approach. Our method follows the traditional process of visual odometry algorithms by focusing on motion estimation between pairs of two consecutive frames. Experiments and evaluations are carried out for trajectory estimation. We use the provided ground truth of the KITTI data-sets to analyse mean rotation and translation errors over distance.

Multi-Run: An Approach for Filling in Missing Information of 3D Roadside Reconstruction

This paper presents an approach for incrementally adding missing information into a point cloud generated for 3D roadside reconstruction. We use a series of video sequences recorded while driving repeatedly through the road to be reconstructed. The video sequences can also be recorded while driving in opposite directions. We call this a multi-run scenario. The only extra input data other than stereo images is the reading from a GPS sensor, which is used as guidance for merging point clouds from different sequences into one. The quality of the 3D roadside reconstruction is in direct relationship to the accuracy of the applied egomotion estimation method. A main part of our motion analysis method is defined by visual odometry following a traditional workflow in this area: first, establish correspondences of tracked features between two subsequent frames; second, use a stereo-matching algorithm to calculate the depth information of the tracked features; then compute the motion data between every two frames using a perspective-n-point solver. Additionally, we propose a technique that uses a Kalman-filter fusion to track the selected feature points, and to filter outliers. Furthermore, we use the GPS data to bound the overall propagation of the positioning errors. Experiments are given with trajectory estimation and 3D scene reconstruction. We evaluate our approach by estimating the recovery of so far missing information when analysing data recorded in a subsequent run.