Haokun Geng

A flexible method for localisation and classification of footprints of small species

In environmental surveillance, ecology experts use a standard tracking tunnel system to acquire tracks or footprints of small animals, so that they can measure the presence of any selected animals or detect threatened species based on the manual analysis of gathered tracks. Unfortunately, distinguishing morphologically similar species through analysing their footprints is extremely difficult, and even very experienced experts find it hard to provide reliable results on footprint identification. This expensive task also requires a great amount of efforts on observation. In recent years, image processing technology has become a model example for applying computer science technology to many other study areas or industries, in order to improve accuracy, productivity, and reliability. In this paper, we propose a method based on image processing technology, it firstly detects significant interest points from input tracking card images. Secondly, it filters irrelevant interest points in order to extract regions of interest. Thirdly, it gathers useful information of footprint geometric features, such as angles, areas, distance, and so on. These geometric features can be generally found in footprints of small species. Analysing the detected features statistically can certainly provide strong proof of footprint localization and classification results. We also present experimental results on extracted footprints by the proposed method. With appropriate developments or modifications, this method has great potential for applying automated identification to any species.

Improved Visual Odometry based on Transitivity Error in Disparity Space: A Third-eye Approach

Accurate estimation of ego-motion heavily relies on correct point correspondences in the context of visual odometry. In order to ensure a metric reconstruction of camera motion, we can refer to the 3D structure of the scene. In this paper we present an indicator for evaluating the accuracy of stereo-based 3D point measurements as well as for filtering out low-confidence correspondences for ego-motion estimation. In a typical binocular system, the left and right images are matched to produce a disparity map. For a trinocular system, however, the map can be derived indirectly via disparity maps of both cameras with respect to the third camera. The difference between an explicitly matched disparity map and its indirect construction defines a transitivity error in disparity space (TED). We evaluate the effectiveness of TED from different perspectives, using a trinocular vehicle-mounted vision system. Results presented in 3D Euclidean space, or in 2D images show improvements of more than 7.5%, indicating that, by taking TED into account, more consistency is ensured for ego-motion estimation.

Feature-matching and extended Kalman filter for stereo ego-motion estimation

Vision-based ego-motion estimation is a widely used method for identifying movements and poses of robots or equipped vehicles utilizing one or more attached cameras. This paper proposes a feature-matching-based method for estimating ego-motion using a calibrated two-camera stereo system. Detected features are separated into two candidate sets. Distant features are selected to provide information, about the rotational components of movements whereas features at closer distance are used to estimate translational components. An extended Kalman filter is used to eliminate the white noise, in order to get a better prediction of both positional and rotational estimations. The method aims to minimise both projection (3D) errors and flow (2D) errors, to ensure a good pair of translation and rotation measures frame by frame. Experiments are carried out for trajectory estimation, and for projection and flow error evaluation.

Multi-frame Feature Integration for Multi-camera Visual Odometry

State-of-the-art ego-motion estimation approaches in the context of visual odometry VO rely either on Kalman filters or bundle adjustment. Recently proposed multi-frame feature integration MFIi¾?[1] techniques aim at finding a compromise between accuracy and computation efficiency. In this paper we generalise an MFI algorithm towards the full use of multi-camera-based visual odometry for achieving more consistent ego-motion estimation in a parallel scalable manner. A series of experiments indicated that the generalised integration technique contributes to an improvement of above 70i¾?% over our direct VO implementation, and further improved the monocular MFI technique by more than 20i¾?%.

Star-Effect Simulation for Photography Using Self-calibrated Stereo Vision

Star effects are an important design factor for night photos. Progress in imaging technologies made it possible that night photos can be taken free-hand. For such camera settings, star effects are not achievable. We present a star-effect simulation method based on self-calibrated stereo vision. Given an uncalibrated stereo pair i.e. a base image and a match image, which can be just two photos taken with a mobile phone with about the same pose, we follow a standard routine: Extract a family of feature-point pairs, calibrate the stereo pair by using the feature-point pairs, and obtain depth information by stereo matching. We detect highlight regions in the base image, estimate the luminance according to available depth information, and, finally, render star patterns with an input texture. Experiments show that our results are similar to real-world star effect photos, and that they are more natural than results of existing commercial applications. The paper reports for the first time research on automatically simulating photo-realistic star effects.

Visual Odometry in Dynamic Environments with Geometric Multi-layer Optimisation

This paper presents a novel approach for optimising visual odometry results in a dynamic outdoor environment. Egomotion estimation is still considered to be one of the more difficult tasks in computer vision because of its continued computation pipeline: every phase of visual odometry can be a source of noise or errors, and influence future results. Also, tracking features in a dynamic environment is very challenging. Since feature tracking can only match two features in integer coordinates, there will be a data loss at sub-pixel level. In this paper we introduce a weighting scheme that measures the geometric relations between different layers: We divide tracked features into three groups based on geometric constrains; each group is recognised as being a “layer”. Each layer has a weight which depends on the distribution of the grouped features on the 2D image and the actual position in 3D scene coordinates. This geometric multi-layer approach can effectively remove all the dynamic features in the scene, and provide more reliable feature tracking results. Moreover, we propose a 3-state Kalman filter optimisation approach. Our method follows the traditional process of visual odometry algorithms by focusing on motion estimation between pairs of two consecutive frames. Experiments and evaluations are carried out for trajectory estimation. We use the provided ground truth of the KITTI data-sets to analyse mean rotation and translation errors over distance.

Star-effect simulation for photography☆

Abstract This paper discusses the creation of star effects in captured night-time photos based on using depth information available by stereo vision. Star effects are an important design factor for night-time photos. Modern imaging technologies support that night-time photos can be taken free-hand, but star effects are not achievable for such camera settings. Self-calibration is an important feature of the presented star-effect simulation method. A photographer is assumed to take just an uncalibrated stereo image pair (i.e. a base and a match image), for example by taking two photos subsequently (e.g. by a mobile phone at about the same pose). For self-calibration we apply the following routine: extract a family of pairs of feature points, calibrate the stereo image pair by using those feature points, and calculate depth data by stereo matching. For creating the star effects, first we detect highlight regions in the base image. Second, we estimate the luminance according to available depth information. Third, we render star patterns with a chosen input texture. Basically we provide a complete tool which is easy to apply for the generation of a user-selected star texture. Minor variations can be introduced in star pattern rendering in order to achieve more natural and vivid looking star effects. By extensive experiments we verified that our rendering results are potentially similar to real-world star effect photos. We demonstrate some of our results, also for illustrating that they appear more natural than results achieved by existing commercial applications. We also illustrate that our method allows us to render more artistic star patterns not available in recorded photographs. In brief, this paper reports research on automatically simulating both photorealistic and non-photorealistic star effects.

Multi-Run: An Approach for Filling in Missing Information of 3D Roadside Reconstruction

This paper presents an approach for incrementally adding missing information into a point cloud generated for 3D roadside reconstruction. We use a series of video sequences recorded while driving repeatedly through the road to be reconstructed. The video sequences can also be recorded while driving in opposite directions. We call this a multi-run scenario. The only extra input data other than stereo images is the reading from a GPS sensor, which is used as guidance for merging point clouds from different sequences into one. The quality of the 3D roadside reconstruction is in direct relationship to the accuracy of the applied egomotion estimation method. A main part of our motion analysis method is defined by visual odometry following a traditional workflow in this area: first, establish correspondences of tracked features between two subsequent frames; second, use a stereo-matching algorithm to calculate the depth information of the tracked features; then compute the motion data between every two frames using a perspective-n-point solver. Additionally, we propose a technique that uses a Kalman-filter fusion to track the selected feature points, and to filter outliers. Furthermore, we use the GPS data to bound the overall propagation of the positioning errors. Experiments are given with trajectory estimation and 3D scene reconstruction. We evaluate our approach by estimating the recovery of so far missing information when analysing data recorded in a subsequent run.

Egomotion Estimation and Reconstruction with Kalman Filters and GPS Integration

This paper presents an approach for egomotion estimation over stereo image sequences combined with extra GPS data. The accuracy of the estimated motion data is tested with 3D roadside reconstruction. Our proposed method follows the traditional flowchart of many visual odometry algorithms: it firstly establishes the correspondences between the keypoints of every two frames, then it uses the depth information from the stereo matching algorithms, and it finally computes the best description of the cameras’ motion. However, instead of simply using keypoints from consecutive frames, we propose a novel technique that uses a set of augmented and selected keypoints, which are carefully tracked by a Kalman filter fusion. We also propose to use the GPS data for each key frame in the input sequence, in order to reduce the positioning errors of the estimations, so that the drift errors could be corrected at each key frame. Finally, the overall growth of the build-up errors can be bounded within a certain range. A least-squares process is used to minimise the reprojection error and to ensure a good pair of translation and rotation measures, frame by frame. Experiments are carried out for trajectory estimation, or combined trajectory and 3D scene reconstruction, using various stereo-image sequences.

Bundle Adjustment with Implicit Structure Modeling Using a Direct Linear Transform

Bundle adjustment (BA) is considered to be the “golden standard” optimisation technique for multiple-view reconstruction over decades of research. The technique simultaneously tunes camera parameters and scene structure to fit a nonlinear function, in a way that the discrepancy between the observed scene points and their reprojections are minimised in a least-squares manner. Computational feasibility and numerical conditioning are two major concerns of todays BA implementations, and choosing a proper parametrization of structure in 3D space could dramatically improve numerical stability, convergence speed, and cost of evaluating Jacobian matrices. In this paper we study several alternative representations of 3D structure and propose an implicit modeling approach based on a Direct Linear Transform (DLT) estimation. The performances of a variety of parametrization techniques are evaluated using simulated visual odometry scenarios. Experimental results show that the computational cost and convergence speed is further improved to achieve similar accuracy without explicit adjustment over the structure parameters.