Haibo Liu

A Flexible Online Camera Calibration Using Line Segments

In order to make the general user take vision tasks more flexibly and easily, this paper proposes a new solution for the problem of camera calibration from correspondences between model lines and their noisy image lines in multiple images. In the proposed method the common planar items in hand with the standard size and structure are utilized as the calibration objects. The proposed method consists of a closed-form solution based on homography optimization, followed by a nonlinear refinement based on the maximum likelihood approach. To automatically recover the camera parameters linearly, we present a robust homography optimization method based on the edge model by redesigning the classic 3D tracking approach. In the nonlinear refinement procedure, the uncertainty of the image line segment is encoded in the error model, taking the finite nature of the observations into account. By developing the new error model between the model line and image line segment, the problem of the camera calibration is expressed in the probabilistic formulation. Simulation data is used to compare this method with the widely used planar pattern based method. Actual image sequences are also utilized to demonstrate the effectiveness and flexibility of the proposed method.

The effects of temperature variation on videometric measurement and a compensation method

When a videometric system operates over a long period, temperature variations in the camera and its environment will affect the measurement results, which cannot be ignored. How to eliminate or compensate for the effects of such variations in temperature is an emergent problem. Starting with the image drift phenomenon, this paper presents an image-drift model that analyzes the relationship between variations in the camera parameters and drift in the coordinates of the image. A simplified model is then introduced by analyzing the coupling relationships among the variations in the camera parameters. Furthermore, a model of the relationship between the camera parameters and temperature variations is established with the system identification method. Finally, several compensation experiments on image drift are carried out, using the parameter-temperature relationship model calibrated with one arbitrary data set to compensate the others. The analyses and experiments demonstrate the feasibility and efficiency of the proposed method. This paper proposed a compensation method for eliminating the effects of temperature variation in the long duration application of image vision.A model of the relationship between the camera parameters and temperature variations is established with the system identification method.Experiments are carried on. The analyses and experiments demonstrate the feasibility and efficiency of the proposed method.

Online cascaded boosting with histogram of orient gradient features for car detection from unmanned aerial vehicle images

Abstract. Car detection from unmanned aerial vehicle (UAV) images has become an important research field. However, robust and efficient car detection is still a challenging problem because of the cars’ appearance variations and complicated background. We present an online cascaded boosting framework with histogram of orient gradient (HOG) features for car detection from UAV images. First, the HOG of the whole sliding window is computed to find the primary gradient direction that is used to estimate the car’s orientation. The sliding window is then rotated according to the estimated car’s orientation, and the HOG features in the rotated window are efficiently computed using the proposed four kinds of integral histograms. Second, to improve the performance of the weak classifiers, a new distance metric is employed instead of the Euclidean distance. Third, we propose an efficient online cascaded boosting for car detection by combining online boosting with soft cascade. Additionally, for the problem of imbalanced training samples, more positive samples are extracted in the rotated images, and for postprocessing, a confidence map is obtained to combine multiple detections and eliminate isolated false negatives. A set of experiments on real images shows the applicability and high efficiency of the proposed car detection method.

Probabilistic approach for maximum likelihood estimation of pose using lines

In this study, the authors have proposed a new solution for the problem of pose estimation from a set of matched 3D model and 2D image lines. Traditional line-based pose estimation methods utilising the finite information of the observations are based on the assumption that the noises for the two endpoints of the image line segment are statistically independent. However, in this study, the authors prove that these two noises are negatively correlative when the image line segment is fitted by the least-squares technique from the noisy edge points. Moreover, the authors derive the noise model describing the probabilistic relationship between the 3D model line and their finite image observations. Based on the proposed noise model, the maximum-likelihood approach is exploited to estimate the pose parameters. The authors have carried out synthetic experiments to compare the proposed method to other pose optimisation methods in the literature. The experimental results show that the proposed methods yield a clear higher precision than the traditional methods. The authors also use real image sequences to demonstrate the performance of the proposed method.

Comparative Study of Visual Tracking Method: A Probabilistic Approach for Pose Estimation Using Lines

In this paper, we propose two perspective-n-line (PnL)-like methods with the presence of line detection process. Compared with the traditional methods, the proposed methods use the new error models derived from the edge points and their corresponding noisy observations rather than relying on the assumption that the noises for the two endpoints are statistically independent. Meanwhile, we improve the performance of the RAPiD-like method—another type of visual tracking approach without extracting image lines by fitting the interpolated location of the corresponding edge pixel in the local region. In addition, we compare the proposed PnL-like methods with the RAPiD-like methods and find that both the types of visual tracking methods for rigid objects are fundamentally equivalent and all of them are maximum-likelihood approaches to estimate the pose parameters, given the error model for the noisy edge points. Special consideration is put into deriving a unifying probabilistic framework to express these two types of methods. Moreover, comparisons under different performance criteria, including computational efficiency, accuracy, and robustness, are also conducted.

Multiple reflectors based autocollimator for three-dimensional angle measurement

This paper designs a multiple reflectors based autocollimator, and proposes a direct linear solution for three-dimensional (3D) angle measurement with the observation vectors of the reflected lights from the reflectors. In the measuring apparatus, the multiple reflectors is fixed with the object to be measured and the reflected lights are received by a CCD camera, then the light spots in the image are extracted to obtain the vectors of the reflected lights in space. Any rotation of the object will induce a change in the observation vectors of the reflected lights, which is used to solve the rotation matrix of the object by finding a linear solution of Wahba problem with the quaternion method, and then the 3D angle is obtained by decomposing the rotation matrix. This measuring apparatus can be implemented easily as the light path is simple, and the computation of 3D angle with observation vectors is efficient as there is no need to iterate. The proposed 3D angle measurement method is verified by a set of simulation experiments.

Review of Calibration Methods for Scheimpflug Camera

The Scheimpflug camera offers a wide range of applications in the field of typical close-range photogrammetry, particle image velocity, and digital image correlation due to the fact that the depth-of-view of Scheimpflug camera can be greatly extended according to the Scheimpflug condition. Yet, the conventional calibration methods are not applicable in this case because the assumptions used by classical calibration methodologies are not valid anymore for cameras undergoing Scheimpflug condition. Therefore, various methods have been investigated to solve the problem over the last few years. However, no comprehensive review exists that provides an insight into recent calibration methods of Scheimpflug cameras. This paper presents a survey of recent calibration methods of Scheimpflug cameras with perspective lens, including the general nonparametric imaging model, and analyzes in detail the advantages and drawbacks of the mainstream calibration models with respect to each other. Real data experiments including calibrations, reconstructions, and measurements are performed to assess the performance of the models. The results reveal that the accuracies of the RMM, PLVM, PCIM, and GNIM are basically equal, while the accuracy of GNIM is slightly lower compared with the other three parametric models. Moreover, the experimental results reveal that the parameters of the tangential distortion are likely coupled with the tilt angle of the sensor in Scheimpflug calibration models. The work of this paper lays the foundation of further research of Scheimpflug cameras.

Similarity-transform invariant similarity measure for robust template matching

A good similarity measure is the key to robust template matching. In this paper, we present a Similarity-Transform invariant Best-Buddies Similarity (SiTi-BBS) to deal with the template matching with obvious geometric distortion. Similar to the BBP, SiTi-BBS still adopts Best-Buddies Pair (BBP) to vote. However, differing from the classic BBS acquiring the point pair via bidirectional matching in xyRGB space, SiTi-BBS takes only the color information (RGB components) to acquire BBPs, while the position information (xy components) of each BBP is employed to calculate the geometric distortion between the template and matching window. To further improve the robustness of template matching, we novelly take advantage of the interval voting to accommodate the case where the two images do not strictly satisfy the similarity transformation. Therefore, SiTi-BBS, to a certain extent, can be applied to the affine and perspective transformation. In this way, the highest number of votes is taken as the similarity measure between the two images. Mathematical analysis indicates that the proposed method is capable of dealing with the case of obvious geometric distortion between images. Furthermore, the test results of simulated and real challenging images show the outstanding performance of the proposed similarity measure for template matching.

An INS data based approach to star image simulation for ship-borne star sensor

The technique of the star image simulation is widely used to test star identification algorithms and the performance of star sensor on the ground. A novel INS data based approach to ship-borne star map simulation is put forward in this paper. The simulation procedure consists of three steps: Firstly, the exact speed and position of the ship in the Conventional Inertial System(CIS) are calculated via the INS data; and then the ship attitude matrix is obtained. Secondly, considering the azimuth angle and elevation angle of the star sensor, the accurate positions of the selected guide stars on the image plane of the star sensor are derived by constructing a pinhole model. At the third, the gray values of simulating star image pixels are evaluated according to the 2D Gaussian distribution law. In order to simulate the star image precisely and actually, the image smear has been considered. Based on the proposed star image simulation approach, the effects of image smear on star sensor recognition capability have been analyzed in different exposure time.

3D measurement and camera attitude estimation method based on trifocal tensor

To simultaneously perform 3D measurement and camera attitude estimation, an efficient and robust method based on trifocal tensor is proposed in this paper, which only employs the intrinsic parameters and positions of three cameras. The initial trifocal tensor is obtained by using heteroscedastic errors-in-variables (HEIV) estimator and the initial relative poses of the three cameras is acquired by decomposing the tensor. Further the initial attitude of the cameras is obtained with knowledge of the three cameras’ positions. Then the camera attitude and the interested points’ image positions are optimized according to the constraint of trifocal tensor with the HEIV method. Finally the spatial positions of the points are obtained by using intersection measurement method. Both simulation and real image experiment results suggest that the proposed method achieves the same precision of the Bundle Adjustment (BA) method but be more efficient.