Ying Kin Yu

Recursive Camera-Motion Estimation With the Trifocal Tensor

In this paper, an innovative extended Kalman filter (EKF) algorithm for pose tracking using the trifocal tensor is proposed. In the EKF, a constant-velocity motion model is used as the dynamic system, and the trifocal-tensor constraint is incorporated into the measurement model. The proposed method has the advantages of those structure- and-motion-based approaches in that the pose sequence can be computed with no prior information on the scene structure. It also has the strengths of those model-based algorithms in which no updating of the three-dimensional (3-D) structure is necessary in the computation. This results in a stable, accurate, and efficient algorithm. Experimental results show that the proposed approach outperformed other existing EKFs that tackle the same problem. An extension to the pose-tracking algorithm has been made to demonstrate the application of the trifocal constraint to fast recursive 3-D structure recovery

Robust 3-D Motion Tracking From Stereo Images: A Model-Less Method

Traditional vision-based 3-D motion-estimation algorithms require given or calculated 3-D models while the motion is being tracked. We propose a high-speed extended-Kalman-filter-based approach that recovers camera position and orientation from stereo image sequences without prior knowledge, as well as the procedure for the reconstruction of 3-D structures. Empowered by the use of a trifocal tensor, the computation step of 3-D models can be eliminated. The algorithm is thus flexible and can be applied to a wide range of domains. The twist motion model is also adopted to parameterize the 3-D motion. It is minimal since it only has six parameters as opposed to seven parameters in quaternion and 12 parameters in matrix representation. The motion representation is robust because it does not suffer from singularities as Euler angles. Due to the fact that the number of parameters to be estimated is reduced, our algorithm is more efficient, stable, and accurate than traditional approaches. The proposed method has been applied to recover the motion from stereo image sequences taken by a robot and a handheld stereo rig. The results are accurate compared to the ground truths. It is shown in the experiment that our algorithm is not susceptible to outlying point features with the application of a validation gate.

Recursive three-dimensional model reconstruction based on Kalman filtering

A recursive two-step method to recover structure and motion from image sequences based on Kalman filtering is described in this paper. The algorithm consists of two major steps. The first step is an extended Kalman filter (EKF) for the estimation of the objects pose. The second step is a set of EKFs, one for each model point, for the refinement of the positions of the model features in the three-dimensional (3-D) space. These two steps alternate from frame to frame. The initial model converges to the final structure as the image sequence is scanned sequentially. The performance of the algorithm is demonstrated with both synthetic data and real-world objects. Analytical and empirical comparisons are made among our approach, the interleaved bundle adjustment method, and the Kalman filtering-based recursive algorithm by Azarbayejani and Pentland. Our approach outperformed the other two algorithms in terms of computation speed without loss in the quality of model reconstruction.

Pose estimation for augmented reality applications using genetic algorithm

This paper describes a genetic algorithm that tackles the pose-estimation problem in computer vision. Our genetic algorithm can find the rotation and translation of an object accurately when the three-dimensional structure of the object is given. In our implementation, each chromosome encodes both the pose and the indexes to the selected point features of the object. Instead of only searching for the pose as in the existing work, our algorithm, at the same time, searches for a set containing the most reliable feature points in the process. This mismatch filtering strategy successfully makes the algorithm more robust under the presence of point mismatches and outliers in the images. Our algorithm has been tested with both synthetic and real data with good results. The accuracy of the recovered pose is compared to the existing algorithms. Our approach outperformed the Lowes method and the other two genetic algorithms under the presence of point mismatches and outliers. In addition, it has been used to estimate the pose of a real object. It is shown that the proposed method is applicable to augmented reality applications.

Merging artificial objects with marker-less video sequences based on the interacting multiple model method

Inserting synthetic objects into video sequences has gained much interest in recent years. Fast and robust vision-based algorithms are necessary to make such an application possible. Traditional pose tracking schemes using recursive structure from motion techniques adopt one Kalman filter and thus only favor a certain type of camera motion. We propose a robust simultaneous pose tracking and structure recovery algorithm using the interacting multiple model (IMM) to improve performance. In particular, a set of three extended Kalman filters (EKFs), each describing a frequently occurring camera motion in real situations (general, pure translation, pure rotation), is applied within the IMM framework to track the pose of a scene. Another set of EKFs,one filter for each model point, is used to refine the positions of the model features in the 3-D space. The filters for pose tracking and structure refinement are executed in an interleaved manner. The results are used for inserting virtual objects into the original video footage. The performance of the algorithm is demonstrated with both synthetic and real data. Comparisons with different approaches have been performed and show that our method is more efficient and accurate.

A fast recursive 3D model reconstruction algorithm for multimedia applications

A recursive two-step method to recover structure and motion from image sequences based on Kalman filtering is described in this paper. The algorithm consists of two major steps. The first step is an extended Kalman filter for the estimation of the objects pose. The second step is a set of extended Kalman filters, one for each model point, for refining the positions of the model features in the 3D space. The initial guess is a planar model formed under the assumption of orthographic projection on the first image. These two steps alternate from frames to frames. The planar model converges to the final structure as the image sequence is scanned sequentially. The performance of the algorithm is demonstrated with both synthetic data and real world objects. Comparisons with different approaches have been performed and show that our method is more efficient.

Recursive Recovery of Position and Orientation from Stereo Image Sequences without Three-Dimensional Structures

Traditional vision-based 3-D motion estimation algorithms for robots require given or calculated 3-D models while the motion is being tracked. We propose a high-speed extended-Kalman-filter-based approach that recovers position and orientation from stereo image sequences without prior knowledge as well as the procedure for the reconstruction of 3-D structures. Empowered by the use of the trifocal tensor, the computation step of 3-D models can be eliminated. The algorithm is thus more flexible and can be applied to a wide range of domains. The twist motion model is also adopted to parameterize the 3-D motion such that the motion representation in the proposed algorithm is robust and minimal. As the number of parameters to be estimated is reduced, our algorithm is more efficient, stable and accurate compared to traditional approaches. The proposed method has been verified using a real image sequence with ground truth.

Extended Kalman filtering approach to stereo video stabilization

Processing of stereo images has become more and more important in recent years because of the availability of various stereo displaying devices. In particular, stabilizing of stereo images is important and useful especially when the images are obtained from cameras held by inexperienced hands or placed on unstable platforms. In this paper, we propose a new frame warping method for such a problem. Most video stabilization methods to date use 2-D geometric transform to approximate the changes between frames. However, these methods fail when there is a large depth variation between the foreground and the background in the scene. We try to solve this by estimating the 3-D motion parameters of the cameras by tri-focal tensor and the extended Kalman filter. And use the motion parameters to stabilize the images. We test the method using synthetic and real images and the results show that the performance of our proposed method is accurate even if the background contains large relative depth variations.

Controlling Virtual Cameras Based on a Robust Model-Free Pose Acquisition Technique

This paper presents a novel method that acquires camera position and orientation from a stereo image sequence without prior knowledge of the scene. To make the algorithm robust, the interacting multiple model probabilistic data association filter (IMMPDAF) is introduced. The interacting multiple model (IMM) technique allows the existence of more than one dynamic system in the filtering process and in return leads to improved accuracy and stability even under abrupt motion changes. The probabilistic data association (PDA) framework makes the automatic selection of measurement sets possible, resulting in enhanced robustness to occlusions and moving objects. In addition to the IMMPDAF, the trifocal tensor is employed in the computation so that the step of reconstructing the 3-D models can be eliminated. This further guarantees the precision of estimation and computation efficiency. Real stereo image sequences have been used to test the proposed method in the experiment. The recovered 3-D motions are accurate in comparison with the ground truth data and have been applied to control cameras in a virtual environment.

A fast and robust simultaneous pose tracking and structure recovery algorithm for augmented reality applications

A robust simultaneous pose tracking and structure recovery algorithm based on the Interacting Multiple Model (IMM) for augmented reality applications is proposed in this paper. A set of three extended Kalman filters (EKFs), each describes a frequently occurring camera motion in real situations (general, pure translation, pure rotation), is applied within the IMM framework to track the pose of an object. Another set of EKFs, one filter for each model point, is used to refine the positions of the model features in the 3D space. The filters for pose tracking and structure refinement are executed in an interleaved manner. The results are used for inserting virtual objects into the original video footage. The performance of the algorithm is demonstrated with both synthetic and real data. Comparisons with different approaches have been performed and show that our method is more efficient and accurate.