Michael Ming-Yuen Chang

Recursive Camera-Motion Estimation With the Trifocal Tensor

In this paper, an innovative extended Kalman filter (EKF) algorithm for pose tracking using the trifocal tensor is proposed. In the EKF, a constant-velocity motion model is used as the dynamic system, and the trifocal-tensor constraint is incorporated into the measurement model. The proposed method has the advantages of those structure- and-motion-based approaches in that the pose sequence can be computed with no prior information on the scene structure. It also has the strengths of those model-based algorithms in which no updating of the three-dimensional (3-D) structure is necessary in the computation. This results in a stable, accurate, and efficient algorithm. Experimental results show that the proposed approach outperformed other existing EKFs that tackle the same problem. An extension to the pose-tracking algorithm has been made to demonstrate the application of the trifocal constraint to fast recursive 3-D structure recovery

Stroke order and stroke number free on-line Chinese character recognition using attributed relational graph matching

A structural method for on-line recognition of Chinese characters is proposed, which is stroke order and stroke number free. Both input characters and the model characters are represented with complete attributed relational graphs (ARGs). A new optimal matching measure between two ARGs is defined. Classification of an input character can be implemented by matching its ARG against every ARG of the model base. The matching procedure is formulated as a search problem of finding the minimum cost path in a state space tree, using the A* algorithm. In order to speed up the search of the A*, besides a heuristic estimate, a novel strategy that utilizes the geometric position information of stroke segments of Chinese characters to prune the tree is employed. The efficiency of our method is demonstrated by the promising experimental results.

Pose estimation for augmented reality applications using genetic algorithm

This paper describes a genetic algorithm that tackles the pose-estimation problem in computer vision. Our genetic algorithm can find the rotation and translation of an object accurately when the three-dimensional structure of the object is given. In our implementation, each chromosome encodes both the pose and the indexes to the selected point features of the object. Instead of only searching for the pose as in the existing work, our algorithm, at the same time, searches for a set containing the most reliable feature points in the process. This mismatch filtering strategy successfully makes the algorithm more robust under the presence of point mismatches and outliers in the images. Our algorithm has been tested with both synthetic and real data with good results. The accuracy of the recovered pose is compared to the existing algorithms. Our approach outperformed the Lowes method and the other two genetic algorithms under the presence of point mismatches and outliers. In addition, it has been used to estimate the pose of a real object. It is shown that the proposed method is applicable to augmented reality applications.

Model reconstruction and pose acquisition using extended Lowe's method

Finding the pose and structure of an unknown object from an image sequence has many applications in graphics, virtual reality, and multimedia processing. In this paper, we address this problem by using a two-stage iterative method. Starting from an initial guess of the structure, the first stage estimates the pose of the object. The second stage uses the estimated pose information to refine the structure. This process is repeated until the difference between the observed data and data re-projected from the estimated model is minimized. This method is a variation of the classical bundle adjustment method, but is faster in execution and is simpler to implement. We used the Kanade-Lucas-Tomasi feature tracker for obtaining the image features. Synthetic and real data have been tested with good results.

An Effective Method for Movable Projector Keystone Correction

Keystone correction is an essential operation for projector-based applications, especially in mobile scenarios. In this paper, we propose a handheld movable projection method that can freely project keystone-free content on a general flat surface without adding any markings or boundary on it. Such a projection system can give the user greater freedom of display control (such as viewing angle, distance, etc.), without suffering from keystone distortion. To achieve this, we attach a camera to the projector to form a camera-projector pair. A green frame with the same resolution as the projector screen is projected onto the screen. Particle filter is employed to track the green frame and the correction of the display content is then achieved by rectifying the projection region of interest into a rectangular area. We built a prototype system to validate the effectiveness of the method. Experimental results show that our method can continuously project distortion free content in real time with good performance.

A fast recursive 3D model reconstruction algorithm for multimedia applications

A recursive two-step method to recover structure and motion from image sequences based on Kalman filtering is described in this paper. The algorithm consists of two major steps. The first step is an extended Kalman filter for the estimation of the objects pose. The second step is a set of extended Kalman filters, one for each model point, for refining the positions of the model features in the 3D space. The initial guess is a planar model formed under the assumption of orthographic projection on the first image. These two steps alternate from frames to frames. The planar model converges to the final structure as the image sequence is scanned sequentially. The performance of the algorithm is demonstrated with both synthetic data and real world objects. Comparisons with different approaches have been performed and show that our method is more efficient.

Recursive Recovery of Position and Orientation from Stereo Image Sequences without Three-Dimensional Structures

Traditional vision-based 3-D motion estimation algorithms for robots require given or calculated 3-D models while the motion is being tracked. We propose a high-speed extended-Kalman-filter-based approach that recovers position and orientation from stereo image sequences without prior knowledge as well as the procedure for the reconstruction of 3-D structures. Empowered by the use of the trifocal tensor, the computation step of 3-D models can be eliminated. The algorithm is thus more flexible and can be applied to a wide range of domains. The twist motion model is also adopted to parameterize the 3-D motion such that the motion representation in the proposed algorithm is robust and minimal. As the number of parameters to be estimated is reduced, our algorithm is more efficient, stable and accurate compared to traditional approaches. The proposed method has been verified using a real image sequence with ground truth.

EKF Based Pose Estimation using Two Back-to-Back Stereo Pairs

In this work, we solve the pose estimation problem for robot motion by placing multiple cameras on the robot. In particular, we use four cameras arranged as two back-to-back stereo pairs combined with the extended Kalman filter (EKF). The reason for using multiple cameras is that the pose estimation problem is more constrained for multiple cameras than for a single camera. Back-to-back cameras are used since they provide more information. Stereo information is used in self initialization and outlier rejection. Different approaches to solve the long-sequence-drift have been suggested. Both the simulations and the real experiments show that our approach is fast, robust, and accurate.

An interactive handheld spherical 3D object display system

Traditional display systems usually display 3D objects on static screens (monitor, wall, etc.) and the manipulation of virtual objects by the viewer is usually achieved via indirect tools such as keyboard or mouse. It would be more natural and direct if we display the object onto a handheld surface and manipulate it with our hands as if we were holding the real 3D object. In this paper, we propose a prototype system by projecting the object onto a handheld foam sphere. The aim is to develop an interactive 3D object manipulation and exhibition tool without the viewer having to wear spectacles. In our system, the viewer holds the sphere with his hands and moves it freely. Meanwhile we project well-tailored images onto the sphere to follow its motion, giving the viewer a virtual perception as if the object were sitting inside the sphere and being moved by the viewer. The design goal is to develop a low-cost, real-time, and interactive 3D display tool. An off-the-shelf projector-camera pair is first calibrated via a simple but efficient algorithm. Vision-based methods are proposed to detect the sphere and track its subsequent motion. The projection image is generated based on the projective geometry among the projector, sphere, camera and the viewer. We describe how to allocate the view spot and warp the projection image. We also present the result and the performance evaluation of the system.

A fast and robust simultaneous pose tracking and structure recovery algorithm for augmented reality applications

A robust simultaneous pose tracking and structure recovery algorithm based on the Interacting Multiple Model (IMM) for augmented reality applications is proposed in this paper. A set of three extended Kalman filters (EKFs), each describes a frequently occurring camera motion in real situations (general, pure translation, pure rotation), is applied within the IMM framework to track the pose of an object. Another set of EKFs, one filter for each model point, is used to refine the positions of the model features in the 3D space. The filters for pose tracking and structure refinement are executed in an interleaved manner. The results are used for inserting virtual objects into the original video footage. The performance of the algorithm is demonstrated with both synthetic and real data. Comparisons with different approaches have been performed and show that our method is more efficient and accurate.