Seungmin Baek

Real-time 3D object pose estimation and tracking for natural landmark based visual servo

A real-time solution for estimating and tracking the 3D pose of a rigid object is presented for image-based visual servo with natural landmarks. The many state-of-the-art technologies that are available for recognizing the 3D pose of an object in a natural setting are not suitable for real-time servo due to their time lags. This paper demonstrates that a real-time solution of 3D pose estimation become feasible by combining a fast tracker such as KLT [7] [8] with a method of determining the 3D coordinates of tracking points on an object at the time of SIFT based tracking point initiation, assuming that a 3D geometric model with SIFT description of an object is known a-priori. Keeping track of tracking points with KLT, removing the tracking point outliers automatically, and reinitiating the tracking points using SIFT once deteriorated, the 3D pose of an object can be estimated and tracked in real-time. This method can be applied to both mono and stereo camera based 3D pose estimation and tracking. The former guarantees higher frame rates with about 1 ms of local pose estimation, while the latter assures of more precise pose results but with about 16 ms of local pose estimation. The experimental investigations have shown the effectiveness of the proposed approach with real-time performance.

Vision-Based Kidnap Recovery with SLAM for Home Cleaning Robots

Emerged as salient in the recent home appliance consumer market is a new generation of home cleaning robot featuring the capability of Simultaneous Localization and Mapping (SLAM). SLAM allows a cleaning robot not only to self-optimize its work paths for efficiency but also to self-recover from kidnappings for user convenience. By kidnapping, we mean that a robot is displaced, in the middle of cleaning, without its SLAM aware of where it moves to. This paper presents a vision-based kidnap recovery with SLAM for home cleaning robots, the first of its kind, using a wheel drop switch and an upward-looking camera for low-cost applications. In particular, a camera with a wide-angle lens is adopted for a kidnapped robot to be able to recover its pose on a global map with only a single image. First, the kidnapping situation is effectively detected based on a wheel drop switch. Then, for an efficient kidnap recovery, a coarse-to-fine approach to matching the image features detected with those associated with a large number of robot poses or nodes, built as a map in graph representation, is adopted. The pose ambiguity, e.g., due to symmetry is taken care of, if any. The final robot pose is obtained with high accuracy from the fine level of the coarse-to-fine hierarchy by fusing poses estimated from a chosen set of matching nodes. The proposed method was implemented as an embedded system with an ARM11 processor on a real commercial home cleaning robot and tested extensively. Experimental results show that the proposed method works well even in the situation in which the cleaning robot is suddenly kidnapped during the map building process.

Robust 3D Line Extraction from Stereo Point Clouds

The paper describes a robust method to extract 3D lines from stereo point clouds. This method combines 2D image information with 3D point clouds from a stereo camera. 2D lines are first extracted from the image in the stereo pair, followed by 3D line regression from the back-projected 3D point set of the images points in the detected 2D lines. In this paper, random sample consensus (RANSAC) is used to estimate 3D line from the 3D point set, the Mahalanobis distance from each 3D point to the 3D line is derived, and the statistically motivated distance measure is used to compute the support for the detected 3D line. Experimental results on real environment with high level of clutter, occlusion, and noise demonstrate the robustness of the algorithm.

Particle Filter Based Robust Recognition and Pose Estimation of 3D Objects in a Sequence of Images

A particle filter framework of multiple evidence fusion and model matching in a sequence of images is presented for robust recognition and pose estimation of 3D objects. It attempts to challenge a long-standing problem in robot vision, so called, how to ensure the dependability of its performance under the large variation in visual properties of a scene due to changes in illumination, texture, occlusion, as well as camera pose. To ensure the dependability, we propose a behavioral process in vision, where possible interpretations are carried probabilistically in space and time for further investigations till they are converged to a credible decision by additional evidences. The proposed approach features 1) the automatic selection and collection of an optimal set of evidences based on in-situ monitoring of environmental variations, 2) the derivation of multiple interpretations, as particles representing possible object poses in 3D space, and the assignment of their probabilities based on matching the object model with evidences, and 3) the particle filtering of interpretations in time with the additional evidences obtained from a sequence of images. The proposed approach has been validated by the stereo-camera based experimentation of 3D object recognition and pose estimation, where a combination of photometric and geometric features are used for evidences

A 3D IR Camera with Variable Structured Light for Home Service Robots

There has shown a significant interest in a high performance of, at the same time, a compact size and low cost of, 3D sensor, in reflection of a growing need of 3D environmental sensing for service robotics. One of the important requirements associated with such a 3D sensor is that sensing does not irritate or disturb human in any way while working in close and continuous contact with human. Furthermore, such a 3D sensor should be reliable and robust to the change of environmental illumination as service robots are required to work day and night. This paper presents a 3D IR camera with variable structured light that is human friendly and robust enough for application to home service robots. Infrared is chosen as the sensing medium in order to meet the requirement of human friendliness and robustness to illumination change. A Digital Mirror Device (DMD) is employed to generate and project variable patterns at a high speed for real-time operation. In implementation, we emphasize the integration of modular components to support real-time sensing and compactness in size. A number of real-world experimentations are conducted, including a human face, a statue, and a plastic model. The experimental results have demonstrated that the implemented 3D IR Camera is robust to illumination change, in addition to its advantage of human friendliness.