Ronald Chung

Use of LCD Panel for Calibrating Structured-Light-Based Range Sensing System

Calibration is a crucial step in structured-light-based range sensing device. The step involves the determination of the intrinsic parameters of both the camera and the projector that constitute the device and the extrinsic parameters between the two instruments. The traditional solution requires the use of an external calibration object with an accurately measured pattern printed on it. This paper presents a calibration design that makes use of a liquid-crystal display (LCD) panel as the calibration object. The LCD panels planarity is of industrial grade and is thus dependable. The pattern shown on the LCD panel is programmable and is thus convenient to produce in high precision. We show that, with the design, the projector-and-camera system parameters can be calibrated with far fewer images with much higher accuracy. Extensive experiments are shown to illustrate the dramatic improvement in performance.

Stereo-motion with stereo and motion in complement

This paper presents a new approach of combining stereo vision and dynamic vision with the objective of retaining their advantages and removing their disadvantages. It is shown that, by assuming affine cameras, the stereo correspondences and motion correspondences, if organized in a particular way in a matrix, can be decomposed into: the 3D structure of the scene, the camera parameters, the motion parameters, and the stereo geometry. With this, the approach can infer stereo correspondences from motion correspondences, requiring only a time linear with respect to the size of the available image data. The approach offers the advantages of simpler correspondence, as in dynamic vision, and accurate reconstruction, as in stereo vision, even with short image sequences.

Use of monocular groupings and occlusion analysis in a hierarchical stereo system

We describe a hierarchical stereo system that computes a hierarchy of descriptions up the surface level from each view using a perceptual grouping technique and matches these features at the different levels. With the description and correspondence processes going hand in hand, we allow high-level abstract features to help reduce correspondence ambiguities, and we exploit the multiple views to help confirm the different levels of descriptions in the scene. Occlusion is a major problem in stereo analysis and is often not treated explicitly. We present a basic property of occlusions in stereo views and show how we can use this property incorporated with the structural descriptions to identify different types of occlusions in the scene. In particular, we identify depth discontinuities and limb boundaries and infer properties of surfaces that are visible only in one of the stereo views. We give some experimental results on scenes with curved objects and with multiple occlusions.

An Accurate and Robust Strip-Edge-Based Structured Light Means for Shiny Surface Micromeasurement in 3-D

Three-dimensional measurement of shiny or reflective surface is a challenging issue for optical-based instrumentations. In this paper, we present a novel structured light approach for direct measurement of shiny target so as to skip the coating preprocedure. In comparison with traditional image-intensity-based structured light coding strategies like sinusoidal and line patterns, strip edges not raw image intensities are encoded in the illuminated patterns. With strip edges generally better preserved than individual image intensity in the image data in the presence of surface reflections, such a coding strategy is more robust. To remove the periodic ambiguity within strip patterns, traditional Gray code patterns are adopted. To localize the strip edges more precisely, both positive and negative strip patterns are used. An improved zero-crossing feature detector that has subpixel accuracy is proposed for strip-edge localization. The experimental setup is configured with merely an off-the-shelf pico-projector and a camera. Extensive experiments including accuracy evaluation, comparison with previous structured light algorithms, and the reconstruction of some real shiny objects are shown to demonstrate the systems accuracy and endurance against reflective nature of surfaces.

Stereo correspondence from motion correspondence

This paper introduces a new framework for stereo correspondence recovery using one motion of a stereo rig. Both the stereo correspondence and the motion of the stereo rig are unknown. By combining the stereo geometry and the motion correspondence we are able to infer the stereo correspondence from motion correspondence without having to systematically use the intensity-based stereo matching algorithms. The stereo correspondence recovery consists of two consecutive steps: the first step uses metric data associated with the stereo rig while the second step uses feature correspondences only. Experiments involving real stereo pairs indicate the feasibility and robustness of the approach.

An Algebraic Approach to Camera Self-Calibration

Abstract This paper describes a new self-calibration method for a single camera undergoing general motions. It has the following main contributions. First, we establish new constraints which relate the intrinsic parameters of the camera to the rotational part of the motions. This derivation is purely algebraic. We propose an algorithm which simultaneously solves for camera calibration and the rotational part of motions. Second, we provide a comparison between the developed method and a Kruppa equation-based method. Extensive experiments on both synthetic and real image data show the reliability and outperformance of the proposed method. The practical contribution of the method is its interesting convergence property compared with that of the Kruppa equations method.

Making any planar surface into a touch-sensitive display by a mere projector and camera

We address how an HCI (Human-Computer Interface) with small device size, large display, and touch input facility can be made possible by a mere projector and camera. The realization is through the use of a properly embedded structured light sensing scheme that enables a regular light-colored table surface to serve the dual roles of both a projection screen and a touch-sensitive display surface. A random binary pattern is employed to code structured light in pixel accuracy, which is embedded into the regular projection display in a way that the user perceives only regular display but not the structured pattern hidden in the display. With the projection display on the table surface being imaged by a camera, the observed image data, plus the known projection content, can work together to probe the 3D world immediately above the table surface, like deciding if there is a finger present and if the finger touches the table surface, and if so at what position of the table surface the finger tip makes the contact. All the decisions hinge upon a careful calibration of the projector-camera-table surface system, intelligent segmentation of the hand in the image data, and exploitation of the homography mapping existing between the projectors display panel and the cameras image plane. Extensive experimentation including evaluation of the display quality, touch detection accuracy, and system efficiency are shown to illustrate the feasibility of the proposed realization.

Cooperative Stereo-Motion

One of the most interesting goals of computer vision is the 3D structure recovery of scenes. Traditionally, two cues are used: structure from motion and structure from stereo, two subfields with complementary sets of assumptions and techniques. This paper introduces a new general framework of cooperation between stereo and motion. This framework combines the advantages of both cues: (i) easy correspondence from motion and (ii) accurate 3D reconstruction from stereo. First, we show how the stereo matching can be recovered from motion correspondences using only geometric constraints. Second, we propose a method of 3D reconstruction of both binocular and monocular features using all stereo pairs in the case of a calibrated stereo rig. Third, we perform an analysis of the performance of the proposed framework as well as a comparison with an affine method. Experiments involving real and synthetic stereo pairs indicate that rich and reliable information can be derived from the proposed framework. They also indicate that robust 3D reconstruction can be obtained even with short image sequences.

Homography-based partitioning of curved surface for stereo correspondence establishment

Planar homography (collineation) is an image-to-image mapping that could be used to pinpoint stereo correspondences, but its usage has been limited to only planar scenes. This paper describes a mechanism that generalizes the use of planar homography for establishing stereo correspondences over curved scenes. Piecewise-linear approximation is used to describe curved scene, so that stereo correspondences over images of the scene are captured by a collection of local homographies. Unlike the classical approaches, the mechanism does not employ the smoothness heuristic in establishing correspondences, but is instead based upon the interplay of two processes in an iterative manner: (1) partitioning of the scene into local planar patches based upon the most current set of confirmed stereo correspondences; and (2) prediction of new stereo correspondences by the use of the local homographies defined by the partitions, plus confirmation of such predictions by the image data, thereby enlarging the set of confirmed correspondences in every iteration until no more improvement could be obtained. A key step of the mechanism is to decide what local planar homography, and thereby what planar patch, is to be used for correspondence prediction of any given unmatched feature in any given iteration of the mechanism. With knowledge of the epipolar geometry, homography could be defined by any 3 non-collinear feature correspondences. Thus for any given unmatched feature, there are choices of the aforementioned local homography as defined by any set of three non-collinear matched features in the vicinity of the feature. In this paper, we explore what criteria are to be used to decide which of such triplet sets of matched features should be used. We analyze what errors in correspondence prediction could come with a planar homography. In particular, we classify the errors into scene-related geometric error and computation-related algebraic error. We examine their effects by simulated data experiments, and propose two ways of deciding which local homography to adopt for any given unmatched feature point. Real image experiments show that the methods could lead to promising results.

Recovering building structures from stereo

Addresses the problem of extracting polyhedral building structures from a stereo pair of aerial intensity images. The authors describe a system that computes a hierarchy of descriptions such as segments, junctions, and links between junctions from each view, and matches these features at the different levels. Such high level features not only help reduce correspondence ambiguity during stereo matching, but also allow us to infer surface boundaries even though the boundaries may be broken because of noise and weak contrast. The authors hypothesize surface boundaries by examining global information such as continuity and coplanarity of linked edges in 3-D, rather than merely by looking at local depth information. When the walls of the buildings are visible, they also exploit the relationship among adjacent surfaces in a polyhedral object to help confirm the different levels of descriptions. The authors give some experimental results for aerial images taken from overhead views and oblique views.<<ETX>>