Yu-Pao Tsai

A Bayesian approach to video object segmentation via merging 3D watershed volumes

In this letter, we propose a Bayesian approach to video object segmentation. Our method consists of two stages. In the first stage, we partition the video data into a set of three-dimensional (3-D) watershed volumes, where each watershed volume is a series of corresponding two-dimensional (2-D) image regions. These 2-D image regions are obtained by applying to each image frame the marker-controlled watershed segmentation, where the markers are extracted by first generating a set of initial markers via temporal tracking and then refining the markers with two shrinking schemes: the iterative adaptive erosion and the verification against a presimplified watershed segmentation. Next, in the second stage, we use a Markov random field to model the spatio-temporal relationship among the 3-D watershed volumes that are obtained from the first stage. Then, the desired video objects can be extracted by merging watershed volumes having similar motion characteristics within a Bayesian framework. A major advantage of this method is that it can take into account the global motion information contained in each watershed volume. Our experiments have shown that the proposed method has potential for extracting moving objects from a video sequence.We propose a Bayesian approach to video object segmentation, which consists of two stages. In the first stage, we partition the video data into a set of 3D watershed volumes, where each watershed volume is a series of corresponding 2D image regions. These 2D image regions are obtained by applying to each image frame the marker-controlled watershed segmentation. In the second stage, we use a Markov random field to model the spatio-temporal relationship among the 3D watershed volume. Then, the desired video objects can be extracted by merging watershed volumes having similar motion characteristics within a Bayesian framework Our experiments have shown that the proposed method has great potential in extracting moving objects from a video sequence.

Augmenting panoramas with object movies by generating novel views with disparity-based view morphing

Our goal is to augment a panorama with object movies in a visually 3D-consistent way. Notice that a panorama is recorded as one single 2D image and an object movie (OM) is composed of a set of 2D images taken around a 3D object. The challenge is how to integrate the above two sources of 2D images in a 3D-consistent way so that the user can easily manipulate object movies in a panorama. To solve this problem, we adopt a purely image-based approach that does not have to reconstruct the geometric models of the 3D objects to be inserted in the panorama. A critical issue of this method is how to generate the novel views required for showing an OM in different places of a panorama, and we have proposed a view morphing technique, called t-DBVM, to solve this problem. Our experiments have shown that this purely image-based approach can effectively generate visually convincing OM-augmented panoramas. This method has great potential for many applications that require integration of panoramas and object movies, such as virtual malls, virtual museum, and interior design. Copyright

A flexible display by integrating a wall-size display and steerable projectors

Many wall-size display systems are built to provide large-scale visualization. These systems may be quite successful for some limited applications, but are very inflexible, since these systems only have fixed display regions. This paper integrates steerable projectors whose beam can be moved under computer control onto a wall-size display system to strengthen its display ability. With the steerable projectors, the integrated display system, named Flexible Display, provide an extendable display region. This consists of a large-scale display region and several movable display regions, such that the integrated display system has great potential in the area of human-computer interaction and information visualization. This paper applies the Flexible Display to a virtual museum application to give the users fluent navigation experience. For the application, the Flexible Display provides the following functions: 1) intensity and resolution enhancement of sub-region of display wall, 2) information augmentation, and 3) “stepping user interfaces” for its viewers interacting with display wall.

Augmented stereo panoramas

Panoramas and object movies are popular image-based techniques for modeling and rendering 3D scenes and objects. In this paper, we present a method which allows the authors of virtual exhibition systems to produce an augmented stereo panorama by interactively integrating stereo object movies into a stereo panorama. When navigating in the augmented stereo panorama with a stereoscopic display, the user can directly browse the stereo object movies that he is interested in. With augmented stereo panoramas, the user can enjoy more persuasive interaction with better depth perception. To our best knowledge, this paper is the first work to integrate stereoscopic image-based scenes with stereo object movies.

A method for calibrating a motorized object rig

Object Movies (OMs) have been successfully used in many applications. However, the techniques for acquiring OMs still need to be improved if high-quality and efficient OMs are desired. In this paper, we present a method for calibrating a motorized object rig to facilitate the acquisition of OMs. We first apply the CPC kinematic model to formulate the 3D configuration of the device, and then propose a method to estimate the parameters of the CPC model of the device. Furthermore, a visual tool is provided for users to adjust the controllable axes of the rig according to the estimated results. After this calibration, more accurate camera parameters can be obtained and then be used for different purposes. In this work, we use the parameters to reconstruct, from an OM, the 3D model of the object, and then adjust the OM according to the center of the 3D model so that a high-quality OM can be obtained for rendering.

Background removal system for object movies

In this paper, we present an interactive system for removing the backgrounds from object movies. Our system automatically extracts initial segmentation results based on observed characteristics of object movies. These characteristics are (1) the distribution of background color is Gaussian, (2) the color difference between foreground and background is distinct, and (3) the background of the images set with the same tilt angle is static. The user can modify misclassified pixels in only a few frames. The corrected result is propagated to all frames through spatial and temporal coherence. After user manipulation, the alpha estimation process is performed to obtain the alpha values for pixels that are composed of both background and foreground. Our automatic process for obtaining initial segmentation results extracts most foreground and background pixels, and thus more accurate results are obtained with little user intervention.

Automatic Geometric and Photometric Calibration for Tiling Multiple Projectors with a Pan-Tilt-Zoom Camera

Research on creating a large, high-resolution, low-cost display system has become increasingly important due to the growing desire in many fields for bigger and better displays. The goal of this research is to build a seamless large-scale display system by tiling multiple projectors with the help of a pan-tilt-zoom camera. In order to achieve this goal, we took two tasks into consideration: geometric calibration and photometric calibration. Compared to the previous work, our method for geometric calibration is more accurate, thanks to the much higher resolution images acquired by combining several zoom-in images and doing lens correction for the camera at first. For photometric calibration, we achieve color uniformity among projectors by utilizing the same PTZ camera, which is calibrated once with a colorimeter in a factory. Furthermore, we adopted the technique of producing a high dynamic range image from several images of different exposures to increase the measurement accuracy of the camera. In our experiments, the average error of photometric measurement achieved by our method is less than the mean perceptibility tolerance. Our method has great potential for many applications that require large and high-resolution displays

Generation of Binocular Object Movies from Monocular Object Movies

Object movie (OM) is a popular technique for producing interactive 3D artifacts because of its simplicity in production and it photo-realistic ability to present the artifacts. At the same time, many stereoscopic vision techniques are developed for a variety of applications. However, the traditional approach for generating binocular object movies require duplicate effort compared with monocular ones both in the process of acquisition and image processing. Therefore, we propose a framework to generate stereo OMs from monocular ones with the help of an automatically constructed 3D model from the monocular OM. Here, a new representation of the 3D model, named billboard clusters, is proposed for efficient generating binocular views. In order to obtain better results, a novel approach to extract view-independent texture is developed in this work. Besides, billboard clusters can be used to compress the storage capacity of OMs, and to perform relighting so that the binocular OMs can be well augmented into virtual environments with different lighting conditions. This paper describes the methods in detail and reports on its wide applications.