Yisong Chen

Adaptive figure-ground classification

We propose an adaptive figure-ground classification algorithm to automatically extract a foreground region using a user-provided bounding-box. The image is first over-segmented with an adaptive mean-shift algorithm, from which background and foreground priors are estimated. The remaining patches are iteratively assigned based on their distances to the priors, with the foreground prior being updated online. A large set of candidate segmentations are obtained by changing the initial foreground prior. The best candidate is determined by a score function that evaluates the segmentation quality. Rather than using a single distance function or score function, we generate multiple hypothesis segmentations from different combinations of distance measures and score functions. The final segmentation is then automatically obtained with a voting or weighted combination scheme from the multiple hypotheses. Experiments indicate that our method performs at or above the current state-of-the-art on several datasets, with particular success on challenging scenes that contain irregular or multiple-connected foregrounds. In addition, this improvement in accuracy is achieved with low computational cost.

Planar metric rectification by algebraically estimating the image of the absolute conic

A new metric rectification method for planar homography is proposed based on a closed form algebraic solution of the image of the absolute conic on the image plane. Our solution allows shape measurement to be made directly on the image plane without explicitly computing the homography matrix or recoreringing the rectified image. We show that the invariance property of the relationship between the circular points and the absolute conic under projective transformation can effectively do planar metric rectification. In this approach, the image of the absolute conic is solved algebraically to achieve metric rectification based only on the vanishing line and the image of one arbitrary circle on the world plane extracted automatically from the image plane. The process of conic solving introduces no errors and the performance of the method is mainly dependent on the robustness of the straight line and ellipse fitting processes. The fitting scheme suggested in the paper is robust and give good results in most cases.

Hidden Markov Model Based Characterization of Content Access Patterns in an e-Learning Environment

Personalized education (PE) emphasizes the importance of individual differences in learning. To deliver personalized e-learning services and content, PE encompasses the abilities of identifying and understanding individual learners needs and competence so as to deploy appropriate learning pedagogy and content to enhance learning. In this paper, we introduce a hidden Markov model based classification approach to enable a multimedia e-learning system to characterize different types of users through their navigation or content access patterns. Our experiments show that the proposed approach is capable of assigning student users to their corresponding categories with high accuracies. The results of such classifications would find applications in adaptive user interface design, user profiling and as supportive tools in personalized e-learning

3D Texture Mapping in Multi-view Reconstruction

This paper proposes a novel framework for texture mapping of 3D models. Given a reconstructed 3D mesh model and a set of calibrated images, a high-quality texture mosaic of the surface can be created after the process of our method. We focus on avoiding noticeable seams, color inconsistency and ghosting artifacts, which is typically due to such facts as modeling inaccuracy, calibration error and photometric disagreement. We extend the multi-band blending technique in a principled manner and apply it to assemble texture images in different frequency domains elaborately. Meanwhile, self-occlusion and highlight problem is taken into account. Then a novel texture map creating method is employed. Experiments based on our 3D Reconstruction System show the effectiveness of our texturing framework.

Simultaneous tracking of rigid head motion and non-rigid facial animation by analyzing local features statistically

A quick and reliable model-based head motion tracking scheme is presented. In this approach, rigid head motion and non-rigid facial animation are robustly tracked simultaneously by statistically analyzing the local regions of several representative facial features. The features are defined and operated based on a mesh model that helps maintain a global constraint on the local features and avoid the time-consuming appearance computation. A statistical model is computed from a moderate training set that is obtained by synthesizing different poses from a given standard initial image. During tracking, feature-based local distributions are obtained directly from the video frames and the troublesome feature detection or model rendering process is avoided. The observed distribution is compared with the pre-computed statistical model and the tracking is achieved by minimizing an error function based on the maximum likelihood principle. Experimental results show that this tracking strategy is robust to a wide range of head motion, facial animation and partial occlusion. The tracking can be conducted in nearly real-time and is easy to recover from failures.

Full Camera Calibration from a Single View of Planar Scene

We present a novel algorithm that applies conics to realize reliable camera calibration. In particular, we show that a single view of two coplanar circles is sufficiently powerful to give a fully automatic calibration framework that estimates both intrinsic and extrinsic parameters. This method stems from the previous work of conic based calibration and calibration-free scene analysis. It eliminates many a priori constraints such as known principal point, restrictive calibration patterns, or multiple views. Calibration is achieved statistically through identifying multiple orthogonal directions and optimizing a probability function by maximum likelihood estimate. Orthogonal vanishing points, which build the basic geometric primitives used in calibration, are identified based on the fact that they represent conjugate directions with respect to an arbitrary circle under perspective transformation. Experimental results from synthetic and real scenes demonstrate the effectiveness, accuracy, and popularity of the approach.

Texture evolution: 3D texture synthesis from single 2D growable texture pattern

An efficient model-independent 3D texture synthesis algorithm based on texture growing and texture turbulence is presented to create vivid 3D solid texture from a single 2D growable texture pattern. Given a 2D texture pattern of some growable material, our technique is able to create an anisotropic 3D volumetric texture cube to simulate the evolution of the material in 3D. An effective tiling scheme is designed to save computation and storage costs. Target objects are directly dipped into the synthesized 3D texture volume to generate creative, sculpture-like models that can be visualized with interactive speed. Our method is conceptually intuitive, computationally fast, and storage efficient compared with other solid texturing methods. As opposed to conventional 2D texture mapping work on polygonal surfaces, our approach is capable of decorating 3D point-rendering systems seamlessly. Furthermore, our combination of texture turbulence and texture growing techniques provides an attractive way to synthesize and tile natural 2D texture patterns, or generate simple but interesting motion textures.

Minimizing Geometric Distance by Iterative Linear Optimization

This paper proposes an algorithm that solves planar homography by iterative linear optimization. we iteratively employ direct linear transformation (DLT) algorithm to robustly estimate the homography induced by a given set of point correspondences under perspective transformation. By simple on-the-fly homogeneous coordinate adjustment we progressively minimize the difference between the algebraic error and the geometric error. When the difference is sufficiently close to zero, the geometric error is equivalently minimized and the homography is reliably solved. Backward covariance propagation is employed to do error analysis. The experiments prove that the algorithm is able to find global minimum despite erroneous initialization. It gives very precise estimate at low computational cost and greatly outperforms existing techniques.

Single view metrology of wide-angle lens images

This paper presents a framework and the associated algorithms to perform 3D scene analysis from a single image with lens distortion. Previous work focuses on either making 3D measurements under the assumption of one or more ideal pinhole cameras or correcting the lens distortion up to a projective transformation with no additional metric analyses. In this work, we bridge the gap between these two areas of work by incorporating metric constraints into lens distortion correction to achieve metric calibration. Lens distortion parameters, especially the lens distortion center, can be precisely recovered with this approach. Subsequent 3D measurements can be made from the corrected image to recover scene structures. In addition, we propose an algorithm based on hybrid backward and forward covariance propagation to yield a quantitative analysis of the confidence of the results. Experimental results show that our approach simultaneously performs image correction and 3D scene analysis.

Enhanced Figure-Ground Classification With Background Prior Propagation

We present an adaptive figure-ground segmentation algorithm that is capable of extracting foreground objects in a generic environment. Starting from an interactively assigned background mask, an initial background prior is defined and multiple soft-label partitions are generated from different foreground priors by progressive patch merging. These partitions are fused to produce a foreground probability map. The probability map is then binarized via threshold sweeping to create multiple hard-label candidates. A set of segmentation hypotheses is formed using different evaluation scores. From this set, the hypothesis with maximal local stability is propagated as the new background prior, and the segmentation process is repeated until convergence. Similarity voting is used to select a winner set, and the corresponding hypotheses are fused to yield the final segmentation result. Experiments indicate that our method performs at or above the current state-of-the-art on several data sets, with particular success on challenging scenes that contain irregular or multiple-connected foregrounds.