Yanghai Tsin

Stereo matching with linear superposition of layers

In this paper, we address stereo matching in the presence of a class of non-Lambertian effects, where image formation can be modeled as the additive superposition of layers at different depths. The presence of such effects makes it impossible for traditional stereo vision algorithms to recover depths using direct color matching-based methods. We develop several techniques to estimate both depths and colors of the component layers. Depth hypotheses are enumerated in pairs, one from each layer, in a nested plane sweep. For each pair of depth hypotheses, matching is accomplished using spatial-temporal differencing. We then use graph cut optimization to solve for the depths of both layers. This is followed by an iterative color update algorithm which we proved to be convergent. Our algorithm recovers depth and color estimates for both synthetic and real image sequences.

Learn to Track Edges

Reliability of a model-based edge tracker critically depends on its ability to establish correct correspondences between points on the model edges and edge pixels in an image. This is a non-trivial problem especially in the presence of large inter-frame motions and in cluttered environments. We propose an online learning approach to solving this problem. An edge pixel is represented by a descriptor composed of a small segment of intensity patterns. From training examples the algorithm utilizes the randomized forest model to learn a posteriori distribution of correspondence given the descriptor. In a new frame, the edge pixels are classified using maximum a posteriori (MAP) estimation. The proposed method is very powerful and it enables us to apply the proposed tracker to many previously impossible scenarios with unprecedented robustness.

Exploiting Occluding Contours for Real-Time 3D Tracking: A Unified Approach

Model-based 3D object tracking is fast and robust using 3D edges. However, traditional edge-based approaches have difficulty handling occluding contours of curved surfaces, since they are not static model edges but change with the viewpoint. In this paper we propose a unified approach to edge-based tracking where 3D edges including occluding contours are utilized. This is achieved through an analysis of local surface differential geometry, which provides the foundation for incorporating occluding contours of curved surfaces into edge-based tracking. This approach uses a simple parametrization of both types of model edges within the same framework. The proposed method has been tested within the context of an existing edge-based tracking system. The system can track both types of model edges in a very fast and robust manner. Experimental results on both synthetic and real scenes are provided, which confirm that occluding contours improve real-time 3D tracking performance.

A Deformation Tracking Approach to 4D Coronary Artery Tree Reconstruction

This paper addresses reconstruction of a temporally deforming 3D coronary vessel tree, i.e., 4D reconstruction from a sequence of angiographic X-ray images acquired by a rotating C-arm. Our algorithm starts from a 3D coronary tree that was reconstructed from images of one cardiac phase. Driven by gradient vector flow (GVF) fields, the method then estimates deformation such that projections of deformed models align with X-ray images of corresponding cardiac phases. To allow robust tracking of the coronary tree, the deformation estimation is regularized by smoothness and cyclic deformation constraints. Extensive qualitative and quantitative tests on clinical data sets suggest that our algorithm reconstructs accurate 4D coronary trees and regularized estimation significantly improves robustness. Our experiments also suggest that a hierarchy of deformation models with increasing complexities are desirable when input data are noisy or when the quality of the 3D model is low.

Explicit 3D Modeling for Vehicle Monitoring in Non-overlapping Cameras

Vehicles are indispensable in modern life. The capability of monitoring them over a long range can play significant roles in many surveillance applications. However, due to high mobility of vehicles, tracking them is difficult and we need to utilize a large network of cameras and reason on discrete sets of observations made from non-overlapping cameras. In this paper, we introduce enabling techniques for such a surveillance need. Specifically, we build explicit 3D models and use them for vehicle signature extraction and matching. The algorithm uses a single active shape model (ASM) for all consumer vehicles. After detecting presence of a vehicle, \eg, by background subtraction, our algorithm then reconstructs a texture mapped 3D model. 3D car models enable us to monitor vehicles in many novel ways otherwise impossible. Two use cases are provided.

Image-based respiratory motion compensation for fluoroscopic coronary roadmapping

We present a new image-based respiratory motion compensation method for coronary roadmapping in fluoroscopic images. A temporal analysis scheme is proposed to identify static structures in the image gradient domain. An extended Lucas-Kanade algorithm involving a weighted sum-of-squared-difference (WSSD) measure is proposed to estimate the soft tissue motion in the presence of static structures. A temporally compositional motion model is used to deal with large image motion incurred by deep breathing. Promising results have been shown in the experiments conducted on clinical data.

Layout consistent segmentation of 3-D meshes via conditional random fields and spatial ordering constraints

We address the problem of 3-D Mesh segmentation for categories of objects with known part structure. Part labels are derived from a semantic interpretation of non-overlapping subsurfaces. Our approach models the label distribution using a Conditional Random Field (CRF) that imposes constraints on the relative spatial arrangement of neighboring labels, thereby ensuring semantic consistency. To this end, each label variable is associated with a rich shape descriptor that is intrinsic to the surface. Randomized decision trees and cross validation are employed for learning the model, which is eventually applied using graph cuts. The method is flexible enough for segmenting even geometrically less structured regions and is robust to local and global shape variations.

Globally optimal affine epipolar geometry from apparent contours

We study the problem of estimating the epipolar geometry from apparent contours of smooth curved surfaces with affine camera models. Since apparent contours are viewpoint dependent, the only true image correspondences are projections of the frontier points, i.e., surface points whose tangent planes are also their epipolar planes. However, frontier points are unknown a priori and must be estimated simultaneously with epipolar geometry. Previous approaches to this problem adopt local greedy search methods which are sensitive to initialization, and may get trapped in local minima. We propose the first algorithm that guarantees global optimality for this problem. We first reformulate the problem using a separable form that allows us to search effectively in a 2D space, instead of on a 5D hypersphere in the classical formulation. Next, in a branch-and-bound algorithm we introduce a novel lower bounding function through interval matrix analysis. Experimental results on both synthetic and real scenes demonstrate that the proposed method is able to quickly obtain the optimal solution.