Tianwei Shen

Graph-Based Consistent Matching for Structure-from-Motion

Pairwise image matching of unordered image collections greatly affects the efficiency and accuracy of Structure-from-Motion (SfM). Insufficient match pairs may result in disconnected structures or incomplete components, while costly redundant pairs containing erroneous ones may lead to folded and superimposed structures. This paper presents a graph-based image matching method that tackles the issues of completeness, efficiency and consistency in a unified framework. Our approach starts by chaining all but singleton images using a visual-similarity-based minimum spanning tree. Then the minimum spanning tree is incrementally expanded to form locally consistent strong triplets. Finally, a global community-based graph algorithm is introduced to strengthen the global consistency by reinforcing potentially large connected components. We demonstrate the superior performance of our method in terms of accuracy and efficiency on both benchmark and Internet datasets. Our method also performs remarkably well on the challenging datasets of highly ambiguous and duplicated scenes.

Structure-driven facade parsing with irregular patterns

We propose a novel method for recognizing irregular patterns in facades. An irregular pattern is an incomplete 2D grid, representing the placements of repetitive structural architectural objects (e.g., windows), which is capable of being generalized to a variety of facade structures. To effectively recognize such a pattern, we jointly model objects and object structures in a unified Marked Point Process framework, where the architectural objects are abstracted as sparsely populated geometric entities and the pairwise spatially interactions are modeled as elliptical repulsion fields. To optimize the proposed model, we introduce a structure-driven Monte Carlo Markov Chain (MCMC) sampler, by which the irregular pattern hypotheses are iteratively constructed in a bottom-up manner and verified in a top-down manner. The solution space is explored more efficiently for fast convergence. Extensive experiments have shown the efficiency and accuracy of our method of parsing a large category of facades.

Color Correction for Image-Based Modeling in the Large

Current texture creation methods for image-based modeling suffer from color discontinuity issues due to drastically varying conditions of illumination, exposure and time during the image capturing process. This paper proposes a novel system that generates consistent textures for triangular meshes. The key to our system is a color correction framework for large-scale unordered image collections. We model the problem as a graph-structured optimization over the overlapping regions of image pairs. After reconstructing the mesh of the scene, we accurately calculate matched image regions by re-projecting images onto the mesh. Then the image collection is robustly adjusted using a non-linear least square solver over color histograms in an unsupervised fashion. Finally, a connectivity-preserving edge pruning method is introduced to accelerate the color correction process. This system is evaluated with crowdsourcing image collections containing medium-sized scenes and city-scale urban datasets. To the best of our knowledge, this system is the first consistent texturing system for image-based modeling that is capable of handling thousands of input images.