Jinyuan Jia

Object proposal by multi-branch hierarchical segmentation

Hierarchical segmentation based object proposal methods have become an important step in modern object detection paradigm. However, standard single-way hierarchical methods are fundamentally flawed in that the errors in early steps cannot be corrected and accumulate. In this work, we propose a novel multi-branch hierarchical segmentation approach that alleviates such problems by learning multiple merging strategies in each step in a complementary manner, such that errors in one merging strategy could be corrected by the others. Our approach achieves the state-of-the-art performance for both object proposal and object detection tasks, comparing to previous object proposal methods.

A GPU Based 3D Object Retrieval Approach Using Spatial Shape Information

In this paper, we present a novel 3D model alignment method by analyzing the voxels of 3D meshes and a visual similarity based 3D model matching and retrieving method using active tabu search. Firstly, each 3D model is voxelized and applied voxels based PCA transformation, then it is represented by six depth images which are projected by rendering in the PCA coordinate system. Hybrid descriptors are extracted from these depth images to represent the origin 3D model shape features. Matching and retrieving is performed when geometric manifold entropy based active tabu search is used to index all the models in the library by its associated sets of depth images, then the dissimilarity between 3D models are computed from this indexed depth images dataset. Finally, in order to accelerate our proposed approach, all the key operations were implemented on GPU platform using its high parallel architecture. Experimental results show that our proposed method achieve better shape matching effect and gain absolutely improvement in retrieval performances on the Princeton 3D Shape Benchmark database.

Intelligent tree modeling based on L-system

This paper proposes an intelligent approach to model realistic tree models using semi-automatic generated L-systems. This approach is different from the traditional way of constructing L-systems that normally requires expertise in botany and manually coding. In this approach, only a few parameters those are highly intuitive and powerful to control the features of the tree need to be specified by users. Then the tree can be modelled automatically using the generated L-system. This approach avoids the complexity of directly using L-system and provides considerable flexibilities to model various trees. Experiments show that it is feasible to model realistic trees using generated L-systems.

Lightweighting for Web3D visualization of large-scale BIM scenes in real-time

Semantics-guided lightweighting is proposed to reduce the amount of data processing in the front end by removing the redundant data and creating an IFC Lightweight Scene Graph (IFC_LSG).Double-Layered Sparse Voxel (DLSV) is proposed for data indexing to improve the access efficiency of real-time Web3D building visualization.Incremental Frustum of Interest (I-FOI) is proposed to manage the scene by combining the rendering pipeline and the current scene index. Display Omitted As a result of informatization in construction, Building Information Modeling (BIM) has now become a core technology for smart construction. We present a Web3D-based lightweighting solution for real-time visualization of large-scale BIM scenes, considering the redundancy, semantics, and the parameterization of BIM data under the limited resources of network bandwidth and web browsers. Taking the Industry Foundation Classes (IFC) as the input data format, we firstly conduct a semantics-guided lightweighting operation on the raw BIM scenes by removing the repetitive objects and parameterizing the swept surfaces. Secondly, we extract the exterior products from the raw BIM buildings for visibility culling and construct a Double-Layered Sparse Voxel (DLSV) index based on sparse voxelization. Thirdly, we integrate the above two together into a new data structure named Incremental Frustum of Interest (I-FOI) to manage the scene data in real-time. Our experiments demonstrate that: (1) with the semantics information, our method is able to significantly reduce the redundancy of raw large-scale BIM scenes; (2) the DLSV structure supports progressive data loading and facilitates the indoor/outdoor visibility culling efficiently; and (3) the I-FOI introduces a frustum incremental-driven mechanism into progressive data loading or unloading to improve the efficiency of resource consumption.

Fast accessing Web3D contents using lightweight progressive meshes

Accessing Web3D contents is relatively slow through Internet under limited bandwidth. Preprocessing of 3D models can certainly alleviate the problem, such as 3D compression and progressive meshes (PM). But none of them considers the similarity between components of a 3D model, so that we could take advantage of this to further improve the efficiency. This paper proposes a similarity‐aware data reduction method together with PM, called lightweight progressive meshes (LPM). LPM aims to excavate similar components in a 3D model, generates PM representation of each component left after removing redundant components, and organizes all the processed data using a structure called lightweight scene graph. The proposed LPM possesses four significant advantages. First, it can minimize the file size of 3D model dramatically without almost any precision loss. Because of this, minimal data is delivered. Second, PM enables the delivery to be progressive, so called streaming. Third, when rendering at client side, due to lightweight scene graph, decompression is not necessary and instanced rendering is fully exerted. Fourth, it is extremely efficient and effective under very limited bandwidth, especially when delivering large 3D scenes. Performance on real data justifies the effectiveness of our LPM, which improves the state‐of‐the‐art in accessing Web3D contents. Copyright

Image-based lightweight tree modeling

This paper presents a novel lightweight tree modeling approach for constructing large scale online virtual forestry on Web. It firstly recovers 3D skeleton of the visible trunk from two source images of a tree, then extracts the rules and parameters of tree L-system from the recovered skeleton, and parses the parametric L-system into very lightweight tree Web3D files. Comparing with rule based tree modeling methods e.g. L-system and AMAP, our method is more convenient for users without requiring botany expertise. Furthermore, our method inherits the merits of both image based tree modeling and rules based tree modeling. Comparing with such 3D modelers as 3DMAX and MAYA, our method is more efficient and economical for users to avoid their heavily manual modeling labors. More important, it can generate very lightweight Web3D tree files even with 1K-2K, which are photorealistic in shape and structure, Experimental results show that the feasibility and perspective of our proposed method in WebVR applications.

An integer incremental AOI algorithm for progressive downloading of large scale VRML environments

Progressive data transmission is critical for implementing interactive walkthrough of large scale distributed virtual environments on Internet. This paper proposes an incremental AOI (Area of Interests) algorithm to determine dynamically which VRML objects should be downloaded while the viewpoint is moving in a VRML environment at each step. Our method has three major steps: (1) to divide the entire VRML ground uniformly into a rectangular mesh (m*n grids) along x-axis and y-axis respectively and store all the VRML objects overlapped by each grid into an adjacent list; (2) statically to determine all the grids overlapped by an AOI according to spatial coherence; (3) dynamically to determine newly visible and invisible grids overlapped by a moving AOI at each step, according to the temporal coherence between two consecutive AOI circles; (4) at each step, only newly overlapped VRML objects should be downloaded and rendered incrementally. Thus, the required VRML data downloads can be decreased dramatically at each moving step. Further, the proposed AOI algorithm is an integer incremental algorithm without multiplication, divisions and floating operations, and also it is suitable for hardware implementation.

LPM: lightweight progressive meshes towards smooth transmission of Web3D media over internet

Transmission of Web3D media over the Internet can be slow, especially when downloading huge 3D models through relatively limited bandwidth. Currently, 3D compression and progressive meshes are used to alleviate the problem, but these schemes do not consider similarity among the 3D components, leaving rooms for improvement in terms of efficiency. This paper proposes a similarity-aware 3D model reduction method, called Lightweight Progressive Meshes (LPM). The key idea of LPM is to search similar components in a 3D model, and reuse them through the construction of a Lightweight Scene Graph (LSG). The proposed LPM offers three significant benefits. First, the size of 3D models can be reduced for transmission without almost any precision loss of the original models. Second, when rendering, decompression is not needed to restore the original model, and instanced rendering can be fully exploited. Third, it is extremely efficient under very limited bandwidth, especially when transmitting large 3D scenes. Performance on real data justifies the effectiveness of our LPM, which improves the state-of-the-art in Web3D media transmission.

Sketch-Based Retrieval Using Content-Aware Hashing

In this paper, we introduce a generic hashing-based approach. It aims to facilitate sketch-based retrieval on large datasets of visual shapes. Unlike previous methods where visual descriptors are extracted from overlapping grids, a content-aware selection scheme is proposed to generate candidate patches instead. Meanwhile, the saliency of each patch is efficiently estimated. Locality-sensitive hashing LSH is employed to integrate and capture both the content and saliency of patches, as well as the spatial information of visual shapes. Furthermore, hash codes are indexed so that a query can be processed in sub-linear time. Experiments on three standard datasets in terms of hand drawn shapes, images and 3D models demonstrate the superiority of our approach.

3D tree skeletonization from multiple images based on PyrLK optical flow

We introduce a novel 3D tree skeletonization method from multiple images.Affine transformation and iterative tracking are added to PyrLK optical flow.Parallel voxel flooding is proposed for determining point cloud of single branch.L-System extraction algorithm for tree skeletons is proposed. Display Omitted 3D tree models are widely applied to construct large-scale virtual scenes. However, converting real trees into computer representation faces two main problems. One is the low quality of reconstructed point cloud. The other is that the skeletonization produces inaccurate results due to the complex structure of trees. We propose a novel pipeline to reconstruct automatically accurate 3D tree skeletons from sequential tree images. It involves three steps: (1) Pre-processing: an optical flow based feature-matching algorithm is proposed to acquire high-quality point clouds. (2) Shape-based parallel skeletonization: tree shape is taken into consideration to detect specific structure features of trees during parallel flooding. (3) Post-processing: an L-System extraction algorithm is applied on the skeletons and the lightweight representation is obtained to meet various requirements. The effectiveness of our approach is demonstrated through experiments.