Ruoyu Yang

Automatic analysis and integration of architectural drawings

Recognition and integration of 2D architectural drawings provide a sound basis for automatically evaluating building designs, simulating safety, estimating construction cost or planning construction sequences. To accomplish these targets, difficulties come from (1) an architectural project is usually composed of a series of related drawings, (2) 3D information of structural objects may be expressed in 2D drawings, annotations, tables, or the composites of above expressions, and (3) a large number of disturbing graphical primitives in architectural drawings complicate the recognition processes. In this paper, we propose new methods to recognize typical structural objects and architectural symbols. Then the recognized results on the same floor and drawings of different floors will be integrated automatically for accurate 3D reconstruction.

Symbol Recognition by Multiresolution Shape Context Matching

We present a multi resolution scheme for symbol representation and recognition based on statistical shape features. We define a symbol as a set of shape points, each of which is then described by a pyramid of shape context features. The pyramid is constructed by successively partitioning the image surrounding one shape point into increasingly finer sub-regions and computing the local shape context descriptor inside each sub-region. To recognize a symbol, we compute the optimal matching between symbol prototypes and the image region, based on the weighted distance measurements across various scales. We also define an adaptive surround suppression measure that assigns different weights to the shape point depending on the complexity of its surrounding context, so as to reduce the effect of local intersections to shape matching. The experimental results show the effectiveness of the proposed shape context pyramid matching method as well as its promising aspects in handling intersecting symbols.

Symbol Recognition Combining Vectorial and Pixel-Level Features for Line Drawings

In this paper, we present an approach for symbol representation and recognition in line drawings, integrating both the vector-based structural description and pixel-level statistical features of the symbol. For the former, a vectorial template is defined on the basis of the vectorization model and exploited in segmenting symbols from the line network. For the latter, a Radon-transform-based signature is employed to characterize shapes on the symbol and the components level. Experimental results on real technical drawings are presented to show the promising aspect of our approach.

Action Segmentation and Recognition Based on Depth HOG and Probability Distribution Difference

The paper presents a method to automatically separate consecutive human actions into subsegments and recognize them. The 3D positions of the joints tracked by depth camera like Kinect sensors and the depth motion maps (DMMs) are used in the method. Both of the two types of data contain useful information to help us extract features for each action video. However, they are also full of noise. So we combine the pairwise relative positions of the 3D joints (Skeleton Joints) and Histograms of Oriented Gradients (HOG) calculated from the DMM together to improve the feature representation. A SVM-based classification ensemble is built to achieve the recognition result. We also build a Probability-Distribution-Difference (PDD) based dynamic boundary detection framework to segment consecutive actions before applying recognition. The segmentation framework is online and reliable. The experimental results applied to the Microsoft Research Action3D dataset outperform the state of the art methods.

Knowledge Extraction from Structured Engineering Drawings

As a typical type of structured documents, table drawings are widely used in engineering fields. Knowledge extraction of such structured documents plays an important role in automatic interpretation systems. In this paper, we propose a new knowledge extraction method based on automatically analyzing drawing layout and extracting physical or logical structures from the given engineering table drawings. Then based on the automatic interpretation results, we further propose normalization method to integrate varied types of engineering tables with other engineering drawings and extract implied domain knowledge.

A dynamic-rule-based framework of engineering drawing recognition and interpretation system

This paper introduces the idea that recognition and interpretation of engineering drawings should be two interwoven phases, with each providing feedback to another, and applies this idea to a dynamic-rule-based method. Recognition rules with attributes are obtained by an automatic object feature extraction procedure, and stored in rule database. During the recognition phase, rules are firstly selected according to two attributes, domain and priority. Then the thresholds of the rules are adjusted automatically to obtain better match results and their priorities are modified dynamically to improve recognition efficiency. Especially, the interpretation phase based on the recognition is also valued in validating and rectifying the recognition result automatically and efficiently. This approach was implemented in a system for recognizing and interpreting architectural structure drawings, and has shown to embody good self-adaptability to various drafting conventions.

Interactive particle filter with occlusion handling for multi-target tracking

Object tracking under occlusion sense is a challenging task. Although appearance-based trackers have been greatly improved in the last decade, they are still struggling with this task. Particle filter tracking has been proven as an efficient way which could overcome nonlinear situations. Unfortunately, conventional particle filter approach encounters tracking failure during severe occlusions. In this paper, we propose an interactive particle filter method, by analyzing the occlusion relationship between different targets, the proposed algorithm select different appearance model adaptively for similarity measurement and then update the particle weight. Our method successfully resolved mutual occlusion problem in tracking multi pedestrians, experimental results show that even target is completely occluded and its trajectory is unpredictable, our algorithm is still able to achieve accurate tracking results.

A new shape descriptor for object recognition and retrieval

We present a new shape descriptor for measuring the similarity between shapes and exploit it in graphical object recognition and retrieval. By statistically integrating the local length-ratio and angle constraints between contour points relative to the shape skeleton, we construct the shape descriptor capturing the global spatial distribution of the shape contour. Then, the dissimilarity between two shapes is computed as a weighted sum of matching errors between corresponding constraint histograms. Experimental results are presented for symbols and shapes data set, showing the effectiveness of our method.

3D building reconstruction based on interpretation of architectural drawings

Many existing 3D reconstruction algorithms either assume a perfect setting or focus only on rebuilding a single object. For architectural drawings, which contain many kinds of abstract information, description formats, and a large number of objects, these algorithms cannot reconstruct 3D models of entire buildings efficiently. This paper presents a new method for 3D reconstruction of buildings through interpreting three-view drawings automatically. Firstly, the feature extraction and recognition algorithm are designed for interpreting the drawings and obtaining the contours of building components. Next, special algorithms including reorientation of orthogonal views, regeneration of omitted views, and unification of coordinate systems are designed mainly based on interpretation and domain knowledge. At last, integrated 3D model of a whole building can be reconstructed and easily used in later automatic applications. Experiments showed that the method is especially efficient and appropriate for reconstruction of buildings composed of large number of components appearing in many different drawings.

Continuous Motion Recognition in Depth Camera Based on Recurrent Neural Networks and Grid-based Average Depth

Inspired by the success of using RNN in some other fields, we propose to apply the RNN to recognize human motion based on depth data. RNN can directly model the depth sequence on the time axis, and learn the temporal information more naturally. For represent the skeleton and depth information in video, we use Orderlet features and Grid-based Average Depth (GbAD) proposed in this paper. Finally, we evaluate our models on the MSR 3D Online Action Dataset in comparison with the state-of-the-art methods. Experimental results show that the proposed models outperforms other ones.