Xi Zhang

Learning from Synthetic Data Using a Stacked Multichannel Autoencoder

Learning from synthetic data has many important and practical applications, An example of application is photo-sketch recognition. Using synthetic data is challenging due to the differences in feature distributions between synthetic and real data, a phenomenon we term synthetic gap. In this paper, we investigate and formalize a general framework -- Stacked Multichannel Autoencoder (SMCAE) that enables bridging the synthetic gap and learning from synthetic data more efficiently. In particular, we show that our SMCAE can not only transform and use synthetic data on the challenging face-sketch recognition task, but that it can also help simulate real images, which can be used for training classifiers for recognition. Preliminary experiments validate the effectiveness of the framework.

Alignment of 3D Building Models with Satellite Images Using Extended Chamfer Matching

Large scale alignment of 3D building models and satellite images has many applications ranging from realistic 3D city modeling to urban planning. In this paper, we address this problem by matching the 2D projection of the building roofs and detected edges of satellite images. To better handle noise and occlusions in alignment, the proposed approach seeks an optimal matching location using an extended Chamfer matching algorithm. In addition the proposed approach attempt to optimize the alignment within large region using a global constraint. We show that the proposed approach can estimate the alignment of matching parts and produce robust result under occlusion. We test the proposed algorithm on two different datasets that covers the downtown areas of San Francisco and Chicago. The results show that the proposed algorithm significantly improves the registration accuracy while maintaining consistent performance.

A learning-based approach for automated quality assessment of computer-rendered images

Computer generated images are common in numerous computer graphics applications such as games, modeling, and simulation. There is normally a tradeoff between the time allocated to the generation of each image frame and and the quality of the image, where better quality images require more processing time. Specifically, in the rendering of 3D objects, the surfaces of objects may be manipulated by subdividing them into smaller triangular patches and/or smoothing them so as to produce better looking renderings. Since unnecessary subdivision results in increased rendering time and unnecessary smoothing results in reduced details, there is a need to automatically determine the amount of necessary processing for producing good quality rendered images. In this paper we propose a novel supervised learning based methodology for automatically predicting the quality of rendered images of 3D objects. To perform the prediction we train on a data set which is labeled by human observers for quality. We are then able to predict the quality of renderings (not used in the training) with an average prediction error of roughly 20%. The proposed approach is compared to known techniques and is shown to produce better results.

CGMOS: Certainty Guided Minority OverSampling

Handling imbalanced datasets is a challenging problem that if not treated correctly results in reduced classification performance. Imbalanced datasets are commonly handled using minority oversampling, whereas the SMOTE algorithm is a successful oversampling algorithm with numerous extensions. SMOTE extensions do not have a theoretical guarantee during training to work better than SMOTE and in many instances their performance is data dependent. In this paper we propose a novel extension to the SMOTE algorithm with a theoretical guarantee for improved classification performance. The proposed approach considers the classification performance of both the majority and minority classes. In the proposed approach CGMOS (Certainty Guided Minority OverSampling) new data points are added by considering certainty changes in the dataset. The paper provides a proof that the proposed algorithm is guaranteed to work better than SMOTE for training data. Further, experimental results on 30 real-world datasets show that CGMOS works better than existing algorithms when using 6 different classifiers.

Learning-based roof style classification in 2D satellite images

Accurately recognizing building roof style leads to a much more realistic 3D building modeling and rendering. In this paper, we propose a novel system for image based roof style classification using machine learning technique. Our system is capable of accurately recognizing four individual roof styles and a complex roof which is composed of multiple parts. We make several novel contributions in this paper. First, we propose an algorithm that segments a complex roof to parts which enable our system to recognize the entire roof based on recognition of each part. Second, to better characterize a roof image, we design a new feature extracted from a roof edge image. We demonstrate that this feature has much better performance compared to recognition results generated by Histogram of Oriented Gradient (HOG), Scale-invariant Feature Transform (SIFT) and Local Binary Patterns (LBP). Finally, to generate a classifier, we propose a learning scheme that trains the classifier using both synthetic and real roof images. Experiment results show that our classifier performs well on several test collections.

Learning from synthetic models for roof style classification in point clouds

Automatic roof style classification using point clouds is useful and can be used as a prior knowledge in various applications, such as the construction of 3D models of real-world buildings. Previous classification approaches usually employ heuristic rules to recognize roof style and are limited to a few roof styles. In this paper, the recognition of roof style is done by a roof style classifier which is trained based on bag of words features extracted from a point cloud. In the computation of bag of words features, a key challenge is the generation of the codebook. Unsupervised learning is often misguided easily by the data and detects uninteresting patterns within the data. In contrast, we propose to integrate existing knowledge of roof structure and cluster the points of target roof styles into several semantic classes which can then be used as code words in the bag of words model. We use synthetic variants of these code words to train a semantics point classifier. We evaluate our approach on two datasets with different levels of degradations. We compare the results of our approach with two unsupervised learning algorithms: K-Means and Gaussian Mixture Model. We show that our approach achieve higher accuracy in classification of the roof styles and maintains consistent performance among different datasets.

Layered Optical Flow Estimation Using a Deep Neural Network with a Soft Mask

Using a layered representation for motion estimation has the advantage of being able to cope with discontinuities and occlusions. In this paper, we learn to estimate optical flow by combining a layered motion representation with deep learning. Instead of pre-segmenting the image to layers, the proposed approach automatically generates a layered representation of optical flow using the proposed soft-mask module. The essential components of the soft-mask module are maxout and fuse operations, which enable a disjoint layered representation of optical flow and more accurate flow estimation. We show that by using masks the motion estimate results in a quadratic function of input features in the output layer. The proposed soft-mask module can be added to any existing optical flow estimation networks by replacing their flow output layer. In this work, we use FlowNet as the base network to which we add the soft-mask module. The resulting network is tested on three well-known benchmarks with both supervised and unsupervised flow estimation tasks. Evaluation results show that the proposed network achieve better results compared with the original FlowNet.

Stacked multichannel autoencoder – an efficient way of learning from synthetic data

Learning from synthetic data has many important applications in case where sufficient amounts of labeled data are not available. Using synthetic data is challenging due to differences in feature distributions between synthetic and actual data, a phenomenon we term synthetic gap. In this paper, we investigate and formalize a general framework – Stacked Multichannel Autoencoder (SMCAE) that enables bridging the synthetic gap and learning from synthetic data more efficiently. In particular, we show that our SMCAE can not only transform and use synthetic data on a challenging face-sketch recognition task, but that it can also help simulate real images which can be used for training classifiers for recognition. Preliminary experiments validate the effectiveness of the proposed framework.

SIG SPATIAL CUP report: constrained map generalization

The 22nd ACM SIGSPATIAL Conference on Advances in Geographic Information Systems (GIS) was held in November of 2014 in Dallas, Texas, US. Following the success of last two events, we organized the 3rd programming contest associated with the conference, called the SIGSPATIAL GIS 2014. The subject of the competition was constrained generalization, which aims to generalize geometric features in the context of topological constraints. We describe the contest details, and the results, as well as the lessons learned during the process.