Xuesong Wang

Research and development of neural network ensembles: a survey

A Neural Network Ensemble (NNE) combines the outputs of several individually trained neural networks in order to improve generalization performance. This article summarizes different approaches on the development and the latest studies on NNE. The introduction of the basic principles of NNE is followed by detailed descriptions of individual neural network generation method, conclusion generation method and fusion based on granular computing and NNE. In addition, for each of these methods we provide a short taxonomy in terms of their relevant characteristics, and analyze several of NNE applications, classic algorithms and contributions on various fields.

Random Forest Classifier for Zero-Shot Learning Based on Relative Attribute

For the zero-shot image classification with relative attributes (RAs), the traditional method requires that not only all seen and unseen images obey Gaussian distribution, but also the classifications on testing samples are made by maximum likelihood estimation. We therefore propose a novel zero-shot image classifier called random forest based on relative attribute. First, based on the ordered and unordered pairs of images from the seen classes, the idea of ranking support vector machine is used to learn ranking functions for attributes. Then, according to the relative relationship between seen and unseen classes, the RA ranking-score model per attribute for each unseen image is built, where the appropriate seen classes are automatically selected to participate in the modeling process. In the third step, the random forest classifier is trained based on the RA ranking scores of attributes for all seen and unseen images. Finally, the class labels of testing images can be predicted via the trained RF. Experiments on Outdoor Scene Recognition, Pub Fig, and Shoes data sets show that our proposed method is superior to several state-of-the-art methods in terms of classification capability for zero-shot learning problems.

Heterogeneous domain adaptation network based on autoencoder

Abstract Heterogeneous domain adaptation is a more challenging problem than homogeneous domain adaptation. The transfer effect is not ideally caused by shallow structure which cannot adequately describe the probability distribution and obtain more effective features. In this paper, we propose a heterogeneous domain adaptation network based on autoencoder, in which two sets of autoencoder networks are used to project the source-domain and target-domain data to a shared feature space to obtain more abstractive feature representations. In the last feature and classification layer, the marginal and conditional distributions can be matched by empirical maximum mean discrepancy metric to reduce distribution difference. To preserve the consistency of geometric structure and label information, a manifold alignment term based on labels is introduced. The classification performance can be improved further by making full use of label information of both domains. The experimental results of 16 cross-domain transfer tasks verify that HDANA outperforms several state-of-the-art methods.

Domain adaptation network based on hypergraph regularized denoising autoencoder

Domain adaptation learning aims to solve the classification problems of unlabeled target domain by using rich labeled samples in source domain, but there are three main problems: negative transfer, under adaptation and under fitting. Aiming at these problems, a domain adaptation network based on hypergraph regularized denoising autoencoder (DAHDA) is proposed in this paper. To better fit the data distribution, the network is built with denoising autoencoder which can extract more robust feature representation. In the last feature and classification layers, the marginal and conditional distribution matching terms between domains are obtained via maximum mean discrepancy measurement to solve the under adaptation problem. To avoid negative transfer, the hypergraph regularization term is introduced to explore the high-order relationships among data. The classification performance of the model can be improved by preserving the statistical property and geometric structure simultaneously. Experimental results of 16 cross-domain transfer tasks verify that DAHDA outperforms other state-of-the-art methods.

Semi-supervised transfer discriminant analysis based on cross-domain mean constraint

In this paper, a novel semi-supervised feature extraction algorithm, i.e., semi-supervised transfer discriminant analysis (STDA) with knowledge transfer capability is proposed, based on the traditional algorithm that cannot get adapted in the change of the learning environment. By using both the pseudo label information from target domain samples and the actual label information from source domain samples in the label iterative refinement process, not only the between-class scatter is maximized while that within-class scatter is minimized, but also the original space structure is maintained via Laplacian matrix, and the distribution difference is reduced by using maximum mean discrepancy as well. Moreover, semi-supervised transfer discriminant analysis based on cross-domain mean constraint (STDA-CMC) is proposed. In this algorithm, the cross-domain mean constraint term is incorporated into STDA, such that knowledge transfer between domains is facilitated by making source and target samples after being projected are located more closely in the low-dimensional feature subspace. The proposed algorithm is proved efficient and feasible from experiments on several datasets.