Hua Mao

Symmetric low-rank representation for subspace clustering

We propose a symmetric low-rank representation (SLRR) method for subspace clustering, which assumes that a data set is approximately drawn from the union of multiple subspaces. The proposed technique can reveal the membership of multiple subspaces through the self-expressiveness property of the data. In particular, the SLRR method considers a collaborative representation combined with low-rank matrix recovery techniques as a low-rank representation to learn a symmetric low-rank representation, which preserves the subspace structures of high-dimensional data. In contrast to performing iterative singular value decomposition in some existing low-rank representation based algorithms, the symmetric low-rank representation in the SLRR method can be calculated as a closed form solution by solving the symmetric low-rank optimization problem. By making use of the angular information of the principal directions of the symmetric low-rank representation, an affinity graph matrix is constructed for spectral clustering. Extensive experimental results show that it outperforms state-of-the-art subspace clustering algorithms. HighlightsA symmetric low-rank representation method for subspace clustering is proposed.A collaborative representation and dimension reduction are combined in learning.The symmetric low-rank representation can be calculated as a closed form solution.The symmetric low-rank representation maintains the subspace structures of data.The proposed method outperforms state-of-the-art subspace clustering algorithms.

Explicit guiding auto-encoders for learning meaningful representation

The auto-encoder model plays a crucial role in the success of deep learning. During the pre-training phase, auto-encoders learn a representation that helps improve the performance of the entire neural network during the fine-tuning phase of deep learning. However, the learned representation is not always meaningful and the network does not necessarily achieve higher performance with such representation because auto-encoders are trained in an unsupervised manner without knowing the specific task targeted in the fine-tuning phase. In this paper, we propose a novel approach to train auto-encoders by adding an explicit guiding term to the traditional reconstruction cost function that encourages the auto-encoder to learn meaningful features. Particularly, the guiding term is the classification error with respect to the representation learned by the auto-encoder, and a meaningful representation means that a network using the representation as input has a low classification error in a classification task. In our experiments, we show that the additional explicit guiding term helps the auto-encoder understand the prospective target in advance. During learning, it can drive the learning toward a minimum with better generalization with respect to the particular supervised task on the dataset. Over a range of image classification benchmarks, we achieve equal or superior results to baseline auto-encoders with the same configuration.

Moving object recognition using multi-view three-dimensional convolutional neural networks

Moving object recognition (MOR) is an important but challenging problem in the field of computer vision. The aim of MOR is to recognize moving objects in a given video dataset. Convolutional neural networks (CNNs) have been extensively used for image recognition and video analysis problems. Recently, a 3D-CNN, which contains 3D convolution layers, was proposed to address MOR problems by successfully extracting spatiotemporal features. In this paper, a multi-view (MV) 3D-CNN is proposed for MOR. This model combines 3D-CNNs with a well-known MV learning technique. Because multi-view learning techniques have the ability to obtain more view-related features from videos captured by different cameras, the proposed model can extract more representative features. Moreover, the model contains a special view-pooling layer that can fuse the feature information from previous layers. The proposed MV3D-CNN is applied to both real-world moving vehicle recognition and sign language recognition tasks. The experimental results show that the proposed model possesses good performance.

Protein secondary structure prediction by using deep learning method

The prediction of protein structures directly from amino acid sequences is one of the biggest challenges in computational biology. It can be divided into several independent sub-problems in which protein secondary structure (SS) prediction is fundamental. Many computational methods have been proposed for SS prediction problem. Few of them can model well both the sequence-structure mapping relationship between input protein features and SS, and the interaction relationship among residues which are both important for SS prediction. In this paper, we proposed a deep recurrent encoder–decoder networks called Secondary Structure Recurrent Encoder–Decoder Networks (SSREDNs) to solve this SS prediction problem. Deep architecture and recurrent structures are employed in the SSREDNs to model both the complex nonlinear mapping relationship between input protein features and SS, and the mutual interaction among continuous residues of the protein chain. A series of techniques are also used in this paper to refine the model’s performance. The proposed model is applied to the open dataset CullPDB and CB513. Experimental results demonstrate that our method can improve both Q3 and Q8 accuracy compared with some public available methods. For Q8 prediction problem, it achieves 68.20% and 73.1% accuracy on CB513 and CullPDB dataset in fewer epochs better than the previous state-of-art method.

Subspace clustering using a symmetric low-rank representation ☆

In this paper, we propose a low-rank representation with sym metric constraint (LRRSC) method for robust subspace clust ering. Given a collection of data points approximately drawn from m ultiple subspaces, the proposed technique can simultaneou sly recover the dimension and members of each subspace. LRRSC extends th original low-rank representation algorithm by integrati ng a symmetric constraint into the low-rankness property of high-d imensional data representation. The symmetric low-rank re presentation, which preserves the subspace structures of high-dimension al data, guarantees weight consistency for each pair of data points so that highly correlated data points of subspaces are represented tog ther. Moreover, it can be e fficiently calculated by solving a convex optimization problem. We provide a rigorous proof for minim izing the nuclear-norm regularized least square problem wi th a symmetric constraint. The a ffinity matrix for spectral clustering can be obtained by furth e exploiting the angular information of the principal directions of the symmetric low-rank representa tion. This is a critical step towards evaluating the members hips between data points. Experimental results on benchmark databases d emonstrate the e ffectiveness and robustness of LRRSC compared with several state-of-the-art subspace clustering algorithms .In this paper, we propose a low-rank representation with symmetric constraint (LRRSC) method for robust subspace clustering. Given a collection of data points approximately drawn from multiple subspaces, the proposed technique can simultaneously recover the dimension and members of each subspace. LRRSC extends the original low-rank representation algorithm by integrating a symmetric constraint into the low-rankness property of high-dimensional data representation. The symmetric low-rank representation, which preserves the subspace structures of high-dimensional data, guarantees weight consistency for each pair of data points so that highly correlated data points of subspaces are represented together. Moreover, it can be efficiently calculated by solving a convex optimization problem. We provide a proof for minimizing the nuclear-norm regularized least square problem with a symmetric constraint. The affinity matrix for spectral clustering can be obtained by further exploiting the angular information of the principal directions of the symmetric low-rank representation. This is a critical step towards evaluating the memberships between data points. Besides, we also develop eLRRSC algorithm to improve the scalability of the original LRRSC by considering its closed form solution. Experimental results on benchmark databases demonstrate the effectiveness and robustness of LRRSC and its variant compared with several state-of-the-art subspace clustering algorithms.

Manifold dimension reduction based clustering for multi-objective evolutionary algorithm

Real world optimization problems always possess multiple objectives which are conflict in nature. Multi-objective evolutionary algorithms (MOEAs), which provide a group of solutions in region of Pareto front, increasingly draw researchers attention for their excellent performance. In this regard, solutions with a wide diversity would be more favored as they give decision makers more choices to evaluate upon their problems. Based on the insight of investigating the evolution, the Pareto front often lies in a manifold space, not Euclidian space. However, most MOEAs utilize Euclidian distance as a sole mechanism to keep a wide range of diversity for solutions, which is not suitable somewhat from this aspect. To this end, manifold dimension reduction algorithm which has the ability to map solutions in the same front of objective space into Euclidian space is adapted in further. And then, general clustering algorithm are utilized. At the end, we use this technology to replace the crowding distance technology in NSGA-II to choose individuals when there is not enough slots in mating selection process. Based on a range of experiments over benchmark problems against state-of-the-art, it is fully expected benefit of performance improvement will be more significant when applied in many objectives optimization problems. This will be pursuit in our future study.

Learning a good representation with unsymmetrical auto-encoder

AbstractAuto-encoders play a fundamental role in unsupervised feature learning and learning initial parameters of deep architectures for supervised tasks. For given input samples, robust features are used to generate robust representations from two perspectives: (1) invariant to small variation of samples and (2) reconstruction by decoders with minimal error. Traditional auto-encoders with different regularization terms have symmetrical numbers of encoder and decoder layers, and sometimes parameters. We investigate the relation between the number of layers and propose an unsymmetrical structure, i.e., an unsymmetrical auto-encoder (UAE), to learn more effective features. We present empirical results of feature learning using the UAE and state-of-the-art auto-encoders for classification tasks with a range of datasets. We also analyze the gradient vanishing problem mathematically and provide suggestions for the appropriate number of layers to use in UAEs with a logistic activation function. In our experiments, UAEs demonstrated superior performance with the same configuration compared to other auto-encoders. n

Symmetric low-rank preserving projections for subspace learning

Abstract Graph construction plays an important role in graph-oriented subspace learning. However, most existing approaches cannot simultaneously consider the global and local structures of high-dimensional data. In order to solve this deficiency, we propose a symmetric low-rank preserving projection (SLPP) framework incorporating a symmetric constraint and a local regularization into low-rank representation learning for subspace learning. Under this framework, SLPP-M is incorporated with manifold regularization as its local regularization while SLPP-S uses sparsity regularization. Besides characterizing the global structure of high-dimensional data by a symmetric low-rank representation, both SLPP-M and SLPP-S effectively exploit the local manifold and geometric structure by incorporating manifold and sparsity regularization, respectively. The similarity matrix is successfully learned by solving the nuclear-norm minimization optimization problem. Combined with graph embedding techniques, a transformation matrix effectively preserves the low-dimensional structure features of high-dimensional data. In order to facilitate classification by exploiting available labels of training samples, we also develop a supervised version of SLPP-M and SLPP-S under the SLPP framework, named S-SLPP-M and S-SLPP-S, respectively. Experimental results in face, handwriting and object recognition applications demonstrate the efficiency of the proposed algorithm for subspace learning.

Audio classification using attention-augmented convolutional neural network

Abstract Audio classification, as a set of important and challenging tasks, groups speech signals according to speakers’ identities, accents, and emotional states. Due to the high dimensionality of the audio data, task-specific hand-crafted features extraction is always required and regarded cumbersome for various audio classification tasks. More importantly, the inherent relationship among features has not been fully exploited. In this paper, the original speech signal is first represented as spectrogram and later be split along the frequency domain to form frequency-distributed spectrogram. This paper proposes a task-independent model, called FreqCNN, to automaticly extract distinctive features from each frequency band by using convolutional kernels. Further more, an attention mechanism is introduced to systematically enhance the features from certain frequency bands. The proposed FreqCNN is evaluated on three publicly available speech databases thorough three independent classification tasks. The obtained results demonstrate superior performance over the state-of-the-art.

Stem cell motion-tracking by using deep neural networks with multi-output

The aim of automated stem cell motility analysis is reliable processing and evaluation of cell behaviors such as translocation, mitosis, death, and so on. Cell tracking plays an important role in this research. In practice, tracking stem cells is difficult because they have frequent motion, deformation activities, and small resolution sizes in microscopy images. Previous tracking approaches designed to address this problem have been unable to generalize the rapid morphological deformation of cells in a complex living environment, especially for real-time tracking tasks. Herein, a deep learning framework with convolutional structure and multi-output layers is proposed for overcoming stem cell tracking problems. A convolutional structure is used to learn robust cell features through deep features learned on massive visual data by a transfer learning strategy. With multi-output layers, this framework tracks the cell’s motion and simultaneously detects its mitosis as an assistant task. This improves the generalization ability of the model and facilitates practical applications for stem cell research. The proposed framework, tracking and detection neural networks, also contains a particle filter-based motion model, a specialized cell sampling strategy, and corresponding model update strategy. Its current application to a microscopy image dataset of human stem cells demonstrates increased tracking performance and robustness compared with other frequently used methods. Moreover, mitosis detection performance was verified against manually labeled mitotic events of the tracked cell. Experimental results demonstrate good performance of the proposed framework for addressing problems associated with stem cell tracking.