Yunqi Lei

Supervised sparse manifold regression for head pose estimation in 3D space

In estimating the head pose angles in 3D space by manifold learning, the results currently are not very satisfactory. We need to preserve the local geometry structure effectively and need a learned projective function that can reveal the dominant features better. To address these problems, we propose a Supervised Sparse Manifold Regression (SSMR) method that incorporates both the supervised graph Laplacian regularization and the sparse regression into manifold learning. In SSMR, on the one hand, a low-dimensional projection is embedded to represent intrinsic features by using supervised information while the local structure can be preserved more effectively by using the Laplacian regularization term in the objective function. On the other hand, by casting the problem of learning projective function into a regression with L1 norm regularizer, a projection is mapped to carry out the sparse representation of high dimension features, rather than a compact linear combination, so as to describe the dominant features better. Experiments show that our proposed method SSMR is beneficial for head pose angle estimation in 3D space. HighlightsA supervised sparse manifold regression method is proposed for manifold learning.A low-dimensional projection is embedded to represent intrinsic features.A projection is used to fulfill the sparse representation of high dimension features.The local geometry is preserved and the dominant features are revealed better.The proposed method is beneficial for head pose angle estimation.

Supervised locality discriminant manifold learning for head pose estimation

In this paper, we propose a novel supervised manifold learning approach, supervised locality discriminant manifold learning (SLDML), for head pose estimation. Traditional manifold learning methods focus on preserving only the intra-class geometric properties of the manifold embedded in the high-dimensional ambient space, so they cannot fully utilize the underlying discriminative knowledge of the data. The proposed SLDML aims to explore both geometric structure and discriminant information of the data, and yields a smooth and discriminative low-dimensional embedding by adding the local discriminant terms in the optimization objectives of manifold learning. Moreover, for efficiently handling out-of-sample extension and learning with the local consistency, we decompose the manifold learning as a two-step approach. We incorporate the manifold learning and the regression with a learned discriminant manifold-based projection function obtained by discriminatively Laplacian regularized least squares. The SLDML provides both the low-dimensional embedding and projection function with better intra-class compactness and inter-class separability, therefore preserves the local geometric structures more effectively. Meanwhile, the SLDML is supervised by both biased distance and continuous head pose angle information when constructing the graph, embedding the graph and learning the projection function. Our experiments demonstrate the superiority of the proposed SLDML over several current state-of-art approaches for head pose estimation on the publicly available FacePix dataset.

Parallelizing Convolutional Neural Networks for Action Event Recognition in Surveillance Videos

In order to deal with action recognition for large scale video data, this paper presents a MapReduce based parallel algorithm for SASTCNN, a sparse auto-combination spatio-temporal convolutional neural network. We design and implement a parallel matrix multiplication algorithm based on MapReduce. We use the MapReduce programming model to parallelize SASTCNN on a Hadoop platform. In order to take advantage of the computing power of multi-core CPU, the Map and Reduce processes of MapReduce are implemented using a multi-thread technique. A series of experiments on both WEIZMAN and KTH data sets are carried out. Compared with traditional serial algorithms, the feasibility, stability and correctness of the parallel SASTCNN are validated and a speedup in computation is obtained. Experimental results also show that the proposed method could provide more competitive results on the two data sets than other benchmark methods.

Temporal sparse feature auto-combination deep network for video action recognition: Temporal sparse feature auto-combination deep network for video action recognition

In order to deal with action recognition for large‐scale video data, we present a spatio‐temporal auto‐combination deep network, which is able to extract deep features from short video segments by making full use of temporal contextual correlation of corresponding pixels among successive video frames. Based on conventional sparse encoding, we further consider the representative features in adjacent nodes of the hidden layers according to activation states similarities. A sparse auto‐combination strategy is applied to multiple input maps in each convolution stage. An information constraint of the representative features of hidden layer nodes is imposed to handle the adaptive sparse encoding of the topology. As a result, the learned features can represent the spatio‐temporal transition relationships better and the number of hidden nodes can be restricted to a certain range. We conduct a series of experiments on two public data sets. The experimental results show that our approach is more effective and robust in video action recognition compared with traditional methods.

Preserving discriminant manifold subspace learning for plant leaf recognition

To achieve the effective plant leaf classification using manifold learning, the local geometry structure of plant leaves is able to be preserved effectively and a discriminant manifold-based projection should be learned to capture the dominant structure features better. We firstly use Gabor filter to model the texture of plant leaf images as the samples. Then for the high-dimensional features, we construct the adjacency information graph based on two constraints, i.e., low rank and sparsity. Thereby, we propose a novel Preserving Discriminant Manifold Subspace Learning (PDMSL) to embed the information graph and learn a common subspace by introducing both graph Lapla-cian and sparse regularizers. The low-dimensional embedding and projection corresponding to the learned manifold subspace have better intra-class similarity and inter-class discriminant ability of Gabor features of the leaf, and can also deal with out-of-sample extension efficiently. Our experiments on Swedish leaf datasets demonstrate that the proposed method is much more effective than other baseline methods.

Face recognition by decision fusion of two-dimensional linear discriminant analysis and local binary pattern

To investigate the robustness of face recognition algorithms under the complicated variations of illumination, facial expression and posture, the advantages and disadvantages of seven typical algorithms on extracting global and local features are studied through the experiments respectively on the Olivetti Research Laboratory database and the other three databases (the three subsets of illumination, expression and posture that are constructed by selecting images from several existing face databases). By taking the above experimental results into consideration, two schemes of face recognition which are based on the decision fusion of the two-dimensional linear discriminant analysis (2DLDA) and local binary pattern (LBP) are proposed in this paper to heighten the recognition rates. In addition, partitioning a face non-uniformly for its LBP histograms is conducted to improve the performance. Our experimental results have shown the complementarities of the two kinds of features, the 2DLDA and LBP, and have verified the effectiveness of the proposed fusion algorithms.

Range clusters based time-of-flight 3D imaging obstacle detection in manifold space.

A new obstacle detection method using time-of-flight 3D imaging sensor based on range clusters is proposed. To effectively reduce the influence of outlier and noise in range images, we utilize intensity images to estimate noise deviation of the range images and a weighted local linear smoothing is used to project the data into a new manifold surface. The proposed method divides the 3D imaging data into range clusters with different shapes and sizes according to the distance ambient relation between the pixels, and some regulation criterions are set to adjust the range clusters into optimal shape and size. Experiments on the SwissRanger sensor data show that, compared to the traditional obstacle detection methods based on regular data patches, the proposed method can give more precious detection results.

Supervised preserving projection for learning scene information based on time-of-flight imaging sensor

In this paper, we propose a new supervised manifold learning approach, supervised preserving projection (SPP), for the depth images of a 3D imaging sensor based on the time-of-flight (TOF) principle. We present a novel manifold sense to learn scene information produced by the TOF camera along with depth images. First, we use a local surface patch to approximate the underlying manifold structures represented by the scene information. The fundamental idea is that, because TOF data have nonstatic noise and distance ambiguity problems, the surface patches can more efficiently approximate the local neighborhood structures of the underlying manifold than TOF data points, and they are robust to the nonstatic noise of TOF data. Second, we propose SPP to preserve the pairwise similarity between the local neighboring patches in TOF depth images. Moreover, SPP accomplishes the low-dimensional embedding by adding the scene region class label information accompanying the training samples and obtains the predictive mapping by incorporating the local geometrical properties of the dataset. The proposed approach has advantages of both classical linear and nonlinear manifold learning, and real-time estimation results of the test samples are obtained by the low-dimensional embedding and the predictive mapping. Experiments show that our approach obtains information effectively from three scenes and is robust to the nonstatic noise of 3D imaging sensor data.

Visual attributes based sparse multitask action recognition

For action recognition, traditional multitask learning can share low-level features among actions effectively, but it neglects high-level semantic relationships between latent visual attributes and actions. Some action classes might be related, where latent visual attributes across categories are shared among them. In this paper, we improve multitask learning model using attribute-actions relationship for action datasets with sparse and incomplete labels. Moreover, the amount of semantic information of visual attributes and action class labels are different, so we carry out attribute task learning and action task learning separately for improving generalization performance. Specifically, for two latent variables, i.e. visual attributes and model parameters, we formulate the joint optimization objective function regularized by low rank and sparsity. To deal with this non-convex optimization, we transform this non-convex objective function into the convex formulation by an auxiliary variable. Experimental results on two datasets show that the proposed approach can learn latent knowledge effectively to enhance discrimination power and is competitive to other baseline methods.

Image compressed sensing based on multi-level adaptive learning dictionaries

Sparse representation matrix is of great significance for Compressed Sensing (CS) reconstruction accuracy. Contourlet Transform (CT) offer a much richer set of directions and shapes, and it is more effective in capturing smooth contours and geometric structures in images. While dictionaries learned by machine learning methods can represent images more effectively. In this paper, we propose a multi-level adaptive dictionary learning (DL) strategy which combines both of the above advantages. We learn sub-dictionaries of high frequency of CT by an improved K-SVD algorithm, and moreover, the stopping criteria of sparse representation stage is associated with the iteratively updated dictionaries to get an adaptive sparse constraint, which gets more effective sparse representation coefficients and then improves the dictionary updating. This approach achieves a good reconstruction accuracy of the high frequency with less CS measurement. Experiment results demonstrate that the reconstructed images using dictionaries learned by the proposed algorithm in CS have better effect.