Zhaoxuan Yang

Multipe/Single-View Human Action Recognition via Part-Induced Multitask Structural Learning

This paper proposes a unified framework for multiple/single-view human action recognition. First, we propose the hierarchical partwise bag-of-words representation which encodes both local and global visual saliency based on the body structure cue. Then, we formulate the multiple/single-view human action recognition as a part-regularized multitask structural learning (MTSL) problem which has two advantages on both model learning and feature selection: 1) preserving the consistence between the body-based action classification and the part-based action classification with the complementary information among different action categories and multiple views and 2) discovering both action-specific and action-shared feature subspaces to strengthen the generalization ability of model learning. Moreover, we contribute two novel human action recognition datasets, TJU (a single-view multimodal dataset) and MV-TJU (a multiview multimodal dataset). The proposed method is validated on three kinds of challenging datasets, including two single-view RGB datasets (KTH and TJU), two well-known depth dataset (MSR action 3-D and MSR daily activity 3-D), and one novel multiview multimodal dataset (MV-TJU). The extensive experimental results show that this method can outperform the popular 2-D/3-D part model-based methods and several other competing methods for multiple/single-view human action recognition in both RGB and depth modalities. To our knowledge, this paper is the first to demonstrate the applicability of MTSL with part-based regularization on multiple/single-view human action recognition in both RGB and depth modalities.

Single/multi-view human action recognition via regularized multi-task learning

Abstract This paper proposes a unified single/multi-view human action recognition method via regularized multi-task learning. First, we propose the pyramid partwise bag of words (PPBoW) representation which implicitly encodes both local visual characteristics and human body structure. Furthermore, we formulate the task of single/multi-view human action recognition into a part-induced multi-task learning problem penalized by graph structure and sparsity to discover the latent correlation among multiple views and body parts and consequently boost the performances. The experiment shows that this method can significantly improve performance over the standard BoW+SVM method. Moreover, the proposed method can achieve competing performance simply with low dimensional PPBoW representation against the state-of-the-art methods for human action recognition on KTH and MV-TJU, a new multi-view action dataset with RGB, depth and skeleton data prepared by our group.

Coupled hidden conditional random fields for RGB-D human action recognition

This paper proposes a human action recognition method via coupled hidden conditional random fields model by fusing both RGB and depth sequential information. The coupled hidden conditional random fields model extends the standard hidden-state conditional random fields model only with one chain-structure sequential observation to multiple chain-structure sequential observations, which are synchronized sequence data captured in multiple modalities. For model formulation, we propose the specific graph structure for the interaction among multiple modalities and design the corresponding potential functions. Then we propose the model learning and inference methods to discover the latent correlation between RGB and depth data as well as model temporal context within individual modality. The extensive experiments show that the proposed model can boost the performance of human action recognition by taking advance of complementary characteristics from both RGB and depth modalities. HighlightsWe propose cHCRF to learn sequence-specific and sequence-shared temporal structure.We contribute a novel RGB-D human action dataset containing 1200 samples.Experiments on 3 popular datasets show the superiority of the proposed method.

Single/cross-camera multiple-person tracking by graph matching

Single and cross-camera multiple person tracking in unconstrained condition is an extremely challenging task in computer vision. Facing the main difficulties caused by the existence of occlusion in single-camera scenario and the occurrence of transition in cross-camera scenario, we propose a unified framework formulated in graph matching with affinity constraints for both single and cross-camera tracking tasks. To our knowledge, our work is the first to unify two kinds of tracking problems with the same framework by graph matching. The proposed method consists of two steps, tracklet generation and tracklet association. First, we implement the modified part-based human detector and the Tracking-Modeling-Detection (TMD) method for tracklet generation. Then we propose to associate tracklets by graph matching which is mathematically formulated into the Rayleigh Quotients Maximization. The comparison experiments show that the proposed method can produce the competing results with the state-of-the-art methods.

Nonnegative Mixed-Norm Convex Optimization for Mitotic Cell Detection in Phase Contrast Microscopy

This paper proposes a nonnegative mix-norm convex optimization method for mitotic cell detection. First, we apply an imaging model-based microscopy image segmentation method that exploits phase contrast optics to extract mitotic candidates in the input images. Then, a convex objective function regularized by mix-norm with nonnegative constraint is proposed to induce sparsity and consistence for discriminative representation of deformable objects in a sparse representation scheme. At last, a Support Vector Machine classifier is utilized for mitotic cell modeling and detection. This method can overcome the difficulty in feature formulation for deformable objects and is independent of tracking or temporal inference model. The comparison experiments demonstrate that the proposed method can produce competing results with the state-of-the-art methods.

HEp-2 cells Classification via clustered multi-task learning

This paper proposes a clustered multi-task learning-based method for automated HEp-2 cells Classification. First, the visual feature is extracted for individual sample to represent its appearance characteristics. Then, the models of multiple HEp-2 cell category are jointly trained in the framework of clustered multi-task learning. The extensive experiments on the HEp 2, cell dataset released by the HEp-2 Cells Classification contest, held at the 2012 International Conference on Patter Recognition, show that the proposed method can discover and share the latent relatedness among multiple tasks and consequently augment the performance. The quantitative comparison against the state-of-the-art methods demonstrates the superiority of the proposed method.

Jointly Learning Multiple Sequential Dynamics for Human Action Recognition

Discovering visual dynamics during human actions is a challenging task for human action recognition. To deal with this problem, we theoretically propose the multi-task conditional random fields model and explore its application on human action recognition. For visual representation, we propose the part-induced spatiotemporal action unit sequence to represent each action sample with multiple partwise sequential feature subspaces. For model learning, we propose the multi-task conditional random fields (MTCRFs) model to discover the sequence-specific structure and the sequence-shared relationship. Specifically, the multi-chain graph structure and the corresponding probabilistic model are designed to represent the interaction among multiple part-induced action unit sequences. Moreover we propose the model learning and inference methods to discover temporal context within individual action unit sequence and the latent correlation among different body parts. Extensive experiments are implemented to demonstrate the superiority of the proposed method on two popular RGB human action datasets, KTH & TJU, and the depth dataset in MSR Daily Activity 3D.

Cell type-independent mitosis event detection via hidden-state conditional neural fields

This paper proposes a cell type-independent mitosis event detection method based on hidden-state conditional neural fields in time-lapse phase contrast microscopy sequences of stem cell populations. This method proceeds through three steps. First, we apply the imaging model-based microscopy image segmentation method and volumetric region growing to extract candidate sequences. Then, we extract the GIST feature of each frame within a candidate sequence for visual representation. Finally, a hidden-state conditional neural field classifier is trained to classify each candidate as mitosis or not. The main contribution is that the proposed method can jointly realize non-linear feature learning for different types of cells and temporal dynamic modeling of mitotic progression. The comparison experiments demonstrated the proposed method can benefit the detection of cell type-independent mitosis.

Sparse Coding Induced Transfer learning for HEp-2 Cell Classification

Automated human larynx carcinoma (HEp-2) cell classification is critical for medical diagnosis. In this paper, we propose a sparse coding-based unsupervised transfer learning method for HEp-2 cell classification. First, the low level image feature is extracted for visual representation. Second, a sparse coding scheme with the Elastic Net penalized convex objective function is proposed for unsupervised feature learning. At last, a Support Vector Machine classifier is utilized for model learning and predicting. To our knowledge, this work is the first to transfer the human-crafted visual feature, sensitive to the variation of appearance and shape during cell movement, to the high level representation which directly denotes the correlation of one sample and the bases in the learnt dictionary. Therefore, the proposed method can overcome the difficulty in discriminative feature formulation for different kinds of cells with irregular and changing visual patterns. Large scale comparison experiments will be conducted to show the superiority of this method.

Spatial Context Constrained Characteristic View Extraction for 3D Model Retrieval

With the development of 3D camera, efficient and effective 3D model retrieval algorithms are highly desired and attracted intensive research attentions. In this paper, we proposed a view-based 3D model retrieval method based on spatial context constrained characteristic view extraction method. First, according to the spatial constraints of views, we cluster all of 2D images, which capture 3D model from different angles. Second, the random-walk algorithm is utilized to update the weight of each view to help us to select the most representative view. Finally, we apply Bayesian model to compute the similarity between query model and candidate 3D model to find the best matching 3D model. Experimental comparisons have been conducted on the ETH and NTU 3D model datasets, and the results have demonstrated the superiority of the proposed method.