Yuting Su

Multipe/Single-View Human Action Recognition via Part-Induced Multitask Structural Learning

This paper proposes a unified framework for multiple/single-view human action recognition. First, we propose the hierarchical partwise bag-of-words representation which encodes both local and global visual saliency based on the body structure cue. Then, we formulate the multiple/single-view human action recognition as a part-regularized multitask structural learning (MTSL) problem which has two advantages on both model learning and feature selection: 1) preserving the consistence between the body-based action classification and the part-based action classification with the complementary information among different action categories and multiple views and 2) discovering both action-specific and action-shared feature subspaces to strengthen the generalization ability of model learning. Moreover, we contribute two novel human action recognition datasets, TJU (a single-view multimodal dataset) and MV-TJU (a multiview multimodal dataset). The proposed method is validated on three kinds of challenging datasets, including two single-view RGB datasets (KTH and TJU), two well-known depth dataset (MSR action 3-D and MSR daily activity 3-D), and one novel multiview multimodal dataset (MV-TJU). The extensive experimental results show that this method can outperform the popular 2-D/3-D part model-based methods and several other competing methods for multiple/single-view human action recognition in both RGB and depth modalities. To our knowledge, this paper is the first to demonstrate the applicability of MTSL with part-based regularization on multiple/single-view human action recognition in both RGB and depth modalities.

Multi-Modal Clique-Graph Matching for View-Based 3D Model Retrieval

Multi-view matching is an important but a challenging task in view-based 3D model retrieval. To address this challenge, we propose an original multi-modal clique graph (MCG) matching method in this paper. We systematically present a method for MCG generation that is composed of cliques, which consist of neighbor nodes in multi-modal feature space and hyper-edges that link pairwise cliques. Moreover, we propose an image set-based clique/edgewise similarity measure to address the issue of the set-to-set distance measure, which is the core problem in MCG matching. The proposed MCG provides the following benefits: 1) preserves the local and global attributes of a graph with the designed structure; 2) eliminates redundant and noisy information by strengthening inliers while suppressing outliers; and 3) avoids the difficulty of defining high-order attributes and solving hyper-graph matching. We validate the MCG-based 3D model retrieval using three popular single-modal data sets and one novel multi-modal data set. Extensive experiments show the superiority of the proposed method through comparisons. Moreover, we contribute a novel real-world 3D object data set, the multi-view RGB-D object data set. To the best of our knowledge, it is the largest real-world 3D object data set containing multi-modal and multi-view information.

Clique-graph matching by preserving global & local structure

This paper originally proposes the clique-graph and further presents a clique-graph matching method by preserving global and local structures. Especially, we formulate the objective function of clique-graph matching with respective to two latent variables, the clique information in the original graph and the pairwise clique correspondence constrained by the one-to-one matching. Since the objective function is not jointly convex to both latent variables, we decompose it into two consecutive steps for optimization: 1) clique-to-clique similarity measure by preserving local unary and pairwise correspondences; 2) graph-to-graph similarity measure by preserving global clique-to-clique correspondence. Extensive experiments on the synthetic data and real images show that the proposed method can outperform representative methods especially when both noise and outliers exist.

Single/multi-view human action recognition via regularized multi-task learning

Abstract This paper proposes a unified single/multi-view human action recognition method via regularized multi-task learning. First, we propose the pyramid partwise bag of words (PPBoW) representation which implicitly encodes both local visual characteristics and human body structure. Furthermore, we formulate the task of single/multi-view human action recognition into a part-induced multi-task learning problem penalized by graph structure and sparsity to discover the latent correlation among multiple views and body parts and consequently boost the performances. The experiment shows that this method can significantly improve performance over the standard BoW+SVM method. Moreover, the proposed method can achieve competing performance simply with low dimensional PPBoW representation against the state-of-the-art methods for human action recognition on KTH and MV-TJU, a new multi-view action dataset with RGB, depth and skeleton data prepared by our group.

Coupled hidden conditional random fields for RGB-D human action recognition

This paper proposes a human action recognition method via coupled hidden conditional random fields model by fusing both RGB and depth sequential information. The coupled hidden conditional random fields model extends the standard hidden-state conditional random fields model only with one chain-structure sequential observation to multiple chain-structure sequential observations, which are synchronized sequence data captured in multiple modalities. For model formulation, we propose the specific graph structure for the interaction among multiple modalities and design the corresponding potential functions. Then we propose the model learning and inference methods to discover the latent correlation between RGB and depth data as well as model temporal context within individual modality. The extensive experiments show that the proposed model can boost the performance of human action recognition by taking advance of complementary characteristics from both RGB and depth modalities. HighlightsWe propose cHCRF to learn sequence-specific and sequence-shared temporal structure.We contribute a novel RGB-D human action dataset containing 1200 samples.Experiments on 3 popular datasets show the superiority of the proposed method.

A video steganalytic algorithm against motion-vector-based steganography

Steganalysis is the art of detecting the information hidden in the media using steganography. In this paper, a steganalytic method is proposed to detect information hidden in the motion vectors of video bit-streams. Based on the statistical analysis of relative properties, the feature classification technique is adopted to determine the existence of hidden messages. The Support Vector Machine (SVM) is used as the discriminator. The experimental results show that the proposed method can detect the hidden data effectively in motion-vector-based steganography algorithms.

Graph-based characteristic view set extraction and matching for 3D model retrieval

In recent times, multi-view representation of the 3D model has led to extensive research in view-based methods for 3D model retrieval. However, most approaches focus on feature extraction from 2D images while ignoring the spatial information of the 3D model. In order to improve the effectiveness of view-based methods on 3D model retrieval, this paper proposes a novel method for characteristic view extraction and similarity measurement. First, the graph clustering method is used for view grouping and the random-walk algorithm is applied to adaptively update the weight of each view. The spatial information of the 3D object is utilized to construct a view-graph model, thus enabling each characteristic view to represent the discriminative visual feature in terms of specific spatial context. Next, by considering the view set as a graph model, the similarity measurement of two models can be converted into a graph matching problem. This problem is solved by mathematically formulating it as a Rayleigh quotient maximization with affinity constraints for similarity measurement. Extensive comparison experiments were conducted on the popular ETH, NTU, PSB, and MV-RED 3D model datasets. The results demonstrate the superiority of the proposed method.

Single/cross-camera multiple-person tracking by graph matching

Single and cross-camera multiple person tracking in unconstrained condition is an extremely challenging task in computer vision. Facing the main difficulties caused by the existence of occlusion in single-camera scenario and the occurrence of transition in cross-camera scenario, we propose a unified framework formulated in graph matching with affinity constraints for both single and cross-camera tracking tasks. To our knowledge, our work is the first to unify two kinds of tracking problems with the same framework by graph matching. The proposed method consists of two steps, tracklet generation and tracklet association. First, we implement the modified part-based human detector and the Tracking-Modeling-Detection (TMD) method for tracklet generation. Then we propose to associate tracklets by graph matching which is mathematically formulated into the Rayleigh Quotients Maximization. The comparison experiments show that the proposed method can produce the competing results with the state-of-the-art methods.

3D object retrieval based on sparse coding in weak supervision

The proposed method does not require the explicit virtual model information.We utilized FDDL to learn dictionary to reconstruct query sample for retrieval.Our approach has high generalization ability and can be used on other applications. With the rapid development of computer vision and digital capture equipment, we can easily record the 3D information of objects. In the recent years, more and more 3D data are generated, which makes it desirable to develop effective 3D retrieval algorithms. In this paper, we apply the sparse coding method in a weakly supervision manner to address 3D model retrieval. First, each 3D object, which is represented by a set of 2D images, is used to learn dictionary. Then, sparse coding is used to compute the reconstruction residual for each query object. Finally, the residual between the query model and the candidate model is used for 3D model retrieval. In the experiment, ETH, NTU and ALOL dataset are used to evaluate the performance of the proposed method. The results demonstrate the superiority of the proposed method.

Rank canonical correlation analysis and its application in visual search reranking

Ranking relevance degree information is widely utilized in the ranking models of information retrieval applications, such as text and multimedia retrieval, question answering, and visual search reranking. However, existing feature dimensionality reduction methods neglect this kind of valuable potential supervised information. In this paper, we extend the pairwise constraints from the traditional class labels to ranking relevance degrees, and propose a novel dimensionality reduction method called Rank-CCA. Rank-CCA effectively incorporates ranking relevance constraints into standard canonical correlation analysis (CCA) algorithm, and is able to employ the knowledge of both unlabeled and labeled data. In the application of visual search reranking, our proposed method is verified through extensive experimental studies. Experimental results show that Rank-CCA is superior to standard CCA and semi-supervised CCA (Semi-CCA) algorithm, and achieves comparable performance with several state-of-the-art reranking methods while preserving the superiority of low dimensional features.