Guorong Li

Object tracking using incremental 2D-LDA learning and Bayes inference

The appearances of the tracked object and its surrounding background usually change during tracking. As for tracking methods using subspace analysis, fixed subspace basis tends to cause tracking failure. In this paper, a novel tracking method is proposed by using incremental 2D-LDA learning and Bayes inference. Incremental 2D-LDA formulates object tracking as online classification between foreground and background. It updates the row- or/and column- projected matrix efficiently. Based on the current object location and the prior knowledge, the possible locations of the object (candidates) in the next frame are predicted using simple sampling method. Applying 2D-LDA projection matrix and Bayes inference, candidate that maximizes the posterior probability is selected as the target object. Moreover, informative background samples are selected to update the subspace basis. Experiments are performed on image sequences with the objects appearance variations due to pose, lighting, etc. We also make comparison to incremental 2D-PCA and incremental FDA. The experimental results demonstrate that the proposed method is efficient and outperforms both the compared methods.

Cross-media topic detection: A multi-modality fusion framework

Detecting topics from Web data attracts increasing attention in recent years. Most previous works on topic detection mainly focus on the data from single medium, however, the rich and complementary information carried by multiple media can be used to effectively enhance the topic detection performance. In this paper, we propose a flexible data fusion framework to detect topics that simultaneously exist in different mediums. The framework is based on a multi-modality graph (MMG), which is obtained by fusing two single-modality graphs together: a text graph and a visual graph. Each node of MMGrepresents a multi-modal data and the edge weight between two nodes jointly measures their content and upload-time similarities. Since the data about the same topic often have similar content and are usually uploaded in a similar period of time, they would naturally form a dense (namely, strongly connected) subgraph in MMG. Such dense subgraph is robust to noise and can be efficiently detected by pair-wise clustering methods. The experimental results on single-medium and cross-media datasets demonstrate the flexibility and effectiveness of our method.

Treat samples differently: Object tracking with semi-supervised online CovBoost

Most feature selection methods for object tracking assume that the labeled samples obtained in the next frames follow the similar distribution with the samples in the previous frame. However, this assumption is not true in some scenarios. As a result, the selected features are not suitable for tracking and the “drift” problem happens. In this paper, we consider datas distribution in tracking from a new perspective. We classify the samples into three categories: auxiliary samples (samples in the previous frames), target samples (collected in the current frame) and unlabeled samples (obtained in the next frame). To make the best use of them for tracking, we propose a novel semi-supervised transfer learning approach. Specifically, we assume only target samples follow the same distribution as the unlabeled samples and develop a novel semi-supervised CovBoost method. It could utilize auxiliary samples and unlabeled samples effectively when training the best strong classifier for tracking. Furthermore, we develop a new online updating algorithm for semi-supervised CovBoost, making our tracker handle with significant variations of the tracked target and background successfully. We demonstrate the excellent performance of the proposed tracker on several challenging test videos.

Effective Multimodality Fusion Framework for Cross-Media Topic Detection

Due to the prevalence of We-Media, information is quickly published and received in various forms anywhere and anytime through the Internet. The rich cross-media information carried by the multimodal data in multiple media has a wide audience, deeply reflects the social realities, and brings about much greater social impact than any single media information. Therefore, automatically detecting topics from cross media is of great benefit for the organizations (i.e., advertising agencies and governments) that care about the social opinions. However, cross-media topic detection is challenging from the following aspects: 1) the multimodal data from different media often involve distinct characteristics and 2) topics are presented in an arbitrary manner among the noisy web data. In this paper, we propose a multimodality fusion framework and a topic recovery (TR) approach to effectively detect topics from cross-media data. The multimodality fusion framework flexibly incorporates the heterogeneous multimodal data into a multimodality graph, which takes full advantage from the rich cross-media information to effectively detect topic candidates (T.C.). The TR approach solidly improves the entirety and purity of detected topics by: 1) merging the T.C. that are highly relevant themes of the same real topic and 2) filtering out the less-relevant noise data in the merged T.C. Extensive experiments on both single-media and cross-media data sets demonstrate the promising flexibility and effectiveness of our method in detecting topics from cross media.

Learning Label Specific Features for Multi-label Classification

Binary relevance (BR) is a well-known framework for multi-label classification. It decomposes multi-label classification into binary (one-vs-rest) classification subproblems, one for each label. The BR approach is a simple and straightforward way for multi-label classification, but it still has several drawbacks. First, it does not consider label correlations. Second, each binary classifier may suffer from the issue of class-imbalance. Third, it can become computationally unaffordable for data sets with many labels. Several remedies have been proposed to solve these problems by exploiting label correlations between labels and performing label space dimension reduction. Meanwhile, inconsistency, another potential drawback of BR, is often ignored by researchers when they construct multi-label classification models. Inconsistency refers to the phenomenon that if an example belongs to more than one class label, then during the binary training stage, it can be considered as both positive and negative example simultaneously. This will mislead binary classifiers to learn suboptimal decision boundaries. In this paper, we seek to solve this problem by learning label specific features for each label. We assume that each label is only associated with a subset of features from the original feature set, and any two strongly correlated class labels can share more features with each other than two uncorrelated or weakly correlated ones. The proposed method can be applied as a feature selection method for multi-label learning and a general strategy to improve multi-label classification algorithms comprising a number of binary classifiers. Comparison with the state-of-the-art approaches manifests competitive performance of our proposed method.

A Multiple Targets Appearance Tracker Based on Object Interaction Models

Kernel-based method has been proved to be effective in solving single-target tracking problem. However, facing more complicated multitarget tracking task, most classic kernel-based multitarget trackers do not model the interaction among targets successfully and simply track each target independently. Thus, they usually could not deal with “singularity” problem and fail in tracking the target when occlusions occur or distracters appear. Although multikernel methods may improve the performance by introducing more constraints, how to simulate the relationship and interaction among the tracked targets is still not fully investigated. In this paper, we discuss a very common scenario of multitarget tracking, in which a moving objects motion is not only determined by its virtual destination but also impacted by other neighboring objects. This phenomenon exists in many usual tracking applications such as human tracking, traffic monitoring, video surveillance, and so on, where an object usually moves toward a particular direction but meanwhile detours when close to others to avoid collision. Specifically, we propose a novel interaction model to explain the above phenomenon. Then by defining a new cost function, we embed this interaction model into a kernel-based tracker and further derive our interactive kernel-based multitarget tracker. Experimental results on various datasets demonstrate that our interaction model can alleviate “singularity” problem and, thus, the proposed tracking method could achieve superior performance in multitarget tracking.

Cross-Modal Retrieval Using Multiordered Discriminative Structured Subspace Learning

This paper proposes a novel method for cross-modal retrieval. In addition to the traditional vector (text)-to-vector (image) framework, we adopt a matrix (text)-to-matrix (image) framework to faithfully characterize the structures of different feature spaces. Moreover, we propose a novel metric learning framework to learn a discriminative structured subspace, in which the underlying data distribution is preserved for ensuring a desirable metric. Concretely, there are three steps for the proposed method. First, the multiorder statistics are used to represent images and texts for enriching the feature information. We jointly use the covariance (second-order), mean (first-order), and bags of visual (textual) features (zeroth-order) to characterize each image and text. Second, considering that the heterogeneous covariance matrices lie on the different Riemannian manifolds and the other features on the different Euclidean spaces, respectively, we propose a unified metric learning framework integrating multiple distance metrics, one for each order statistical feature. This framework preserves the underlying data distribution and exploits complementary information for better matching heterogeneous data. Finally, the similarity between the different modalities can be measured by transforming the multiorder statistical features to the common subspace. The performance of the proposed method over the previous methods has been demonstrated through the experiments on two public datasets.

PL-ranking: A Novel Ranking Method for Cross-Modal Retrieval

This paper proposes a novel method for cross-modal retrieval named Pairwise-Listwise \textbf{ranking} (PL-ranking) based on the low-rank optimization framework. Motivated by the fact that optimizing the top of ranking is more applicable in practice, we focus on improving the precision at the top of ranked list for a given sample and learning a low-dimensional common subspace for multi-modal data. Concretely, there are three constraints in PL-ranking. First, we use a pairwise ranking loss constraint to optimize the top of ranking. Then, considering that the pairwise ranking loss constraint ignores class information, we further adopt a listwise constraint to minimize the intra-neighbors variance and maximize the inter-neighbors separability. By this way, class information is preserved while the number of iterations is reduced. Finally, low-rank based regularization is applied to exploit the correlations between features and labels so that the relevance between the different modalities can be enhanced after mapping them into the common subspace. We design an efficient low-rank stochastic subgradient descent method to solve the proposed optimization problem. The experimental results show that the average MAP scores of PL-ranking are improved 5.1%, 9.2%, 4.7% and 4.8% than those of the state-of-the-art methods on the Wiki, Flickr, Pascal and NUS-WIDE datasets, respectively.

Learning Label-Specific Features and Class-Dependent Labels for Multi-Label Classification

Binary Relevance is a well-known framework for multi-label classification, which considers each class label as a binary classification problem. Many existing multi-label algorithms are constructed within this framework, and utilize identical data representation in the discrimination of all the class labels. In multi-label classification, however, each class label might be determined by some specific characteristics of its own. In this paper, we seek to learn label-specific data representation for each class label, which is composed of label-specific features. Our proposed method LLSF can not only be utilized for multi-label classification directly, but also be applied as a feature selection method for multi-label learning and a general strategy to improve multi-label classification algorithms comprising a number of binary classifiers. Inspired by the research works on modeling high-order label correlations, we further extend LLSF to learn class-Dependent Labels in a sparse stackingway, denoted as LLSF-DL. It incorporates both second-order- and high-order label correlations. A comparative study with the state-of-the-art approaches manifests the effectiveness and efficiency of our proposed methods.

Online selection of the best k-feature subset for object tracking

In this paper, we propose a new feature subset evaluation method for feature selection in object tracking. According to the fact that a feature which is useless by itself could become a good one when it is used together with some other features, we propose to evaluate feature subsets as a whole for object tracking instead of scoring each feature individually and find out the most distinguishable subset for tracking. In the paper, we use a special tree to formalize the feature subset space. Then conditional entropy is used to evaluating feature subset and a simple but efficient greedy search algorithm is developed to search this tree to obtain the optimal k-feature subset quickly. Furthermore, our online k-feature subset selection method is integrated into particle filter for robust tracking. Extensive experiments demonstrate that k-feature subset selected by our method is more discriminative and thus can improve tracking performance considerably.