Wanqi Yang

Multi-instance dictionary learning for detecting abnormal events in surveillance videos.

In this paper, a novel method termed Multi-Instance Dictionary Learning (MIDL) is presented for detecting abnormal events in crowded video scenes. With respect to multi-instance learning, each event (video clip) in videos is modeled as a bag containing several sub-events (local observations); while each sub-event is regarded as an instance. The MIDL jointly learns a dictionary for sparse representations of sub-events (instances) and multi-instance classifiers for classifying events into normal or abnormal. We further adopt three different multi-instance models, yielding the Max-Pooling-based MIDL (MP-MIDL), Instance-based MIDL (Inst-MIDL) and Bag-based MIDL (Bag-MIDL), for detecting both global and local abnormalities. The MP-MIDL classifies observed events by using bag features extracted via max-pooling over sparse representations. The Inst-MIDL and Bag-MIDL classify observed events by the predicted values of corresponding instances. The proposed MIDL is evaluated and compared with the state-of-the-art methods for abnormal event detection on the UMN (for global abnormalities) and the UCSD (for local abnormalities) datasets and results show that the proposed MP-MIDL and Bag-MIDL achieve either comparable or improved detection performances. The proposed MIDL method is also compared with other multi-instance learning methods on the task and superior results are obtained by the MP-MIDL scheme.

MRM-Lasso: A Sparse Multiview Feature Selection Method via Low-Rank Analysis

Learning about multiview data involves many applications, such as video understanding, image classification, and social media. However, when the data dimension increases dramatically, it is important but very challenging to remove redundant features in multiview feature selection. In this paper, we propose a novel feature selection algorithm, multiview rank minimization-based Lasso (MRM-Lasso), which jointly utilizes Lasso for sparse feature selection and rank minimization for learning relevant patterns across views. Instead of simply integrating multiple Lasso from view level, we focus on the performance of sample-level (sample significance) and introduce pattern-specific weights into MRM-Lasso. The weights are utilized to measure the contribution of each sample to the labels in the current view. In addition, the latent correlation across different views is successfully captured by learning a low-rank matrix consisting of pattern-specific weights. The alternating direction method of multipliers is applied to optimize the proposed MRM-Lasso. Experiments on four real-life data sets show that features selected by MRM-Lasso have better multiview classification performance than the baselines. Moreover, pattern-specific weights are demonstrated to be significant for learning about multiview data, compared with view-specific weights.

TRASMIL: A local anomaly detection framework based on trajectory segmentation and multi-instance learning

Local anomaly detection refers to detecting small anomalies or outliers that exist in some subsegments of events or behaviors. Such local anomalies are easily overlooked by most of the existing approaches since they are designed for detecting global or large anomalies. In this paper, an accurate and flexible three-phase framework TRASMIL is proposed for local anomaly detection based on TRAjectory Segmentation and Multi-Instance Learning. Firstly, every motion trajectory is segmented into independent sub-trajectories, and a metric with Diversity and Granularity is proposed to measure the quality of segmentation. Secondly, the segmented sub-trajectories are modeled by a sequence learning model. Finally, multi-instance learning is applied to detect abnormal trajectories and sub-trajectories which are viewed as bags and instances, respectively. We validate the TRASMIL framework in terms of 16 different algorithms built on the three-phase framework. Substantial experiments show that algorithms based on the TRASMIL framework outperform existing methods in effectively detecting the trajectories with local anomalies in terms of the whole trajectory. In particular, the MDL-C algorithm (the combination of HDP-HMM with MDL segmentation and Citation kNN) achieves the highest accuracy and recall rates. We further show that TRASMIL is generic enough to adopt other algorithms for identifying local anomalies.

Abnormal event detection via multi-instance dictionary learning

In this paper, we present a method for detecting abnormal events in videos. In the proposed method, we define an event containing several sub-events. Sub-events can be viewed as instances and an event as a bag of instances in the multi-instance learning formulation. Given labeled events but with the labels of sub-events unknown, the proposed method is able to learn a dictionary together with a classification function. The dictionary is capable of generating discriminant sparse codes of sub-events while the classification function is able to classify an event. This method is suited for scenarios where the label of a sub-event is ambiguous, while the label of a set of sub-events is definite and is easy to obtain. Once the sparse codes of sub-events are generated, the classification of an event is carried out according to the result given by the classification function. An efficient optimization procedure of the proposed method is presented. Experiments show that the method is able to detect abnormal events with comparable or improved accuracy compared with other methods.

Heterogeneous Face Recognition by Margin-Based Cross-Modality Metric Learning

Heterogeneous face recognition deals with matching face images from different modalities or sources. The main challenge lies in cross-modal differences and variations and the goal is to make cross-modality separation among subjects. A margin-based cross-modality metric learning (MCM2L) method is proposed to address the problem. A cross-modality metric is defined in a common subspace where samples of two different modalities are mapped and measured. The objective is to learn such metrics that satisfy the following two constraints. The first minimizes pairwise, intrapersonal cross-modality distances. The second forces a margin between subject specific intrapersonal and interpersonal cross-modality distances. This is achieved by defining a hinge loss on triplet-based distance constraints for efficient optimization. It allows the proposed method to focus more on optimizing distances of those subjects whose intrapersonal and interpersonal distances are hard to separate. The proposed method is further extended to a kernelized MCM2L (KMCM2L). Both methods have been evaluated on an ID card face dataset and two other cross-modality benchmark datasets. Various feature extraction methods have also been incorporated in the study, including recent deep learned features. In extensive experiments and comparisons with the state-of-the-art methods, the MCM2L and KMCM2L methods achieved marked improvements in most cases.

mPadal: a joint local-and-global multi-view feature selection method for activity recognition

The selection of multi-view features plays an important role for classifying multi-view data, especially the data with high dimension. In this paper, a novel multi-view feature selection method via joint local pattern-discrimination and global label-relevance analysis (mPadal) is proposed. Different from the previous methods which globally select the multi-view features directly via view-level analysis, the proposed mPadal employs a new joint local-and-global way. In the local selection phase, the pattern-discriminative features will be first selected by considering the local neighbor structure of the most discriminative patterns. In the global selection phase, the features with the topmost label-relevance, which can well separate different classes in the current view, are selected. Finally, the two parts selected are combined to form the final features. Experimental results show that compared with several baseline methods in publicly available activity recognition dataset IXMAS, mPadal performs the best in terms of the highest accuracy, precision, recall and F1 score. Moreover, the features selected by mPadal are highly complementary among views for classification, which is able to improve the classification performance according to previous theoretical studies.

Does manual delineation only provide the side information in CT prostate segmentation

Prostate segmentation, for accurate prostate localization in CT images, is regarded as a crucial yet challenging task. Nevertheless, due to the inevitable factors (e.g., low contrast, large appearance and shape changes), the most important problem is how to learn the informative feature representation to distinguish the prostate from non-prostate regions. We address this challenging feature learning by leveraging the manual delineation as guidance: the manual delineation does not only indicate the category of patches, but also helps enhance the appearance of prostate. This is realized by the proposed cascaded deep domain adaptation (CDDA) model. Specifically, CDDA constructs several consecutive source domains by employing a mask of manual delineation to overlay on the original CT images with different mask ratios. Upon these source domains, convnet will guide better transferrable feature learning until to the target domain. Particularly, we implement two typical methods: patch-to-scalar (CDDA-CNN) and patch-to-patch (CDDA-FCN). Also, we theoretically analyze the generalization error bound of CDDA. Experimental results show the promising results of our method.

Interactive image segmentation via cascaded metric learning

In this paper, we propose an interactive image segmentation method from a novel perspective of cascaded metric learning. Given an image with user-marked scribbles that are essentially uncertain and noisy, our method completes the segmentation task by solving a binary classification problem. Starting from the initial training samples with known class labels (i.e., regions of the image that are believed with high confidence to be foreground or background), we first find an optimal metric that can best describe the classification of these samples. After that, we classify the unlabeled samples using the learnt metric. Samples classified with high confidence are used as new training samples to refine the metric. This cycle of metric learning and classification repeats until the accomplishment of the image segmentation task. The proposed method is extensively evaluated on the MSRC image set. Experiment results show that our method outperforms the state-of-the-art methods.

Spatial-aware hyperspectral image classification via multifeature kernel dictionary learning

Sparse representation based on dictionary learning has yielded impressive effects on hyperspectral image (HSI) classification. But most of these methods utilize only the single spectral feature of HSI and advanced features are not considered, such that the discriminability of sparse representation coefficients is relatively weak. In this paper, we propose a novel multifeature spatial aware dictionary learning model by incorporating complementary across-feature and contextual information obtaining from HSI. The newly developed model, by designing a joint sparse regularization term for pixels represented by several complementary yet correlated features in a contextual group, makes the learning sparse coefficients have enough discriminability. Also, in order to further improve the discrimination ability of coding coefficients, utilizing kernel trick, we design the corresponding kernel extension of the newly proposed model. Based on the newly presented models, we give two corresponding discriminant dictionary learning algorithms. The experimental results on Indian Pines and University of Pavia images show that the effectiveness of the proposed algorithms, which also validate that our algorithms can obtain more discriminant coding coefficients.

Online multi-view subspace learning via group structure analysis for visual object tracking

In this paper, we focus on incrementally learning a robust multi-view subspace representation for visual object tracking. During the tracking process, due to the dynamic background variation and target appearance changing, it is challenging to learn an informative feature representation of tracking object, distinguished from the dynamic background. To this end, we propose a novel online multi-view subspace learning algorithm (OMEL) via group structure analysis, which consistently learns a low-dimensional representation shared across views with time changing. In particular, both group sparsity and group interval constraints are incorporated to preserve the group structure in the low-dimensional subspace, and our subspace learning model will be incrementally updated to prevent repetitive computation of previous data. We extensively evaluate our proposed OMEL on multiple benchmark video tracking sequences, by comparing with six related tracking algorithms. Experimental results show that OMEL is robust and effective to learn dynamic subspace representation for online object tracking problems. Moreover, several evaluation tests are additionally conducted to validate the efficacy of group structure assumption.