Moin Nabi

Abnormality Detection with Improved Histogram of Oriented Tracklets

Recently the histogram of oriented tracklets (HOT) was shown to be an efficient video representation for abnormality detection and achieved state-of-the-arts on the available datasets. Unlike standard video descriptors that mainly employ low level motion features, e.g. optical flow, the HOT descriptor simultaneously encodes magnitude and orientation of tracklets as a mid-level representation over crowd motions. However, extracting tracklets in HOT suffers from poor salient point initialization and tracking drift in the presence of occlusion. Moreover, count-based HOT histogramming does not properly take into account the motion characteristics of abnormal motions. This paper extends the HOT by addressing these drawbacks introducing an enhanced version of HOT, named Improved HOT. First, we propose to initialize salient points in each frame instead of the first frame, as the HOT does. Second, we replace the naive count-based histogramming by the richer statistics of crowd movement (i.e., motion distribution). The evaluation of the Improved HOT on different datasets, namely UCSD, BEHAVE and UMN, yields compelling results in abnormality detection, by outperforming the original HOT and the state-of-the-art descriptors based on optical flow, dense trajectories and the social force models.

Crowd motion monitoring using tracklet-based commotion measure.

Abnormal detection in crowd is a challenging vision task due to the scarcity of real-world training examples and the lack of a clear definition of abnormality. To tackle these challenges, we propose a novel measure to capture the commotion of a crowd motion for the task of abnormality detection in crowd. The unsupervised nature of the proposed measure allows to detect abnormality adaptively (i.e. context dependent) with no training cost. The extensive experiments on three different levels (e.g. pixel, frame and video) show the superiority of the proposed approach compared to the state of the arts.

Temporal Poselets for Collective Activity Detection and Recognition

Detection and recognition of collective human activities are important modules of any system devoted to high level social behavior analysis. In this paper, we present a novel semantic-based spatio-temporal descriptor which can cope with several interacting people at different scales and multiple activities in a video. Our descriptor is suitable for modelling the human motion interaction in crowded environments - the scenario most difficult to analyse because of occlusions. In particular, we extend the Pose let detector approach by defining a descriptor based on Pose let activation patterns over time, named TPOS. We will show that this descriptor can effectively tackle complex real scenarios allowing to detect humans in the scene, to localize (in space-time) human activities, and perform collective group activity recognition in a joint manner, reaching state-of-the-art results.

Novel dataset for fine-grained abnormal behavior understanding in crowd

Despite the huge research on crowd on behavior understanding in visual surveillance community, lack of publicly available realistic datasets for evaluating crowd behavioral interaction led not to have a fair common test bed for researchers to compare the strength of their methods in the real scenarios. This work presents a novel crowd dataset contains around 45,000 video clips which annotated by one of the five different fine-grained abnormal behavior categories. We also evaluated two state-of-the-art methods on our dataset, showing that our dataset can be effectively used as a benchmark for fine-grained abnormality detection. The details of the dataset and the results of the baseline methods are presented in the paper.

Detection and localization of crowd behavior using a novel tracklet-based model

In this paper, two novel descriptors are introduced to detect and localize abnormal behaviors in crowded scenes. The first proposed descriptor is based on the orientation and magnitude of short trajectories extracted by tracking interest points in spatio-temporal 3D patches. The proposed descriptor employs a novel simplified Histogram of Oriented Tracklets (sHOT), which is shown to be very effective in the task of crowd abnormal behavior detection. In this scheme, abnormal behaviors are detected at different levels, namely spatio-temporal level and frame level. By combining the first proposed descriptor and the dense optical flow model, we propose our second framework which is able to localize the abnormal behavior areas in video sequences. The evaluation of our simple but yet effective descriptors on different state-of-the-art datasets, namely UCSD, UMN and Violence in Crowds yields very promising results in abnormality detection and outperforming different former state-of-the-art descriptors.

Learning with dataset bias in latent subcategory models

Latent subcategory models (LSMs) offer significant improvements over training linear support vector machines (SVMs). Training LSMs is a challenging task due to the potentially large number of local optima in the objective function and the increased model complexity which requires large training set sizes. Often, larger datasets are available as a collection of heterogeneous datasets. However, previous work has highlighted the possible danger of simply training a model from the combined datasets, due to the presence of bias. In this paper, we present a model which jointly learns an LSM for each dataset as well as a compound LSM. The method provides a means to borrow statistical strength from the datasets while reducing their inherent bias. In experiments we demonstrate that the compound LSM, when tested on PASCAL, LabelMe, Caltech101 and SUN09 in a leave-one-dataset-out fashion, achieves an average improvement of over 6.5% over a previous SVM-based undoing bias approach and an average improvement of over 8.5% over a standard LSM trained on the concatenation of the datasets.

CNN-aware binary MAP for general semantic segmentation

In this paper we introduce a novel method for general semantic segmentation that can benefit from general semantics of Convolutional Neural Network (CNN). Our segmentation proposes visually and semantically coherent image segments. We use binary encoding of CNN features to overcome the difficulty of the clustering on the high-dimensional CNN feature space. These binary codes are very robust against noise and non-semantic changes in the image. These binary encoding can be embedded into the CNN as an extra layer at the end of the network. This results in real-time segmentation. To the best of our knowledge our method is the first attempt on general semantic image segmentation using CNN. All the previous papers were limited to few number of category of the images (e.g. PASCAL VOC). Experiments show that our segmentation algorithm outperform the state-of-the-art non-semantic segmentation methods by large margin.

Self-Crowdsourcing Training for Relation Extraction

In this paper we introduce a self-training strategy for crowdsourcing. The training examples are automatically selected to train the crowd workers. Our experimental results show an impact of 5% Improvement in terms of F1 for relation extraction task, compared to the method based on distant supervision.

Sparse-coded cross-domain adaptation from the visual to the brain domain

Brain decoding (i.e., retrieving information from brain signals by employing machine learning algorithms) has recently received considerable attention across many communities. In a typical brain decoding paradigm, different types of stimuli are shown to the participant of the neuroimaging experiment, while his/her concurrent brain activity is captured using neuroimaging techniques. Then a machine learning algorithm is employed to categorize the measured brain signal into the target stimuli classes. Accurate prediction of the stimulus category by the algorithm is considered a positive evidence of the hypothesis of the existence of stimulus-related information in brain data. However, most of the brain decoding studies suffer from the constraint of having few and noisy samples. In order to overcome this limitation, in this paper, an adaptation paradigm is employed in order to transfer knowledge from visual domain to brain domain. We experimentally show that such adaptation procedure leads to improved results for the object recognition task in the brain domain, outperforming significantly the results achieved by the brain features alone. This is the first study in the direction of transferring knowledge by adapting representations learned on visual domain to the brain modality. We believe this paper opens up avenues for exploiting large-scale visual datasets to achieve performance gain in brain decoding.

A cross-modal adaptation approach for brain decoding

Brain decoding has become a hot topic in many recent brain studies. In a typical neuroimaging experiment, participants are presented with different categories of stimuli while their concurrent brain activity is recorded. Then a classifier is trained on the features extracted from the recorded brain data to discriminate different target stimuli classes. It is a common practice to hypothesize that the stimulus-related information exists in the brain data if the decoder can accurately predict the target stimulus category. However, most of the neuroimaging studies suffer from few and noisy samples. These constraints affects the performance of such decoding systems. In order to cope with this limitation, a dictionary learning approach is used in this paper to transfer knowledge from the multimedia domain to the brain domain. We show that such cross-modal domain adaptation yields better performance of the learning algorithm in the brain domain. This is the first study in the direction of cross-modal adaptation by joint dictionary learning on multimedia and brain modality.