Hossein Mousavi

Analyzing Tracklets for the Detection of Abnormal Crowd Behavior

This paper presents a novel video descriptor, referred to as Histogram of Oriented Tracklets, for recognizing abnormal situation in crowded scenes. Unlike standard approaches that use optical flow, which estimates motion vectors only from two successive frames, we built our descriptor over long-range motion trajectories which is called tracklets in the literature. Following the standard procedure, we divided video sequences in spatio-temporal cuboids within which we collected statistics on the tracklets passing through them. In particular, we quantized orientation and magnitude in a 2-dimensional histogram which encodes the motion patterns expected in each cuboid. We classify frames as normal and abnormal by using Latent Dirichlet Allocation and Support Vector Machines. We evaluated the effectiveness of the proposed descriptors on three datasets: UCSD, Violence in Crowds and UMN. The experiments demonstrated (i) very promising results in abnormality detection, (ii) setting new state-of-the-art on two of them, and (iii) outperforming former descriptors based on the optical flow, dense trajectories and the social force model.

Abnormality Detection with Improved Histogram of Oriented Tracklets

Recently the histogram of oriented tracklets (HOT) was shown to be an efficient video representation for abnormality detection and achieved state-of-the-arts on the available datasets. Unlike standard video descriptors that mainly employ low level motion features, e.g. optical flow, the HOT descriptor simultaneously encodes magnitude and orientation of tracklets as a mid-level representation over crowd motions. However, extracting tracklets in HOT suffers from poor salient point initialization and tracking drift in the presence of occlusion. Moreover, count-based HOT histogramming does not properly take into account the motion characteristics of abnormal motions. This paper extends the HOT by addressing these drawbacks introducing an enhanced version of HOT, named Improved HOT. First, we propose to initialize salient points in each frame instead of the first frame, as the HOT does. Second, we replace the naive count-based histogramming by the richer statistics of crowd movement (i.e., motion distribution). The evaluation of the Improved HOT on different datasets, namely UCSD, BEHAVE and UMN, yields compelling results in abnormality detection, by outperforming the original HOT and the state-of-the-art descriptors based on optical flow, dense trajectories and the social force models.

Crowd motion monitoring using tracklet-based commotion measure.

Abnormal detection in crowd is a challenging vision task due to the scarcity of real-world training examples and the lack of a clear definition of abnormality. To tackle these challenges, we propose a novel measure to capture the commotion of a crowd motion for the task of abnormality detection in crowd. The unsupervised nature of the proposed measure allows to detect abnormality adaptively (i.e. context dependent) with no training cost. The extensive experiments on three different levels (e.g. pixel, frame and video) show the superiority of the proposed approach compared to the state of the arts.

Detecting Abnormal Behavioral Patterns in Crowd Scenarios

This Chapter presents a framework for the the task of abnormality detection in crowded scenes based on the analysis of trajectories, build up upon a novel video descriptor, called Histogram of Oriented Tracklets. Unlike standard approaches that employ low level motion features, e.g. optical flow, to form video descriptors, we propose to exploit mid-level features extracted from long-range motion trajectories called tracklets, which have been successfully applied for action modeling and video analysis. Following standard procedure, a video sequence is divided into spatio-temporal cuboids within which we collect statistics of the tracklets passing through them. Specifically, tracklets orientation and magnitude are quantized in a two-dimensional histogram which encodes the actual motion patterns in each cuboid. These histograms are then fed into machine learning models (e.g., Latent Dirichlet allocation and Support Vector Machines) to detect abnormal behaviors in video sequences. The evaluation of the proposed descriptor on different datasets, namely UCSD, BEHAVE, UMN and Violence in Crowds, yields compelling results in abnormality detection, by setting new state-of-the-art and outperforming former descriptors based on the optical flow, dense trajectories and social force models.

Novel dataset for fine-grained abnormal behavior understanding in crowd

Despite the huge research on crowd on behavior understanding in visual surveillance community, lack of publicly available realistic datasets for evaluating crowd behavioral interaction led not to have a fair common test bed for researchers to compare the strength of their methods in the real scenarios. This work presents a novel crowd dataset contains around 45,000 video clips which annotated by one of the five different fine-grained abnormal behavior categories. We also evaluated two state-of-the-art methods on our dataset, showing that our dataset can be effectively used as a benchmark for fine-grained abnormality detection. The details of the dataset and the results of the baseline methods are presented in the paper.

Crowd behavior representation: an attribute-based approach

In crowd behavior studies, a model of crowd behavior needs to be trained using the information extracted from video sequences. Most of the previous methods are based on low-level visual features because there are only crowd behavior labels available as ground-truth information in crowd datasets. However, there is a huge semantic gap between low-level motion/appearance features and high-level concept of crowd behaviors. In this paper, we tackle the problem by introducing an attribute-based scheme. While similar strategies have been employed for action and object recognition, to the best of our knowledge, for the first time it is shown that the crowd emotions can be used as attributes for crowd behavior understanding. We explore the idea of training a set of emotion-based classifiers, which can subsequently be used to indicate the crowd motion. In this scheme, we collect a large dataset of video clips and provide them with both annotations of “crowd behaviors” and “crowd emotions”. We test the proposed emotion based crowd representation methods on our dataset. The obtained promising results demonstrate that the crowd emotions enable the construction of more descriptive models for crowd behaviors. We aim at publishing the dataset with the article, to be used as a benchmark for the communities.

CNN-aware binary MAP for general semantic segmentation

In this paper we introduce a novel method for general semantic segmentation that can benefit from general semantics of Convolutional Neural Network (CNN). Our segmentation proposes visually and semantically coherent image segments. We use binary encoding of CNN features to overcome the difficulty of the clustering on the high-dimensional CNN feature space. These binary codes are very robust against noise and non-semantic changes in the image. These binary encoding can be embedded into the CNN as an extra layer at the end of the network. This results in real-time segmentation. To the best of our knowledge our method is the first attempt on general semantic image segmentation using CNN. All the previous papers were limited to few number of category of the images (e.g. PASCAL VOC). Experiments show that our segmentation algorithm outperform the state-of-the-art non-semantic segmentation methods by large margin.

Erratum to: Crowd behavior representation: an attribute-based approach

[This corrects the article DOI: 10.1186/s40064-016-2786-0.].