Sovan Biswas

Real time anomaly detection in H.264 compressed videos

Real time anomaly detection is the need of the hour for any security applications. In this paper, we have proposed a real-time anomaly detection algorithm by utilizing cues from the motion vectors in H.264/AVC compressed domain. The discussed work is principally motivated by the observation that motion vectors (MVs) exhibit different characteristics during anomaly. We have observed that H.264 motion vector magnitude contains relevant information which can be used to model the usual behavior (UB) effectively. This is subsequently extended to detect abnormality/anomaly based on the probability of occurrence of a behavior. Additionally, we have suggested a hierarchical approach through Motion Pyramid for High Resolution videos to further increase the detection rate. The proposed algorithm has performed extremely well on UMN and Peds Anomaly Detection Video datasets, with a detection speed of >150 and 65-75 frames per sec in respective datasets resulting in more than 200× speedup along with comparable accuracy to pixel domain state-of-the-art algorithms.

H.264 compressed video classification using Histogram of Oriented Motion Vectors (HOMV)

In this paper, we have proposed a simple and effective approach to classify H.264 compressed videos, by capturing orientation information from the motion vectors. Our major contribution involves computing Histogram of Oriented Motion Vectors (HOMV) for overlapping hierarchical Space-Time cubes. The Space-Time cubes selected are partially overlapped. HOMV is found to be very effective to define the motion characteristics of these cubes. We then use Bag of Features (BOF) approach to define the video as histogram of HOMV keywords, obtained using k-means clustering. The video feature, thus computed, is found to be very effective in classifying videos. We demonstrate our results with experiments on two large publicly available video database.

Anomaly detection in compressed H.264/AVC video

Real time anomaly detection is the need of the hour for any security applications. In this article, we have proposed a real time anomaly detection for H.264 compressed video streams utilizing pre-encoded motion vectors (MVs). The proposed work is principally motivated by the observation that MVs have distinct characteristics during anomaly than usual. Our observation shows that H.264 MV magnitude and orientation contain relevant information which can be used to model the usual behavior (UB) effectively. This is subsequently extended to detect abnormality/anomaly based on the probability of occurrence of a behavior. The performance of the proposed algorithm was evaluated and bench-marked on UMN and Ped anomaly detection video datasets, with a detection rate of 70 frames per sec resulting in 90× and 250× speedup, along with on-par detection accuracy compared to the state-of-the-art algorithms.

Super-pixel based crowd flow segmentation in H.264 compressed videos

In this paper, we have proposed a simple yet robust novel approach for segmentation of high density crowd flows based on super-pixels in H.264 compressed videos. The collective representation of the motion vectors of the compressed video sequence is transformed to color map and super-pixel segmentation is performed at various scales for clustering the coherent motion vectors. The number of dynamically meaningful flow segments is determined by measuring the confidence score of the accumulated multi-scale super-pixel boundaries. The final crowd flow segmentation is obtained from the edges that are consistent across all the super-pixel resolutions. Hence, our major contribution involves obtaining the flow segmentation by clustering the motion vectors and determination of number of flow segments using only motion super-pixels without any prior assumption of the number of flow segments. The proposed approach was bench-marked on standard crowd flow dataset. Experiments demonstrated better accuracy and speedup for the proposed approach compared to the state-of-the-art methods.

Anomaly detection via short local trajectories

Trajectory provides an important motion cue in describing the behavior of the moving object and can be used effectively for anomaly detection. But traditional trajectories or tracklets used for analysis have limitations due to various tracking irregularities. In this paper, we propose a novel idea of detecting anomalies in a video, based on short history of a region in motion. These histories are defined as short local trajectories (SLT). In contrast to traditional tracklets, these SLTs are extracted for super-pixels belonging to foreground objects. This captures both spatial and temporal information of a candidate moving object. Furthermore, the proposed trajectory extraction is suitable across videos having different crowd density, occlusions, etc. The trajectories of persons/objects at a particular location under usual condition have certain attributes. Thus, we have used Hidden Markov Model (HMM) for characterizing the usual trajectory patterns for each defined region. The proposed algorithm takes SLTs as observations and measures the likelihood for each super-pixel of being anomaly based on learned HMMs. In order to avoid the influence of noisy trajectories, we have computed spatial consistency measure for each SLT based on the neighboring trajectories. Thus, anomalies detected by the proposed approach are highly localized as demonstrated from the experiments conducted on three anomaly datasets, namely UCSD Ped1, Ped2 and a newly proposed CHUK-Crowd Anomaly Dataset.

Short Local Trajectory based moving anomaly detection

A high level abstraction of the behavior a moving object can be obtained by analyzing its trajectory. However, traditional trajectories or tracklets are bound by the limitations of the underlying tracking algorithm used. In this paper, we propose a novel idea of detecting anomalous objects amid other moving objects in a video based on its short history. This history is defined as short local trajectory (SLT). The unique approach of generating SLTs from super-pixels belonging to a foreground object that incorporates both spatial and temporal information is the key in detection of anomaly. Additionally, the proposed trajectory extraction is robust across videos having different crowd density, occlusions, etc. Generally the trajectories of persons/objects moving at a particular region under usual conditions has certain fixed characteristics, thus we use Hidden Markov Model (HMM) for capturing the usual trajectory patterns during training. Whereas during detection, the proposed algorithm takes SLTs as observations for each super-pixel and measures its likelihood of being anomaly using the learned HMMs. Furthermore, we compute the spatial consistency measure for each SLT based on the neighboring trajectories. Thus, anomaly detected by the proposed approach is highly localized as demonstrated from the experiments conducted on two widely used anomaly datasets, namely UCSD Ped1 and UCSD Ped2.

Sparse representation based anomaly detection using HOMV in H.264 compressed videos

In this paper, we have proposed an anomaly detection algorithm based on Histogram of Oriented Motion Vectors (HOMV) [1] in sparse representation framework. Usual behavior is learned at each location by sparsely representing the HOMVs over learnt normal feature bases obtained using an online dictionary learning algorithm. In the end, anomaly is detected based on the likelihood of the occurrence of sparse coefficients at that location. The proposed approach is found to be robust compared to existing methods as demonstrated in the experiments on UCSD Ped1 and UCSD Ped2 datasets.