Arslan Basharat

Learning object motion patterns for anomaly detection and improved object detection

We present a novel framework for learning patterns of motion and sizes of objects in static camera surveillance. The proposed method provides a new higher-level layer to the traditional surveillance pipeline for anomalous event detection and scene model feedback. Pixel level probability density functions (pdfs) of appearance have been used for background modelling in the past, but modelling pixel level pdfs of object speed and size from the tracks is novel. Each pdf is modelled as a multivariate Gaussian mixture model (GMM) of the motion (destination location & transition time) and the size (width & height) parameters of the objects at that location. Output of the tracking module is used to perform unsupervised EM-based learning of every GMM. We have successfully used the proposed scene model to detect local as well as global anomalies in object tracks. We also show the use of this scene model to improve object detection through pixel-level parameter feedback of the minimum object size and background learning rate. Most object path modelling approaches first cluster the tracks into major paths in the scene, which can be a source of error. We avoid this by building local pdfs that capture a variety of tracks which are passing through them. Qualitative and quantitative analysis of actual surveillance videos proved the effectiveness of the proposed approach.

Chaotic Invariants for Human Action Recognition

The paper introduces an action recognition framework that uses concepts from the theory of chaotic systems to model and analyze nonlinear dynamics of human actions. Trajectories of reference joints are used as the representation of the non-linear dynamical system that is generating the action. Each trajectory is then used to reconstruct a phase space of appropriate dimension by employing a delay-embedding scheme. The properties of the reconstructed phase space are captured in terms of dynamical and metric invariants that include Lyapunov exponent, correlation integral and correlation dimension. Finally, the action is represented by a feature vector which is a combination of these invariants over all the reference trajectories. Our contributions in this paper include :1) investigation of the appropriateness of theory of chaotic systems for human action modelling and recognition, 2) a new set of features to characterize nonlinear dynamics of human actions, 3) experimental validation of the feasibility and potential merits of carrying out action recognition using methods from theory of chaotic systems.

Content based video matching using spatiotemporal volumes

This paper presents a novel framework for matching video sequences using the spatiotemporal segmentation of videos. Instead of using appearance features for region correspondence across frames, we use interest point trajectories to generate video volumes. Point trajectories, which are generated using the SIFT operator, are clustered to form motion segments by analyzing their motion and spatial properties. The temporal correspondence between the estimated motion segments is then established based on most common SIFT correspondences. A two pass correspondence algorithm is used to handle splitting and merging regions. Spatiotemporal volumes are extracted using the consistently tracked motion segments. Next, a set of features including color, texture, motion, and SIFT descriptors are extracted to represent a volume. We employ an Earth Movers Distance (EMD) based approach for the comparison of volume features. Given two videos, a bipartite graph is constructed by modeling the volumes as vertices and their similarities as edge weights. Maximum matching of this graph produces volume correspondences between the videos, and these volume matching scores are used to compute the final video matching score. Experiments for video retrieval were performed on a variety of videos obtained from different sources including BBC Motion Gallery and promising results were achieved. We present qualitative and quantitative analysis of retrieval along with a comparison with two baseline methods.

A framework for intelligent sensor network with video camera for structural health monitoring of bridges

Wireless sensor network (WSN) gives the characteristics of an effective, feasible and fairly reliable monitoring system which shows promise for structural health monitoring (SHM) applications. Monitoring of civil structures generates a large amount of sensor data that is used for structural anomaly detection. Efficiently dealing with this large amount of data in a resource-constrained WSN is a challenge. This paper proposes a, WSN based, novel framework that triggers smart events from sensor data. These events are useful for both intelligent data recording and video camera control. The operation of this framework consists of active & passive sensing modes. In passive mode, selected nodes can intelligently interpret local sensor data to trigger appropriate events. In active mode, most of the sensing nodes perform high frequency sampling and record useful data. Unnecessary data is suppressed which improves the lifespan of the network and simplifies data management.

Time series prediction by chaotic modeling of nonlinear dynamical systems

We use concepts from chaos theory in order to model nonlinear dynamical systems that exhibit deterministic behavior. Observed time series from such a system can be embedded into a higher dimensional phase space without the knowledge of an exact model of the underlying dynamics. Such an embedding warps the observed data to a strange attractor, in the phase space, which provides precise information about the dynamics involved. We extract this information from the strange attractor and utilize it to predict future observations. Given an initial condition, the predictions in the phase space are computed through kernel regression. This approach has the advantage of modeling dynamics without making any assumptions about the exact form (linear, polynomial, radial basis, etc.) of the mapping function. The predicted points are then warped back to the observed time series. We demonstrate the utility of these predictions for human action synthesis, and dynamic texture synthesis. Our main contributions are: multivariate phase space reconstruction for human actions and dynamic textures, a deterministic approach to model dynamics in contrast to the popular noise-driven approaches for dynamic textures, and video synthesis from kernel regression in the phase space. Experimental results provide qualitative and quantitative analysis of our approach on standard data sets.

Real-time multi-target tracking at 210 megapixels/second in Wide Area Motion Imagery

We present a real-time, full-frame, multi-target Wide Area Motion Imagery (WAMI) tracking system that utilizes distributed processing to handle high data rates while maintaining high track quality. The proposed architecture processes the WAMI data as a series of geospatial tiles and implements both process- and thread-level parallelism across multiple compute nodes. Each tile is processed independently, from decoding the image through generating tracks that are finally merged across all tiles by an inter-tile linker (ITL) module. A high performance PostgreSQL database with GIS extensions is used to control the flow of intermediate data between each tracking process. High quality tracks are produced efficiently due to robust, effective algorithmic modules including: multi-frame moving object detection and track initialization; tracking based on the fusion of motion and appearance with a goal of very pure tracks; and online track linking based on multiple features. In addition, we have configured a high-performance compute cluster using high density blade servers, Infiniband networking, and an HPC filesystem. The compute cluster enables full-frame, state-of-the-art tracking of vehicles or dismounts at the WAMI sensors native 1.25Hz frame-rate, while only taking 7u of rack space and providing 210 megapixels/second throughput.

Track Initialization in Low Frame Rate and Low Resolution Videos

The problem of object detection and tracking has received relatively less attention in low frame rate and low resolution videos. Here we focus on motion segmentation in videos where objects appear small (less than 30-pixel tall people) and have low frame rate (less than 5 Hz). We study challenging cases where some of the, otherwise successful, approaches may break down. We investigate a number of popular techniques in computer vision that have been shown to be useful for discriminating various spatio-temporal signatures. These include: Histogram of oriented Gradients (HOG), Histogram of oriented optical Flow (HOF) and Haar-features (Viola and Jones). We use these feature to classify the motion segmentations into person vs. other and vehicle vs. other. We rely on aligned motion history images to create a more consistent object representation across frames. We present results on these features using webcam data and wide-area aerial video sequences.

Person and Vehicle Tracking in Surveillance Video

This evaluation for person and vehicle tracking in surveillance presented some new challenges. The dataset was large and very high-quality, but with difficult scene properties involving illumination changes, unusual lighting conditions, and complicated occlusion of objects. Since this is a well-researched scenario [1], our submission was based primarily on our existing projects for automated object detection and tracking in surveillance. We also added several new features that are practical improvements for handling the difficulties of this dataset.

Geometric constraints on 2D action models for tracking human body

We propose a 2D model-based approach for tracking human body parts during articulated motion. A human is modeled as a stick figure with thirteen landmarks, and an action is a sequence of these stick figures. Given the locations of these joints in a model video and only the first frame of a test video, the joint locations are automatically estimated throughout the test video using two geometric constraints. The first constraint is based on the invariance of the ratio of areas under an affine transformation, and provides initial estimates. The second one is based on the fundamental matrix, defined by the corresponding landmarks of the two actors, and refines the initial estimates. Using these estimated locations, the tracking algorithm determines the exact location of each joint in the test video. The novelty of our approach lies in the geometric formulation of human actions and the use of geometric constraints for body joints estimation. The approach is able to handle variations in anthropometry of individuals, viewpoints, execution rate, and style of action execution. Experimental results provide encouraging quantitative and qualitative performance analysis.

Moving object detection for vehicle tracking in Wide Area Motion Imagery using 4D filtering

Most Wide Area Motion Imagery (WAMI) based trackers use motion based cueing for detecting and tracking moving objects. The results are very high false alarm rates in urban environments with tall structures due to parallax effects. This paper proposes an accurate moving object detection method using a precise orthorectification approach for ground stabilization combined with accurate multiview depth maps to reduce the number of false positives induced by parallax effects by 90 percent. Proposed hybrid moving vehicle detection approach for large scale aerial urban imagery is based on fusion of motion detection mask obtained from median-based background subtraction and tall structures height mask provided by image depth map information. Using buildings mask enables us to improve the object level detection accuracy in terms of F-measure by 57 percent from 22.2% to 79.2%.