Faisal I. Bashir

Object Trajectory-Based Activity Classification and Recognition Using Hidden Markov Models

Motion trajectories provide rich spatiotemporal information about an objects activity. This paper presents novel classification algorithms for recognizing object activity using object motion trajectory. In the proposed classification system, trajectories are segmented at points of change in curvature, and the subtrajectories are represented by their principal component analysis (PCA) coefficients. We first present a framework to robustly estimate the multivariate probability density function based on PCA coefficients of the subtrajectories using Gaussian mixture models (GMMs). We show that GMM-based modeling alone cannot capture the temporal relations and ordering between underlying entities. To address this issue, we use hidden Markov models (HMMs) with a data-driven design in terms of number of states and topology (e.g., left-right versus ergodic). Experiments using a database of over 5700 complex trajectories (obtained from UCI-KDD data archives and Columbia University Multimedia Group) subdivided into 85 different classes demonstrate the superiority of our proposed HMM-based scheme using PCA coefficients of subtrajectories in comparison with other techniques in the literature.

Real-Time Motion Trajectory-Based Indexing and Retrieval of Video Sequences

This paper presents a novel motion trajectory-based compact indexing and efficient retrieval mechanism for video sequences. Assuming trajectory information is already available, we represent trajectories as temporal ordering of subtrajectories. This approach solves the problem of trajectory representation when only partial trajectory information is available due to occlusion. It is achieved by a hypothesis testing-based method applied to curvature data computed from trajectories. The subtrajectories are then represented by their principal component analysis (PCA) coefficients for optimally compact representation. Different techniques are integrated to index and retrieve subtrajectories, including PCA, spectral clustering, and string matching. We assume a query by example mechanism where an example trajectory is presented to the system and the search system returns a ranked list of most similar items in the dataset. Experiments based on datasets obtained from University of California at Irvines KDD archives and Columbia Universitys DVMM group demonstrate the superiority of our proposed PCA-based approaches in terms of indexing and retrieval times and precision recall ratios, when compared to other techniques in the literature

Segmented trajectory based indexing and retrieval of video data

In this paper, we present a novel principal component analysis (PCA) based approach towards modeling the object trajectory in a video clip. An eigenspace decomposition of high-dimensional trajectory data leads to very compact representation, which is then used as indexing structure. To cutback on PCA computation during indexing, we first segment the trajectories into atomic subtrajectories using a curvature zero-crossing based approach followed by clustering of these subtrajectories. A two-level PCA operation with coarse-to-fine retrieval for query trajectory is then performed to generate retrieval results. Our experimental results show that our global PCA based approach performs better when input query trajectory is of similar length compared to the matching trajectories in the database. However, when partial trajectories are posed as queries our segmented trajectory based approach provides superior results for all precision-recall ratios.

View-invariant motion trajectory-based activity classification and recognition

Motion trajectories provide rich spatio-temporal information about an objects activity. The trajectory information can be obtained using a tracking algorithm on data streams available from a range of devices including motion sensors, video cameras, haptic devices, etc. Developing view-invariant activity recognition algorithms based on this high dimensional cue is an extremely challenging task. This paper presents efficient activity recognition algorithms using novel view-invariant representation of trajectories. Towards this end, we derive two Affine-invariant representations for motion trajectories based on curvature scale space (CSS) and centroid distance function (CDF). The properties of these schemes facilitate the design of efficient recognition algorithms based on hidden Markov models (HMMs). In the CSS-based representation, maxima of curvature zero crossings at increasing levels of smoothness are extracted to mark the location and extent of concavities in the curvature. The sequences of these CSS maxima are then modeled by continuous density (HMMs). For the case of CDF, we first segment the trajectory into subtrajectories using CDF-based representation. These subtrajectories are then represented by their Principal Component Analysis (PCA) coefficients. The sequences of these PCA coefficients from subtrajectories are then modeled by continuous density hidden Markov models (HMMs). Different classes of object motions are modeled by one Continuous HMM per class where state PDFs are represented by GMMs. Experiments using a database of around 1750 complex trajectories (obtained from UCI-KDD data archives) subdivided into five different classes are reported.

HMM-based motion recognition system using segmented PCA

In this paper, we propose a novel technique for model-based recognition of complex object motion trajectories using hidden Markov models (HMM). We build our models on principal component analysis (PCA)-based representation of trajectories after segmenting them into small units of perceptually similar pieces of motions. These subtrajectories are then grouped using spectral clustering to decide on the number of states for each HMM representing a class of object motion. The hidden states of the HMMs are represented by Gaussian mixtures (GMs). This way the HMM topology as well as the parameters are automatically derived from the training data in a fully unsupervised sense. Experiments are performed on two data sets; the ASL data set (from UCIs KDD archives) consists of 207 trajectories depicting signs for three words, from Australian Sign Language (ASL); the HJSL data set contains 108 trajectories from sports videos. Our experiments yield an accuracy of 90+% performing much better than existing approaches.

Compressed Domain Video Object Segmentation

We present a compressed domain video object segmentation method for the MPEG encoded video sequences. For a fraction of the raw domain analysis, compressed domain segmentation provides the essential a priori information to many vision tasks from surveillance to transcoding that require fast processing of large volumes of data where pixel-resolution boundary extraction is not required. Our method generates accurate segmentation maps in block resolution at hierarchically varying object levels, which empowers application to determine the most pertinent partition of images. It exploits the block structure of the compressed video to minimize the amount of data to be processed. All the available motion flow within a group of pictures is projected onto a single layer, which also consists of the frequency decomposition of color pattern. Then, by starting from the blocks where the spatial energy is small, it expands homogeneous regions while automatically adapting local similarity criteria. We also formulate an alternative solution that applies a kernel-based clustering where separate spatial, transform, and motion kernels are used to establish the affinity. We show that both region expansion and mean shift produce similar results as the computationally expensive raw domain segmentation. Finally, a binary clustering iteratively merges the most similar regions to generate a hierarchical partition tree.

A hybrid system for affine-invariant trajectory retrieval

This paper studies efficient feature spaces for content based indexing and retrieval of object motion trajectories. Taking object trajectory data as input, we first investigate highly compact affine invariant feature spaces based on Fourier Descriptors (FD) and Principal Component Analysis (PCA) techniques. Based on these feature spaces, we then develop a hybrid content based indexing and retrieval system that employs a two-stage matching scheme. The first stage uses affine-invariant Fourier Descriptor (FD) for indexing and retrieval. Top few results from this stage along with the original query are then posed to the second stage of the matching system that employs Principal Component Analysis (PCA) for fast retrieval. We compare our systems performance with two other approaches borrowed from 2-D shape representation in image analysis. For quantitative analysis of the system performance, we report query results in terms of precision-recall metrics

Automatic Object Trajectory-Based Motion Recognition Using Gaussian Mixture Models

In this paper, we propose a novel technique for model-based recognition of complex object motion trajectories using Gaussian mixture models (GMM). We build our models on principal component analysis (PCA)-based representation of trajectories after segmenting them into small units of perceptually similar pieces of motions. These subtrajectories are then fitted with automatically learnt mixture of Gaussians to estimate the underlying class probability distribution. Experiments are performed on two data sets; the ASL data set (from UCIs KDD archives) consists of 207 trajectories depicting signs for three words, from Australian sign language (ASL); the HJSL data set contains 108 trajectories from sports videos. Our experiments yield an accuracy of 85+% performing much better than existing approaches

Event Analysis Based on Multiple Interactive Motion Trajectories

Motion information is regarded as one of the most important cues for developing semantics in video data. Yet it is extremely challenging to build semantics of video clips particularly when it involves interactive motion of multiple objects. Most of the existing research has focused on capturing and modelling the motion of each object individually thus loosing interaction information. Such approaches yield low precision-recall ratios and limited indexing and retrieval performances. This paper presents a novel framework for compact representation of multi-object motion trajectories. Three efficient multi-trajectory indexing and retrieval algorithms based on multilinear algebraic representations are proposed. These include: (i) geometrical multiple-trajectory indexing and retrieval (GMIR), (ii) unfolded multiple-trajectory indexing and retrieval (UMIR), and (iii) concentrated multiple-trajectory indexing and retrieval (CMIR). The proposed tensor-based representations not only remarkably reduce the dimensionality of the indexing space but also enable the realization of fast retrieval systems. The proposed representations and algorithms can be robustly applied to both full and partial (segmented) multiple motion trajectories with varying number of objects, trajectory lengths, and sampling rates. The proposed algorithms have been implemented and evaluated using real video datasets. Simulation results demonstrate that the CMIR algorithm provides superior precision-recall metrics, and smaller query processing time compared to the other approaches.

A motion trajectory based video retrieval system using parallel adaptive self organizing maps

We present a novel motion trajectory based video retrieval system using LAMSTAR-based adaptive self organizing maps (PASOMs) in this paper. The trajectories are extracted from video by a robust tracker. To reduce the high dimension of motion trajectories, we first decompose each trajectory into sub-trajectories by using a maximum acceleration based approach. Each subtrajectory is then modeled and coded by two different methods, polynomial curving fitting and independent component analysis. To fuse the different features of subtrajectories for more efficient and flexible retrieval, we use PASOMs as the searching tool. Experimental results show the superior performance of the proposed approach for video retrieval comparing with prior approaches.