Saira Saleem Pathan

Spatio-temporal feature extraction-based hand gesture recognition for isolated American Sign Language and Arabic numbers

This paper proposes a system to recognize isolated American Sign Language and Arabic numbers in real-time from stereo color image sequences using Hidden Markov Models (HMMs). Our system is based on three main stages; preprocessing, feature extraction and classification. In preprocessing stage, color and 3D depth map are used to detect and track the hand. The second stage, 3D combined features of location, orientation and velocity with respected to Cartesian and Polar systems are used. Additionally, k-means clustering is employed for HMMs code-word. In the final stage, the hand gesture path is recognized using Left-Right Banded topology (LRB) in conjunction Viterbi path. Experimental results demonstrate that, our system can successfully recognize isolated hand gestures with 98.33% recognition rate.

Crowd behavior detection by statistical modeling of motion patterns

The governing behaviors of individuals in crowded places offer unique and difficult challenges, and limit the scope of conventional surveillance systems. In this paper, we investigate the crowd behaviors and localize the anomalies due to individuals abrupt dissipation. The novelty of the proposed approach can be described in three aspects. First, we introduce block-clips by sectioning the video segments into non-overlapping spatio-temporal patches to marginalize the arbitrarily complicated and dense flow field. Second, we treat the flow field in each block-clip as 2d distribution of samples and mixtures of Gaussian is used to parameterize it keeping generality of flow field intact. K-means algorithm is employed to initialize the mixture model and is followed by Expectation Maximization for optimization. These mixtures of Gaussian result in the distinct flow patterns precisely a sequence of dynamic patterns for each block-clip. Third, a bank of Conditional Random Field model is employed one for each block clip and is learned from the sequence of dynamic patterns and classifies each block-clip as normal and abnormal. We conduct experiment on two challenging benchmark crowd datasets PETS 2009 and University of Minnesota and results show that our method achieves higher recognition rates in detecting specific and overall crowd behaviors. In addition, the proposed approach shows dominating performance during the comparative analysis with similar approaches in crowd behavior detection.

Intelligent feature-guided multi-object tracking using Kalman filter

Kalman filtering, a recursive state estimation filter is a robust method for tracking objects. It has been proven that Kalman filter gives a good estimation when tested on various tracking systems. However, unsatisfying tracking results may be produced due to different real-time conditions. These conditions include: inter-object occulusion and separation which are observed when objects are being tracked in real-time. Thus, it is challenging to handle for the classical Kalman filter. In this paper, we proposed an idea of intelligent feature-guided tracking using Kalman filtering. A new method is developed named Correlation-Weighted Histogram Intersection (CWHI), in which correlation weights are applied to Histogram Intersection (HI) method. We focus on multi-object tracking in traffic sequences and our aim is to achieve efficient tracking of multiple moving objects under the confusing situations. The proposed algorithm achieves robust tracking with 97.3% accuracy and 0.07% covariance error in different real-time scenarios.

Incorporating social entropy for crowd behavior detection using SVM

Crowd behavior analysis is a challenging task for computer vision. In this paper, we present a novel approach for crowd behavior analysis and anomaly detection in coherent and incoherent crowded scenes. Two main aspects describe the novelty of the proposed approach: first, modeling the observed flow field in each non-overlapping block through social entropy to measure the concerning uncertainty of underlying field. Each block serves as an independent social system and social entropy determine the optimality criteria. The resulted in distributions of the flow field in respective blocks are accumulated statistically and the flow feature vectors are computed. Second, Support Vector Machines are used to train and classify the flow feature vectors as normal and abnormal. Experiments are conducted on two benchmark datasets PETS 2009 and University of Minnesota to characterize the specific and overall behaviors of crowded scenes. Our experiments show promising results with 95.6% recognition rate for both the normal and abnormal behavior in coherent and incoherent crowded scenes. Additionally, the similar method is tested using flow feature vectors without incorporating social entropy for comparative analysis and the detection results indicate the dominating performance of the proposed approach.

Using conditional random field for crowd behavior analysis

The governing behaviors of individuals in crowded places offer unique and difficult challenges. In this paper, a novel framework is proposed to investigate the crowd behaviors and to localize the anomalous behaviors. Novelty of the proposed approach can be revealed in three aspects. First, we introduce block-clips by sectioning video segments into non-overlapping patches to marginalize the arbitrarily complicated dense flow field. Second, flow field is treated as a 2d distribution of samples in block-clips, which is parameterized by using mixtures of Gaussian keeping the generality intact. The parameters of each Gaussian model, particularly mean values are transformed into a sequence of Gaussian mean densities for each block-clip namely a sequence of latent-words. A bank of Conditional Random Field model is employed, one for each block-clip, which is learned from the sequence of latent-words and classifies each block-clip as normal and abnormal. Experiments are conducted on two challenging benchmark datasets PETS 2009 and University of Minnesota and results show that our method achieves higher accuracy in behavior detection and can effectively localize specific and overall anomalies. Besides, a comparative analysis is presented with similar approaches which demonstrates the dominating performance of our approach.

Multi-object tracking using semantic analysis and Kalman filter

A generic approach for tracking humans and objects under occlusion using semantic analysis is presented. The aim is to exploit knowledge representation schemes, precisely semantic logic where each detected object is represented by a node and the association among the nodes is interpretated as flow paths. Besides, maximum likelihood is computed using our CWHI technique and Bhattacharyya coefficient. These likelihood weights are mapped onto the semantic network to efficiently infer the multiple possibilities of tracking by the manipulation of ldquopropositional logicrdquo at a time window. The logical propositions are built by formularizing facts, semantic rules and constraints associated with tracking. Currently, we are able to handle tracking under normal, occlusion, and split conditions. The experimental results show that the proposed approach enables accurate and reliable tracking by resolving the ambiguities of online data association under occlusions.

Multi-Object Tracking Based on Particle Filter and Data Association in Color Image Sequences

Robust tracking of multi-objects is still challenging in real scenarios such as crowed scenes. In this paper a novel method in color image sequences is proposed for tracking multiple objects in non-cooperative situations. A system of independent particle filters with an adaptive motion model is used which tracks the moving objects under complex situations. Besides, in order to handle the conflicted situations, an integrated data association technique is exploited which adjusts the particle filters accordingly. Results have shown the good performance of the proposed method on various complex-situation image sequences.

Geometric and Optical Flow Based Method for Facial Expression Recognition in Color Image Sequences

This work proposes new static and dynamic based methods for facial expression recognition in stereo image sequences. Computer vision 3-d techniques are applied to determine real world geometric measures and to build a static geometric feature vector. Optical flow based motion detection is also carried out which delivers the dynamic flow feature vector. Support vector machine classification is used to recognize the expression using geometric feature vector while k-nearest neighbor classification is used for flow feature vector. The proposed method achieves robust feature detection and expression classification besides covering the in/out of plane head rotations and back and forth movements. Further, a wide range of human skin color is exploited in the training and the test samples.

Crowd behaviour analysis and anomaly detection by statistical modelling of flow patterns

In this paper, we investigate the crowd behaviours and localise the anomalies due to individuals abrupt dissipation. The novelty of proposed approach is described in three aspects. First, we create the spatio-temporal flow-blocks of the video sequence allowing the marginalisation of arbitrarily flow field. Second, the observed flow field in each flow-block is treated as 2D distribution of samples and mixtures of Gaussian is used to parameterise the flow field. These mixtures of Gaussian result in the distinct representation of flow field named as flow patterns for each flow-block. Third, conditional random field is employed to classify the flow patterns as normal and abnormal for each flow-block. Experiments are conducted on two challenging benchmark datasets PETS 2009 and UMN, and results show that our method achieves higher recognition rates in detecting specific and overall crowd behaviours. In addition, proposed approach shows dominating performance during the comparative analysis with similar approaches.

Integrating statistical and cognitive model for multi-object tracking in realistic scenarios

In this paper, we have addressed a quite researched problem in vision for tracking objects in realistic scenarios containing multifarious situations. We explore cognitive modeling approaches with statistical modeling for tracking objects in contrast to conventional multi-hypothesis and global data association approaches. Our framework comprises of three phases: object detection, integrated cognitive and statistical model, and object tracker. The objects are detected using improved background subtraction with shadow removal technique. Second module is the key to proposed approach and the motivation is to tackle the tracking problem by axiomatizing and reasoning human-tracking abilities with associated weights. An undirected network of detected objects is built in space. Each object contains a unique identity and a data structure of cognitive and statistical attributes whilst satisfying the global constraints of continuity during motion. Consequently, results are linked with Kalman filter based tracker to estimate the trajectories of moving objects. We show that combining cognitive and statistical information gives a straightforward way to interpret and disambiguate the uncertainties due to con icted situations in tracking. The performance of the proposed approach is demonstrated on a set of videos representing various challenges. Besides, quantitative evaluation with annotated ground truth is presented.