Alok Kumar Singh Kushwaha

Multi-view human activity recognition based on silhouette and uniform rotation invariant local binary patterns

This paper addresses the problem of silhouette-based human activity recognition. Most of the previous work on silhouette based human activity recognition focus on recognition from a single view and ignores the issue of view invariance. In this paper, a system framework has been presented to recognize a view invariant human activity recognition approach that uses both contour-based pose features from silhouettes and uniform rotation local binary patterns for view invariant activity representation. The framework is composed of three consecutive modules: (1) detecting and locating people by background subtraction, (2) combined scale invariant contour-based pose features from silhouettes and uniform rotation invariant local binary patterns (LBP) are extracted, and (3) finally classifying activities of people by Multiclass Support vector machine (SVM) classifier. The rotation invariant nature of uniform LBP provides view invariant recognition of multi-view human activities. We have tested our approach successfully in the indoor and outdoor environment results on four multi-view datasets namely: our own view point dataset, VideoWeb Multi-view dataset [28], i3DPost multi-view dataset [29], and WVU multi-view human action recognition dataset [30]. The experimental results show that the proposed method of multi-view human activity recognition is robust, flexible and efficient.

An approach towards wavelet transform based multiclass object classification

Object classification is an important problem in computer vision, in which multiclass object classification is more difficult one in comparison to single class object classification. In this paper, we proposed a new method for multiclass object classification based on discrete wavelet transform. We have used discrete wavelet transform coefficients as a feature of object, because of its multi-resolution property. We have used multiclass support vector machine as a classifier for classification of objects. The proposed method has been tested on own dataset prepared by authors of this paper. We have tested the proposed method on multiple levels of discrete wavelet transform. Quantitative evaluation results shows that the proposed method gives better performance for multiclass object classification at higher level of discrete wavelet transform and other state-of-the-art methods.

Automatic moving object segmentation methods under varying illumination conditions for video data: comparative study, and an improved method

In recent past, many moving object segmentation methods under varying lighting changes have been proposed in literature and each of them has their own benefits and limitations. The various methods available in literature for moving object segmentation may be broadly classified into four categories i.e., moving object segmentation methods based on (i) motion information (ii) motion and spatial information (iii) learning (iv) and change detection. The objective of this paper is two-fold i.e., firstly, this paper presents a comprehensive comparative study of various classical as well as state-of-the art methods for moving object segmentation under varying illumination conditions under each of the above mentioned four categories and secondly this paper presents an improved approximation filter based method in complex wavelet domain and its comparison with other methods under four categories mentioned as above. The proposed approach consist of seven steps applied on given video frames which include: wavelet decomposition of frames using Daubechies complex wavelet transform; use of improved approximate median filter on detail co-efficient (LH, HL, HH); use of background modeling on approximate co-efficient (LL sub-band); soft thresholding for noise removal; strong edge detection; inverse wavelet transformation for reconstruction; and finally using closing morphology operator. The qualitative and quantitative comparative study of the various methods under four categories as well as the proposed method is presented for six different datasets. The merits, demerits, and efficacy of each of the methods under consideration have been examined. The extensive experimental comparative analysis on six different challenging benchmark data sets demonstrate that proposed method is performing better to other state-of-the-art moving object segmentation methods and is well capable of dealing with various limitations of existing methods.

Multiview human activity recognition system based on spatiotemporal template for video surveillance system

Abstract. An efficient view invariant framework for the recognition of human activities from an input video sequence is presented. The proposed framework is composed of three consecutive modules: (i) detect and locate people by background subtraction, (ii) view invariant spatiotemporal template creation for different activities, (iii) and finally, template matching is performed for view invariant activity recognition. The foreground objects present in a scene are extracted using change detection and background modeling. The view invariant templates are constructed using the motion history images and object shape information for different human activities in a video sequence. For matching the spatiotemporal templates for various activities, the moment invariants and Mahalanobis distance are used. The proposed approach is tested successfully on our own viewpoint dataset, KTH action recognition dataset, i3DPost multiview dataset, MSR viewpoint action dataset, VideoWeb multiview dataset, and WVU multiview human action recognition dataset. From the experimental results and analysis over the chosen datasets, it is observed that the proposed framework is robust, flexible, and efficient with respect to multiple views activity recognition, scale, and phase variations.

Performance evaluation of various moving object segmentation techniques for intelligent video surveillance system

Moving object segmentation is an essential process for many computer vision algorithms. Many different methods have been proposed over the recent years but expert can be confused about their benefits and limitations. In this paper, review and comparative studyof various moving object segmentation approachesis presented in terms of qualitative and quantitative performances with the aim of pointing out their strengths and weaknesses, and suggesting new research directions. For evaluation and analysis purposes, the various standard spatial domain methods include as proposed by McFarlane and Schofield [13], Kim et al [18], Oliver et al [27], Liu et al [9], Stauffer and Grimsons [15], Zivkovic [12], Lo and Velastin [25], Cucchiara et al. [26], Bradski [24], and Wren et al. [16]. For quantitative evaluation of these standard methods the various metrics used are RFAM (relative foreground area measure), MP (misclassification penalty), RPM (relative position based measure), and NCC (normalized cross correlation). The strengths and weaknesses of various segmentation approaches are discussed. From the results obtained, it is observed that codebook based segmentation method performs better in comparison to other methods in consideration.

Rule based human activity recognition for surveillance system

This paper presents a framework for classification and recognition of human activities in complex motion. We propose a template matching based method to classify the objects and a rule-based approach to recognize human activities. First, moving objects are detected and their silhouettes are generated in each frame. Second, template matching based approach is used to classify the generated silhouette and then a rule based classifier is applied to classify human activities such as running, walking, bending, boxing and jogging etc. The experimental results show that the system can recognize seven types of primitive actions with high accuracy.

Depth based enlarged temporal dimension of 3D deep convolutional network for activity recognition

An activity takes many seconds to complete which makes it a spatiotemporal structure. Many contemporary techniques tried to learn activity representation using convolutional neural network from such structures to recognize activities from videos. Nevertheless, these representation failed to learn complete activity because they utilized very few video frames for learning. In this work we use raw depth sequences considering its capabilities to record geometric information of objects and apply proposed enlarged time dimension convolution to learn features. Due to these properties, depth sequences are more discriminatory and insensitive to lighting changes as compared to RGB video. As we use raw depth data, time to do preprocessing are also saved. The 3 dimensional space-time filters have been used over increased time dimension for feature learning. Experimental results demonstrated that by lengthening the temporal resolution over raw depth data, accuracy of activity recognition has been improved significantly. We also studied the impact of different spatial resolution and conclude that accuracy stabilizes at larger spatial sizes. We shows the state-of-the-art results on three human activity recognition depth datasets: NTU-RGB + D, MSRAction3D and MSRDailyActivity3D.

A Framework of Moving Object Segmentation in Maritime Surveillance Inside a Dynamic Background

Maritime surveillance represents a challenging scenario for moving object segmentation due to the complexity of the observed scenes. The waves on the water surface, boat wakes, and weather issues contribute to generate a highly dynamic background. Moving object segmentation using change detection under maritime environment is a challenging problem for the maritime surveillance system. To address these issues, a fast and robust moving object segmentation approach is proposed which consist of seven steps applied on given video frames which include wavelet decomposition of frames using complex wavelet transform; use of change detection on detail coefficients (LH, HL, HH); use of background modeling on approximate co-efficient (LL sub-band); cast shadow suppression; strong edge detection; inverse wavelet transformation for reconstruction; and finally using closing morphology operator. For dynamic background modeling in the water surface, we have used background registration, background difference, and background difference mask in the complex wavelet domain. For shadow detection and suppression problem in water surface, we exploit the high frequency sub-band in the complex wavelet domain. A comparative analysis of the proposed method is presented both qualitatively and quantitatively with other standard methods available in the literature for seven datasets. The various performance measures used for quantitative analysis include relative foreground area measure (RFAM), misclassification penalty (MP), relative position based measure (RPM), normalized cross correlation (NCC), Precision (PR), Recall (RE), shadow detection rate (SDR), shadow discrimination rate, execution time and memory consumption. Experimental results indicate that the proposed method is performing better in comparison to other methods in consideration for all the test cases as well as addresses all the issues effectively.

A framework for moving object segmentation under rapidly changing illumination conditions in complex wavelet domain

Moving object segmentation using change detection in wavelet domain under dynamic background changes is a challenging problem in video surveillance system. There are several literature surveys available in change detection using wavelet domain for moving object segmentation but most of the research work are based on static background changes. Change detection under background changes is a challenging task and it has not been addressed in effectively in literature. To address this issues, a fast and robust moving object segmentation approach is proposed in dynamic background changes which consist of six steps applied on given video frames which include: wavelet decomposition of frames using complex wavelet transform; use of change detection on detail coefficients (LH, HL, HH); use of background modeling on approximate co-efficient (LL sub-band); strong edge detection; inverse wavelet transformation for reconstruction; and finally using closing morphology operator. For dynamic background modeling, we have improved the Gaussian mixture model and use mode value to find the variance of K-Gaussian. A comparative analysis of the proposed method is presented both quantitatively and qualitatively with other standard methods available in the literature. The various performance measure used for quantitative analysis include RFAM, RPM, NCC and MP. From the obtained result, it is observed that proposed approach is performing better in comparison to other methods in consideration.

Framework for dynamic background modeling and shadow suppression for moving object segmentation in complex wavelet domain

Abstract. Moving object segmentation using change detection in wavelet domain under continuous variations of lighting condition is a challenging problem in video surveillance systems. There are several methods proposed in the literature for change detection in wavelet domain for moving object segmentation having static backgrounds, but it has not been addressed effectively for dynamic background changes. The methods proposed in the literature suffer from various problems, such as ghostlike appearance, object shadows, and noise. To deal with these issues, a framework for dynamic background modeling and shadow suppression under rapidly changing illumination conditions for moving object segmentation in complex wavelet domain is proposed. The proposed method consists of eight steps applied on given video frames, which include wavelet decomposition of frame using complex wavelet transform; use of change detection on detail coefficients (LH, HL, and HH), use of improved Gaussian mixture-based dynamic background modeling on approximate coefficient (LL subband); cast shadow suppression; use of soft thresholding for noise removal; strong edge detection; inverse wavelet transformation for reconstruction; and finally using closing morphology operator. A comparative analysis of the proposed method is presented both qualitatively and quantitatively with other standard methods available in the literature for six datasets in terms of various performance measures. Experimental results demonstrate the efficacy of the proposed method.