Shengrong Gong

Object Detection in Dynamic Scenes Based on Codebook with Superpixels

This paper proposed a new object detection method based on codebook with super pixels (CBSP-OD), which overcomes large memory requirements, calculated quantity problems and code words created are not exact in dynamic scenes. The CBSP-OD algorithm converts pixels from RGB space to HSL space and cluster pixels into super pixels. In this way, we just use L component as light value to reduce computational complexity and enforce robust to light. CBSP-OD use super pixel instead of single pixel to build background model, in which consider spatial consistency well and create code words more exactly. And then, build code words on every super pixel. In this way, large memory requirements and calculated quantity problems can be avoided. Experiments in the dynamic scenes demonstrate the proposed method outperform recent state-of the-art methods.

Trajectory-pooled Spatial-temporal Architecture of Deep Convolutional Neural Networks for Video Event Detection

Nowadays content-based video event detection faces great challenges due to complex scenes and blurred actions in surveillance videos. To alleviate these challenges, we propose a novel spatial-temporal architecture of deep convolutional neural networks for this task. By taking advantage of spatial-temporal information, we fine-tune two-stream networks, and then, fuse spatial and temporal features at convolution layers using a 2D pooling fusion method to enforce the consistence of spatial-temporal information. Based on the two-stream networks and spatial-temporal layer, a triple-channel model is obtained. Furthermore, we implement trajectory-constrained pooling to deep features and hand-crafted features to combine their merits. A fusion method on triple-channel yields the final detection result. The experiments on two benchmark surveillance video data sets including VIRAT 1.0 and VIRAT 2.0, which involve a suit of challenging events, such as person loading an object to a vehicle or person opening a vehicle trunk, manifest that the proposed method can achieve superior performance compared with the state-of-the-art methods on these event benchmarks.

Improved algorithm for Zernike moments

The Zernike moments can achieve high accuracy and strong robustness for the classification and retrieval of images, but involve huge amount of computation caused by its complex definition. It has limited its exploitation in online real-time applications or big data processing. So researches on how to improve the computation speed of Zernike moments are carried out. One of the existing high-accuracy algorithms for Zernike moments, which is called ZMGM algorithm, treats Zernike moments as the linear combination of geometric moments. Based on the ZMGM algorithm, we make two accelerating improvements and propose a fast algorithm. Firstly, a simplified linear combination is achieved by merging all the terms corresponding to the same geometric moment. So that the multiplication times is reduced. In this case, combined coefficients can be separated, pre-computed and stored for further computation of Zernike moments. Secondly, to speed up the computation of combined coefficients, a fast algorithm for the coefficient matrix of Zernike radial polynomials is proposed. The elements of this matrix are the main components of combined coefficients. Complexity analysis and numerical experiments show that, compared with the ZMGM algorithm, our proposed algorithm can significantly reduce the complexity and improve the computation speed. The optimization effect becomes more obvious as the order increases.

Person re-identification by enhanced local maximal occurrence representation and generalized similarity metric learning

Abstract To solve the challenging person re-identification problem, great efforts have been devoted to feature representation and metric learning. However, existing feature extractors are either stripe-based or dense-block-based, the fine details and coarse appearance are not well integrated. What is more, the metrics are generally learned independently from distance view or bilinear similarity view. Few works have exploited the mutual complementary effects of their combination. To address these issues, we propose a new feature representation termed enhanced Local Maximal Occurrence (eLOMO) which fuses a new overlapping-stripe-based descriptor with the Local Maximal Occurrence (LOMO) extracted from dense blocks. Such integration makes eLOMO resemble the coarse-to-fine recognition mechanism of human vision system, thus it can provide a more discriminative descriptor for re-identification. Besides, we show the advantages of learning generalized similarity by combining the Mahalanobis distance and bilinear similarity together. Specifically, we derive a logistic metric learning method to jointly learn a distance metric and a bilinear similarity metric, which exploits both the distance and angle information from training data. Taking advantage of learning in the intra-class subspace, the proposed method can be solved efficiently by coordinate descent optimization. Experiments on four challenging datasets including VIPeR, PRID450S, QMUL GRID, and CUHK01, show that the proposed method outperforms the state-of-the-art approaches significantly.

Person re-identification by kernel null space marginal Fisher analysis

Abstract Distance metric learning has been widely applied for person re-identification. However, the typical Small Sample Size (SSS) problem, which is induced by high dimensional feature and limited training samples in most re-identification datasets, may lead to a sub-optimal metric. In this work, we propose to embed samples into a discriminative null space based on Marginal Fisher Analysis (MFA) to overcome the SSS problem. In such a null space, multiple images of the same pedestrian are collapsed to a single point, resulting the extreme Marginal Fisher Criterion. We theoretically analyze the null space and derive its closed-form solution which can be computed very efficiently. To deal with the heavy storage burden in computation, we further extend the proposed method to kernel version, which is called Kernel Null Space Marginal Fisher Analysis (KNSMFA). Experiments on four challenging person re-identification datasets show that KNSMFA uniformly outperforms state-of-the-art approaches.

Large margin relative distance learning for person re-identification

Distance metric learning has achieved great success in person re-identification. Most existing methods that learn metrics from pairwise constraints suffer the problem of imbalanced data. In this study, the authors present a large margin relative distance learning (LMRDL) method which learns the metric from triplet constraints, so that the problem of imbalanced sample pairs can be bypassed. Different from existing triplet-based methods, LMRDL employs an improved triplet loss that enforces penalisation on the triplets with minimal inter-class distance, and this leads to a more stringent constraint to guide the learning. To suppress the large variations of pedestrians appearance in different camera views, the authors propose to learn the metric over the intra-class subspace. The proposed method is formulated as a logistic metric learning problem with positive semi-definite constraint, and the authors derive an efficient optimisation scheme to solve it based on the accelerated proximal gradient approach. Experimental results show that the proposed method achieves state-of-the-art performance on three challenging datasets (VIPeR, PRID450S, and GRID).

Learning topic of dynamic scene using belief propagation and weighted visual words approach

In this paper, we are tackling the problem of distinguishing scenes, including static and dynamic scenes. We propose a framework of scene recognition, based on bag of visual words and topic model. We achieve the task using the topic model by belief propagation (TMBP), which belongs to the family of the latent Dirichlet allocation model. We also extend the TMBP model, called as the knowledge TMBP model, by introducing the prior information of visual words and scenes. Experimental results on the static and dynamic scenes demonstrated that our proposed framework is effective and efficient. The scene semantics can be obtained from two levels of visual words and topics in our framework. Our result significantly outperforms the others using low-level visual features, such as spatial, temporal and spatiotemporal features.

Robust Dynamic Background Model with Adaptive Region Based on T2FS and GMM

For many tracking and surveillance applications, Gaussian mixture model GMM provides an effective mean to segment the foreground from background. Though, because of insufficient and noisy data in complex dynamic scenes, the estimated parameters of the GMM, which are based on the assumption that the pixel process meets multi-modal Gaussian distribution, may not accurately reflect the underlying distribution of the observations. And the existing block-based GMM BGMM method may be able to segment only rough foreground objects with time-consuming calculations. To solve these difficulties, this paper proposes to use type-2 fuzzy sets T2FSs to handle GMMs uncertain parameters T2GMM. Furthermore, this paper also introduces a novel representation of contextual spatial information including the color, edge and texture features for each block which is faster and almost lossless T2BGMM. Experimental results demonstrate the efficiency of the proposed methods.

Pedestrian Recognition in Aerial Video Using Saliency and Multi-Features

uf020 Abstract—Pedestrian recognition in aerial video is a challenge problem for the problem of low resolution, camera movement and targets blurred detail in aerial video. This paper proposes weighted region matching algorithm with Kalman filter, Multi-features fusion model and saliency segmentation (KMFS-WRM) to detect and recognize pedestrian. The KMFS-WRM algorithm first uses Kalman filter algorithm to mark candidates region, which can avoid the problem of selecting candidates under supervision. Then we proposed the fusion algorithm of multi-feature, including HOG, LBP and SIFT features, namely HLS model to detect the pedestrian in aerial video. Our proposed detection method is robust for whether the camera is moving. And instructing human percept and concept, we segment the pedestrians in marked region using Context-Aware saliency detection algorithm that proposed by Goferman et al. and revised the segmentation results by HST model (Head Shoulder and Torso) and AAM model (Active Appearance Model) to obtain the candidates set. Last the matching of voter and candidates set using weighted region matching algorithm. Experimental results in complex aerial video demonstrated that our KMFS-WRM algorithm not only cuts down calculated complexity, but also improves adaptive and real-time ability. Moreover proposed method outperforms recent state-of-the-art methods.