Huiwen Guo

Spatio-temporal context analysis within video volumes for anomalous-event detection and localization

In this paper, we propose an anomaly-detection approach applied for video surveillance in crowded scenes. This approach is an unsupervised statistical learning framework based on analysis of spatio-temporal video-volume configuration within video cubes. It learns global activity patterns and local salient behavior patterns via clustering and sparse coding, respectively. Upon the composition-pattern dictionary learned from normal behavior, a sparse reconstruction cost criterion is designed to detect anomalies that occur in video both globally and locally. In addition, a multiple scale analysis is employed for obtaining accurate anomaly localization, considering scale variations of abnormal events. This approach is verified on publically available anomaly-detection datasets and compared with other existing work. The experiment results demonstrate that it not only detects various anomalies more efficiently, but also locates anomalous regions more accurately.

Anomaly detection and localization in crowded scenes using short-term trajectories

In this paper we present a method to detect and localize abnormal events in crowded scene. Most existing methods use the patch of optical flow or human tracking based trajectory as representation for crowd motion, which inevitably suffer from noises. Instead, we propose the employment of a new and efficient feature, short-term trajectory, which represent the motion of the visible and constant part of human body that move consistently, for modeling the complicated crowded scene. To extract the short-term trajectory, 3D mean-shift is firstly used to smooth the video frames and 3D seed filling algorithm is performed. In order to detect the abnormal events, all short-term trajectories are treated as point set and mapped into the image plane to obtain probability distribution of normalcy for every pixel. A cumulative energy is calculated based on these probability distributions to identify and localize the abnormal event. Experiments are conducted on known crowd data sets, and the results show that our method can achieve high accuracy in anomaly detection as well as effectiveness in anomalies localization.

Quaternion discrete cosine transformation signature analysis in crowd scenes for abnormal event detection

In this paper, an abnormal event detection approach inspired by the saliency attention mechanism of human visual system is presented. Conventionally, statistics-based methods suffer from visual scale, complexity of normal events and insufficiency of training data, for the reason that a normal behavior model established from normal video data is used to detect unusual behaviors with an assumption that anomalies are events with rare appearance. Instead, we make the assumption that anomalies are events that attract human attention. Temporal and spatial anomaly saliency are considered consistently by representing the pixel value in each frame as a quaternion, with weighted components that composed of intensity, contour, motion-speed and motion-direction feature. For each quaternion frame, Quaternion Discrete Cosine Transformation (QDCT) and signature operation are applied. The spatio-temporal anomaly saliency map is developed by inverse QDCT and Gaussian smoothing. By multi-scale analyzing, abnormal events appear at those areas with high saliency score. Experiments on typical datasets show that our method can achieve high accuracy results.

Anomaly Detection in Video Surveillance via Gaussian Process

In this paper, we propose a new approach for anomaly detection in video surveillance. This approach is based on a nonparametric Bayesian regression model built upon Gaussian process priors. It esta...

Multi-stream deep networks for human action classification with sequential tensor decomposition

Abstract Effective spatial-temporal representation of motion information is crucial to human action classification. In spite of the attempt of most existing methods capturing spatial-temporal structure and learning motion representations with deep neural networks, such representations are failing to model action at their full temporal extent. To address this problem, this paper proposes a global motion representation by using sequential low-rank tensor decomposition. Specifically, we model an action sequence as a third-order tensor with spatiotemporal structure. Then, by using low-rank tensor decomposition, partial motion of objects in global context were preserved which will be feeding into deep architecture to automatically learning global-term motion features. To simultaneously exploit static spatial features, short-term motion and global-term motion in the video, we describe a multi-stream framework with recurrent convolutional architectures which is end-to-end trainable. Gated Recurrent Unit (GRU) is used as our recurrent unit which have fewer parameters than Long Short-Term Memory (LSTM). Extensive experiments were conducted on two challenging dataset: HMDB51 and UCF101. Experimental results show that our method achieves state-of-the-art performance on the HMDB51 dataset, and is comparable to the state-of-the-art methods on the UCF101 dataset.

Calibration of non-overlapping cameras based on a mobile robot

This paper presents a calibration framework for calibrating the pose of two cameras with non-overlapping region with the help of a mobile robot. Firstly, intrinsic parameters are calibrated separately by using camera calibration toolbox for MATLAB. To establish the position relationship between the two fixed cameras, the movement of mobile robot at two different sites is obtained for utilizing. In our model, the unknown parameters include the pose relationship between two cameras and the poses of the calibration marker, and the aim is to minimize the errors between the detected and re-projection positions of the chess-board corners. Based on least square estimation (LSE) method, we get the optimal solution. The proposed method is verified on the platform of Pioneer patrol-bot, and the results demonstrate its effectiveness, allowing the use of collaborative calibration and computing the topology of a multi-camera system.

Multi-scale analysis of contextual information within spatio-temporal video volumes for anomaly detection

In this paper, we present a novel approach for video anomaly detection in crowded scenes. The proposed approach detects anomalies based on the contextual information analysis within spatio-temporal video volume. Around each pixel, spatio-temporal volumes are built and clustered to construct the activity pattern codebook. Then, the composition information of the volumes within a large spatiotemporal window is described via a dictionary learned by sparse representation. Furthermore, multi-scale analysis is employed to adapt the size change of abnormal events. Finally, the sparse reconstruction cost is designed to evaluate the abnormal level of an input motion pattern. We demonstrate the efficiency of the proposed method on the existing public available anomaly-detection datasets and the performance comparasion with three existing methods validates that the proposed method detects anomalies more accurately.

Detecting action tubes via spatial action estimation and temporal path inference

Abstract In this paper, we address the problem of action detection in unconstrained video clips. Our approach starts from action detection on object proposals at each frame, then aggregates the frame-level detection results belonging to the same actor across the whole video via linking, associating, and tracking to generate action tubes that are spatially compact and temporally continuous. To achieve the target, a novel action detection model with two-stream architecture is firstly proposed, which utilizes the fused feature from both appearance and motion cues and can be trained end-to-end. Then, the association of the action paths is formulated as a maximum set coverage problem with the results of action detection as a priori. We utilize an incremental search algorithm to obtain all the action proposals at one-pass operation with great efficiency, especially while dealing with the video of long duration or with multiple action instances. Finally, a tracking-by-detection scheme is designed to further refine the generated action paths. Extensive experiments on three validation datasets, UCF-Sports, UCF-101 and J-HMDB, show that the proposed approach advances state-of-the-art action detection performance in terms of both accuracy and proposal quality.

Real-time running event detection via a community patrol robot

Security surveillance is an important application for patrol robots. In this article, a real-time running event detection method is proposed for the community patrol robot. Although sliding window-based approaches have been quite successful in detecting objects in images, directly extending them to real-time object detection in video is not simple. This is due to the huge samples and diversity of object appearances with multivisual view and scale. To address these limitations, first, a simple and effective spatial–temporal filtering-based approach is proposed to obtain moving object proposals in each frame; then, two-stream convolutional networks fusion architecture is introduced to best take advantage of the spatial–temporal information from the proposal. The algorithm is applied on PatrolBot in community environments and runs at 15 fps on a consumer laptop. Two benchmark data sets (the Kungliga Tekniska Högskolan [KTH] data set and Nanyang Technological University [NTU] running data set) were also used to compare results with previous works. Experimental results show higher accuracy and lower detection error rate in the proposed method.

Robust localization system for an autonomous mower

This paper presents a robust vision-based localization system for an autonomous mower, which is significant for both the meadow map building and the successful area covering. Instead of setting the monocular camera toward the scene, which suffers from the disturbance of moving objects, less mark points or variation of illumination, we equip the camera toward the ground with constant illumination compensation. To achieve the localization of the mower, point features are extracted and matched between pairs of frames. Motion is incremental obtained by calculate the rotation and translation transformation of matched feature point pairs. As the angle accumulated error has greater contribution to the location error, angular acceleration sensor is adopted to compensate the angle error especially in the steep turning case. Experiments on meadow with our mowers demonstrate the robustness of our localization system.