Jongmin Yu

Representation Learning, Scene Understanding, and Feature Fusion for Drowsiness Detection

We propose a novel drowsiness detection method based on 3D-Deep Convolutional Neural Network (3D-DCNN). We design a learning architecture for the drowsiness detection, which consists of three building blocks for representation learning, scene understanding, and feature fusion. In this framework, the model generates a spatio-temporal representation from multiple consecutive frames and analyze the scene conditions which are defined as head, eye, and mouth movements. The result of analysis from the scene condition understanding model is used to auxiliary information for the drowsiness detection. Then the method subsequently generates fusion features using the spatio-temporal representation and the results of the classification of scene conditions. By using the fusion features, we show that the proposed method can boost the performance of drowsiness detection. The proposed method demonstrates with the NTHU Drowsy Driver Detection (NTHU-DDD) video dataset.

Feature flow-based abnormal event detection using a scene-adaptive cuboid determination method

As closed circuit television which had been used only for surveillance or identification has developed rapidly the research on intelligent surveillance systems is getting increased interest. Above all, abnormal event detection is becoming an essential part of surveillance systems by detecting or identifying actions or situations which are not commonly occurred in general. In this work, we propose an abnormal event detection method using trajectory modeling with an automatic scene-adaptive cuboid determination scheme. First, we constructed a human appearance model to determine the human size without using any detection method. Then, HOG feature extracted from human images which is the predetermined input is used to construct a human appearance model. We applied a background subtraction to input datasets and then compared HOG feature extracted from the bounding box of the foreground with the human appearance model. The human size is determined by the size of the foreground bounding box with the highest similarity. With the ratio obtained through the experiments, the cuboid size is calculated according to the human size and histogram of oriented tracklets model is constructed by the cuboid size. We used the UCSD dataset to validate the proposed approach. From the experimental results, we verified the significance of the proposed AED method adopting the automatic scene-adaptive cuboid size determination scheme.

An incremental learning approach for restricted boltzmann machines

Determination of model complexity is a challenging issue to solve computer vision problems using restricted boltzmann machines (RBMs). Many algorithms for feature learning depend on cross-validation or empirical methods to optimize the number of features. In this work, we propose an learning algorithm to find the optimal model complexity for the RBMs by incrementing the hidden layer. The proposed algorithm is composed of two processes: 1) determining incrementation necessity of neurons and 2) computing the number of additional features for the increment. Specifically, the proposed algorithm uses a normalized reconstruction error in order to determine incrementation necessity and prevent unnecessary increment for the number of features during training. Our experimental results demonstrated that the proposed algorithm converges to the optimal number of features in a single layer RBMs. In the classification results, our model could outperform the non-incremental RBM.

Heart beat detection using estimated regular intervals from electrocardiography and blood pressure

In this paper, we propose a robust heart beat detection method using signals representing cardiac activity directly in complex noisy environment. Proposed method extracts and matches characteristic points which provide essential information on estimating heart beats. The proposed method was applied to a data set of the CinC/Challenge 2014, and achieved a sensitivity of 99.65% and a positive predictivity of 99.64%.

Joint representation learning of appearance and motion for abnormal event detection

In this paper, we propose a joint learning of spatio-temporal representation based on 3D deep convolutional neural network for simultaneous representation of appearance and motion information in 3D volumes which are extracted from the multiple consecutive frames, and an end-to-end learning framework to detect abnormal events in surveillance scenes. By using the joint learning approach, the proposed framework can detect various abnormal events which can appear with diverse motion and appearance patterns. The proposed framework detects abnormal events in each volume by analyzing the spatio-temporal representation trained by the joint learning method. This volume-level event detection approach makes it possible to localize an abnormal event. We verify the proposed joint learning and the framework on the publicly available abnormal event datasets containing UMN dataset, UCSD dataset, and subway dataset, by comparing it with existing state-of-the-art methods. The experimental results demonstrate that the proposed joint learning and event detection method not only detect various abnormal events more efficiently but also localize anomalous regions more accurately.

Learning Object-Level Spatio-Temporal Representation for Abnormal Event Detection

In this paper, we propose an approach for abnormal event detection, using the object-level spatio-temporal representation. Our approach detects an abnormal event in complex scenes which contain objects classified in various categories. We compute the object-level 3D Region-of-interest (3D RoI) and extract object-level 3D volume. Then, the object-level 3D volume is inputted to 3D deep convolutional neural network (3D-DCNN) for detecting the abnormal event. In the experiments, we compare our method with several methods on our experimental dataset.

Gaussian-Poisson mixture model for anomaly detection of crowd behaviour

This paper presents a Gaussian-Poisson mixture model (GPMM) which can reflect a frequency of event occurrence, for detecting anomaly of crowd behaviours. GPMM exploits the complementary information of both a statistics of crowd behaviour patterns and a count of the observed behaviour, and we learn the statistics of normal crowd behaviours for behaviours that occur frequently in the past by placing different weights, depending on the frequency occur. GPMM implicitly accounts for the motion patterns and the count of occurrence. The dense optical flow and an interactive force are used to represent a scene. We demonstrate the proposed method on a publicly available dataset, and the experimental results show that the proposed method could achieves competitive performances with respect to state-of-the-art approaches.

Abnormal Event Detection using Scene Partitioning by Regional Activity Pattern Analysis

This paper presents a method for detecting abnormal events based on scene partitioning. To develop the practical application for abnormal event detection, the proposed method focuses on handling various activity patterns caused by diverse moving objects and geometric conditions such as camera angles and distances between the camera and objects. We divide a frame into several blocks and group the blocks with similar motion patterns. Then, the proposed method constructs normal-activity models for local regions by using the grouped blocks. These regional models allow to detect unusual activities in complex surveillance scenes by considering specific regional local activity patterns. We construct a new dataset called GIST Youtube dataset, using the Youtube videos to evaluate performance in practical scenes. In the experiments, we used the dataset of the university of minnesota, and our dataset. From the experimental study, we verified that the proposed method is efficient in the complex scenes which contain the various activity patterns.