Yu-Ming Liang

Learning Atomic Human Actions Using Variable-Length Markov Models

Visual analysis of human behavior has generated considerable interest in the field of computer vision because of its wide spectrum of potential applications. Human behavior can be segmented into atomic actions, each of which indicates a basic and complete movement. Learning and recognizing atomic human actions are essential to human behavior analysis. In this paper, we propose a framework for handling this task using variable-length Markov models (VLMMs). The framework is comprised of the following two modules: a posture labeling module and a VLMM atomic action learning and recognition module. First, a posture template selection algorithm, based on a modified shape context matching technique, is developed. The selected posture templates form a codebook that is used to convert input posture sequences into discrete symbol sequences for subsequent processing. Then, the VLMM technique is applied to learn the training symbol sequences of atomic actions. Finally, the constructed VLMMs are transformed into hidden Markov models (HMMs) for recognizing input atomic actions. This approach combines the advantages of the excellent learning function of a VLMM and the fault-tolerant recognition ability of an HMM. Experiments on realistic data demonstrate the efficacy of the proposed system.

Human Object Inpainting Using Manifold Learning-Based Posture Sequence Estimation

We propose a human object inpainting scheme that divides the process into three steps: 1) human posture synthesis; 2) graphical model construction; and 3) posture sequence estimation. Human posture synthesis is used to enrich the number of postures in the database, after which all the postures are used to build a graphical model that can estimate the motion tendency of an object. We also introduce two constraints to confine the motion continuity property. The first constraint limits the maximum search distance if a trajectory in the graphical model is discontinuous, and the second confines the search direction in order to maintain the tendency of an objects motion. We perform both forward and backward predictions to derive local optimal solutions. Then, to compute an overall best solution, we apply the Markov random field model and take the potential trajectory with the maximum total probability as the final result. The proposed posture sequence estimation model can help identify a set of suitable postures from the posture database to restore damaged/missing postures. It can also make a reconstructed motion sequence look continuous.

Human action segmentation and classification based on the Isomap algorithm

Visual analysis of human behavior has attracted a great deal of attention in the field of computer vision because of the wide variety of potential applications. Human behavior can be segmented into atomic actions, each of which indicates a single, basic movement. To reduce human intervention in the analysis of human behavior, unsupervised learning may be more suitable than supervised learning. However, the complex nature of human behavior analysis makes unsupervised learning a challenging task. In this paper, we propose a framework for the unsupervised analysis of human behavior based on manifold learning. First, a pairwise human posture distance matrix is derived from a training action sequence. Then, the isometric feature mapping (Isomap) algorithm is applied to construct a low-dimensional structure from the distance matrix. Consequently, the training action sequence is mapped into a manifold trajectory in the Isomap space. To identify the break points between the trajectories of any two successive atomic actions, we represent the manifold trajectory in the Isomap space as a time series of low-dimensional points. A temporal segmentation technique is then applied to segment the time series into sub series, each of which corresponds to an atomic action. Next, the dynamic time warping (DTW) approach is used to cluster atomic action sequences. Finally, we use the clustering results to learn and classify atomic actions according to the nearest neighbor rule. If the distance between the input sequence and the nearest mean sequence is greater than a given threshold, it is regarded as an unknown atomic action. Experiments conducted on real data demonstrate the effectiveness of the proposed method.

Recognition of blurred license plate images

In video forensics, to recognize objects in low-resolution frames is a commonly seen problem. In this paper, we propose a systematic way to recognize blurred license plate images. Our method only uses one license plate image and character segmentation is not necessary. The process involves three steps. First, using single-character templates, we identify the positions of characters on a license plate and estimate the corresponding character list. Then, the position of a special symbol on the license plate is estimated. Finally, to refine the recognition results, we expand the single-character templates to multiple-character templates. The experiment results demonstrate the efficacy of our method in recognizing characters in blurred license plate images.

A Language Modeling Approach to Atomic Human Action Recognition

Visual analysis of human behavior has generated considerable interest in the field of computer vision because it has a wide spectrum of potential applications. Atomic human action recognition is an important part of a human behavior analysis system. In this paper, we propose a language modeling framework for this task. The framework is comprised of two modules: a posture labeling module, and an atomic action learning and recognition module. A posture template selection algorithm is developed based on a modified shape context matching technique. The posture templates form a codebook that is used to convert input posture sequences into training symbol sequences or recognition symbol sequences. Finally, a variable-length Markov model technique is applied to learn and recognize the input symbol sequences of atomic actions. Experiments on real data demonstrate the efficacy of the proposed system.

Video object inpainting using manifold-based action prediction

This paper presents a novel scheme for object completion in a video. The framework includes three steps: posture synthesis, graphical model construction, and action prediction. In the very beginning, a posture synthesis method is adopted to enrich the number of postures. Then, all postures are used to build a graphical model of object action which can provide possible motion tendency. We define two constraints to confine the motion continuity property. With the two constraints, possible candidates between every two consecutive postures are significantly reduced. Finally, we apply the Markov Random Field model to perform global matching. The proposed approach can effectively maintain the temporal continuity of the reconstructed motion. The advantage of this action prediction strategy is that it can handle the cases such as non-periodic motion or complete occlusion.

Unsupervised analysis of human behavior based on manifold learning

In this paper, we propose a framework for unsupervised analysis of human behavior based on manifold learning. First, a pairwise human posture distance matrix is calculated from a training action sequence. Then, the isometric feature mapping (Isomap) algorithm is applied to construct a low-dimensional structure from the distance matrix. The data points in the Isomap space are consequently represented as a time-series of low-dimensional points. A temporal segmentation technique is then applied to segment the time series into subseries corresponding to atomic actions. Next, a dynamic time warping (DTW) approach is applied for clustering atomic action sequences. Finally, we use the clustering results to learn and classify atomic actions using the nearest neighbor rule. Experiments conducted on real data demonstrate the efficacy of the proposed method.

Linear production game solution to a PTZ camera network

Reconfiguring the PTZ parameters of a camera network is an combinatorial optimization problem and computing the optimal solution is very time consuming. Therefore, existing methods can only provide sub-optimal solutions. In this paper, a nonlinear objective function for better utilizing the cameras to track multiple targets is proposed. Furthermore, it is shown that by expanding the unknown parameters and imposing new constraints, the nonlinear objective function can be converted into a linear production game (LPG) problem. Since an LPG possesses an optimal solution which can be evaluated with polynomial time, the proposed method is efficient and accurate. Computer simulations have been conducted and the results show that the proposed method is very promising.

Background modeling using phase space for day and night video surveillance systems

This paper presents a novel background modeling approach for day and night video surveillance. A great number of background models have been proposed to represent the background scene for video surveillance. In this paper, we propose a novel background modeling approach by using the phase space trajectory to represent the change of intensity over time for each pixel. If the intensity of a pixel which originally belongs to the background deviates from the original trajectory in phase space, then it is considered a foreground object pixel. In this manner, we are able to separate the foreground object from the background scene easily. The experimental results show the feasibility of the proposed background model.

An RST-Tolerant Shape Descriptor for Object Detection

In this paper, we propose a new object detection method that does not need a learning mechanism. Given a hand-drawn model as a query, we can detect and locate objects that are similar to the query model in cluttered images. To ensure the invariance with respect to rotation, scaling, and translation (RST), high curvature points (HCPs) on edges are detected first. Each pair of HCPs is then used to determine a circular region and all edge pixels covered by the circular region are transformed into a polar histogram. Finally, we use these local descriptors to detect and locate similar objects within any images. The experiment results show that the proposed method outperforms the existing state-of-the-art work.