Chih-Chang Yu

Efficient human action and gait analysis using multiresolution motion energy histogram

Average Motion Energy (AME) image is a good way to describe human motions. However, it has to face the computation efficiency problem with the increasing number of database templates. In this paper, we propose a histogram-based approach to improve the computation efficiency. We convert the human action/gait recognition problem to a histogram matching problem. In order to speed up the recognition process, we adopt a multiresolution structure on the Motion Energy Histogram (MEH). To utilize the multiresolution structure more efficiently, we propose an automated uneven partitioning method which is achieved by utilizing the quadtree decomposition results of MEH. In that case, the computation time is only relevant to the number of partitioned histogram bins, which is much less than the AME method. Two applications, action recognition and gait classification, are conducted in the experiments to demonstrate the feasibility and validity of the proposed approach.

Vision-Based Fingertip-Writing Character Recognition

This paper proposes a vision-based fingertip handwriting recognition system to provide an alternative to input devices. Traditional handwriting recognition systems are limited because they require a specific or expensive input device, such as pen, tablet, or touch panel. Recently, cameras have gradually become standard components in many computer-based products. Therefore, a fingertip and camera combination provides a flexible and convenient input device. The proposed system combines fingertip detection, trajectory feature extraction, and character recognition. First, fingertip moving trajectories are tracked and recoded. The proposed cyclic chain code histograms are then obtained from the trajectories and used as features in the following recognition process. An improved radial basis function (RBF) neural network is used to recognize handwritten characters. Experimental results show that the proposed novel input system is feasible and effective. This study also presents possible applications for camera input devices.

Distinguishing falsification of human faces from true faces based on optical flow information

Falsification of Human Faces using face photos has been an arising problem for face recognition and verification systems. In this paper, we propose a system to distinguish face photos from true faces by their motion models. In order to enhance the difference between the two classes, we design an enhanced optical flow method which generates a larger difference between the motion model of true faces and that of face photos. The feature vector we adopted is the dense optical flow field across a short period of time. An LDA-based training method is adopted to separate the projection of the training data into two classes, and a Bayes classifier is used to classify the testing samples. Under the specified motion of true faces and face photos, our proposed method can effectively distinguish the two classes with high verification rate. Even if the motion is arbitrary for both classes, the proposed system can also report satisfying results.

Human-centered fingertip mandarin input system using single camera

Designing a user friendly Chinese input system is a challenging task due to the logographic nature of Chinese characters. Using fingertips and cameras to replace pens and touch panels as input devices could reduce the cost and improve ease-of-use and comfort of the computer-human interface. In this work, Chinese character entry is achieved using Mandarin Phonetic Symbol (MPS) recognition via on-line fingertip tracking. In the proposed system, particle filters are applied for robust fingertip tracking. Afterwards, MPS recognition is performed on the tracked fingertip trajectories using Hidden Markov Models. In the proposed system, the challenges of entering, leaving, and virtual strokes caused by video-based fingertip input can be overcome. We conduct experiments to validate that the MPS symbols written by fingertips are successfully and efficiently recognized using the proposed framework.

An Efficient Way to Classify Human Gaits

The computation efficiency in human identification problem is a very important issue when the number of database templates is large. In this paper, we propose a histogram based approach to improve the computation efficiency for human gait classification. We convert the human gait classification problem to a histogram matching problem. In order to speed up the recognition process, we adopt a multi resolution structure on the Motion Energy Histogram (MEH). To utilize the multi resolution structure more efficiently, we propose an automated uneven partitioning method which is achieved by utilizing the quadtree decomposition results of MEH. In that case, the computation time is only relevant to the number of partitioned histogram bins. Experiments demonstrate the feasibility and validity of the proposed approach.

Connectivity Based Human Body Modeling from Monocular Camera

In this paper, we develop a system for automated human body tracking and modeling based on a monocular camera. In this system, ten body parts including head, torso, arms and legs are extracted to build a 2D human body model. One way to decompose human silhouette into different parts is to generate cuts between the negative minimum curvature (NMC) points. However, due to the self-occlusion problem and left-right ambiguity, each individual body part cannot be successfully identified in every frame. Therefore, in addition to utilizing the NMC points, we design a forward and backward tracking mechanism to identify the location of head in each frame. The torso angle and size are determined by integrating multiple-frame information with the modified solution of Poisson equation. Hands and feet can then be identified correctly based on a modified star skeleton approach along with the nearest-neighbor tracking mechanism. The rest of joint points can also be located by making use of the notion “connectivity”. In the experiments, we analyze the performance of the proposed human body modeling mechanism. We also demonstrate a behavior analysis application by employing the proposed method. The experiment results verify the robustness of the proposed approach and the feasibility of the employing the proposed approach to the action recognition application.

Detecting and retrieving texts from electronic marquees in natural scenes

This paper proposed a system which is able to detect the electronic marquee in natural scenes and retrieve the full contents displayed by the electronic marquee automatically. First, the system locates the marquee in natural images by analyzing the optical flow fields of the video sequence. Meanwhile, the starting and ending time of scrolling texts can also be obtained. Then, text blocks are retrieved by the integration of temporal information and geometric projections. Finally, text blocks which have the same contents in adjacent frames are associated together. The full contents displayed by the electronic marquee are reconstructed by rearranging these text blocks. Experiments demonstrate that the proposed method is effective in displaying texts in the electronic marquee regardless of the language or the color of texts.

Mixture models with skin and shadow probabilities for fingertip input applications

This paper proposes an accurate moving skin region detection method for video-based human-computer interface using gestures or fingertips. Using Gaussian mixture models as groundwork, the proposed method expresses the features of skins in a probability form and incorporates them into the mixture-based framework. Moreover, to alleviate the influence of shadows, the properties of shadows are also formulated as probabilities and used for shadow detection and elimination. In addition to moving skin region detection, this paper also develops two practical fingertip input applications to demonstrate the accuracy of the proposed detection method. The two applications are Mandarin Phonetic Symbol combination recognition system and single fingertip virtual keyboard implementation. Experimental results have shown the advantages of the proposed detection method and the effectiveness of the two application implementations.

Preprocessing for Images Captured by Cameras

Due to the rapid development of mobile devices equipped with cameras, the realization of what you get is what you see is not a dream anymore. In general, texts in images often draw people’s attention due to the following reasons: semantic meanings to objects in the image (e.g., the name of the book), information about the environment (e.g., a traffic sign), or commercial purpose (e.g., an advertisement). The mass development of mobile device with low cost cameras boosts the demand of recognizing characters in nature scenes via mobile devices such as smartphones. Employing text detection algorithms along with character recognition techniques on mobile devices assists users in understanding or gathering useful information around them. A useful mobile application is the translation tool. Using handwriting as the input is widely used in current translation tools on smartphones. However, capturing images and recognizing texts directly is more intuitive and convenient for users. A translation tool with character recognition techniques recognizes texts on the road signs or restaurant menus. Such application greatly helps travelers and blinds.

Video-based Chinese Input System via Fingertip Tracking

In this paper, we propose a system to detect and track fingertips online and recognize Mandarin Phonetic Symbol (MPS) for user-friendly Chinese input purposes. Using fingertips and cameras to replace pens and touch panels as input devices could reduce the cost and improve the ease-of-use and comfort of computer-human interface. In the proposed framework, particle filters with enhanced appearance models are applied for robust fingertip tracking. Afterwards, MPS combination recognition is performed on the tracked fingertip trajectories using Hidden Markov Models. In the proposed system, the fingertips of the users could be robustly tracked. Also, the challenges of entering, leaving and virtual strokes caused by video-based fingertip input can be overcome. Experimental results have shown the feasibility and effectiveness of the proposed work.