Hongming Zhang

Object detection using spatial histogram features

In this paper, we propose an object detection approach using spatial histogram features. As spatial histograms consist of marginal distributions of an image over local patches, they can preserve texture and shape information of an object simultaneously. We employ Fisher criterion and mutual information to measure discriminability and features correlation of spatial histogram features. We further train a hierarchical classifier by combining cascade histogram matching and support vector machine. The cascade histogram matching is trained via automatically selected discriminative features. A forward sequential selection method is presented to construct uncorrelated and discriminative feature sets for support vector machine classification. We evaluate the proposed approach on two different kinds of objects: car and video text. Experimental results show that the proposed approach is efficient and robust in object detection.

HandTalker: A Multimodal Dialog System Using Sign Language and 3-D Virtual Human

In this paper, we describe HandTalker: a system we designed for making friendly communication reality between deaf people and normal hearing society. The system consists of GTS (Gesture/Sign language To Spoken language) part and STG (Spoken language To Gesture/Sign language) part. GTS is based on the technology of sign language recognition, and STG is based on 3D virtual human synthesis. Integration of the sign language recognition and 3D virtual human techniques greatly improves the system performance. The computer interface for deaf people is data-glove, camera and computer display, and the interface for hearing-abled is microphone, keyboard, and display. HandTalker now can support no domain limited and continuously communication between deaf and hearing-abled Chinese people.

Combining Skin Color Model and Neural Network for Rotation Invariant Face Detection

Face detection is a key problem in human-computer interaction. In this paper, we present an algorithm for rotation invariant face detection in color images of cluttered scenes. It is a hierarchical approach, which combines a skin color model, a neural network, and an upright face detector. Firstly, the skin color model is used to process the color image to segment the face-like regions from the background. Secondly, the neural network computing and an operation for locating irises are performed to acquire rotation angle of each input window in the face-like regions. Finally, we provide an upright face detector to determine whether or not the rotated window is a face. Those techniques are integrated into a face detection system. The experiments show that the algorithm is robust to different face sizes and various lighting conditions.

Individual 3D face synthesis based on orthogonal photos and speech-driven facial animation

In the paper, a methodology for individual face synthesis using given orthogonal photos is proposed. An integrated speech-driven facial animation system is presented. Firstly, in order to capture a given subjects personal facial configuration, a novel coarse-to-fine strategy based on facial texture and deformable template is proposed to localize some facial feature points in the image of frontal view. The corresponding feature points in the profile are extracted by using the polygonal approximation. Secondly, all these feature points are aligned to fit the generic 3D face model to a specialized one to reflect the given persons facial configuration. Then a multi-direction texture-mapping technique is presented to synthesize a lifelike personal face. Finally, muscle-based expression and lip-motion models are built up. All the above technologies are integrated into a speech-driven face animation system. We are aiming at a MPEG-4 compatible video-driven face animation system.

Learning informative features for spatial histogram-based object detection

Feature extraction for object representation plays an important role in automatic object detection system. As the spatial histograms consist of marginal distribution of image over local patches, object texture and shape are simultaneously preserved by the spatial histogram representation. In this paper, we propose methods of learning informative features for spatial histogram-based object detection. We employ Fisher criterion to measure the discriminability of each spatial histogram feature and calculate features correlation using mutual information. In order to construct compact feature sets for efficient classification, we propose informative selection algorithm to select uncorrelated and discriminative spatial histogram features. The proposed approaches are tested on two different kinds of objects: car and video text. The experimental results show that the proposed approaches are efficient in object detection.

A novel two-tier Bayesian based method for hair segmentation

In this paper, a novel two-tier Bayesian based method is proposed for hair segmentation. In the first tier, we construct a Bayesian model by integrating hair occurrence prior probabilities (HOPP) with a generic hair color model (GHCM) to obtain some reliable hair seed pixels. These initial seeds are further propagated to their neighborhood pixels by utilizing segmentation results of Mean Shift, to obtain more seeds. In the second tier, all of these selected seeds are used to train a hair-specific Gaussian model, which are combined with HOPP to build the second Bayesian model for pixel classification. Mean Shift results are further utilized to remove holes and spread hair regions. The experimental results illustrate the effectiveness of our approach.

A novel coarse-to-fine hair segmentation method

Segmenting hair regions from human images facilitates many tasks like hair synthesis and hair style trends forecast. However, hair segmentation is quite challenging due to hair/background confusion and large hair pattern diversity. To address these problems to some extent, this paper proposes a novel coarse-to-fine hair segmentation method. In our approach, firstly, the recently proposed “Active Segmentation with Fixation” (ASF) is used to coarsely define an enclosed candidate region with high-recall (but possibly low-precision) of hair pixels and exclude considerable part of the backgrounds which are easily confused with hair. Then Graph Cuts (GC) method is applied to the candidate regions to remove additional false positives by incorporating hair-specific information. Specifically, Bayesian method is employed to select some reliable hair and background regions (seeds) among the ones over-segmented by Mean Shift. SVM classifier is then learnt online from these seeds and explored to predict hair/background likelihood probability, which is subsequently fed into GC algorithm. The novelty of the proposed approach lies in three folds: 1) an elaborate design of hair segmentation framework, which utilizes ASF to reduce the candidate hair regions and adopts GC to achieve more accurate hair region contours; 2) the region-based strategy for seed selection; 3) the exploration of the discriminative method, SVM, to predict the probability of each pixel belonging to hair and background regions. Extensive experimental results demonstrate the approach outperforms recently proposed methods.

A SYSTEM FOR HUMAN FACE AND FACIAL FEATURE LOCATION

This paper presents an implementation of a system designed for the location of human faces and facial features such as pupils, eyes, nose and mouth. The kernel of the system is an integration of several algorithms, such as the human face center-of-gravity template, illumination compensation, and so on. A false-face removal algorithm is proposed in this paper specially for the distinguishing of cartoon faces from true faces. The testing experiments of the system have produced quite good results, with the average detection accuracy rates for face detection and facial feature location being 97.8% and 87.5% respectively.

Robust multi-view face detection using error correcting output codes

This paper presents a novel method to solve multi-view face detection problem by Error Correcting Output Codes (ECOC). The motivation is that face patterns can be divided into separated classes across views, and ECOC multi-class method can improve the robustness of multi-view face detection compared with the view-based methods because of its inherent error-tolerant ability. One key issue with ECOC-based multi-class classifier is how to construct effective binary classifiers. Besides applying ECOC to multi-view face detection, this paper emphasizes on designing efficient binary classifiers by learning informative features through minimizing the error rate of the ensemble ECOC multi-class classifier. Aiming at designing efficient binary classifiers, we employ spatial histograms as the representation, which provide an over-complete set of optional features that can be efficiently computed from the original images. In addition, the binary classifier is constructed as a coarse to fine procedure using fast histogram matching followed by accurate Support Vector Machine (SVM). The experimental results show that the proposed method is robust to multi-view faces, and achieves performance comparable to that of state-of-the-art approaches to multi-view face detection.

Data-driven hair segmentation with isomorphic manifold inference ☆

Abstract Hair segmentation is challenging due to the diverse appearance, irregular region boundary and the influence of complex background. To deal with this problem, we propose a novel data-driven method, named Isomorphic Manifold Inference (IMI). The IMI method assumes the coarse probability map and the binary segmentation map as a couple of isomorphic manifolds and tries to learn hair specific priors from manually labeled training images. For an input image, firstly, the method calculates a coarse probability map. Then it exploits regression techniques to obtain the relationship between the coarse probability map of the test image and those of training images. Finally, this relationship, i.e., a coefficient set, is transferred to the binary segmentation maps and a soft segmentation of the test image will be achieved by a linear combination of those binary maps. Further, we employ this soft segmentation as a shape cue and integrate it with color and texture cues into a unified segmentation framework. A better segmentation is achieved by the Graph Cuts optimization. Extensive experiments are conducted to validate effectiveness of the IMI method, compare contributions of different cues and investigate the generalization of IMI method. The results strongly encourage our method.