Tatsuo Kozakaya

Facial Deblur Inference Using Subspace Analysis for Recognition of Blurred Faces

This paper proposes a novel method for recognizing faces degraded by blur using deblurring of facial images. The main issue is how to infer a Point Spread Function (PSF) representing the process of blur on faces. Inferring a PSF from a single facial image is an ill-posed problem. Our method uses learned prior information derived from a training set of blurred faces to make the problem more tractable. We construct a feature space such that blurred faces degraded by the same PSF are similar to one another. We learn statistical models that represent prior knowledge of predefined PSF sets in this feature space. A query image of unknown blur is compared with each model and the closest one is selected for PSF inference. The query image is deblurred using the PSF corresponding to that model and is thus ready for recognition. Experiments on a large face database (FERET) artificially degraded by focus or motion blur show that our method substantially improves the recognition performance compared to existing methods. We also demonstrate improved performance on real blurred images on the FRGC 1.0 face database. Furthermore, we show and explain how combining the proposed facial deblur inference with the local phase quantization (LPQ) method can further enhance the performance.

Facial feature localization using weighted vector concentration approach

We propose an efficient and generic facial feature localization method based on a weighted vector concentration approach. Our method does not require any specific priors on facial shape but implicitly learns its structural information from a training data. Unlike previous work, facial feature points are globally estimated by the concentration of directional vectors from sampling points on a face region, and those vectors are weighted by using local likelihood patterns which discriminate the appropriate position of the feature points. The directional vectors and local likelihood patterns are provided through nearest neighbor search between local patterns around the sampling points and a trained codebook of extended templates. The combination of the global vector concentration and the verification with the local likelihood patterns achieves robust facial feature point detection. We demonstrate that our method outperforms state-of-the-art method based on the Active Shape Models in our evaluation.

COUNT Forest: CO-Voting Uncertain Number of Targets Using Random Forest for Crowd Density Estimation

This paper presents a patch-based approach for crowd density estimation in public scenes. We formulate the problem of estimating density in a structured learning framework applied to random decision forests. Our approach learns the mapping between patch features and relative locations of all objects inside each patch, which contribute to generate the patch density map through Gaussian kernel density estimation. We build the forest in a coarse-to-fine manner with two split node layers, and further propose a crowdedness prior and an effective forest reduction method to improve the estimation accuracy and speed. Moreover, we introduce a semi-automatic training method to learn the estimator for a specific scene. We achieved state-of-the-art results on the public Mall dataset and UCSD dataset, and also proposed two potential applications in traffic counts and scene understanding with promising results.

Facial deblur inference to improve recognition of blurred faces

This paper proposes a novel method for deblurring facial images to recognize faces degraded by blur. The main problem is how to infer a point spread function (PSF) representing the process of blur. Inferring a PSF from a single facial image is an ill-posed problem. To make this problem more tractable, our method uses learned prior information derived from a training set of blurred facial images of several individuals. We construct a feature space such that blurred faces degraded by the same PSF are similar to one another and form a cluster. During training, we compute a statistical model of each PSF cluster in this feature space. For PSF inference we compare a query image of unknown blur with each model and select the closest one. Using the PSF corresponding to that model, the query image is deblurred, ready for recognition. Experiments on a standard face database artificially degraded by focus or motion blur show that our method substantially improves the recognition performance compared with state-of-the-art methods. We also demonstrate improved performance on real blurred images.

Random ensemble metrics for object recognition

This paper presents a novel and generic approach for metric learning, random ensemble metrics (REMetric). To improve generalization performance, we introduce the concept of ensemble learning to the metric learning scheme. Unlike previous methods, our method does not optimize the global objective function for the whole training data. It learns multiple discriminative projection vectors obtained from linear support vector machines (SVM) using randomly subsampled training data. The final metric matrix is then obtained by integrating these vectors. As a result of using SVM, the learned metric has an excellent scalability for the dimensionality of features. Therefore, it does not require any prior dimensionality reduction techniques such as PCA. Moreover, our method allows us to unify dimensionality reduction and metric learning by controlling the number of the projection vectors. We demonstrate through experiments, that our method can avoid overfitting even though a relatively small number of training data is provided. The experiments are performed with three different datasets; the Viewpoint Invariant Pedestrian Recognition (VIPeR) dataset, the Labeled Face in the Wild (LFW) dataset and the Oxford 102 category flower dataset. The results show that our method achieves equivalent or superior performance compared to existing state-of-the-art metric learning methods.

Illumination Normalization using Quotient Image-based Techniques

This chapter focuses on correctly recognizing faces in the presence of large illumination variation. Our aim is to do this by synthesizing an illumination normalized image using Quotient Image-based techniques (Shashua et al., 2001, Wang et al., 2004, Chen et al., 2005, Nishiyama et al., 2006, Zhang et al., 2007, and An et al., 2008). These techniques extract an illumination invariant representation of a face from a raw facial image. To discuss the variation of facial appearances caused by illumination, the appearances are classified into four main components: diffuse reflection, specular reflection, attached shadow and cast shadow (see Figure 1) as described in (Shashua, 1999).

Cat face detection with two heterogeneous features

In this paper, we propose a generic and efficient object detection framework based on two heterogeneous features and demonstrate effectiveness of our method for a cat face detection problem. Simple Haar-like features with AdaBoost are fast to compute but they are not discriminative enough to deal with complicated shape and texture. Therefore, we cascade joint Haar-like features with AdaBoost and CoHOG descriptors with a linear classifier. Since the CoHOG descriptors are extremely high dimensional pattern descriptors based on gradient orientations, they have a strong classification capability to represent various cat face patterns. The combination of these two distinct classifiers enables fast and accurate cat face detection. The experimental result with about 10,000 cat images shows that our method gives better performance in comparison with the state-of-the-art cat head detection method, although our method does not exploit any cat specific characteristics.

Face recognition by projection-based 3D normalization and shading subspace orthogonalization

This paper describes a new face recognition method using a projection-based 3D normalization and a shading subspace orthogonalization under variation in facial pose and illumination. The proposed method does not need any reconstruction and reillumination for a personalized 3D model, thus it can avoid these troublesome problems and the recognition process can be done rapidly. The facial size and pose including out of plane rotation can be normalized to a generic 3D model from one still image and the input subspace is generated by perturbed cropped patterns in order to absorb the localization errors. Furthermore, by exploiting the fact that a normalized pattern is fitted to the generic 3D model, illumination robust features are extracted through the shading subspace orthogonalization. Evaluation experiments are performed using several databases and the results show the effectiveness of our method under various facial poses and illuminations

Development of a Face Recognition System on an Image Processing LSI Chip

We present a real-time and high-precision face recognition system using an image processing LSI chip:Visconti[Visconti: Multi-VLIW Image Recognition Processor]. The system is compact and operates at low power, making it suitable for many purposes, including home security and robot vision. The LSI includes three media processing modules and peripherals which are suitable for machine vision. Face recognition is based on the constrained mutual sub-space method (CMSM), implemented on the LSI and optimized to make the best use of the hardware features. The optimization consists of four different levels: instruction, data, task and algorithm. It shows possible implementation of face recognition with multi-core CPUs or other LSI chips. Experimental results show the system operates at 20 frames/sec and a recognition rate is 99.59% when a threshold value is set to 0.55; performance comparable to that of a state-of-the-art system.

An Efficient 3D Geometrical Consistency Criterion for Detection of a Set of Facial Feature Points

We propose a novel efficient three-dimensional geometrical consistency criterion for detection of a set of facial feature points. Many face recognition methods employing a single image require localization of particular facial feature points and their performance is highly dependent on localization accuracy in detecting these feature points. The proposed method is able to calculate alignment error of a point set rapidly because calculation is not iterative. Also the method does not depend on the type of point detection method used and no learning is needed. Independently detected point sets are evaluated through matching to a three-dimensional generic face model. Correspondence error is defined by the distance between the feature points defined in the model and those detected. The proposed criterion is evaluated through experiment using various facial feature point sets on face images.