Wen-Yi Zhao

Discriminant analysis of principal components for face recognition

In this paper we describe a face recognition method based on PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis). The method consists of two steps: first we project the face image from the original vector space to a face subspace via PCA, second we use LDA to obtain a best linear classifier. The basic idea of combining PCA and LDA is to improve the generalization capability of LDA when only few samples per class are available. Using PCA, we are able to construct a face subspace in which we apply LDA to perform classification. Using FERET dataset we demonstrate a significant improvement when principal components rather than original images are fed to the LDA classifier. The hybrid classifier using PCA and LDA provides a useful framework for other image recognition tasks as well.

SFS based view synthesis for robust face recognition

Sensitivity to variations in pose is a challenging problem in face recognition using appearance-based methods. More specifically, the appearance of a face changes dramatically when viewing and/or lighting directions change. Various approaches have been proposed to solve this difficult problem. They can be broadly divided into three classes: (1) multiple image-based methods where multiple images of various poses per person are available; (2) hybrid methods where multiple example images are available during learning but only one database image per person is available during recognition; and (3) single image-based methods where no example-based learning is carried out. We present a method that comes under class 3. This method, based on shape-from-shading (SFS), improves the performance of a face recognition system in handling variations due to pose and illumination via image synthesis.

Illumination-insensitive face recognition using symmetric shape-from-shading

Sensitivity to variations in illumination is a fundamental and challenging problem in face recognition. In this paper, we describe a new method based on symmetric shape-from-shading (SSFS) to develop a face recognition system that is robust to changes in illumination. The basic idea of this approach is to use the SSFS algorithm as a tool to obtain a prototype image which is illumination-normalized. It has been shown that the SSFS algorithm has a unique point-wise solution. But it is still difficult to recover accurate shape information given a single real face image with complex shape and varying albedo. In stead, we utilize the fact that all faces share a similar shape making the direct computation of the prototype image from a given face image feasible. Finally, to demonstrate the efficacy of our method, we have applied it to several publicly available face databases.

Empirical performance analysis of linear discriminant classifiers

In face recognition literature, holistic template matching systems and geometrical local feature based systems have been pursued. In the holistic approach, PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis) are popular ones. More recently, the combination of PCA and LDA has been proposed as a superior alternative over pure PCA and LDA. In this paper, we illustrate the rationales behind these methods and the pros and cons of applying them to pattern classification task. A theoretical performance analysis of LDA suggests applying LDA over the principal components from the original signal space or the subspace. The improved performance of this combined approach is demonstrated through experiments conducted on both simulated data and real data.

Robust image based face recognition

In face recognition literature, 2D image based approaches are possibly the most promising ones. However, the 2D images/patterns can change dramatically in practice. We first study the performance degradation due to 2D distortions and illumination variations on the input images. We then propose several methods to improve the system performance. Finally experiments are carried out using FERET and other face databases to demonstrate the improvement of one particular system-the subspace LDA system.

3D model enhanced face recognition

A personal identification system based on the analysis of frontal or profile images of the face has applications in human-computer interfaces, access control, surveillance etc. Among many practical face recognition schemes, image based approaches are possibly the most promising ones. However, the 2D images/patterns of 3D face objects can change dramatically due to lighting and viewing variations. In this paper, we propose a generic 3D model to enhance existing face recognition systems. More specifically, we use a 3D model to synthesize the so-called prototype image from a given image acquired under different lighting and viewing conditions.

Improving color based video shot detection

Video shot detection plays a fundamental role in video access/analysis. Among many shot detection schemes, color based schemes seem to be the most widely used ones which can be applied to videos of different domains. However color based techniques are not always very efficient. For example, over-detection (high false-alarm rate) may occur, especially for difficult videos such as home video which often contain significant object motion and illumination change. In this paper we propose a new learning method to improve the color-based shot detection schemes by employing a better similarity measure obtained through a minmax optimization procedure. This new measure is not tied to a particular feature such as color, hence it can be applied to video browsing, indexing based on other features as well.

Pose-encoded spherical harmonics for face recognition and synthesis using a single image

Face recognition under varying pose is a challenging problem, especially when illumination variations are also present. In this paper, we propose to address one of the most challenging scenarios in face recognition. That is, to identify a subject from a test image that is acquired under different pose and illumination condition from only one training sample (also known as a gallery image) of this subject in the database. For example, the test image could be semifrontal and illuminated by multiple lighting sources while the corresponding training image is frontal under a single lighting source. Under the assumption of Lambertian reflectance, the spherical harmonics representation has proved to be effective in modeling illumination variations for a fixed pose. In this paper, we extend the spherical harmonics representation to encode pose information. More specifically, we utilize the fact that 2D harmonic basis images at different poses are related by close-form linear transformations, and give a more convenient transformation matrix to be directly used for basis images. An immediate application is that we can easily synthesize a different view of a subject under arbitrary lighting conditions by changing the coefficients of the spherical harmonics representation. A more important result is an efficient face recognition method, based on the orthonormality of the linear transformations, for solving the above-mentioned challenging scenario. Thus, we directly project a nonfrontal view test image onto the space of frontal view harmonic basis images. The impact of some empirical factors due to the projection is embedded in a sparse warping matrix; for most cases, we show that the recognition performance does not deteriorate after warping the test image to the frontal view. Very good recognition results are obtained using this method for both synthetic and challenging real images.

Efficient Scene-based Nonuniformity Correction and Enhancement

We propose a unified framework for scene-based nonuniformity correction (NUC) and enhancement that is required for the FPA-like (focal plane array) sensors to remove fixed-pattern noise and to enhance the image quality. In contrast to existing scene-based NUC methods, the new framework allows us to process image sequences under severe and structured nonuniformity efficiently and to obtain high quality images. We achieve this goal by applying an efficient registration-based method that is bootstrapped by statistical scene-based NUC methods. Specifically, we initialize the whole NUC-enhancement process by applying statistical methods in order to obtain images with quality just enough for image registration. To obtain high-quality images, we integrate the NUC process with super-resolution techniques to reduce noise and enhance resolution. This is achieved by adopting a new imaging model that includes linear NUC model and image blurring and subsampling. Experiments with real data demonstrate the efficacy of the proposed framework.

Super-resolution with significant illumination change

In this paper, we study the impact of illumination change on super-resolution. Based on analysis and experimental results, we show that large illumination change, not a global/parametric one, poses a significant challenge to existing super-resolution algorithms. Existing robust techniques of handling outliers do not offer a good solution either. We propose a two-step framework to super-resolve the object shape at high-resolution first and then synthesize high-resolution images. For applications where appropriate illumination models are not available, we propose a pre-processing framework. Extensive experimental results are presented based on synthetic and real examples.