Seong G. Kong

Recent advances in visual and infrared face recognition: a review

Face recognition is a rapidly growing research area due to increasing demands for security in commercial and law enforcement applications. This paper provides an up-to-date review of research efforts in face recognition techniques based on two-dimensional (2D) images in the visual and infrared (IR) spectra. Face recognition systems based on visual images have reached a significant level of maturity with some practical success. However, the performance of visual face recognition may degrade under poor illumination conditions or for subjects of various skin colors. IR imagery represents a viable alternative to visible imaging in the search for a robust and practical identification system. While visual face recognition systems perform relatively reliably under controlled illumination conditions, thermal IR face recognition systems are advantageous when there is no control over illumination or for detecting disguised faces. Face recognition using 3D images is another active area of face recognition, which provides robust face recognition with changes in pose. Recent research has also demonstrated that the fusion of different imaging modalities and spectral components can improve the overall performance of face recognition.

Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition

AbstractThis paper describes a new software-based registration and fusion of visible and thermal infrared (IR) image data for face recognition in challenging operating environments that involve illumination variations. The combined use of visible and thermal IR imaging sensors offers a viable means for improving the performance of face recognition techniques based on a single imaging modality. Despite successes in indoor access control applications, imaging in the visible spectrum demonstrates difficulties in recognizing the faces in varying illumination conditions. Thermal IR sensors measure energy radiations from the object, which is less sensitive to illumination changes, and are even operable in darkness. However, thermal images do not provide high-resolution data. Data fusion of visible and thermal images can produce face images robust to illumination variations. However, thermal face images with eyeglasses may fail to provide useful information around the eyes since glass blocks a large portion of thermal energy. In this paper, eyeglass regions are detected using an ellipse fitting method, and replaced with eye template patterns to preserve the details useful for face recognition in the fused image. Software registration of images replaces a special-purpose imaging sensor assembly and produces co-registered image pairs at a reasonable cost for large-scale deployment. Face recognition techniques using visible, thermal IR, and data-fused visible-thermal images are compared using a commercial face recognition software (FaceIt®) and two visible-thermal face image databases (the NIST/Equinox and the UTK-IRIS databases). The proposed multiscale data-fusion technique improved the recognition accuracy under a wide range of illumination changes. Experimental results showed that the eyeglass replacement increased the number of correct first match subjects by 85% (NIST/Equinox) and 67% (UTK-IRIS).

Comparison of fuzzy and neural truck backer-upper control systems

A simple fuzzy control system and a simple neural control system for backing up a truck in an open parking lot are developed. The choice of control problem was prompted by the recent, successful, neural network truck backer-upper simulation of Nguyen and Widrow (Proc. Int. Joint Conference on Neural Networks, vol.2, p.357-363, June, 1989). The authors were unable to exactly replicate the neural network they used. Instead the authors built the best backpropagation network they could with essentially the same kinematics and compared it to the best fuzzy controller they could develop. The fuzzy controller compares favorably with the neural controller in terms of black-box computation load, smoothness of truck trajectories, and robustness. Robustness of the fuzzy controller is studied by deliberately adding confusing FAM (fuzzy associative memory), rules-sabotage rules-to the system and by randomly removing different subsets of FAM rules. Robustness of the neural controller is studied by randomly removing different portions of the training data. It is concluded that fuzzy control shows optimal truck backing-up performance

Fusion of Visual and Thermal Signatures with Eyeglass Removal for Robust Face Recognition

This paper describes a fusion of visual and thermal infrared (IR) images for robust face recognition. Two types of fusion methods are discussed: data fusion and decision fusion. Data fusion produces an illumination-invariant face image by adaptively integrating registered visual and thermal face images. Decision fusion combines matching scores of individual face recognition modules. In the data fusion process, eyeglasses, which block thermal energy, are detected from thermal images and replaced with an eye template. Three fusion-based face recognition techniques are implemented and tested: Data fusion of visual and thermal images (Df), Decision fusion with highest matching score (Fh), and Decision fusion with average matching score (Fa). A commercial face recognition software FaceIt® is used as an individual recognition module. Comparison results show that fusion-based face recognition techniques outperformed individual visual and thermal face recognizers under illumination variations and facial expressions.

Analysis of hyperspectral fluorescence images for poultry skin tumor inspection

We present a hyperspectral fluorescence imaging system with a fuzzy inference scheme for detecting skin tumors on poultry carcasses. Hyperspectral images reveal spatial and spectral information useful for finding pathological lesions or contaminants on agricultural products. Skin tumors are not obvious because the visual signature appears as a shape distortion rather than a discoloration. Fluorescence imaging allows the visualization of poultry skin tumors more easily than reflectance. The hyperspectral image samples obtained for this poultry tumor inspection contain 65 spectral bands of fluorescence in the visible region of the spectrum at wavelengths ranging from 425 to 711 nm. The large amount of hyperspectral image data is compressed by use of a discrete wavelet transform in the spatial domain. Principal-component analysis provides an effective compressed representation of the spectral signal of each pixel in the spectral domain. A small number of significant features are extracted from two major spectral peaks of relative fluorescence intensity that have been identified as meaningful spectral bands for detecting tumors. A fuzzy inference scheme that uses a small number of fuzzy rules and Gaussian membership functions successfully detects skin tumors on poultry carcasses. Spatial-filtering techniques are used to significantly reduce false positives.

DETECTION OF SKIN TUMORS ON CHICKEN CARCASSES USING HYPERSPECTRAL FLUORESCENCE IMAGING

This article presents a method for detecting skin tumors on chicken carcasses using hyperspectral fluorescence imaging data, which provide fluorescence information in both spectral and spatial dimensions. Since these two kinds of information are complementary to each other, it is necessary to exploit them in a synergistic manner. Chicken carcasses are examined first using spectral information, and the results are used to determine candidate regions for skin tumors. Next, a spatial classifier selects the real tumor spots from the candidate regions. It was shown that the method detected chicken carcasses with tumors, but failed to detect some tumors that were smaller than 3 mm in diameter. This study uncovered meaningful spectral bands for detecting tumors using hyperspectral imaging data. A detection system based on this method can improve the detection rate, and processing time can also be reduced, because the detection procedure is simplified by using a limited number of features, which reduces computational complexity. The resultant detection rate, false positive rate, and missing rate of the proposed method are 76%, 28%, and 24%, respectively.

Visual Analysis of Eye State and Head Pose for Driver Alertness Monitoring

This paper presents visual analysis of eye state and head pose (HP) for continuous monitoring of alertness of a vehicle driver. Most existing approaches to visual detection of nonalert driving patterns rely either on eye closure or head nodding angles to determine the driver drowsiness or distraction level. The proposed scheme uses visual features such as eye index (EI), pupil activity (PA), and HP to extract critical information on nonalertness of a vehicle driver. EI determines if the eye is open, half closed, or closed from the ratio of pupil height and eye height. PA measures the rate of deviation of the pupil center from the eye center over a time period. HP finds the amount of the drivers head movements by counting the number of video segments that involve a large deviation of three Euler angles of HP, i.e., nodding, shaking, and tilting, from its normal driving position. HP provides useful information on the lack of attention, particularly when the drivers eyes are not visible due to occlusion caused by large head movements. A support vector machine (SVM) classifies a sequence of video segments into alert or nonalert driving events. Experimental results show that the proposed scheme offers high classification accuracy with acceptably low errors and false alarms for people of various ethnicity and gender in real road driving conditions.

Brief paper: Output feedback variable structure control for linear systems with uncertainties and disturbances

This paper proposes a dynamic output feedback variable structure controller for linear MIMO systems with mismatched and matched norm-bounded uncertainties and matched nonlinear disturbances. The proposed controller consists of nonlinear and linear parts, similar to the unit vector type of controller. The nonlinear part of the new controller takes care only of matched uncertainties and disturbances and, on the other hand, the linear part with full dynamics completely handles mismatched uncertainties. Designing such a linear part leads to use a Lyapunov function associated with the full states, which achieves the global stability against the mismatched uncertainties. The resulting criteria are, furthermore, converted into solvable ones, by using the so-called cone complementary linearization algorithm for bi-convex problems.

Band-Subset-Based Clustering and Fusion for Hyperspectral Imagery Classification

This paper proposes a band-subset-based clustering and fusion technique to improve the classification performance in hyperspectral imagery. The proposed method can account for the varying data qualities and discrimination capabilities across spectral bands, and utilize the spectral and spatial information simultaneously. First, the hyperspectral data cube is partitioned into several nearly uncorrelated subsets, and an eigenvalue-based approach is proposed to evaluate the confidence of each subset. Then, a nonparametric technique is used to extract the arbitrarily-shaped clusters in spatial-spectral domain. Each cluster offers a reference spectral, based on which a pseudosupervised hyperspectral classification scheme is developed by using evidence theory to fuse the information provided by each subset. The experimental results on real Hyperspectral Digital Imagery Collection Experiment (HYDICE) demonstrate that the proposed pseudosupervised classification scheme can achieve higher accuracy than the spatially constrained fuzzy c-means clustering method. It can achieve nearly the same accuracy as the supervised K-Nearest Neighbor (KNN) classifier but is more robust to noise.

Enhancement of feature extraction for low-quality fingerprint images using stochastic resonance

This paper presents a new approach to enhancing feature extraction for low-quality fingerprint images by adding noise to the original signal. Feature extraction often fails for low-quality fingerprint images obtained from excessively dry or wet fingers. In nonlinear signal processing systems, a moderate amount of noise can help amplify a faint signal while excessive amounts of noise can degrade the signal. Stochastic resonance (SR) refers to a phenomenon where an appropriate amount of noise added to the original signal can increase the signal-to-noise ratio. Experimental results show that Gaussian noise added to low-quality fingerprint images enables the extraction of useful features for biometric identification. SR was applied to 20 fingerprint images in the FVC2004 DB2 database that were rejected by a state-of-the-art fingerprint verification algorithm due to failures in feature extraction. SR enabled feature extraction from 10 out of 11 low-quality images with poor contrast. The remaining nine images were damaged fingerprints from which no meaningful features can be obtained. Improved feature extraction using SR decreases an equal error rate of fingerprint verification from 6.55% to 5.03%. The receiver operating characteristic curve shows that the genuine acceptance rates are improved for all false acceptance rates.