Saleh Aly

Face recognition: challenges, achievements and future directions

Face recognition has received significant attention because of its numerous applications in access control, law enforcement, security, surveillance, Internet communication and computer entertainment. Although significant progress has been made, the state-of-the-art face recognition systems yield satisfactory performance only under controlled scenarios and they degrade significantly when confronted with real-world scenarios. The real-world scenarios have unconstrained conditions such as illumination and pose variations, occlusion and expressions. Thus, there remain plenty of challenges and opportunities ahead. Latterly, some researchers have begun to examine face recognition under unconstrained conditions. Instead of providing a detailed experimental evaluation, which has been already presented in the referenced works, this study serves more as a guide for readers. Thus, the goal of this study is to discuss the significant challenges involved in the adaptation of existing face recognition algorithms to build successful systems that can be employed in the real world. Then, it discusses what has been achieved so far, focusing specifically on the most successful algorithms, and overviews the successes and failures of these algorithms to the subject. It also proposes several possible future directions for face recognition. Thus, it will be a good starting point for research projects on face recognition as useful techniques can be isolated and past errors can be avoided.

Face recognition across illumination

Illumination variation on images of faces is one of the most difficult problems in face recognition systems. The performance of a self-organizing map-based face recognition system is highly degraded when the illumination in test images differs from that of the training images. Illumination normalization is a way to solve this problem. Both global and local image enhancement methods are studied in this article. A local histogram equalization method strongly improves the recognition accuracy of the CMU-PIE face database.

An Effective Face Detection Algorithm Based on Skin Color Information

Face detection approach is presented in this paper combines skin color detection and neural network. The first motivation for our paper is to decide which color space is the best in order to build efficient skin color detector can be embedded in the overall face detection system. The proposed skin detection approach uses a chrominance distribution model of skin-color information in the input image in order to detect skin pixels over the entire image. Next, morphological operations are used in order to smooth the detected skin region and generate, finally, face candidates for face-base applications. Finally, neural network is used in order to verify these face candidates. Many experiments using color images gathered from the Internet and from our own database are conducted and give encouraging results. It is expected to combine the proposed face detector with face recognition approach to be embedded later in human computer interaction applications.

Face recognition under varying illumination using Mahalanobis self-organizing map

We present an appearance-based method for face recognition and evaluate its robustness against illumination changes. Self-organizing map (SOM) is utilized to transform the high dimensional face image into low dimensional topological space. However, the original learning algorithm of SOM uses Euclidean distance to measure similarity between input and codebook images, which is very sensitive to illumination changes. In this paper, we present Mahalanobis SOM, which uses Mahalanobis distance instead of the original Euclidean distance. The effectiveness of the proposed method is demonstrated by conducting some experiments on Yale B and CMU-PIE face databases.

Robust Face Recognition Using Multiple Self-Organized Gabor Features and Local Similarity Matching

Gabor-based face representation has achieved enormous success in face recognition. However, one drawback of Gabor-based face representation is the huge amount of data that must be stored. Due to the nonlinear structure of the data obtained from Gabor response, classical linear projection methods like principal component analysis fail to learn the distribution of the data. A nonlinear projection method based on a set of self-organizing maps is employed to capture this nonlinearity and to represent face in a new reduced feature space. The Multiple Self-Organized Gabor Features (MSOGF) algorithm is used to represent the input image using all winner indices from each SOM map. A new local matching algorithm based on the similarity between local features is also proposed to classify unlabeled data. Experimental results on FERET database prove that the proposed method is robust to expression variations.

Partially occluded pedestrian classification using histogram of oriented gradients and local weighted linear kernel support vector machine

One of the main challenges in pedestrian classification is partial occlusion. This study presents a new method for pedestrian classification with partial occlusion handling. The proposed method involves a set of part-based classifiers trained on histogram of oriented gradients features derived from non-occluded pedestrian data set. The score of each part classifier is then employed to weight features used to train a second stage full-body classifier. The full-body classifier based on local weighted linear kernel support vector machine is trained using both non-occluded and artificially generated partial occlusion pedestrian dataset. The new kernel allows to significantly focus on the non-occluded parts and reduce the impact of the occluded ones. Experimental results on real-world dataset, with both partially occluded and non-occluded data, show high performance of the proposed method compared with other state-of-the-art methods.

Visual feature extraction using variable map-dimension Hypercolumn Model

Hypercolumn model (HCM) is a neural network model previously proposed to solve image recognition problem. In this paper, we propose an improved version of HCM network and demonstrate its ability to solve face recognition problem. HCM network is a hierarchical model based on self-organizing map (SOM) that closely follows the organization of visual cortex and builds an increasingly complex and invariant feature representation. This invariance achieved by alternating between feature extraction and feature integration operation. To improve the recognition rate of HCM, we propose a variable dimension for each map in the feature extraction layer. The number of neurons in each map-side is decided automatically from training data. We demonstrate the performance of the approach using ORL face database.

Arabic sign language fingerspelling recognition from depth and intensity images

Automatic Arabic sign language recognition (ArSL) and fingerspelling considered to be the preferred communication method among deaf people. In this paper, we propose a system for alphabetic Arabic sign language recognition using depth and intensity images which acquired from SOFTKINECT™ sensor. The proposed method does not require any extra gloves or any visual marks. Local features from depth and intensity images are learned using unsupervised deep learning method called PCANet. The extracted features are then recognized using linear support vector machine classifier. The performance of the proposed method is evaluated on dataset of real images captured from multi-users. Experiments using a combination of depth and intensity images and also using depth and intensity images separately are performed. The obtained results show that the performance of the proposed system improved by combining both depth and intensity information which give an average accuracy of 99:5%.

Learning invariant local image descriptor using convolutional Mahalanobis self-organising map

The image descriptor is a critical issue for most image recognition problems. Local image descriptors based on Gabor filters, SIFT, HOG and LBP have exhibited good image representation for many applications. However, these descriptors are designed in a hand-crafted way and the extracted features may not be appropriate from one application to another. In this paper, a new learning based mechanism is proposed to learn invariant image features that are optimal for image representation in a data-driven way. Particularly, a new stochastic mini-batch learning algorithm is proposed to train a Mahalanobis self-organising map (MSOM) model. The MSOM model consists of a group of connected neurons where each neuron is attached with a set of sub-neurons. In this model, neurons learn local features from training images while sub-neurons learn local variations. In the proposed MSOM model, invariance is achieved by learning both neuron means and covariance matrix for each neuron. The principal eigenvectors of the learned covariance matrices are then used to represent local variations of the learned features. Two new models based on SOM and MSOM, namely convolutional SOM (CSOM) and convolutional Mahalanobis SOM (CMSOM), are also proposed as local image descriptors using local feature histograms. The CSOM and CMSOM have a similar structure as other convolutional neural networks (CNNs). However, the CSOM and CMSOM employ an index of the best learned feature instead of the feature response used in CNNs. Experimental results on the ORL and Yale face databases and the MNIST handwritten digit database show the robustness of the proposed methods.

Arabic Sign Language Recognition Using Spatio-Temporal Local Binary Patterns and Support Vector Machine

One of the most common ways of communication in deaf community is sign language recognition. This paper focuses on the problem of recognizing Arabic sign language at word level used by the community of deaf people. The proposed system is based on the combination of Spatio-Temporal local binary pattern (STLBP) feature extraction technique and support vector machine (SVM) classifier. The system takes a sequence of sign images or a video stream as input, and localize head and hands using IHLS color space and random forest classifier. A feature vector is extracted from the segmented images using local binary pattern on three orthogonal planes (LBP-TOP) algorithm which jointly extracts the appearance and motion features of gestures. The obtained feature vector is classified using support vector machine classifier. The proposed method does not require that signers wear gloves or any other marker devices. Experimental results using Arabic sign language (ArSL) database contains 23 signs (words) recorded by 3 signers show the effectiveness of the proposed method. For signer dependent test, the proposed system based on LBP-TOP and SVM achieves an overall recognition rate reaching up to 99.5%.