Gaurav Goswami

On RGB-D face recognition using Kinect

Face recognition algorithms generally use 2D images for feature extraction and matching. In order to achieve better performance, 3D faces captured via specialized acquisition methods have been used to develop improved algorithms. While such 3D images remain difficult to obtain due to several issues such as cost and accessibility, RGB-D images captured by low cost sensors (e.g. Kinect) are comparatively easier to acquire. This research introduces a novel face recognition algorithm for RGB-D images. The proposed algorithm computes a descriptor based on the entropy of RGB-D faces along with the saliency feature obtained from a 2D face. The probe RGB-D descriptor is used as input to a random decision forest classifier to establish the identity. This research also presents a novel RGB-D face database pertaining to 106 individuals. The experimental results indicate that the RGB-D information obtained by Kinect can be used to achieve improved face recognition performance compared to existing 2D and 3D approaches.

RGB-D Face Recognition With Texture and Attribute Features

Face recognition algorithms generally utilize 2D images for feature extraction and matching. To achieve higher resilience toward covariates, such as expression, illumination, and pose, 3D face recognition algorithms are developed. While it is challenging to use specialized 3D sensors due to high cost, RGB-D images can be captured by low-cost sensors such as Kinect. This research introduces a novel face recognition algorithm using RGB-D images. The proposed algorithm computes a descriptor based on the entropy of RGB-D faces along with the saliency feature obtained from a 2D face. Geometric facial attributes are also extracted from the depth image and face recognition is performed by fusing both the descriptor and attribute match scores. The experimental results indicate that the proposed algorithm achieves high face recognition accuracy on RGB-D images obtained using Kinect compared with existing 2D and 3D approaches.

FaceDCAPTCHA: Face detection based color image CAPTCHA

With data theft and computer break-ins becoming increasingly common, there is a great need for secondary authentication to reduce automated attacks while posing a minimal hindrance to legitimate users. CAPTCHA is one of the possible ways to classify human users and automated scripts. Though text-based CAPTCHAs are used in many applications, they pose a challenge due to language dependency. In this paper, we propose a face image-based CAPTCHA as a potential solution. To solve the CAPTCHA, users must correctly identify visually-distorted human faces embedded in a complex background without selecting any non-human faces. The proposed algorithm generates a CAPTCHA that offers better human accuracy and lower machine attack rates compared to existing approaches.

MDLFace: Memorability augmented deep learning for video face recognition

Videos have ample amount of information in the form of frames that can be utilized for feature extraction and matching. However, face images in not all of the frames are “memorable” and useful. Therefore, utilizing all the frames available in a video for recognition does not necessarily improve the performance but significantly increases the computation time. In this research, we present a memorability based frame selection algorithm that enables automatic selection of memorable frames for facial feature extraction and matching. A deep learning algorithm is then proposed that utilizes a stack of denoising autoencoders and deep Boltzmann machines to perform face recognition using the most memorable frames. The proposed algorithm, termed as MDLFace, is evaluated on two publicly available video face databases, Youtube Faces and Point and Shoot Challenge. The results show that the proposed algorithm achieves state-of-the-art performance at low false accept rates.

Group sparse representation based classification for multi-feature multimodal biometrics

Group sparse representation based classification algorithm is proposed for feature-level multimodal biometrics.The algorithm is able to handle missing features in multimodal scenario.Experimental results on WVU and real world LEA databases show efficacy of the proposed algorithm. Multimodal biometrics technology consolidates information obtained from multiple sources at sensor level, feature level, match score level, and decision level. It is used to increase robustness and provide broader population coverage for inclusion. Due to the inherent challenges involved with feature-level fusion, combining multiple evidences is attempted at score, rank, or decision level where only a minimal amount of information is preserved. In this paper, we propose the Group Sparse Representation based Classifier (GSRC) which removes the requirement for a separate feature-level fusion mechanism and integrates multi-feature representation seamlessly into classification. The performance of the proposed algorithm is evaluated on two multimodal biometric datasets. Experimental results indicate that the proposed classifier succeeds in efficiently utilizing a multi-feature representation of input data to perform accurate biometric recognition.

Recognizing composite sketches with digital face images via SSD dictionary

Sketch recognition has important law enforcement applications in detecting and apprehending suspects. Compared to hand drawn sketches, software generated composite sketches are faster to create and require lesser skill sets as well as bring consistency in sketch generation. While sketch generation is one side of the problem, recognizing composite sketches with digital images is another side. This paper presents an algorithm to address the second problem, i.e. matching composite sketches with digital images. The proposed algorithm utilizes a SSD based dictionary generated via 50,000 images from the CMU Multi-PIE database. The gallery-probe feature vectors created using SSD dictionary are matched using GentleBoostKO classifier. The results on extended PRIP composite sketch database show the effectiveness of the proposed algorithm.

Composite sketch recognition using saliency and attribute feedback

Propose composite sketch to photo matching algorithm using visual saliency and combination of texture features.Attributes such as gender and ethnicity information is used to improve recognition performance.Multiple experts such as sketches from different artists are combined for further increasing accuracy. Recent interest and requirement of law enforcement agencies in matching composite sketches with digital images has instigated research in this important face recognition problem. In this paper, we propose feature extraction and matching algorithm using visual saliency and combination of texture features for matching composite sketches with digital photos. The attributes such as gender, ethnicity, and skin color are utilized for re-ordering the ranked list. Further, information from multiple experts such as multiple composite sketch generation tools or artists is combined for improving the matching performance. The results obtained on the extended PRIP database show that the proposed algorithm improves the state-of-art in matching composite sketch and digital face images and yields the rank 50 identification accuracy of 70.3% on a database of 1500 subjects.

FR-CAPTCHA: CAPTCHA based on recognizing human faces.

A Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA) is designed to distinguish humans from machines. Most of the existing tests require reading distorted text embedded in a background image. However, many existing CAPTCHAs are either too difficult for humans due to excessive distortions or are trivial for automated algorithms to solve. These CAPTCHAs also suffer from inherent language as well as alphabet dependencies and are not equally convenient for people of different demographics. Therefore, there is a need to devise other Turing tests which can mitigate these challenges. One such test is matching two faces to establish if they belong to the same individual or not. Utilizing face recognition as the Turing test, we propose FR-CAPTCHA based on finding matching pairs of human faces in an image. We observe that, compared to existing implementations, FR-CAPTCHA achieves a human accuracy of 94% and is robust against automated attacks.

Face recognition CAPTCHA

CAPTCHA is one of the Turing tests used to classify human users and automated scripts. Existing CAPTCHAs, especially text-based CAPTCHAs, are used in many applications, however they pose challenges due to language dependency and high attack rates. In this paper, we propose a face recognition-based CAPTCHA as a potential solution. To solve the CAPTCHA, users must correctly find one pair of human face images, that belong to same subject, embedded in a complex background without selecting any nonface image or impostor pair. The proposed algorithm generates CAPTCHA that offer better human accuracy and lower attack rates compared to existing approaches.

Class sparsity signature based Restricted Boltzmann Machine

Abstract Restricted Boltzmann Machines (RBMs) have been extensively utilized in machine learning as core units in constructing deep learning architectures such as Deep Boltzmann Machines (DBMs) and Deep Belief Networks (DBNs). However, they are prone to overfitting and several regularization techniques have been proposed to mitigate this effect. In this paper, we propose the semi-supervised class sparsity signature based RBM formulation by combining unsupervised generative training of the RBM with a supervised sparsity regularizer. The proposed approach, termed as cssRBM, enforces sparsity at the class level to ensure that coherent and discriminative representations are learnt during training. Combining unsupervised learning with supervised learning allows the model to utilize external training data to learn better generative features while the supervised learning enables fine-tuning for discrimination using the learned features. We construct both DBMs and DBNs with cssRBM units and evaluate the performance on multiple publicly available benchmark datasets. Experiments on the MNIST and CIFAR-10 databases demonstrate that the proposed approaches are comparable with state-of-the-art deep learning architectures in the literature. We also evaluate the performance on one of the most challenging face databases, i.e., the Point and Shoot Challenge dataset. The results show that the proposed approaches improve state-of-the-art results by 15% on the PaSC database.