Dipan K. Pal

NIR-VIS heterogeneous face recognition via cross-spectral joint dictionary learning and reconstruction

A lot of real-world data is spread across multiple domains. Handling such data has been a challenging task. Heterogeneous face biometrics has begun to receive attention in recent years. In real-world scenarios, many surveillance cameras capture data in the NIR (near infrared) spectrum. However, most datasets accessible to law enforcement have been collected in the VIS (visible light) domain. Thus, there exists a need to match NIR to VIS face images. In this paper, we approach the problem by developing a method to reconstruct VIS images in the NIR domain and vice-versa. This approach is more applicable to real-world scenarios since it does not involve having to project millions of VIS database images into learned common subspace for subsequent matching. We present a cross-spectral joint ℓ0 minimization based dictionary learning approach to learn a mapping function between the two domains. One can then use the function to reconstruct facial images between the domains. Our method is open set and can reconstruct any face not present in the training data. We present results on the CASIA NIR-VIS v2.0 database and report state-of-the-art results.

Hallucinating the Full Face from the Periocular Region via Dimensionally Weighted K-SVD

Identifying a suspect wearing a mask (where only the suspects periocular region is visible) is one of the toughest real-world challenges in biometrics that exist. In this paper, we present a practical method to hallucinate the full frontal face given only the periocular region of a face. This is an important problem faced in many law-enforcement applications on almost a daily basis. In such real-world situations, we only have access to the periocular region of a persons face. Unfortunately commercial matchers are unable to process these images successfully. We propose in this paper, an approach that will reconstruct the entire frontal face using just the periocular region. We empirically show that our reconstruction technique, based on a modified sparsifying dictionary learning algorithm, can effectively reconstruct faces which we show are actually very similar to the original ground-truth faces. Further, our method is open set, thus can reconstruct any face not seen in training. We show the real-world applicability of method by benchmarking face verification results using the reconstructed faces to show that they still match competitively compared to the original faces when evaluated under a large-scale face verification protocol such as NISTs FRGC protocol where over 256 million face matches are made.

Driver cell phone usage detection on Strategic Highway Research Program (SHRP2) face view videos

The harmful effects of cell phone usage on driver behavior have been well investigated and the growing problem has motivated several several research efforts aimed at developing automated cell phone usage detection systems. Computer vision based approaches for dealing with this problem have only emerged in recent years. In this paper, we present a vision based method to automatically determine if a driver is holding a cell phone close to one of his/her ears (thus keeping only one hand on the steering wheel) and quantitatively demonstrate the methods efficacy on challenging Strategic Highway Research Program (SHRP2) face view videos from the head pose validation data that was acquired to monitor driver head pose variation under naturalistic driving conditions. To the best of our knowledge, this is the first such evaluation carried out using this relatively new data. Our approach utilizes the Supervised Descent Method (SDM) based facial landmark tracking algorithm to track the locations of facial landmarks in order to extract a crop of the region of interest. Following this, features are extracted from the crop and are classified using previously trained classifiers in order to determine if a driver is holding a cell phone. We adopt a through approach and benchmark the performance obtained using raw pixels and Histogram of Oriented Gradients (HOG) features in combination with various classifiers.

Investigating the feasibility of image-based nose biometrics

The search for new biometrics is never ending. In this work, we investigate the use of image based nasal features as a biometric. In many real-world recognition scenarios, partial occlusions on the face leave the nose region visible (e.g. sunglasses). Face recognition systems often fail or perform poorly in such settings. Furthermore, the nose region naturally contain more invariance to expression than features extracted from other parts of the face. In this study, we extract discriminative nasal features using Kernel Class-Dependence Feature Analysis (KCFA) based on Optimal Trade-off Synthetic Discriminant Function (OTSDF) filters. We evaluate this technique on the FRGC ver2.0 database and the AR Face database, training and testing exclusively on nasal features and have compared the results to the full face recognition using KCFA features. We find that the between-subject discriminability in nasal features is comparable to that found in facial features. This shows that nose biometrics have a potential to support and boost biometric identification, that has largely been under utilized. Moreover, our extracted KCFA nose features have significantly outperformed the PittPatt face matcher which works with the original JPEG images on the AR facial occlusion database. This shows that nose biometrics can be used as a stand-alone biometric trait when the subjects are under occlusions.

A preliminary investigation on the sensitivity of COTS face recognition systems to forensic analyst-style face processing for occlusions

Modern day law enforcement banks heavily on the use of commercial off-the-shelf (COTS) face recognition systems (FRS) as a tool for biometric evaluation and identification. However, in many real-world scenarios, when the face of an individual is occluded or degraded in some way, commercial recognition systems fail to accept the face for evaluation or simply return unusable matched faces. In these kinds of cases, forensic experts rely on image processing techniques and tools, to make the face fit to be processed by the commercial recognition systems (e.g. use partial face images from another subject to fill in the occluded parts of the face of interest, or have a tight crop around the face). In this study, we evaluate the sensitivity of commercial recognition systems to such forensic techniques. More specifically, we study the change in the rank-1 identification result that is caused by forensic processing of faces-of-interest that are unusable by the commercial recognition systems. Further, forensic processing of such faces is more of an art and it is extremely difficult to process faces consistently such that there is a predictable effect on the rank-n identification result. This study is meant to serve as an evaluation of the effect of a few forensic techniques intended to allow commercial recognition systems to process and match face images that were otherwise unusable. Our results indicate that COTS FRS can be sensitive to the subjectivity in facial part swapping and cropping, resulting in inconsistencies in the identification rankings and similarity scores.

Stochastic CoSaMP: Randomizing Greedy Pursuit for Sparse Signal Recovery

In this paper, we formulate the K-sparse compressed signal recovery problem with the

Discriminative Invariant Kernel Features: A Bells-and-Whistles-Free Approach to Unsupervised Face Recognition and Pose Estimation

L_0

Ring Loss: Convex Feature Normalization for Face Recognition

norm within a Stochastic Local Search SLS framework. Using this randomized framework, we generalize the popular sparse recovery algorithm CoSaMP, creating Stochastic CoSaMP StoCoSaMP. Interestingly, our deterministic worst case analysis shows that under the Restricted Isometric Property RIP, even a purely random version of StoCoSaMP is guaranteed to recover a notion of strong components of a sparse signal, thereby leading to support convergence. Empirically, we find that StoCoSaMP outperforms CoSaMP, both in terms of signal recoverability and computational cost, on different problems with upi¾źto 1 million dimensions. Further, StoCoSaMP outperforms several other popular recovery algorithms, including StoGradMP and StoIHT, on large real-world gene-expression datasets.