Yingbo Zhou

Deep Secure Encoding for Face Template Protection

In this paper we present a framework for secure identification using deep neural networks, and apply it to the task of template protection for face password authentication. We use deep convolutional neural networks (CNNs) to learn a mapping from face images to maximum entropy binary (MEB) codes. The mapping is robust enough to tackle the problem of exact matching, yielding the same code for new samples of a user as the code assigned during training. These codes are then hashed using any hash function that follows the random oracle model (like SHA-512) to generate protected face templates. The algorithm makes no unrealistic assumptions and offers high template security, cancelability, and matching performance comparable to the state-of-the-art. The efficacy of the approach is shown on CMU-PIE, Extended Yale B, and Multi-PIE face databases. We achieve high (~ 95%) genuine accept rates (GAR) at zero false accept rate (FAR) while maintaining a high level of template security.

An ontology for generating descriptions about natural outdoor scenes

We present an image ontology useful for generating descriptive texts about highly unconstrained natural outdoor images, taken under many different conditions - lighting, varying viewpoints, etc. The ontology pre-defines the visual content we are interested in describing. Unlike other image description techniques, which tend to be purely object-centric, we utilize a holistic scene ontology for description. The primitive units defined by the ontology are extracted from an image via stochastic processes. Similarly, attributes of the units, also specified by the ontology, are evaluated. Binary and tertiary relationships between relevant primitives are also evaluated. The values, attributes and relationships of the primitive units are combined, based on a pre-defined set of production rules, in such a way as to generate rich, descriptive sentences about the image. Evaluation strategies are implemented to quantitatively test the meaningfulness of the generated sentences. Results indicate that the proposed scene ontology aids in generating highly relevant, naturalistic and meaningful sentences describing natural outdoor images.

Shared features for multiple face-based biometrics

People often make instant judgments about the age, health, mood, personality and character of others based on their facial features. It is not clear from a cognitive aspect whether these different traits require different sets of features or a shared feature set. Till date, much of the computational face image analysis work such as face recognition, face-based deceit detection, age estimation, gender estimation, etc, have been developed on datasets and features specific only to the problem-at-hand. In this paper, we explore an approach for performing face image analysis using a shared set of features for different tasks. By performing unsupervised learning on a large collection of face images, we learn the parameters of a probabilistic generative face model, and by projecting a new face image into this probabilistic space, we obtain a set of face features not created for any specific face analysis tasks. We investigate the use of such shared features and successfully predict the level of attractiveness, whether or not a face is made-up, the facial expression, and the gender of a person, given any arbitrary, near-frontal face image.

Learning Representations for Cryptographic Hash Based Face Template Protection

In this chapter, we discuss the impact of recent advancements in deep learning in the field of biometric template protection. The representation learning ability of neural networks has enabled them to achieve state-of-the-art results in several fields, including face recognition. Consequently, biometric authentication using facial images has also benefited from this, with deep convolutional neural networks pushing the matching performance numbers to all time highs. This chapter studies the ability of neural networks to learn representations which could benefit template security in addition to matching accuracy. Cryptographic hashing is generally considered most secure form of protection for the biometric data, but comes at the high cost of requiring an exact match between the enrollment and verification templates. This requirement generally leads to a severe loss in matching performance (FAR and FRR) of the system. We focus on two relatively recent face template protection algorithms that study the suitability of representations learned by neural networks for cryptographic hash based template protection. Local region hashing tackles hash-based template security by attempting exact matches between features extracted from local regions of the face as opposed to the entire face. A comparison of the suitability of different feature extractors for the task is presented and it is found that a representation learned by an autoencoder is the most promising. Deep secure encoding tackles the problem in an alternative way by learning a robust mapping of face classes to secure codes which are then hashed and stored as the secure template. This approach overcomes several pitfalls of local region hashing and other face template algorithms. It also achieves state-of-the-art matching performance with a high standard of template security.

Labeling Spain With Stanford

We present an end-to-end framework for outdoor scene region decomposition, learned on a small set of randomly selected images that generalizes well to multiple data sets containing images from around the world. We discuss the different aspects of the framework especially a generalized variational inference method with better approximations to the true marginals of a graphical model. Experimentally, we explain why the framework is robust and performs competitively on many diverse scene data sets, including several unseen scene types. We have obtained high pixel-level accuracies ( ≈ 80%) in three of the four data sets, which include a benchmark data set known as the Stanford background data set. Our model obtained over 70% accuracy on the fourth data set, which contained a number of indoor and close-up images that are significantly different from our training examples.