John R. Sullins

Hand-Drawn Face Sketch Recognition by Humans and a PCA-Based Algorithm for Forensic Applications

Because face sketches represent the original faces in a very concise yet recognizable form, they play an important role in criminal investigations, human visual perception, and face biometrics. In this paper, we compared the performances of humans and a principle component analysis (PCA)-based algorithm in recognizing face sketches. A total of 250 sketches of 50 subjects were involved. All of the sketches were drawn manually by five artists (each artist drew 50 sketches, one for each subject). The experiments were carried out by matching sketches in a probe set to photographs in a gallery set. This study resulted in the following findings: 1) A large interartist variation in terms of sketch recognition rate was observed; 2) fusion of the sketches drawn by different artists significantly improved the recognition accuracy of both humans and the algorithm; 3) human performance seems mildly correlated to that of PCA algorithm; 4) humans performed better in recognizing the caricature-like sketches that show various degrees of geometrical distortion or deviation, given the particular data set used; 5) score level fusion with the sum rule worked well in combining sketches, at least for a small number of artists; and 6) the algorithm was superior with the sketches of less distinctive features, while humans seemed more efficient in handling tonality (or pigmentation) cues of the sketches that were not processed with advanced transformation functions.

Human and Computer Evaluations of Face Sketches with Implications for Forensic Investigations

Because sketches represent the original faces in a much concise yet recognizable form, they play an important role in criminal investigations, human perceptions and biometrics. In this work, we compared the performances of humans and a PCA-based algorithm in recognizing face sketches. A total of 250 sketches of 50 subjects were involved. All sketches were drawn manually by five artists (each artist drew 50 sketches, one for each subject). Experiments were carried out by matching sketches in a probe set to photos in a gallery set. This study resulted in the following findings: (i) A large inter-artist variation in sketch recognition rate was observed; (ii) Fusing sketches from different artists significantly improved the performance; (iii) Human performance seems correlated with that of the algorithm; (iv) The algorithm was superior with sketches of less distinctive features, while humans used tonality (or pigmentation) cues more efficiently.

Face Recognition by Multi-Frame Fusion of Rotating Heads in Videos

This paper presents a face recognition study that implicitly utilizes the 3D information in 2D video sequences through multi-sample fusion. The approach is based on the hypothesis that continuous and coherent intensity variations in video frames caused by a rotating head can provide information similar to that of explicit shapes or range images. The fusion was done on the image level to prevent information loss. Experiments were carried out using a data set of over 100 subjects and promising results have been obtained: (1) under regular indoor lighting conditions, rank one recognition rate increased from 91% using a single frame to 100% using 7-frame fusion; (2) under strong shadow conditions, rank one recognition rate increased from 63% using a single frame to 85% using 7-frame fusion.

A biometric database with rotating head videos and hand-drawn face sketches

The past decade has witnessed a significant progress in biometric technologies, to a large degree, due to the availability of a wide variety of public databases that enable benchmark performance evaluations. In this paper, we describe a new database that includes: (i) Rotating head videos of 259 subjects; (ii) 250 hand-drawn face sketches of 50 subjects. Rotating head videos were acquired under both normal indoor lighting and shadow conditions. Each video captured four expressions: neutral, smile, surprise, and anger. For each subject, video frames of ten pose angles were manually labeled using reference images and empirical rules, to facilitate the investigation of multi-frame fusion. The database can also be used to study 3D face recognition by reconstructing a 3D face model from videos. In addition, this is the only currently available database that has a large number of face sketches drawn by multiple artists. The face sketches are valuable resource for many researches, such as forensic analysis of eyewitness recollection, impact assessment of face degradation on recognition rate, as well as comparative evaluation of sketch recognitions by humans and algorithms.

Recognizing face sketches by a large number of human subjects: A perception-based study for facial distinctiveness

Understanding how humans recognize face sketches drawn by artists is of significant value to both criminal investigators and researchers in computer vision, face biometrics and cognitive psychology. However, large scale experimental studies of hand-drawn face sketches are still very limited in terms of the number of artists, the number of sketches, and the number of human evaluators involved. In this paper, we reported the results of a series of psychological experiments in which 406 volunteers were asked to recognize 250 sketches drawn by 5 different artists. The primary findings are: (i) Sketch quality (artist factor) has a significant effect on human performance. Inter-artist variation as measured by the mean recognition rate can be as high as 31%; (ii) Participants showed a higher tendency to match multiple sketches to one photo than to second-guess their answers. The multi-match ratio seems correlated to the recognition rate, while second-guessing had no significant effect on human performance; (iii) For certain highly recognized faces, their rankings were very consistent using three measuring parameters: recognition rate, multi-match ratio, and second-guess ratio, suggesting that the three parameters could provide valuable information to quantify facial distinctiveness.

Probabilistic Smart Terrain

“Smart terrain” is a very efficient algorithm used in many games (such as The Sims), in which objects that meet needs transmit signals to non-player characters with those needs, influencing the character to move towards those objects. We describe how probabilities can be added to this algorithm, allowing an object to broadcast that it “may” meet a need with a given probability, as well as how expected distances to an object that meets a need can be used to allow the non-player characters to make plausible decisions about which direction to move. We also describe a set of benchmarks for realistic behavior in an uncertain environment, how the algorithm can factor in learned knowledge of whether an object actually does meet a need, and how probabilistic smart terrain can be implemented and integrated into games.

Evaluation of Multi-frame Fusion Based Face Classification Under Shadow

A video sequence of a head moving across a large pose angle contains much richer information than a single-view image, and hence has greater potential for identification purposes. This paper explores and evaluates the use of a multi-frame fusion method to improve face recognition in the presence of strong shadow. The dataset includes videos of 257 subjects who rotated their heads by 0° to 90°. Experiments were carried out using ten video frames per subject that were fused on the score level. The primary findings are: (i) A significant performance increase was observed, with the recognition rate being doubled from 40% using a single frame to 80% using ten frames; (ii) The performance of multi-frame fusion is strongly related to its inter-frame variation that measures its information diversity.

Exploration strategies for learned probabilities in smart terrain

Consider a mobile agent (such as a robot) surrounded by objects that may or may not meet its needs. An important goal of such an agent is to learn probabilities that different types of objects meet needs, based on objects it has previously explored. This requires a rational strategy for determining which objects to explore next based on distances to objects, prevalence of similar objects, and amount of information the agent expects to gain. We define information gain in terms of how additional examples increase the certainty of the probabilities (represented as beta distributions), based on how that certainty reduces future travel time by preventing the agent from moving to objects which do not actually meet needs. This is used to create a smart terrain-based influence map in which objects send signals proportional to their information gain (with inverse falloff over distance) to enable simple agent navigation to those objects.

Multi-agent probabilistic smart terrain

Our previous work has added probabilistic reasoning to the “smart terrain” algorithm commonly used for goal-based navigation in games, allowing targets to have a given probability of meeting a characters goals and navigating based on expected distances until goals are met. We now extend it to enable cooperative behavior by a set of characters seeking the same goal (such as a group of guards searching for the player). Each agent computes a global expected distance until goals are met, in terms of its distance to targets it is moving towards and the distances from other agents to targets it is moving away from, and chooses the direction that minimizes this measure. We demonstrate that the algorithm produces plausible cooperative behavior on a set of benchmark examples.

An empirical comparison of high definition video and regular video in optical flow computation

This paper presents a comparative study of high definition videos and regular videos in the context of optical flow estimation. The hypothesis is that, because of its higher resolution, a high definition video can yield more accurate optical flow data, which is critical for many motion-based researches. The experiments were carried out using videos that captured a wide variety of motions in both natural and indoor settings, which ensures a statistically sound comparison. Analysis through visual examinations and quantitative violation measures indicates that, in general, high definition videos are superior to regular videos, but the presence of fast moving objects could complicate a specific application.