Kathryn Bonnen

Matching Composite Sketches to Face Photos: A Component-Based Approach

The problem of automatically matching composite sketches to facial photographs is addressed in this paper. Previous research on sketch recognition focused on matching sketches drawn by professional artists who either looked directly at the subjects (viewed sketches) or used a verbal description of the subjects appearance as provided by an eyewitness (forensic sketches). Unlike sketches hand drawn by artists, composite sketches are synthesized using one of the several facial composite software systems available to law enforcement agencies. We propose a component-based representation (CBR) approach to measure the similarity between a composite sketch and mugshot photograph. Specifically, we first automatically detect facial landmarks in composite sketches and face photos using an active shape model (ASM). Features are then extracted for each facial component using multiscale local binary patterns (MLBPs), and per component similarity is calculated. Finally, the similarity scores obtained from individual facial components are fused together, yielding a similarity score between a composite sketch and a face photo. Matching performance is further improved by filtering the large gallery of mugshot images using gender information. Experimental results on matching 123 composite sketches against two galleries with 10,123 and 1,316 mugshots show that the proposed method achieves promising performance (rank-100 accuracies of 77.2% and 89.4%, respectively) compared to a leading commercial face recognition system (rank-100 accuracies of 22.8% and 52.0%) and densely sampled MLBP on holistic faces (rank-100 accuracies of 27.6% and 10.6%). We believe our prototype system will be of great value to law enforcement agencies in apprehending suspects in a timely fashion.

Component-Based Representation in Automated Face Recognition

This paper presents a framework for component-based face alignment and representation that demonstrates improvements in matching performance over the more common holistic approach to face alignment and representation. This work is motivated by recent evidence from the cognitive science community demonstrating the efficacy of component-based facial representations. The component-based framework presented in this paper consists of the following major steps: 1) landmark extraction using Active Shape Models (ASM), 2) alignment and cropping of components using Procrustes Analysis, 3) representation of components with Multiscale Local Binary Patterns (MLBP), 4) per-component measurement of facial similarity, and 5) fusion of per-component similarities. We demonstrate on three public datasets and an operational dataset consisting of face images of 8000 subjects, that the proposed component-based representation provides higher recognition accuracies over holistic-based representations. Additionally, we show that the proposed component-based representations: 1) are more robust to changes in facial pose, and 2) improve recognition accuracy on occluded face images in forensic scenarios.

Dynamic mechanisms of visually guided 3D motion tracking

The continuous perception of motion-through-depth is critical for both navigation and interacting with objects in a dynamic three-dimensional (3D) world. Here we used 3D tracking to simultaneously assess the perception of motion in all directions, facilitating comparisons of responses to motion-through-depth to frontoparallel motion. Observers manually tracked a stereoscopic target as it moved in a 3D Brownian random walk. We found that continuous tracking of motion-through-depth was selectively impaired, showing different spatiotemporal properties compared with frontoparallel motion tracking. Two separate factors were found to contribute to this selective impairment. The first is the geometric constraint that motion-through-depth yields much smaller retinal projections than frontoparallel motion, given the same object speed in the 3D environment. The second factor is the sluggish nature of disparity processing, which is present even for frontoparallel motion tracking of a disparity-defined stimulus. Thus, despite the ecological importance of reacting to approaching objects, both the geometry of 3D vision and the nature of disparity processing result in considerable impairments for tracking motion-through-depth using binocular cues.NEW & NOTEWORTHY We characterize motion perception continuously in all directions using an ecologically relevant, manual target tracking paradigm we recently developed. This approach reveals a selective impairment to the perception of motion-through-depth. Geometric considerations demonstrate that this impairment is not consistent with previously observed spatial deficits (e.g., stereomotion suppression). However, results from an examination of disparity processing are consistent with the longer latencies observed in discrete, trial-based measurements of the perception of motion-through-depth.

Beyond trial-based paradigms: Continuous behavior, ongoing neural activity, and natural stimuli

The vast majority of experiments examining perception and behavior are conducted using experimental paradigms that adhere to a rigid trial structure: each trial consists of a brief and discrete series of events and is regarded as independent from all other trials. The assumptions underlying this structure ignore the reality that natural behavior is rarely discrete, brain activity follows multiple time courses that do not necessarily conform to the trial structure, and the natural environment has statistical structure and dynamics that exhibit long-range temporal correlation. Modern advances in statistical modeling and analysis offer tools that make it feasible for experiments to move beyond rigid independent and identically distributed trial structures. Here we review literature that serves as evidence for the feasibility and advantages of moving beyond trial-based paradigms to understand the neural basis of perception and cognition. Furthermore, we propose a synthesis of these efforts, integrating the characterization of natural stimulus properties with measurements of continuous neural activity and behavioral outputs within the framework of sensory-cognitive-motor loops. Such a framework provides a basis for the study of natural statistics, naturalistic tasks, and/or slow fluctuations in brain activity, which should provide starting points for important generalizations of analytical tools in neuroscience and subsequent progress in understanding the neural basis of perception and cognition.

Continuous Psychophysics: measuring visual sensitivity by dynamic target tracking

We introduce a novel framework for estimating visual sensitivity using a continuous target-tracking task in concert with a dynamic internal model of human visual performance. In our main experiment, observers used a mouse cursor to track the center of a 2D Gaussian luminance target as it moved in a Brownian walk in a field of dynamic Gaussian luminance noise. To estimate visual sensitivity, we fit a Kalman filter to the tracking data assuming that humans behave roughly as Bayesian ideal observers. Such observers optimally combine prior information with noisy observations to produce an estimate of target location at each point in time. We found that estimates of human sensory noise obtained from the Kalman filter model were highly correlated with traditional psychophysical measures of human sensitivity (R2 > 0.97). Because data can be collected at the display frame rate, the amount of time required to measure sensitivity is greatly reduced relative to traditional psychophysics. While our modeling framework provides principled estimates of sensitivity that are directly comparable with those from traditional psychophysics, easily-computed summary statistics based on cross-correlograms of the tracking data also accurately predict relative sensitivity, and are thus good empirical substitutes for the more computationally-intensive Kalman filter fitting. As a second example, we show contrast sensitivity functions quickly determined using target tracking. Finally, we show that psychophysical reverse-correlation can also be quickly done via tracking. We conclude that dynamic target tracking is a viable and faster alternative to traditional psychophysical methods in many situations. Meeting abstract presented at VSS 2015.

Two eyes are identical to one: three-dimensional motor tracking of visual targets

The threshold for detecting stereoscopic motion is notably higher than that for detecting the equivalent fronto-parallel motion. Thus, for small excursions, an apparently stationary line can easily be seen as moving by simply closing one eye - two eyes being less sensitive than one (Tyler, 1971). Here, we used a 3D tracking paradigm to examine motion perception with a continuous motor task. Observers used a cursor to track a target as it moved in a 3D Gaussian random walk. In the main experiment, each eye saw a veridical 2D projection of a square target and cursor moving in 3D. A Leap Motion controller was used to move the cursor by simply pointing at the target location. The controller was calibrated such that there was a one-to-one mapping between finger movement and simulated cursor movement. Tracking data were analyzed by computing the cross-correlograms (CCGs) between the target and cursor velocities for each of the three cardinal motion directions, and assessing the lag, peak correlation, and width of the CCGs. Responses to the depth component of the motion were markedly weaker and more sluggish than responses to the fronto-parallel directions. Further experiments revealed that these depth responses were identical when either only horizontal disparity information was available or when a disparity-only target was viewed monocularly. Our primary finding was that depth tracking is worse than fronto-parallel tracking, but this was simply due to the much smaller amplitudes of projected fronto-parallel motion that motion-in-depth produces. We did not find evidence for the stereo-motion suppression observed with traditional psychophysics, nor did we find a need to invoke a specific disparity processing mechanism. Rather, continuous tracking behavior is parsimoniously explained by reliance on the 2D projections of 3D motion. Meeting abstract presented at VSS 2015.