Scott Spurlock

Gamesourcing to acquire labeled human pose estimation data

In this paper, we present a gamesourcing method for automatically and rapidly acquiring labeled images of human poses to obtain ground truth data as input for human pose estimation from 2D images. Typically, these datasets are constructed manually through a tedious process of clicking on joint locations in images. By using a low-cost RGBD sensor, we capture synchronized, registered images, depth maps, and skeletons of users playing a movement-based game and automatically filter the data to keep a subset of unique poses. Using a recently-developed, learning-based human pose estimation method, we demonstrate how data collected in this manner is as suitable for use as training data as existing, manually-constructed data sets.

Dynamic subset selection for multi-camera tracking

While multi-camera methods for object tracking tend to out-perform their single-camera counterparts, the data aggregation schemes can introduce new challenges, such as resource management and algorithm complexity. We present a framework for dynamically choosing the best subset of available cameras for tracking in real-time, which reduces aggregate tracking error and resource consumption and can be applied to a variety of existing base tracking models. We demonstrate on challenging video sequences of players in a basketball game. Our method is able to successfully track targets entering and exiting camera views and through occlusions, and overcome instances of single-view tracking drift.

Multi-view action recognition one camera at a time

For human action recognition methods, there is often a trade-off between classification accuracy and computational efficiency. Methods that include 3D information from multiple cameras are often computationally expensive and not suitable for real-time application. 2D, frame-based methods are generally more efficient, but suffer from lower recognition accuracies. In this paper, we present a hybrid keypose-based method that operates in a multi-camera environment, but uses only a single camera at a time. We learn, for each keypose, the relative utility of a particular viewpoint compared with switching to a different available camera in the network for future classification. On a benchmark multi-camera action recognition dataset, our method outperforms approaches that incorporate all available cameras.

GeoFaceExplorer: exploring the geo-dependence of facial attributes

The images uploaded to social networking websites are a rich source of information about the appearance of people around the world. We present a system, GeoFaceExplorer, for collecting, processing, browsing, and analyzing this data. GeoFaceExplorer allows for the crowdsourced collection of human facial images, as well as automated and interactive visual analysis of the geo-dependence of facial appearance and visual attributes, such as ethnicity, gender, and whether or not a person is wearing glasses. As a case study, automated approaches are applied to detect common facial attributes in a large set of geo-tagged human faces, leading to several analysis results that illuminate the relationship between raw facial appearance, facial attributes, and geographic location. We show how the distribution of these attributes differs in ten major urban areas. Our analysis also shows a similar expected distribution of ethnicity within large urban areas in comparison to manually collected U.S. census data. In addition, by applying automated hierarchical clustering to facial attribute similarity, we find a large degree of overlap between discovered regional clusters and geographical and national boundaries.

An Evaluation of Gamesourced Data for Human Pose Estimation

Gamesourcing has emerged as an approach for rapidly acquiring labeled data for learning-based, computer vision recognition algorithms. In this article, we present an approach for using RGB-D sensors to acquire annotated training data for human pose estimation from 2D images. Unlike other gamesourcing approaches, our method does not require a specific game, but runs alongside any gesture-based game using RGB-D sensors. The automatically generated datasets resulting from this approach contain joint estimates within a few pixel units of manually labeled data, and a gamesourced dataset created using a relatively small number of players, games, and locations performs as well as large-scale, manually annotated datasets when used as training data with recent learning-based human pose estimation methods for 2D images.

Pedestrian Verification for Multi-Camera Detection

In this paper, we introduce an approach to multi-camera, multi-object detection that builds on low-level object localization with the targeted use of high-level pedestrian detectors. Low-level detectors often identify a small number of candidate locations, but suffer from false positives. We introduce a method of pedestrian verification, which takes advantage of geometric and scene information to (1) drastically reduce the search space in both the spatial and scale domains, and (2) select the camera(s) with the highest likelihood of providing accurate high-level detection. The proposed framework is modular and can incorporate a variety of existing detection methods. Compared to recent methods on a benchmark dataset, our method improves detection performance by 2.4 %, while processing more than twice as fast.

Combining automated and interactive visual analysis of biomechanical motion data

We present a framework for combining automated and interactive visual analysis techniques for use on high-resolution biomechanical data. Analyzing the complex 3D motion of, e.g., pigs chewing or bats flying, can be enhanced by providing investigators with a multiview interface that allows interaction across multiple modalities and representations. In this paper, we employ nonlinear dimensionality reduction to automatically learn a low-dimensional representation of the data and hierarchical clustering to learn patterns inherent within the motion segments. Our multi-view framework allows investigators to simultaneously view a low-dimensional embedding, motion segment clustering, and 3D visual representation of the data side-by-side.We describe an application to a dataset containing thousands of frames of high-speed, 3D motion data collected over multiple experimental trials.

Multi-camera head pose estimation using an ensemble of exemplars

We present a method for head pose estimation for moving targets in multi-camera environments. Our approach utilizes an ensemble of exemplar classifiers for joint head detection and pose estimation and provides finer-grained predictions than previous approaches. We incorporate dynamic camera selection, which allows a variable number of cameras to be selected at each time step and provides a tunable trade-off between accuracy and speed. On a benchmark dataset for multi-camera head pose estimation, our method predicts head pan angle with a mean absolute error of ~ 8° for different moving targets.

Discriminative poses for early recognition in multi-camera networks

We present a framework for early action recognition in a multi-camera network. Our approach balances recognition accuracy with speed by dynamically selecting the best camera for classification. We follow an iterative clustering approach to learn sets of keyposes that are discriminative for recognition as well as for predicting the best camera for classification of future frames. Experiments on multi-camera datasets demonstrate the applicability of our view-shifting framework to the problem of early recognition.

Keeping an Eye Out: Real Time, Real World Modeling of Behavior in Health Care Settings

Imagine a health care facility that is able to track and understand the meaningful behaviors of the patients 24 h a day, 365 days a year, understanding individual variation in behavior over both the short and the long term. Now consider the needs of patients with Alzheimers, who typically have trouble with spatial and visual issues. They are sometimes unable to distinguish between a shadow cast on the floor and a step; they can also spend the entire day at the “front door” anticipating arrivals and departures. The families of these patients, to the best of their ability, want to be able to maintain surveillance and understand the changes in the behavior of their parents or spouses. Continuing on the work of the Computing in Place research group that includes faculty with specialties in architecture, computer vision, ubiquitous computing and anthropology, we propose in this paper a new paradigm for intelligent architectural settings. Health care settings, like most architecture, are generally conceptualized as a spatial volume containing human and technical elements. There is an implicit distinction between the active contents and the passive container. From our research group’s expertise in ethnography, we emphasize the importance of meaning to the understanding of behavior, to the idea of place as a construed setting: or, as Clifford Geertz describes it, the difference between “a wink and a blink”. Our new paradigm proposes the creation of “intelligent” architectural settings that capture such meaningful behavior in real time and generate knowledge that is useful both in the real world and in the evaluation of design revisions.