Dashan Gao

Multi-camera person tracking in crowded environments

Reliably tracking people throughout a camera network is an important capability in areas such as law enforcement, homeland protection, and healthcare. In this paper we will provide an overview of GE Global Researchs tracking system and evaluate it against a subset of the PETS 2009 dataset. The tasks of counting, density estimation, multiperson tracking as well as the tracking of selected individuals will be addressed. Qualitative and quantitative performance results will be reported.

Learning to recognize people in a smart environment

In this paper, we address the problem of online learning to recognize people from visual appearances, a prerequisite step towards building a fully intelligent and context-aware smart environment. While the trajectories of tracked individuals are responsible for producing samples to the appearance signature learning process, it is highly risky to directly label these appearance samples with tracker IDs, due to possible tracker switches and temporary tracker losses. Through the exploration of trajectory fidelity in terms of temporal continuity and spatial locality, we show that the side information from tracking, in the form of pairwise constraints, such as “must-link” and “cannot-link”, could significantly benefit signature learning. Furthermore, to learn and update an online identity signature pool, a two-step approach is proposed: 1) a data clustering step based on spectral kernel learning with pairwise constraints, and 2) a large-margin based discriminative signature model learning step. A real-world setup in a smart office environment is used to evaluate the performance of the learning paradigm. Consistent recognition of individuals from live videos verifies the efficacy and effectiveness of our proposal.

Action Recognition from Experience

A reinforcement learning model, which allows for an agent to interact with a simulated 3D learning environment under the initial guidance of an all knowing oracle is proposed. Methods are presented that allow the agent to learn how to perform a set of task oriented actions. It is then hypothesized that the ability to recognize an action may in fact be a byproduct of learning how to perform an action. Evidence supporting this conjecture is presented using both simulated and real world imagery.

A systems level approach to perimeter protection

Effective perimeter protection mechanisms for industrial sites and critical infrastructure must contend with a large variety of potential threats as well as with the fact that normal site activity can be both complex and diverse. This paper documents the development of a system level approach capable of functioning under such challenging conditions. A multi-view tracking system is used to provide real-time site wide trajectories of all observed individuals. A Radar-based system is also used for tracking if and when camera coverage of various regions is not available. Track information is then analyzed with respect to articulated motion analysis, complex event analysis and normalcy analysis. In addition, object recognition is used to classify left behind objects using high resolution PTZ imagery. A real-time integrated version of this comprehensive approach to perimeter protection was deployed using a single standard off-the-shelf desktop computer.

Action based video summarization for convenience stores

Given the size and congestion associated with a standard convenience store, it is possible to achieve automated site-wide tracking for each customer via a network of in-store video cameras. For the purposes of reducing customer related shrinkage (theft), it may be advantageous to produce summary videos of the activities associated with each customer. As the customer approaches an exit or checkout station, the cashier could quickly review the associated summery video for the purposes of determining whether or not a theft has taken place. Such a summery video would be automatically generated from clips produced by a network of in-store cameras. The value of the summery videos can be determined based on their brevity and the cashiers ability to detect the presence or absence of a theft event. To this end we propose that a site-wide tracking system coupled with a clip nomination process that is based on a set of action saliency measures be used to produce the desired summery videos.

AVSS 2011 demo session: A systems level approach to perimeter protection

Summary form only given. The rapid evolution of tools and software systems to design experiments, automatically monitor, collect and warehouse large amounts of data, from applications such as life sciences and industrial processes has resulted in a new paradigm shift. This change of paradigm is so fast that some of the practices for optimization and management of these processes that were valid only 5–10 years ago may no longer be fully acceptable or sufficient for todays business optimization and management. This has a direct influence on the best practices for knowledge discovery and management of the discovered knowledge in real-world data mining applications. Establishing and managing a real-world data mining project in any domain, in particular in todays life science industry, is not a trivial task. A few approaches have been proposed in the literature. However, initiation and successful management of such efforts may depend on where a given case study fits in the overall classification of data mining approaches. Todays knowledge discovery from data can be classified in several ways: (i) data mining on engineered systems (e.g. complex equipment) or systems designed by nature (e.g. life sciences), (ii) explanatory or predictive data mining, (iii) data mining from static data (e.g. data warehouse) or dynamic data (e.g. data streams), (iv) user operated or automated data mining. There could still be other ways to classify data mining applications. This talk provides an overview of the above listed knowledge discovery applications. We provide examples where we demonstrate how small or large amounts of data, when understood from a real-world data mining point of view and the required data is properly integrated, can result in novel knowledge discovery case studies. We explain motivations and challenges of establishing real-world dat