Yogesh Raja

Tracking colour objects using adaptive mixture models

The use of adaptive Gaussian mixtures to model the colour distributions of objects is described. These models are used to perform robust, real-time tracking under varying illumination, viewing geometry and camera parameters. Observed log-likelihood measurements were used to perform selective adaptation.

MODELLING FACIAL COLOUR AND IDENTITY WITH GAUSSIAN MIXTURES

Abstract An integrated system for the acquisition, normalisation and recognition of moving faces in dynamic scenes is introduced. Four face recognition tasks are defined and it is argued that modelling person-specific probability densities in a generic face space using mixture models provides a technique applicable to all four tasks. The use of Gaussian colour mixtures for face detection and tracking is also described. Results are presented using data from the integrated system.

Tracking and segmenting people in varying lighting conditions using colour

Colour cues were used to obtain robust detection and tracking of people in relatively unconstrained dynamic scenes. Gaussian mixture models were used to estimate probability densities of colour for skin, clothing and background. These models were used to detect, track and segment people, faces and hands. A technique for dynamically updating the models to accommodate changes in apparent colour due to varying lighting conditions was used. Two applications are highlighted: (1) actor segmentation for virtual studios, and (2) focus of attention for face and gesture recognition systems. A system implemented on a 200 MHz PC tracks multiple objects in real time.

Colour Model Selection and Adaption in Dynamic Scenes

We use colour mixture models for real-time colour-based object localisation, tracking and segmentation in dynamic scenes. Within such a framework, we address the issues of model order selection, modelling scene background and model adaptation in time. Experimental results are given to demonstrate our approach in different scale and lighting conditions.

Segmentation and Tracking Using Color Mixture Models

A system is described that provides robust and real-time focus-of-attention for tracking and segmentation of multi-coloured objects. Gaussian mixture models were used to estimate the probability densities of object foreground and scene background colours. Tracking was performed by fitting dynamic bounding boxes to image regions of maximum probability. Two scenarios are presented: (1) real-time face tracking based upon a skin colour model and (2) dynamic body segmentation for virtual studios based upon combined foreground and background models.

Object Tracking Using Adaptive Color Mixture Models

The use of adaptive Gaussian mixtures to model the colour distributions of objects is described. These models are used to perform robust, real-time tracking under varying illumination, viewing geometry and camera parameters. Observed log-likelihood measurements were used to perform selective adaptation.

Sparse Multiscale Local Binary Patterns

In a Local Binary Pattern (LBP) representation, circular point features are taken in their entirety as predicates and restricted to unif orm patterns with limited scales of small numbers of features in order to avoid large bin complexity. Such a design cannot fully exploit the discriminat ive capacity of the features available. To address the problem, this paper proposes (1) a pairwise-coupled reformulation of LBP-type classificatio n which involves selecting single-point features for each pair of classes ac ross multiple scales to form compact, contextually-relevant multiscale predicates known as Multiscale Selected Local Binary Features (MSLBF), and (2) a novel binary feature selection procedure, known as Binary Histogram Intersection Minimisation (BHIM) designed to choose features with minimal redundancy. Experiments show the advantages of MSLBF over traditional LBP representation and of BHIM over feature selection schemes such as AdaBoost.

Scalable Multi-camera Tracking in a Metropolis

The majority of work in person re-identification is focused primarily on the matching process at an algorithmic level, from identifying reliable features to formulating effective classifiers and distance metrics in order to improve matching scores on established ‘closed-world’ benchmark datasets of limited scope and size. Very little work has explored the pragmatic and ultimately challenging question of how to engineer working systems that best leverage the strengths and tolerate the weaknesses of the current state of the art in re-identification techniques, and which are capable of scaling to ‘open-world’ operational requirements in a large urban environment. In this work, we present the design rationale, implementational considerations and quantitative evaluation of a retrospective forensic tool known as Multi-Camera Tracking (MCT). The MCT system was developed for re-identifying and back-tracking individuals within huge quantities of open-world CCTV video data sourced from a large distributed multi-camera network encompassing different public transport hubs in a metropolis. There are three key characteristics of MCT, associativity, capacity and accessibility, that underpin its scalability to spatially large, temporally diverse, highly crowded and topologically complex urban environments with transport links. We discuss a multitude of functional features that in combination address these characteristics. We consider computer vision techniques and machine learning algorithms, including relative feature ranking for inter-camera matching, global (crowd-level) and local (person-specific) space–time profiling, attribute re-ranking and machine-guided data mining using a ‘man-in-the-loop’ interactive paradigm. We also discuss implementational considerations designed to facilitate linear scalability to an aribitrary number of cameras by employing a distributed computing architecture. We conduct quantitative trials to illustrate the potential of the MCT system and its performance characteristics in coping with very large-scale open-world multi-camera data covering crowded transport hubs in a metropolis.

Scaling up multi-camera tracking for real-world deployment

A user-assisted multi-camera tracking system employing several key novel methodologies has previously been shown to be highly effective in assisting human users in tracking targets of interest through industry-standard i-LIDS multi-camera benchmark data.1 A prototype system was developed in order to test and evaluate the effectiveness of this approach. In this paper, we develop this system further in order to improve tracking accuracy and further facilitate scalability to arbitrary numbers of camera views across much larger spatial areas and different locations. Specifically, we describe the following three areas of improvement: (1) dynamic learning mechanisms apply user feedback in adapting internal models to improve performance over time; (2) modular design and hardware acceleration techniques are explored with a view to real-time performance, extensive configurability to leverage available hardware and scalability to larger datasets; and (3) re-design of the user interface for deployment as a secure asynchronous remote web-based service. We conduct an extensive evaluation of the system in terms of: (1) tracking performance; and (2) the speed of the system in computation and in usage over a network. We use a newly collected real-world dataset significantly more challenging than i-LIDS, which comprises six cameras covering two London Underground stations. We show that: (1) dynamic learning is effective; (2) the user-assisted paradigm retains its effectiveness with this significantly more challenging dataset; (3) large-scale deployment and real-time computation is feasible due to linear scalability; (4) context-aware user search strategies and external non-visual information can aid search convergence; and (5) storage and querying of meta-data is a bottleneck to be overcome.

User-assisted visual search and tracking across distributed multi-camera networks

Human CCTV operators face several challenges in their task which can lead to missed events, people or associations, including: (a) data overload in large distributed multi-camera environments; (b) short attention span; (c) limited knowledge of what to look for; and (d) lack of access to non-visual contextual intelligence to aid search. Developing a system to aid human operators and alleviate such burdens requires addressing the problem of automatic re-identification of people across disjoint camera views, a matching task made difficult by factors such as lighting, viewpoint and pose changes and for which absolute scoring approaches are not best suited. Accordingly, we describe a distributed multi-camera tracking (MCT) system to visually aid human operators in associating people and objects effectively over multiple disjoint camera views in a large public space. The system comprises three key novel components: (1) relative measures of ranking rather than absolute scoring to learn the best features for matching; (2) multi-camera behaviour profiling as higher-level knowledge to reduce the search space and increase the chance of finding correct matches; and (3) human-assisted data mining to interactively guide search and in the process recover missing detections and discover previously unknown associations. We provide an extensive evaluation of the greater effectiveness of the system as compared to existing approaches on industry-standard i-LIDS multi-camera data.