Graeme A. Jones

Distributed intelligence for multi-camera visual surveillance

Latest advances in hardware technology and state of the art of computer vision and artificial intelligence research can be employed to develop autonomous and distributed monitoring systems. The paper proposes a multi-agent architecture for the understanding of scene dynamics merging the information streamed by multiple cameras. A typical application would be the monitoring of a secure site, or any visual surveillance application deploying a network of cameras. Modular software (the agents) within such architecture controls the different components of the system and incrementally builds a model of the scene by merging the information gathered over extended periods of time. The role of distributed artificial intelligence composed of separate and autonomous modules is justified by the need for scalable designs capable of co-operating to infer an optimal interpretation of the scene. Decentralizing intelligence means creating more robust and reliable sources of interpretation, but also allows easy maintenance and updating of the system. Results are presented to support the choice of a distributed architecture, and to prove that scene interpretation can be incrementally and efficiently built by modular software.

Video-Based Surveillance Systems

The proliferation of cheap sensors and increased processing power has made the acquisition and processing of video information more feasible. Real-time video analysis tasks performing object detection, tracking can increasingly be performed efficiently on standard PCs. Smart cameras are being designed that enable on-camera applications to be designed so that they act as intelligent sensors that provide compressed data or meta event information directly. These advances along with major breakthroughs in communication and Internet technology is making possible real-time video monitoring and communication for a variety of application sectors such as: Industrial automation, Transportation, Automotive, Security /Surveillance, and Communications. The real-time imaging group at SCR is focusing on the development of integrated end-to-end solutions for applications requiring object detection, tracking, and action classification / event analysis from video. This paper will present an overview of our research in statistical methods for realtime video surveillance systems. Solutions for subway/highway monitoring for emergency assistance and resource management, real-time tracking technologies for applications in video conferencing, intelligent video communications, and industrial automation, will be highlighted.

A multi-agent framework for visual surveillance

We describe an architecture for implementing scene understanding algorithms in the visual surveillance domain. To achieve a high level description of events observed by multiple cameras, many inter-related event-driven processes must be executed. We use the agent paradigm to provide a framework in which these processes can be managed. Each camera has an associated agent, which detects and tracks moving regions of interest. This is used to construct and update object agents. Each camera is calibrated so that image co-ordinates can be transformed into ground plane locations. By comparing properties, two object agents can infer that they have the same referent, i.e. that two cameras are observing the same entity, and as a consequence merge identities. Each objects trajectory is classified with a type of activity, with reference to a ground plane agent. We demonstrate objects simultaneously tracked by two cameras, which infer this shared observation. The combination of the agent framework, and visual surveillance application provides an excellent environment for development and evaluation of scene understanding algorithms.

Constraint, Optimization, and Hierarchy

To extract three dimensional data from a pair of images, it is essential to solve the correspondence problem. In the literature, a large number of algorithms have been implemented which differ in the token type, match constraints, and search methods employed. Recently, hierarchical matching schemes have utilized multiple token types of increasing complexity. In previous reviews of stereopsis, no general framework has emerged within which to evaluate all the different contributions. This paper breaks down the correspondence problem into its general components: token type, match constraints, and method employed to encode and search match information. In common with other reported work, matching is cast as an optimization problem, and the definition of match functionals may be separated from the method employed to search the solution space. Within this very general framework, hierarchical matching is discussed at some length including suggestions on how hierarchical constraints may be formally embedded within the matching algorithm. The benefits of the hierarchical approach are illustrated with some examples.

Real-Time Modeling of 3-D Soccer Ball Trajectories From Multiple Fixed Cameras

In this paper, model-based approaches for real-time 3-D soccer ball tracking are proposed, using image sequences from multiple fixed cameras as input. The main challenges include filtering false alarms, tracking through missing observations, and estimating 3-D positions from single or multiple cameras. The key innovations are: 1. incorporating motion cues and temporal hysteresis thresholding in ball detection; 2. modeling each ball trajectory as curve segments in successive virtual vertical planes so that the 3-D position of the ball can be determined from a single camera view; and 3. introducing four motion phases (rolling, flying, in possession, and out of play) and employing phase-specific models to estimate ball trajectories which enables high-level semantics applied in low-level tracking. In addition, unreliable or missing ball observations are recovered using spatio-temporal constraints and temporal filtering. The system accuracy and robustness are evaluated by comparing the estimated ball positions and phases with manual ground-truth data of real soccer sequences.

Multi-camera colour tracking

We propose a colour tracker for use in visual surveillance. The tracker is part of a framework designed to monitor a dynamic scene with more than one camera. Colour tracking complements spatial tracking: it can also be used over large temporal intervals, and between spatially uncalibrated cameras. The colour distributions from objects are modelled, and measures of difference between them are discussed. A context is required for assessing the significance of any difference. It is provided by an analysis of the noise processes: first on the camera capture, then on the underlying variability of the signal. We present results comparing parametric and explicit representations, the inclusion and omission of intensity data, and single and multiple cameras.

Tracking the soccer ball using multiple fixed cameras

This paper demonstrates innovative techniques for estimating the trajectory of a soccer ball from multiple fixed cameras. Since the ball is nearly always moving and frequently occluded, its size and shape appearance varies over time and between cameras. Knowledge about the soccer domain is utilized and expressed in terms of field, object and motion models to distinguish the ball from other movements in the tracking and matching processes. Using ground plane velocity, longevity, normalized size and color features, each of the tracks obtained from a Kalman filter is assigned with a likelihood measure that represents the ball. This measure is further refined by reasoning through occlusions and back-tracking in the track history. This can be demonstrated to improve the accuracy and continuity of the results. Finally, a simple 3D trajectory model is presented, and the estimated 3D ball positions are fed back to constrain the 2D processing for more efficient and robust detection and tracking. Experimental results with quantitative evaluations from several long sequences are reported.

Learning Surveillance Tracking Models for the Self-Calibrated Ground Plane

Tracking strategies usually employ motion and appearance models to locate observations of the tracked object in successive frames. The subsequent model update procedure renders the approach highly sensitive to the inevitable observation and occlusion noise processes. In this work, two robust mechanisms are proposed which rely on knowledge about the ground plane. First a highly constrained bounding box appearance model is proposed which is determined solely from predicted image location and visual motion. Second, tracking is performed on the ground plane enabling global real-world observation and dynamic noise models to be defined. Finally, a novelauto-calibrationprocedureis developedto recoverthe imageto ground plane homographyby simply accumulating event observations.

Classifying surveillance events from attributes and behaviour

In order to develop a high-level description of events unfolding in a typical surveillance scenario, each successfully tracked event must be classified into type and behaviour. In common with a number of approaches this paper employs a Bayesian classifier to determine type from event attribute such as height, width and velocity. The classifier, however, is extended to integrate all available evidence from the entire track. A not untypical Hidden Markov Model approach has been employed to model the common event behaviours typical of a car-park environment. Both techniques have been probabilistically integrated to generate accurate type and behaviour classifications.

An object-based comparative methodology for motion detection based on the F-Measure

The majority of visual surveillance algorithms rely on effective and accurate motion detection. However, most evaluation techniques described in literature do not address the complexity and range of the issues which underpin the design of a good evaluation methodology. In this paper, we explore the problems associated with both the optimising the operating point of any motion detection algorithms and the objective performance comparison of competing algorithms. In particular, we develop an object-based approach based on the F-Measure-a single-valued ROC-like measure which enables a straight-forward mechanism for both optimising and comparing motion detection algorithms. Despite the advantages over pixel-based ROC approaches, a number of important issues associated with parameterising the evaluation algorithm need to be addressed. The approach is illustrated by a comparison of three motion detection algorithms including the well-known Stauffer and Grimson algorithm, based on results obtained on two datasets.