Hilary Buxton

Visual surveillance in a dynamic and uncertain world

Advanced visual surveillance systems not only need to track moving objects but also interpret their patterns of behaviour. This means that solving the information integration problem becomes very important. We use conceptual knowledge of both the scene and the visual task to provide constraints. We also control the system using dynamic attention and selective processing. Bayesian belief networks support this and allow us to model dynamic dependencies between parameters involved in visual interpretation. We illustrate these arguments using experimental results from a traffic surveillance application. In particular, we demonstrate that using expectations of object trajectory, size and speed for the particular scene improves robustness and sensitivity in dynamic tracking and segmentation. We also demonstrate behavioral evaluation under attentional control using a combination of a static BBN TASKNET and dynamic network. The causal structure of these networks provides a framework for the design and integration of advanced vision systems.

A computational model of the analysis of some first-order and second-order motion patterns by simple and complex cells.

Although spatio-temporal gradient schemes are widely used in the computation of image motion, algorithms are ill conditioned for particular classes of input. This paper addresses this problem. Motion is computed as the space-time direction in which the difference in image illuminance from the local mean is conserved. This method can reliably detect motion in first-order and some second-order motion stimuli. Components of the model can be identified with directionally asymmetric and directionally selective simple cells. A stage in which we compute spatial and temporal derivatives of the difference between image illuminance and the local mean illuminance using a truncated Taylor series gives rise to a phase-invariant output reminiscent of the response of complex cells.

Learning and understanding dynamic scene activity: a review

We are entering an era of more intelligent cognitive vision systems. Such systems can analyse activity in dynamic scenes to compute conceptual descriptions from motion trajectories of moving people and the objects they interact with. Here we review progress in the development of flexible, generative models that can explain visual input as a combination of hidden variables and can adapt to new types of input. Such models are particularly appropriate for the tasks posed by cognitive vision as they incorporate learning as well as having sufficient structure to represent a general class of problems. In addition, generative models explain all aspects of the input rather than attempting to ignore irrelevant sources of variation as in exemplar-based learning. Applications of these models in visual interaction for education, smart rooms and cars, as well as surveillance systems is also briefly reviewed.

Computation of optic flow from the motion of edge features in image sequences

Abstract Three-dimensional scene information relating to the depth and orientations of the visible surfaces may be obtained from the optic flow field in time varying imagery. The computation of optic flow is therefore an important step in computer vision. We review our work on calculating optic flow from the motion of edge features in an image sequence. The method is based on a spatiotemporal extension of the Marr-Hildreth edge detection scheme that smooths the data over time as well as over the spatial, image, coordinates. Edge features are defined as the zero crossings of the resultant convolution signal and their motion obtained to subpixel accuracy by a leastsquares interpolation. The details of the method are described and some computational examples are given, including a brief description of how the algorithms may be implemented on a single-instruction multiple-data machine. Some novel effects associated with the choice of metric in the spatiotemporal convolution operator that may be useful for obtaining the time to contact (depth) of objects in the periphery of the field of view are discussed.

Conceptual Descriptions from Monitoring and Watching Image Sequences.

Abstract This paper contrasts two ways of forming conceptual descriptions from images. The first, called “monitoring”, just follows the flow of data from images to interpretation, having little need for top-level control. The second, called “watching”, emphasizes the use of top-level control and actively selects evidence for task-based descriptions of the dynamic scenes. Here we look at the effect this has on forming conceptual descriptions. First, we look at how motion verbs and the perception of events contribute to an effective representational scheme. Then we go on to discuss illustrated examples of computing conceptual descriptions from images in our implementations of the monitoring and watching systems. Finally, we discuss future plans and related work.

Monocular depth perception from optical flow by space time signal processing

A theory of monocular depth determination is presented. The effect of finite temporal resolution is incorporated by generalizing the Marr–Hildreth edge detected operator –∇ 2G(r) where ∇2 is the Laplacian and G (r) is a two-dimensional Gaussian. The constraint that the edge detection operator in space–time should produce zero-crossings at the same place in different channels, i. e. at different resolutions of the Gaussian, led to the conclusion that the Marr–Hildreth operator should be replaced by – □2G(r, t) where □2 is the d’Alembertian ∇2 – (1/u2)(∂2/∂t2) and G(r, t) is a Gaussian in space–time. To ensure that the locations of the zero-crossings are independent of the channel width, G(r, t) has to be isotropic in the sense that the velocity u appearing in the defintion of the d’Alembertian must also be used to relate the scales of length and time in G. However, the new operatior –□2G(r, t) produces two types of zero-crossing for each isolated edge feature in the image I(r, t). One of these, termed the ‘static edge’, corresponds to the position of the image edge at time t as defined by ∇2I(r, t) = 0; the other, called a ‘depth zero’, depends only on the relative motion of the observer and object and is usually found only in the periphery of the field of view. When an edge feature is itself in the periphery of the visual field and these zeros coincide, there is an additional cross-over effect. It is shown how these zero-crossings may be used to infer the depth of an object when the observer and object are in relative motion. If an edge feature is near the centre of the image (i. e. near the focus of expansion), the spatial and temporal slopes of the zeros crossing at the static edge may be used to infer the depth, but, if the edge feature is in the periphery of the image, the cross-over effect enables the depth to be obtained immediately. While the former utilizes sharp spatial and temporal resolution to give detailed three-dimensional information, the cross-over effect relies on longer integration times to give a direct measure of the time-to-contact. We propose that both mechanisms could be used to extract depth information in computer vision systems and speculate on how our theory could be used to model depth perception in early visual processing in humans where there is evidence of both monocular perception of the environment in depth and of looming detection in the periphery of the field of view. In addition it is shown how a number of previous models are included in our theory, in particular the directional sensor proposed by Marr & Ullman and a method of depth determination proposed by Prazdny.

Face recognition using radial basis function neural networks

This paper presents experiments using an adaptive learning compo nent based on Radial Basis Function RBF networks to tackle the unconstrained face recognition problem using low resolution video in formation Firstly we performed preprocessing of face images to mimic the e ects of receptive eld functions found at various stages of the hu man vision system These were then used as input representations to RBF networks that learnt to classify and generalise over di erent views for a standard face recognition task Two main types of preprocessing Di erence of Gaussian ltering and Gabor wavelet analysis are com pared Secondly we provide an alternative face unit RBF network model that is suitable for large scale implementations by decomposi tion of the network which avoids the unmanagability of neural net works above a certain size Finally we show the D shift scale and y axis rotation invariance properties of the standard RBF network Quantitative and qualitative di erences in these schemes are described and conclusions drawn about the best approach for real applications to address the face recognition problem using low resolution images

Learning identity with radial basis function networks

Radial basis function (RBF) networks are compared with other neural network techniques on a face recognition task for applications involving identification of individuals using low-resolution video information. The RBF networks are shown to exhibit useful shift, scale and pose (y-axis head rotation) invariance after training when the input representation is made to mimic the receptive field functions found in early stages of the human vision system. In particular, representations based on difference of Gaussian (DoG) filtering and Gabor wavelet analysis are compared. Extensions of the techniques to the case of image sequence analysis are described and a time delay (TD) RBF network is used for recognising simple movement-based gestures. Finally, we discuss how these techniques can be used in real-life applications that require recognition of faces and gestures using low-resolution video images.

Analogical representation of space and time

Abstract In computer vision, the usual level of ‘interpretation’ is the identification of the objects in the image. In this paper, we extend the level of interpretation to include spatial event detection using a knowledge base for a known scene. This will allow us to formulate a computational theory for forming conceptual descriptions about the behaviours of the objects. Here we describe an analogical representation of space and time that supports the formation of an event level description giving a local interpretation. These, in turn, can be used to form the global level conceptual descriptions. The analogical representation provides a characterization of the real world making explicit the behavioural data that is usually implicit. The knowledge base includes information about the scene layout and expected behaviours of the scene objects. Some of this behavioural information is spatially invariant and is used to contextually index the scene. The anlogical reasoning uses this contextual indexing of the spatial knowledge to provide a behavioural interpretation that describes what the objects are doing in the scene. We illustrate the approach with examples of road traffic surveillance at a German roundabout.

Error Thresholds and Their Relation to Optimal Mutation Rates

The error threshold -- a notion from molecular evolution -- is the critical mutation rate beyond which structures obtained by the evolutionary process are destroyed more frequently than selection can reproduce them. We argue that this notion is closely related to the more familiar notion of optimal mutation rates in Evolutionary Algorithms (EAs). This correspondence has been intuitively perceived before ([9], [11]). However, no previous study, to our knowledge, has been aimed at explicitly testing the hypothesis of such a relationship. Here we propose a methodology for doing so. Results on a restricted range of fitness landscapes suggest that these two notions are indeed correlated. There is not, however, a critically precise optimal mutation rate but rather a range of values producing similar near-optimal performance. When recombination is used, both error thresholds and optimal mutation ranges are lower than in the asexual case. This knowledge may have both theoretical relevance in understanding EA behavior, and practical implications for setting optimal values of evolutionary parameters.