Geoff A. W. West

Policy recognition in the abstract hidden Markov model

In this paper, we present a method for recognising an agents behaviour in dynamic, noisy, uncertain domains, and across multiple levels of abstraction. We term this problem on-line plan recognition under uncertainty and view it generally as probabilistic inference on the stochastic process representing the execution of the agents plan. Our contributions in this paper are twofold. In terms of probabilistic inference, we introduce the Abstract Hidden Markov Model (AHMM), a novel type of stochastic processes, provide its dynamic Bayesian network (DBN) structure and analyse the properties of this network. We then describe an application of the Rao-Blackwellised Particle Filter to the AHMM which allows us to construct an efficient, hybrid inference method for this model. In terms of plan recognition, we propose a novel plan recognition framework based on the AHMM as the plan execution model. The Rao-Blackwellised hybrid inference for AHMM can take advantage of the independence properties inherent in a model of plan execution, leading to an algorithm for online probabilistic plan recognition that scales well with the number of levels in the plan hierarchy. This illustrates that while stochastic models for plan execution can be complex, they exhibit special structures which, if exploited, can lead to efficient plan recognition algorithms. We demonstrate the usefulness of the AHMM framework via a behaviour recognition system in a complex spatial environment using distributed video surveillance data.

Nonparametric segmentation of curves into various representations

This paper describes and demonstrates the operation and performance of an algorithm for segmenting connected points into a combination of representations such as lines, circular, elliptical and superelliptical arcs, and polynomials. The algorithm has a number of interesting properties including being scale invariant, nonparametric, general purpose, and efficient.

Recognizing and monitoring high-level behaviors in complex spatial environments

The recognition of activities from sensory data is important in advanced surveillance systems to enable prediction of high-level goals and intentions of the target under surveillance. The problem is complicated by sensory noise and complex activity spanning large spatial and temporal extents. The paper presents a system for recognizing high-level human activities from multi-camera video data in complex spatial environments. The Abstract Hidden Markov mEmory Model (AHMEM) is used to deal with noise and scalability. The AHMEM is an extension of the Abstract Hidden Markov Model (AHMM) that allows us to represent a richer class of both state-dependent and context-free behaviors. The model also supports integration with low-level sensory models and efficient probabilistic inference. We present experimental results showing the ability of the system to perform real-time monitoring and recognition of complex behaviors of people from observing their trajectories within a real, complex indoor environment.

Hierarchical recognition of intentional human gestures for sports video annotation

We present a novel technique for the recognition of complex human gestures for video annotation using accelerometers and the hidden Markov model. Our extension to the standard hidden Markov model allows us to consider gestures at different levels of abstraction through a hierarchy of hidden states. Accelerometers in the form of wrist bands are attached to humans performing intentional gestures, such as umpires in sports. Video annotation is then performed by populating the video with time stamps indicating significant events, where a particular gesture occurs. The novelty of the technique lies in the development of a probabilistic hierarchical framework for complex gesture recognition and the use of accelerometers to extract gestures and significant events for video annotation.

Tracking and surveillance in wide-area spatial environments using the abstract hidden Markov model

In this paper, we consider the problem of tracking an object and predicting the objects future trajectory in a wide-area environment, with complex spatial layout and the use of multiple sensors/cameras. To solve this problem, there is a need for representing the dynamic and noisy data in the tracking tasks, and dealing with them at different levels of detail. We employ the Abstract Hidden Markov Models (AHMM), an extension of teh well-known Hidden Markov Model (HMM) and a special type of Dynamic Probabilistic Network (DPN), as our underlying representation framework. The AHMM allows us to explicitly encode the hierarchy of connected spatial locations, making it scalable to the size of the environment being modeled. We describe an application for tracking human movement in an office-like spatial layout where the AHMM is used to track and predict the evolution of object trajectories at different levels of detail.

Salience Distance Transforms

Abstract The distance transform has been used in computer vision for a number of applications such as matching and skeletonization. This paper proposes two things: (1) a multiscale distance transform to overcome the need to choose the appropriate scale and (2) the addition of various saliency factors such as edge strength, length, and curvature to the basic distance transform to eliminate the need for (e.g., edge magnitude) thresholds and to improve its effectiveness. Results are presented for applications of matching and snake fitting.

Recognition of human activity through hierarchical stochastic learning

Seeking to extend the functional capability of the elderly, we explore the use of probabilistic methods to learn and recognise human activity in order to provide monitoring support. We propose a novel approach to learning the hierarchical structure of sequences of human actions through the application of the hierarchical hidden Markov model (HHMM). Experimental results are presented for learning and recognising sequences of typical activities in a home.

Modeling edges at subpixel accuracy using the local energy approach

In this paper we describe a new technique for ID and 2D edge feature extraction to subpixel accuracy using edge models and the local energy approach. A candidate edge is modeled as one of a number of parametric edge models, and the fit is refined by a least-squared error fitting technique. >

Recognition of emergent human behaviour in a smart home: A data mining approach

Motivated by a growing need for intelligent housing to accommodate ageing populations, we propose a novel application of intertransaction association rule (IAR) mining to detect anomalous behaviour in smart home occupants. An efficient mining algorithm that avoids the candidate generation bottleneck limiting the application of current IAR mining algorithms on smart home data sets is detailed. An original visual interface for the exploration of new and changing behaviours distilled from discovered patterns using a new process for finding emergent rules is presented. Finally, we discuss our observations on the emergent behaviours detected in the homes of two real world subjects.

Using dynamic time warping for online temporal fusion in multisensor systems

Sensor fusion is concerned with gaining information from multiple sensors by fusing across raw data, features or decisions. Traditionally these fusion processes only concern fusion at specific points in time. However recently, there is a growing interest in inferring the behavioural aspects of environments or objects that are monitored by multisensor systems, rather than just their states at specific points in time. In order to infer environmental behaviours, it may be necessary to fuse data acquired from (i) geographically distributed sensors at specific points of time and (ii) specific sensors over a period of time. Fusing multisensor data over a period of time (also known as Temporal fusion) is a challenging task, since the data to be fused consists of complex sequences that are multi-dimensional, multimodal, interacting, and time-varying in nature. Additionally, performing temporal fusion efficiently in real-time is another challenge due to the large amounts of data to be fused. To address this issue, we propose a robust and efficient framework that uses dynamic time warping (DTW) as the core recognizer to perform online temporal fusion on either the raw data or the features. We evaluate the performance of the online temporal fusion system on two real world datasets: (1) accelerometer data acquired from performing two hand gestures, and (2) a benchmark dataset acquired from carrying a mobile device and performing the predefined user scenarios. Performance results of the DTW-based system are compared with those of a Hidden Markov Model (HMM) based system. The experimental results from both datasets demonstrate that the proposed system outperforms HMM based systems, and has the capability to perform online temporal fusion efficiently and accurately in real-time.