David Windridge

A Linguistic Feature Vector for the Visual Interpretation of Sign Language

This paper presents a novel approach to sign language recognition that provides extremely high classification rates on minimal training data. Key to this approach is a 2 stage classification procedure where an initial classification stage extracts a high level description of hand shape and motion. This high level description is based upon sign linguistics and describes actions at a conceptual level easily understood by humans. Moreover, such a description broadly generalises temporal activities naturally overcoming variability of people and environments. A second stage of classification is then used to model the temporal transitions of individual signs using a classifier bank of Markov chains combined with Independent Component Analysis. We demonstrate classification rates as high as 97.67% for a lexicon of 43 words using only single instance training outperforming previous approaches where thousands of training examples are required.

Detection of Face Spoofing Using Visual Dynamics

Rendering a face recognition system robust is vital in order to safeguard it against spoof attacks carried out using printed pictures of a victim (also known as print attack) or a replayed video of the person (replay attack). A key property in distinguishing a live, valid access from printed media or replayed videos is by exploiting the information dynamics of the video content, such as blinking eyes, moving lips, and facial dynamics. We advance the state of the art in facial antispoofing by applying a recently developed algorithm called dynamic mode decomposition (DMD) as a general purpose, entirely data-driven approach to capture the above liveness cues. We propose a classification pipeline consisting of DMD, local binary patterns (LBPs), and support vector machines (SVMs) with a histogram intersection kernel. A unique property of DMD is its ability to conveniently represent the temporal information of the entire video as a single image with the same dimensions as those images contained in the video. The pipeline of DMD + LBP + SVM proves to be efficient, convenient to use, and effective. In fact only the spatial configuration for LBP needs to be tuned. The effectiveness of the methodology was demonstrated using three publicly available databases: (1) print-attack; (2) replay-attack; and (3) CASIA-FASD, attaining comparable results with the state of the art, following the respective published experimental protocols.

An evaluation of bags-of-words and spatio-temporal shapes for action recognition

Bags-of-visual-Words (BoW) and Spatio-Temporal Shapes (STS) are two very popular approaches for action recognition from video. The former (BoW) is an un-structured global representation of videos which is built using a large set of local features. The latter (STS) uses a single feature located on a region of interest (where the actor is) in the video. Despite the popularity of these methods, no comparison between them has been done. Also, given that BoW and STS differ intrinsically in terms of context inclusion and globality/locality of operation, an appropriate evaluation framework has to be designed carefully. This paper compares these two approaches using four different datasets with varied degree of space-time specificity of the actions and varied relevance of the contextual background. We use the same local feature extraction method and the same classifier for both approaches. Further to BoW and STS, we also evaluated novel variations of BoW constrained in time or space. We observe that the STS approach leads to better results in all datasets whose background is of little relevance to action classification.

A morphologically optimal strategy for classifier combination: multiple expert fusion as a tomographic process

We specify an analogy in which the various classifier combination methodologies are interpreted as the implicit reconstruction, by tomographic means, of the composite probability density function spanning the entirety of the pattern space, the process of feature selection in this scenario amounting to an extremely bandwidth-limited Radon transformation of the training data. This metaphor, once elaborated, immediately suggests techniques for improving the process, ultimately defining, in reconstructive terms, an optimal performance criterion for such combinatorial approaches.

Artificial Co-Drivers as a Universal Enabling Technology for Future Intelligent Vehicles and Transportation Systems

This position paper introduces the concept of artificial “co-drivers” as an enabling technology for future intelligent transportation systems. In Sections I and II, the design principles of co-drivers are introduced and framed within general human-robot interactions. Several contributing theories and technologies are reviewed, specifically those relating to relevant cognitive architectures, human-like sensory-motor strategies, and the emulation theory of cognition. In Sections III and IV, we present the co-driver developed for the EU project interactIVe as an example instantiation of this notion, demonstrating how it conforms to the given guidelines. We also present substantive experimental results and clarify the limitations and performance of the current implementation. In Sections IV and V, we analyze the impact of the co-driver technology. In particular, we identify a range of application fields, showing how it constitutes a universal enabling technology for both smart vehicles and cooperative systems, and naturally sets out a program for future research.

Domain Anomaly Detection in Machine Perception: A System Architecture and Taxonomy

We address the problem of anomaly detection in machine perception. The concept of domain anomaly is introduced as distinct from the conventional notion of anomaly used in the literature. We propose a unified framework for anomaly detection which exposes the multifaceted nature of anomalies and suggest effective mechanisms for identifying and distinguishing each facet as instruments for domain anomaly detection. The framework draws on the Bayesian probabilistic reasoning apparatus which clearly defines concepts such as outlier, noise, distribution drift, novelty detection (object, object primitive), rare events, and unexpected events. Based on these concepts we provide a taxonomy of domain anomaly events. One of the mechanisms helping to pinpoint the nature of anomaly is based on detecting incongruence between contextual and noncontextual sensor(y) data interpretation. The proposed methodology has wide applicability. It underpins in a unified way the anomaly detection applications found in the literature. To illustrate some of its distinguishing features, in here the domain anomaly detection methodology is applied to the problem of anomaly detection for a video annotation system.

Evaluation of face recognition system in heterogeneous environments (visible vs NIR)

Performing facial recognition between Near Infrared (NIR) and visible-light (VIS) images has been established as a common method of countering illumination variation problems in face recognition. In this paper we present a new database to enable the evaluation of cross-spectral face recognition. A series of preprocessing algorithms, followed by Local Binary Pattern Histogram (LBPH) representation and combinations with Linear Discriminant Analysis (LDA) are used for recognition. These experiments are conducted on both NIR→VIS and the less common VIS→NIR protocols, with permutations of uni-modal training sets. 12 individual baseline algorithms are presented. In addition, the best performing fusion approaches involving a subset of 12 algorithms are also described.

Anomaly Detection and Knowledge Transfer in Automatic Sports Video Annotation

A key question in machine perception is how to adaptively build upon existing capabilities so as to permit novel functionalities. Implicit in this are the notions of anomaly detection and learning transfer. A perceptual system must firstly determine at what point the existing learned model ceases to apply, and secondly, what aspects of the existing model can be brought to bear on the newly-defined learning domain. Anomalies must thus be distinguished from mere outliers, i.e. cases in which the learned model has failed to produce a clear response; it is also necessary to distinguish novel (but meaningful) input from misclassification error within the existing models. We thus apply a methodology of anomaly detection based on comparing the outputs of strong and weak classifiers [10] to the problem of detecting the rule-incongruence involved in the transition from singles to doubles tennis videos. We then demonstrate how the detected anomalies can be used to transfer learning from one (initially known) rule-governed structure to another. Our ultimate aim, building on existing annotation technology, is to construct an adaptive system for court-based sport video annotation.

Ball event recognition using hmm for automatic tennis annotation

A key prerequisite of automatic video indexing and summarisation is the description of events and actions. In the context of many sports, the motion of the ball and agents plays an essential role in describing events. However, the only existing solution for the tennis event recognition problem in the literature is the work in [8] which relies on a set of heuristic rules such as proximity between ball and players or court lines to classify ball event candidates. We present hidden Markov models (HMMs) paradigm to automatically learn to identify events from ball trajectories and demonstrate that its ability to capture the dynamics of the ball movement lead to a much higher performance.

Characterizing Driver Intention via Hierarchical Perception–Action Modeling

We seek a mechanism for the classification of the intentional behavior of a cognitive agent, specifically a driver, in terms of a psychological Perception-Action (P-A) model, such that the resulting system would be potentially suitable for use in intelligent driver assistance. P-A models of human intentionality assume that a cognitive agents perceptual domain is learned in response to the outcome of the agents actions rather than vice versa. In this way, the perceptual domain is maintained at an appropriate level of complexity in relation to the agents embodied motor capabilities, greatly simplifying visual processing. A subsumptive P-A model further captures the hierarchical nature of the subtask structure implicit in human actions and assumes that a parallel hierarchical structuring exists within the perceptual domain. Adopting this model enables us to characterize intentions at each level of the P-A hierarchy in terms of a range of descriptors derived from the U.K. Highway Code by examining their correlation with driver gaze behavior. The problem of classifying intentions thus becomes one of reconciling high-level protocols (i.e., Highway Code rules) with low-level perceptual features. We perform a “proof-of-concept” assessment of the model by comparative evaluation of a number of logic-based methods (both stochastic and deductive) for carrying out this classification utilizing the control, signal, and motor inputs of an instrumented vehicle driven by a single driver, and find that a deductive model gives superior intentional classification performance due to the strongly protocol-governed nature of the driving environment.