Anil C. Kokaram

N-dimensional probability density function transfer and its application to color transfer

This article proposes an original method to estimate a continuous transformation that maps one N-dimensional distribution to another. The method is iterative, non-linear, and is shown to converge. Only 1D marginal distribution is used in the estimation process, hence involving low computation costs. As an illustration this mapping is applied to color transfer between two images of different contents. The paper also serves as a central focal point for collecting together the research activity in this area and relating it to the important problem of automated color grading

Browsing sports video: trends in sports-related indexing and retrieval work

This paper aims to identify the current trends in sports-based indexing and retrieval work. It discusses the essential building blocks for any semantic-level retrieval system and acts as a case study in content analysis system design. While one of the major benefits of digital media and digital television in particular has been to provide users with more choices and a more interactive viewing experience, the freedom to choose has in fact manifested as the freedom to choose from the options the broadcaster provides. It is only through the use of automated content-based analysis that sports viewers will be given a chance to manipulate content at a much deeper level than that intended by broadcasters, and hence put true meaning into interactivity

Joint audio visual retrieval for tennis broadcasts

In recent years, there has been increasing work in the area of content retrieval for sports. The idea is generally to extract important events or create summaries to allow personalisation of the media stream. While previous work in sports analysis has employed either the audio or video stream to achieve some goal, there is little work that explores how much can be achieved by combining the two streams. This paper combines both audio and image features to identify the key episode in tennis broadcasts. The image feature is based on image moments and is able to capture the essence of scene geometry without recourse to 3D modelling. The audio feature uses PCA to identify the sound of the ball hitting the racket. The features are modelled as stochastic processes and the work combines the features using a likelihood approach. The results show that combining the features yields a much more robust system than using the features separately.

Content-based analysis for video from snooker broadcasts

This paper presents three new tools appropriate for content analysis of sports, applied to footage from snooker broadcasts in particular. The first tool is a new feature for parsing a sequence based on geometry without the need for deriving 3D information. The second tool allows events to be detected where an event is characterised by an object leaving the scene at a particular location. The final tool is a mechanism for summarising motion in a shot for use in a content-based summary. As a matter of course, the paper considers a number of enabling techniques such as the removal of irrelevant objects and object tracking using a particle filter. The paper shows that by exploiting context, a convincing summary can be made for snooker footage.

Semantic Event Detection in Sports Through Motion Understanding

In this paper we investigate the retrieval of semantic events that occur in broadcast sports footage. We do so by considering the spatio-temporal behaviour of an object in the footage as being the embodiment of a particular semantic event. Broadcast snooker footage is used as an example of the sports footage for the purpose of this research. The system parses the sports video using the geometry of the content in view and classifies the footage as a particular view type. A colour based particle filter is then employed to robustly track the snooker balls, in the appropriate view, to evoke the semantics of the event. Over the duration of a player shot, the position of the white ball on the snooker table is used to model the high level semantic structure occurring in the footage. Upon collision of the white ball with another coloured ball, a separate track is instantiated allowing for the detection of pots and fouls, providing additional clues to the event in progress.

System for the removal of impulsive noise in image sequences

This paper presents a system for the restoration of image sequences that are degraded by impulsive noise such as scratches or dropouts. The proposed system uses a multilevel block matching algorithm to estimate the motion between frames and considers the use of an impulsive noise detector to improve the quality of restoration as compared to a global median operation. The detector considers the temporal continuity of motion compensated image information and makes a decision as to whether a suspected discontinuity is due to an impulsive distortion or occlusion in the sequence. When a corrupted portion of the image is detected a motion compensated median filter is used to remove the distortion. The paper introduces an extended multistage filter for image sequence processing. It is found that the use of the detector cannot adversely affect the filtered image when compared to the globally filtered image, and the detail preservation is generally better. The speed of processing is also increased since the number of median filtering operations is considerably reduced.

A Wavelet Packet representation of audio signals for music genre classification using different ensemble and feature selection techniques

The vast amount of music available electronically presents considerable challenges for information retrieval. There is a need to annotate music items with descriptors in order to facilitate retrieval. In this paper we present a process for determining the music genre of an item using a new set of descriptors. A Wavelet Packet Transform is applied to obtain the signal representation at different levels. Time and frequency features are extracted from these levels taking into account the nature of music. Using round-robin and one-against-all ensembles of simple classifiers, together with feature selection methods, we evaluate the best signal representation for music genre classification. Ensembles based on different feature sub-spaces are explored as well in order to overcome over-fitting issues. Our evaluation shows that Wavelet Packet analysis together with ensemble methods achieves very good classification accuracy.

A statistical framework for picture reconstruction using 2D AR models

Abstract This paper presents a framework for ‘Filling In’ missing gaps in images and particularly patches with texture. The algorithm can also be used as a fallback mode in treating missing data for video sequence reconstruction. The underlying idea is to construct a parametric model of the p.d.f. of the texture to be re-synthesised and then draw samples from that p.d.f. to create the resulting reconstruction. A Bayesian approach is used to articulate 2D Autoregressive Models as generative models for texture (using the Gibbs sampler) given surrounding boundary conditions. A fast implementation is presented that iterates between pixelwise updates and blockwise parametric model estimation. The novel ideas in this paper are joint parameter estimation and fast, efficient texture reconstruction using linear models.

Classification and representation of semantic content in broadcast tennis videos

This paper investigates the semantic analysis of broadcast tennis footage. We consider the spatio-temporal behaviour of an object in the footage as being the embodiment of a semantic event. This object is tracked using a colour based particle filter. The video syntax and audio features are used to help delineate the temporal boundaries of these events. For broadcast tennis footage, the system firstly parses the video sequence based on the geometry of the content in view and classifies the clip as a particular view type. The temporal behaviour of the serving player is modelled using a HMM. As a result, each model is representative of a particular semantic episode. Events are then summarised using a number of synthesised keyframes.

A System for Reconstruction of Missing Data in Image Sequences Using Sampled 3D AR Models and MRF Motion Priors

This paper presents a new technique for interpolating missing data in image sequences. A 3D autoregressive (AR) model is employed and a sampling based interpolator is developed in which reconstructed data is generated as a typical realization from the underlying AR process. rather than e.g. least squares (LS). In this way a perceptually improved result is achieved. A hierarchical gradient-based motion estimator, robust in regions of corrupted data, employing a Markov random field (MRF) motion prior is also presented for the estimation of motion before interpolation.