Neill W. Campbell

Iterative refinement by relevance feedback in content-based digital image retrieval

Many image-database retr-ieual systems rely heavily on the success oj one-shot queries, using optimised jemkre set-s to obtain the best possible results. What is ojten missing jt-om this appToach is acceptance oj the fact that the user knows considerably mor-e about the query being made than cm. be conveyed in szch Relatively simple terms. Ij the query jails then the useT must try and improve the description using only the avai[able Jeatwe descriptors. This papeT describes how a query system can. exploit the user’s lrnourledge to a higher extent by employing relevance jeedback to itemtively rejine queries at ran-time. Subjects oj inierest aTe chosen by selection oj i-egions jrom pre-processed, segment ed imag~, giving access to object-specific, local information which zs not possible in a global pattern-matching approach. AfteT an initial retrieved attempt, jeedback is given in the jmm oj acceptance or rejection oj imagtx ofemd. This information b used as a collection oj positive and negative training ezamples joT a class-speci

Interpreting image databases by region classification

c classijkation network by identifying clusterings in the data and the spread along jeatuTe axes. Each. networ-k consists OJ a set of Radial BaSiSFunction nodes with a non-linear perception output layer. Network training is carried out of-line mung the data gathered o%ing an on-line query session with the user. The aseT can i-eview and adjust the behaviotw of the network in the next session. Over time, collections oj these networks can be built into a bier-a~chical ck.ss database, resulting into highly usefil Tetm”eval tooi specifically trained jor the nature oj the user’s database.

Efficient Layout of Comic-Like Video Summaries

Abstract This paper addresses automatic interpretation of images of outdoor scenes. The method allows instances of objects from a number of generic classes to be identified: vegetation, buildings; vehicles; roads, etc., thereby enabling image databases to be queried on scene content. The feature set is based, in part, on psychophysical principles and includes measures of colour, texture and shape. Using a large database of ground-truth labelled images, a neural network is trained as a pattern classifier. The method is demonstrated on a large test set to provide highly accurate image interpretations, with over 90% of the image area labelled correctly.

Using a tensor framework for the analysis of facial dynamics

In order to represent large amounts of information in the form of a video key-frame summary, this paper studies narrative grammar of comics, and using its universal and intuitive rules, lays out visual summaries in an efficient and user centered way. The system ranks importance of key-frame sizes in the final layout by balancing the dominant visual representability and discovery of unanticipated content utilizing a specific cost function and an unsupervised robust spectral clustering technique. A final layout is created using an optimization algorithm based on dynamic programming. Algorithm efficiency and robustness are demonstrated by comparing the results with the optimal panelling solutions.

Automatic segmentation and classification of outdoor images using neural networks.

Research has shown that the dynamics of facial motion are important in the perception of gender, identity, and emotion. In this paper we show that it is possible to use a multilinear tensor framework to extract facial motion signatures and to cluster these signatures by gender or by emotion. Here we consider only the dynamics of internal features of the face (e.g. eyebrows, eyelids and mouth) so as to remove structural and shape cues to identity and gender. Such structural gender biases include jaw width and forehead shape and their removal ensures dynamic cues alone are being used. Additionally, we demonstrate the generative capabilities of using a tensor framework, by consistently synthesising new motion signatures

Multi-perspective Panoramas of Long Scenes

The paper describes how neural networks may be used to segment and label objects in images. A self-organising feature map is used for the segmentation phase, and we quantify the quality of the segmentations produced as well as the contribution made by colour and texture features. A multi-layer perception is trained to label the regions produced by the segmentation process. It is shown that 91.1% of the image area is correctly classified into one of eleven categories which include cars, houses, fences, roads, vegetation and sky.

Practical generation of video textures using the auto-regressive process

This paper presents a system for producing multiperspective panoramas for long scenes from dense collections of images. Multi-perspective panoramas are generated by combining different perspectives, including original and novel perspectives. The latter are rendered by our synthesis algorithm, which uses estimated depth information to remove distortions caused by depth parallax. Then, different perspectives are properly combined without noticeable visual artifacts caused by both color inconsistencies and structural misalignments.

Facial behaviour mapping-From video footage to a robot head

Abstract Recently, there have been several attempts at creating ‘video textures’, that is, synthesising new (potentially infinitely long) video clips based on existing ones. One method for achieving this is to transform each frame of the video into an eigenspace using Principal Components Analysis so that the original sequence can be viewed as a signature through a low-dimensional space. A new sequence can be generated by moving through this space and creating ‘similar’ signatures. These signatures may be derived using an auto-regressive process (ARP). Such an ARP assumes that the signature has Gaussian statistics. For many sequences this assumption is valid, however, some sequences are strongly non-linearly correlated, in which case their statistical properties are non-Gaussian. We examine two methods by which such non-linearities may be overcome. The first is by modelling the non-linearity automatically using a spline, and the second using a combined appearance model. New video sequences created using these approaches contain images never present in the original sequence and appear very convincing.

Visual abstraction of wildlife footage using Gaussian mixture models and the minimum description length criterion

As autonomous robotic systems advance, they will be required and designed for interaction with humans in order to exchange information, which is essential for fulfilling their tasks. It is well established that human-machine interactions are more believable and memorable when a physical entity is present, provided that the machine behaves in a realistic manner. It is desirable to adopt face-to-face communication, because it is the most natural and efficient way of exchanging information, and does not require users to alter their habits. In this context, this paper describes a process for animating a robot head, based on video input of a human head. We map from the 2D coordinates of feature points into the robots servo space, using Partial Least Squares (PLS). Learning is done using a small set of keyframes manually created by an animator. The method is efficient, robust to tracking errors and independent of the scale of the face being tracked.

A Survey on Multimodal Video Representation for Semantic Retrieval

We present a novel approach for clip-based key frame extraction. Our framework allows both clips with subtle changes as well as clips containing rapid shot changes, fades and dissolves to be well approximated. We show that creating key frame video abstractions can be achieved by transforming each frame of a video sequence into an eigenspace and then clustering this space using Gaussian mixture models (GMMs). A minimum description length (MDL) criterion is then used to determine the optimal number of GMM components to use in the clustering. The image nearest to the centres of each of the GMM components are selected as key frames. Unlike previous work, this technique relies on global video clip properties and results show that the key frames extracted give a very good representation of the overall clip content. We demonstrate the application of this technique on a database of 307 clips of wildlife footage containing dissolves, shot changes, fades, pans, zooms and a wide range of animal behaviours.