John P. Eakins

Towards intelligent image retrieval

Research into techniques for the retrieval of images by semantic content is still in its infancy. This paper reviews recent trends in the field, distinguishing four separate lines of activity: automatic scene analysis, model-based and statistical approaches to object classification, and adaptive learning from user feedback. It compares the strengths and weaknesses of model-based and adaptive techniques, and argues that further advances in the field are likely to involve the increasing use of techniques from the field of artificial intelligence.

Similarity retrieval of trademark images

The Artisan system retrieves abstract trademark images by shape similarity. It analyzes each image to characterize key shape components, grouping image regions into families that potentially mirror human image perception, and then derives characteristic indexing features from these families and from the image as a whole. We have evaluated the retrieval effectiveness of our prototype system on more than 10,000 images from the UK Trade Marks Registry.

ARTISAN: a shape retrieval system based on boundary family indexing

Successful retrieval of images by shape feature is likely to be achieved only if we can mirror human similarity judgments. Following Biedermans theory of recognition-by-components, we postulate that shape analysis for retrieval should characterize an image by identifying properties such as collinearity, shape similarity and proximity in component boundaries. Such properties can then be used to group image components into families, from which indexing features can be derived. We are currently applying these principles in the development of the ARTISAN shape retrieval system for the UK Patent Office. The trademark images, supplied in compressed bit-map format, are processed using standard edge-extraction techniques to derive a set of region boundaries, which are approximated as a sequence of straight-line and circular-arc segments. These are then grouped into families using criteria such as proximity and shape similarity. Shape features for retrieval are then extracted from the image as a whole, each boundary family, and each individual boundary. Progress to date with the project is analyzed, evaluation plans described, and possible future directions for the research discussed.

Shape feature matching for trademark image retrieval

Shape retrieval from image databases is a complex problem. This paper reports an investigation on the comparative effectiveness of a number of different shape features (including those included in the recent MPEG-7 standard) and matching techniques in the retrieval of multi-component trademark images. Experiments were conducted within the framework of the ARTISAN shape retrieval system, and retrieval effectiveness assessed on a database of over 10 000 images, using 24 queries and associated ground truth supplied by the UK Patent Office. Our results show clearly that multi-component matching can give better results than whole-image matching. However, only minor differences in retrieval effectiveness were found between different shape features or distance measures, suggesting that a wide variety of shape feature combinations and matching techniques can provide adequate discriminating power for effective retrieval.

Image Retrieval Interfaces: A User Perspective

Surprisingly little is known about how different users conduct image searches. As a result, even the most sophisticated systems available have limited appeal to the end-user. This paper describes a study eliciting user requirements for future image databases through an online questionnaire. 125 experienced image searchers were questioned about the functions and modes of interaction that they currently use, and those they would like to see in future systems. The results of this survey, and their implications for retrieval systems design, are discussed in some detail.

A Taxonomy of the Image: On the Classification of Content for Image Retrieval

Image database (IDB) systems are at present often designed to test technology and the efficacy of retrieval algorithms, rather than being oriented towards delivering functionality to users. Research is necessary to design interfaces geared towards human usage of images. The starting point of this research needs to be consideration at a fundamental, user-centred level of how people perceive and interpret images. This article considers literature from many disciplines to describe a taxonomy of image content, from direct sensory elements to high-level abstractions. The nine categories derived will later be validated and used to direct the design of visual query interfaces for IDB systems.

Challenges of Image and Video Retrieval

What use is the sum of human knowledge if nothing can be found? Although significant advances have been made in text searching, only preliminary work has been done in finding images and videos in large digital collections. In fact, if we examine the most frequently used image and video retrieval systems (i.e. www.google.com) we find that they are typically oriented around text searches where manual annotation was already performed.

Retrieval of Still Images by Content

This chapter summarises the current state of the art in content based image retrieval (CBIR). It discusses the need for image retrieval by content, and the types of query which might be encountered. It describes the main techniques currently used to retrieve images by content at both primitive and semantic levels, describes the features of some commercial and experimental CBIR systems, assesses the capabilities of current technology, and outlines possible future development in the field.

Comparison of the effectiveness of alternative feature sets in shape retrieval of multicomponent images

Many different kinds of features have been used as the basis for shape retrieval from image databases. This paper investigates the relative effectiveness of several types of global shape feature, both singly and in combination. The features compared include well-established descriptors such as Fourier coefficients and moment invariants, as well as recently-proposed measures of triangularity and ellipticity. Experiments were conducted within the framework of the ARTISAN shape retrieval system, and retrieval effectiveness assessed on a database of over 10,000 images, using 24 queries and associated ground truth supplied by the UK Patent Office . Our experiments revealed only minor differences in retrieval effectiveness between different measures, suggesting that a wide variety of shape feature combinations can provide adequate discriminating power for effective shape retrieval in multi-component image collections such as trademark registries. Marked differences between measures were observed for some individual queries, suggesting that there could be considerable scope for improving retrieval effectiveness by providing users with an improved framework for searching multi-dimensional feature space.

Component-based visual clustering using the self-organizing map

In this paper we present a new method for visual clustering of multi-component images such as trademarks, using the topological properties of the self-organizing map, and show how it can be used for similarity retrieval from a database. The method involves two stages: firstly, the construction of a 2D map based on features extracted from image components, and secondly the derivation of a Component Similarity Vector from a query image, which is used in turn to derive a 2D map of retrieved images. The retrieval effectiveness of this novel component-based shape matching approach has been evaluated on a set of over 10 000 trademark images, using a spatially-based precision-recall measure. Our results suggest that our component-based matching technique performs markedly better than matching using whole-image clustering, and is relatively insensitive to changes in input parameters such as network size.