William Plant

Visualisation and browsing of image databases

In this chapter we provide a comprehensive overview of the emerging field of visualising and browsing image databases. We start with a brief introduction to content-based image retrieval and the traditional query-by-example search paradigm that many retrieval systems employ. We specify the problems associated with this type of interface, such as users not being able to formulate a query due to not having a target image or concept in mind. The idea of browsing systems is then introduced as a means to combat these issues, harnessing the cognitive power of the human mind in order to speed up image retrieval.We detail common methods in which the often high-dimensional feature data extracted from images can be used to visualise image databases in an intuitive way. Systems using dimensionality reduction techniques, such as multi-dimensional scaling, are reviewed along with those that cluster images using either divisive or agglomerative techniques as well as graph-based visualisations. While visualisation of an image collection is useful for providing an overview of the contained images, it forms only part of an image database navigation system. We therefore also present various methods provided by these systems to allow for interactive browsing of these datasets. A further area we explore are user studies of systems and visualisations where we look at the different evaluations undertaken in order to test usability and compare systems, and highlight the key findings from these studies. We conclude the chapter with several recommendations for future work in this area.

Navigation and Browsing of Image Databases

Image collections are growing at an exponential rate and solutions to manage vast databases of images are hence highly sought after. Content-based image retrieval techniques have shown great potential, yet commonly employed approaches like query-by-example are only of limited usefulness. An interesting alternative is provided by systems that allow visual exploration of an image dataset through a browsing interface. In these methods the complete database, or parts thereof, is visualised through application of dimensionality reduction techniques, clustered visualisations or display of a graph structure. Once visualised, it should then be possible to browse through the collection in an interactive, intuitive and efficient way. In this paper we present various browsing techniques that can be employed for this purpose. Browsing can be achieved in several ways. We can distinguish between horizontal browsing which works on images of the visualisation plane, and includes operations such as panning, zooming, magnification and scaling, and vertical browsing which allows navigation to a different level of a hierarchically organised visualisation. Furthermore, browsing can also be accomplished by taking into account time stamp information, hence enabling temporal browsing. We conclude, highlighting the need for objective evaluation and benchmarking of browsing system and see one of the next research challenges in the development of effective image browsing tools for mobile devices.

Visualising image databases

In this paper we explore different ways in which large collections of images can be visualised. We discuss the three principle visualisation techniques employed for this purpose, namely dimensionality reduced mappings, clustering-based visualisations and graph-based representations. Mapping-based techniques try to present the relationships between images described by high-dimensional features in a low-dimensional visualisation space. Clustered visualisations group similar images based on content, metadata or time stamp information, while in graph-based approaches links between images are exploited to arrive at an intuitive display of the dataset. We highlight advantages and disadvantages of the various approaches and emphasise the need for a benchmark which allows objective evaluation of these systems.

Image retrieval on the Honeycomb Image Browser

Efficient and effective approaches of dealing with the vast amount of visual information available nowadays are highly sought after. This is particularly the case for image collections, both personal and commercial. Due to the magnitude of these ever expanding image repositories, annotation of all images images is infeasible, and search in such an image collection therefore becomes inherently difficult. Although content-based image retrieval techniques have shown much potential, such approaches also suffer from various problems making it difficult to adopt them in practice. In this paper, we follow a different approach, namely that of browsing image databases for image retrieval. In our Honeycomb Image Browser, large image databases are visualised on a hexagonal lattice with image thumbnails occupying hexagons. Arranged in a space filling manner, visually similar images are located close together enabling large image datasets to be navigated in a hierarchical manner. Various browsing tools are incorporated to allow for interactive exploration of the database. Experimental results confirm that our approach affords efficient image retrieval.

The Mosaic Test: measuring the effectiveness of colour-based image retrieval

A variety of content-based image retrieval systems exist which enable users to perform image retrieval based on colour content—i.e., colour-based image retrieval. For the production of media for use in television and film, colour-based image retrieval is useful for retrieving specifically coloured animations, graphics or videos from large databases (by comparing user queries to the colour content of extracted key frames). It is also useful to graphic artists creating realistic computer-generated imagery (CGI). Unfortunately, current methods for evaluating colour-based image retrieval systems have 2 major drawbacks. Firstly, the relevance of images retrieved during the task cannot be measured reliably. Secondly, existing methods do not account for the creative design activity known as reflection-in-action. Consequently, the development and application of novel and potentially more effective colour-based image retrieval approaches, better supporting the large number of users creating media for use in television and film productions, is not possible as their efficacy cannot be reliably measured and compared to existing technologies. As a solution to the problem, this paper introduces the Mosaic Test. The Mosaic Test is a user-based evaluation approach in which participants complete an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. In this paper, we introduce the Mosaic Test and report on a user evaluation. The findings of the study reveal that the Mosaic Test overcomes the 2 major drawbacks associated with existing evaluation methods and does not require expert participants.

Intuitive mobile image browsing on a hexagonal lattice

Following miniaturisation of cameras and their integration into mobile devices such as smartphones combined with the intensive use of the latter, it is likely that in the near future the majority of digital images will be captured using such devices rather than using dedicated cameras. Since many users decide to keep their photos on their mobile devices, effective methods for managing these image collections are required. Common image browsers prove to be only of limited use, especially for large image sets [1].

An overview of Genetic Algorithms in simulation soccer

This paper discusses the use of genetic algorithms and genetic programming within the simulation soccer domain. Genetic algorithms (GAs) are based on the Darwinian theory of evolution and provide techniques to execute an effective search on a large range of potential solutions to a specific problem. Genetic Programming (GP) uses GA concepts to evolve a computer program. We show how GAs and GP have been applied to the challenging real-time and noisy domain of RoboCup simulation soccer. Among others, genetic approaches can be used to find appropriate actions for a soccer agent during a game, to improve different aspects of team strategy as well as to strengthen the ability of a player or a team in training exercises.

Interactive exploration of large remote image databases

Mapping-based visualisations of image databases are well suited to users wanting to survey the overall content of a collection. Given the large amount of image data contained within such visualisations, however, this approach has yet to be applied to large image databases stored remotely. In this technical demonstration, we showcase our Web-Based Images Browser (WBIB). Our novel system makes use of image pyramids so that users can interactively explore mapping-based visualisations of large remote image databases.

Truly Interactive Approaches to Browsing Image Databases

Image collections are ever growing and hence efficient and effective tools to manage these repositories are highly sought after. In this paper, we present effective image browsing systems that are operated on a large multi-touch environment for truly interactive exploration. Not only do image browsers pose a useful alternative to retrieval-based systems, they also provide a visualisation of the whole image collection and allow users to interactively explore particular parts of the collection. Our systems are based on the idea that visually similar images are located close to each other in the visualisation, that image thumbnails are arranged on a regular lattice (either a regular grid projected onto a sphere or a hexagonal lattice), and that large image datasets can be accessed through a hierarchical tree structure. A pilot study has shown that the presented systems do indeed work well and are preferred compared to conventional image browsers.

Exploring mapping-based visualisations of large remote image databases

Image database visualisations, in particular mapping-based visualisations, provide an interesting approach to accessing image repositories as they are able to overcome some of the drawbacks associated with retrieval based approaches. However, making a mapping-based approach work efficiently on large remote image databases, has yet to be explored. In this paper, we present Web-Based Images Browser (WBIB), a novel system that efficiently employs image pyramids to reduce bandwidth requirements so that users can interactively explore large remote image databases.