David McG. Squire

Performance evaluation in content-based image retrieval: overview and proposals

Abstract Evaluation of retrieval performance is a crucial problem in content-based image retrieval (CBIR). Many different methods for measuring the performance of a system have been created and used by researchers. This article discusses the advantages and shortcomings of the performance measures currently used. Problems such as defining a common image database for performance comparisons and a means of getting relevance judgments (or ground truth) for queries are explained. The relationship between CBIR and information retrieval (IR) is made clear, since IR researchers have decades of experience with the evaluation problem. Many of their solutions can be used for CBIR, despite the differences between the fields. Several methods used in text retrieval are explained. Proposals for performance measures and means of developing a standard test suite for CBIR, similar to that used in IR at the annual Text REtrieval Conference (TREC), are presented.

Content-based query of image databases: inspirations from text retrieval

In this paper we report the application of techniques inspired by text retrieval research to the content-based query of image databases. In particular, we show how the use of an inverted file data structure permits the use of a feature space of

Strategies for positive and negative relevance feedback in image retrieval

\mathcal{O}(104)

Learning from User Behavior in Image Retrieval: Application of Market Basket Analysis

dimensions, by restricting search to the subspace spanned by the features present in the query. A suitably sparse set of colour and texture features is proposed. A scheme based on the frequency of occurrence of features in both individual images and in the whole collection provides a means of weighting possibly incommensurate features in a compatible manner, and naturally extends to incorporate relevance feedback queries. The use of relevance feedback is shown consistently to improve system performance, as measured by precision and recall.

Efficient access methods for content-based image retrieval with inverted files

Relevance feedback has been shown to be a very effective tool for enhancing retrieval results in text retrieval. It has also been increasingly used in content-based image retrieval and very good results have been obtained. However, too much negative feedback may destroy a query as good features get negative weightings. This paper compares a variety of strategies for positive and negative feedback. The performance evaluation of feedback algorithms is a hard problem. To solve this, we obtain judgments from several users and employ an automated feedback scheme. We then evaluate different techniques using the same judgements. Using automated feedback, the ability of a system to adapt to the users needs can be measured very effectively. Our study highlights the utility of negative feedback, especially over several feedback steps.

Learning a similarity-based distance measure for image database organization from human partitionings of an image set

This article describes an approach to learn feature weights for content-based image retrieval (CBIR) from user interaction log files. These usage log files are analyzed for images marked together by a user in the same query step. The problem is somewhat similar to one of the traditional data mining problems, the market basket analysis problem, where items bought together in a supermarket are analyzed. This paper outlines similarities and differences between the two fields and explains how to use the interaction data for deriving a better feature weighting.Experiments with existing log files are done and a significant improvement in performance is reached with a feature weighting calculated from the information contained in the log files. Even with several steps of relevance feedback the results remain much better than without the learning, which means that not only information from feedback is taken into account earlier, but a better quality of retrieval is reached in all steps.

Correspondence analysis and hierarchical indexing for content-based image retrieval

As human factor studies over the last thirty years have shown, response time is a very important factor for the usability of an interactive system, especially on the world wide web. In particular, response times of under one second are often specified as a usability requirement. This paper compares several methods for improving the evaluation time in a content-based image retrieval system which uses inverted file technology. The use of the inverted file technology facilitates search pruning in a variety of ways, as is shown in this paper. For large databases and a high number o possible features, efficient and fast access is necessary to allow interactive querying and browsing. Parallel access to the inverted file can reduce the response time. This parallel access is very easy to implement with little communication overhead, and thus scales well. Other search pruning methods, similar to methods used in information retrieval, can also reduce the response time significantly without reducing the performance of the system. The performance of the system is evaluated using precision vs. recall graphs, which are an established evaluation method in information retrieval. A user survey was carried out in order to obtain relevance judgments for the queries reported in this work.

Assessing Agreement Between Human and Machine Clusterings of Image Databases

In this paper we employ human judgments of image similarity to improve the organization of an image database. We first derive a statistic, /spl kappa//sub B/ which measures the agreement between two partitionings of an image set. /spl kappa//sub B/ is used to assess agreement both amongst and between human and machine partitionings. This provides a rigorous means of choosing between competing image database organization systems, and of assessing the performance of such systems with respect to human judgments. Human partitionings of an image set are used to define a similarity value based on the frequency with which images are judged to be similar. When this measure is used to partition an image set using a clustering technique, the resultant partitioning agrees better with human partitionings than any of the feature-space-based techniques investigated. Finally, we investigate the use of multilayer perceptrons and a distance learning network to learn a mapping from feature space to this perceptual similarity space. The distance learning network is shown to learn a mapping which results in partitionings in excellent agreement with those produced by human subjects.

Invariance Signatures

This paper describes a two-stage statistical approach supporting content-based search in image databases. The first stage performs correspondence analysis, a factor analysis method transforming image attributes into a reduced-size, uncorrelated factor space. The second stage performs ascendant hierarchical classification, an iterative clustering method which constructs a hierarchical index structure for the images of the database. Experimental results supporting the applicability of both techniques to data sets of heterogeneous images are reported.

Visualization of cluster changes by comparing self-organizing maps

There is currently much interest in the organization and content-based querying image databases. The usual hypothesis is that image similarity can be characterized by low-level features, without further abstraction. This assumes that agreement between machine and human measures of similarity is sufficient for the database to be useful. To assess this assumption, we develop measures of the agreement between partitionings of an image set, showing that chance agreements must be considered. These measures are used to assess the agreement between human subjects and several machine clustering techniques on an image set. The results can be used to select and refine distance measures for querying and organizing image databases.