Dionysius P. Huijsmans

Toward improved ranking metrics

In many computer vision algorithms, a metric or similarity measure is used to determine the distance between two features. The Euclidean or SSD (sum of the squared differences) metric is prevalent and justified from a maximum likelihood perspective when the additive noise distribution is Gaussian. Based on real noise distributions measured from international test sets, we have found that the Gaussian noise distribution assumption is often invalid. This implies that other metrics, which have distributions closer to the real noise distribution, should be used. In this paper, we consider three different applications: content-based retrieval in image databases, stereo matching, and motion tracking. In each of them, we experiment with different modeling functions for the noise distribution and compute the accuracy of the methods using the corresponding distance measures. In our experiments, we compared the SSD metric, the SAD (sum of the absolute differences) metric, the Cauchy metric, and the Kullback relative information. For several algorithms from the research literature which used the SSD or SAD, we showed that greater accuracy could be obtained by using the Cauchy metric instead.

How to complete performance graphs in content-based image retrieval: add generality and normalize scope

The performance of a content-based image retrieval (CBIR) system, presented in the form of precision-recall or precision-scope graphs, offers an incomplete overview of the system under study: the influence of the irrelevant items (embedding) is obscured. We propose a comprehensive and well-normalized description of the ranking performance compared to the performance of an ideal retrieval system defined by ground-truth for a large number of predefined queries. We advocate normalization with respect to relevant class size and restriction to specific normalized scope values (the number of retrieved items). We also propose new three and two-dimensional performance graphs for total recall studies in a range of embeddings.

Multi-scale sub-image search

If an image should be retrieved by its subregions from a large image database, an immense number of possible queries will appear. Therefore, the index which encodes the spatial information of an image, should make only few assumptions about possible queries. In addition, this index has to consider different scales of objects in the image. In this paper, we propose a novel approach using a hierarchical index encoding image regions, gained by a fixed partition. The suggested index uses color features and is easy to implement. The index is tested on a database with more than 12,000 images.

Extended performance graphs for cluster retrieval

Performance evaluations in probabilistic information retrieval are often presented as precision-recall or precision-scope graphs avoiding the otherwise dominating effect of the embedding irrelevant fraction. However, precision and recall values as such offer an incomplete overview of the information retrieval system under study: information about system parameters like generality (the embedding of the relevant fraction), random performance, and the effect of varying the scope is missed In this paper two cluster performance graphs are presented In those cases where complete ground truth is available (both cluster size and database size) the cluster precision-recall (Cluster PR) graph and the generality-precision=recall graph are proposed.

Quality Measures for Interactive Image Retrieval with a Performance Evaluation of Two 3x3 Texel-based Methods

The aim of the Leiden Imaging and Multi-media Group in collaboration with Philips is to develop and evaluate content-based indexing and interactive retrieval methods for large photo collections and to integrate them with annotation based methods. Ground-truth is provided by copy pairs in the Leiden Portrait Database, a database of scanned-in images of 19th-century Dutch studio portraits (“Cartes de Visite” ).

Content-based indexing performance: size normalized precision, recall, generality evaluation

Progress in content-based image retrieval (CBIR) is hampered by the lack of good evaluation practice and test-benches. In this paper, we raise the awareness of all the parameters that define a content-based indexing and retrieval method. Extensive ground-truth, 15,324 hand-checked image queries, was developed for a portrait database of gray-level images and their backside studio logos. Our aim was to clearly demonstrate the diminishing effect of a growing embedding on performance figures, and the establishment of a reliable ranking of several suggested CBIR gray-level indexing methods. This evaluation scheme was used first to optimize a number of parameters defining the detailed workings of each method. The database, standard image queries, ground-truth, and evaluation scripts are offered for inclusion in an evaluation site like Benchathlon.

Adapting k-d Trees to Visual Retrieval

The most frequently occurring problem in image retrieval is find-the-similar-image, which in general is finding the nearest neighbor. From the literature, it is well known that k-d trees are efficient methods of finding nearest neighbors in high dimensional spaces. In this paper we survey the relevant k-d tree literature, and adapt the most promising solution to the problem of image retrieval by finding the best parameters for the bucket size and threshold. We also test the system on the Corel Studio photo database of 18,724 images and measure the user response times and retrieval accuracy.

Interactive voxel-based graphics for 3D reconstruction of biological structures

Abstract To aid the visualization and manipulation of complex biological structures, a reconstruction system is developed that is based on a volume representation of objects. This approach has several advantages for numerical quantification of parameters as well as interactive manipulation of the objects. In addition to developing the software, we decided to try and exploit readily available image processing hardware to support the display and manipulation of voxel models. At display time, encoded normal vectors on the visible surface are stored in the framebuffer. Changing the light direction then merely involves recalculating the output lookup table. The removal and addition of voxels require only local modifications to the displayed image, allowing interactive editing of the model. The ability of the hardware to calculate bitwise logical functions on two frames is used for the modelling of objects through union, intersection and difference of two volume-element sets that are stored in the framebuffers.

Efficient content-based image retrieval in digital picture collections using projections: (near)-copy location

Digital storage of large photo collections opens the way to computer-aided queries based on visual rather than thematic search patterns. The objective of our queries in this research was the 19th-century mass-produced studio portrait or carte-de-visite, whose front and back sides provide a testbed for gray level and binary images classes. We established a ground truth for detecting highly similar images (former copies) in different classes of B/W images. The results will serve as a reference benchmark for yet to be developed visual search methods. The similarity measure used for locating near-copies was the average distance in pixel intensity for shifted image pairs with normalized position, orientation, resolution and lighting. To measure the performance of possible hierarchical comparison and ranking protocols, we scanned in test sets of known copies and near-copies together with over a thousand similar format pictures. The results show that projections are highly effective and efficient in indexing raw image data: reduced dimensionality, low noise, conservation of pattern characteristics, separable x- and y-translation (best shift) and suitable for hierarchical indexing. A query by example WWW demo program with precalculated ranking result files was developed for visual inspection and evaluation of similar image location methods.

Which ranking metric is optimal? With applications in image retrieval and stereo matching

Euclidean metric is frequently used in computer vision, mostly ad-hoc without any justification. However we have found that other metrics like a double exponential or Cauchy metric provide better results, in accordance with the maximum likelihood approach. In this paper we experiment with different modeling functions for similarity noise and compute the accuracy of different methods using these modeling functions in two kinds of applications: content-based image retrieval from a large database and stereo matching. We provide a way to determine the modeling distribution which fits best the similarity noise distribution according to the ground truth. In the optimum case, when one has chosen the best modeling distribution, its corresponding metric will give the best ranking results for the ground truth provided.