David Edmundson

Efficient and effective online image retrieval

With visual information becoming increasingly important, efficient and effective methods for querying and retrieving this kind of information are highly sought after. In this paper, we focus on image information and querying from image collections in an online retrieval fashion. In online retrieval, image features for performing retrieval are not pre-calculated but need to be extracted during the retrieval stage. Consequently, and in particular for large image datasets, the time required for feature extraction becomes crucial so as to not exceed interactive retrieval speeds. Our aim in this work is to match the retrieval accuracy of a high performing yet rather slow image retrieval method, but perform the retrieval in only a fraction of the time. We achieve this by a combination of two carefully crafted filtering stages both of which are based on the way data is stored in JPEG compressed images. The first of these performs extremely fast image retrieval using solely information contained in the JPEG headers. The second stage employs a compressed domain retrieval method that utilises features calculated from JPEG coefficient data. The first filter discards a large part of irrelevant images in a very fast fashion. The remaining images are filtered by the second technique in order to arrive at a relatively small subset of the complete database in a timely fashion. Finally, on this subset the high performing algorithm of choice, the MPEG-7 colour structure descriptor in this paper, is applied to produce a final ranking of the images to be returned to the user. Our experimental results demonstrate that on a large dataset of over 25,000 images our approach achieves retrieval scores nearly identical to those of the high performing technique while reducing the overall retrieval time by a factor of 15.

Performance comparison of JPEG compressed domain image retrieval techniques

Although content-based image retrieval (CBIR) has been an active research area for more than two decades, relatively little work takes into account that virtually all images exist in compressed form, mostly in (lossy) JPEG format. In this paper, we benchmark eight state-of-the-art CBIR algorithms that operate directly in the compressed domain of JPEG by performing retrieval based on DCT coefficients. We analyse their performance on a benchmark dataset, compare them against common pixel-domain techniques, investigate whether they are affected by compression ratio, and measure their computational complexity. We conclude that several of the JPEG CBIR techniques allow much faster feature calculation and faster image retrieval, while providing retrieval performance similar to common pixel-domain algorithms.

DC stream based JPEG compressed domain image retrieval

The vast majority of images are stored in compressed JPEG format. When performing content-based image retrieval, faster feature extraction is possible when calculating them directly in the compressed domain, avoiding full decompression of the images. Algorithms that operate in this way calculate image features based on DCT coefficients and hence still require partial decoding of the image to arrive at these. In this paper, we introduce a JPEG compressed domain retrieval algorithm that is based not directly on DCT coefficients but on differences of these, which are readily available in a JPEG compression stream. In particular, we utilise solely the DC stream of JPEG files and make direct use of the fact that DC terms are differentially coded. We build histograms of these differences and utilise them as image features, thus eliminating the need to undo the differential coding as in other methods. In combination with a colour histogram, also extracted from DC data, we show our approach to give (to our knowledge) the best retrieval accuracy of a JPEG domain retrieval algorithm, outperforming other compressed domain methods and reaching a performance close to that of the best performing MPEG-7 descriptor.

Robust texture retrieval of compressed images

Almost all images are stored in compressed form, most commonly in (lossy) JPEG format. In this paper, we show that compression leads to a drop in performance of texture retrieval algorithms, and propose a method that reverses this performance drop. We achieve this by what might at first glance seem counter-intuitive, namely by compressing the images even more. In particular, we recompress images (or rather re-quantising their DCT coefficients) to their lowest common image quality setting. We demonstrate, on a large benchmark texture retrieval database and using standard texture algorithms, that this results in improved image retrieval performance close to that obtained on uncompressed images.

Recompressing images to improve image retrieval performance

Virtually all images are stored in compressed form, most in (lossy) JPEG format. Compressing images however has been shown to cause a small but not negligible drop in performance for content-based image retrieval (CBIR) algorithms. In this paper, we show that it is possible to reverse this performance drop. We achieve this by what might at a first glance seem counter-intuitive, namely by compressing the images even more. In detail, what we perform is recompressing images (or rather re-quantising the DCT coefficients) to their lowest common image quality setting. We demonstrate, on a benchmark image retrieval database and using standard CBIR algorithms, that this results in improved image retrieval performance rivalling that of running the algorithms on uncompressed data.

Intuitive mobile image browsing on a hexagonal lattice

Following miniaturisation of cameras and their integration into mobile devices such as smartphones combined with the intensive use of the latter, it is likely that in the near future the majority of digital images will be captured using such devices rather than using dedicated cameras. Since many users decide to keep their photos on their mobile devices, effective methods for managing these image collections are required. Common image browsers prove to be only of limited use, especially for large image sets [1].

Fast mobile image retrieval

Image collections, in particular those accessed via photo sharing sites, are expanding at a rapid rate. At the same time, internet traffic from mobile devices such as smartphones is growing at a similar speed, and hence efficient and effective techniques for managing and querying these large image repositories from mobile devices are highly sought after. In this paper, we show that this is possible using a very fast method for performing content-based image retrieval of JPEG compressed images that naturally lends itself to mobile retrieval. Our approach performs retrieval directly in the compressed domain of JPEG but, unlike previous techniques, does not require partial decompression of the encoded image data. We employ image adapted Huffman tables, which are stored in the header of JPEG files, as image descriptors and thus not only eliminate the need for decoding but require transfer of only a fraction of the image files. Image similarity is defined as the similarity between DC and AC Huffman table entries, and is shown to lead to good retrieval performance. On a benchmark database, we demonstrate retrieval accuracy similar to common compressed domain and pixel domain retrieval algorithms, yet achieve a speedup of more than 30-fold (compared to JPEG domain techniques) and more than 250-fold (compared to MPEG-7 descriptors) respectively for online image retrieval. A mobile retrieval application is shown to provide an effective way of performing image retrieval from the photo sharing website Flickr while significantly reducing bandwidth and power consumption requirements.

Effective and Efficient Filtering of Retrieved Images Based on JPEG Header Information

Visual information on the web, in particular in form of images, is increasing at a rapid rate. Consequently, efficient and effective techniques to retrieve visual information are sought after, especially as it can be usefully employed to augment textual information. Since users rarely annotate images, this proves to be a challenging task, however much progress has been reported in the area of content-based image retrieval which is based on visual features extracted from images for retrieval purposes. In this paper, we present two strategies for very fast image retrieval which use solely information contained in the header of JPEG compressed files. One is based on the tables that are responsible for the lossy quantisation step in JPEG, while the other is related to the Huffman tables used for entropy coding. In both cases, we employ the tables directly as image features in the context of online image retrieval. We then utilise them to discard irrelevant images, while a compressed-domain image retrieval technique is used for ranking the remaining image set. Experimental results convincingly show that our algorithms lead to a significant reduction of overall retrieval time while maintaining retrieval accuracy. They could thus be integrated into web-based recommender systems to augment and improve search results.

JIRL - A C++ Library for JPEG Compressed Domain Image Retrieval

In this paper we present JIRL, an open source C++ software suite that allows to perform content-based image retrieval in the JPEG compressed domain. We provide implementations of nine retrieval algorithms representing the current state-of-the-art. For each algorithm, methods for compressed domain feature extraction as well as feature comparison are provided in an object-oriented framework. In addition, our software suite includes functionality for benchmarking retrieval algorithms in terms of retrieval performance and retrieval time. An example full image retrieval application is also provided to demonstrate how the library can be used. JIRL is made available to fellow researchers under the LGPL.

Fast JPEG image retrieval using tuned quantisation tables

In this paper, we present an extremely fast method for online image retrieval of JPEG compressed images. We exploit minimal perceptual error image compression which optimises JPEG quantisation tables to improve the resulting image quality. In particular, we demonstrate that thus tuned quantisation tables can be used as image descriptors for performing content-based image retrieval. Image similarity is expressed as similarity between the respective quantisation tables and feature extraction and comparison be performed in an extremely fast fashion as it is based on information only from the JPEG headers. We show that our method takes only about 2-2.5% of the time of standard compressed domain algorithms, yet achieves retrieval accuracy within 3.5% of these techniques on a large dataset.