Ricardo da Silva Torres

Comparative study of global color and texture descriptors for web image retrieval

This paper presents a comparative study of color and texture descriptors considering the Web as the environment of use. We take into account the diversity and large-scale aspects of the Web considering a large number of descriptors (24 color and 28 texture descriptors, including both traditional and recently proposed ones). The evaluation is made on two levels: a theoretical analysis in terms of algorithms complexities and an experimental comparison considering efficiency and effectiveness aspects. The experimental comparison contrasts the performances of the descriptors in small-scale datasets and in a large heterogeneous database containing more than 230 thousand images. Although there is a significant correlation between descriptors performances in the two settings, there are notable deviations, which must be taken into account when selecting the descriptors for large-scale tasks. An analysis of the correlation is provided for the best descriptors, which hints at the best opportunities of their use in combination.

VISON: VIdeo Summarization for ONline applications

Recent advances in technology have increased the availability of video data, creating a strong requirement for efficient systems to manage those materials. Making efficient use of video information requires that data to be accessed in a user-friendly way. This has been the goal of a quickly evolving research area known as video summarization. Most of existing techniques to address the problem of summarizing a video sequence have focused on the uncompressed domain. However, decoding and analyzing of a video sequence are two extremely time-consuming tasks. Thus, video summaries are usually produced off-line, penalizing any user interaction. The lack of customization is very critical, as users often have different demands and resources. Since video data are usually available in compressed form, it is desirable to directly process video material without decoding. In this paper, we present VISON, a novel approach for video summarization that works in the compressed domain and allows user interaction. The proposed method is based on both exploiting visual features extracted from the video stream and on using a simple and fast algorithm to summarize the video content. Results from a rigorous empirical comparison with a subjective evaluation show that our technique produces video summaries with high quality relative to the state-of-the-art solutions and in a computational time that makes it suitable for online usage.

Visual structures for image browsing

Content-Based Image Retrieval (CBIR) presents several challenges and has been subject to extensive research from many domains, such as image processing or database systems. Database researchers are concerned with indexing and querying, whereas image processing experts worry about extracting appropriate image descriptors. Comparatively little work has been done on designing user interfaces for CBIR systems. This, in turn, has a profound effect on these systems since the concept of image similarity is strongly influenced by user perception. This paper describes an initial effort to fill this gap, combining recent research in CBIR and Information Visualization, studied from a Human-Computer Interface perspective. It presents two visualization techniques based on Spiral and Concentric Rings implemented in a CBIR system to explore query results. The approach is centered on keeping user focus on both the query image, and the most similar retrieved images. Experiments conducted so far suggest that the proposed visualization strategies improves system usability.

Visual word spatial arrangement for image retrieval and classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization. HighlightsSpatial arrangement of visual words (WSA) for image retrieval and classification.WSA generates vectors more compact than the traditional spatial pooling methods.WSA outperforms Spatial Pyramids in the retrieval scenario.WSA presents adequate performance in the classification scenario.

Learning to rank for content-based image retrieval

In Content-based Image Retrieval (CBIR), accurately ranking the returned images is of paramount importance, since users consider mostly the topmost results. The typical ranking strategy used by many CBIR systems is to employ image content descriptors, so that returned images that are most similar to the query image are placed higher in the rank. While this strategy is well accepted and widely used, improved results may be obtained by combining multiple image descriptors. In this paper we explore this idea, and introduce algorithms that learn to combine information coming from different descriptors. The proposed learning to rank algorithms are based on three diverse learning techniques: Support Vector Machines (CBIR-SVM), Genetic Programming (CBIR-GP), and Association Rules (CBIR-AR). Eighteen image content descriptors(color, texture, and shape information) are used as input and provided as training to the learning algorithms. We performed a systematic evaluation involving two complex and heterogeneous image databases (Corel e Caltech) and two evaluation measures (Precision and MAP). The empirical results show that all learning algorithms provide significant gains when compared to the typical ranking strategy in which descriptors are used in isolation. We concluded that, in general, CBIR-AR and CBIR-GP outperforms CBIR-SVM. A fine-grained analysis revealed the lack of correlation between the results provided by CBIR-AR and the results provided by the other two algorithms, which indicates the opportunity of an advantageous hybrid approach.

Exploring digital libraries: integrating browsing, searching, and visualization

Exploring services for digital libraries (DLs) include two major paradigms, browsing and searching, as well as other services such as clustering and visualization. In this paper, we formalize and generalize DL exploring services within a DL theory. We develop theorems to indicate that browsing and searching can be converted or mapped to each other under certain conditions. The theorems guide the design and implementation of exploring services for an integrated archaeological DL, ETANA-DL. Its integrated browsing and searching can support users in moving seamlessly between these operations, minimizing context switching, and keeping users focused. It also integrates browsing and searching into a single visual interface for DL exploration. A user study to evaluate ETANA-DLs exploring services helped validate our hypotheses

Online video summarization on compressed domain

Abstract Recent advances in technology have increased the availability of video data, creating a strong requirement for efficient systems to manage those materials. Making efficient use of video information requires that data to be accessed in a user-friendly way. Ideally, one would like to understand a video content, without having to watch it entirely. This has been the goal of a quickly evolving research area known as video summarization. In this paper, we present a novel approach for video summarization that works in the compressed domain and allows the progressive generation of a video summary. The proposed method relies on exploiting visual features extracted from the video stream and on using a simple and fast algorithm to summarize the video content. Experiments on a TRECVID 2007 dataset show that our approach presents high quality relative to the state-of-the-art solutions and in a computational time that makes it suitable for online usage.

A scalable re-ranking method for content-based image retrieval

Content-based Image Retrieval (CBIR) systems consider only a pairwise analysis, i.e., they measure the similarity between pairs of images, ignoring the rich information encoded in the relations among several images. However, the user perception usually considers the query specification and responses in a given context. In this scenario, re-ranking methods have been proposed to exploit the contextual information and, hence, improve the effectiveness of CBIR systems. Besides the effectiveness, the usefulness of those systems in real-world applications also depends on the efficiency and scalability of the retrieval process, imposing a great challenge to the re-ranking approaches, once they usually require the computation of distances among all the images of a given collection. In this paper, we present a novel approach for the re-ranking problem. It relies on the similarity of top-k lists produced by efficient indexing structures, instead of using distance information from the entire collection. Extensive experiments were conducted on a large image collection, using several indexing structures. Results from a rigorous experimental protocol show that the proposed method can obtain significant effectiveness gains (up to 12.19% better) and, at the same time, improve considerably the efficiency (up to 73.11% faster). In addition, our technique scales up very well, which makes it suitable for large collections.

Comparison of video sequences with histograms of motion patterns

Making efficient use of video information requires the development of a video signature and a similarity measure to rapidly identify similar videos in a huge database. Most of existing techniques to address this problem have focused on the uncompressed domain. However, decoding and analyzing of a video sequence are extremely time-consuming tasks. Since video data are usually available in compressed form, it is desirable to directly process video material without decoding. In this paper, we present a novel approach for comparing video sequences that works in the compressed domain. The proposed method is based on recognizing motion patterns extracted from the video stream and their occurrence histogram is proven to be a powerful feature for describing the video content. Experiments on a TRECVID 2010 dataset show that our approach presents high accuracy relative to the state-of-the-art solutions and in a computational time that makes it suitable for large collections.

Exploiting pairwise recommendation and clustering strategies for image re-ranking

In Content-based Image Retrieval (CBIR) systems, accurately ranking collection images is of great relevance. Users are interested in the returned images placed at the first positions, which usually are the most relevant ones. Commonly, image content descriptors are used to compute ranked lists in CBIR systems. In general, these systems perform only pairwise image analysis, that is, compute similarity measures considering only pairs of images, ignoring the rich information encoded in the relations among several images. This paper presents a novel re-ranking approach used to improve the effectiveness of CBIR tasks by exploring relations among images. In our approach, a recommendation-based strategy is combined with a clustering method. Both exploit contextual information encoded in ranked lists computed by CBIR systems. We conduct several experiments to evaluate the proposed method. Our experiments consider shape, color, and texture descriptors and comparisons with other post-processing methods. Experimental results demonstrate the effectiveness of our method.