Chryssanthi Iakovidou

Co.Vi.Wo.: Color Visual Words Based on Non-Predefined Size Codebooks

Due to the rapid development of information technology and the continuously increasing number of available multimedia data, the task of retrieving information based on visual content has become a popular subject of scientific interest. Recent approaches adopt the bag-of-visual-words (BOVW) model to retrieve images in a semantic way. BOVW has shown remarkable performance in content-based image retrieval tasks, exhibiting better retrieval effectiveness over global and local feature (LF) representations. The performance of the BOVW approach depends strongly, however, on predicting the ideal codebook size, a difficult and database-dependent task. The contribution of this paper is threefold. First, it presents a new technique that uses a self-growing and self-organized neural gas network to calculate the most appropriate size of a codebook for a given database. Second, it proposes a new soft-weighting technique, whereby each LF is classified into only one visual word (VW) with a degree of participation. Third, by combining the information derived from the method that automatically detects the number of VWs, the soft-weighting method, and a color information extraction method from the literature, it shapes a new descriptor, called color VWs. Experimental results on two well-known benchmarking databases demonstrate that the proposed descriptor outperforms 15 contemporary descriptors and methods from the literature, in terms of both precision at K and its ability to retrieve the entire ground truth.

Mean Normalized Retrieval Order (MNRO): a new content-based image retrieval performance measure

The results of a content based image retrieval system can be evaluated by several performance measures, each one employing different evaluation criteria. Many of the methods used in the field of information retrieval have been adopted for use in image retrieval systems. This paper reviews the most widely used performance measures for retrieval evaluation with particular emphasis on the assumptions made during their design. More specifically, it focuses on the design principles of the commonly used Mean Average Precision (MAP) and Average Normalized Modified Retrieval Rank (ANMRR), pinpointing their limitations. It also proposes a new performance measure for image retrieval systems, the Mean Normalized Retrieval Order (MNRO), whose effectiveness is demonstrated through a wide range of experiments. Initial experiments were conducted on artificially produced query trials and evaluations. Experiments on a large database demonstrate the ability of MNRO to take into account the generality of the queries during the retrieval procedure. Furthermore, the results of a case study show that the proposed performance measure is closer to human evaluations, in comparison to MAP and ANMRR. Lastly, in order to encourage researchers and practitioners to use the proposed performance measure, we present the experimental results produced by a large number of state of the art descriptors applied on three well-known benchmarking databases.

Localizing global descriptors for content-based image retrieval

In this paper, we explore, extend and simplify the localization of the description ability of the well-established MPEG-7 (Scalable Colour Descriptor (SCD), Colour Layout Descriptor (CLD) and Edge Histogram Descriptor (EHD)) and MPEG-7-like (Color and Edge Directivity Descriptor (CEDD)) global descriptors, which we call the SIMPLE family of descriptors. Sixteen novel descriptors are introduced that utilize four different sampling strategies for the extraction of image patches to be used as points of interest. Designing with focused attention for content-based image retrieval tasks, we investigate, analyse and propose the preferred process for the definition of the parameters involved (point detection, description, codebook sizes and descriptors’ weighting strategies). The experimental results conducted on four different image collections reveal an astonishing boost in the retrieval performance of the proposed descriptors compared to their performance in their original global form. Furthermore, they manage to outperform common SIFT- and SURF-based approaches while they perform comparably, if not better, against recent state-of-the-art methods that base their success on much more complex data manipulation.

Real-time indexing for large image databases: color and edge directivity descriptor on GPU

In this paper, we focus on implementing the extraction of a well-known low-level image descriptor using the multicore power provided by general-purpose graphic processing units (GPGPUs). The color and edge directivity descriptor, which incorporates both color and texture information achieving a successful trade-off between effectiveness and efficiency, is employed and reassessed for parallel execution. We are motivated by the fact that image/frame indexing should be achieved real time, which in our case means that a system should be capable of indexing a frame or an image as it becomes part of a database (ideally, calculating the descriptor as the images are captured). Two strategies are explored to accelerate the method and bypass resource limitations and architectural constrains. An approach that exclusively uses the GPU together with a hybrid implementation that distributes the computations to both available GPU and CPU resources are proposed. The first approach is strongly based on the compute unified device architecture and excels compared to all other solutions when the GPU resources are abundant. The second implementation suggests a hybrid scheme where the extraction process is split in two sequential stages, allowing the input data (images or video frames) to be pipelined through the central and the graphic processing units. Experimental results were conducted on four different combinations of GPU–CPU technologies in order to highlight the strengths and the weaknesses of all implementations. Real-time indexing is obtained over all computational setups for both GPU-only and Hybrid techniques. An impressive 22 times acceleration is recorded for the GPU-only method. The proposed Hybrid implementation outperforms the GPU-only implementation and becomes the preferred solution when a low-cost setup (i.e., more advanced CPU combined with a relatively weak GPU) is employed.

Golden retriever: a Java based open source image retrieval engine

Golden Retriever Image Retrieval Engine (GRire) is an open source light weight Java library developed for Content Based Image Retrieval (CBIR) tasks, employing the Bag of Visual Words (BOVW) model. It provides a complete framework for creating CBIR system including image analysis tools, classifiers, weighting schemes etc., for efficient indexing and retrieval procedures. Its eminent feature is its extensibility, achieved through the open source nature of the library as well as a user-friendly embedded plug-in system. GRire is available on-line along with install and development documentation on http://www.grire.net and on its Google Code page http://code.google.com/p/grire. It is distributed either as a Java library or as a standalone Java application, both GPL licensed.

Two-staged image colorization based on salient contours

In this paper we present a novel colorization technique that manages to significantly reduce color bleeding artifacts caused by weak object boundaries and also requires only abstract color indications and placement from the user. It is essentially a two-staged color propagation algorithm. Guided by the extracted salient contours of the image, we roughly mark and divide the image in two differently treated image area categories: Homogeneous color areas of high confidence and critical attention-needing areas of edges and region boundaries. The method was tested with user drawn scribble images, but can be easily adopted by image exemplars employing techniques, as well.

Spatial pyramids for boosting global features in content based image retrieval

Image retrieval deals with the problem of finding relevant images to satisfy a specific user need. Many methods for content based image retrieval have been developed over the years, ranging from global to local features and, lately, to convolutional neural networks. Each of the approaches has its own benefits and drawbacks, but they also have similarities. In this paper we investigate how a method initially developed for local features, pyramid matching, then employed on texture features, spatial pyramids, can enhance general global features. We apply a spatial pyramid based approach to add spatial information to well known and established global descriptors, and present the results of an extensive evaluation that shows that this combination is able to outperform the original versions of the global features.

Content based image retrieval using visual-words distribution entropy

Bag-of-visual-words (BOVW) is a representation of images which is built using a large set of local features. To date, the experimental results presented in the literature have shown that this approach achieves high retrieval scores in several benchmarking image databases because of their ability to recognize objects and retrieve near-duplicate (to the query) images. In this paper, we propose a novel method that fuses the idea of inserting the spatial relationship of the visual words in an image with the conventional Visual Words method. Incorporating the visual distribution entropy leads to a robust scale invariant descriptor. The experimental results show that the proposed method demonstrates better performance than the classic Visual Words approach, while it also outperforms several other descriptors from the literature.

Color and Edge Directivity Descriptor on GPGPU

Image indexing refers to describing the visual multimedia content of a medium, using high level textual information or/and low level descriptors. In most cases, images and videos are associated with noisy and incomplete user-supplied textual annotations, possibly due to omission or the excessive cost associated with the metadata creation. In such cases, Content Based Image Retrieval (CBIR) approaches are adopted and low level image features are employed for indexing and retrieval. We employ the Colour and Edge Directivity Descriptor (CEDD), which incorporates both colour and texture information in a compact representation and reassess it for parallel execution, utilizing the multicore power provided by General Purpose Graphic Processing Units (GPGPUs). Experiments conducted on four different combinations of GPU-CPU technologies revealed an impressive gained acceleration when using a GPU, which was up to 22 times faster compared to the respective CPU implementation, while real-time indexing was achieved for all tested GPU models.

A LoCATe‐based visual place recognition system for mobile robotics and GPGPUs

In this paper, a novel visual Place Recognition approach is evaluated based on a visual vocabulary of the Color and Edge Directivity Descriptor (CEDD) to address the loop closure detection task. Even though CEDD was initially designed so as to globally describe the color and texture information of an input image addressing Image Indexing and Retrieval tasks, its scalability on characterizing single feature points has already been proven. Thus, instead of using CEDD as a global descriptor, we adopt a bottom‐up approach and use its localized version, Local Color And Texture dEscriptor, as an input to a state‐of‐the‐art visual Place Recognition technique based on Visual Word Vectors. Also, we use a parallel execution pipeline based on a previous work of ours using the well established General Purpose Graphics Processing Unit (GPGPU) computing. Our experiments show that the usage of CEDD as a local descriptor produces high accuracy visual Place Recognition results, while the parallelization used allows for a real‐time implementation even in the case of a low‐cost mobile device.