Esin Guldogan

Feature selection for content-based image retrieval

In this article, we propose a novel system for feature selection, which is one of the key problems in content-based image indexing and retrieval as well as various other research fields such as pattern classification and genomic data analysis. The proposed system aims at enhancing semantic image retrieval results, decreasing retrieval process complexity, and improving the overall system usability for end-users of multimedia search engines. Three feature selection criteria and a decision method construct the feature selection system. Two novel feature selection criteria based on inner-cluster and intercluster relations are proposed in the article. A majority voting-based method is adapted for efficient selection of features and feature combinations. The performance of the proposed criteria is assessed over a large image database and a number of features, and is compared against competing techniques from the literature. Experiments show that the proposed feature selection system improves semantic performance results in image retrieval systems.

MUVIS: a content-based multimedia indexing and retrieval framework

MUVIS is a series of CBIR systems. The first one has been developed in late 90s to support indexing and retrieval in large image databases using visual and semantic features such as color, texture and shape. During recent years, MUVIS has been reformed to become a PC-based framework, which supports indexing, browsing and querying of various multimedia types such as audio, video, audio/video interlaced and several image formats. MUVIS system allows real-time audio and video capturing, encoding by last generation codecs such as MPEG-4, H.263+, MP3 and AAC. It supports several audio/video file format such as AVI, MP4, MP3 and AAC. Furthermore, MUVIS system provides a well-defined interface for third parties to integrate their own feature extraction algorithms into the framework and for this reason it has recently been adopted by COST 211quat as COST framework for CBIR. In this paper, we describe the general system features with underlying applications and outline the main philosophy.

Compression effects on color and texture based multimedia indexing and retrieval

This paper presents an evaluation of digital compression effects on content-based multimedia retrieval using color and texture attributes. Subjective evaluation tests that are applied on digital image and video databases using different compression and visual feature extraction techniques have been performed and reported. Simulations show that a satisfactory retrieval performance can be obtained from the compressed databases with 10% compression quality (i.e. 97.6% compression ratio in JPEG). Image retrieval based on HSV color histogram performs better than retrieval based on YUV color histogram in the uncompressed domain, and the other way around in the compressed domain. In general, video retrieval based on color histogram in MPEG-4 compressed databases performs better compared to H.263+ compressed databases. However, retrieval performance from H.263+ compressed databases at lower bit rates is more stable, where it drastically decreases in MPEG-4 compressed databases below 128 Kb/s. Retrieval based on texture features produces more robust performance than retrieval based on color. Subjective tests show that 25% compression quality achieves high compression ratio without loosing significant retrieval performance. The results are particularly relevant to applications in which a mobile device is involved in a multimedia retrieval system.

Instance based personalized multi-form image browsing and retrieval

It is important to adapt and personalize image browsing and retrieval systems based on users’ preferences for improved user experience and satisfaction. In this paper, we present a novel instance based personalized multi-form image representation with implicit relevance feedback and adaptive weighting approach for image browsing and retrieval systems. In the proposed system, images are grouped into forms, which represent different information on images such as location, content etc. We conducted user interviews on image browsing, sharing and retrieval systems for understanding image browsing and searching behaviors of users. Based on the insights gained from the user interview study we propose an adaptive weighting method and implicit relevance feedback for multi-form structures that aim to improve the efficiency and accuracy of the system. Statistics of the past actions are considered for modeling the target of the users. Thus, on each iteration weights of the forms are updated adaptively. Moreover, retrieval results are modified according to the users’ preferences on iterations in order to improve personalized user experience. The proposed method has been evaluated and results are illustrated in the paper. It is shown that, satisfactory improvements can be achieved with proposed approaches in the multi-form scheme.

Personalized representative image selection for shared photo albums

In this paper, we present a novel method for personalizing the representative image selection using visual modeling and user statistics. For a given shared image album, a set of images that have the highest interest for a single user are determined. A personalized visual model is created using the given “interest set” to be able to select the most representative image(s) for that user from that album. The experiments show satisfactory performance in the task of representative image selection.

Dynamic feature weights with relevance feedback in content-based image retrieval

In this paper, we present a novel relevance feedback method for Content-Based Image Retrieval systems based on dynamic feature weights. The proposed method utilizes intra-cluster and inter-cluster information for representing the descriptive and discriminative properties of the features according to the labeled images by the user. Afterwards, feature weights are updated dynamically according to the users preferences for improving retrieval results. The proposed method has been thoroughly evaluated and selected results are illustrated in the paper. It is shown that, satisfactory improvements can be achieved with small number of iterations and labeled samples. Furthermore, it is a low-complex and flexible method that can be used on various databases and Content-Based Image Retrieval applications.

Visual media retrieval using transform-based layered query scheme

This paper presents a visual media querying scheme referred to as transform-based layered query (TLQ) scheme. The TLQ scheme mainly aims at decreasing retrieval processing time and run-time memory consumption without degrading retrieval results semantically. The scheme contains abstract layers in indexing and retrieval phases, where each indexing layer corresponds to a retrieval layer. The layers are constructed based on transformations for reducing visual frame and feature data dimensions. The proposed TLQ scheme also involves an unsupervised method for eliminating irrelevant media items between the retrieval layers. A two-layer TLQ system is implemented and integrated into MUVIS content-based multimedia indexing and retrieval framework, and its theoretical advantages are verified with dedicated experiments on image and video databases. The experiments reveal that 75% retrieval performance improvement in terms of process time can be achieved depending on transformation parameters.

Audiovisual video context recognition using SVM and genetic algorithm fusion rule weighting

The recognition of the surrounding context from video recordings offers interesting possibilities for context awareness of video capable mobile devices. Multimodal analysis provides means for improved recognition accuracy and robustness in different use conditions. We present a mul-timodal video context recognition system fusing audio and video cues with support vector machines (SVM) and simple rules with genetic algorithm (GA) optimized weights. Mul-timodal recognition is shown to outperform the unimodal approaches in recognizing between 21 everyday contexts. The highest correct classification rate of 0.844 is achieved with SVM-based fusion.

System profiles in content-based image indexing and retrieval

In this paper, a novel study on system profiles and adaptation of parameters for end-users of content-based indexing and retrieval (CBIR) applications are presented. The main objective of the study is improving the overall CBIR application performance in different hardware platforms having different technical capabilities and conditions. We define CBIR system profiles in terms of hardware and system platform attributes and propose CBIR parameters for each profile. Hence, the study consists of two main parts: system profiling and adaptation of indexing and retrieval parameters for each profile. The proposed CBIR parameters are appropriate configurations for optimal CBIR use on every platform. The proposed parameters for each system profile are assessed over a large set of experiments. Experimental studies show that the proposed parameters for each system profile have satisfactory semantic retrieval performance, with reduced computational complexity and storage space requirement. 45 to 78% improvement is achieved in the computational complexity of the retrieval process depending on the profile.

Unsupervised Elimination of Media Items in Content-Based Image Retrieval

A novel unsupervised method for elimination of irrelevant media items for retrieval is presented. The proposed method aims to decrease retrieval complexity and memory consumption of content-based indexing and retrieval systems. The method is suitable for layered systems that iterates query process more than one time to improve and/or refine the query results. The elimination is performed on the query results of the first step of retrieval. Experiments show that, the error rate of missing relevant images in the eliminated part of the database is ~0.01%. Low complexity and successful accuracy allow the proposed method to be used in various platforms such as distributed and limited systems.