Amel Ksibi

Improving image search effectiveness by integrating contextual information

Recently, image retrieval approaches shift to context-based reasoning. Context-based approaches proved their efficiency to improve retrieval process. In fact, conventional image search engines are often not able to satisfy the users intent as they provide noisy or/and redundant results. In addition, when a query is ambiguous, such systems can hardly distinguish different meanings for one query and therefore, they fail to show images with different contexts. A good system should provide, at top-k results, images which are the most relevant and diverse to guarantee users satisfaction. Our objective is to improve the retrieval process performance by harnessing the contextual information to measure the relevance score and diversity score. The proposed approach implies the relevance-based ranking where a random walk with restart offers a refining step, the diversity-based ranking and the combination. Our approach was evaluated in the context of ImageCLEF1 benchmark. Obtained results are promising especially for diversity-based ranking.

Effective Diversification for Ambiguous Queries in Social Image Retrieval

Recent years have witnessed a great popularity of social photos sharing websites, which host a tremendous volume of digital images accompanied by their associated tags. Thus, extensive research efforts have been dedicated to tag-based social image search which enables users to formulate their queries using tags. However, tag queries are often ambiguous and typically short. Search results diversification approach is the common solution which aims to increase the number of satisfied users using only a single results set that cover the maximum of query aspects. However, not all queries are uniformly ambiguous and hence different diversification strategies might be suggested. In such context, we propose a new ranking process which dynamically predicts an effective trade-off between the relevance and diversity based results ranking according to the ambiguity level of a given query. Thorough experiments using 12 ambiguous queries over the NUS-WIDE dataset show the effectiveness of our approach over classical uniform diversification approaches.

Effective concept detection using Second order Co-occurence Flickr context similarity measure SOCFCS

Automatic photo annotation task aims to describe the semantic content by detecting high level concepts. Most existing approaches are performed by training independent concept detectors omitting the interdependencies between concepts. The obtained annotations are often not so satisfactory. Therefore, a process of annotation refinement is mondatory to improve the imprecise annotation results. Recently, harnessing the contextual correlation between concepts is shown to be an important resource to improve concept detection. In this paper, we propose a new context based concept detection process. For this purpose, we define a new semantic measure called Second order Co-occurence Flickr context similarity (SOCFCS), which aggregates the FCS values of common Flickr related-tags of two target concepts in order to calculate their relative semantic context relatedness (SCR). Our proposed measure is applied to build a concept network as the context space. A Random Walk with Restart process is performed over this network to refine the annotation results by exploring the contextual correlation among concepts. Experimental studies are conducted on ImageCLEF 2011 Collection containing 99 concepts. The results demonstrate the effectiveness of our proposed approach.

Adaptive diversification for tag-based social image retrieval

With the great popularity of social photos sharing websites, a tremendous volume of digital images is hosted together with their associated tags. Thus, extensive research efforts have been dedicated to tag-based social image search which enables users to formulate their queries using tags. However, tag queries are often ambiguous and typically short. Diversifying search results is a common solution in the absence of further knowledge about the user’s intention. Such approach aims to retrieve relevant images covering as much of the diverse meanings the query may have. However, not all queries are uniformly ambiguous and hence different diversification strategies might be suggested. In such a context, two new processes are jointly investigated at query pre-processing and post-processing levels. On the one hand, we propose a multi-view concept-based query expansion process, using a predefined list of semantic concepts, which aims to weight concepts from different views or contexts, aggregate the obtained weights and select the most representative ones using a dynamic threshold. On the other hand, we propose a new ranking process called “adaptive diverse relevance ranking” which automatically predicts an effective trade-off between relevance scores and diversity scores according to the query ambiguity level. Thorough experiments using 12 ambiguous queries over the NUS-WIDE dataset show the effectiveness of our approach versus classical uniform diversification approaches.

Flickr-based semantic context to refine automatic photo annotation

Automatic photo annotation task aims to describe the semantic content by detecting high level concepts in order to further facilitate concept based video retrieval. Most of existing approaches are based on independent semantic concept detectors without considering the contextual correlation between concepts. This drawback has its impact over the efficiency of such systems. Recently, harnessing contextual information to improve the effectiveness of concepts detection becomes a promising direction in such field. In this paper, we propose a new contextbased annotation refinement process. For this purpose, we define a new semantic measure called “Second Order Co-occurence Flickr context similarity” (SOCFCS) which aims to extract the semantic context correlation between two concepts by exploring Flickr resources (Flickr related-tags). Our measure is an extension of FCS measure by taking into consideration the FCS values of common Flickr related-tags of the two target concepts. Our proposed measure is applied to build a concept network which models the semantic context inter-relationships among concepts. A Random Walk with Restart process is performed over this network to refine the annotation results by exploring the contextual correlation among concepts. Experimental studies are conducted on ImageCLEF 2011 Collection containing 10000 images and 99 concepts. The results demonstrate the effectiveness of our proposed approach.

Enhanced context-based query-to-concept mapping in social image retrieval

In a concept based image retrieval system, query-by-concept mapping is a new trend of query formulation using a set of predefined concepts in order to improve retrieval effectiveness. The key challenge is how to select the appropriate concepts since many of them will not directly be named in the query? In this paper, we propose a new approach for query to concept mapping based on the contextual correlations inter-concepts. Our idea is to explore Flickr resources in order to extract such correlations which will be presented as an inter-concepts graph. A random walk process will be performed over this graph to discover implicit concepts which are relevant to the query. Experimental studies are conducted on ImageCLEF 2011 Collection containing 250000 images, 99 concepts and 40 queries. The results show that our system runs reasonably and confirm the effectiveness of the proposed approach.

Enhancing relevance re-ranking using nature-inspired meta-heuristic optimization algorithms

Over the last years, relevance re-ranking has been an attractive research, aiming to re-order the initial image search result list by which relevant ones should be at the top ranking list and irrelevant ones should be pruned. In this paper, we propose to explore two population-based meta-heuristic algorithms, which are Particle Swarm optimization(PSO), and Cuckoo search(CS), in order to solve the relevance re-ranking problem as a constrained regularisation framework. By doing so, we define two reranking processes, refereed as APSO-Rank and CS-Rank that converge to the optimal ranked list. Results are further provided to demonstrate the effectiveness and performance of these two reranking processes.

SOFF: Scalable and oriented FAST-based local features

Local feature detection is a fundamental module in several mobile vision applications such as mobile object recognition and mobile visual search. The effectiveness and the efficiency of a local feature detector decide to what extent it is suitable for a mobile application. Over the past decades, several local feature detectors have been developed. In this paper, we are interested in FAST (Features from Accelerated Segment Test) local feature detector for its efficiency. However, FAST detector shows poor robustness against both scale and rotation changes. Therefore, we aim at enhancing FAST robustness against both scale and rotation changes while maintaining good efficiency. To this end, we propose a Scalable and Oriented FAST-based local Feature detector (SOFF). A comprehensive comparison against FAST detector and its variants is performed on benchmark datasets. Experimental results demonstrate that SOFF detector outperforms other FAST-based detectors in many cases. Furthermore, it is efficient to compute, thereby suitable for mobile vision applications.

Semantic-aware framework for Mobile Image Search

Social and Mobile are the two very characterizing trends of the Internet. Subsequently, the volume of photos with rich social, textual and contextual information increases exponentially either on mobile devices or social networks. Performing an efficient and effective mobile Image Search over social photo collection is therefore a crucial challenge. Indeed, capture the complex connections among social photos is as important as speeding up similarity search at large scale. This paper present a generic Mobile Image Search framework with hypergraph hashing. On the mobile side, users are enabled to formulate whether visual, textual or vocal queries. On the server side, we start by modeling complex connections that may exist among photos and social features using an hypergraph. To accelerate the nearest neighbor search over the hypergraph, a spectral hashing is performed. Namely, each hypergraph vertex is mapped to a binary string without loss of similarity. For unseen items in the hypergraph, a query-adaptive supervised learning is carried out to learn binary strings based on the query type. We report the initial results over NUS-WIDE collection which show that the proposed framework is promising in the field of Mobile Image Search.