Anis Ben Ammar

A fuzzy ontology: based framework for reasoning in visual video content analysis and indexing

Multimedia indexing systems based on semantic concept detectors are incomplete in the semantic sense. We can improve the effectiveness of these systems by using knowledge-based approaches which utilize semantic knowledge. In this paper, we propose a novel and efficient approach to enhance semantic concept detection in multimedia content, by exploiting contextual information about concepts from visual modality. First, a semantic knowledge is extracted via a contextual annotation framework. Second, a Fuzzy ontology is proposed to represent the fuzzy relationships (roles and rules) among every context and its semantic concepts. We use an abduction engine based on βeta function as a membership function for fuzzy rules. Third, a deduction engine is used to handle richer results in our video indexing system by running the proposed fuzzy ontology. Experiments on TRECVID 2010 benchmark have been performed to evaluate the performance of this approach. The obtained results show consistent improvement in semantic concepts detection, when a context space is used, and a good degree of indexing effectiveness as compared to existing approaches.

Improving image search effectiveness by integrating contextual information

Recently, image retrieval approaches shift to context-based reasoning. Context-based approaches proved their efficiency to improve retrieval process. In fact, conventional image search engines are often not able to satisfy the users intent as they provide noisy or/and redundant results. In addition, when a query is ambiguous, such systems can hardly distinguish different meanings for one query and therefore, they fail to show images with different contexts. A good system should provide, at top-k results, images which are the most relevant and diverse to guarantee users satisfaction. Our objective is to improve the retrieval process performance by harnessing the contextual information to measure the relevance score and diversity score. The proposed approach implies the relevance-based ranking where a random walk with restart offers a refining step, the diversity-based ranking and the combination. Our approach was evaluated in the context of ImageCLEF1 benchmark. Obtained results are promising especially for diversity-based ranking.

Effective Diversification for Ambiguous Queries in Social Image Retrieval

Recent years have witnessed a great popularity of social photos sharing websites, which host a tremendous volume of digital images accompanied by their associated tags. Thus, extensive research efforts have been dedicated to tag-based social image search which enables users to formulate their queries using tags. However, tag queries are often ambiguous and typically short. Search results diversification approach is the common solution which aims to increase the number of satisfied users using only a single results set that cover the maximum of query aspects. However, not all queries are uniformly ambiguous and hence different diversification strategies might be suggested. In such context, we propose a new ranking process which dynamically predicts an effective trade-off between the relevance and diversity based results ranking according to the ambiguity level of a given query. Thorough experiments using 12 ambiguous queries over the NUS-WIDE dataset show the effectiveness of our approach over classical uniform diversification approaches.

Effective concept detection using Second order Co-occurence Flickr context similarity measure SOCFCS

Automatic photo annotation task aims to describe the semantic content by detecting high level concepts. Most existing approaches are performed by training independent concept detectors omitting the interdependencies between concepts. The obtained annotations are often not so satisfactory. Therefore, a process of annotation refinement is mondatory to improve the imprecise annotation results. Recently, harnessing the contextual correlation between concepts is shown to be an important resource to improve concept detection. In this paper, we propose a new context based concept detection process. For this purpose, we define a new semantic measure called Second order Co-occurence Flickr context similarity (SOCFCS), which aggregates the FCS values of common Flickr related-tags of two target concepts in order to calculate their relative semantic context relatedness (SCR). Our proposed measure is applied to build a concept network as the context space. A Random Walk with Restart process is performed over this network to refine the annotation results by exploring the contextual correlation among concepts. Experimental studies are conducted on ImageCLEF 2011 Collection containing 99 concepts. The results demonstrate the effectiveness of our proposed approach.

Adaptive diversification for tag-based social image retrieval

With the great popularity of social photos sharing websites, a tremendous volume of digital images is hosted together with their associated tags. Thus, extensive research efforts have been dedicated to tag-based social image search which enables users to formulate their queries using tags. However, tag queries are often ambiguous and typically short. Diversifying search results is a common solution in the absence of further knowledge about the user’s intention. Such approach aims to retrieve relevant images covering as much of the diverse meanings the query may have. However, not all queries are uniformly ambiguous and hence different diversification strategies might be suggested. In such a context, two new processes are jointly investigated at query pre-processing and post-processing levels. On the one hand, we propose a multi-view concept-based query expansion process, using a predefined list of semantic concepts, which aims to weight concepts from different views or contexts, aggregate the obtained weights and select the most representative ones using a dynamic threshold. On the other hand, we propose a new ranking process called “adaptive diverse relevance ranking” which automatically predicts an effective trade-off between relevance scores and diversity scores according to the query ambiguity level. Thorough experiments using 12 ambiguous queries over the NUS-WIDE dataset show the effectiveness of our approach versus classical uniform diversification approaches.

Adaptive Semantic Construction for Diversity-based Image Retrieval

In recent years, the explosive growth of multimedia databases and digital libraries reveals crucial problems in indexing and retrieving images, what led us to develop our own approach. Our proposed approach TAD consists in disambiguating web queries to build an adaptive semantic for diversity-based image retrieval. In fact, the TAD approach is a puzzle constituted by three main components which are the TAWQU (Thesaurus-Based Ambiguous Web Query Understanding) process, the ASC (Adaptive Semantic Construction) process and the DR (Diversity-based Retrieval) process. The Wikipedia pages represent our main source of information. The NUS-WIDE dataset is the bedrock of our adaptive semantic. Actually, it permits us to perform a respectful evaluation. Fortunately, the experiments demonstrate promising results for the majority of the twelve ambiguous queries.

Flickr-based semantic context to refine automatic photo annotation

Automatic photo annotation task aims to describe the semantic content by detecting high level concepts in order to further facilitate concept based video retrieval. Most of existing approaches are based on independent semantic concept detectors without considering the contextual correlation between concepts. This drawback has its impact over the efficiency of such systems. Recently, harnessing contextual information to improve the effectiveness of concepts detection becomes a promising direction in such field. In this paper, we propose a new contextbased annotation refinement process. For this purpose, we define a new semantic measure called “Second Order Co-occurence Flickr context similarity” (SOCFCS) which aims to extract the semantic context correlation between two concepts by exploring Flickr resources (Flickr related-tags). Our measure is an extension of FCS measure by taking into consideration the FCS values of common Flickr related-tags of the two target concepts. Our proposed measure is applied to build a concept network which models the semantic context inter-relationships among concepts. A Random Walk with Restart process is performed over this network to refine the annotation results by exploring the contextual correlation among concepts. Experimental studies are conducted on ImageCLEF 2011 Collection containing 10000 images and 99 concepts. The results demonstrate the effectiveness of our proposed approach.

Enhanced context-based query-to-concept mapping in social image retrieval

In a concept based image retrieval system, query-by-concept mapping is a new trend of query formulation using a set of predefined concepts in order to improve retrieval effectiveness. The key challenge is how to select the appropriate concepts since many of them will not directly be named in the query? In this paper, we propose a new approach for query to concept mapping based on the contextual correlations inter-concepts. Our idea is to explore Flickr resources in order to extract such correlations which will be presented as an inter-concepts graph. A random walk process will be performed over this graph to discover implicit concepts which are relevant to the query. Experimental studies are conducted on ImageCLEF 2011 Collection containing 250000 images, 99 concepts and 40 queries. The results show that our system runs reasonably and confirm the effectiveness of the proposed approach.

Fuzzy reasoning framework to improve semantic video interpretation

A video retrieval system user hopes to find relevant information when the proposed queries are ambiguous. The retrieval process based on detecting concepts remains ineffective in such a situation. Potential relationships between concepts have been shown as a valuable knowledge resource that can enhance the retrieval effectiveness, even for ambiguous queries. Recent researches in multimedia retrieval have focused on ontology modeling as a common framework to manage knowledge. Handling these ontologies has to cope with issues related to generic knowledge management and processing scalability. Considering these issues, we suggest a context-based fuzzy ontology framework for video content analysis and indexing. In this paper, we focused on the way in which we modeled our fuzzy ontology: First, we populate automatically the generated ontology by gathering various available video annotation datasets. Then, the ontology content was used to infer enhanced video semantic interpretation. Finally, considering user feedback, the content of the ontology was improved. Experimental results showed that our approach achieves the goal of scalability while at the same time allowing better video content semantic interpretation.

A generic framework for semantic video indexing based on visual concepts/contexts detection

Providing a semantic access to video data requires the development of concept detectors. However, semantic concepts detection is a hard task due to the large intra-class and the small inter-class variability of content. Moreover, semantic concepts co-occur together in various contexts and their occurrence may vary from one to another. Thus, it is interesting to exploit this knowledge in order to achieve satisfactory performances. In this paper we present a generic semantic video indexing scheme, called SVI_REGIMVid. It is based on three levels of analysis. The first level (level1) focuses on low-level processing such as video shot boundary/key-frame detection, annotation tools, key-points detection and visual features extraction tools. The second level (level2) aims to build the semantic models for supervised learning of concepts/contexts. The third level (level3) enriches the semantic interpretation of concepts/contexts by exploiting fuzzy knowledge. The obtained experimental results are promising for a semantic concept/context detection process.