David Zellhöfer

ImageCLEF 2013: The Vision, the Data and the Open Challenges

This paper presents an overview of the ImageCLEF 2013 lab. Since its first edition in 2003, ImageCLEF has become one of the key initiatives promoting the benchmark evaluation of algorithms for the cross-language annotation and retrieval of images in various domains, such as public and personal images, to data acquired by mobile robot platforms and botanic collections. Over the years, by providing new data collections and challenging tasks to the community of interest, the ImageCLEF lab has achieved an unique position in the multi lingual image annotation and retrieval research landscape. The 2013 edition consisted of three tasks: the photo annotation and retrieval task, the plant identification task and the robot vision task. Furthermore, the medical annotation task, that traditionally has been under the ImageCLEF umbrella and that this year celebrates its tenth anniversary, has been organized in conjunction with AMIA for the first time. The paper describes the tasks and the 2013 competition, giving an unifying perspective of the present activities of the lab while discussion the future challenges and opportunities.

A preference-based approach for interactive weight learning: learning weights within a logic-based query language

The result quality of queries incorporating impreciseness can be improved by the specification of user-defined weights. Existing approaches evaluate weighted queries by applying arithmetic evaluations on top of the query’s intrinsic logic. This complicates the usage of logic-based optimization. Therefore, we suggest a weighting approach that is completely embedded in a logic.In order to facilitate the user interaction with the system, we exploit the intuitively comprehensible concept of preferences. In addition, we use a machine-based learning algorithm to learn weighting values in correspondence to the user’s intended semantics of a posed query. Experiments show the utility of our approach.

Towards quantum-based DB+IR processing based on the principle of polyrepresentation

The cognitively motivated principle of polyrepresentation still lacks a theoretical foundation in IR. In this work, we discuss two competing polyrepresentation frameworks that are based on quantum theory. Both approaches support different aspects of polyrepresentation, where one is focused on the geometric properties of quantum theory while the other has a strong logical basis. We compare both approaches and outline how they can be combined to express further aspects of polyrepresentation.

An extensible personal photograph collection for graded relevance assessments and user simulation

In this paper, we present a document collection with graded relevance assessments that has been sampled from real photographers. In order to reflect both vagueness in the commonly used retrieval calculations as well as in the users query, we argue that current document collections being based on binary relevance judgments have drawbacks -- in particular if user-centered or relevance feedback-related experiments are conducted. In addition, such system-centric collections are based on documents, which do not necessarily reflect a laypersons personal photo collection. To overcome this issue, we suggest a test set of documents that is based on a study of 19 real photographers discriminating it from Flickr downloads or the like. The collection has been categorized on basis of different criteria such as document quality or motif quality plus the aforementioned graded relevance assessments. Reflecting the photograph taking behavior of the investigated photographers, we are also providing an event-based ground-truth in addition to a topic-based one. In total, 130 different topics are available for the collection. In order to provide means to address different photographer and user types, e.g. in user simulations or in usability engineering, we make the demographic information of both photographers and assessors available. Eventually, this links interactive information retrieval evaluation with persona-based interaction design -- a factor that has been neglected in multimedia information retrieval so far.

PythiaSearch: a multiple search strategy-supportive multimedia retrieval system

PythiaSearch is a multimedia information retrieval system supporting multiple search strategies. Based on the promising results of the underlying query model in 2011s Image-CLEF Wikipedia task, we have implemented an interactive retrieval system which supports multimodal data such as images, (multilingual) texts, and various metadata formats that can be used to query or browse a collection. The support of multiple search strategies is crucial, because it is subject to change during the users interaction with the retrieval system. The directed search and browsing mechanisms rely both on the same formal query model providing a seamless adaption to the users search strategy. Additionally, it features a relevance feedback process that can be used to adjust or even learn specific queries based on the users interaction with the system alone which can be saved for later usage.

A user interaction model based on the principle of polyrepresentation

Recently, the cognitively motivated principle of polyrepresentation has been shown to correlate with quantum mechanics-inspired IR models. The principles core hypothesis is that a document is defined by different representations such as low-level features, textual content, or the users context. Eventually, these representations can be used to form a cognitive overlap in which highly relevant documents are likely to be contained. In this work, we present a user interaction model mapping the principle onto a quantum logic-based retrieval model. The novelty of our approach is that - because of the cognitive basis of our work - the same model can be used from the retrieval model up to the user interaction model resulting in a new consistency throughout the interactive retrieval process. To cope with the inherent dynamics of the search process, the model allows users to use relevance feedback to further personalize the cognitive overlap. In addition, we address the open issue of information need drifts, i.e., if the cognitive overlap has to be adjusted. The accompanying experiments in the CBIR domain indicate the utility of our approach. Regarding relevance feedback, the retrieval model adjusts well even to small feedback data. If relevance feedback is not feasible, the retrieval performs appropriately for structured or unstructured queries, which are both supported to model the cognitive overlap. In conclusion, we discuss the potential of the presented approach for multimodal retrieval, how it can contribute to bridging DB and IR, and relate it to other contributions.

Towards quantum logic based multimedia retrieval

Searching in a huge collection of multimedia objects is a very complex task comprising different search paradigms such as classical retrieval search, database querying, and search with proximity conditions as well as a way to assign weights to different conditions. Thus, we need a sound formalism unifying underlying search concepts. The framework of fuzzy logic is often seen as such a unifying framework. Our work identifies some problems with fuzzy logic for multimedia retrieval, namely the dominance problem and the problem of violated logical laws such as idempotence and associativity especially when weighting is involved. Our approach is to apply a mathematical framework inspired by the theory of quantum mechanics and logic. We show that our quantum logic based query language overcomes problems of fuzzy logic in our context. This is due to the fact that we regard more semantics of a query and its involved conditions in comparison to fuzzy logic. A fuzzy t-norm, for example, deals with membership values whereas the conjunction in our framework is performed on subqueries.

A poset based approach for condition weighting

Current research in multimedia retrieval (MR) does not satisfactorily mirror research results from psychology revealing a different significance of certain characteristics of a media object to a query in terms of similarity. Although the relevance of user-controlled condition weights has been demonstrated, there is a lack of systems supporting users in setting these weights. In this work, we present a relevance feedback based approach that supports users to set condition weights in order to retrieve results from the MR system that are consistent with their perception of similarity. Condition weights are learned by a machine based learning algorithm from user preferences based on a partially ordered set.

Approaching multimedia retrieval from a polyrepresentative perspective

Multimedia documents such as videos, images, or music are characterized by an amount of different qualities that can become relevant during a search task. These qualities are seldom reflected as a whole by retrieval models. Thus, we present a new query model, which fully supports the principle of polyrepresentation by taking advantage of quantum logic. We offer means to model document relevance as a cognitive overlap from various features describing a multimedia document internally. Using our query model, the combination of the aforementioned polyrepresentative features is supported by the mechanisms of a Boolean algebra. In addition, these overlaps can be personalized by user preferences during a machine-based learning supported relevance feedback process. The input for the relevance feedback is based on qualitative judgments between documents, which are known from daily life, to keep the cognitive load on users low. We further discuss how our model contributes to the unification of different aspects of polyrepresentation into one sound theory.

Condition learning from user preferences

The utility of preferences within the database domain is widely accepted. Preferences provide an effective means for query personalization and information filtering. Nevertheless, two preference approaches - qualitative and quantitative ones - do still compete. In this paper, we contribute to the bridging of both approaches and compare their expressive power and different usage scenarios. In order to combine qualitative and quantitative preferences, mappings are introduced and discussed, which transform a query from one approach into its counter-part. We consider Chomickis preference formulas and as a quantitative approach our CQQL approach that extends the relational calculus with proximity predicates. In order to facilitate query formulation for the user, we extend the CQQL approach to condition learning. That is, user-defined preferences amongst database objects serve as input to learn logical conditions within a CQQL query. Hereby, we can support the user in the cognitively demanding task of query formulation.