M. de Rijke

Adding Semantics to Detectors for Video Retrieval

In this paper, we propose an automatic video retrieval method based on high-level concept detectors. Research in video analysis has reached the point where over 100 concept detectors can be learned in a generic fashion, albeit with mixed performance. Such a set of detectors is very small still compared to ontologies aiming to capture the full vocabulary a user has. We aim to throw a bridge between the two fields by building a multimedia thesaurus, i.e., a set of machine learned concept detectors that is enriched with semantic descriptions and semantic structure obtained from WordNet. Given a multimodal user query, we identify three strategies to select a relevant detector from this thesaurus, namely: text matching, ontology querying, and semantic visual querying. We evaluate the methods against the automatic search task of the TRECVID 2005 video retrieval benchmark, using a news video archive of 85 h in combination with a thesaurus of 363 machine learned concept detectors. We assess the influence of thesaurus size on video search performance, evaluate and compare the multimodal selection strategies for concept detectors, and finally discuss their combined potential using oracle fusion. The set of queries in the TRECVID 2005 corpus is too small for us to be definite in our conclusions, but the results suggest promising new lines of research.

A Note on Graded Modal Logic

We introduce a notion of bisimulation for graded modal logic. Using these bisimulations the model theory of graded modal logic can be developed in a uniform manner. We illustrate this by establishing the finite model property, and proving invariance and definability results.We introduce a notion of bisimulation for graded modal logic. Using this notion, the model theory of graded modal logic can be developed in a uniform manner. We illustrate this by establishing the finite model property and proving invariance and definability results.

Hybrid logics on linear structures: expressivity and complexity

We investigate expressivity and complexity of hybrid logics on linear structures. Hybrid logics are an enrichment of modal logics with certain first-order features which are algorithmically well behaved. Therefore, they are well suited for the specification of certain properties of computational systems. We show that hybrid logics are more expressive than usual modal and temporal logics on linear structures, and exhibit a hierarchy of hybrid languages. We determine the complexities of the satisfiability problem for these languages and define an existential fragment of hybrid logic for which satisfiability is still NP-complete. Finally, we examine the linear time model checking problem for hybrid logics and its complexity.

CTL model checking for processing simple XPath queries

The eXtensible Markup Language (XML) was designed to describe the content of a document and its hierarchical structure, and the XML Path language (XPath) is a language for selecting elements from XML documents. There is a close connection between the query processing problem for XPath and the model checking problem for temporal logics. Both boil down to checking which nodes of a graph satisfy a property. We investigate the potential of a technique based on computation tree logic (CTL) model checking for evaluating queries expressed in (a subset of) XPath. To this aim, we isolate a simple fragment of XPath that is naturally embeddable into CTL. We report on experiments based on the model checker NuSMV, and compare our results with alternative academic XPath processors. We comment on the advantages and drawbacks of the application of our model checking-based approach to XPath processing.

Fact Discovery in Wikipedia

We address the task of extracting focused salient information items, relevant and important for a given topic, from a large encyclopedic resource. Specifically, for a given topic (a Wikipedia article) we identify snippets from other articles in Wikipedia that contain important information for the topic of the original article, without duplicates. We compare several methods for addressing the task, and find that a mixture of content-based, link-based, and layout-based features outperforms other methods, especially in combination with the use of so-called reference corpora that capture the key properties of entities of a common type.

Content-Based Analysis Improves Audiovisual Archive Retrieval

Content-based video retrieval is maturing to the point where it can be used in real-world retrieval practices. One such practice is the audiovisual archive, whose users increasingly require fine-grained access to broadcast television content. In this paper, we take into account the information needs and retrieval data already present in the audiovisual archive, and demonstrate that retrieval performance can be significantly improved when content-based methods are applied to search. To the best of our knowledge, this is the first time that the practice of an audiovisual archive has been taken into account for quantitative retrieval evaluation. To arrive at our main result, we propose an evaluation methodology tailored to the specific needs and circumstances of the audiovisual archive, which are typically missed by existing evaluation initiatives. We utilize logged searches, content purchases, session information, and simulators to create realistic query sets and relevance judgments. To reflect the retrieval practice of both the archive and the video retrieval community as closely as possible, our experiments with three video search engines incorporate archive-created catalog entries as well as state-of-the-art multimedia content analysis results. A detailed query-level analysis indicates that individual content-based retrieval methods such as transcript-based retrieval and concept-based retrieval yield approximately equal performance gains. When combined, we find that content-based video retrieval incorporated into the archives practice results in significant performance increases for shot retrieval and for retrieving entire television programs. The time has come for audiovisual archives to start accommodating content-based video retrieval methods into their daily practice.

Feature interaction as a satisfiability problem

We present a formal model for the specification of telephone features by means of description logics. Our framework permits the formal definition of the basic telephone system as well as the specification of additional features (call waiting, call forwarding, etc.). Furthermore, by using standard reasoning tasks from description logics, the properties of features can be formally proved and interactions detected. An EXPTIME upper bound for the complexity of detecting feature interaction as a satisfiability problem is provided by exploiting well-known results for expressive description languages.

Machine Learning for Question Answering from Tabular Data

Question Answering (QA) systems automatically answer natural language questions in a human-like manner. One of the practical approaches to open domain QA consists in extracting facts from free text offline and using a lookup mechanism when answering users questions online. This approach is related to natural language interfaces to databases (NLIDBs) that were studied extensively from the 1970s to the 1990s. NLIDB systems employed a range of techniques, from simple pattern-matching rules to formal logical calculi such as the lambda calculus, but most were restricted to specific domains. In this paper we describe a machine learning approach to querying tabular data for QA which is not restricted to specific domains. Our approach consists of two steps: for an incoming question, we first use a classifier to identify appropriate tables and columns in a structured database, and then employ a free-text retrieval to look up answers. The system uses part-of-speech tagging, named-entity normalization and a statistical classifier trained on data from the TREC QA task. With the TREC QA data, our system is shown to significantly outperform an existing rule-based table lookup method.