Thomas Mandl

Advances in Multilingual and Multimodal Information Retrieval

This book constitutes the thoroughly refereed proceedings of the 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, held in Budapest, Hungary, September 2007. The revised and extended papers were carefully reviewed and selected for inclusion in the book. There are 115 contributions in total and an introduction. The seven distinct evaluation tracks in CLEF 2007, are designed to test the performance of a wide range of multilingual information access systems or system components. The papers are organized in topical sections on Multilingual Textual Document Retrieval (Ad Hoc), Domain-Specific Information Retrieval (Domain-Specific), Multiple Language Question Answering (QA@CLEF), cross-language retrieval in image collections (Image CLEF), cross-language speech retrieval (CL-SR), multilingual Web retrieval (WebCLEF), cross-language geographical retrieval (GeoCLEF), and CLEF in other evaluations.

GeoCLEF 2008: the CLEF 2008 cross-language geographic information retrieval track overview

GeoCLEF is an evaluation task running under the scope of the Cross Language Evaluation Forum (CLEF). The purpose of GeoCLEF is to test and evaluate cross-language geographic information retrieval (GIR). The GeoCLEF 2008 task presented twenty-five geographically challenging search topics for English, German and Portuguese. Eleven participants submitted 131 runs, based on a variety of approaches, including sample documents, named entity extraction and ontology based retrieval. The evaluation methodology and results are presented in the paper.

Implementation and evaluation of a quality-based search engine

In this paper, an approach for the implementation of a quality-based Web search engine is proposed. Quality retrieval is introduced and an overview on previous efforts to implement such a service is given. Machine learning approaches are identified as the most promising methods to determine the quality of Web pages. Features for the most appropriate characterization of Web pages are determined. A quality model is developed based on human judgments. This model is integrated into a meta search engine which assesses the quality of all results at run time. The evaluation results show that quality based ranking does lead to better results concerning the perceived quality of Web pages presented in the result set. The quality models are exploited to identify potentially important features and characteristics for the quality of Web pages.

The effect of named entities on effectiveness in cross-language information retrieval evaluation

The large number of experiments carried out within evaluation initiatives for information retrieval has led to an invaluable source for further research and meta-analysis. In this study, an analysis of the results of the Cross Language Evaluation Forum (CLEF) campaigns for the years 2000 to 2003 is presented. This study considers the performance of the systems for each individual topic. It is dedicated to the influence of named entities on retrieval performance. Named entities in topics lead to significant improvement of the retrieval quality in general and for most systems and tasks. The performance of systems varies for topics without, with one or two and with three or more named entities. This knowledge gained by data mining on the evaluation results can be exploited for the improvement of retrieval systems as well as for the design of topics for future CLEF campaigns.

Barriers to Information Access across Languages on the Internet: Network and Language Effects

This paper investigates the role of language in accessing information on the Internet. We combined data about website visitors through log-file analysis with data about web-hosts and links obtained from a crawler. Results suggest that language may represent a double barrier: first, the number of native speakers determines the number of web-hosts, and hence the amount of information and the interconnectedness of information sources. Second, to access information on a particular website the languages offered are an even more important factor than network effects: non-native speakers and links from websites in other languages are always underrepresented. Our results are in line with the Information Foraging Theory, the Revised Hierarchy Model, network and market theories, and emphasize the role of language on the Internet. Insight into these processes is helpful when website translation represents important investment decisions, or when aiming to diminish the digital divide.

Linguistic and Statistical Analysis of the CLEF Topics

This paper reports on an analysis of the CLEF 2001 topics. In particular, we investigated potential correlations between features of the topics and the performance of retrieval systems. Although there are some weak relations, we claim that the properties of the CLEF topics do not influence the results of the retrieval systems. We found just one correlation for the English topics. The more linguistic challenges contained in the topic texts, the better the systems performed. However, no correlation for the length of a topic could be found.

CLEF 2009 ad hoc track overview: robust-WSD task

The Robust-WSD at CLEF 2009 aims at exploring the contribution ofWord Sense Disambiguation to monolingual and multilingual Information Retrieval. The organizers of the task provide documents and topics which have been automatically tagged with Word Senses from WordNet using several state-of-the-art Word Sense Disambiguation systems. The Robust-WSD exercise follows the same design as in 2008. It uses two languages often used in previous CLEF campaigns (English, Spanish). Documents were in English, and topics in both English and Spanish. The document collections are based on the widely used LA94 and GH95 news collections. All instructions and datasets required to replicate the experiment are available from the organizers website (http://ixa2.si.ehu.es/clirwsd/). The results show that some top-scoring systems improve their IR and CLIR results with the use of WSD tags, but the best scoring runs do not use WSD.

Comparing Chinese and German blogs

Blogs in different countries do not only differ in the language of their texts but in many other aspects as well. This study explores how these differences can be identified and related to known cultural differences. A thorough intellectual analysis of several hundreds of blog pages from China and Germany revealed culturally diverse patterns. Chinese blogs are more graphically oriented. They emphasize the communication between bloggers and commentators. Especially, the distinction between high and low context communication in both cultures seems to have a large impact on the blog communication.

Tolerant Information Retrieval with Backpropagation Networks

Neural networks can learn from human decisions and preferences. Especially in human-computer interaction, adaptation to the behaviour and expectations of the user is necessary. In information retrieval, an important area within human-computer interaction, expectations are difficult to meet. The inherently vague nature of information retrieval has led to the application of vague processing techniques. Neural networks seem to have great potential to model the cognitive processes involved more appropriately. Current models based on neural networks and their implications for human-computer interaction are analysed. COSIMIR (Cognitive Similarity Learning in Information Retrieval), an innovative model integrating human knowledge into the core of the retrieval process, is presented. It applies backpropagation to information retrieval, integrating human-centred and soft and tolerant computing into the core of the retrieval process. A further backpropagation model, the transformation network for heterogeneous data sources, is discussed. Empirical evaluations have provided promising results.

Evaluation of a language identification system for mono- and multilingual text documents

Language identification is an important task for web information retrieval. This paper presents the implementation of a tool for language identification in mono- and multi-lingual documents. The tool implements four algorithms for language identification. Furthermore, we present a n-gram approach for the identification of languages in multi-lingual documents. An evaluation for monolingual texts of varied length is presented. Results for eight languages including Ukrainian and Russian are presented. It could be shown that n-gram-based approaches outperform word-based algorithms for short texts. For longer texts, the performance is comparable. The evaluation for multi-lingual documents is based on real world web documents. Our tool is able to recognize the languages present in a document with reasonable accuracy.