Fernando López-Ostenero

Interactive Cross-Language Document Selection

The problem of finding documents written in a language that the searcher cannot read is perhaps the most challenging application of cross-language information retrieval technology. In interactive applications, that task involves at least two steps: (1) the machine locates promising documents in a collection that is larger than the searcher could scan, and (2) the searcher recognizes documents relevant to their intended use from among those nominated by the machine. This article presents the results of experiments designed to explore three techniques for supporting interactive relevance assessment: (1) full machine translation, (2) rapid term-by-term translation, and (3) focused phrase translation. Machine translation was found to better support this task than term-by-term translation, and focused phrase translation further improved recall without an adverse effect on precision. The article concludes with an assessment of the strengths and weaknesses of the evaluation framework used in this study and some remarks on implications of these results for future evaluation campaigns.

Noun phrases as building blocks for cross-language search assistance

This paper presents a Foreign-Language Search Assistant that uses noun phrases as fundamental units for document translation and query formulation, translation and refinement. The system (a) supports the foreign-language document selection task providing a cross-language indicative summary based on noun phrase translations, and (b) supports query formulation and refinement using the information displayed in the cross-language document summaries. Our results challenge two implicit assumptions in most of cross-language Information Retrieval research: first, that once documents in the target language are found, Machine Translation is the optimal way of informing the user about their contents; and second, that in an interactive setting the optimal way of formulating and refining the query is helping the user to choose appropriate translations for the query terms.

Interactive Cross-Language Searching: Phrases Are Better than Terms for Query Formulation and Refinement

This paper summarizes the participation of the UNED group in the CLEF 2002 Interactive Track. We focused on interactive query formulation and refinement, comparing two approaches: a) a reference system that assists the user to provide adequate translations for terms in the query; and b) a proposed system that assists the user to formulate the query as a set of relevant phrases, and to select promising phrases in the documents to enhance the query. All collected evidence indicates that the phrase-based approach is preferable: the official Fα=0.8 measure is 65% better for the proposed system, and all users in our experiment preferred the phrase-based system as a simpler and faster way of searching.

Noun Phrase Translations for Cross-Language Document Selection

This paper presents results for the CLEF Interactive Cross-Language Document Selection task at the UNED. Two translation techniques were compared: the standard Systran translations provided by the CLEF organizers as a baseline, and a phrase-based pseudo-translation approach that uses a phrase alignment algorithm based on comparable corpora. The hypothesis being tested was that noun phrase translations could serve as summarized information for relevance judgment without compromising the precision of such judgments. In addition, we wanted to have an indirect measure of the quality of our phrase extraction process, that had been previously developed for an interactive CLIR application. The results of the experiment confirm that the hypothesis is reasonable: a set of 8 monolingual Spanish speakers judged English documents with the same precision for both systems, but achieved 52% more recall using phrasal translations than using full Systran translations.

Interactive question answering: Is Cross-Language harder than monolingual searching?

Is Cross-Language answer finding harder than Monolingual answer finding for users? In this paper we provide initial quantitative and qualitative evidence to answer this question. In our study, which involves 16 users searching questions under four different system conditions, we find that interactive cross-language answer finding is not substantially harder (in terms of accuracy) than its monolingual counterpart, using general purpose Machine Translation systems and standard Information Retrieval machinery, although it takes more time. We have also seen that users need more context to provide accurate answers (full documents) than what is usually considered by systems (paragraphs or passages). Finally, we also discuss the limitations of standard evaluation methodologies for interactive Information Retrieval experiments in the case of cross-language question answering.

UNED at ImageCLEF 2004: detecting named entities and noun phrases for automatic query expansion and structuring

This paper describes UNED experiments at the Image CLEF bilingual ad hoc task. Two different strategies are attempted: i) automatic expansion and translation using noun phrases; ii) automatic detection of named entities in the query for structured search on image caption fields. All our experiments obtain results above the average MAP for the bilingual task. Structured searches using named entities improve performance over a strong baseline (Pirkolas structured query approach), achieving one of the best results for the whole bilingual track. Expansion with noun phrases, however, degrades results, possibly due to the mismatch between train and test collections.

UNED at iCLEF 2003: Searching Cross-Language Summaries

The UNED phrase-based cross-language summaries were first introduced at iCLEF 2001 as a translation strategy which permitted faster document selection with roughly the same accuracy than full Machine Translation. For our iCLEF 2003 participation, we tested the validity of our summaries as cross-language indexes for the retrieval stage of the interactive search process. We compared a reference system that performs query translation (and then retrieves target-language documents) with a system that directly retrieves cross-language summaries with the source-language query. The performance of both systems is very similar, confirming that UNED summaries are viable for cross-language indexing. This approach is trivially scalable to more than one target language, opening an interesting path for truly multilingual search assistance.

Interactive cross-language question answering: searching passages versus searching documents

iCLEF 2004 is the first comparative evaluation of interactive Cross-Language Question Answering systems. The UNED group has participated in this task comparing two strategies to help users in the answer finding task: the baseline system is just a standard document retrieval engine searching machine-translated versions of the documents; the contrastive system is identical, but searches passages which contain expressions of the appropriate answer type. Although the users prefer the passage system because searching is faster and simpler, it leads to slightly worse results, because the document context (which is not available in the passage retrieval system) turns out to be useful to verify the correctness of candidate answers; this makes an interesting difference with automatic Q&A systems. In addition, our experiment sets a strong baseline of 69% strict accuracy, showing that Cross-Language Question Answering can be efficiently accomplished by users without using dedicated Q&A technology.