Khalid Choukri

European speech databases for telephone applications

The SpeechDat project aims to produce speech databases for all official languages of the European Union and some major dialectal variants and minority languages resulting in 28 speech databases. They will be recorded over fixed and mobile telephone networks. This will provide a realistic basis for training and assessment of both isolated and continuous-speech utterances, employing whole-word or subword approaches, and thus can be used for developing voice driven teleservices including speaker verification. The specification of the databases has been developed jointly, and is essentially the same for each language to facilitate dissemination and use. There will be a controlled variation among the speakers concerning sex, age, dialect, environment of call, etc. The validation of all databases will be carried out centrally. The SpeechDat databases will be transferred to ELRA for distribution. The next databases to be recorded will cover East European languages.

The Maurdor Project: Improving Automatic Processing of Digital Documents

This paper presents the achievements of an experimental project called Maurdor (Moyens AUtomatisés deReconnaissance de Documents ecRits - Automatic Processingof Digital Documents) funded by the French DGA that aims at improving processing technologies for handwritten and typewritten documents in French, English and Arabic. The first part describes the context and objectives of the project. The second part deals with the challenge of creating a realistic corpus of 10,000 annotated documents to support the efficient development and evaluation of processing modules. The third part presents the organisation, metric definition and results of the Maurdor International evaluation campaign. The last part presents the Maurdor demonstrator with a functional and technical perspective.

PROMISE retreat report prospects and opportunities for information access evaluation

The PROMISE network of excellence organized a two-days brainstorming workshop on 30th and 31st May 2012 in Padua, Italy, to discuss and envisage future directions and perspectives for the evaluation of information access and retrieval systems in multiple languages and multiple media. This document reports on the outcomes of this event and provides details about the six envisaged research lines: search applications; contextual evaluation; challenges in test collection design and exploitation; component-based evaluation; ongoing evaluation; and signal-aware evaluation. The ultimate goal of the PROMISE retreat is to stimulate and involve the research community along these research lines and to provide funding agencies with effective and scientifically sound ideas for coordinating and supporting information access research.

Overview of CLEF 2008 INFILE pilot track

The INFILE campaign was run for the first time as a pilot track in CLEF 2008. Its purpose was the evaluation of cross-language adaptive filtering systems. It used a corpus of 300,000 newswires from Agence France Presse (AFP) in three languages: Arabic, English and French, and a set of 50 topics in general and specific domain (scientific and technological information). Due to delays in the organization of the task, the campaign only had 3 submissions (from one participant) which are presented in this article.

ELRA – European Language Resources Association-Background, Recent Developments and Future Perspectives

The European Language Resources Association (ELRA) was founded in 1995 with the mission of providing language resources (LR) to European research institutions and companies. In this paper we describe the background, the mission and the major activities since then.

Information filtering evaluation: overview of CLEF 2009 INFILE track

The INFILE@CLEF 2009 is the second edition of a track on the evaluation of cross-language adaptive filtering systems. It uses the same corpus as the 2008 track, composed of 300,000 newswires from Agence France Presse (AFP) in three languages: Arabic, English and French, and a set of 50 topics in general and specific domains (scientific and technological information). In 2009, we proposed two tasks : a batch filtering task and an interactive task to test adaptive methods. Results for the two tasks are presented in this paper.

The language resource Strategic Agenda: the FLaReNet synthesis of community recommendations

Abstract The main purpose of this paper is to serve as a landmark for future research and in particular for future strategic, infrastructural and coordination initiatives. It presents a preliminary plan for actions and infrastructures that could become the basis for future initiatives in the sector of Language Resources and Technologies (LRTs). The FLaReNet Language Resource Strategic Agenda presents a set of recommendations for the development and progress of LRT in Europe, as issued from a three-year consultation of the FLaReNet European project. Recommendations cover a broad range of topics and activities, spanning over production and use of language resources, licensing, maintenance and preservation issues, infrastructures for language resources, resource identification and sharing, evaluation and validation, interoperability and policy issues. The intended recipients belong to a large set of players and stakeholders in LRT, ranging from individuals to research and education institutions, to policy-makers, funding agencies, SMEs and large companies, service and media providers. The main goal of these recommendations is to serve as an instrument to support stakeholders in planning for and addressing the urgencies of the LRT of the future.

A PROMISE for experimental evaluation

Participative Research laboratory for Multimedia and Multilingual Information Systems Evaluation (PROMISE) is a Network of Excellence, starting in conjunction with this first independent CLEF 2010 conference, and designed to support and develop the evaluation of multilingual and multimedia information access systems, largely through the activities taking place in Cross-Language Evaluation Forum (CLEF) today, and taking it forward in important new ways. PROMISE is coordinated by the University of Padua, and comprises 10 partners: the Swedish Institute for Computer Science, the University of Amsterdam, Sapienza University of Rome, University of Applied Sciences of Western Switzerland, the Information Retrieval Facility, the Zurich University of Applied Sciences, the Humboldt University of Berlin, the Evaluation and Language Resources Distribution Agency, and the Centre for the Evaluation of Language Communication Technologies. The single most important step forward for multilingual and multimedia information access which PROMISE will work towards is to provide an open evaluation infrastructure in order to support automation and collaboration in the evaluation process.

Linguistic Data Resources

This chapter provides an overview of available language resources, from both U.S. and European perspectives. Multilingual data repositories as well as large ongoing and planned collection efforts are introduced, along with a description of the major challenges of collection efforts, such as transcription issues due to inconsistent writing standards, subject recruitment, recording equipment, legal aspects, and costs in terms of time and money. The overview of multilingual resources comprises multilingual audio and text data, pronunciation dictionaries, and parallel bilingual/multilingual corpora. This chapter provides an overview of existing language resources in Europe. A number of projects in Europe have been working toward the production of multilingual speech and language resources, many of which have become key databases for the human language technology (HLT) community. The SpeechDat projects are a set of speech data-collection efforts funded by the European Commission with the aim of establishing databases for the development of voice-operated teleservices and speech interfaces. The resulting databases are available via European Language Resources Association (ELRA).

Perceptual Component Evaluation and Data Collection

Systematic evaluation is essential to drive the rapid progress of a broad range of audiovisual perceptual technologies. Within the CHIL project, such evaluations were undertaken on an annual basis, so that improvements could be measured objectively, and different approaches compared and assessed.