Joachim Köhler

IFINDER: an MPEG-7-based retrieval system for distributed multimedia content

This paper describes the MPEG-7 compliant indexing and retrieval system iFinder based on XML and open source database technology. The iFinder system automatically extracts metadata from A/V-content and allows access to the enriched content by means of a client/server-based retrieval engine. This multimedia retrieval system allows for search and retrieval of short video segments in huge multimedia archives. As a reference application, the iFinder system is used to index speeches from the German Parliament. The user can search for fragments of political speeches and receives matching video segments through a video streaming service.

Spoken content retrieval: Searching spontaneous conversational speech

The second workshop on Searching Spontaneous Conversational Speech (SSCS 2008) was held in Singapore on July 24, 2008 in conjunction with the 31st Annual International ACM SIGIR Conference. The goal of the workshop was to bring the speech community and the information retrieval community together. The forum was designed to be conducive to the close interaction and the intense discussion necessary to promote fusion of these fields into a single discipline with a concerted vision of spoken content retrieval. At the workshop, talks and posters were presented covering a wide range of topics including vocabulary independent search, spoken term detection, combination of models/indexes, use of speech recognition lattices for search, segmentation, temporal analysis, benchmarking, exploitation of prosody, speech surrogates for user interfaces and multi-language collections. Demonstrations of speech-based retrieval systems from a variety of application domains introduced a strong practical emphasis into the workshop program. The workshop concluded with a panel discussion, whose goal it was to identify future research directions for speech retrieval. Among the important challenges identified during the panel discussions were: dealing with large scale multimedia collections, representing audio/video content effectively in the user interface, focusing on perfecting the component technologies on which speech retrieval systems are based, and developing systems and approaches that will enable users (both content seekers and content providers) to actively create their own speech search applications or contribute to the indexability of their content.

SHARE: supporting large-scale rescue operations with communication and information services over mobile networks

This paper presents the EU project SHARE which is developing a mobile service architecture to support large-scale rescue operations with multimedia communication and information services. The task of planning and controlling large-scale rescue operations requires flexible and robust tools which help the rescue forces to do their search and rescue work with maximum efficiency. Today the main channels of communication are analog radio and paper text forms. The SHARE system introduces an advanced multimedia communication and information system to the highly mobile working environment which supports well established command hierarchies of the rescue organizations. Different operation scenarios for mobile networks using WiMAX, WLAN or UMTS are discussed regarding their usability for large-scale rescue operations.

Multimedia with a speech track: searching spontaneous conversational speech

After two successful years at SIGIR in 2007 and 2008, the third workshop on Searching Spontaneous Conversational Speech (SSCS 2009) was held conjunction with the ACM Multimedia 2009. The goal of the SSCS series is to serve as a forum that brings together the disciplines that collaborate on spoken content retrieval, including information retrieval, speech recognition and multimedia analysis. Multimedia collections often contain a speech track, but in many cases it is ignored or not fully exploited for information retrieval. Currently, spoken content retrieval research is expanding beyond highly-conventionalized domains such as broadcast news in to domains involving speech that is produced spontaneously and in conversational settings. Such speech is characterized by wide variability of speaking styles, subject matter and recording conditions. The work presented at SSCS 2009 included techniques for searching meetings, interviews, telephone conversations, podcasts and spoken annotations. The work encompassed a large range of approaches including using subword units, exploiting dialogue structure, fusing retrieval models, modeling topics and integrating visual features. Taken in sum, the workshop demonstrated the high potential of new ideas emerging in the area of speech search and also reinforced the need for concentrated research devoted to the classic challenges of spoken content retrieval, many of which remain yet unsolved.

Searching multimedia content with a spontaneous conversational speech track

Spoken document retrieval research effort invested into developing broadcast news retrieval systems has yielded impressive results. This paper is the introduction the proceedings of the 3rd workshop aiming at the advancement of the field in less explored domains (SSCS2009) which was organized in conjunction to the ACM Multimedia Conference in Beijing.

The MoveOn database: motorcycle environment speech and noise database for command and control applications

The MoveOn speech and noise database was purposely designed and implemented in support of research on spoken dialogue interaction in a motorcycle environment. The distinctiveness of the MoveOn database results from the requirements of the application domain—an information support and operational command and control system for the two-wheel police force—and also from the specifics of the adverse open-air acoustic environment. In this article, we first outline the target application, motivating the database design and purpose, and then report on the implementation details. The main challenges related to the choice of equipment, the organization of recording sessions, and some difficulties that were experienced during this effort, are discussed. We offer a detailed account of the database statistics, the suggested data splits in subsets, and discuss results from automatic speech recognition experiments which illustrate the degree of complexity of the operational environment.