Peter Poller

An Efficient Kernel for Multilingual Generation in Speech-to-Speech Dialogue Translation

We present core aspects of a fully implemented generation component in a multilingual speech-to-speech dialogue translation system. Its design was particularly influenced by the necessity of real-time processing and usability for multiple languages and domains. We developed a general kernel system comprising a microplanning and a syntactic realizer module. The microplanner performs lexical and syntactic choice, based on constraint-satisfaction techniques. The syntactic realizer processes HPSG grammars reflecting the latest developments of the underlying linguistic theory, utilizing their pre-processing into the TAG formalism. The declarative nature of the knowledge bases, i.e., the microplanning constraints and the HPSG grammars allowed an easy adaption to new domains and languages. The successful integration of our component into the translation system Verbmobil proved the fulfillment of the specific real-time constraints.

Multilingual Summary Generation in a Speech-To-Speech Translation System for Multilingual Dialogues

This paper describes a novel functionality of the VERBMOBIL system, a large scale translation system designed for spontaneously spoken multilingual negotiation dialogues. The task is the on-demand generation of dialogue scripts and result summaries of dialogues. We focus on summary generation and show how the relevant data are selected from the dialogue memory and how they are packed into an appropriate abstract representation. Finally, we demonstrate how the existing generation module of VERBMOBIL was extended to produce multilingual and result summaries from these representations.

The Verbmobil Generation Component VM-GECO

This chapter presents the Verbmobil generation component VM-GECO. The main modules of our component—microplanner and syntactic generator—are illuminated in detail focusing on the problems of real-time computation, multilinguality, dependencies among choices and the use of different representation formalisms. We discuss robustness as an important feature of large-scale systems with spontaneous and erroneous input.

Hybrid Multi-step Disfluency Detection

Previous research has shown that speech disfluencies - speech errors that occur in spoken language - affect NLP systems and hence need to be repaired or at least marked. This study presents a hybrid approach that uses different detection techniques for this task where each of these techniques is specialized within its own disfluency domain. A thorough investigation of the used disfluency scheme, which was developed by [1], led us to a detection design where basic rule-matching techniques are combined with machine learning approaches. The aim was both to reduce computational overhead and processing time and also to increase the detection performance. In fact, our system works with an accuracy of 92.9% and an F-Score of 90.6% while working faster than real-time.

An extended architecture for robust generation

Based on our experiences in VERBMOBIL, a large scale speech-to-speech translation system, we identify two types of problems that a generation component must address in a realistic implementation and present relevant examples. As an extension to the architecture of a translation system, we present a module for robustness preprocessing on the interface between translation and generation.

The SAMMIE System: Multimodal In-Car Dialogue

The SAMMIE system is an in-car multi-modal dialogue system for an MP3 application. It is used as a testing environment for our research in natural, intuitive mixed-initiative interaction, with particular emphasis on multimodal output planning and realization aimed to produce output adapted to the context, including the drivers attention state w.r.t. the primary driving task.

Generating Multilingual Dialog Summaries and Minutes

This chapter describes the on-demand generation of dialog minutes and result summaries of dialogs. We focus on summary generation since the generation of minutes is performed using almost the same techniques. We describe how the relevant data are selected from the dialog memory, how the data are converted into sequences of VITs and, finally, we demonstrate how the existing generation module of Verbmobil was extended to generate textual documents. Multilinguality is achieved by utilizing the transfer module.