Stephan Peitz

EU-BRIDGE MT: Combined Machine Translation

This paper describes one of the collaborative efforts within EU-BRIDGE to further advance the state of the art in machine translation between two European language pairs, German→English and English→German. Three research institutes involved in the EU-BRIDGE project combined their individual machine translation systems and participated with a joint setup in the shared translation task of the evaluation campaign at the ACL 2014 Eighth Workshop on Statistical Machine Translation (WMT 2014). We combined up to nine different machine translation engines via system combination. RWTH Aachen University, the University of Edinburgh, and Karlsruhe Institute of Technology developed several individual systems which serve as system combination input. We devoted special attention to building syntax-based systems and combining them with the phrasebased ones. The joint setups yield empirical gains of up to 1.6 points in BLEU and 1.0 points in TER on the WMT newstest2013 test set compared to the best single systems.

A Guide to Jane, an Open Source Hierarchical Translation Toolkit

A Guide to Jane, an Open Source Hierarchical Translation Toolkit Jane is RWTHs hierarchical phrase-based translation toolkit. It includes tools for phrase extraction, translation and scaling factor optimization, with efficient and documented programs of which large parts can be parallelized. The decoder features syntactic enhancements, reorderings, triplet models, discriminative word lexica, and support for a variety of language model formats. In this article, we will review the main features of Jane and explain the overall architecture. We will also indicate where and how new models can be included.

Hierarchical Phrase-Based Translation with Jane 2

Hierarchical Phrase-Based Translation with Jane 2 In this paper, we give a survey of several recent extensions to hierarchical phrase-based machine translation that have been implemented in version 2 of Jane, RWTHs open source statistical machine translation toolkit. We focus on the following techniques: Insertion and deletion models, lexical scoring variants, reordering extensions with non-lexicalized reordering rules and with a discriminative lexicalized reordering model, and soft string-to-dependency hierarchical machine translation. We describe the fundamentals of each of these techniques and present experimental results obtained with Jane 2 to confirm their usefulness in state-of-the-art hierarchical phrase-based translation (HPBT).

Local System Voting Feature for Machine Translation System Combination

In this paper, we enhance the traditional confusion network system combination approach with an additional model trained by a neural network. This work is motivated by the fact that the commonly used binary system voting models only assign each input system a global weight which is responsible for the global impact of each input system on all translations. This prevents individual systems with low system weights from having influence on the system combination output, although in some situations this could be helpful. Further, words which have only been seen by one or few systems rarely have a chance of being present in the combined output. We train a local system voting model by a neural network which is based on the words themselves and the combinatorial occurrences of the different system outputs. This gives system combination the option to prefer other systems at different word positions even for the same sentence.

A Comparison of Update Strategies for Large-Scale Maximum Expected BLEU Training

This work presents a flexible and efficient discriminative training approach for statistical machine translation. We propose to use the RPROP algorithm for optimizing a maximum expected BLEU objective and experimentally compare it to several other updating schemes. It proves to be more efficient and effective than the previously proposed growth transformation technique and also yields better results than stochastic gradient descent and AdaGrad. We also report strong empirical results on two large scale tasks, namely BOLT Chinese!English and WMT German!English, where our final systems outperform results reported by Setiawan and Zhou (2013) and on matrix.statmt.org. On the WMT task, discriminative training is performed on the full training data of 4M sentence pairs, which is unsurpassed in the literature.

German Compounds and Statistical Machine Translation. Can they get along

This paper reports different experiments created to study the impact of using linguistics to preprocess German compounds prior to translation in Statistical Machine Translation (SMT). Compounds are a known challenge both in Machine Translation (MT) and Translation in general as well as in other Natural Language Processing (NLP) applications. In the case of SMT, German compounds are split into their constituents to decrease the number of unknown words and improve the results of evaluation measures like the Bleu score. To assess to which extent it is necessary to deal with German compounds as a part of preprocessing in SMT systems, we have tested different compound splitters and strategies, such as adding lists of compounds and their translations to the training set. This paper summarizes the results of our experiments and attempts to yield better translations of German nominal compounds into Spanish and shows how our approach improves by up to 1.4 Bleu points with respect to the baseline.

Simple and Effective Approach for Consistent Training of Hierarchical Phrase-based Translation Models

In this paper, we present a simple approach for consistent training of hierarchical phrase-based translation models. In order to consistently train a translation model, we perform hierarchical phrasebased decoding on training data to find derivations between the source and target sentences. This is done by synchronous parsing the given sentence pairs. After extracting k-best derivations, we reestimate the translation model probabilities based on collected rule counts. We show the effectiveness of our procedure on the IWSLT German!English and English!French translation tasks. Our results show improvements of up to 1.6 points BLEU.

Generative Training and Smoothing of Hierarchical Phrase-Based Translation Models

Hierarchical phrase-based translation is a common machine translation approach for translating between languages with significantly different word order. The focus of the first part of this thesis is set on smoothing and training of the translation models used in hierarchical translation. Additionally, we present an improved implementation of the search algorithm and show that our implementation is competitive compared to other state-of-the-art hierarchical phrase-based translation engines. Within the second part of this work, we apply hierarchical phrase-based translation in the context of spoken language translation. In the state-of-the-art hierarchical translation model extraction process, translation rules and their corresponding translation probabilities are obtained from word-aligned training data by applying simple heuristics. A common issue is that even if a large set of training data is provided, the resulting translation model may suffer from data sparseness. Smoothing is an approach to remedy this problem and is well-known from other natural language processing tasks (e.g. language modeling). The goal of smoothing applied in the scope of machine translation is to model rarely seen translation rules better. In this thesis, we investigate and compare different smoothing techniques for hierarchical phrase-based translation. Furthermore, the extraction and translation processes are two separated steps. Therefore, the extraction does not take into account whether the obtained translation rules are actually needed in the translation process. To learn whether a translation rule is relevant for the translation process, we pursue the approach of force-decoding the training data. Given a sentence pair of the training data, the translation of the source sentence is constrained to produce the corresponding target sentence. The applied translation rules are then determined and the corresponding translation probabilities re-estimated. In order to be able to translate a large set of training data, an efficient and fast framework is needed. In this work, we introduce such a framework for re-estimating hierarchical translation models. This approach enables us to obtain smaller translation models while simultaneously improving the translation quality. We further compare our proposed scheme with another state-of-the-art translation model training approach, namely discriminative training, on a large-scale Chinese→English translation task. Spoken language translation is the task of translating automatically transcribed speech. Since most automatic speech recognition systems provide transcriptions without punctuation marks and case information, this information has to be re-introduced before the actual translation takes place. In this work, we show that performing punctuation prediction and re-casing by applying a machine translation system helps to improve the translation quality. In particular, we propose to apply hierarchical translation rather than phrase-based translation for this task. Finally, experiments were conducted on a large-scale English→French spoken language translation task. All methods described in this thesis have been made freely available to the research community as they were integrated into the open-source translation toolkit Jane.