Menno van Zaanen

The tenjinno machine translation competition

This paper describes the Tenjinno Machine Translation Competition held as part of the International Colloquium on Grammatical Inference 2006. The competition aimed to promote the development of new and better practical grammatical inference algorithms used in machine translation. Tenjinno focuses on formal models used in machine translation. We discuss design issues and decisions made when creating the competition. For the purpose of setting the competition tasks, a measure of the complexity of learning a transducer was developed. This measure has enabled us to compare the competition tasks to other published results, and it can be seen that the problems solved in the competition were of a greater complexity and were solved with lower word error rates than other published results. In addition the complexity measures and benchmark problems can be used to track the progress of the state-of-the-art into the future.

INTRODUCTION: SPECIAL ISSUE ON APPLICATIONS OF GRAMMATICAL INFERENCE

This special issue of Applied Artificial Intelligence contains articles on applications of grammar induction (GI)—a research area concerned, not surprisingly, with learning grammars from examples. A grammar is a rule-based, generative model of the elements in a possibly infinite set, where these elements are typically complex, structured objects like strings, trees, and graphs. The GI problem is to identify a grammar given some of the elements in the set it generates (and possibly some elements that are not in that set). In the context of GI, the most familiar grammars are those for formal languages that generate sets of strings. In the early days of the GI field, researchers focused on inferring regular grammars, those at the lowest level of the Chomsky hierarchy. Negative learnability results showed that even this problem is computationally hard (Gold, 1967; Pitt and Warmuth, 1989). However, since those early days, research in GI has produced deep theoretical insights into the learnability of grammars at all levels of the Chomsky hierarchy, resulting in powerful, efficient algorithms for inferring a wide variety of grammars, including many that are not represented in the hierarchy at all. Grammatical representations of sets of structured objects have a number of advantages, perhaps foremost among them being explicit representation

Classifying sentences using induced structure

In this article we will introduce a new approach (and several implementations) to the task of sentence classification, where pre-defined classes are assigned to sentences. This approach concentrates on structural information that is present in the sentences. This information is extracted using machine learning techniques and the patterns found are used to classify the sentences. The approach fits in between the existing machine learning and hand-crafting of regular expressions approaches, and it combines the best of both. The sequential information present in the sentences is used directly, classifiers can be generated automatically and the output and intermediate representations can be investigated and manually optimised if needed.

Unsupervised measurement of translation quality using multi-engine, bi-directional translation

Lay people discussing machine translation systems often perform a round trip translation, that is translating a text into a foreign language and back, to measure the quality of the system. The idea behind this is that a good system will produce a round trip translation that is exactly (or perhaps very close to) the original text. However, people working with machine translation systems intuitively know that round trip translation is not a good evaluation method. In this article we will show empirically that round trip translation cannot be used as a measure of the quality of a machine translation system. Even when using translations of multiple machine translation systems into account, to reduce the impact of errors of a single system, round trip translation cannot be used to measure machine translation quality.

Grammatical inference for syntax-based statistical machine translation

In this article we present a syntax-based translation system, called TABL (Translation using Alignment-Based Learning). It translates natural language sentences by mapping grammar rules (which are induced by the Alignment-Based Learning grammatical inference framework) of the source language to those of the target language. By parsing a sentence in the source language, the grammar rules in the derivation are translated using the mapping and subsequently, a derivation in the target language is generated. The initial results are encouraging, illustrating that this is a valid machine translation approach.