Doug Arnold

TSNLP: Test Suites for Natural Language Processing

The growing language technology industry needs measurement tools to allow researchers, engineers, managers, and customers to track development, evaluate and assure quality, and assess suitability for a variety of applications.The TSNLP (Test Suites for Natural Language Processing) project has investigated various aspects of the construction, maintenance and application of systematic test suites as diagnostic and evaluation tools for NLP applications. The paper summarizes the motivation and main results of TSNLP: besides the solid methodological foundation of the project, TSNLP has produced substantial (i.e. larger than any existing general test suites) multi-purpose and multi-user test suites for three European languages together with a set of specialized tools that facilitate the construction, extension, maintenance, retrieval, and customization of the test data.The publicly available results of TSNLP represent a valuable linguistic resource that has the potential of providing a wide-spread pre-standard diagnostic and evaluation tool for both developers and users of NLP applications.

Non-restrictive relatives are not orphans

According to a ‘radical orphanage’ approach, non-restrictive relative clauses are not part of the syntactic representation of the sentence that contains them. It is an appealing view, and seems to capture some important properties of non-restrictive relative clauses. Focusing mainly on empirical shortcomings, this paper aims to show that the appeal of such approaches is illusory. It also outlines an empirically superior ‘syntactically integrated’ account.

Evaluation: An assessment

The primary aim of this contribution is to provide an editorial introduction to this Special Issue ofMachine Translation dedicated to Evaluation. The intention is to describe the rationale for the Issue, outline the various contributions of the papers in this issue, and try to situate them in a wider context. As part of providing this wider context, we give an overview and assessment of the main current approaches to Evaluation of Natural Language Processing, and especially Machine Translation systems.

The ,T framework in Eurotra: a theoretically committed notation for MT

This paper describes a model for MT, developed within the Eurotra MT project, based on the idea of compositional translation , by describing a basic, experimental notation which embodies the idea. The introduction provides background, section 1 introduces the basic ideas and the notation, and section 2 discusses some of the theoretical and practical implications of the model, including some concrete extensions, and some more speculative discussion.

Eurotra: A European perspective on MT

This paper describes the perspective on Machine Translation taken in the Eurotra project: an R & D project of the European Communities at a relatively early stage. It describes: i) the projects rather challenging aims; ii) the framework of design principles and linguistic and software methodology which are of particular importance given its size, scope, and organization; iii) the proposed representational theories and expressive (programming) languages with which actual instances of MT systems can be constructed. The final section sketches some recent proposals and directions for future research.

The theoretical basis of MiMo

The purpose of this paper is to describe and motivate the leading ideas behind the MiMo notation, a formalism for transfer-based machine translation which is especially appropriate for intermediate representations based on Dependency Grammar. The most important of these ideas concern properties of the translation relation such as compositionality, reversibility, modularity, and autonomy of levels of description. The final section considers some issues relating to rule interaction.

A model for preference

In this paper we address the problem of choosing the best solution(s) from a set of interpretations of the same object (in our case a segment of text). A notion of preference is stated, based on pairwise comparisons of complete interpretations in order to obtain a partial order among the competing interpretations. An experimental implementation is described, which uses Prolog-like preference statements.

Automatic Test Suite generation

A Test Suite (TS) is typically a collection of Natural Language sentences against which the coverage of a Natural Language Processing system can be evaluated. We describe a method by which such suites can be produced automatically, involving a modification and extension of the Definite Clause Grammar formalism, and describe some of the advantages of the method over the traditional method of manual construction.

A constraint-based approach to translating anaphoric dependencies

The normal method for representing anaphoric dependencies in Unification Based grammar formalisms is that of re-entrance. In this paper, we address the problems that this representational device poses when such formalisms are used for translation. We demonstrate the inadequacies of existing proposals, and describe an approach which exploits the expressive possibilities of the equational constraint language in LFG and involves an inferential procedure combining underspecification in the statement of bilingual correspondences with the use of target language knowledge.

Web access to corpora: the W3Corpora project

In this day an age, some corpus linguistics should be part of every course to do with language. But learning about corpus linguistics -- its possibilities and limitations --- is not just a matter of acquiring information. The best way to learn about corpus linguistics is to do it, and the best way to teach corpus linguistics is to put students into a position where they can do it ((Leech, 1997), (Fligelstone, 1993)). This requires corpora, and tools, in addition to teaching materials.