Lorna Balkan

TSNLP: Test Suites for Natural Language Processing

The growing language technology industry needs measurement tools to allow researchers, engineers, managers, and customers to track development, evaluate and assure quality, and assess suitability for a variety of applications.The TSNLP (Test Suites for Natural Language Processing) project has investigated various aspects of the construction, maintenance and application of systematic test suites as diagnostic and evaluation tools for NLP applications. The paper summarizes the motivation and main results of TSNLP: besides the solid methodological foundation of the project, TSNLP has produced substantial (i.e. larger than any existing general test suites) multi-purpose and multi-user test suites for three European languages together with a set of specialized tools that facilitate the construction, extension, maintenance, retrieval, and customization of the test data.The publicly available results of TSNLP represent a valuable linguistic resource that has the potential of providing a wide-spread pre-standard diagnostic and evaluation tool for both developers and users of NLP applications.

An experiment in automatic indexing using the HASSET thesaurus

In this paper we present the tools, techniques and evaluation results of an automatic indexing experiment we conducted on the UK Data Archive/UK Data Service data-related document collection, as part of the Jisc-funded SKOS-HASSET project. We examined the quality of an automatic indexer based on a controlled vocabulary called the Humanities and Social Science Electronic Thesaurus (HASSET). We used the Keyphrase Extraction Algorithm (KEA), a text mining and a machine learning tool. KEA builds a classifier model using training documents with known keywords which is then applied to help assign keywords to new documents. We performed extensive manual and automatic evaluation on the results using recall, precision and F1 scores. The quality of the KEA indexing was measured a) automatically by the degree of overlap between the automated indexing decisions and those originally made by the human indexer and b) manually by comparing KEAs output with the source text. This paper explains how and why we applied the chosen technical solutions, and how we intend to take forward any lessons learned from this work in the future.

Book Review: Lynne Bowker, Computer Aided Translation Technology: A Practical Introduction, University of Ottawa Press, Ottawa, Canada, 2002. xx + 185 pp.

A gas sensor is disclosed which comprises a sintered metal oxide the resistance value of which changes with the ambient temperature, a gas sensor element having at least one electrode connected to the sintered metal oxide, a ceramic insulator for insulatively holding the gas sensor, a lead connection and a lead wire connected between the electrodes. The diameter of the ceramic insulator for passing the lead wire is reduced and the length of the electrode is shortened, thus realizing a compact gas sensor.