Sean Boisen

BBN: description of the PLUM system as used for MUC-4

Traditional approaches to the problem of extracting data from texts have emphasized hand-crafted linguistic knowledge. In contrast, BBNs PLUM system (Probabilistic Language Understanding Model) was developed as part of a DARPA-funded research effort on integrating probabilistic language models with more traditional linguistic techniques. Our research and development goals are• more rapid development of new applications,• the ability to train (and re-train) systems based on user markings of correct and incorrect output,• more accurate selection among interpretations when more than one is found, and• more robust partial interpretation when no complete interpretation can be found.

Named Entity Extraction from Noisy Input: Speech and OCR

In this paper, we analyze the performance of name finding in the context of a variety of automatic speech recognition (ASR) systems and in the context of one optical character recognition (OCR) system. We explore the effects of word error rate from ASR and OCR, performance as a function of the amount of training data, and for speech, the effect of out-of-vocabulary errors and the loss of punctuation and mixed case

Developing an evaluation methodology for spoken language systems

There has been a long-standing methodology for evaluating work in speech recognition (SR), but until recently no community-wide methodology existed for either natural language (NL) researchers or speech understanding (SU) researchers for evaluating the systems they developed.Recently considerable progress has been made by a number of groups involved in the DARPA Spoken Language Systems (SLS) program to agree on a methodology for comparative evaluation of SLS systems, and that methodology is being used in practice for the first time.This paper gives an overview of the process that was followed in creating a meaningful evaluation mechanism, describes the current mechanism, and presents some directions for future development.

Partial parsing: a report on work in progress

This paper reports a handful of experiments designed to test the feasibility of applying well-known partial parsing techniques to the problem of automatic data base update from an open-ended source of messages. and the feasiblity of automatically learning semantic knowledge from annotated examples. The challenges arise from the incompleteness of any lexicon, sentences that average over 20 words in length, and the lack of a complete semantics.

BBN ATIS system progress report—June 1990

This paper reports recent progress on the development of the Delphi natural language component of the BBN spoken language system for the ATIS domain, focussing on the comparative evaluation performed by NIST in June, 1990.

A new approach to text understanding

This paper first briefly describes the architecture of PLUM, BBNs text Processing system, and then reports on some experiments evaluating the effectiveness of the design at the component level. Three features are unusual in PLUMs architecture: a domain-independent deterministic parser, processing of (the resulting) fragments at the semantic and discourse level, and probabilistic models.

CHINESE INFORMATION EXTRACTION AND RETRIEVAL

This paper provides a summary of the following topics:1. what was learned from porting the INQUERY information retrieval engine and the INFINDER term finder to Chinese2. experiments at the University of Massachusetts evaluating INQUERY performance on Chinese newswire (Xinhua),3. what was learned from porting selected components of PLUM to Chinese4. experiments evaluating the POST part of speech tagger and named entity recognition on Chinese.5. program issues in technology development.

THE HOOKAH INFORMATION EXTRACTION SYSTEM

This paper describes Project HOOKAH, a TIPSTER Implementation Project with the Drug Enforcement Administration to extract information from the DEA-6 field report. The paper overviews Project HOOKAH, describes the system architecture and modules, and discusses several lessons that have been learned from this application of TIPSTER technology.

BBN's PLUM Probabilistic Language Understanding system

Traditional approaches to the problem of extracting data from texts have emphasized hand-crafted linguistic knowledge. In contrast, BBNs PLUM system (Probabilistic Language Understanding Model) was developed as part of an ARPA-funded research effort on integrating probabilistic language models with more traditional linguistic techniques. Our research and development goals are:• Achieving high performance in objective evaluations, such as the Tipster evaluations.• Reducing human effort in porting the natural language algorithms to new domains and to new languages.• Providing technology that is scalable to realistic applications.

PROGRESS IN INFORMATION EXTRACTION

This paper provides a quick summary of the following topics: enhancements to the PLUM information extraction engine, what we learned from MUC-6 (the Sixth Message Understanding Conference), the results of an experiment on merging templates from two different information extraction engines, a learning technique for named entity recognition, and towards information extraction from speech.