Barbara A. Norgard

An association-based method for automatic indexing with a controlled vocabulary

In this article, we describe and test a two-stage algorithm based on a lexical collocation technique which maps from the lexical clues contained in a document representation into a controlled vocabulary list of subject headings. Using a collection of 4,626 INSPEC documents, we create a “dictionary” of associations between the lexical items contained in the titles, authors, and abstracts, and controlled vocabulary subject headings assigned to those records by human indexers using a likelihood ratio statistic as the measure of association. In the deployment stage, we use the dictionary to predict which of the controlled vocabulary subject headings best describe new documents when they are presented to the system. Our evaluation of this algorithm, in which we compare the automatically assigned subject headings to the subject headings assigned to the test documents by human catalogers, shows that we can obtain results comparable to, and consistent with, human cataloging. In effect, we have cast this as a classic partial match information retrieval problem. We consider the problem to be one of “retrieving” (or assigning) the most probably “relevant” (or correct) controlled vocabulary subject headings to a document based on the clues contained in that document.

The Online Catalog: From Technical Services to Access Service

A whole generation of library users has traveled the path to computer and library literacy through the use of the online catalog. Implementaiton of online library catalogs has given nex impetus to programs for user education, formal bibliographic instruction, and information desks, building, in the process, an infrastructure for electronic access to the resourcs of the library and the community. Nt the least of the changes within the profession is the gradual redifinition of the catalog. Present-day online catalogs are beginning to provide access to abstracting and indexing databases, full text, and pass through access to catalogs of material housed in physically diverse locations in different institutions. Online catalogs are providing printing, downloading, electronic mailing of results, requests for document delivery, and access to electronic full text.

Advanced search technologies for unfamiliar metadata (demonstration abstract)

Searching of databases, textual or numeric, is likely to be effective and efficient only if the user is familiar with the classification, categorizing, and indexing schemes (metadata vocabularies) being searched. Therefore, it is obviously beneficial to provide a bridge between the user’s ordinary language and the metadata vocabularies of the unfamiliar database in order to compensate for abbreviated, cryptic, or specialized terminologies. Advanced search technologies would utilize customized Entry Vocabulary Modules (EVM) which respond adaptively to the searcher’s ordinary language query with a ranked list of search terms in the target metadata vocabularies that may more accurately represent what is sought in the unfamiliar database [l]. Experienced searchers know that familiarity with the source being searched is critical for effective searching. Each source has its own special characteristics, and familiarity comes from frequent use. The rapid increase in network accessible repositories increases the number and proportion of information resources that are unfamiliar. Entry Vocabulary Modules axe designed to respond to a searcher’s query with a ranked list of terms from a system vocabulary to help the searcher to deal with unfamiliar metadata. In order to accomplish this goal, EVMs make use of association dictionaries that map ordinary language terms to metadata vocabularies based on a co-occurrence measure. The source of these ordinary language terms is the titles, abstracts, and sometimes full text from documents indexed with metadata terms. Associations are recorded between ordinary language and the domain-specific, and often technical, metadata vocabularies that are typically used to describe databases. The technique of mapping terms in the target metadata vocabulary to ordinary language terms is a twostage lexical collocation process. In the creation of an Entry Vocabulary Module a “dictionary” of associations between the lexical items found in the titles, authors, and abstracts and the metadata terms (e.g., classification numbers or thesaural terms) assigned using a likelihood ratio statistic as a measure of association [2, 31. The terms can be words or noun phrases extracted using natural language parsing software. The dictionary is used to predict which of the metadata terms best represent the topic being searched. We have developed several prototype EVMs to map from ordinary language to the United States Patent Classification and the International Patent Classification, ordinary language to BIOSIS (biological abstracts) concept codes, and ordinary language to INSPEC thesaurus terms’. Entry vocabulary functionality has very extensive implementation potential and can be usefully positioned in several ways: As an aid on the searcher’s desktop to provide assistance when accessing an unfamiliar remote repository or as an aid on a repository server to help remote searchers unfamiliar with the local metadata scheme of that repository. An Entry Vocabulary Module can also be used for computer-assisted categorization. Because it can provide a ranked list of probably relevant metadata terms for any fragment of text, it can also be used for automating the process of categorization. For example, an Entry Vocabulary Module for the Patent Classification might be a useful support for the assignment of classification numbers as well as for patent searching.