David C. Blair

An evaluation of retrieval effectiveness for a full-text document-retrieval system

An evaluation of a large, operational full-text document-retrieval system (containing roughly 350,000 pages of text) shows the system to be retrieving less than 20 percent of the documents relevant to a particular search. The findings are discussed in terms of the theory and practice of full-text document retrieval.

Full-text information retrieval: further analysis and clarification

Abstract In 1985, an article by Blair and Maron described a detailed evaluation of the effectiveness of an operational full text retrieval system used to support the defense of a large corporate lawsuit. The following year Salton published an article which called into question the conclusions of the 1985 study. The following article briefly reviews the initial study, replies to the objections raised by the second article, and clarifies several confusions and misunderstandings about the 1985 study.

STAIRS redux: thoughts on the STAIRS evaluation, ten years after

The test of retrieval effectiveness performed on IBMs STAIRS and reported in Communications of the ACM ten years ago, continues to be cited frequently in the information retrieval literature. The reasons for the studys continuing pertinence to todays research are discussed, and the political, legal, and commercial aspects of the study are presented. In addition, the method of calculating recall that was used in the STAIRS study is discussed in some detail, especially how it reduces the five major types of uncertainty in recall estimations. It is also suggested that this method of recall estimation may serve as the basis for recall estimations that might be truly comparable between systems.

Communicating MIS research: a citation study of journal influence

Due to difficulties with objectively evaluating the quality of MIS research, at- titudes about journals in which this research is published play an important role in deter- mining the allocation of research resources. To provide a more objective basis for these attitudes, we examine journal influence in communicating MIS research over a 9-year pe- riod using citation analyses, researcher perceptions, and publishing patterns of top MIS research universities. As a result of these analyses, we identify a cohesive stable group of highly influential journals which can reasonably be called an MIS core. An internal rank- ing of this core is then determined which is significantly different from prior rankings.

Wittgenstein, language and information: “back to the rough ground!”

First of all, why are the issues of language and meaning important to the study of information systems? Information systems are, of course, tools that are used to search for information of various kinds: data, text, images, etc. Information searches themselves inevitably require the searcher to ask for or describe the information he or she wants and to match those descriptions with the descriptions of the information that is available: in short, when we ask for or describe information we must mean something by these statements. This places the requests for information as properly within the study of language and meaning. Surely, requests for information, or descriptions of available information, can be clear or ambiguous, precise or imprecise, just as statements in natural language can. In short, understanding how requests for, and descriptions of, information work, and, more importantly, how they can go wrong, is an issue of language, meaning and understanding.

Searching biases in large interactive document retrieval systems

The way that individuals construct and modify search queries on a large interactive document retrieval system is subject to systematic biases similar to those that have been demonstrated in experiments on judgments under uncertainty. These biases are shared by both naive and sophisticated subjects and cause the inquirer searching for documents on a large interactive system to construct and modify queries inefficiently. A searching algorithm is suggested that helps the inquirer to avoid the effect of these biases.

Information retrieval and the philosophy of language

This discussion takes the position that information retrieval systems are fundamentally linguistic in nature - in essence, the languages of document representation and searching are dialects of natural language. Because of this, the discipline of the Philosophy of Language should have some bearing on the problems of document representation and search query formulation. The philosophies of Austin, Searle, Grice and Wittgenstein are briefly examined and their relevance to information retrieval theory is discussed

Indeterminacy in the subject access to documents

Abstract Subject access to documents is influenced by two kinds of indeterminacy: the indeterminacy of the indexers selection of indexing descriptors and the indeterminacy of the inquirers selection of search terms. The possibility of successful retrieval depends on how these two indeterminacies interact. Five types of interaction are discussed and a change in the traditional method of subject searching is suggested as a way of reducing the effect of one of these two indeterminacies and of avoiding those types of interactions where the retrieval of the desired document(s) is impossible.

Exemplary documents: a foundation for information retrieval design

Documents are generally represented for retrieval by either extracting index terms from them or by creating and selecting from an external set of candidate terms. There are many procedures for doing this, but while work continues along these dimensions, there have been relatively few attempts to change this basic process. Of particular importance is the creation of indexing schemes for retrieval systems in nonlibrary contexts. Here, the cost of developing an indexing scheme independent of the documents to be retrieved is often considered too high to implement. As a result, simple full-text retrieval or, to a lesser extent, automatic extractive or associative indexing methods are the predominant methods used in nonlibrary contexts. This paper suggests an alternative document representation method based on what we call exemplary documents. Exemplary documents are those documents that describe or exhibit the intellectual structure of a particular field of interest. In so doing, they provide both an indexing vocabulary for that area and, more importantly, a narrative context in which the indexing terms have a clearer meaning. Further, it is much easier to develop an indexing scheme by using exemplary documents than it is to do so from scratch.

The data-document distinction in information retrieval

The computerized retrieval of documents or texts from large databases is an area of increasing concern for those who design or use information management systems. The eXplosive growth of word processing and electronic mail systems is creating document databases of substantial size that will require sophisticated retrieval systems if users are to have satisfactory access. Unfortunately, document retrieval design has been the poor stepchild of the computer revolution. Largescale document retrieval system evaluations are few and often inconclusive, and commercial implementations of any but the most simple logical designs are almost unheard of. Popularized retrieval systems such as LEXIS, DIALOG, ORBIT, STAIRS, and SPIRES are based on logical retrieval designs that date back to the 1950s. The logical design and implementation of document retrieval systems have lagged behind other fields of information technology in large part because the fundamental issues or problems are often unclear. Commercially developed document retrieval systems have frequently treated document retrieval as merely a variant of data retrieval, assuming that advances in data retrieval technology will automatically translate into