Kurt A. Shoens

Incremental updates of inverted lists for text document retrieval

With the proliferation of the worlds “information highways” a renewed interest in efficient document indexing techniques has come about. In this paper, the problem of incremental updates of inverted lists is addressed using a new dual-structure index. The index dynamically separates long and short inverted lists and optimizes retrieval, update, and storage of each type of list. To study the behavior of the index, a space of engineering trade-offs which range from optimizing update time to optimizing query performance is described. We quantitatively explore this space by using actual data and hardware in combination with a simulation of an information retrieval system. We then describe the best algorithm for a variety of criteria.

Synthetic workload performance analysis of incremental updates

Declining disk and CPU costs have kindled a renewed interest in efficient document indexing techniques. In this paper, the problem of incremental updates of inverted lists is addressed using a dual-structure index data structure that dynamically separates long and short inverted lists and optimizes the retrieval, update, and storage of each type of list. The behavior of this index is studied with the use of a synthetically-generated document collection and a simulation model of the algorithm. The index structure is shown to support rapid insertion of documents, fast queries, and to scale well to large document collections and many disks.

Programming Language Constructs for Screen Definition

This paper describes a screen-oriented I/O facility, designed as a part of Rigel, a high-level database programming language. The novel features of the I/O facility, called Screen Rigel, include the use of formatting techniques to lay out text and data on the screen. Applications written in Screen Rigel can directly access the database using the database constructs already in Rigel. In addition, Screen Rigel has access to the data dictionary so the programmer can use the data descriptions in the database instead of respecifying them. These features make Screen Rigel programs simple to write and highly data and terminal independent.

Information organization using Rufus

Computer system users today are inundated with a flood of semi-structured information, such as documents, electronic mail, programs, and images. Today, this information is typically stored in filesystems that provide limited support for organizing, searching, and operating upon this data, all operations that are vital to the ability of users to effectively use this data. Database systems provide good function for organizing, searching, managing and writing applications on structured data. Current database systems are inappropriate for semi-structured information because moving the data into the database breaks all existing applications that use the data. The Rufus system attacks the problems of semi-structured information by using database function to help users manage semi-structured information without requiring that the users information reside in the database.

Managing change in the Rufus system

Rufus is an information system that models user data with objects taken from a class system. Due to the importance of coping with changes to the schema, Rufus has adopted the conformity-based model of Melampus. This model enables Rufus to cope with schema changes more easily than traditional class- and inheritance-based data models. The paper reviews the Melampus data model and describes how it was implemented in the Rufus system. The authors show how changes to the schema can be accommodated with minimum disruption. They also review design decisions that contributed to streamlined schema evolution and compare this approach with those proposed in the literature.<<ETX>>