James R. Driscoll

Method and system for searching for relevant documents from a text database collection, using statistical ranking, relevancy feedback and small pieces of text

Search system and method for retrieving relevant documents from a text data base collection comprised of patents, medical and legal documents, journals, news stories and the like. Each small piece of text within the documents such as a sentence, phrase and semantic unit in the data base is treated as a document. Natural language queries are used to search for relevant documents from the data base. A first search query creates a selected group of documents. Each word in both the search query and in the documents are given weighted values. Combining the weighted values creates similarity values for each document which are then ranked according to their relevant importance to the search query. A user reading and passing through this ranked list checks off which documents are relevant or not. Then the system automatically causes the original search query to be updated into a second search query which can include the same words, less words or different words than the first search query. Words in the second search query can have the same or different weights compared to the first search query. The system automatically searches the text data base and creates a second group of documents, which as a minimum does not include at least one of the documents found in the first group. The second group can also be comprised of additional documents not found in the first group. The ranking of documents in the second group is different than the first ranking such that the more relevant documents are found closer to the top of the list.

Incorporating a semantic analysis into a document retrieval strategy

Current information retrieval systems focus on the use of keywords to respond to user queries. We propose the additional use of surface level knowledge in order to improve the accuracy of information retrieval. Our approach is based on the database concept of semantic modeling (particularly entities and relationships among entities). We extend the concept of query-document similarity by recognizing basic entity properties (attributes) which appear in text. We also extend query-document similarity using the linguistic concept of thematic roles. Thematic roles allow us to recognize relationship properties which appear in text. We include several examples to illustrate our approach. Test results which support our approach are reported. The test results concern searching documents and using their contents to perform the intelligent task of answering a question.

Batch insertion for tree structured file organizations—improving differential database representation

Abstract This paper presents an algorithm to perform batch insertions into tree structured files. The efficiency of the algorithm is formally analyzed. To demonstrate the usefulness of the algorithm, optimum batch insertion points are determined in one particular example. The example demonstrates that batch operations for tree structured files may be used to achieve the advantages of a differential database representation and, at the same time, avoid the drawbacks previously attributed to the use of differential files.

Improving the differential file technique via batch operations for tree structured file organizations

This paper presents a combined algorithm to perform batch insertion, deletion, and update for tree structured files. The efficiency of the algorithm is analyzed for performing updates only and insertions only. A cost analysis example is reviewed to demonstrate that batch operations for tree structured files achieve the advantages of a differential database representation and, at the same time, avoid the drawbacks previously attributed to the use of differential files.

A unified analysis of batched searching of sequential and tree-structured files

A direct and unified approach is used to analyze the efficiency of batched searching of sequential and tree-structured files. The analysis is applicable to arbitrary search distributions, and closed-form expressions are obtained for the expected batched searching cost and savings. In particular, we consider a search distribution satisfying Zipfs law for sequential files and four types of uniform (random) search distribution for sequential and tree-structured files. These results unify and extend earlier research on batched searching and estimating block accesses for database systems.

The operation and performance of an artificially intelligent keywording system

Abstract A new approach to text analysis for automating the key phrase indexing process, using artificial intelligence techniques, is presented. This new approach mimics the behavior of human experts by using a rule base consisting of “insertion” and “deletion” rules generated by subject-matter experts. The insertion rules are based on the idea that some phrases found in a text imply or trigger other phrases. The deletion rules apply to semantically ambiguous phrases where text presence alone does not determine appropriateness as a key phrase. The insertion and deletion rules are used to transform a list of found phrases to a list of key phrases for indexing a document. Statistical data are provided to demonstrate the performance of this expert rule based system.

Modeling B-tree insertion activity

Abstract In this paper we model the root node insertion activity of block split tree structured files such as the B-tree and its variants. Specially, we derive formulas for the events of inserting each entry into a B+ tree root node. The formulas also apply to B-trees and may be modified for other block split tree structured files. Our model of the root node insertion activity enables us to better characterize three aspects of efficient index management—storage space, search time, and reorganization.

Structuring Text within a Relational System

We introduce a preprocessor that uses a relational system and semantic modeling to impose structure on text. Our intent is to show that document retrieval applications can be easily developed within the relational model. We illustrate several operations that are typically found in information retrieval systems, and show how each can be performed in the relational model. These include keywording, proximity searches, and relevance ranking. We also include a discussion of an extension to relevance based on semantic modeling.

A generalized method for maintaining views

A generalized method for maintaining views, which takes into account each views pattern of usage, is described. This method involves storing views in both actual and potential forms. When views exist in actual form (concrete views), updates are deferred until the view is queried. Differential files are formed from tuples inserted and deleted from the defining relations. These differential files are then used in conjunction with specified update schemes to update the view. Views alternate between actual and potential form based on their usage and the storage replacement algorithm.

Dynamic derived relations within the RAQUEL II DBMS

An implementation technique for supporting dynamic derived relations within a particular relational DBMS (the RAQUEL II DBMS) is presented. Included is a discussion of the RAQUEL II concepts of permanent and temporary relations, base and derived relations, and explicit and implicit updates. The methods presented here for support of updates in regard to dynamic derived relations take the form of immediate updates. Proofs are presented to show the validity of the update methods.