Igor S. Nekrestyanov

Building Topic-Specific Collections with Intelligent Agents

As part of the OASIS project, an autonomous intelligent agent, called an OASIS Crawler, was designed to address the problem of automatically constructing topic-specific Internet indices. Each OASIS Crawler independently constructs an index of Internet documents relevant to the topic of its associated OASIS Collection. OASIS Crawlers interact asynchronously through the propagation of discovered URLs to improve system wide index quality. The model of interaction is one of self-interested cooperation between independent agents.

Open Architecture for Distributed Search Systems

This paper describes an open architecture for distributed Internet search engines and the experience derived from implementation of a conforming prototype. The architecture enables competing collaboration of independent information retrieval service providers. It allows integration of multiple cheap private servers into a powerful distributed system, which still guards independence and commercial interests of every player. Special emphasis was made on demonstrating the ability of the architecture to make effective use of latest advances in information retrieval technology. Prototype implementation has proved the feasibility of the approach. It has also exposed a wide area of optimisations desirable at the component level. The source code of the prototype is publicly available.

Text Retrieval Systems for the Web

The Web information retrieval is a highly popular, but also a challenging research area, which considerably differs from the classical information retrieval theory. The focus of this paper is to survey the modern approaches to the accomplishment of different Web search engine tasks. The main attention is paid to specific issues of Web information retrieval rather than to a detailed description of Web search engine implementation. Approaches to solving crawling and ranking tasks, as well as modifications of the retrieval problem associated with heterogeneity of both users needs and architectures of the search systems used, are discussed. Special consideration is given to the problem of independent evaluation of Web search engine effectiveness.

Evaluation of Text Retrieval Systems

Evaluation is one of the main driving forces in studies and developments related to text retrieval. It is a basic tool for the comparison of efficiencies of alternative approaches. In this paper, the state of the art in the field of evaluation of text retrieval systems is surveyed. Two basic—system-oriented and user-oriented— paradigms, which are commonly accepted in this field, are often considered as incompatible. In this survey, both paradigms are considered in the context of a unique framework based on attributes affecting the innovation distribution and adaptation. A detailed discussion of the evaluation of text retrieval systems is based on the consideration of required components of the evaluation process for an arbitrary system. Methodological problems related to the verification of the results obtained are also discussed.

An Analysis of Alternative Methods for Storing Semistructured Data in Relations

Although the major source of semistructured data is WWW and therefore the representation of data cannot be controlled by any single site, in many cases the data are replicated and stored locally in a database to support sophisticated query processing. To date a number of approaches to store data in relations were proposed. In this paper we present an analysis of alternative mapping schemes.

Designing Persistence for Real-Time Distributed Object Systems

An implementation of persistent object store for real-time systems with strict processing time constraints is a challenging task, because many traditional database techniques, e.g. transaction management schemes, are not applicable for such systems.

Using intelligent internet robots for building topic-specific collections

The problem of developing an intelligent Internet search engine for creating topic-specific collections is considered. To improve the efficiency of discovering topic-related resources, a specialized network crawling algorithm is developed that uses information on the relevance of the Web pages already visited to evaluate the usefulness of new pages.

Consistency control for semistructured data

Semistructured data models provide an important tool for studying methods for representing, searching, and processing heterogeneous data in distributed systems with autonomous components (especially, on the WWW). However, the problem of data consistency remains little investigated. In this paper, various approaches to this problem are discussed, and the usability of various mechanisms suggested in the context of databases, including real-time databases, is analyzed. An approach based on the concept of multilevel transactions is presented in detail. In the framework of this approach, commutativity predicates for basic operations on semistructured data are constructed.