Marian H. Nodine

InfoSleuth: agent-based semantic integration of information in open and dynamic environments

The goal of the InfoSleuth project at MCC is to exploit and synthesize new technologies into a unified system that retrieves and processes information in an ever-changing network of information sources. InfoSleuth has its roots in the Carnot project at MCC, which specialized in integrating heterogeneous information bases. However, recent emerging technologies such as internetworking and the World Wide Web have significantly expanded the types, availability, and volume of data available to an information management system. Furthermore, in these new environments, there is no formal control over the registration of new information sources, and applications tend to be developed without complete knowledge of the resources that will be available when they are run. Federated database projects such as Carnot that do static data integration do not scale up and do not cope well with this ever-changing environment. On the other hand, recent Web technologies, based on keyword search engines, are scalable but, unlike federated databases, are incapable of accessing information based on concepts. In this experience paper, we describe the architecture, design, and implementation of a working version of InfoSleuth. We show how InfoSleuth integrates new technological developments such as agent technology, domain ontologies, brokerage, and internet computing, in support of mediated interoperation of data and services in a dynamic and open environment. We demonstrate the use of information brokering and domain ontologies as key elements for scalability.

ACTIVE INFORMATION GATHERING IN INFOSLEUTHTM

InfoSleuth is an agent-based system that can be configured to perform many different information management activities in a distributed environment. InfoSleuthTM agents provide a number of complex query services that require resolving ontology-based queries over dynamically changing, distributed, heterogeneous resources. These include distributed query processing, location-independent single-resource updates, event and information monitoring, statistical or inferential data analysis, and trend discovery in complex event streams. It has been used in numerous applications, including the Environmental Data Exchange Network and the Competitive Intelligence System.

Ensuring relaxed atomicity for flexible transactions in multidatabase systems

Global transaction management requires cooperation from local sites to ensure the consistent and reliable execution of global transactions in a distributed database system. In a heterogeneous distributed database (or multidatabase) environment, various local sites make conflicting assertions of autonomy over the execution of global transactions. A flexible transaction model for the specification of global transactions makes it possible to deal robustly with these conflicting requirements. This paper presents an approach that preserves the semi-atomicity (a weaker form of atomicity) of flexible transactions, allowing local sites to autonomously maintain serializability and recoverability. We offer a fundamental characterization of the flexible transaction model and precisely define the semi-atomicity. We investigate the commit dependencies among the subtransactions of a flexible transaction. These dependencies are used to control the commitment order of the subtransactions. We next identify those restrictions that must be placed upon a flexible transaction to ensure the maintenance of its semi-atomicity. As atomicity is a restrictive criterion, semi-atomicity enhances the class of executable global transactions.

Agent-based semantic interoperability in infosleuth

The InfoSleuth T M Project i at MCC has developed a distributed agent architecture that addresses the need for semantic interoperability among information sources and analytical tools within diverse application domains [4, 13]. InfoSleuth is being used as a significant component of the Environmental Data Exchange Network (EDEN) 2. The current EDEN pilot demonstration enables integrated access via web browser to environmental information resources provided by offices of these agencies located in several states. At the application level, InfoSleuth provides for semantic interchange among users by allowing an application developer to express the concepts and relationships of the application domain in high-level terms that are then translated into the low-level types of database schemas or semantic analyses of text and image resources. At the system level, InfoSleuth employs accepted standards where possible, to simplify data interchange and communication among processes. To apply InfoSleuth in a specific application domain, it is necessary to identify the key elements of the business environment of the application, and create or discover an appropriate ontology for the domain, as well as identify the kinds of data that will be appropriate to the application. For the

Global scheduling for flexible transactions in heterogeneous distributed database systems

A heterogeneous distributed database environment integrates a set of autonomous database systems to provide global database functions. A flexible transaction approach has been proposed for the heterogeneous distributed database environments. In such an environment, flexible transactions can increase the failure resilience of global transactions by allowing alternate (but in some sense equivalent) executions to be attempted when a local database system fails or some subtransactions of the global transaction abort. We study the impact of compensation, retry, and switching to alternative executions on global concurrency control for the execution of flexible transactions. We propose a new concurrency control criterion for the execution of flexible and local transactions, termed F-serializability, in the error-prone heterogeneous distributed database environments. We then present a scheduling protocol that ensures F-serializability on global schedules. We also demonstrate that this scheduler avoids unnecessary aborts and compensation.

Cooperative transaction hierarchies: transaction support for design applications

Traditional atomic and nested transactions are not always well-suited to cooperative applications, such as design applications. Cooperative applications place requirements on the database that may conflict with the serializability requirement. They require transactions to be long, possibly nested, and able to interact with each other in a structured way. We define a transaction framework, called acooperative transaction hierarchy, that allows us to relax the requirement for atomic, serializable transactions to better support cooperative applications. In cooperative transaction hierarchies, we allow the correctness specification for groups of designers to be tailored to the needs of the application. We usepatterns andconflicts to specify the constraints imposed on a groups history for it to be correct. We also provide some primitives to smooth the operation of the members. We characterize deadlocks in a cooperative transaction hierarchy, and provide mechanisms for deadlock detection and resolution. We examine issues associated with failure and recovery.

Constructing Robust Conversation Policies in Dynamic Agent Communities

Conversation policies codify allowable exchanges of speech acts among agents as they execute specific types of tasks. Both the set of agents in a community, and the nature of those agents may change over time; however, these agents must conform to a common set of conversation policies that are robust to change and failure. We describe aspects of the implementation of conversation policies in InfoSleuth, including the integral use of finite-state automata for defining those policies. We identify features of those automata and the underlying performatives that are necessary for their robust and correct execution in an operational community. We describe the construction of new conversation policies from simpler underlying components using two mechanisms, extension and concatenation. In this way, we can ensure that the specification of these new policies is easily sharable, and that certain shared characteristics of multiple conversation policies are enforced consistently.

The InfoSleuth Project

The InfoSleuth Project at MCC [7, 9, 8, 1] is developing and deploying technologies for finding information in corp~ rate networks and in external networka, such as networks baaed on the emerging National Information Infrastructure. InfoSleuth is baaed on MCC’S previously developed Carnot technology [2, 6, 10], which was successfully used to integrate heterogeneous information resources. The Carnot project developed semantic modeling techniques that enable description of the information resources and pioneered the use of agents to provide interoperation among autonomous systems. The InfoSleuth Project investigates the use of Carnot technologies in a dynamically changing environment, such as the Internet, where there is no formal control of the registration of new information sources and the identities of the resources to be used may be unknown at the time the application is developed. InfoSleuth deploys semantic agents [9, 5, 3] that carry out coordinated searches and cooperate with each other to merge the retrieved data into understandable information. The project is developing technologies to support mediated interoperation of data and services over information networks in a dynamically changing environment, including:

Supporting long-running tasks on an evolving multidatabase using interactions and events

An open nested multi-database transaction model called Interactions is proposed. Interactions sit above a layer that provide atomic, serializable global multidatabase transactions. It provides additional support for planning applications, which are long-running applications that may need to respond to changes in the underlying multidatabase. Interactions and the multidatabase architecture that they run on are described, and synchronization of Interactions is discussed. The nature of events, which the applications can use to monitor changes in the information in the multidatabase is described, and how they are detected and how to respond to them are defined. This response usually involves semantically undoing part of at least one Interaction, so a brief description of how recovery works in that situation is given.<<ETX>>

Transaction Optimization Techniques

Replication introduces a tension between query optimization and remote access control in a distributed database system. If we view a transaction as a partially-ordered set of queries and updates, then factors that affect quorum selection for the fragments accessed by a transaction as a whole are currently orthogonal to factors that affect the replica selection during the planning of individual queries. Therefore, the two processes may act at cross-purposes to one another. Query optimization considers an individual query and selects a set of fragments that minimizes the computation and communication cost and allows computation to be pushed into the local site. Transaction management, on the other hand, selects quorums (sets of replicas to retrieve) based on replica availability and on mutual consistency constraints such as quorum intersection among write operations or between read and write operations. Thus, transaction optimization narrows the “optimal” solution space for the queries it contains Hence, transaction management should cooperate with query optimization to optimize transaction processing.