Steve Fisher

R-GMA: An Information Integration System for Grid Monitoring

Computational Grids are distributed systems that provide access to computational resources in a transparent fashion. Collecting and providing information about the status of the Grid itself is called Grid monitoring.

The Relational Grid Monitoring Architecture: Mediating Information about the Grid

We have developed and implemented the Relational Grid Monitoring Architecture (R-GMA) as part of the DataGrid project, to provide a flexible information and monitoring service for use by other middleware components and applications.R-GMA presents users with a virtual database and mediates queries posed at this database: users pose queries against a global schema and R-GMA takes responsibility for locating relevant sources and returning an answer. R-GMA’s architecture and mechanisms are general and can be used wherever there is a need for publishing and querying information in a distributed environment.We discuss the requirements, design and implementation of R-GMA as deployed on the DataGrid testbed. We also describe some of the ways in which R-GMA is being used.

Fault tolerance in the R-GMA information and monitoring system

R-GMA (Relational Grid Monitoring Architecture) [1] is a grid monitoring and information system that provides a global view of data distributed across a grid system. R-GMA creates the impression of a single centralised repository of information, but in reality the information can be stored at many different locations on the grid. The Registry and Schema are key components of R-GMA. The Registry matches queries for information to data sources that provide the appropriate information. The Schema defines the tables that can be queried. Without the combined availability of these components, R-GMA ceases to operate as a useful service. This paper presents an overview of R-GMA and describes the Registry replication design and implementation. A replication algorithm for the Schema has also been designed.

Information and Monitoring Services within a Grid Environment

The R-GMA (Relational Grid Monitoring Architecture) was developed within the EU DataGrid project, to bring the power of SQL to an information and monitoring system for the grid. It provides producer and consumer services to both publish and retrieve information from anywhere within a grid environment. Users within a Virtual Organization may define their own tables dynamically into which to publish data. Within the DataGrid project R-GMA was used for the information system, making details about grid resources available for use by other middleware components. RGMA has also been used for monitoring grid jobs by members of the CMS and D0 collaborations where information about jobs is published from within a job wrapper, transported across the grid by R-GMA and made available to users. An accounting package for processing PBS logging data and sending it to one or more Grid Operation Centres using R-GMA has been written and is being deployed within LCG. There are many other existing and potential applications. R-GMA is currently being re-engineered to fit into a Web Service environment as part of the EU Enabling Grids for E-science in Europe (EGEE) project. Improvements being developed include fine grained authorization, an improved user interface and measures to ensure superior scaling behaviour.

The CanonicalProducer: an instrument monitoring component of the Relational Grid Monitoring Architecture (R-GMA)

We describe how the R-GMA (Relational Grid Monitoring Architecture) can be used to allow for instrument monitoring in a Grid environment. The R-GMA has been developed within the European DataGrid Project (EDG) as a Grid Information and Monitoring System. It is based on the Grid Monitoring Architecture (GMA) from the Global Grid Forum (GGF), which is a simple Consumer-Producer model. The special strength of this implementation comes from the power of the relational model. It offers a global view of the information as if each Virtual Organisation had one large relational database. It provides a number of different Producer types with different characteristics; for example some support streaming of information. We describe the R-GMA component that allows for instrument monitoring, the CanonicalProducer. We also describe an example use of this approach in the European CrossGrid project, SANTA-G, a network monitoring tool.

An event-driven information dissemination model

We note that over the past decade publish-subscribe systems have improved the ability of users to exchange information but we argue that these systems have been either restrictive or simplistic - relying on narrowly defined channels for data exchange. The Information Dissemination (INFOD) approach we present here introduces a flexible and dynamic framework for brokering information in publish-subscribe systems. INFOD enables communities of interest to constitute and use vocabularies to describe their interests as well as their capabilities (available information). Publishers, consumers and subscribers are real-world entities characterized in terms of vocabularies and constraints within an INFOD registry. Subscribers define subscriptions expressing events of interest at candidate publishers and constraints on the run-time event data. The registry matches entities based on subscriptions and constraints, and sets up event channels between them.

INFOrmation Dissemination (INFOD) middleware

The increasing volume, diversity, and complexity of resources and data continues to raise challenges in the sharing and dissemination of information. Although publish-subscribe systems have improved the ability of users to exchange information, we argue that these systems have been either restrictive or simplistic - relying significantly on narrowly defined channels for data exchange. The Information Dissemination (INFOD) approach we present here introduces a flexible and dynamic framework for brokering information in publish-subscribe systems. INFOD enables communities of interest to themselves constitute and use vocabularies for describing their interests as well as their capabilities (available information). Publishers, consumers and subscribers are real-world entities characterized in terms of vocabularies and their interests as constraints within an INFOD registry. Subscribers define subscriptions primarily as XQuery constraints expressing events of interest at candidate publishers. Subscriptions also specify constraints on the run-time data that must be disseminated to specific consumers. Entity descriptions, property constraints, and subscriptions comprise the metadata information that INFOD uses to associate and link entities within a community. We refer to the process of associating entities in INFOD as mutual filtering, which we realize with a three-way join across publishers, consumers, and subscriber entities. We demonstrate and evaluate the INFOD approach in an emergency response use case scenario that employs INFOD to support changing event dynamics and varying publisher-consumer-subscriber requirements. We use industry standard technologies and present system performance results for the mutual filtering steps for a variety of subscription constraints and broad classes of publisher, subscriber, consumer property descriptions.