Deborah A. Agarwal

Totem: a fault-tolerant multicast group communication system

When Totem delivers multicast messages, it invokes operations in the same total order throughout the distributed system. The result: consistency of replicated data and simpli ed programming of applications.

The Totem single-ring ordering and membership protocol

Fault-tolerant distributed systems are becoming more important, but in existing systems, maintaining the consistency of replicated data is quite expensive. The Totem single-ring protocol supports consistent concurrent operations by placing a total order on broadcast messages. This total order is derived from the sequence number in a token that circulates around a logical ring imposed on a set of processors in a broadcast domain. The protocol handles reconfiguration of the system when processors fail and restart or when the network partitions and remerges. Extended virtual synchrony ensures that processors deliver messages and configuration changes to the application in a consistent, systemwide total order. An effective flow control mechanism enables the Totem single-ring protocol to achieve message-ordering rates significantly higher than the best prior total-ordering protocols.

Extended virtual synchrony

We formulate a model of extended virtual synchrony that defines a group communication transport service for multicast and broadcast communication in a distributed system. The model extends the virtual synchrony model of the Isis system to support continued operation in all components of a partitioned network. The significance of extended virtual synchrony is that, during network partitioning and remerging and during process failure and recovery, it maintains a consistent relationship between the delivery of messages and the delivery of configuration changes across all processes in the system and provides well-defined self-delivery and failure atomicity properties. We describe an algorithm that implements extended virtual synchrony and construct a filter that reduces extended virtual synchrony to virtual synchrony.<<ETX>>

Network Characterization Service (NCS)

Distributed applications require information to effectively utilize the network. Some of the information they require is the current and maximum bandwidth, current and minimum latency, bottlenecks, burst frequency and congestion extent. This type of information allows applications to determine parameters like the optimal TCP buffer size. In this paper, we present a cooperative information-gathering tool called the Network Characterization Service (NCS). NCS runs in the user space and is used to acquire network information. Its protocol is designed for scalable and distributed deployment, similar to DNS. Its algorithms provide efficient, speedy and accurate detection of bottlenecks, especially dynamic bottlenecks. On current and future networks, dynamic bottlenecks do and will affect network performance dramatically.

Fast message ordering and membership using a logical token-passing ring

The Totem protocol supports consistent concurrent operations by placing a total order on broadcast messages. This total order is achieved by including a sequence number in a token circulated around a logical ring that is imposed on a set of processors in a broadcast domain. A membership algorithm handles reconfiguration, including restarting of a failed processor and remerging of a partitioned network. Effective flow-control allows the protocol to achieve message ordering rates two to three times higher than the best prior protocols. The single-ring total ordering protocol of Totem provides fault-tolerant agreed and safe delivery of messages within a broadcast domain.<<ETX>>

A reliable ordered delivery protocol for interconnected local area networks

We present-the Totem multiple-ring protocol, a novel reliable ordered multicast protocol for multiple interconnected local-area networks. The protocol exhibits excellent performance and maintains a consistent network-wide total order of messages despite network partitioning and remerging, or processor failure and recovery with stable storage intact. The Totem protocol is designed for fault-tolerant distributed systems, which replicate data to guard against failures and must ensure that replicated data remain consistent despite failures. The network-wide total order of messages provided by Totem simplifies the maintenance of consistency of replicated data, and, thus, eases the development of fault-tolerant distributed systems.

An integrated solution for secure group communication in wide-area networks

Many distributed applications require a secure reliable group communication system to provide coordination among the application components. This paper describes a secure group layer (SGL) which bundles a reliable group communication system, a group authorization and access control mechanism, and a group key agreement protocol to provide a comprehensive and practical secure group communication platform. The SGL also encapsulates the standard message security services (i.e., confidentiality, authenticity and integrity). A number of challenging issues encountered in the design of SGL are brought to light and experimental results obtained with a prototype implementation are discussed.

The Totem system

The Totem system supports fault-tolerant applications in which distributed processes cooperate to perform a common task and in which replicated data must be updated consistently in the presence of asynchrony and faults. Reliable totally ordered delivery of messages to processes within process groups is provided on a single local-area network or over multiple local-area networks interconnected by gateways. Message ordering is consistent across the entire network, despite processor and communication faults, without requiring all processes to deliver all messages. The Totem system handles processor failure and recovery, as well as network partitioning and remerging, and provides membership and topology maintenance services.<<ETX>>

Carbon Capture Simulation Initiative: A Case Study in Multiscale Modeling and New Challenges

Advanced multiscale modeling and simulation have the potential to dramatically reduce the time and cost to develop new carbon capture technologies. The Carbon Capture Simulation Initiative is a partnership among national laboratories, industry, and universities that is developing, demonstrating, and deploying a suite of such tools, including basic data submodels, steady-state and dynamic process models, process optimization and uncertainty quantification tools, an advanced dynamic process control framework, high-resolution filtered computational-fluid-dynamics (CFD) submodels, validated high-fidelity device-scale CFD models with quantified uncertainty, and a risk-analysis framework. These tools and models enable basic data submodels, including thermodynamics and kinetics, to be used within detailed process models to synthesize and optimize a process. The resulting process informs the development of process control systems and more detailed simulations of potential equipment to improve the design and reduce scale-up risk. Quantification and propagation of uncertainty across scales is an essential part of these tools and models.

Deep scientific computing requires deep data

Increasingly, scientific advances require the fusion of large amounts of complex data with extraordinary amounts of computational power. The problems of deep science demand deep computing and deep storage resources. In addition to teraflop-range computing engines with their own local storage, facilities must provide large data repositories of the order of 10-100 petabytes, and networking to allow the movement of multi-terabyte files in a timely and secure manner. This paper examines such problems and identifies associated challenges. The paper discusses some of the storage systems and data management methods that are needed for computing facilities to address the challenges and describes some ongoing improvements.