H. Raymond Strong

Extendible hashing—a fast access method for dynamic files

Extendible hashing is a new access technique, in which the user is guaranteed no more than two page faults to locate the data associated with a given unique identifier, or key. Unlike conventional hashing, extendible hashing has a dynamic structure that grows and shrinks gracefully as the database grows and shrinks. This approach simultaneously solves the problem of making hash tables that are extendible and of making radix search trees that are balanced. We study, by analysis and simulation, the performance of extendible hashing. The results indicate that extendible hashing provides an attractive alternative to other access methods, such as balanced trees.

Early stopping in Byzantine agreement

Two different kinds of Byzantine Agreement for distributed systems with processor faults are defined and compared. The first is required when coordinated actions may be performed by each participant at different times. This kind is called Simultaneous Byzantine Agreement (SBA). This paper deals with the number of rounds of message exchange required to reach Byzantine Agreement of either kind (BA). If an algorithm allows its participants to reach Byzantine agreement in every execution in which at most <italic>t</italic> participants are faulty, then the algorithm is said to tolerate <italic>t</italic> faults. It is well known that any BA algorithm that tolerates <italic>t</italic> faults (with <italic>t</italic> < <italic>n</italic> - 1 where <italic>n</italic> denotes the total number of processors) must run at least <italic>t</italic> + 1 rounds in some execution. However, it might be supposed that in executions where the number <italic>f</italic> of actual faults is small compared to <italic>t</italic>, the number of rounds could be correspondingly small. A corollary of our first result states that (when <italic>t</italic> < <italic>n</italic> - 1) any algorithm for SBA must run <italic>t</italic> + 1 rounds in some execution where there are no faults. For EBA (with <italic>t</italic> < <italic>n</italic> - 1), a lower bound of min(<italic>t</italic> + 1,<italic>f</italic> + 2) rounds is proved. Finally, an algorithm for EBA is presented that achieves the lower bound, provided that <italic>t</italic> is on the order of the square root of the total number of processors.

Fault-tolerant clock synchronization

This paper gives two simple efficient distributed algorithms: one for keeping clocks in a network synchronized and one for allowing new processors to join the network with their clocks synchronized. The algorithms tolerate both link and node failures of any type. The algorithm for maintaining synchronization will work for arbitrary networks (rather than just completely connected networks) and tolerates any number of processor or communication link faults as long as the correct processors remain connected by fault-free paths. It thus represents an improvement over other clock synchronization algorithms such as [LM1,LM2,LL1]. Our algorithm for allowing new processors to join requires that more than half the processors be correct, a requirement which is provably necessary.

Polynomial algorithms for multiple processor agreement

Reaching agreement in a distributed system while handling malfunctioning behavior is a central issue for reliable computer systems. All previous algorithms for reaching the agreement required an exponential number of messages to be sent, with or without authentication. We give polynomial algorithms for reaching (Byzantine) agreement, both with and without the use of authentication protocols. We also prove that no matter what kind of information is exchanged, there is no way to reach agreement with fewer than t+1 rounds of exchange, where t is the upper bound on the number of faults.

An efficient algorithm for byzantine agreement without authentication

Byzantine Agreement involves a system of n processes, of which some t may be faulty. The problem is for the correct processes to agree on a binary value sent by a transmitter that may itself be one of the n processes. If the transmitter sends the same value to each process, then all correct processes must agree on that value, but in any case, they must agree on some value. An explicit solution not using authentication for n = 3t + 1 processes is given, using 2t + 3 rounds and O(t3 log t) message bits. This solution is easily extended to the general case of n ⩾ 3t + 1 to give a solution using 2t + 3 rounds and O(nt + t3 log t) message bits.

A new look at fault-tolerant network routing

Abstract We model a communication network as a graph in which a processor is a node and a communication link is an edge. A routing for such a network is a fixed path, or route, between each pair of nodes. Given a network with a predefined routing, we study the effects of faulty components on the routing. Of particular interest is the number of routes along which a message must travel between any two non-faulty nodes. This problem is analyzed for specific families of graphs and for classes of routings. We also give some bounds for general versions of the problem. Finally, we conclude with one of the most important contributions of this paper, a list of interesting and apparently difficult open problems.

On the possibility and impossibility of achieving clock synchronization

It is known that clock synchronization can be achieved in the presence of faulty clocks numbering more than one-third of the total number of participating clocks provided that some authentication technique is used. Without authentication the number of faults that can be tolerated has been an open question. Here we show that if we restrict logical clocks to running within some linear function of real time, then clock synchronization is impossible, without authentication, when one-third or more of the processors are faulty. However, if there is a bound on the rate at which a processor can generate messages, then we show that clock synchronization is achievable, without authentication, as long as the faults do not disconnect the network. Finally, we provide a lower bound on the closeness to which simultaneity can be achieved in the network as a function of the transmission and processing delay properties of the network.

Atomic Broadcast

In distributed systems subject to random communication delays and component failures, atomic broadcast can be used to implement the abstraction of synchronous replicated storage, a distributed storage that displays the same contents at every correct processor as of any clock time. This paper presents a systematic derivation of a family of atomic broadcast protocols that are tolerant of increasingly general failure classes: omission failures, timing failures, and authentication-detectable Byzantine failures. The protocols work for arbitrary point-to-point network topologies, and can tolerate any number of link and process failures up to network partitioning. After proving their correctness, we also prove two lower bounds that show that the protocols provide in many cases the best possible termination times.

Efficient Message Passing Interface (MPI) for Parallel Computing on Clusters of Workstations

Parallel computing on clusters of workstations and personal computers has very high potential, since it leverages existing hardware and software. Parallel programming environments offer the user a convenient way to express parallel computation and communication. In fact, recently, a Message Passing Interface (MPI) has been proposed as an industrial standard for writing “portable” message-passing parallel programs. The communication part of MPI consists of the usual point-to-point communication as well as collective communication. However, existing implementations of programming environments for clusters are built on top of a point-to-point communication layer (send and receive) over local area networks (LANs) and, as a result, suffer from poor performance in the collective communication part. In this paper, we present an efficient design and implementation of the collective communication part in MPI that is optimized for clusters of workstations. Our system consists of two main components: the MPI-CCL layer that includes the collective communication functionality of MPI and a User-Level Reliable Transport Protocol (URTP) that interfaces with the LAN Data-Link Layer and leverages the fact that the LAN is a broadcast medium. Our system is integrated with the operating system via an efficient kernel extension mechanism that we developed. The kernel extension significantly improves the performance of our implementation as it can handle part of the communication overhead without involving user space. We have implemented our system on a collection of IBM RS/6000 workstations connected via a 10-Mbit Ethernet LAN. Our performance measurements are taken from typical scientific programs that run in a parallel mode by means of the MPI. The hypothesis behind our design is that the systems performance will be bounded by interactions between the kernel and user space rather than by the bandwidth delivered by the LAN Data-Link Layer. Our results indicate that the performance of our MPI Broadcast (on top of Ethernet) is about twice as fast as a recently published software implementation of broadcast on top of ATM.

Shifting gears: changing algorithms on the fly to expedite Byzantine agreement

We describe several new algorithms for Byzantine agreement . The first of-these is a simplification of the original exponential-time Byzantine agreement algorithm due to Pease, Shostak, and Lamport, and is of comparable complexity to their algorithm . However, its proof is very intuitively appealing . A technique of shifting between algorithms for solving the Byzantine agreement problem is then studied . We present two families of algorithms obtained by applying a shift operator to our first algorithm . These families obtain the same rounds to message length trade-off as do Coans families but do not require the exponential local computation time (and space of his algorithms . We also describe a modification of an O(/ )-resilient algorithm for Byzantine agreement of Dolev, Reischuk, and Strong . Finally, we obtain a hybrid algorithm that dominates all our others, by beginning execution of an algorithm in one family and shifting first into an algorithm of the second family and finally shifting into an execution of the adaptation of the Dolev, Reischuk, and Strong algorithm . 0 `IBM T. J. Watson Research Center, P . 0. Box 704, Yorktown Heights, NY 10598. This work was carried out while this author was a Ph .D . student of the Hebrew University, Jerusalem, Israel . tIBM Almaden Research Center, 650 Harry Road, San Jose, CA 95120 and the Computer Science Department, Hebrew University, Jerusalem, Israel .