Martin Biely

Tolerating corrupted communication

Consensus encalpsulates the inherent problems of building fault tolerant distributed systems. In this context, the classic model of Byzantine faulty processes can be restated such that messages from a subset of processes can be arbitrarily corrupted (including addition and omission of messages). We consider the case of dynamic and transient faults,that may affect all processes and that are not permanent, and we model them via corrupted communication. For corrupted communication it is natural to distinguish between the safety of communication, which is concerned with the number of altered messages, and the liveness of communication, which restricts message loss. We present two consensus algorithms, together with sufficient conditions on the system to ensure correctness. Our first algorithm needs strong conditions on safety but requires weak conditions on liveness in order to terminate. Our second algorithm tolerates a lower degree of communication safety at the price of stronger liveness conditions. Our algorithms allow us to circumvent the resilience lower bounds from Santoro/Widmayer and Martin/Alvisi.

S-Paxos: Offloading the Leader for High Throughput State Machine Replication

Implementations of state machine replication are prevalently using variants of Paxos or other leader-based protocols. Typically these protocols are also leader-centric, in the sense that the leader performs more work than the non-leader replicas. Such protocols scale poorly, because as the number of replicas or the load on the system increases, the leader replica quickly reaches the limits of one of its resources. In this paper we show that much of the work performed by the leader in a leader-centric protocol can in fact be evenly distributed among all the replicas, thereby leaving the leader only with minimal additional workload. This is done (i) by distributing the work of handling client communication among all replicas, (ii) by disseminating client requests among replicas in a distributed fashion, and (iii) by executing the ordering protocol on ids. We derive a variant of Paxos incorporating these ideas. Compared to leader-centric protocols, our protocol not only achieves significantly higher throughput for any given number of replicas, but also increases its throughput with the number of replicas.

Agreement in directed dynamic networks

We study the fundamental problem of achieving consensus in a synchronous dynamic network, where an omniscient adversary controls the unidirectional communication links. Its behavior is modeled as a sequence of directed graphs representing the active (i.e. timely) communication links per round. We prove that consensus is impossible under some natural weak connectivity assumptions, and introduce vertex-stable root components as a--practical and not overly strong--means for circumventing this impossibility. Essentially, we assume that there is a short period of time during which an arbitrary part of the network remains strongly connected, while its interconnect topology keeps changing continuously. We present a consensus algorithm that works under this assumption, and prove its correctness. Our algorithm maintains a local estimate of the communication graphs, and applies techniques for detecting stable network properties and univalent system configurations. Our possibility results are complemented by several impossibility results and lower bounds, which reveal that our algorithm is asymptotically optimal.

Synchronous consensus under hybrid process and link failures

We introduce a comprehensive hybrid failure model for synchronous distributed systems, which extends a conventional hybrid process failure model by adding communication failures: Every process in the system is allowed to commit up to fℓs send link failures and experience up to fℓr receive link failures per round here, without being considered faulty; up to some fℓsa≤fℓs and fℓra≤fℓr among those may even cause erroneous messages rather than just omissions. In a companion paper (Schmid et al. (2009) [14]), devoted to a complete suite of related impossibility results and lower bounds, we proved that this model surpasses all existing link failure modeling approaches in terms of the assumption coverage in a simple probabilistic setting. In this paper, we show that several well-known synchronous consensus algorithms can be adapted to work under our failure model, provided that the number of processes required for tolerating process failures is increased by small integer multiples of fℓs, fℓr, fℓsa, fℓra. This is somewhat surprising, given that consensus in the presence of unrestricted link failures and mobile (moving) process omission failures is impossible. We provide detailed formulas for the required number of processes and rounds, which reveal that the lower bounds established in our companion paper are tight. We also explore the power and limitations of authentication in our setting, and consider uniform consensus algorithms, which guarantee their properties also for benign faulty processes.

Optimal message-driven implementations of omega with mute processes

We investigate the complexity of algorithms in message-driven models. In such models, events in the computation can only be caused by message receptions, but not by the passage of time. Hutle and Widder [2005a] have shown that there is no deterministic message-driven self-stabilizing implementation of the eventually strong failure detector and thus Ω in systems with uncertainty in message delays and channels of unknown capacity using only bounded space. Under stronger assumptions it was shown that even the eventually perfect failure detector can be implemented in message-driven systems consisting of at least f + 2 processes (f being the upper bound on the number of processes that crash during an execution). In this article we show that f + 2 is in fact a lower bound in message-driven systems, even if nonstabilizing algorithms are considered. This contrasts time-driven models where f + 1 is sufficient for failure detector implementations. Moreover, we investigate algorithms where not all processes send message, that is, are active, but some (in a predetermined set) remain passive. Here, we show that the f + 2 processes required for message-driven systems must be active, while in time-driven systems it suffices that f processes are active. We also provide message-driven implementations of Ω. Our algorithms are efficient in the sense that not all processes have to send messages forever, which is an improvement to previous message-driven failure detector implementations.

Gracefully Degrading Consensus and k-Set Agreement in Directed Dynamic Networks

We present (This work has been supported the Austrian Science Fund (FWF) project P26436 (SIC) and S11405 (RiSE).) the first consensus/k-set agreement algorithm for synchronous dynamic networks with unidirectional links, controlled by an omniscient message adversary, which automatically adapts to the actual network properties in a run: If the network is sufficiently well-connected, it solves consensus, while it degrades gracefully to general k-set agreement in less well-connected communication graphs. The actual number k of system-wide decision values is determined by the number of certain vertex-stable root components occurring in a run, which are strongly connected components without incoming links from outside. Related impossibility results reveal that our condition is reasonably close to the solvability border for k-set agreement.

Distal: A framework for implementing fault-tolerant distributed algorithms

We introduce Distal, a new framework that simplifies turning pseudocode of fault tolerant distributed algorithms into efficient executable code. Without proper tool support, even small amounts of pseudocode normally ends up in several thousands of non-trivial lines of Java or C++. Distal is implemented as a library in Scala and consists of two main parts: a domain specific language (DSL) in which algorithms are expressed and an efficient messaging layer that deals with low level issues such as connection management, threading and (de)serialization. The DSL is designed such that implementations of distributed algorithms highly resemble the pseudocode found in research papers. By writing code that is close to the protocol description, one can be more convinced that the implemented system really reflects the protocol specification on paper. Distal does not only make it simple and intuitive to implement distributed algorithms but it also leads to efficient implementations.

Easy impossibility proofs for k-set agreement in message passing systems

Despite of being quite similar (agreement) problems, 1-set agreement (consensus) and general k-set agreement require surprisingly different techniques for proving the impossibility in asynchronous systems with crash failures: Rather than the relatively simple bivalence arguments as in the impossibility proof for consensus in the presence of a single crash failure, known proofs for the impossibility of k-set agreement in shared memory systems with f ≥ k > 1 crash failures use algebraic topology or a variant of Sperner’s Lemma. In this paper, we present a generic theorem for proving the impossibility of k-set agreement in various message passing settings, which is based on a reduction to the consensus impossibility in a certain subsystem resulting from a partitioning argument.

Optimal message-driven implementation of omega with mute processes

We consider the complexity of algorithms in message-driven models, i.e., models of distributed computations where events can only be caused by message receptions but not by the passage of time. Hutle and Widder (2005) have shown that there is no self-stabilizing implementation of the eventually strong failure detector, and thus the eventual leader oracle Ω in such models under certain assumptions. Under stronger assumptions it was shown that even the eventually perfect failure detector can be implemented in systems consisting of at least f + 2 processes-- f being the upper bound on the number of processes that crash during an execution. In this paper we show that f + 2 is in fact a lower bound in messagedriven systems, even if non stabilizing algorithms are considered. This contrasts time-driven models where f + 1 is sufficient for failure detector implementations. After that, we provide an efficient message-driven implementation of Ω. Our algorithm is efficient in the sense that not all processes have to send messages forever, which is an improvement to previous message-driven failure detector implementations.

Brief announcement: gracefully degrading consensus and k-set agreement under dynamic link failures

We present a k-set agreement algorithm for synchronous dynamic distributed systems with unidirectional links controlled by an omniscient adversary. Our algorithm automatically adapts to the actual network properties: If the network is sufficiently well-connected, it solves consensus, while degrading gracefully to general k-set agreement in less well-behaved runs. The algorithm is oblivious to the maximum number of system-wide decision values k, which is bounded by the number of certain strongly connected components occurring in the dynamically changing network in a run. Related impossibility results reveal that this bound is close to the solvability border for k-set agreement. To the best of our knowledge, this is the first consensus algorithm that degrades in a graceful way in a dynamic network.