Marco Serafini

Leader election for replicated services using application scores

Replicated services often rely on a leader to order client requests and broadcast state updates. In this work, we present POLE, a leader election algorithm that select leaders using application-specific scores. This flexibility given to the application enables the algorithm to tailor leader election according to metrics that are relevant in practical settings and that have been overlooked by existing approaches. Recovery time and request latency are examples of such metrics. To evaluate POLE, we use ZooKeeper, an open-source replicated service used for coordinating Web-scale applications. Our evaluation over realistic wide-area settings shows that application scores can have a significant impact on performance, and that just optimizing the latency of consensus does not translate into lower latency for clients. An important conclusion from our results is that obtaining a general strategy that satisfies a wide range of requirements is difficult, which implies that configurability is indispensable for practical leader election.

On Barriers and the Gap between Active and Passive Replication

Active replication is commonly built on top of the atomic broadcast primitive. Passive replication, which has been recently used in the popular ZooKeeper coordination system, can be naturally built on top of the primary-order atomic broadcast primitive. Passive replication differs from active replication in that it requires processes to cross a barrier before they become primaries and start broadcasting messages. In this paper, we propose a barrier function tau that explains and encapsulates the differences between existing primary-order atomic broadcast algorithms, namely semi-passive replication and Zookeeper atomic broadcast (Zab), as well as the differences between Paxos and Zab. We also show that implementing primary-order atomic broadcast on top of a generic consensus primitive and tau inherently results in higher time complexity than atomic broadcast, as witnessed by existing algorithms. We overcome this problem by presenting an alternative, primary-order atomic broadcast implementation that builds on top of a generic consensus primitive and uses consensus itself to form a barrier. This algorithm is modular and matches the time complexity of existing tau-based algorithms.

P-Store: An Elastic Database System with Predictive Provisioning

OLTP database systems are a critical part of the operation of many enterprises. Such systems are often configured statically with sufficient capacity for peak load. For many OLTP applications, however, the maximum load is an order of magnitude larger than the minimum, and load varies in a repeating daily pattern. It is thus prudent to allocate computing resources dynamically to match demand. One can allocate resources reactively after a load increase is detected, but this places additional burden on the already-overloaded system to reconfigure. A predictive allocation, in advance of load increases, is clearly preferable. We present P-Store, the first elastic OLTP DBMS to use prediction, and apply it to the workload of B2W Digital (B2W), a large online retailer. Our study shows that P-Store outperforms a reactive system on B2Ws workload by causing 72% fewer latency violations, and achieves performance comparable to static allocation for peak demand while using 50% fewer servers.

Graph Data Mining with Arabesque

Graph data mining is defined as searching in an input graph for all subgraphs that satisfy some property that makes them interesting to the user. Examples of graph data mining problems include frequent subgraph mining, counting motifs, and enumerating cliques. These problems differ from other graph processing problems such as PageRank or shortest path in that graph data mining requires searching through an exponential number of subgraphs. Most current parallel graph analytics systems do not provide good support for graph data mining. One notable exception is Arabesque, a system that was built specifically to support graph data mining. Arabesque provides a simple programming model to express graph data mining computations, and a highly scalable and efficient implementation of this model, scaling to billions of subgraphs on hundreds of cores. This demonstration will showcase the Arabesque system, focusing on the end-user experience and showing how Arabesque can be used to simply and efficiently solve practical graph data mining problems that would be difficult with other systems.

Social piggybacking: leveraging common friends to generate event streams

Social networking systems present users with online event streams that include recent activities of their friends. Materialized per-user views are a common way to generate such event streams on-line and with low latency. We propose improving throughput by using social piggybacking: process the requests of two friends by just querying and updating the view of a third common friend. By using one such hub view, the system can serve requests of the first friend without querying nor updating the view of the second. This reduces the overall load on views, and can be implemented with minimal adaptations by existing social networking architectures. We show that, given a social graph, there exist hub-based schedules that minimize the overall number of view requests, but computing them is NP-hard. We propose a heuristic to solve the problem and evaluate it using samples from the Twitter and Flickr social graphs. The evaluation shows that existing solutions generate up to 2.4 times the amount of view requests induced by our heuristic.

Brief announcement: consensus and efficient passive replication

Passive replication is a popular practical approach to fault tolerance [1]. Using the Paxos consensus protocol [4] to implement it is seeing a growing popularity lately, but requires taking care of peculiar constraints. State updates must be applied using the same sequence of generation: if a primary is in state A and executes an operation making it transition to state B, the resulting state update δAB must be applied to the state A. Applying it to a different state C≠A is not safe because it might lead to an incorrect state, which is inconsistent with the history observed by replicas and clients. Paxos does not necessarily preserve the dependency between A and the delivery of δAB, as observed in [3].