Patrick Eugster

The many faces of publish/subscribe

Well adapted to the loosely coupled nature of distributed interaction in large-scale applications, the publish/subscribe communication paradigm has recently received increasing attention. With systems based on the publish/subscribe interaction scheme, subscribers register their interest in an event, or a pattern of events, and are subsequently asynchronously notified of events generated by publishers. Many variants of the paradigm have recently been proposed, each variant being specifically adapted to some given application or network model. This paper factors out the common denominator underlying these variants: full decoupling of the communicating entities in time, space, and synchronization. We use these three decoupling dimensions to better identify commonalities and divergences with traditional interaction paradigms. The many variations on the theme of publish/subscribe are classified and synthesized. In particular, their respective benefits and shortcomings are discussed both in terms of interfaces and implementations.

Epidemic information dissemination in distributed systems

Easy to deploy, robust, and highly resilient to failures, epidemic algorithms are a potentially effective mechanism for propagating information in large peer-to-peer systems deployed on Internet or ad hoc networks. It is possible to adjust the parameters of epidemic algorithm to achieve high reliability despite process crashes and disconnections, packet losses, and a dynamic network topology. Although researchers have used epidemic algorithms in applications such as failure detection, data aggregation, resource discovery and monitoring, and database replication, their general applicability to practical, Internet-wide systems remains open to question. We describe four key problems: membership maintenance, network awareness, buffer management, and message filtering, and suggest some preliminary approaches to address them.

On objects and events

This paper presents linguistic primitives for publish/subscribe programming using events and objects. We integrate our primitives into a strongly typed object-oriented language through four mechnisms: (1) serialization, (2) multiple subtyping, (3)closures, and (4) deferred code evaluation. We illustrate our primitives through Java, showing how we have overcome its respective lacks. A precompiler transforms statements based on our publish/subscribe primitives into calls to specifically generated typed adapters, which resemble the typed stubs and skeletons by the rmic precompiler for remote method invocations in Java

PAN: providing reliable storage in mobile ad hoc networks with probabilistic quorum systems

Reliable storage of data with concurrent read/write accesses (or query/update) is an ever recurring issue in distributed settings. In mobile ad hoc networks, the problem becomes even more challenging due to highly dynamic and unpredictable topology changes. It is precisely this unpredictability that makes probabilistic protocols very appealing for such environments. Inspired by the principles of probabilistic quorum systems, we present a Probabilistic quorum system for ad hoc networks Pan), a collection of protocols for the reliable storage of data in mobile ad hoc networks. Our system behaves in a predictable way due to the gossip-based diffusion mechanism applied for quorum accesses, and the protocol overhead is reduced by adopting an asymmetric quorum construction. We present an analysis of our Pan system, in terms of both reliability and overhead, which can be used to fine tune protocol parameters to obtain the desired tradeoff between efficiency and fault tolerance. We confirm the predictability and tunability of Pan through simulations with ns-2.

Distributed Asynchronous Collections: Abstractions for Publish/Subscribe Interaction

Publish/subscribe is considered one of the most important interaction styles for the explosive market of enterprise application integration. Producers publish information on a software bus and consumers subscribe to the information they want to receive from that bus. The decoupling nature of the interaction between the publishers and the subscribers is not only important for enterprise computing products but also for many emerging e-commerce and telecommunication applications. It is often claimed that object-orientation is inherently incompatible with the publish/subscribe interaction style. This flawed argument is due to the persistent confusion between object-orientation as a modeling discipline and the specific request/reply mechanism promoted by CORBA-like middleware systems. This paper describes object-oriented abstractions for publish/subscribe interaction in the form of Distributed Asynchronous Collections (DACs). DACs are general enough to capture the commonalities of various publish/subscribe interaction styles, and flexible enough to allow the exploitation of the differences between these flavors.

EventJava: An Extension of Java for Event Correlation

Event correlation has become the cornerstone of many reactive applications, particularly in distributed systems. However, support for programming with complex events is still rather specific and rudimentary. This paper presents EventJava, an extension of Java with generic support for event-based distributed programming. EventJava seamlessly integrates events with methods, and broadcasting with unicasting of events; it supports reactions to combinations of events, and predicates guarding those reactions. EventJava is implemented as a framework to allow for customization of event semantics, matching, and dispatching. We present its implementation, based on a compiler transforming specific primitives to Java, along with a reference implementation of the framework. We discuss ordering properties of EventJava through a formalization of its core as an extension of Featherweight Java. In a performance evaluation, we show that EventJava compares favorably to a highly tuned database-backed event correlation engine as well as to a comparably lightweight concurrency mechanism.

DICTATE: DIstributed CerTification Authority with probabilisTic frEshness for ad hoc networks

Securing ad hoc networks is notoriously challenging, notably due to the lack of an online infrastructure. In particular, key management is a problem that has been addressed by many researchers but with limited results. In this paper, we consider the case where an ad hoc network is under the responsibility of a mother certification authority (mCA). Since the nodes can frequently be collectively isolated from the mCA (e.g., for a remote mission) but still need the access to a certification authority, the mCA preassigns a special role to several nodes (called servers) that constitute a distributed certification authority (dCA) during the isolated period. We propose a solution, called DICTATE (DIstributed CerTification Authority with probabilisTic frEshness), to manage the dCA. This solution ensures that the dCA always processes a certificate update (or query) request in a finite amount of time and that an adversary cannot forge a certificate. Moreover, it guarantees that the dCA responds to a query request with the most recent version of the queried certificate in a certain probability; this probability can be made arbitrarily close to 1, but at the expense of higher overhead. Our contribution is twofold: 1) a set of certificate management protocols that allow trading protocol overhead for certificate freshness or the other way around, and 2) a combination of threshold and identity-based cryptosystems to guarantee the security, availability, and scalability of the certification function. We describe DICTATE in detail and, by security analysis and simulations, we show that it is robust against various attacks.

Type-based publish/subscribe: Concepts and experiences

A continuously increasing number of interconnected computer devices makes the requirement for programming abstractions for remote one-to-many interaction yet more stringent. The publish/subscribe paradigm has been advocated as a candidate abstraction for such one-to-many interaction at large scale. Common practices in publish/subscribe, however, include low-level abstractions which hardly leverage type safety, and provide only poor support for object encapsulation. This tends to put additional burden on software developers; guarantees such as the aforementioned type safety and object encapsulation become of increasing importance with an accrued number of software components, which modern applications also involve, besides an increasing number of hardware components.Type-based publish/subscribe (TPS) is a high-level variant of the publish/subscribe paradigm which aims precisely at providing guarantees such as type safety and encapsulation. We present the rationale and principles underlying TPS, as well as two implementations in Java: the first based on a specific extension of the Java language, and a second novel implementation making use of recent general-purpose features of Java, such as generics and behavioral reflection. We compare the two approaches, thereby evaluating the aforementioned features---as well as additional features which have been included in the most recent Java 1.5 release---in the context of distributed and concurrent programming. We discuss the benefits of alternative programming languages and features for implementing TPS. By revisiting alternative abstractions for distributed programming, including “classic” and recent ones, we extend our investigations to programming language support for distributed programming in general, pointing out that overall, the support in current mainstream programming languages is still insufficient.

From the Cloud to the Atmosphere: Running MapReduce across Data Centers

Efficiently analyzing big data is a major issue in our current era. Examples of analysis tasks include identification or detection of global weather patterns, economic changes, social phenomena, or epidemics. The cloud computing paradigm along with software tools such as implementations of the popular MapReduce framework offer a response to the problem by distributing computations among large sets of nodes. In many scenarios, input data are, however, geographically distributed (geodistributed) across data centers, and straightforwardly moving all data to a single data center before processing it can be prohibitively expensive. Above-mentioned tools are designed to work within a single cluster or data center and perform poorly or not at all when deployed across data centers. This paper deals with executing sequences of MapReduce jobs on geo-distributed data sets. We analyze possible ways of executing such jobs, and propose data transformation graphs that can be used to determine schedules for job sequences which are optimized either with respect to execution time or monetary cost. We introduce G-MR, a system for executing such job sequences, which implements our optimization framework. We present empirical evidence in Amazon EC2 and VICCI of the benefits of G-MR over common, naïve deployments for processing geodistributed data sets. Our evaluations show that using G-MR significantly improves processing time and cost for geodistributed data sets.

VNsnap: Taking snapshots of virtual networked environments with minimal downtime

A virtual networked environment (VNE) consists of virtual machines (VMs) connected by a virtual network. It has been adopted to create “virtual infrastructures” for individual users on a shared cloud computing infrastructure. The ability to take snapshots of an entire VNE — including images of the VMs with their execution, communication and storage states — yields a unique approach to reliability as a snapshot can restore the operation of an entire virtual infrastructure. We present VNsnap, a system that takes distributed snapshots of VNEs. Unlike existing distributed snapshot/checkpointing solutions, VNsnap does not require any modifications to the applications, libraries, or (guest) operating systems running in the VMs. Furthermore, VNsnap incurs only seconds of downtime as much of the snapshot operation takes place concurrently with the VNEs normal operation. We have implemented VNsnap on top of Xen. Our experiments with real-world parallel and distributed applications demonstrate VNsnaps effectiveness and efficiency.