Helge Parzyjegla

Analyzing content-based publish/subscribe systems

Publish/subscribe communication is increasingly often the basis for distributed, event-based, and loosely-coupled applications. With this communication paradigm, application components communicate indirectly by publishing notifications and by subscribing to those notifications of interest. To precisely specify the subscribers interests, content-based publish/subscribe systems provide expressive filter models that, on the one hand, ease implementing versatile and entangled interaction patterns. On the other hand, however, resulting notification flows evolve dynamically and implicitly making an analysis the more challenging and complex. In this paper, we focus on modeling the interrelationship between the notifications being advertised, produced, and requested by publishers and subscribers, respectively. The presented analytical models, in their general form, support various data and filter models to compute matching and forwarding probabilities, which provide the basis to derive message rates as well as more advanced performance metrics. To demonstrate their applicability, we instantiate the general models with a specific data and filter model based on a non-uniform, multi-dimensional distribution of published and subscribed notifications. Our results show the impact of the distributions skewness and, thereby, prove the necessity to precisely model the content interrelationships in order to make reliable performance predictions.

Distributed Composite Event Detection in Publish/Subscribe Networks - A Case for Self-Organization

Event-based cooperation is well suited to model the interaction of components in distributed, dynamically changing environments prevalent, for example, in ubiquitous computing scenarios. Publish/subscribe middleware can be used to efficiently implement event-based cooperation. However, while application components may be interested not only in single events, but also in spatio-temporal patterns of events, called composite events, research in the area of routing algorithms for publish/subscribe systems has focused mainly on efficiently routing individual notifications from producers to their consumers without providing means for correlation. In order to avoid every application having to subscribe to all events that may form an interesting event pattern to detect, which can waste large amounts of network bandwidth, nwe propose to realize composite event detection as a middleware service. nWhile a centralized implementation of this service would be simpler to realize, we favor distributed composite event detection inside the broker network because this way locality in publication rates and subscriber interests can be exploited. However, placing detectors for composite events such that the required network bandwidth is minimized is a complex on-line optimization problem. In this paper, we present our ideas to place composite event detectors inside the publish/subscribe broker network and to adapt this placement at runtime. The placement is based on a self-organizing optimization using a spring relaxation heuristic considering multiple event patterns.

Dynamic Sets: A Programming Abstraction for Object Bundling

Due to the dynamics of distributed ensembles in ubiquitous environments, developers can neither predetermine all environment settings at design time, nor rely on particular services or devices to be available at runtime. To better cope with such uncertainties, dynamic sets provide a programming abstraction that enables developers to program to the interface of a single device at design time, but to bundle and control a variable number of instances at runtime. Moreover, selection criteria for devices as well as the behavior of the set can be customized upon deployment or even be adapted at runtime.

Improving random write performance in heterogeneous erasure-coded drive arrays by offloading code block requests

Erasure coding is used together with data striping in RAIDs for almost three decades. These techniques combine individual hard disk drives (HDDs) to satisfy requirements on performance, capacity, and reliability that cannot be met by a single drive. While these techniques are well known, the advent of NAND ash memory based solid-state drives (SSDs) now opens up new options to mix heterogeneous drives to combine the specific advantages of the different technologies. In this paper, we address a main weakness of traditional (i.e., homogeneous) erasure-coded drive arrays, namely their inherent random write penalty. We focus on a specific use of heterogeneous drives to improve the performance of small to medium-sized random writes. This is of particular interest, because erasure coding provides significantly better space efficiency compared to replication-based approaches, but performs worse for random writes. We describe a drive array organization alleviating this penalty by storing data blocks on slower and code blocks on faster drives, and we provide an analysis allowing to predict the performance of such arrays. An experimental evaluation of the approach (based on HDDs and SSDs) confirms the performance claims, and demonstrates the viability of the approach.

Dynamic Sets: A Programming Abstraction for Ubiquitous Computing and the Internet of Things

Applications in the areas of Ubiquitous Computing and the Internet of Things are often facing the problem that at design time it is not known what devices will be available at runtime and what preferences the user will have. Due to this gap between design time and runtime, mechanisms are needed to postpone decisions until the application is deployed and to adapt these decisions while it is running. Unfortunately, such mechanisms are currently mostly missing. In this paper, we propose dynamic sets as a programming abstraction that allows an application originally written for using a single device to interact with a set of devices of the same type instead. Using a dynamic set, an application can transparently call a method on a set of devices using a local proxy of the set that implements the very same interface as a device the application was originally written to interact with. Applications that are aware of dynamic sets can change the criteria used to select the members of a set or derive a new set from existing sets. Besides invocation mechanisms, dynamic sets provide functionality for result aggregation, automatic member management, update notifications, and derived business methods. Finally, applications can be exported making it possible to combine existing applications to flexible and dynamic mashups.

Adaptive information distribution for dynamic sets

While ubiquitous computing is becoming an integral part of our daily lives, developing applications for dynamic device ensembles is still a challenging and cumbersome task. Besides other reasons (e. g., heterogeneity), a major obstacle for application development is that the developer often does not know at design time which and how many devices will be available when the user deploys the application in his personal environment. To tackle this problem, a new programming abstraction called dynamic sets [7, 8] was proposed. A dynamic set allows an application to interact transparently with a set of remote objects in the same way it interacts with a single object. To achieve this, the developer programs against a proxy that replicates the calls to the objects and aggregates the return values. However, if an application is calling methods with high frequency, this can waste large amounts of network and computing resources. In this paper, we propose three adaptive heuristics tackling this problem for methods that only return information about the objects state. The heuristics dynamically switch between push and pull communication and use unicast as well as multicast communication to optimize resource usage. We discuss and prove algorithm details and present an evaluation based on simulations that shows that our approach can indeed reduce resource usage substantially.

ILP-based joint routing and scheduling for time-triggered networks

Networks in the automotive and aerospace area as well as in production facilities have to support time-critical (i.e., hard real-time) communication. For such applications, time-triggered Ethernet-based networking solutions provide the required timeliness, i.e., reliable packet delivery with deterministic latencies and low jitter. However, the routing and scheduling of the time-triggered traffic is an NP-hard problem. Hence, existing solutions to this problem make certain abstractions to reduce complexity if necessary. Nonetheless, such abstractions exclude feasible routing and scheduling options from the design space. Specifically, it is a typical approach to model routing and scheduling as separate problems, which are solved successively or with heuristic coupling. Therefore, we present a novel ILP formulation that can be used to jointly solve the routing and scheduling problem for time-triggered Ethernet networks. Using this formulation, it is possible to solve various scheduling problems that are infeasible when using a fixed shortest path routing with separate scheduling. Compared to a fixed load balanced routing with separate scheduling, schedules computed with our formulation offer lower communication latencies.

Adaptive information distribution for dynamic sets using multicast push and pull

When developing applications in the area of ubiquitous computing, the developer often does not know at design time which and how many devices will be available at runtime when the user deploys the application in his personal environment. To tackle this problem, we proposed a new programming abstraction called dynamic sets in previous work. A dynamic set allows an application to interact transparently with a set of remote objects in the same way it interacts with a single object. The developer programs against a proxy that replicates the calls to the objects and aggregates the return values. While dynamic sets make application development easier and more flexible, efficiency problems can arise when an application is calling proxy methods with high frequency as if the object was local (since the calls are replicated to all members of the dynamic set).n In this paper, we propose adaptive heuristics that tackle this problem for getter methods returning information about the objects state. The heuristics use unicast and multicast communication and dynamically switch between pull (application requests update from object) and push (object sends update to application) depending on the getter rates of applications and the setter rates of objects to optimize communication costs. After describing the basic interaction patterns and presenting the details of our heuristics, we present the results of a comprehensive evaluation based on discrete event simulations. The results show that our heuristics effectively reduce communication costs although they operate on the application layer and, thus, have no detailed knowledge about the physical network topology.

Improving random write performance in homogeneous and heterogeneous erasure-coded drive arrays

In data storage systems, drive arrays known as RAIDs are often used in order to avoid data loss and to maintain availability in the event of drive failure(s). RAID schemes define various drive array organizations (denoted as RAID levels) that can be used in arrays of hard disk drives (HDDs) and arrays of NAND flash memory solid-state drives (SSDs). For larger drive arrays, using data striping with erasure coding is appealing due to its notably higher space efficiency compared to data replication. However, the main issue of data striping with erasure coding is the performance of random writes smaller than a stripe. This problem is even aggravated if the random access performance characteristics of the deployed device type (HDD or SSD) and device model are not properly considered when choosing the data striping configuration (in particular the stripe unit size).n In this article, we provide an analytical model allowing to predict the random write throughput of homogeneous drive arrays as well as of a heterogeneous drive array with code blocks stored on the faster drives. Based on our model, we develop a method to improve the random write throughput in homogeneous drive arrays (comprising only one device type, e.g., HDDs or SSDs) by adapting the data striping configuration to the used device type and model in relation to the workload. Then, based on our previous work, we describe an organization for heterogeneous drive arrays, which is especially suitable for arrays combining HDDs with SSDs, and permits to further increase the random write throughput by storing data blocks on slower and code blocks on faster drives. Finally, we experimentally evaluate our analytical claims and show that random write throughput can indeed be notably increased in drive arrays that use data striping with erasure coding.

Organizing and evaluating publish/subscribe systems with scopes

Modern applications and services need to process and respond to an ever growing number of internal and external events. In principle, publish/subscribe middleware would be the ideal platform for these event-driven applications, which range from the gathering of simple sensor data in ubiquitous environments up to the correlation and processing of complex events in business workflows. Most publish/subscribe middleware, however, offers no or just little support to adequately structure and organize network, services, and applications making it hard to deploy and adapt them in complex dynamic infrastructures. In this paper, we refine and concretize a scope concept for publish/subscribe systems to give application developers and system administrators powerful and effective means to organize these systems by controlling the visibility of events, subscriptions, and advertisements. Besides presenting the scope concept and discussing the integration into a middleware, our main contribution is a comprehensive evaluation based on the first publish/subscribe middleware prototype with integrated scoping. The evaluation analyzes the behavior a scoped publish/subscribe system in various settings and quantifies its benefits.