Kai Sachs

Event-based applications and enabling technologies

Event processing has become the paradigm of choice in many monitoring and reactive applications. However, the understanding of events, their composition and level of abstraction, the style of processing and the quality of service requirements vary drastically across application domains. We introduce the basic notions of event processing to create a common understanding, present the enabling technologies that are used for the implementation of event-based systems, survey a wide range of applications identifying their main features, and discuss open research issues.

Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark

Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 standard benchmark. SPECjms2007 is based on a novel application in the supply chain management domain, designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: (i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, (ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, (iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses a standard benchmark, providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.

Benchmarking in the Cloud: What It Should, Can, and Cannot Be

With the increasing adoption of Cloud Computing, we observe an increasing need for Cloud Benchmarks, in order to assess the performance of Cloud infrastructures and software stacks, to assist with provisioning decisions for Cloud users, and to compare Cloud offerings. We understand our paper as one of the first systematic approaches to the topic of Cloud Benchmarks. Our driving principle is that Cloud Benchmarks must consider end-to-end performance and pricing, taking into account that services are delivered over the Internet. This requirement yields new challenges for benchmarking and requires us to revisit existing benchmarking practices in order to adopt them to the Cloud.

Statistical inference of software performance models for parametric performance completions

Software performance engineering (SPE) enables software architects to ensure high performance standards for their applications. However, applying SPE in practice is still challenging. Most enterprise applications include a large software basis, such as middleware and legacy systems. In many cases, the software basis is the determining factor of the system’s overall timing behavior, throughput, and resource utilization. To capture these influences on the overall system’s performance, established performance prediction methods (model-based and analytical) rely on models that describe the performance-relevant aspects of the system under study. Creating such models requires detailed knowledge on the system’s structure and behavior that, in most cases, is not available. In this paper, we abstract from the internal structure of the system under study. We focus on message-oriented middleware (MOM) and analyze the dependency between the MOM’s usage and its performance. We use statistical inference to conclude these dependencies from observations. For ActiveMQ 5.3, the resulting functions predict the performance with a relative mean square error 0.1.

Performance Evaluation: Metrics, Models and Benchmarks

This book constitutes the thoroughly refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008 . The 17 revised full papers presented were carefully selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.

Performance modeling and analysis of message-oriented event-driven systems

Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.

Benchmarking publish/subscribe-based messaging systems

Publish/subscribe-based messaging systems are used increasingly often as a communication mechanism in data-oriented web applications. Such applications often pose serious performance and scalability challenges. To address these challenges, it is important that systems are tested using benchmarks to evaluate their performance and scalability before they are put into production. In this paper, we present jms2009-PS, a new benchmark for publish/subscribe-based messaging systems built on top of the SPECjms2007 standard workload.We introduce the benchmark and discuss its configuration parameters showing how the workload can be customized to evaluate various aspects of publish/subscribe communication. Finally, we present a case study illustrating how the benchmark can be used for performance analysis of messaging servers.

Communication-Aware and Energy-Efficient Scheduling for Parallel Applications in Virtualized Data Centers

In this paper, we propose Peer VMs Aggregation (PVA) to enable dynamic discovery of communication patterns and reschedule VMs based on the determined communication patterns using VM migration. In the implementation, we consider that communication delays occur at the server (i.e., memory-bus) and at the data center network. To evaluate our approach, we modeled a network and a memory subsystem on CloudSim simulator. We then used NAS Parallel Benchmarks, which consists of six different applications as parallel applications. We thoroughly evaluated our proposed approach measuring several assessment metrics including VMs placement, performance degradation, and network utilization of each link. The results of the simulation show that our proposed approach significantly reduces the total amount of traffic in the network where it reduces the average of the networks utilization by 25%.

Towards benchmarking of AMQP

With the increasing importance of event-based systems the performance of underlying event transporting systems, such as message oriented middleware (MOM), becomes business critical. Therefore, we see a strong need for benchmarks for such environments. Several messaging standards and protocols for middleware exist; most popular is the Java Message Service (JMS) which is defining an API rather than a wire protocol. An emerging standard is the new wire level protocol Advanced Message Queuing Protocol (AMQP). It originated in the financial sector and is developed by a consortium of over 20 member organizations.

Benchmarking of message-oriented middleware

Message-oriented middleware (MOM) is increasingly used as enabling technology for modern event-driven applications typically based on publish/subscribe (pub/sub) communication [1]. Many of these applications are designed for maximum scalability and flexibility and as such, they pose some serious performance issues for the underlying pub/sub middleware. Additionally, software designers face a new challenge: designing message-based communication flows which rely on asynchronous decoupled communication patterns. In order to develop good designs, system designers have to understand quality of service aspects and their performance costs. We believe that benchmarks are very helpful tools to analyse these aspects.