Lubomír Bulej

Capturing performance assumptions using stochastic performance logic

Compared to functional unit testing, automated performance testing is difficult, partially because correctness criteria are more difficult to express for performance than for functionality. Where existing approaches rely on absolute bounds on the execution time, we aim to express assertions on code performance in relative, hardware-independent terms. To this end, we introduce Stochastic Performance Logic (SPL), which allows making statements about relative method performance. Since SPL interpretation is based on statistical tests applied to performance measurements, it allows (for a special class of formulas) calculating the minimum probability at which a particular SPL formula holds. We prove basic properties of the logic and present an algorithm for SAT-solver-guided evaluation of SPL formulas, which allows optimizing the number of performance measurements that need to be made. Finally, we propose integration of SPL formulas with Java code using higher-level performance annotations, for performance testing and documentation purposes.

CoCoME in Fractal

This chapter presents our solution to the CoCoME assignment that is based on the Fractal component model. The solution involves (i) modeling architecture in Fractal ADL, (ii) specification of component behavior via behavior protocols, (iii) checking compatibility of components, (iv) verification of correspondence between component code and behavior specification, and (v) run-time monitoring of non-functional properties. Among the issues we have faced was the need to modify the architecture - the component hierarchy was reorganized in order to improve clarity of the design and the hierarchical bus was split into two independent buses. These were modeled by primitive components, since Fractal does not support message bus as a first-class entity. Since the CoCoME assignment does not include a complete UML behavior specification (e.g. via activity diagrams and state charts), behavior protocols for all the components are based on the provided plain-English use cases, the UML sequence diagrams, and the reference Java implementation.

Analyzing the Impact of CPU Pinning and Partial CPU Loads on Performance and Energy Efficiency

While workload collocation is a necessity to increase energy efficiency of contemporary multi-core hardware, it also increases the risk of performance anomalies due to workload interference. Pinning certain workloads to a subset of CPUs is a simple approach to increasing workload isolation, but its effect depends on workload type and system architecture. Apart from common sense guidelines, the effect of pinning has not been extensively studied so far. In this paper we study the impact of CPU pinning on performance interference and energy efficiency for pairs of collocated workloads. Besides various combinations of workloads, virtualization and resource isolation, we explore the effects of pinning depending on the level of background load. The presented results are based on more than 1000 experiments carried out on an Intel-based NUMA system, with all power management features enabled to reflect real-world settings. We find that less common CPU pinning configurations improve energy efficiency at partial background loads, indicating that systems hosting collocated workloads could benefit from dynamic CPU pinning based on CPU load and workload type.

Using Connectors for Deployment of Heterogeneous Applications in the Context of OMG D&C Specification

The OMG Deployment and Configuration specification is an attempt at standardizing the deployment process of component-based applications in distributed environment. A software connector is an abstraction capturing interaction among components. Apart from middleware independence, connectors provide additional services (e.g. adaptation, monitoring, etc.) and benefits, especially in the area of integration of heterogeneous componentbased applications. This paper presents an approach for using connectors in the context of the deployment process defined by the OMG Deployment and Configuration specification to overcome incompatibilities in different component models and allow for deployment of truly heterogeneous component

ShadowVM: robust and comprehensive dynamic program analysis for the java platform

Dynamic analysis tools are often implemented using instrumentation, particularly on managed runtimes including the Java Virtual Machine (JVM). Performing instrumentation robustly is especially complex on such runtimes: existing frameworks offer limited coverage and poor isolation, while previous work has shown that apparently innocuous instrumentation can cause deadlocks or crashes in the observed application. This paper describes ShadowVM, a system for instrumentation-based dynamic analyses on the JVM which combines a number of techniques to greatly improve both isolation and coverage. These centre on the offload of analysis to a separate process; we believe our design is the first system to enable genuinely full bytecode coverage on the JVM. We describe a working implementation, and use a case study to demonstrate its improved coverage and to evaluate its runtime overhead.

Unit testing performance with Stochastic Performance Logic

Unit testing is an attractive quality management tool in the software development process, however, practical obstacles make it difficult to use unit tests for performance testing. We present Stochastic Performance Logic, a formalism for expressing performance requirements, together with interpretations that facilitate performance evaluation in the unit test context. The formalism and the interpretations are implemented in a performance testing framework and evaluated in multiple experiments, demonstrating the ability to identify performance differences in realistic unit test scenarios.

A programming model and framework for comprehensive dynamic analysis on Android

The multi-process architecture of Android applications combined with the lack of suitable APIs make dynamic program analysis (DPA) on Android challenging and unduly difficult. Existing analysis tools and frameworks are tailored mainly to the needs of security-related analyses and are not flexible enough to support the development of generic DPA tools. In this paper we present a framework that, besides providing the fundamental support for the development of DPA tools for Android, enables development of cross-platform analyses that can be applied to applications targeting the Android and Java platforms. The framework provides a convenient high-level programming model, flexible instrumentation support, and strong isolation of the base program from the analysis. To boost developer productivity, the framework retains Java as the main development language, while seamless integration with the platform overcomes the recurring obstacles hindering development of DPA tools for Android. We evaluate the framework on two diverse case studies, demonstrating key concepts, the flexibility of the framework, and analysis portability.

On the limits of modeling generational garbage collector performance

Garbage collection is an element of many contemporary software platforms whose performance is determined by complex interactions and is therefore difficult to quantify and model. We investigate the difference between the behavior of a real garbage collector implementation and a simplified model on a selection of workloads, focusing on the accuracy achievable with particular input information (sizes, references, lifetimes). Our work highlights the limits of performance modeling of garbage collection and points out issues of existing evaluation tools that may lead to incorrect experimental conclusions.

A comprehensive toolchain for workload characterization across JVM languages

The Java Virtual Machine (JVM) today hosts implementations of numerous languages. To achieve high performance, JVM implementations rely on heuristics in choosing compiler optimizations and adapting garbage collection behavior. Historically, these heuristics have been tuned to suit the dynamics of Java programs only. This leads to unnecessarily poor performance in case of non-Java languages, which often exhibit systematic differences in workload behavior. Dynamic metrics characterizing the workload help to identify and quantify useful optimizations, but so far, no cohesive suite of metrics has adequately covered properties that vary systematically between Java and non-Java workloads. We present a suite of such metrics, justifying our choice with reference to a range of guest languages. These metrics are implemented on a common portable infrastructure which ensures ease of deployment and customization.

Performance Awareness in Component Systems: Vision Paper

Resource awareness is a key requirement for dynamic adaptation in resource-constrained systems. Achieving resource awareness with clean separation of concerns and reasonable overhead is still a challenge - especially where this awareness concerns runtime performance. Among the difficult issues are for example transparent performance monitoring or platform independent performance evaluation. To advance the current state of the art in resource awareness, we propose a performance awareness framework for the domain of component-based systems. The framework is based on the Stochastic Performance Logic (SPL), which enables explicit description and automatic evaluation of assumptions about performance using logic formulas. We demonstrate the potential of the framework on multiple use-cases and outline extensions that facilitate the runtime resource awareness.