Stephan Kraft

Estimating service resource consumption from response time measurements

We propose a linear regression method and a maximum likelihood technique for estimating the service demands of requests based on measurement of their response times instead of their CPU utilization. Our approach does not require server instrumentation or sampling, thus simplifying the parameterization of performance models. The benefit of this approach is further highlighted when utilization measurement is difficult or unreliable, such as in virtualized systems or for services controlled by third parties. Both experimental results from an industrial ERP system and sensitivity analyses based on simulations indicate that the proposed methods are often much more effective for service demand estimation than popular utilization based linear regression methods. In particular, the maximum likelihood approach is found to be typically two to five times more accurate than utilization based regression, thus suggesting that estimating service demands from response times can help in improving performance model parameterization.

A Model of Storage I/O Performance Interference in Virtualized Systems

In this paper, we propose simple performance models to predict the impact of consolidation on the storage I/O performance of virtualized applications. We use a measurement-based approach based on tools such as blktrace and tshark for storage workload characterization in a commercial virtualized solution, namely VMware ESX server. Our approach allows a distinct characterization of read/write performance attributes on a per request level and provides valuable information for parameterization of storage I/O performance models. In particular, based on measures of quantities such as the mean queue-length seen upon arrival by requests, we define simple linear prediction models for the throughput, response times, and mix of read/write requests in consolidation based only on information collected in isolation experiments for the individual virtual machines.

Performance models of storage contention in cloud environments

We propose simple models to predict the performance degradation of disk requests due to storage device contention in consolidated virtualized environments. Model parameters can be deduced from measurements obtained inside Virtual Machines (VMs) from a system where a single VM accesses a remote storage server. The parameterized model can then be used to predict the effect of storage contention when multiple VMs are consolidated on the same server. We first propose a trace-driven approach that evaluates a queueing network with fair share scheduling using simulation. The model parameters consider Virtual Machine Monitor level disk access optimizations and rely on a calibration technique. We further present a measurement-based approach that allows a distinct characterization of read/write performance attributes. In particular, we define simple linear prediction models for I/O request mean response times, throughputs and read/write mixes, as well as a simulation model for predicting response time distributions. We found our models to be effective in predicting such quantities across a range of synthetic and emulated application workloads.

Predictive modelling of SAP ERP applications: challenges and solutions

Analytic performance models are being increasingly used to support system runtime optimization. This paper considers the modelling features needed to predict the response time behaviour of an industrial enterprise resource planning (ERP) application, SAP ERP. A number of studies have reported modelling success with the application of basic product-form Queueing Network Models (QNMs) to multi-tier systems. Such QNMs are often preferred in the context of optimization studies due to the low computational costs of their solution. However, we show that these simple models do not support many important features required to accurately characterize industrial applications such as ERP systems. Specifically, our results indicate that software threading levels, asynchronous database calls, priority scheduling, multiple phases of processing, and the parallelism offered by multi-core processors all have a significant impact on response time that cannot be neglected. n nStarting from these observations, the paper shows that Layered Queueing Models (LQMs) are a robust alternative to basic QNMs, while still enjoying analytical solution algorithms that facilitate their integration in optimization studies. A case study for a sales and distribution workload demonstrates that many of the features supported by LQMs are critical for achieving good prediction accuracy. Results show that, remarkably, all of the features we considered that are not captured by basic product-form QNMs are needed to predict mean response times to within 15% of measured values for a wide range of load levels. If any key feature is absent, the mean response time estimates could differ by 36% to 117% compared to the measured values, thus making the case that such non-product-form modelling features are needed for complex real-world applications.

IO performance prediction in consolidated virtualized environments

We propose a trace-driven approach to predict the performance degradation of disk request response times due to storage device contention in consolidated virtualized environments. Our performance model evaluates a queueing network with fair share scheduling using trace-driven simulation. The model parameters can be deduced from measurements obtained inside Virtual Machines (VMs) from a system where a single VM accesses a remote storage server. The parameterized model can then be used to predict the effect of storage contention when multiple VMs are consolidated on the same virtualized server. The model parameter estimation relies on a search technique that tries to estimate the splitting and merging of blocks at the the Virtual Machine Monitor (VMM) level in the case of multiple competing VMs. Simulation experiments based on traces of the Postmark and FFSB disk benchmarks show that our model is able to accurately predict the impact of workload consolidation on VM disk IO response times.

IO performance prediction in consolidated virtualized environments (abstracts only)

We propose a trace-driven approach to predict the performance degradation of disk request response times due to storage device contention in consolidated virtualized environments. Our performance model evaluates a queueing network with fair share scheduling using trace-driven simulation. The model parameters can be deduced from measurements obtained inside Virtual Machines (VMs) from a system where a single VM accesses a remote storage server. The parameterized model can then be used to predict the effect of storage contention when multiple VMs are consolidated on the same virtualized server. The model parameter estimation relies on a search technique that tries to estimate the splitting and merging of blocks at the the Virtual Machine Monitor (VMM) level in the case of multiple competing VMs. Simulation experiments based on traces of the Postmark and FFSB disk benchmarks show that our model is able to accurately predict the impact of workload consolidation on VM disk IO response times.