Qais Noorshams

Predictive performance modeling of virtualized storage systems using optimized statistical regression techniques

Modern virtualized environments are key for reducing the operating costs of data centers. By enabling the sharing of physical resources, virtualization promises increased resource efficiency with decreased administration costs. With the increasing popularity of I/O-intensive applications, however, the virtualized storage used in such environments can quickly become a bottleneck and lead to performance and scalability issues. Performance modeling and evaluation techniques applied prior to system deployment help to avoid such issues. In current practice, however, virtualized storage and its performance-influencing factors are often neglected or treated as a black-box. In this paper, we present a measurement-based performance prediction approach for virtualized storage systems based on optimized statistical regression techniques. We first propose a general heuristic search algorithm to optimize the parameters of regression techniques. Then, we apply our optimization approach and create performance models using four regression techniques. Finally, we present an in-depth evaluation of our approach in a real-world representative environment based on IBM System z and IBM DS8700 server hardware. Using our optimized techniques, we effectively create performance models with less than 7% prediction error in the most typical scenario. Furthermore, our optimization approach reduces the prediction error by up to 74%.

I/O Performance Modeling of Virtualized Storage Systems

Server virtualization is a key technology to share physical resources efficiently and flexibly. With the increasing popularity of I/O-intensive applications, however, the virtualized storage used in shared environments can easily become a bottleneck and cause performance and scalability issues. Performance modeling and evaluation techniques applied prior to system deployment help to avoid such issues. In current practice, however, virtualized storage and its effects on the overall system performance are often neglected or treated as a black-box. In this paper, we present a systematic I/O performance modeling approach for virtualized storage systems based on queueing theory. We first propose a general performance model building methodology. Then, we demonstrate our methodology creating I/O queueing models of a real-world representative environment based on IBM System z and IBM DS8700 server hardware. Finally, we present an in-depth evaluation of our models considering both interpolation and extrapolation scenarios as well as scenarios with multiple virtual machines. Overall, we effectively create performance models with less than 11% mean prediction error in the worst case and less than 5% prediction error on average.

Experimental evaluation of the performance-influencing factors of virtualized storage systems

Virtualized cloud environments introduce an additional abstraction layer on top of physical resources enabling their collective use by multiple systems to increase resource efficiency. In I/O-intensive applications, however, the virtualized storage of such shared environments can quickly become a bottleneck and lead to performance and scalability issues. In software performance engineering, application performance is analyzed to assess the non-functional properties taking into account the many performance-influencing factors. In current practice, however, virtualized storage is either modeled as a black-box or tackled with full-blown and fine-granular simulations. This paper presents a systematic performance analysis approach of I/O-intensive applications in virtualized environments. First, we systematically identify storageperformance- influencing factors in a representative storage environment. Second, we quantify them using a systematic experimental analysis. Finally, we extract simple performance analysis models based on regression techniques. Our approach is applied in a real world environment using the state-of-the-art virtualization technology of the IBM System z and IBM DS8700.

Interactive Visual Analytics for Efficient Maintenance of Model Transformations

Maintaining model transformations remains a demanding task due to the sheer amount of metamodel elements and transformation rules that need to be understood. Several established techniques for software maintenance have been ported to model transformation development. Most available techniques proactively help to design and implement maintainable transformations, yet however, a growing number of legacy transformations needs to be maintained. Interactive visualization techniques to support model transformation maintenance still do not exist. We propose an interactive visual analytics process for understanding model transformations for maintenance. Data and control dependencies are statically analyzed and displayed in an interactive graph-based view with cross-view navigation and task-oriented filter criteria. We present results of an empirical study, where we asked programmers to carry out typical maintenance tasks on a real-world transformation in QVT-O. Subjects using our view located relevant code spots significantly more efficiently.

Designing information hiding modularity for model transformation languages

Development and maintenance of model transformations make up a substantial share of the lifecycle costs of software products that rely on model-driven techniques. In particular large and heterogeneous models lead to poorly understandable transformation code due to missing language concepts to master complexity. At the present time, there exists no module concept for model transformation languages that allows programmers to control information hiding and strictly declare model and code dependencies at module interfaces. Yet only then can we break down transformation logic into smaller parts, so that each part owns a clear interface for separating concerns. In this paper, we propose a module concept suitable for model transformation engineering. We formalize our concept based on cQVTom, a compact subset of the transformation language QVT-Operational. To meet the special demands of transformations, module interfaces give control over both model and code accessibility. We also implemented the approach for validation. In a case study, we examined the effort required to carry out two typical maintenance tasks on a real-world transformation. We are able to attest a significant reduction of effort, thereby demonstrating the practical effects of a thorough interface concept on the maintainability of model transformations.

Using quality of service bounds for effective multi-objective software architecture optimization

Quantitative prediction of non-functional properties, such as performance, reliability, and cost, of software architectures supports systematic software engineering. Even though there usually is a rough idea on bounds for quality of service, the exact required values may be unclear and subject to tradeoffs. Designing architectures that exhibit such good tradeoff between multiple quality attributes is hard. Even with a given functional design, many degrees of freedom in the software architecture (e.g. component deployment or server configuration) span a large design space. Automated approaches search the design space with multi-objective meta-heuristics such as evolutionary algorithms. However, as quality prediction for a single architecture is computationally expensive, these approaches are time consuming. In this work, we enhance an automated improvement approach to take into account bounds for quality of service in order to focus the search on interesting regions of the objective space, while still allowing trade-offs after the search. To validate our approach, we applied it to an architecture model of a component-based business information system. We compared the search to an unbounded search by running the optimization 8 times, each investigating around 800 candidates. The approach decreases the time needed to find good solutions in the interesting regions of the objective space by more than 35% on average.

Constructing performance model of JMS middleware platform

Middleware performance models are useful building blocks in the performance models of distributed software applications. We focus on performance models of messaging middleware implementing the Java Message Service standard, showing how certain system design properties -- including pipelined processing and message coalescing -- interact to create performance behavior that the existing models do not capture accurately. We construct a performance model of the ActiveMQ messaging middleware that addresses the outlined issues and discuss how the approach extends to other middleware implementations.

Modeling of I/O Performance Interference in Virtualized Environments with Queueing Petri Nets

Virtualization technology allows to share the physical resources used in IT infrastructures for efficient and flexible system operation. Sharing of physical resources, however, comes usually at the cost of performance and poses significant challenges to respect the Quality-of-Service of consolidated data-intensive applications due to the mutual performance interference among the applications. The non-trivial impact of workload consolidation on the I/O performance can be anticipated using explicit performance analysis techniques. In current practice, however, explicit modeling of I/O performance interference effects in virtualized environments is usually avoided due to their complexity. In this paper, we present an explicit performance modeling approach of I/O performance interference in virtualized environments with queueing Petri nets (QPNs). More specifically, we first highlight major challenges when modeling I/O performance in virtualized environments. Then, we create a single-VM I/O performance model calibrated with response time measurements to capture the complex behavior of a representative, real-world environment based on IBM System z and IBM DS8700 server hardware. Finally, we use the I/O performance model to evaluate the I/O performance when the workload is distributed heterogeneously on colocated virtual machines. Overall, we effectively create an I/O performance interference model capturing the I/O performance effects in a multi-VM environment with less than 10% prediction error on average.

Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems

Modern IT systems frequently employ virtualization technology to maximize resource efficiency. By sharing physical resources, however, the virtualized storage used in such environments can quickly become a bottleneck. Performance modeling and evaluation techniques applied prior to system deployment help to avoid performance issues. In current practice, however, modeling I/O performance is usually avoided due to the increasing complexity of modern virtualized storage systems. In this paper, we present an automated modeling approach based on statistical regression techniques to analyze I/O performance and interference effects in the context of virtualized storage systems. We demonstrate our approach in three case studies creating performance models with two I/O benchmarks. The case studies are conducted in a real-world environment based on IBM System z and IBM DS8700 server hardware. Using our approach, we effectively create performance models with excellent prediction accuracy for both I/O-intensive applications and I/O performance interference effects with a mean prediction error up to 7%.

Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

Next generation IT infrastructures are highly driven by virtualization technology. The latter enables flexible and efficient resource sharing allowing to improve system agility and reduce costs for IT services. Due to the sharing of resources and the increasing requirements of modern applications on I/O processing, the performance of storage systems is becoming a crucial factor. In particular, when migrating or consolidating different applications the impact on their performance behavior is often an open question. Performance modeling approaches help to answer such questions, a prerequisite, however, is to find an appropriate workload characterization that is both easy to obtain from applications as well as sufficient to capture the important characteristics of the application. In this paper, we present an automated workload characterization approach that extracts a workload model to represent the main aspects of I/O-intensive applications using relevant workload parameters, e.g., request size, read-write ratio, in virtualized environments. Once extracted, workload models can be used to emulate the workload performance behavior in real-world scenarios like migration and consolidation scenarios. We demonstrate our approach in the context of two case studies of representative system environments. We present an in-depth evaluation of our workload characterization approach showing its effectiveness in workload migration and consolidation scenarios. We use an IBM System z equipped with an IBM DS8700 and a Sun Fire system as state-of-the-art virtualized environments. Overall, the evaluation of our workload characterization approach shows promising results to capture the relevant factors of I/O-intensive applications.