Galen S. Swint

Expertus: A Generator Approach to Automate Performance Testing in IaaS Clouds

Cloud computing is an emerging technology paradigm that revolutionizes the computing landscape by providing on-demand delivery of software, platform, and infrastructure over the Internet. Yet, architecting, deploying, and configuring enterprise applications to run well on modern clouds remains a challenge due to associated complexities and non-trivial implications. The natural and presumably unbiased approach to these questions is thorough testing before moving applications to production settings. However, thorough testing of enterprise applications on modern clouds is cumbersome and error-prone due to a large number of relevant scenarios and difficulties in testing process. We address some of these challenges through Expertus---a flexible code generation framework for automated performance testing of distributed applications in Infrastructure as a Service (IaaS) clouds. Expertus uses a multi-pass compiler approach and leverages template-driven code generation to modularly incorporate different software applications on IaaS clouds. Expertus automatically handles complex configuration dependencies of software applications and significantly reduces human errors associated with manual approaches for software configuration and testing. To date, Expertus has been used to study three distributed applications on five IaaS clouds with over 10,000 different hardware, software, and virtualization configurations. The flexibility and extensibility of Expertus and our own experience on using it shows that new clouds, applications, and software packages can easily be incorporated.

Detecting bottleneck in -tier IT applications through analysis

As the complexity of large-scale enterprise applications increases, providing performance verification through staging becomes an important part of reducing business risks associated with violating sophisticated service-level agreement (SLA). Currently, performance verification during the staging process is accomplished through either an expensive, cumbersome manual approach or ad hoc automation. This paper describes an automation approach as part of the Elba project supporting monitoring and performance analysis of distributed multi-tiered applications that helps in bottleneck detection. We use machinelearning to determine service-level objectives (SLOs) satisfaction and locate bottlenecks in candidate deployment scenarios. We evaluate our tools with TPC-W, an on-line bookstore, and RUBiS, an on-line auction site.

Clearwater: extensible, flexible, modular code generation

Distributed applications typically interact with a number of heterogeneous and autonomous components that evolve independently. Methodical development of such applications can benefit from approaches based on domain-specific languages (DSLs). However, the evolution and customization of heterogeneous components introduces significant challenges to accommodating the syntax and semantics of a DSL in addition to the heterogeneous platforms on which they must run. In this paper, we address the challenge of implementing code generators for two such DSLs that are flexible (resilient to changes in generators or input formats), extensible (able to support multiple output targets and multiple input variants), and modular (generated code can be re-written). Our approach, Clearwater, leverages XML and XSLT standards: XML supports extensibility and mutability for in-progress specification formats, and XSLT provides flexibility and extensibility for multiple target languages. Modularity arises from using XML meta-tags in the code generator itself, which supports controlled addition, subtraction, or replacement to the generated code via XML-weaving. We discuss the use of our approach and show its advantages in two non-trivial code generators: the Infopipe Stub Generator (ISG) to support distributed flow applications, and the Automated Composable Code Translator to support automated distributed application deployment. As an example, the ISG accepts as input an XML description and generates output for C, C++, or Java using a number of communications platforms such as sockets and publish-subscribe.

Automated Staging for Built-to-Order Application Systems

The increasing complexity of enterprise and distributed systems demands automated design, testing, deployment, and monitoring of applications. Testing, or staging, in particular poses unique challenges. In this paper, we present the Elba project and Mulini generator. The goal of Elba is creating automated staging and testing of complex enterprise systems before deployment to production. Automating the staging process lowers the cost of testing applications. Feedback from staging, especially when coupled with appropriate resource costs, can be used to ensure correct functionality and provisioning for the application. The Elba project extracts test parameters from production specifications (such as SLAs) and deployment specifications, and via the Mulini generator, creates staging plans for the application. We then demonstrate Mulini on an example application, TPC-W, and show how information from automated staging and monitoring allows us to refine application deployments easily based on performance and cost

Towards automated deployment of built-to-order systems

End-to-end automated application design and deployment poses a significant technical challenge. With increasing scale and complexity of IT systems and the manual handling of existing scripts and configuration files for application deployment that makes them increasingly error-prone and brittle, this problem has become more acute. Even though design tools have been used to automate system design, it is usually difficult to translate these designs to deployed systems in an automated manner due to both syntactic obstacles and the synchronization of multiple activities involved in such a deployment. We describe a generic process of automated deployment from design documents and evaluate this process for 1, 2, and 3-tier distributed applications.

Mulini: an automated staging framework for QoS of distributed multi-tier applications

The increasing scale and success of distributed multi-tier applications have created increasingly dynamic workload variations that made system performance less predictable. Consequently, staging has become a significant and useful method to characterize the performance and Quality of Service (QoS) of such applications. Manual staging is an expensive, time consuming and error-prone process. In particular, manually exploring a large configuration parameter space of the applications is a cumbersome task. In this article, we outline the design of Mulini, an automated staging framework for large-scale multi-tier applications that realizes the automation via an extensible and flexible code generator. Mulini adopts XSLT/XPath tools and aspect-oriented programming (AOP) techniques to manipulate XML-encoded high-level specifications and weave non-functional specifications (e.g., QoS) into staging implementation. To illustrate the usability of the Mulini code generator in complex staging, we apply Mulini to bottleneck detection and observation-based performance characterization of the RUBiS e-Commerce benchmark.

Issues in Bottleneck Detection in Multi-Tier Enterprise Applications

In this work, the performance of various machine learning classifiers with regard to bottleneck detection in enterprise, multi-tier applications governed by service level objectives is described. Specifically, in this paper, it demonstrates the effectiveness of three classifiers, a tree-augmented Naive Bayesian network, a J48 decision tree, and LogitBoost, using our bottleneck detection process, which delves into a new area of performance analysis based on the trends of metrics (first order derivative) rather than the metric value itself. Furthermore, the efficiency of each classifier by measuring the convergence speed, or the number of staging trials required in order to provide positive results is illustrated. Finally, the effectiveness of the classifiers used in the bottleneck detection process as each classifier strongly identifies the enterprise system bottleneck

DSCWeaver: Synchronization-Constraint Aspect Extension to Procedural Process Specification Languages

BPEL is emerging as an open-standards language for Web service composition. However, its procedural style can lead to inflexible and tangled code for managing a crosscutting aspect - synchronization constraints that define permissible sequences of execution for activities in a process. In this paper, we present DSCWeaver, a tool that enables a synchronization-aspect extension to BPEL. It uses DSCL, a synchronization expression language, to specify constraints. DSCL has the desirable features of declarative syntax, fine granularity, and validation support. A designer can use DSCL to describe and validate the synchronization behavior and rely on DSCWeaver to generate BPEL code. We demonstrate the advantages of our approach in a service deployment process and evaluate its performance using two metrics: lines of code (LoC) and places to visit (PtV). Evaluation results show that our approach can effectively reduce development effort of process designers while providing performance competitive to un-woven BPEL code

Code generation for WSLAs using AXpect

WSLAs can be viewed as describing the service aspect of Web services. By their nature, Web services are distributed. Therefore, integrating support code into a Web service application is potentially costly and error prone. Viewed from this AOP perspective, then, we present a method for integrating WSLAs into code generation using the AXpect weaver, the AOP technology for Infopipes. This helps to localize the code physically and therefore increase the eventual maintainability and enhance the reuse of the WSLA code. We then illustrate the weavers capability by using a WSLA document to codify constraints and metrics for a streaming image application that requires CPU resource monitoring.

Infopipes: concepts and ISG implementation

We describe Infopipes, a distributed computational and communications abstraction for information flow applications and I/O intensive distributed real-time embedded (DRE) systems. Infopipes are specified by the syntax, semantics, and quality of service requirements for information flows. Software tools generate executable code from the specification. We explain a DRE scenario and then provide a microbenchmark comparison of generated Infopipe code to standard, handwritten TCP code. Measurements show that Infopipe-generated code has the same execution overhead as the manually written original version.