Paul Brebner

Is your cloud elastic enough?: performance modelling the elasticity of infrastructure as a service (IaaS) cloud applications

Elasticity, the ability to rapidly scale resources up and down on demand, is an essential feature of public cloud platforms. However, it is difficult to understand the elasticity requirements of a given application and workload, and if the elasticity provided by a cloud provider will meet those requirements. We introduce the elasticity mechanisms of a typical Infrastructure as a Service (IaaS) cloud platform (inspired by Amazon EC2). We have enhanced our Service Oriented Performance Modelling method and tool to model and predict the elasticity characteristics of three realistic applications and workloads on this cloud platform. We compare the pay-as-you-go instance costs and end-user response time service level agreements for different elasticity scenarios. The model is also able to predict the elasticity requirements (in terms of the maximum instance spin-up time) for the three applications. We conclude with an analysis of the results.

Performance modeling for service oriented architectures

We present a tool for performance modeling of Service Oriented Architectures (SOAs). As mission-critical use of whole-of-government SOAs become pervasive, the capability to model and predict the run-time performance of interdependent composite applications is critical. The tool can be used by architects early in the software engineering lifecycle to predict performance and scalability, to evaluate architectural alternatives, to provide guidance for capacity planning and the negotiation of Service Level Agreements (SLAs). It directly models and produces metrics for SOA applications in terms that are familiar to architects (services, workflows, and compositions of services). The tool enables the performance model to be generated from available architectural artifacts and performance data, making it easy to use. It is highly dynamic to facilitate interactive evaluation of alternative architectural choices. The tool can model complex deployment scenarios such as server virtualisation. Development and evaluation of the tool was carried out in the context of architectural modeling for large-scale SOA-based Australian e-Government systems. The tool radically simplified the construction and execution of SOA performance models, and contributed critical insights for the architecting of these systems.

Business transformation to SOA: aspects of the migration and performance and QoS issues

Organizations face challenges to be more adaptable and transform to meet new customer demands with fewer resources and streamlining of its business activities. There is a growing move to introduce SOAs with their promise of cost-efficiency, agility, adaptability and legacy leverage. However there are many aspects of transforming an organization to use SOA and many obstacles and issues that the organization has to address when introducing SOAs. In this paper we outline some of the major aspects of SOA introduction and focus on some of the open issues that still need to be tackled. In the discussion on the various aspects of SOA introduction we focus in on performance and QoS which are major pieces to get right if the SOA implementation is to be successful. We outline some of the work that we are doing in this area and some problem areas where further research is needed.

Performance and cost assessment of cloud services

Architecting applications for the Cloud is challenging due to significant differences between traditional hosting and Cloud infrastructure setup, unknown and unproven Cloud performance and scalability characteristics, as well as variable quota limitations. Building workable cloud applications therefore requires in-depth insight into the architectural and performance characteristics of each cloud offering, and the ability to reason about tradeoffs and alternatives of application designs and deployments. NICTA has developed a Service Oriented Performance Modeling technology for modeling the performance and scalability of Service Oriented applications architected for a variety of platforms. Using a suite of cloud testing applications we conducted in-depth empirical evaluations of a variety of real cloud infrastructures, including Google App Engine, Amazon EC2, and Microsoft Azure. The insights from these experimental evaluations, and other public/published data, were combined with the modeling technology to predict the resource requirements in terms of cost, application performance, and limitations of a realistic application for different deployment scenarios.

Service-Oriented Performance Modeling the MULE Enterprise Service Bus (ESB) Loan Broker Application

Since 2006 NICTA has been developing and trialing Service-Oriented Performance Modeling (SOPM), a method and tool support for performance modeling of large-scale heterogeneous Service Oriented Architectures (SOAs). This technology enables software architects to rapidly build performance models of SOAs directly in terms of service compositions. Enterprise Service Buses (ESBs) are an increasingly common style of SOA infrastructure and implementation technology that we have encountered and modeled in e-Government SOA projects. In this paper we show the application of our SOPM approach to the MULE Enterprise Service Bus Loan Broker application in a laboratory context. We give a high-level outline of the SOPM method, and introduce the MULE ESB and Loan Broker application. We describe how a SOPM of the Loan Broker application is built in terms of application business-logic services and MULE infrastructure service components, and parameterized with measurements from an experimental test-bed. We demonstrate the validity of the approach in an initial scenario, and then explore the modeling of alternative deployment and application scenarios.

Performance modelling power consumption and carbon emissions for Server Virtualization of Service Oriented Architectures (SOAs)

Server Virtualization is driven by the goal of reducing the total number of physical servers in an organisation by consolidating multiple applications on shared servers. Expected benefits include more efficient server utilisation, and a decrease in green house gas emissions. However, Service Oriented Architectures combined with Server Virtualization may significantly increase risks such as saturation and Service Level Agreement (SLA) violations.

Modeling cloud cost and performance

Architecting applications for the Cloud is challenging due to significant differences between traditional hosting and Cloud infrastructure setup, unknown and unproven Cloud performance and scalability characteristics, and unpredictable availability as well as variable quota limitations. Clouds claim to offer benefits in terms of cost and elasticity under some situations, but may introduce significant risks in areas such as latency and security which need to be carefully managed. Building workable cloud applications therefore requires in-depth insight into the architectural and performance characteristics of each cloud offering, and the ability to reason about tradeoffs and alternatives of application designs and deployments. NICTA has developed a Service Oriented Performance Modeling technology for modeling the performance and scalability of Service Oriented applications architected for a variety of platforms. Using a suite of cloud testing application we conducted in-depth empirical evaluations of a variety of real cloud infrastructures, including Google App Engine, Amazon EC2, and Microsoft Azure. The insights from these experimental evaluations, and other public/published data, were combined with the modeling technology to predict the resource requirements in terms of cost, application performance, and limitations of a realistic application for different deployment scenarios. We conclude with some architectural lessons learnt.

Real-world performance modelling of enterprise service oriented architectures: delivering business value with complexity and constraints

Performance and Scalability Modelling of real-world enterprise systems is challenging due to both the complexity and size of the system being modelled, and constraints imposed by real projects such as the need to provide business value, deadlines, and the accessibility, relevance, quality and quantity of available documentation and performance data. Our hypothesis is that enterprise Service Oriented Architectures (SOAs) are more amenable to performance modelling as services are more granular, visible, and measurable. Since 2007 we have developed, trialled and refined a method with modeldriven tool support for directly modelling the performance and scalability of increasingly complex Service Oriented Architectures. This paper reports an illustrative experience modelling a large-scale production SOA Enterprise Service Bus (ESB) upgrade, focussing on lessons learnt related to the complexity and constraints of modelling in the real-world. The key observations are that model construction is a type of theory formation and therefore: (1) Models (functioning as theories) can be simple but powerful enough to model large complex SOAs within the boundaries of real project constraints; (2) Model formation can be incremental, starting with a simple model (as simple theories are easier to refute) and refining as required; (3) Building multiple competing models can be a useful approach if information is inadequate or ambiguous, as the rival models can be tested with the aim of discarding incorrect ones; (4) If insufficient information is available to build a single über model to answer all the performance questions, it is often possible to build multiple specialised models for different purposes.

Performance modeling evolving Enterprise Service Oriented Architectures

NICTA has developed a Service Oriented Performance Modeling technology, and for several years has conducted field-trials of the technology in collaboration with government and non-government projects. The technology enables rapid modeling and performance prediction of large-scale heterogeneous Service Oriented Architectures (SOAs), using a methodology and tool support to model and simulate complex service compositions. Evolution in Enterprise Service Oriented Architectures (ESOAs) presents significant challenges to managing risks associated with performance and scalability. Four of the field-trials involved the construction and use of models to specifically address performance risks related to the evolution of service architectures, resulting from, and to explore changes in: architecture, services, deployment, resources, and loads. This paper first presents some unifying observations on the architectural commonalities of evolving enterprise services, discusses the performance and scalability challenges, and describes some common approaches to addressing them. We then demonstrate Service Oriented Performance Modeling applied to an illustrative example of enterprise service evolution. We conclude with a presentation of four case-studies which summarize our experiences with performance modeling evolving service architectures.

Automatic Performance Modelling from Application Performance Management (APM) Data: An Experience Report

Traditional testing approaches for enterprise systems are no longer possible, agile enough, affordable, or accurate in many cases. This is due to ongoing changes, reduced time between production updates and the inability to test all system components because of third party services and the expense of maintaining a test environment. One alternative approach has been to manually build predictive performance models to mitigate performance risk. Even this has become impractical and cannot keep pace with changes in complex enterprise systems. In response to these challenges we have developed a way to automatically build and parameterize performance models for large scale enterprise systems from Application Performance Management (APM) data. This industry experience report summaries our experiences with automatically building performance models for commercial customers over the last two years. For each project we summarize the problem context, the performance risks to be addressed, the automatic modelling process, the range in complexity of the resulting models, the accuracy of the predictions, and the benefits and limitations of the models in practice.