Cristian Klein

Brownout: building more robust cloud applications

Self-adaptation is a first class concern for cloud applications, which should be able to withstand diverse runtime changes. Variations are simultaneously happening both at the cloud infrastructure level - for example hardware failures - and at the user workload level - flash crowds. However, robustly withstanding extreme variability, requires costly hardware over-provisioning. In this paper, we introduce a self-adaptation programming paradigm called brownout. Using this paradigm, applications can be designed to robustly withstand unpredictable runtime variations, without over-provisioning. The paradigm is based on optional code that can be dynamically deactivated through decisions based on control theory. We modified two popular web application prototypes - RUBiS and RUBBoS - with less than 170 lines of code, to make them brownout-compliant. Experiments show that brownout self-adaptation dramatically improves the ability to withstand flash-crowds and hardware failures.

Software engineering meets control theory

The software engineering community has proposed numerous approaches for making software self-adaptive. These approaches take inspiration from machine learning and control theory, constructing software that monitors and modifies its own behavior to meet goals. Control theory, in particular, has received considerable attention as it represents a general methodology for creating adaptive systems. Control-theoretical software implementations, however, tend to be ad hoc. While such solutions often work in practice, it is difficult to understand and reason about the desired properties and behavior of the resulting adaptive software and its controller. This paper discusses a control design process for software systems which enables automatic analysis and synthesis of a controller that is guaranteed to have the desired properties and behavior. The paper documents the process and illustrates its use in an example that walks through all necessary steps for self-adaptive controller synthesis.

Towards Faster Response Time Models for Vertical Elasticity

Resource provisioning in cloud computing is typically coarse-grained. For example, entire CPU cores may be allocated for periods of up to an hour. The Resource-as-a-Service cloud concept has been introduced to improve the efficiency of resource utilization in clouds. In this concept, resources are allocated in terms of CPU core fractions, with granularities of seconds. Such infrastructures could be created using existing technologies such as lightweight virtualization using LXC or by exploiting the Xen hyper visors capacity for vertical elasticity. However, performance models for determining how much capacity to allocate to each application are currently lacking. To address this deficit, we evaluate two performance models for predicting mean response times: the previously proposed queue length model and the novel inverse model. The models are evaluated using 3 applications under both open and closed system models. The inverse model reacted rapidly and remained stable even with targets as low as 0.5 seconds.

Improving Cloud Service Resilience Using Brownout-Aware Load-Balancing

We focus on improving resilience of cloud services (e.g., e-commerce website), when correlated or cascading failures lead to computing capacity shortage. We study how to extend the classical cloud service architecture composed of a load-balancer and replicas with a recently proposed self-adaptive paradigm called brownout. Such services are able to reduce their capacity requirements by degrading user experience (e.g., disabling recommendations). Combining resilience with the brownout paradigm is to date an open practical problem. The issue is to ensure that replica self-adaptivity would not confuse the load-balancing algorithm, overloading replicas that are already struggling with capacity shortage. For example, load-balancing strategies based on response times are not able to decide which replicas should be selected, since the response times are already controlled by the brownout paradigm. In this paper we propose two novel brownout-aware load-balancing algorithms. To test their practical applicability, we extended the popular lighttpd web server and load-balancer, thus obtaining a production-ready implementation. Experimental evaluation shows that the approach enables cloud services to remain responsive despite cascading failures. Moreover, when compared to Shortest Queue First (SQF), believed to be near-optimal in the non-adaptive case, our algorithms improve user experience by 5%, with high statistical significance, while preserving response time predictability.

Coordinating CPU and Memory Elasticity Controllers to Meet Service Response Time Constraints

Vertical elasticity is recognized as a key enabler for efficient resource utilization of cloud infrastructure through fine-grained resource provisioning, e.g., allowing CPU cycles to be leased for as short as a few seconds. However, little research has been done to support vertical elasticity where the focus is mostly on a single resource, either CPU or memory, while an application may need arbitrary combinations of these resources at different stages of its execution. Nonetheless, the existing techniques cannot be readily used as-is without proper orchestration since they may lead to either under-or over-provisioning of resources and consequently result in undesirable behaviors such as performance disparity. The contribution of this paper is the design of an autonomic resource controller using a fuzzy control approach as a coordination technique. The novel controller dynamically adjusts the right amount of CPU and memory required to meet the performance objective of an application, namely its response time. We perform a thorough experimental evaluation using three different interactive benchmark applications, RUBiS, RUBBoS, and Olio, under workload traces generated based on open and closed system models. The results show that the coordination of memory and CPU elasticity controllers using the proposed fuzzy control provisions the right amount of resources to meet the response time target without over-committing any of the resource types. In contrast, with no coordinating between controllers, the behaviour of the system is unpredictable e.g., the application performance may be met but at the expense of over-provisioning of one of the resources, or application crashing due to severe resource shortage as a result of conflicting decisions.

The Straw that Broke the Camel's Back: Safe Cloud Overbooking with Application Brownout

Resource overbooking is an admission control technique to increase utilization in cloud environments. However, due to uncertainty about future application workloads, overbooking may result in overload situations and deteriorated performance. We mitigate this using brownout, a feedback approach to application performance steering, that ensures graceful degradation during load spikes and thus avoids overload. Additionally, brownout management information is included into the overbooking system, enabling the development of improved reactive methods to overload situations. Our combined brownout-overbooking approach is evaluated based on real-life interactive workloads and non-interactive batch applications. The results show that our approach achieves an improvement of resource utilization of 11 to 37 percentage points, while keeping response times lower than the set target of 1 second, with negligible application degradation.

Control-theoretical load-balancing for cloud applications with brownout

Cloud applications are often subject to unexpected events like flash crowds and hardware failures. Without a predictable behaviour, users may abandon an unresponsive application. This problem has been partially solved on two separate fronts: first, by adding a self-adaptive feature called brownout inside cloud applications to bound response times by modulating user experience, and, second, by introducing replicas - copies of the applications having the same function-alities - for redundancy and adding a load-balancer to direct incoming traffic.

Control strategies for predictable brownouts in cloud computing

Cloud computing is an application hosting model providing the illusion of infinite computing power. However, even the largest datacenters have finite computing capacity, thus cloud infrastructures have experienced overload due to overbooking or transient failures. The topic of this paper is the comparison of different control strategies to mitigate overload for datacenters, that assume that the running cloud applications are cooperative and help the infrastructure in recovering from critical events. Specifically, the paper investigates the behavior of different controllers when they have to keep the average response time of a cloud application below a certain threshold by acting on the probability of serving requests with optional computations disabled, where the pressure exerted by each request on the infrastructure is diminished, at the expense of user experience. (Less)

How Beneficial Are Intermediate Layer Data Centers in Mobile Edge Networks

To reduce the congestion due to the future bandwidth-hungry applications in domains such as Health care, Internet of Things (IoT), etc., we study the benefit of introducing additional Data Centers (DCs) closer to the network edge for the optimal application placement. Our study shows that the edge layer DCs in a Mobile Edge Network (MEN) infrastructure is cost beneficial for the bandwidth-hungry applications having their strong demand locality and in the scenarios where large capacity is deployed at the edge layer DCs. The cost savings for such applications can go up to 67%. Additional intermediate layer DCs close to the root DC can be marginally cost beneficial for the compute intensive applications with medium or low demand locality. Hence, a Telecom Network Operator should start building an edge DC first having capacity up to hundreds of servers at the network edge to cater the emerging bandwidth-hungry applications and to minimize its operational cost.

KPI-agnostic Control for Fine-Grained Vertical Elasticity

Applications hosted in the cloud have become indispensable in several contexts, with their performance often being key to business operation and their running costs needing to be minimized. To minimize running costs, most modern virtualization technologies such as Linux Containers, Xen, and KVM offer powerful resource control primitives for individual provisioning – that enable adding or removing of fraction of cores and/or megabytes of memory for as short as few seconds. Despite the technology being ready, there is a lack of proper techniques for fine-grained resource allocation, because there is an inherent challenge in determining the correct composition of resources an application needs, with varying workload, to ensure deterministic performance. This paper presents a control-based approach for the management of multiple resources, accounting for the resource consumption, together with the application performance, enabling fine-grained vertical elasticity. The control strategy ensures that the application meets the target performance indicators, consuming as less resources as possible. We carried out an extensive set of experiments using different applications – interactive with response-time requirements, as well as noninteractive with throughput desires – by varying the workload mixes of each application over time. The results demonstrate that our solution precisely provides guaranteed performance while at the same time avoiding both resource over-and underprovisioning.