Ganesha Shanmuganathan

Pesto: online storage performance management in virtualized datacenters

Virtualized datacenters strive to reduce costs through workload consolidation. Workloads exhibit a diverse set of IO behaviors and varying IO load that makes it difficult to estimate the IO performance on shared storage. As a result, system administrators often resort to gross overprovisioning or static partitioning of storage to meet application demands. In this paper, we introduce Pesto, a unified storage performance management system for heterogeneous virtualized datacenters. Pesto is the first system that completely automates storage performance management for virtualized datacenters, providing IO load balancing with cost-benefit analysis, per-device congestion management, and initial placement of new workloads. At its core, Pesto constructs and adapts approximate black-box performance models of storage devices automatically, leveraging our analysis linking device throughput and latency to outstanding IOs.Experimental results for a wide range of devices and configurations validate the accuracy of these models. We implemented Pesto in a commercial product and tested its performance on tens of devices, running hundreds of test cases over the past year. End-to-end experiments demonstrate that Pesto is efficient, adapts to changes quickly and can improve workload performance by up to 19%, achieving our objective of lowering storage management costs through automation.

Defragmenting the cloud using demand-based resource allocation

Current public cloud offerings sell capacity in the form of pre-defined virtual machine (VM) configurations to their tenants. Typically this means that tenants must purchase individual VM configurations based on the peak demands of the applications, or be restricted to only scale-out applications that can share a pool of VMs. This diminishes the value proposition of moving to a public cloud as compared to server consolidation in a private virtualized datacenter, where one gets the benefits of statistical multiplexing between VMs belonging to the same or different applications. Ideally one would like to enable a cloud tenant to buy capacity in bulk and benefit from statistical multiplexing among its workloads. This requires the purchased capacity to be dynamically and transparently allocated among the tenants VMs that may be running on different servers, even across datacenters. In this paper, we propose two novel algorithms called BPX and DBS that are able to provide the cloud customer with the abstraction of buying bulk capacity. These algorithms dynamically allocate the bulk capacity purchased by a customer between its VMs based on their individual demands and user-set importance. Our algorithms are highly scalable and are designed to work in a large-scale distributed environment. We implemented a prototype of BPX as part of VMwares management software and showed that BPX is able to closely mimic the behavior of a centralized allocator in a distributed manner.