Jack Li

Micro-Blog: sharing and querying content through mobile phones and social participation

Recent years have witnessed the impacts of distributed content sharing (Wikipedia, Blogger), social networks (Facebook, MySpace), sensor networks, and pervasive computing. We believe that significant more impact is latent in the convergence of these ideas on the mobile phone platform. Phones can be envisioned as people-centric sensors capable of aggregating participatory as well as sensory inputs from local surroundings. The inputs can be visualized in different dimensions, such as space and time. When plugged into the Internet, the collaborative inputs from phones may enable a high resolution view of the world. This paper presents the architecture and implementation of one such system, called Micro-Blog. New kinds of application-driven challenges are identified and addressed in the context of this system. Implemented on Nokia N95 mobile phones, Micro-Blog was distributed to volunteers for real life use. Promising feedback suggests that Micro-Blog can be a deployable tool for sharing, browsing, and querying global information.

Automated control for elastic n-tier workloads based on empirical modeling

Elastic n-tier applications have non-stationary workloads that require adaptive control of resources allocated to them. This presents not only an opportunity in pay-as-you-use clouds, but also a challenge to dynamically allocate virtual machines appropriately. Previous approaches based on control theory, queuing networks, and machine learning work well for some situations, but each model has its own limitations due to inaccuracies in performance prediction. In this paper we propose a multi-model controller, which integrates adaptation decisions from several models, choosing the best. The focus of our work is an empirical model, based on detailed measurement data from previous application runs. The main advantage of the empirical model is that it returns high quality performance predictions based on measured data. For new application scenarios, we use other models or heuristics as a starting point, and all performance data are continuously incorporated into the empirical models knowledge base. Using a prototype implementation of the multi-model controller, a cloud testbed, and an n-tier benchmark (RUBBoS), we evaluated and validated the advantages of the empirical model. For example, measured data show that it is more effective to add two nodes as a group, one for each tier, when two tiers approach saturation simultaneously.

Variations in Performance and Scalability When Migrating n-Tier Applications to Different Clouds

The increasing popularity of computing clouds continues to drive both industry and research to provide answers to a large variety of new and challenging questions. We aim to answer some of these questions by evaluating performance and scalability when an n-tier application is migrated from a traditional datacenter environment to an IaaS cloud. We used a representative n-tier macro-benchmark (RUBBoS) and compared its performance and scalability in three different test beds: Amazon EC2, Open Cirrus (an open scientific research cloud), and Emulab (academic research test bed). Interestingly, we found that the best-performing configuration in Emulab can become the worst-performing configuration in EC2. Subsequently, we identified the bottleneck components, high context switch overhead and network driver processing overhead, to be at the system level. These overhead problems were confirmed at a finer granularity through micro-benchmark experiments that measure component performance directly. We describe concrete alternative approaches as practical solutions for resolving these problems.

Performance Overhead among Three Hypervisors: An Experimental Study Using Hadoop Benchmarks

Hyper visors are widely used in cloud environments and their impact on application performance has been a topic of significant research and practical interest. We conducted experimental measurements of several benchmarks using Hadoop MapReduce to evaluate and compare the performance impact of three popular hyper visors: a commercial hyper visor, Xen, and KVM. We found that differences in the workload type (CPU or I/O intensive), workload size and VM placement yielded significant performance differences among the hyper visors. In our study, we used the three hyper visors to run several MapReduce benchmarks, such as Word Count, TestDSFIO, and TeraSort and further validated our observed hypothesis using micro benchmarks. In our observation for CPU-bound benchmark, the performance difference between the three hyper visors was negligible, however, significant performance variations were seen for I/O-bound benchmarks. Moreover, adding more virtual machines on the same physical host degraded the performance on all three hyper visors, yet we observed different degradation trends amongst them. Concretely, the commercial hyper visor is 46% faster at TestDFSIO Write than KVM, but 49% slower in the TeraSort benchmark. In addition, increasing the workload size for TeraSort yielded completion times for CVM that were two times that of Xen and KVM. The performance differences shown between the hyper visors suggests that further analysis and consideration of hyper visors are needed in the future when deploying applications to cloud environments.

Expertus: A Generator Approach to Automate Performance Testing in IaaS Clouds

Cloud computing is an emerging technology paradigm that revolutionizes the computing landscape by providing on-demand delivery of software, platform, and infrastructure over the Internet. Yet, architecting, deploying, and configuring enterprise applications to run well on modern clouds remains a challenge due to associated complexities and non-trivial implications. The natural and presumably unbiased approach to these questions is thorough testing before moving applications to production settings. However, thorough testing of enterprise applications on modern clouds is cumbersome and error-prone due to a large number of relevant scenarios and difficulties in testing process. We address some of these challenges through Expertus---a flexible code generation framework for automated performance testing of distributed applications in Infrastructure as a Service (IaaS) clouds. Expertus uses a multi-pass compiler approach and leverages template-driven code generation to modularly incorporate different software applications on IaaS clouds. Expertus automatically handles complex configuration dependencies of software applications and significantly reduces human errors associated with manual approaches for software configuration and testing. To date, Expertus has been used to study three distributed applications on five IaaS clouds with over 10,000 different hardware, software, and virtualization configurations. The flexibility and extensibility of Expertus and our own experience on using it shows that new clouds, applications, and software packages can easily be incorporated.

Detecting Transient Bottlenecks in n-Tier Applications through Fine-Grained Analysis

Identifying the location of performance bottlenecks is a non-trivial challenge when scaling n-tier applications in computing clouds. Specifically, we observed that an n-tier application may experience significant performance loss when there are transient bottlenecks in component servers. Such transient bottlenecks arise frequently at high resource utilization and often result from transient events (e.g., JVM garbage collection) in an n-tier system and bursty workloads. Because of their short lifespan (e.g., milliseconds), these transient bottlenecks are difficult to detect using current system monitoring tools with sampling at intervals of seconds or minutes. We describe a novel transient bottleneck detection method that correlates throughput (i.e., request service rate) and load (i.e., number of concurrent requests) of each server in an n-tier system at fine time granularity. Both throughput and load can be measured through passive network tracing at millisecond-level time granularity. Using correlation analysis, we can identify the transient bottlenecks at time granularities as short as 50ms. We validate our method experimentally through two case studies on transient bottlenecks caused by factors at the system software layer (e.g., JVM garbage collection) and architecture layer (e.g., Intel SpeedStep).

Variations in Performance and Scalability: An Experimental Study in IaaS Clouds Using Multi-Tier Workloads

The increasing popularity of clouds drives researchers to find answers to a large variety of new and challenging questions. Through extensive experimental measurements, we show variance in performance and scalability of clouds for two non-trivial scenarios. In the first scenario, we target the public Infrastructure as a Service (IaaS) clouds, and study the case when a multi-tier application is migrated from a traditional datacenter to one of the three IaaS clouds. To validate our findings in the first scenario, we conduct similar study with three private clouds built using three mainstream hypervisors. We used the RUBBoS benchmark application and compared its performance and scalability when hosted in Amazon EC2, Open Cirrus, and Emulab. Our results show that a best-performing configuration in one cloud can become the worst-performing configuration in another cloud. Subsequently, we identified several system level bottlenecks such as high context switching and network driver processing overheads that degraded the performance. We experimentally evaluate concrete alternative approaches as practical solutions to address these problems. We then built the three private clouds using a commercial hypervisor (CVM), Xen, and KVM respectively and evaluated performance characteristics using both RUBBoS and Cloudstone benchmark applications. The three clouds show significant performance variations; for instance, Xen outperforms CVM by 75 percent on the read-write RUBBoS workload and CVM outperforms Xen by over 10 percent on the Cloudstone workload. These observed problems were confirmed at a finer granularity through micro-benchmark experiments that measure component performance directly.

Impact of DVFS on n-tier application performance

Dynamic Voltage and Frequency Scaling (DVFS) has been widely deployed and proven to reduce energy consumption at low CPU utilization levels; however, our measurements of the n-tier application benchmark (RUBBoS) performance showed significant performance degradation at high utilization levels, with response time several times higher and throughput loss of up to 20%, when DVFS is turned on. Using a combination of benchmark measurements and simulation, we found two kinds of problems: large response time fluctuations due to push-back wave queuing in n-tier systems and throughput loss due to rapidly alternating bottlenecks. These problems arise from anti-synchrony between DVFS adjustment period and workload burst cycles (similar cycle length but out of phase). Simulation results (confirmed by extensive measurements) show the anti-synchrony happens routinely for a wide range of configurations. We show that a workload-sensitive DVFS adaptive control mechanism can disrupt the anti-synchrony and reduce the performance impact of DVFS at high utilization levels to 25% or less of the original.

Profit-Based Experimental Analysis of IaaS Cloud Performance: Impact of Software Resource Allocation

High resource utilization is an important goal in achieving high return on investment in cloud environments. Guaranteed quality of service (QoS) is an important goal for web-facing applications such as e-commerce. Achieving both high utilization and high QoS simultaneously is a significant challenge, since high utilization often implies more QoS failures such as long response times. In this paper, we adopt a profit model based on response time (i.e., decreasing or negative revenues for increasing query answer response time) to represent the QoS requirements. Our data shows that such a profit model often yields different analytical results compared to traditional performance metrics such as average throughput. Using extensive experimental measurements (on the same hardware platform and software stack) of the standard RUBBoS n-tier benchmark, we study the impact of different allocations of software resources such as the size of thread pools in various servers in an n-tier system. First, the profit model emphasizes the importance of appropriate allocations, showing a difference of up to 48.6% when system utilization is high (over 80%). Second, our experiments show that over-allocation of thread pool may lead to unnecessary consumption of critical resources (e.g., CPU) that reduce profits by up to 84.8%. Third, we found that under-allocation of thread pool in one server may lead to under-utilization of several servers downstream in an n-tier system, also reducing profits by up to 52.8%. Our data shows that the best allocation depends on several system parameters, including resource availability. We designed an adaptive algorithm to find the best allocations and show its effectiveness through our experiments and analyses.

Revisiting Performance Interference among Consolidated n-Tier Applications: Sharing is Better Than Isolation

Performance unpredictability is one of the major concerns slowing down the migration of mission-critical applications into cloud computing infrastructures. An example of non-intuitive result is the measured n-tier application performance in a virtualized environment that showed increasing workload caused a competing, co-located constant workload to decrease its response time. In this paper, we investigate the sensitivity of measured performance in relation to two factors: (1) consolidated server specification of virtual machine resource availability, and (2) burstiness of n-tier application workload. Our first and surprising finding is that specifying a complete isolation, e.g., 50-50 even split of CPU between two co-located virtual machines (VMs) results in significantly lower performance compared to a fully-shared allocation, e.g., up to 100% CPU for both co-located VMs. This happens even at relatively modest resource utilization levels (e.g., 40% CPU in the VMs). Second, we found that an increasingly bursty workload also increases the performance loss among the consolidated servers, even at similarly modest utilization levels (e.g., 70% overall). A potential solution to the first problem (performance loss due to resource allocation) is cross-tier-priority scheduling (giving higher priority to shorter jobs), which can reduce the performance loss by a factor of two in our experiments. In contrast, bursty workloads are a more difficult problem: our measurements show they affect both the isolation and sharing strategies in virtual machine resource allocation.