Xingjian Lu

BURSE: A Bursty and Self-Similar Workload Generator for Cloud Computing

As two of the most important characteristics of workloads, burstiness and self-similarity are gaining more and more attention. Workload generation, which is a key technique for performance analysis and simulations, has also attracted an increasing interest in cloud community in recent years. Though a large number of methods for synthetically generating bursty or self-similar workloads have been proposed in the literature, none of them can deal with workload generation with both of the two characteristics. In this paper, a configurable and intelligible synthetic generator (BURSE) is proposed for bursty and self-similar workloads in cloud computing based on a superposition of two-state Markov Modulated Poisson Processes (MMPP2s). The proposed generator can produce workloads with both specified intension of burstiness and self-similarity. Detailed experimental evaluation demonstrates the accuracy, robustness and good applicability of BURSE.

JTangCSB: A Cloud Service Bus for Cloud and Enterprise Application Integration

Cloud and enterprise application integration (CEAI) is a new challenge as more enterprise applications migrate successfully to cloud environments. However, achieving CEAI properties such as multitenancy, cloud-level scalability, and environmental heterogeneity is nontrivial for enterprise applications originally built under the assumption of centralized management. A concrete implementation of the cloud service bus approach, called JTangCSB, addresses the challenge of handling these properties and demonstrates its feasibility in practice.

System resource utilization analysis and prediction for cloud based applications under bursty workloads

Abstract Performance analysis and prediction need a solid understanding of the system workload. As a salient workload characteristic, burstiness has critical impact on resource provisioning and performance of cloud based applications. Thus performance analysis and prediction under bursty workloads are of crucial importance to cloud based applications. However, it is yet challenging for such analysis and prediction, since no accurate and effective bursty workload generator exists, as well as the fine-grained bursty workload analysis and prediction method. In this article, to deal with these challenges, a bursty workload generator has been proposed for Cloudstone (a cloud benchmark) based on 2-state Markovian Arrival Process (MAP2). Then based on this generator, a fine-grained performance analysis method, which can be used to predict the probability density function of CPU utilization, has been suggested for cloud based applications, to support better resource provisioning decision making and system performance optimization. Finally, extensive experiments are conducted in a Xen-based virtualized environment to evaluate the accuracy and effectiveness of the two methods. By comparing the actual value of Indices of Dispersion for Count with the target value deduced from MAP2 model, the experiments show the precision of our method is superior to existing works. By comparing the real and predicted system resource utilization under a variety of bursty workloads generated by the proposed generator, the experiments also demonstrate the effectiveness and accuracy of the proposed fine-grained system resource utilization prediction method.

An Approach for Bursty and Self-similar Workload Generation

As two of the most important characteristics of Web systems’ workloads, burstiness and self-similarity are gaining more and more attentions. And synthetically generating bursty and self-similar workloads is a key technique for Web system performance analysis. In this paper, a configurable synthetic approach for bursty and self-similar workload generation has been proposed based on a superposition of 2-state Markovian arrival processes (MAP2). This method can generate workload with both specified intension of burstiness and self-similarity. The detailed evaluation show the accuracy and robustness of our method.

Geographical Job Scheduling in Data Centers with Heterogeneous Demands and Servers

The fast proliferation of cloud computing promotes the rapid development of large-scale commercial data centers. Tens or even hundreds of geographically distributed data centers have been deployed for better reliability and quality of services. This brings huge energy consumption for data centers. Previous research has proved that the geographical load balancing technique can achieve significant energy cost savings for geographically distributed data centers. However, existing methods for geographical load balancing often assume data centers with homogeneous servers, and workloads with single-dimension or uniform resource demands. This is an over-simplification in reality, especially when modern data centers are typically constructed from a variety of server classes. In this paper, we systematically study the problem of job scheduling for geographically distributed data centers to embrace the heterogeneity of underlying platforms and workloads. We develop a novel distributed algorithm to solve the problem efficiently based on the alternating direction method of multipliers. Extensive evaluations based on real-life data center topology, traffic traces, and electricity price data show high efficiency and efficacy of our method.

Distributed Optimal Datacenter Bandwidth Allocation for Dynamic Adaptive Video Streaming

Video streaming systems such as YouTube and Netflix are usually supported by the content delivery networks and datacenters that can consume many megawatts of power. Most existing works independently study the issues of improving quality of experience (QoE) for viewers and reducing the cost and emissions associated with the enormous energy usage of datacenters. By contrast, this paper addresses them both, and jointly optimizes the QoE, the energy cost and emissions by intelligently allocating datacenter bandwidth among different client groups. Specially, we propose a distributed algorithm for achieving the optimal bandwidth allocation. The algorithm novelly decomposes the optimization process into separate ones, which are solved iteratively across datacenters and clients. We demonstrate its convergence by both theoretical proof and experimental validation. The experimental results show that the proposed algorithm converges very fast and achieves much better QoE-cost balance than existing approaches.

Distance-aware virtual cluster performance optimization: A hadoop case study

Cloud computing and big data are becoming two important developing trends in information technology area. However, data-intensive computing has some challenges to work well on virtual machines in cloud computing for virtualized resource competition and complex network communication. Network becomes one of the most notorious bottlenecks, which highlights strategies to lower communication and transmission cost in virtual cluster. In this paper, we present a novel cluster performance optimization strategy named vClusterOpt. vClusterOpt finds out centralized subgraphs of node graph and choose node with the shortest logical distance as kernel node of the subgraph to reduce inter-machine communication and transmission cost under virtual cluster. To calculate logical distance accurately, we define two kinds of logical distance: Logical Communication Distance(LCD) and Logical Transmission Distance(LTD). VM with the shortest LCD with others is used as the communication kernel node who has the most information communication stress, while VM with the shortest LTD is treated as transmission kernel node who has the most data transmission stress. We choose benchmarks running on Hadoop as the represent of data-intensive computing service to demonstrate effectiveness of our approach. Experiments show that an average of 20% performance improvement can get by our distance-aware virtual cluster optimization strategy.

JTangCMS: An efficient monitoring system for cloud platforms

Abstract Cloud computing has attracted increasing interest in industry and academia. However, due to the constantly expanding scale of cloud platforms, the monitoring of clouds encounters a number of critical challenges related to flexibility, scalability, efficiency, and performance. In this paper, we present JTang Cloud Monitoring System (JTangCMS), an efficient monitoring system for cloud platforms. Our contributions cover the collection, delivery, and processing of monitoring data. For data collection, a flexible and scalable agent is implemented with pluggable monitoring components to collect runtime information from different entities. For data delivery, an efficient and robust data dissemination framework is implemented to transmit the runtime information reliably with high throughput and low latency, based on the Data Distribution Service (DDS). For data processing, a cloud action platform is developed to support cloud management decision-making, based on complex event processing (CEP). Finally, detailed experimental evaluations show the feasibility and efficiency of JTangCMS.

Bulk Savings for Bulk Transfers: Minimizing the Energy-Cost for Geo-Distributed Data Centers

With the fast proliferation of cloud computing, major cloud service providers, e.g., Amazon, Google, Facebook, etc., have been deploying more and more geographically distributed data centers to provide customers with better reliability and quality of services. A basic demand in such a geo-distributed data center system is to transfer bulk volumes of data from one data center to another. Geographic distribution and large delay-tolerance of such inter-data-center bulk data transfers provide cloud service providers opportunities to optimize the operating cost. Most existing studies on inter-data-center bulk data transfers focus on minimizing the network bandwidth cost. However, the energy-cost of the bulk data transfers, which also accounts for a large proportion of operating cost in the data centers, still remains unexplored. This is an important problem, especially in the multi-electricity-market environment, where the electricity price exhibits both spatial and temporal diversities. In this paper, we systematically study the problem of how to route and schedule inter-data-center bulk data transfers to minimize the energy-cost for geo-distributed data centers. We model this problem as a min-cost multi-commodity flow problem and develop an efficient two-stage optimization method to solve it. Extensive evaluations with real-life inter-data-center network and electricity prices show that our method brings significant energy-cost savings over existing bulk data transfer methods.

Distributed Data Center Bandwidth Allocation for Cloud-Based Streaming

Cloud-based video streaming systems such as YouTube and Netflix are usually supported by the content delivery networks and data centers that can consume many megawatts of power. Most existing work independently studies the issues of improving quality of experience (QoE) for viewers and reducing the cost and emissions associated with the enormous energy usage of data centers. By contrast, this paper addresses them both, and jointly optimizes the QoE, the energy cost and emissions by intelligently allocating data center bandwidth among different client groups. Specially, we propose a distributed algorithm to achieve the optimal bandwidth allocation, given the prediction of future workload. The algorithm novelly decomposes the optimization process into separate ones, which are solved iteratively across data centers and clients. Further, the algorithm has robust performance guarantee in terms of the variance of the prediction error. We demonstrate its convergence and robustness by both proofs using theoretical analysis and validation based on trace-driven simulations. The results further show that the proposed algorithm converges very fast and achieves much better QoE-cost balance than existing approaches.