Weikun Wang

Quality-of-service in cloud computing: modeling techniques and their applications

Recent years have seen the massive migration of enterprise applications to the cloud. One of the challenges posed by cloud applications is Quality-of-Service (QoS) management, which is the problem of allocating resources to the application to guarantee a service level along dimensions such as performance, availability and reliability. This paper aims at supporting research in this area by providing a survey of the state of the art of QoS modeling approaches suitable for cloud systems. We also review and classify their early application to some decision-making problems arising in cloud QoS management.

Evaluating Weighted Round Robin Load Balancing for Cloud Web Services

Weighted round robin load balancing is a common routing policy offered in cloud load balancers. However, there is a lack of effective mechanisms to decide the weights assigned to each server to achieve an overall optimal revenue of the system. In this paper, we first experimentally explore the relation between probabilistic routing and weighted round robin load balancing policies. From the experiment a similar behavior is found between these two policies, which makes it possible to assign the weights according to the routing probability estimated from queueing theoretic heuristic and optimization algorithms studied in the literature. We focus in particular on algorithms based on closed queueing networks for multi-class workloads, which can be used to describe application with service level agreements differentiated across users. We also compare the efficiency of queueing theoretic methods with simple heuristics that do not require to specify a stochastic model of the application. Results indicate that queueing theoretical algorithms yield significantly better results than routings proportional to the VM capacity with respect to throughput maximization.

A Bayesian Approach to Parameter Inference in Queueing Networks

The application of queueing network models to real-world applications often involves the task of estimating the service demand placed by requests at queueing nodes. In this article, we propose a methodology to estimate service demands in closed multiclass queueing networks based on Gibbs sampling. Our methodology requires measurements of the number of jobs at resources and can accept prior probabilities on the demands. Gibbs sampling is challenging to apply to estimation problems for queueing networks since it requires one to efficiently evaluate a likelihood function on the measured data. This likelihood function depends on the equilibrium solution of the network, which is difficult to compute in closed models due to the presence of the normalizing constant of the equilibrium state probabilities. To tackle this obstacle, we define a novel iterative approximation of the normalizing constant and show the improved accuracy of this approach, compared to existing methods, for use in conjunction with Gibbs sampling. We also demonstrate that, as a demand estimation tool, Gibbs sampling outperforms other popular Markov Chain Monte Carlo approximations. Experimental validation based on traces from a cloud application demonstrates the effectiveness of Gibbs sampling for service demand estimation in real-world studies.

Towards a DevOps Approach for Software Quality Engineering

DevOps is a novel trend in software engineering that aims at bridging the gap between development and operations, putting in particular the developer in greater control of deployment and application runtime. Here we consider the problem of designing a tool capable of providing feedback to the developer on the performance, reliability, and in general quality characteristics of the application at runtime. This raises a number of questions related to what measurement information should be carried back from runtime to design-time and what degrees of freedom should be provided to the developer in the evaluation of performance data. To answer these questions, we describe the design of a filling-the-gap (FG) tool, a software system capable of automatically analyzing performance data either directly or through statistical inference. A natural application of the FG tool is the continuous training of stochastic performance models, such as layered queueing networks, that can inform developers on how to refactor the software architecture.

Maximum Likelihood Estimation of Closed Queueing Network Demands from Queue Length Data

We propose maximum likelihood (ML) estimators for service demands in closed queueing networks with load-independent and load-dependent stations. Our ML estimators are expressed in implicit form and require only to compute mean queue lengths and marginal queue length probabilities from an empirical dataset. Further, in the load-independent case, we provide an explicit approximate formula for the ML estimator together with confidence intervals.

Filling the gap: a tool to automate parameter estimation for software performance models

Software performance engineering heavily relies on application and resource models that enable the prediction of Quality-of-Service metrics. Critical to these models is the accuracy of their parameters, the value of which can change with the application and the resources where it is deployed. In this paper we introduce the Filling-the-gap (FG) tool, which automates the parameter estimation of application performance models. This tool implements a set of statistical routines to estimate the parameters of performance models, which are automatically executed using monitoring information kept in a local database.

QD-AMVA

Workload measurements in enterprise systems often lead to observe a dependence between the number of requests running at a resource and their mean service requirements. However, multiclass performance models that feature these dependences are challenging to analyze, a fact that discourages practitioners from characterizing workload dependences. We here focus on closed multiclass queueing networks and introduce QD-AMVA, the first approximate mean-value analysis (AMVA) algorithm that can efficiently and robustly analyze queue-dependent service times in a multiclass setting. A key feature of QD-AMVA is that it operates on mean values, avoiding the computation of state probabilities. This property is an innovative result for state-dependent models, which increases the computational efficiency and numerical robustness of their evaluation. Extensive validation on random examples, a cloud load-balancing case study and comparison with a fluid method and an existing AMVA approximation prove that QD-AMVA is efficient, robust and easy to apply, thus enhancing the tractability of queue-dependent models.

QMLE: A Methodology for Statistical Inference of Service Demands from Queueing Data

Estimating the demands placed by services on physical resources is an essential step for the definition of performance models. For example, scalability analysis relies on these parameters to predict queueing delays under increasing loads. In this article, we investigate maximum likelihood (ML) estimators for demands at load-independent and load-dependent resources in systems with parallelism constraints. We define a likelihood function based on state measurements and derive necessary conditions for its maximization. We then obtain novel estimators that accurately and inexpensively obtain service demands using only aggregate state data. With our approach, and also thanks to approximation methods for computing marginal and joint distributions for the load-dependent case, confidence intervals can be rigorously derived, explicitly taking into account both topology and concurrency levels of the services. Our estimators and their confidence intervals are validated against simulations and real system measurements for two multi-tier applications, showing high accuracy also in models with load-dependent resources.

Load Balancing for Multi-cloud

Load balancing is an integral part of software systems that require to serve requests with multiple concurrent computing resources such as servers, clusters, network links, central processing units or disk drives.

Closing the Loop Between Ops and Dev

DevOps [1] is a recent trend in software engineering that bridges the gap between software development and operations, putting the developer in greater control of the operational environment in which the application runs. To support Quality-of-Service (QoS) analysis, the developer may rely on software performance models. However, to provide reliable estimates, the input parameters must be continuously updated and accurately estimated. Accurate estimation is challenging because some parameters are not explicitly tracked by log files requiring deep monitoring instrumentation that poses large overheads, unacceptable in production environments.