Bahman Javadi

Cost-benefit analysis of Cloud Computing versus desktop grids

Cloud Computing has taken commercial computing by storm. However, adoption of cloud computing platforms and services by the scientific community is in its infancy as the performance and monetary cost-benefits for scientific applications are not perfectly clear. This is especially true for desktop grids (aka volunteer computing) applications. We compare and contrast the performance and monetary cost-benefits of clouds for desktop grid applications, ranging in computational size and storage. We address the following questions: (i) What are the performance tradeoffs in using one platform over the other? (ii) What are the specific resource requirements and monetary costs of creating and deploying applications on each platform? (iii) In light of those monetary and performance cost-benefits, how do these platforms compare? (iv) Can cloud computing platforms be used in combination with desktop grids to improve cost-effectiveness even further? We examine those questions using performance measurements and monetary expenses of real desktop grids and the Amazon elastic compute cloud.

The Failure Trace Archive: Enabling Comparative Analysis of Failures in Diverse Distributed Systems

With the increasing functionality and complexity of distributed systems, resource failures are inevitable. While numerous models and algorithms for dealing with failures exist, the lack of public trace data sets and tools has prevented meaningful comparisons. To facilitate the design, validation, and comparison of fault-tolerant models and algorithms, we have created the Failure Trace Archive (FTA) as an online public repository of availability traces taken from diverse parallel and distributed systems. Our main contributions in this study are the following. First, we describe the design of the archive, in particular the rationale of the standard FTA format, and the design of a toolbox that facilitates automated analysis of trace data sets. Second, applying the toolbox, we present a uniform comparative analysis with statistics and models of failures in nine distributed systems. Third, we show how different interpretations of these data sets can result in different conclusions. This emphasizes the critical need for the public availability of trace data and methods for their analysis.

Discovering Statistical Models of Availability in Large Distributed Systems: An Empirical Study of SETI@home

In the age of cloud, Grid, P2P, and volunteer distributed computing, large-scale systems with tens of thousands of unreliable hosts are increasingly common. Invariably, these systems are composed of heterogeneous hosts whose individual availability often exhibit different statistical properties (for example stationary versus nonstationary behavior) and fit different models (for example exponential, Weibull, or Pareto probability distributions). In this paper, we describe an effective method for discovering subsets of hosts whose availability have similar statistical properties and can be modeled with similar probability distributions. We apply this method with about 230,000 host availability traces obtained from a real Internet-distributed system, namely SETI@home. We find that about 21 percent of hosts exhibit availability, that is, a truly random process, and that these hosts can often be modeled accurately with a few distinct distributions from different families. We show that our models are useful and accurate in the context of a scheduling problem that deals with resource brokering. We believe that these methods and models are critical for the design of stochastic scheduling algorithms across large systems where host availability is uncertain.

Statistical Modeling of Spot Instance Prices in Public Cloud Environments

The surge in demand for utilizing public Cloud resources has introduced many trade-offs between price, performance and recently reliability. Amazons Spot Instances (SIs) create a competitive bidding option for the public Cloud users at lower prices without providing reliability on services. It is generally believed that SIs reduce monetary cost to the Cloud users, however it appears from the literature that their characteristics have not been explored and reported. We believe that characterization of SIs is fundamental in the design of stochastic scheduling algorithms and fault tolerant mechanisms in public Cloud environments for spot market. In this paper, we have done a comprehensive analysis of SIs based on one year price history in four data centers of Amazons EC2. For this purpose, we have analyzed all different types of SIs in terms of spot price and the inter-price time (time between price changes) and determined the time dynamics for spot price in hour-in-day and day-of-week. Moreover, we have proposed a statistical model that fits well these two data series. The results reveal that we are able to model spot price dynamics as well as the inter-price time of each SI by the mixture of Gaussians distribution with three or four components. The proposed model is validated through extensive simulations, which demonstrate that our model exhibits a good degree of accuracy under realistic working conditions.

Editorial: recent advances in communication networks and multimedia technologies

To successfully support multimedia applications, it is necessary to improve the state-of-theart multimedia technologies and multimedia transmissions over communication networks in order to provide quality-of-service (QoS) guarantees between end-users and to achieve a high quality-of-experience for end-users. The accepted papers in this special issue deal with a wide range of important aspects and challenging issues of communication networks and multimedia technologies, and the contents are built on analytical modelling, experimental and simulation studies. The contributions of these papers are outlined below. With the development of wireless access and multimedia compression technologies, much attention has been devoted to wireless video communications. Wan, Xiong and Yang propose an integrated routing metric to evaluate path quality taking the hop count, congestion bottleneck and other parameters of a path into account. They design the traffic assignment and traffic adjustment modules to make the routing scheme flexible to the changing environments, and present an enhanced link layer scheduling algorithm. Simulation results show that compared to commonly used schemes, the proposed network layer scheme always provides better performance with various traffic modes; combined with link layer scheduling algorithm, the performance could be further improved. High speed and quality multimedia and data transmission requirements exist widely in wireless networks. Wang et al. investigate the performance of opportunistic scheduling in wireless multimedia and data networks based on popular stochastic network calculus (SNC). The authors firstly bound the traffic arrival process using the exponentially bounded burstiness traffic model of SNC and establish a stochastic arrival curve, and then propose a new wireless opportunistic scheduling, i.e., modified proportional fairness (MPF) scheduling algorithm, which can provide better fairness for input traffic to guarantee QoS performance, and obtain a stochastic service curve. The numerical results show that the proposed stochastic model and MPF scheduling algorithm can provide better QoS performance. Multimed Tools Appl (2015) 74:1–3 DOI 10.1007/s11042-013-1647-x

Characterizing spot price dynamics in public cloud environments

The surge in demand for utilizing public Cloud resources has introduced many trade-offs between price, performance and recently reliability. Amazons Spot Instances (SIs) create a competitive bidding option for public Cloud users at lower prices without providing reliability on services. It is generally believed that SIs reduce monetary cost to the Cloud users, however it appears from the literature that their characteristics have not been explored and reported. We believe that characterization of SIs is fundamental in the design of stochastic scheduling algorithms and fault tolerant mechanisms in public Cloud environments for the spot market. In this paper, we have done a comprehensive analysis of SIs based on one year price history in four data centers of Amazons EC2. For this purpose, we have analyzed all different types of SIs in terms of spot price and the inter-price time (time between price changes) and determined the time dynamics for spot price in hour-in-day and day-of-week. Moreover, we have proposed a statistical model that fits well these two data series. The results reveal that we are able to model spot price dynamics as well as the inter-price time of each SI by a mixture of Gaussians distribution with three or four components. The proposed model is validated through extensive simulations, which demonstrate that our model exhibits a good degree of accuracy under realistic working conditions.

The Failure Trace Archive: Enabling the comparison of failure measurements and models of distributed systems

With the increasing presence, scale, and complexity of distributed systems, resource failures are becoming an important and practical topic of computer science research. While numerous failure models and failure-aware algorithms exist, their comparison has been hampered by the lack of public failure data sets and data processing tools. To facilitate the design, validation, and comparison of fault-tolerant models and algorithms, we have created the Failure Trace Archive (FTA)-an online, public repository of failure traces collected from diverse parallel and distributed systems. In this work, we first describe the design of the archive, in particular of the standard FTA data format, and the design of a toolbox that facilitates automated analysis of trace data sets. We also discuss the use of the FTA for various current and future purposes. Second, after applying the toolbox to nine failure traces collected from distributed systems used in various application domains (e.g., HPC, Internet operation, and various online applications), we present a comparative analysis of failures in various distributed systems. Our analysis presents various statistical insights and typical statistical modeling results for the availability of individual resources in various distributed systems. The analysis results underline the need for public availability of trace data from different distributed systems. Last, we show how different interpretations of the meaning of failure data can result in different conclusions for failure modeling and job scheduling in distributed systems. Our results for different interpretations show evidence that there may be a need for further revisiting existing failure-aware algorithms, when applied for general rather than for domain-specific distributed systems.

Mining for statistical models of availability in large-scale distributed systems: An empirical study of SETI@home

In the age of cloud, Grid, P2P, and volunteer distributed computing, large-scale systems with tens of thousands of unreliable hosts are increasingly common. Invariably, these systems are composed of heterogeneous hosts whose individual availability often exhibit different statistical properties (for example stationary versus non-stationary behavior) and fit different models (for example Exponential, Weibull, or Pareto probability distributions). In this paper, we describe an effective method for discovering subsets of hosts whose availability have similar statistical properties and can be modelled with similar probability distributions. We apply this method with about 230,000 host availability traces obtained from a real large-scale Internet-distributed system, namely SETI@home. We find that about 34% of hosts exhibit availability that is a truly random process, and that these hosts can often be modelled accurately with a few distinct distributions from different families. We believe that this characterization is fundamental in the design of stochastic scheduling algorithms across large-scale systems where host availability is uncertain.

Modeling and Analysis of Communication Networks in Multicluster Systems under Spatio-Temporal Bursty Traffic

Multicluster systems have emerged as a promising infrastructure for provisioning of cost-effective high-performance computing and communications. Analytical models of communication networks in cluster systems have been widely reported. However, for tractability and simplicity, the existing models are based on the assumptions that the network traffic follows the nonbursty Poisson arrival process and the message destinations are uniformly distributed. Recent measurement studies have shown that the traffic generated by real-world applications reveals the bursty nature in both the spatial domain (i.e., nonuniform distribution of message destinations) and temporal domain (i.e., bursty message arrival process). In order to obtain a comprehensive understanding of the system performance, a novel analytical model is developed for communication networks in multicluster systems in the presence of the spatio-temporal bursty traffic. The spatial traffic burstiness is captured by the communication locality and the temporal traffic burstiness is modeled by the Markov-modulated Poisson process. After validating its accuracy through extensive simulation experiments, the model is used to investigate the impact of bursty message arrivals and communication locality on network performance. The analytical results demonstrate that the communication locality can relieve the degrading effects of bursty message arrivals on the network performance.

Reliability and energy efficiency in cloud computing systems

With the popularity of cloud computing, it has become crucial to provide on-demand services dynamically according to the users requirements. Reliability and energy efficiency are two key challenges in cloud computing systems (CCS) that need careful attention and investigation. The recent survey articles are either focused on the reliability techniques or energy efficiency methods in cloud computing. This paper presents a thorough review of existing techniques for reliability and energy efficiency and their trade-off in cloud computing. We also discuss the classifications on resource failures, fault tolerance mechanisms and energy management mechanisms in cloud systems. Moreover, various challenges and research gaps in trade-off between reliability and energy efficiency are identified for future research and developments.