Marcos Dias de Assunção

Big Data computing and clouds

This paper discusses approaches and environments for carrying out analytics on Clouds for Big Data applications. It revolves around four important areas of analytics and Big Data, namely (i) data management and supporting architectures; (ii) model development and scoring; (iii) visualisation and user interaction; and (iv) business models. Through a detailed survey, we identify possible gaps in technology and provide recommendations for the research community on future directions on Cloud-supported Big Data computing and analytics solutions. Survey of solutions for carrying out analytics and Big Data on Clouds.Identification of gaps in technology for Cloud-based analytics.Recommendations of research directions for Cloud-based analytics and Big Data.

A cost-benefit analysis of using cloud computing to extend the capacity of clusters

In this paper, we investigate the benefits that organisations can reap by using “Cloud Computing” providers to augment the computing capacity of their local infrastructure. We evaluate the cost of seven scheduling strategies used by an organisation that operates a cluster managed by virtual machine technology and seeks to utilise resources from a remote Infrastructure as a Service (IaaS) provider to reduce the response time of its user requests. Requests for virtual machines are submitted to the organisation’s cluster, but additional virtual machines are instantiated in the remote provider and added to the local cluster when there are insufficient resources to serve the users’ requests. Naïve scheduling strategies can have a great impact on the amount paid by the organisation for using the remote resources, potentially increasing the overall cost with the use of IaaS. Therefore, in this work we investigate seven scheduling strategies that consider the use of resources from the “Cloud”, to understand how these strategies achieve a balance between performance and usage cost, and how much they improve the requests’ response times.

AN EVALUATION OF COMMUNICATION DEMAND OF AUCTION PROTOCOLS IN GRID ENVIRONMENTS

Dynamic pricing and good level of Pareto optimality make auctions more attractive for resource allocation over other economic models. However, some auction models present drawbacks regarding the high demand of communication when applied to large-scale scenarios. In a complex Grid environment, the communication demand can become a bottleneck; that is, a number of messages need to be exchanged for matching suitable service providers and consumers. In this context, it is worthwhile to investigate the communication demand or complexity of auction protocols in Grid environments. This work presents an analysis on the communication requirements of four auction protocols, namely First-Price Sealed, English, Dutch, and Continuous double auctions, in Grid environments. In addition, we provide a framework supporting auction protocols within a Grid simulating toolkit called GridSim.

Evaluating Auto-scaling Strategies for Cloud Computing Environments

Auto-scaling is a key feature in clouds responsible for adjusting the number of available resources to meet service demand. Resource pool modifications are necessary to keep performance indicators, such as utilisation level, between user-defined lower and upper bounds. Auto-scaling strategies that are not properly configured according to user workload characteristics may lead to unacceptable QoS and large resource waste. As a consequence, there is a need for a deeper understanding of auto-scaling strategies and how they should be configured to minimise these problems. In this work, we evaluate various auto-scaling strategies using log traces from a production Google data centre cluster comprising millions of jobs. Using utilisation level as performance indicator, our results show that proper management of auto-scaling parameters reduces the difference between the target utilisation interval and the actual values-we define such difference as Auto-scaling Demand Index. We also present a set of lessons from this study to help cloud providers build recommender systems for auto-scaling operations.

The Green Grid’5000: Instrumenting and Using a Grid with Energy Sensors

Targeting mostly application performance, distributed systems have constantly increased in size and computing power of their resources. The power supply requirements of these systems have increased in a similar fashion, which has raised concerns about the energy they consume. This paper presents the Green Grid’5000 approach, a large-scale energy-sensing infrastructure with software components that allow users to precisely measure and understand the energy usage of their system. It also discusses a set of use-cases describing how an energy instrumented platform can be utilised by various categories of users, including administrators, Grid component designers, and distributed application end-users.

Performance analysis of multiple site resource provisioning: effects of the precision of availability information

Emerging deadline-driven Grid applications require a numberof computing resources to be available over a time frame, startingat a specific time in the future. To enable these applications, it is importantto predict the resource availability and utilise this informationduring provisioning because it affects their performance. It is impracticalto request the availability information upon the scheduling of everyjob due to communication overhead. However, existing work has notconsidered how the precision of availability information influences theprovisioning. As a result, limitations exist in developing advanced resourceprovisioning and scheduling mechanisms. This work investigateshow the precision of availability information affects resource provisioningin multiple site environments. Performance evaluation is conductedconsidering both multiple scheduling policies in resource providers andmultiple provisioning policies in brokers, while varying the precision ofavailability information. Experimental results show that it is possible toavoid requesting availability information for every Grid job scheduledthus reducing the communication overhead. They also demonstrate thatmultiple resource partition policies improve the slowdown of Grid jobs.

A Generic and Extensible Framework for Monitoring Energy Consumption of OpenStack Clouds

Although cloud computing has been transformational to the IT industry, it is built on large data centres that often consume massive amounts of electrical power. Efforts have been made to reduce the energy clouds consume, with certain data centres now approaching a Power Usage Effectiveness (PUE) factor of 1.08. While this is an incredible mark, it also means that the IT infrastructure accounts for a large part of the power consumed by a data centre. Hence, means to monitor and analyse how energy is spent have never been so crucial. Such monitoring is required not only for understanding how power is consumed, but also for assessing the impact of energy management policies. In this article, we draw lessons from experience on monitoring large-scale systems and introduce an energy monitoring software framework called Kilo Watt API (KWAPI), able to handle Open-Stack clouds. The framework - whose architecture is scalable, extensible, and completely integrated into Open Stack - supports several wattmeter devices, multiple measurement formats, and minimises communication overhead.

Optimising resource costs of cloud computing for education

There is a growing interest around the utilisation of cloud computing in education. As organisations involved in the area typically face severe budget restrictions, there is a need for cost optimisation mechanisms that explore unique features of digital learning environments. In this work, we introduce a method based on Maximum Likelihood Estimation that considers heterogeneity of IT infrastructure in order to devise resource allocation plans that maximise platform utilisation for educational environments. We performed experiments using modelled datasets from real digital teaching solutions and obtained cost reductions of up to 30%, compared with conservative resource allocation strategies. Context-aware algorithm for allocating computing resources for class- rooms.Experiment setup based on real-world school data.Evaluation analysis considering security margin, costs, and QoS.

An architecture and algorithm for context-aware resource allocation for digital teaching platforms

Digital Teaching Platforms (DTPs) are aimed to support personalization of classroom education to help optimize the learning process. A trend for research and development exists regarding methods to analyze multimodal data, aiming to infer how students interact with delivered content and understanding student behavior, academic performance, and the way teachers react to student engagement. Existing DTPs can deliver several types of insights, some of which teachers can use to adjust learning activities in real-time. These technologies require a computing infrastructure capable of collecting and analyzing large volumes of data, and, for this, cloud computing is an ideal candidate solution. Nonetheless, preliminary field tests with DTPs demonstrate that applying fully remote services is prohibitive in scenarios with limited bandwidth and a constrained communication infrastructure. Therefore, we propose an architecture for DTPs and an algorithm to promote the adjustable balance between local and federated cloud resources. The solution works by deciding where tasks should be executed, based on resource availability and the quality of insights they may provide to teachers during learning sessions. In this work, we detail the system architecture, describe a proof-of-concept, and discuss the viability of the proposed approach for practical scenarios.

An integration of global and enterprise grid computing: gridbus broker and xgrid perspective

In this work, we present two perspectives of Grid computing by using two different Grid middleware as examples: an Enterprise Grid using Xgrid and a Global Grid with Gridbus. We also present the integration of Enterprise and Global Grids by proposing the integration of Gridbus Broker with diverse enterprise Grid middleware including Xgrid, PBS, Condor and SGE. The sample performance results demonstrate the usefulness of this integration effort.