Carmen Carrión

Extending GridSim with an architecture for failure detection

Grid technologies are emerging as the next generation of distributed computing, allowing the aggregation of resources that are geographically distributed across different locations. However, these resources are independent and managed separately by various organizations with different policies. This will have a major impact to users who submit their jobs to the Grid, as they have to deal with issues such as policy heterogeneity, security and fault tolerance. Moreover, the changes of Grid conditions, such as resources that may become unavailable for a period of time due to maintenance and/or suffer failures, would significantly affect the quality of service (QoS) requirements of users. Therefore, it is essential for users to take into account the effects of resource failures during jobs execution. In this paper, we present our work on introducing resource failures and failure detection into the GridSim simulation toolkit. As we need to conduct repeatable and controlled experiments, it is easier to use simulation as a means of studying complex scenarios. We also give a detailed description of the overall design and a use case scenario demonstrating the conditions of resources varied over time.

Network-aware meta-scheduling in advance with autonomous self-tuning system

The provision of Quality of Service (QoS) in Grid environments is still an open issue that needs attention from the research community. One way of contributing to the provision of QoS in Grids is by performing meta-scheduling of jobs in advance, that is, jobs are scheduled some time before they are actually executed. In this way, it becomes more likely that the appropriate resources are available to run the job when needed, so that QoS requirements of jobs are met (i.e. jobs are finished within a deadline). This paper presents a framework built on top of Globus and the GridWay meta-scheduler to provide QoS by means of performing meta-scheduling in advance. Thanks to this, QoS requirements of jobs are met. This framework manages idle/busy periods of resources in order to choose the most suitable resource for each job, and uses red-black trees for this task. Besides, no prior knowledge on the duration of jobs is required, as opposed to other works using similar techniques. This framework uses heuristics that consider the network as a first level resource. Furthermore, this framework presents an autonomous behaviour so that it adapts to the dynamic changes of the Grid resources. The autonomous behaviour is obtained by means of computing a trust for each resource and performing job rescheduling. All this set of features make this framework suitable for real Grids. Finally, a performance evaluation using a real testbed is presented that illustrates the efficiency of this approach to meet the QoS requirements of users.

Network-aware heuristics for inter-domain meta-scheduling in Grids

Grid computing generally involves the aggregation of geographically distributed resources in the context of a particular application. As such resources can exist within different administrative domains, requirements on the communication network must also be taken into account when performing meta-scheduling, migration or monitoring of jobs. Similarly, coordinating efficient interaction between different domains should also be considered when performing such meta-scheduling of jobs. A strategy to perform peer-to-peer-inspired meta-scheduling in Grids is presented. This strategy has three main goals: (1) it takes the network characteristics into account when performing meta-scheduling; (2) communication and query referral between domains is considered, so that efficient meta-scheduling can be performed; and (3) the strategy demonstrates scalability, making it suitable for many scientific applications that require resources on a large scale. Simulation results are presented that demonstrate the usefulness of this approach, and it is compared with other proposals from literature.

Exponential Smoothing for Network-Aware Meta-scheduler in Advance in Grids

Grid computing involves the coordinated use of disperse heterogeneous computing resources. This heterogeneity and dispersion makes Quality of Service (QoS) still an open issue requiring attention from the research community. One way of contributing to the provision of QoS in Grids is by performing meta-scheduling of jobs in advance, that is, the computing resource where a job will be executed is decided some time before jobs are actually executed. But this way of scheduling needs to do predictions about the future status of resources, including network. The main aim of this work is to provide QoS in Grid environments through network-aware job scheduling in advance. In our case, QoS means the fulfillment of a deadline for the completion of jobs. For this, predictions about future status of computing and network resources are made by using exponential smoothing functions. This paper presents a performance evaluation using a real testbed that illustrates the efficiency of this approach to meet the QoS requirements of users. This evaluation highlights the effects of using Exponential Smoothing (ES) to tune predictions in order to deliver the requested QoS.

A Cost-Effective Hardware Link Scheduling Algorithm for the Multimedia Router (MMR)

The primary objective of the Multimedia Router (MMR) project is the design and implementation of a compact router optimized for multimedia applications. The router is targeted for use in cluster and LAN interconnection networks, which offer different constraints and therefore differing router solutions than WANs. One of the key elements in order to achieve these goals is the scheduling algorithm. In a previous paper, the authors have proposed a link/switch scheduling algorithm capable of providing different QoS guarantees to flows as needed. This work focuses on the reduction of the hardware complexity necessary to implement such algorithm. A novel priority algorithm is presented, and its hardware complexity is compared to that of the original proposal.

Using network information to perform meta-scheduling in advance in grids

In extremely heterogeneous and distributed systems, like Grid environments, it is quite difficult to provide quality of service (QoS). In addition, the dynamic behaviour of the resources makes the time needed to complete the execution of a job highly variable. So, fulfilling the user QoS requirements in a Grid is still an open issue. The main aim of this work is to provide QoS in Grid environments through network-aware job scheduling in advance. This paper presents a technique to manage idle/busy periods of resources using red-black trees which considers the network as a first level resource. Besides, no a priori knowledge on the duration of jobs is required, as opposed to other works. A performance evaluation using a real testbed is presented which illustrates the efficiency of this approach to meet the QoS requirements of users, and highlights the importance of taking the network into account when predicting the duration of jobs.

Analyzing Hadoop power consumption and impact on application QoS

Energy efficiency is often identified as one of the key reasons for migrating to Cloud environments. It is stated that a data center hosting the Cloud environment is likely to achieve greater energy efficiency (at a reduced cost) compared to a local deployment. With increasing energy prices, it is also estimated that a large percentage of operational costs within a Cloud environment can be attributed to energy. In this work, we investigate and measure energy consumption of a number of virtual machines running the Hadoop system, over an OpenNebula Cloud. Our workload is based on sentiment analysis undertaken over Twitter messages. Our objective is to understand the tradeoff between energy efficiency and performance for such a workload. From our results we generalize and speculate on how such an analysis could be used as a basis to establish a Service Level Agreement (SLA) with a Cloud provider-especially where there is likely to be a high level of variability (both in performance and energy use) over multiple runs of the same application (at different times). Among the service level objectives that might be included in a SLA, Quality of Service (QoS) related metrics (i.e., latency) are one of the most challenging to support. This work provides some insight on the relationship between power consumption and QoS related metrics, describing how a combined consideration of these two metrics could be supported for a particular workload. Power consumption characterization of Hadoop Clouds (with a social media use case).Study of the QoS related to power consumption (in terms of processing time).Experimentation on two different Cloud infrastructures (single node-multi node).OpenNebula based private Cloud environments.

A GridWay-based autonomic network-aware metascheduler

One of the key motivations of computational and data grids is the ability to make coordinated use of heterogeneous computing resources which are geographically dispersed. Consequently, the performance of the network linking all the resources present in a grid has a significant impact on the performance of an application. It is therefore essential to consider network characteristics when carrying out tasks such as scheduling, migration or monitoring of jobs. This work focuses on an implementation of an autonomic network-aware meta-scheduling architecture that is capable of adapting its behavior to the current status of the environment, so that jobs can be efficiently mapped to computing resources. The implementation extends the widely used GridWay meta-scheduler and relies on exponential smoothing to predict the execution and transfer times of jobs. An autonomic control loop (which takes account of CPU use and network capability) is used to alter job admission and resource selection criteria to improve overall job completion times and throughput. The implementation has been tested using a real testbed involving heterogeneous computing resources distributed across different national organizations.

Meta-scheduling in advance using red-black trees in heterogeneous Grids

The provision of Quality of Service in Grid environments is still an open issue that needs attention from the research community. One way of contributing to the provision QoS in Grids is by performing meta-scheduling of jobs in advance, that is, jobs are scheduled some time before they are actually executed. In this way, the aproppriate resources will be available to run the job when needed, so that QoS requirements (i.e., deadline) are met. This paper presents two new techniques, implemented over the red-black tree data structure, to manage the idle/busy periods of resources. One of them takes into account the heterogeneity of resources when estimating the execution times of jobs. A performance evaluation using a real testbed is presented that illustrates the efficiency of this approach to meet the QoS requirements of users.

Studying the Influence of Network-Aware Grid Scheduling on the Performance Received by Users

Grid computing is the key enabling technology to aggregate geographically distributed resources in the context of a particular application. As Grids are extremely distributed systems, requirements on the communication network should also be taken into account when performing usual tasks such as scheduling, migrating or monitoring of jobs. Note that users, services, and data need to communicate with each other over networks, thus the network should be used in an efficient and fault-tolerant way. There are Grid schedulers that consider the network when performing their tasks, but the way they have been implemented does not allow easy extensions. Thus, they are not suitable to be modified and try different scheduling approaches. The authors have extended the GridWay metascheduler to perform scheduling considering the network status. This is the first step in order to proceed with more complicated and efficient scheduling and reservation processes. In this work, the extension has been evaluated by means of a testbed, in which users simultaneously submit different jobs with different frequencies to GridWay. Results presented here show that the response time perceived by Grid users is reduced when data on network performance are considered in the job scheduling process.