Fabien Hermenier

An energy aware framework for virtual machine placement in cloud federated data centres

Data centres are powerful ICT facilities which constantly evolve in size, complexity, and power consumption. At the same time users and operators requirements become more and more complex. However, existing data centre frameworks do not typically take energy consumption into account as a key parameter of the data centres configuration. To lower the power consumption while fulfilling performance requirements we propose a flexible and energy-aware framework for the (re)allocation of virtual machines in a data centre. The framework, being independent from the data centre management system, computes and enacts the best possible placement of virtual machines based on constraints expressed through service level agreements. The frameworks flexibility is achieved by decoupling the expressed constraints from the algorithms using the Constraint Programming (CP) paradigm and programming language, basing ourselves on a cluster management library called Entropy. Finally, the experimental and simulation results demonstrate the effectiveness of this approach in achieving the pursued energy optimization goals.

BtrPlace: A Flexible Consolidation Manager for Highly Available Applications

The massive amount of resources found in data centers makes it possible to provide high availability to multitier applications. Virtualizing these applications makes it possible to consolidate them on servers, reducing runtime costs. Nevertheless, replicated VMs have to be carefully placed within the data center to provide high availability and good performance. This requires resolving potentially conflicting application and data center requirements, while scaling up to the size of modern data centers. We present BtrPlace, a flexible consolidation manager that is customized through configuration scripts written by the application and data center administrators. BtrPlace relies on constraint programming and an extensible library of placement constraints. The present library of 14 constraints subsumes and extends the capabilities of existing commercial consolidation managers. Scalability is achieved by splitting the data center into partitions and computing placements in parallel. Overall, BtrPlace repairs a nonviable placement after a major load increase or a maintenance operation for a 5,000 server data center hosting 30,000 VMs and involving thousands of constraints in 3 minutes. Using partitions of 2,500 servers, placement computing is reduced to under 30 seconds.

Cluster-wide context switch of virtualized jobs

Clusters are mostly used through Resources Management Systems (RMS) with a static allocation of resources for a bounded amount of time. Those approaches are known to be insufficient for an efficient use of clusters. To provide a finer RMS, job preemption, migration and dynamic allocation of resources are required. However due to the complexity of developing and using such mechanisms, advanced scheduling strategies have rarely been deployed. This trend is currently evolving thanks to the use of migration and preemption capabilities of Virtual Machines (VMs). However, although the manipulation of jobs composed of VM enables to change the state of the jobs according to the scheduling objective, changing the state and the location of numerous VMs at each decision is tedious and degrades the overall performance. In addition to the scheduling policy implementation, developers have to focus on the feasibility of the actions while executing them in the most efficient way.n In this paper, we argue such an operation is independent from the policy itself and can be addressed through a generic mechanism, the cluster-wide context switch. Thanks to it, developers can implement sophisticated algorithms to schedule jobs without handling the issues related to their manipulations. They only focus on the implementation of their algorithm to select the jobs to run while the cluster-wide context switch system performs the necessary actions to switch from the current to the new situation. As a proof of concept, we evaluate the interest of the cluster-wide context switch through a sample scheduler that executes jobs as early as possible, even partially, regarding to their current resources requirements and their priority.

Plug4Green: A flexible energy-aware VM manager to fit data centre particularities

To maintain an energy footprint as low as possible, data centres manage their VMs according to conventional and established rules. Each data centre is however made unique due to its hardware and workload specificities. This prevents the ad hoc design of current VM managers from taking these particularities into account to provide additional energy savings. In this paper, we present Plug4Green, an energy-aware VM placement algorithm that can be easily specialized and extended to fit the specificities of the data centres. Plug4Green computes the placement of the VMs and state of the servers depending on a large number of constraints, extracted automatically from SLAs. The flexibility of Plug4Green is achieved by allowing the constraints to be formulated independently from each other but also from the power models. This flexibility is validated through the implementation of 23 SLA constraints and 2 objectives aiming at reducing either the power consumption or the greenhouse gas emissions. On a heterogeneous test bed, Plug4Green specialization to fit the hardware and the workload specificities allowed to reduce the energy consumption and the gas emission by up to 33% and 34%, respectively. Finally, simulations showed that Plug4Green is capable of computing an improved placement for 7500 VMs running on 1500 servers within a minute.

Aspect-Based Patterns for Grid Programming

The development of grid algorithms is frequently hampered by limited means to describe topologies and lack of support for the invasive composition of legacy components in order to pass data between them. In this paper we present a solution to overcome these limitations using the notion of invasive patterns for the construction of distributed algorithms, a recent extension of well-known computation and communication patterns. Concretely, we present two contributions. First, based on a study of how patterns are instantiated in NAS grid, a well-known benchmark used for evaluating performance of computational grids, we show how invasive patterns can be used for the declarative definition of large-scale grid topologies and checkpointing algorithms. Second, we qualitatively and quantitatively evaluate how our approach can be used to implement the checkpointing on top of grid applications.

Multi-objective job placement in clusters

One of the key decisions made by both MapReduce and HPC cluster management frameworks is the placement of jobs within a cluster. To make this decision, they consider factors like resource constraints within a node or the proximity of data to a process. However, they fail to account for the degree of collocation on the clusters nodes. A tight process placement can create contention for the intra-node shared resources, such as shared caches, memory, disk, or network bandwidth. A loose placement would create less contention, but exacerbate network delays and increase cluster-wide power consumption. Finding the best job placement is challenging, because among many possible placements, we need to find one that gives us an acceptable trade-off between performance and power consumption. We propose to tackle the problem via multi-objective optimization. Our solution is able to balance conflicting objectives specified by the user and efficiently find a suitable job placement.

An energy aware application controller for optimizing renewable energy consumption in data centres

Sustainable energy sources such as renewable energies are replacing dirty sources of energy in order to address the environmental challenges of the century. In order to operate data centres with renewable energies we have to mitigate their volatile and variable nature. In this paper, we present the Energy Adaptive Software Controller (EASC), a generic software controller and interface that developers can use to make their application adaptive to renewable energy availability. Adaptivity is realized through the concept of working modes which allow to run an application under various performance levels. We advocate for a collaborative approach involving the developers of the applications in order to use the renewable energies more efficiently. The notion of EASC allows to abstract away the details of application scheduling, execution, and monitoring. We demonstrate the applicability and genericity of the EASC concept through four different instantiations. These instantiations cover two types of applications: task-oriented and service-oriented, and two kind of computing environments: Infrastructure-as-a-Service, and Platform-as-a-Service. The EASC has been trialled in the data centre of the healthcare agency of Trento, Italy and in the laboratory of HP Milan, Italy, with a mix of energy sources: national grid and local solar panels. The experimental results show how the EASC allowed to increase the renewable energies usage of 14% and 4.73% for Trento and HP Labs trials, respectively.

Scheduling live-migrations for fast, adaptable and energy-efficient relocation operations

Every day, numerous VMs are migrated inside a datacenter to balance the load, save energy or prepare production servers for maintenance. Despite VM placement problems are carefully studied, the underlying migration scheduler rely on vague adhoc models. This leads to unnecessarily long and energy-intensive migrations. We present mVM, a new and extensible migration scheduler. mVM takes into account the VM memory workload and the network topology to estimate precisely the migration duration and take wiser scheduling decisions. mVM is implemented as a plugin of BtrPlace and can be customized with additional scheduling constraints to finely control the migrations. Experiments on a real testbed show mVM outperforms schedulers that cap the migration parallelism by a constant to reduce the completion time. Besides an optimal capping, mVM reduces the migration duration by 20.4% on average and the completion time by 28.1%. In a maintenance operation involving 96 VMs to migrate between 72 servers, mVM saves 21.5% Joules against BtrPlace. Finally, its current library of 6 constraints allows administrators to address temporal and energy concerns, for example to adapt the schedule and fit a power budget.

Scheduling Live Migration of Virtual Machines

Every day, numerous VMs are migrated inside a datacenter to balance the load, save energy or prepare production servers for maintenance. Although VM placement problems are carefully studied, the underlying migration schedulers rely on vague adhoc models. This leads to unnecessarily long and energy-intensive migrations. We present mVM, a new and extensible migration scheduler. To provide schedules with minimal completion times, mVM parallelizes and sequentializes the migrations with regards to the memory workload and the network topology. mVM is implemented as a plugin of BtrPlace and its current library allows administrators to address temporal and energy concerns. Experiments on a real testbed shows mVM outperforms state-of-the-art migration schedulers. Compared to schedulers that cap the migration parallelism, mVM reduces the individual migration duration by 20.4 percent on average and the schedule completion time by 28.1 percent. In a maintenance operation involving 96 VMs migrated between 72 servers, mVM saves 21.5 percent Joules against BtrPlace. Compared to the migration model inside the cloud simulator CloudSim, the prediction error of the migrations duration is about 5 times lower with mVM. By computing schedules involving thousands of migrations performed over various fat-tree network topologies, we observed that the mVM solving time accounts for about 1 percent of the schedule execution time.

Ordonnancement contrôlé de migrations à chaud

Migrer a chaud une machine virtuelle (VM) est une operation basique dans un centre de don-nees. Tous les jours, des VM sont migrees pour repartir la charge, economiser de lenergie ou preparer la maintenance de serveurs en production. Bien que les problemes de placement des VM soient beaucoup etudies, on observe que la gestion des migrations permettant de transiter vers ces nouveaux placements reste un domaine de second plan. On observe alors des algo-rithmes de placement de qualite, couples a des algorithmes dordonnancement prenant des decisions peu pertinentes causees par des hypotheses irrealistes. Nous presentons dans ce papier mVM, un ordonnanceur de migrations reposant sur un modele precis du reseau et du protocole de migration a chaud. Cet ordonnanceur a ete integre en place de celui du gestionnaire de VM BtrPlace. Nos premieres experimentations montrent que les durees des migrations sont estimees a 1.5 secondes pres. Cette precision a permis de calculer de meilleurs ordonnancements, reduisant la duree des migrations par 3.5 comparee a BtrPlace.