Axel Keller

Scheduling in HPC resource management systems: Queuing vs. planning

Nearly all existing HPC systems are operated by resource management systems based on the queuing approach. With the increasing acceptance of grid middleware like Globus, new requirements for the underlying local resource management systems arise. Features like advanced reservation or quality of service are needed to implement high level functions like co-allocation. However it is difficult to realize these features with a resource management system based on the queuing concept since it considers only the present resource usage.

The virtual resource manager: an architecture for SLA-aware resource management

The next generation Grid will demand the Grid middleware to provide flexibility, transparency, and reliability. This implies the appliance of service level agreements to guarantee a negotiated level of quality of service. These requirements also affect the local resource management systems providing resources for the Grid. At this a gap between these demands and the features of todays resource management systems becomes apparent. In this paper we present an approach which closes this gap. Introducing the architecture of the virtual resource manager we highlight its main features of runtime responsibility, resource virtualization, information hiding, autonomy provision, and smooth integration of existing resource management system installations.

Rule-Based Mapping of Virtual Machines in Clouds

Infrastructure as a Service providers use virtualization to abstract their hardware and to create a dynamic data center. Virtualization enables the consolidation of virtual machines as well as the migration of them to other hosts during runtime. Each provider has its own strategy to efficiently operate a data center. We present a rule based mapping algorithm for VMs, which is able to automatically adapt the mapping between VMs and physical hosts. It offers an interface where policies can be defined and combined in a generic way. The algorithm performs the initial mapping at request time as well as a remapping during runtime. It deals with policy and infrastructure changes. We extended the open source IaaS solution Eucalyptus and we evaluated it with typical policies: maximizing the compute performance and VM locality to achieve a high performance and minimizing energy consumption. The evaluation was done on state-of-the-art servers in our own data center and by simulations using a workload of the Parallel Workload Archive. The results show that our algorithm performs well in dynamic data centers environments.

RSD — Resource and Service Description

RSD (Resource and Service Description) is a scheme for specifying resources and services in complex heterogeneous computing systems and metacomputing environments. At the system administrator level, RSD is used to specify the available system components, such as the number of nodes, their interconnection topology, CPU speeds, and available software packages. At the user level, a GUI provides a comfortable, high-level interface for specifying system requests. A textual editor can be used for defining repetitive and recursive structures. This gives service providers the necessary flexibility for fine-grained specification of system topologies, interconnection networks, system and software dependent properties. All these representations are mapped onto a single, coherent internal object-oriented resource representation.

Virtualized HPC: a contradiction in terms?

System virtualization has become the enabling technology to manage the increasing number of different applications inside data centers. The abstraction from the underlying hardware and the provision of multiple virtual machines (VM) on a single physical server have led to a consolidation and more efficient usage of physical servers. The abstraction from the hardware also eases the provision of applications on different data centers, as applied in several cloud computing environments. In this case, the application need not adapt to the environment of the cloud computing provider, but can travel around with its own VM image, including its own operating system and libraries. System virtualization and cloud computing could also be very attractive in the context of high‐performance computing (HPC). Today, HPC centers have to cope with both, the management of the infrastructure and also the applications. Virtualization technology would enable these centers to focus on the infrastructure, while the users, collaborating inside their virtual organizations (VOs), would be able to provide the software. Nevertheless, there seems to be a contradiction between HPC and cloud computing, as there are very few successful approaches to virtualize HPC centers. This work discusses the underlying reasons, including the management and performance, and presents solutions to overcome the contradiction, including a set of new libraries. The viability of the presented approach is shown based on evaluating a selected parallel, scientific application in a virtualized HPC environment. Copyright

RsdEditor: a graphical user interface for specifying metacomputer components

RsdEditor is a graphical user interface which produces specifications of computational resources. It is used in the RSD (Resource and Service Description) environment for specifying, registering, requesting and accessing resources and services in a metacomputer. RsdEditor was designed to be used by the administrators and users of metacomputing environments. At the administrator level, the GUI is used to describe the available computing and networking components of a metacomputer. At the user level, RsdEditor can be used to specify which characteristics of the computational resources are needed to execute a meta-application. This paper is organized as follows: it first introduces RsdEditor. It then briefly describes the RSD environment, and finally, it highlights various features and implementation issues of RsdEditor.

Quality assurance of Grid service provisioning by risk aware managing of resource failures

Quality assurance is a key aspect in scope of the provisioning of grid services since end-users ask for specific quality of service (QoS) criteria defined in service level agreements (SLA). To commit to an SLA, grid providers need a risk analysis during SLA negotiation in order to estimate the probability of an SLA violation. In addition, such a risk analysis is necessary in the post-negotiation phase, in order to find the most profitable solution if not all SLAs can be fulfilled. Current job failure rates in grids (10-45%) highlight the necessity of fault-tolerance mechanisms. If not enough resources exist to compensate for all resource outages, the provider has to prefer those jobs which are in expectation the most profitable ones. Hence, this quality assurance ensures that obligations from the most important jobs will be fulfilled.

Specifying resources and services in metacomputing environments

With a steadily increasing number of services, metacomputing is now gaining importance in science and industry. Virtual organizations, autonomous agents, mobile computing services, and high-performance client–server applications are among the many examples of metacomputing services. For all of them, resource description plays a major role in organizing access, use, and administration of the computing components and software services. We present a generic Resource and Service Description (RSD) for specifying the hardware and software components of (meta-) computing environments. Its graphical interface allows metacomputer users to specify their resource requests. Its textual counterpart gives service providers the necessary flexibility to specify topology and properties of the available system and software resources. Finally, its internal object-oriented representation is used to link different resource management systems and service tools. With these three representations, our generic RSD approach is a key component for building metacomputer environments.

Cost-Aware and SLO-Fulfilling Software as a Service

Virtualization technology makes data centers more dynamic and easier to administrate. Today, cloud providers offer customers access to complex applications running on virtualized hardware. Nevertheless, big virtualized data centers become stochastic environments and the simplification on the user side leads to many challenges for the provider. He has to find cost-efficient configurations and has to deal with dynamic environments to ensure service level objectives (SLOs). We introduce a software solution that reduces the degree of human intervention to manage clouds. It is designed as a multi-agent system (MAS) and placed on top of the Infrastructure as a Service (IaaS) layer. Worker agents allocate resources, configure applications, check the feasibility of requests, and generate cost estimates. They are equipped with application specific knowledge allowing it to estimate the type and number of necessary resources. During runtime, a worker agent monitors the job and adapts its resources to ensure the specified quality of service—even in noisy clouds where the job instances are influenced by other jobs. They interact with a scheduler agent, which takes care of limited resources and does a cost-aware scheduling by assigning jobs to times with low costs. The whole architecture is self-optimizing and able to use public or private clouds. Building a private cloud needs to face the challenge to find a mapping of virtual machines (VMs) to hosts. We present a rule-based mapping algorithm for VMs. It offers an interface where policies can be defined and combined in a generic way. The algorithm performs the initial mapping at request time as well as a remapping during runtime. It deals with policy and infrastructure changes. An energy-aware scheduler and the availability of cheap resources provided by a spot market are analyzed. We evaluated our approach by building up an SaaS stack, which assigns resources in consideration of an energy function and that ensures SLOs of two different applications, a brokerage system and a high-performance computing software. Experiments were done on a real cloud system and by simulations.

Resource management for high-performance PC clusters

With the recent availability of cost-effective network cards for the PCI bus, researchers have been tempted to build up large compute clusters with standard PCs. Many of them are operated with workstation cluster management software in high-throughput or single user mode.