Stephen Childs

A single-computer grid gateway using virtual machines

Grid middleware is enabling resource sharing between computing centres across the world and sites with existing clusters are eager to connect to the grid using middleware such as that developed by the LHC Computing Grid (LCG) project. However, the hardware requirements for access to the grid remain high: a standard LCG grid gateway requires four separate servers. We propose the use of virtual machine (VM) technology to run multiple OS instances, allowing a full grid gateway to be hosted on a single computer. This would significantly reduce the hardware, installation and management commitments required of a site that wants to connect to the grid. In this paper, we outline the architecture of a single-computer grid gateway. We evaluate implementations of this architecture using two popular open-source VMs: Xen and user-mode Linux (UML). Our results show that Xen outperforms UML for installation tasks and standard gateway operations. Configuration is similar to that of sites running multi-computer gateways, making it easy to keep site installation profiles synchronised. Our VM gateway architecture provides a low-cost entry path to the grid and will be of interest to many institutions wishing to connect their existing facilities.

GridBuilder: A Tool for Creating Virtual Grid Testbeds

Grid software developers and Grid site administrators both require realistic testbeds where they can test applications and middleware before deployment on production infrastructure. Such testbeds should be dynamically reconfigurable to allow replication of real-world configurations. While the combination of service nodes needed for a particular application may vary greatly, there is a fixed set of node types which are used frequently: these are the building blocks needed to construct testbeds. We present GridBuilder: a web-based virtual machine (VM) manager that supports the rapid creation and customisation of Grid nodes based on standard configurations. GridBuilder allows users to create a library of filesystem images and then generate independent filesystems based on these images. Copy-on-write is used for fast initialisation and efficient use of disk space. GridBuilder uses standard Grid configuration tools to automatically configure the middleware on new Grid service nodes. Users can save and restore snapshots of nodes and can import images to create new node types.

Deployment of grid gateways using virtual machines

Grid-Ireland, the national computational grid for Ireland, has a centrally managed core infrastructure: the installation and configuration of grid gateways at constituent sites are controlled from an Operations Centre based at Trinity College Dublin. We have developed tools to automate and simplify the deployment of Grid middleware to these sites. Virtual machine (VM) technology has performed an important role, allowing us to maximise the utilisation of server hardware and to simplify installation and management procedures. By running multiple OS instances, each on a VM, a full LCG-compatible Grid gateway can be hosted on a single computer. This has significantly reduced the hardware, installation and management investment needed to deploy a new site. In this paper, we summarise an evaluation of competing VM technologies and relate our experience with virtual machines to date. We also describe a single-computer Grid gateway based on the Xen VM system which we plan to deploy to eleven new sites in early 2005.

Principles of transactional grid deployment

This paper examines the availability of grid infrastructures, describes the principles of transactional grid deployment, outlines the design and implementation of a transactional deployment system for the Grid-Ireland national grid infrastructure based on these principles, and estimates the resulting availability.

Heterogeneous grid computing: issues and early benchmarks

A heterogeneous implementation of the current LCG2/EGEE grid computing software is supported in the Grid-Ireland infrastructure. The porting and testing of the current software version of LCG2 is presented for different flavours of Linux, namely Red Hat 7.3, Red Hat 9 and Fedora Core 2. The GridBench micro-benchmarks developed in CrossGrid are used to compare the different platforms.

Dynamic virtual worker nodes in a production grid

There is a growing body of opinion that virtual machines (VMs) provide a good environment for executing user jobs on Grid compute nodes. Sites which execute jobs in specially-created virtual machines can provide levels of isolation and customisation that are unobtainable when jobs run directly on the hardware. Various solutions have been proposed for initiating and controlling such dynamic virtual environments, but issues of integration with a production Grid middleware stack have not received much attention. In addition, solutions proposed to date often require significant user involvement in the process of locating and initiating VMs. We outline a scheme for transparently providing dynamically-instantiated VM-based worker nodes in the EGEE production grid. By extending server-side software, the use of virtual machines is made invisible to the user. Users simply specify the details of their required execution environment in the standard job description language. Resource brokers then locate sites that advertise support for that particular environment. Sites that support dynamic virtual worker nodes advertise support for the various environments that they know how to create; the sites compute element is responsible for instantiating a VM that conforms to the environment description requested and for executing the job in that VMs context. We also evaluate the VM management tools available to implement such a scheme and describe their possible integration with LCG and gLite middleware.

Heterogeneity of computing nodes for grid computing

A heterogeneous implementation of the current LCG2/EGEE grid computing software is supported in the Grid-Ireland infrastructure. The porting and testing of the current softwareversionofLCG2iswellunder way on Red Hat 9, Fedora Core 2, IRIX and AIX. The issues concerned are discussed, and recommendations are presented for improvement of portability to non-reference operating systems.

The Back-End of a Two-Layer Model for a Federated National Datastore for Academic Research VOs that Integrates EGEE Data Management

This paper proposes an architecture for the back-end of a federated national datastore for use by academic research communities, developed by the e-INIS (Irish National e-InfraStructure) project, and describes in detail one member of the federation, the regional datastore at Trinity College Dublin. It builds upon existing infrastructure and services, including Grid-Ireland, the National Grid Initiative and EGEE, Europe’s leading Grid infrastructure. It assumes users are in distinct research communities and that their data access patterns can be described via two properties, denoted as mutability and frequency-of-access. The architecture is for a back-end—individual academic communities are best qualified to define their own front-end services and user interfaces. The proposal is designed to facilitate front-end development by placing minimal restrictions on how the front-end is implemented and on the internal community security policies. The proposal also seeks to ensure that the communities are insulated from the back-end and from each other in order to ensure quality of service and to decouple their front-end implementation from site-specific back-end implementations.

An Introduction to Grid Computing Using EGEE

Grid is an evolving and maturing architecture based on several well-established services, including amongst others, distributed computing, role and group management, distributed data management and Public Key Encryption systems Currently the largest scientific grid infrastructure is Enabling Grids e-Science (EGEE), comprised of approximately ∼250 sites, ∼50,000 CPUs and tens of petabytes of storage. Moreover, EGEE covers a large variety of scientific disciplines including Astrophysics. The scope of this work is to provide the keen astrophysicist with an introductory overview of the motivations for using Grid, and of the core production EGEE services and its supporting software and/or middleware (known by the name gLite). We present an overview of the available set of commands, tools and portals as used within these Grid communities. In addition, we present the current scheme for supporting MPI programs on these Grids.

Integrating Xen with the Quattor fabric management system

While the deployment of virtual machines (VMs) within the high-performance computing (HPC) community is proceeding at a great pace, tools for system management of VMs are still lagging behind those available for physical machines. In order to make further progress, VM management must be fully integrated with existing fabric management infrastructure. We present the results of work done to integrate Xen [4] with the Quattor [15] fabric management suite. The principal contributions are the development of a network bootloader for para-virtualised Xen VMs and a Quattor management component for setting up hosted VMs. The combination of these tools allows for full unattended installation of Xen VMs and the automatic configuration of services, all from a single configuration database.