Renato J. O. Figueiredo

From virtualized resources to virtual computing grids: the In-VIGO system

This paper describes the architecture of the first implementation of the In-VIGO grid-computing system. The architecture is designed to support computational tools for engineering and science research In Virtual Information Grid Organizations (as opposed to in vivo or in vitro experimental research). A novel aspect of In-VIGO is the extensive use of virtualization technology, emerging standards for grid-computing and other Internet middleware. In the context of In-VIGO, virtualization denotes the ability of resources to support multiplexing, manifolding and polymorphism (i.e. to simultaneously appear as multiple resources with possibly different functionalities). Virtualization technologies are available or emerging for all the resources needed to construct virtual grids which would ideally inherit the above mentioned properties. In particular, these technologies enable the creation of dynamic pools of virtual resources that can be aggregated on-demand for application-specific user-specific grid-computing. This change in paradigm from building grids out of physical resources to constructing virtual grids has many advantages but also requires new thinking on how to architect, manage and optimize the necessary middleware. This paper reviews the motivation for In-VIGO approach, discusses the technologies used, describes an early architecture for In-VIGO that represents a first step towards the end goal of building virtual information grids, and reports on first experiences with the In-VIGO software under development.

Guest Editors' Introduction: Resource Virtualization Renaissance

Virtualization technologies encompass a variety of mechanisms and techniques used to address computer system problems such as security, performance, and reliability by decoupling the architecture and user-perceived behavior of hardware and software resources from their physical implementation.

Experimental study of virtual machine migration in support of reservation of cluster resources

Virtual Machines are becoming increasingly valuable to resource consolidation and management, providing efficient and secure resource containers, along with desired application execution environments. This paper focuses on the VM-based resource reservation problem, that is, the reservations of CPU, memory and network resources for individual VM instances, as well as for VM clusters. In particular, it considers the scenario where one or several physical servers need to be vacated to start a cluster of VMs for dedicated execution of parallel jobs. VMs provide a primitive for transparently vacating workloads through migration; however, the process of migrating several VMs can be time-consuming and needs to be estimated. To achieve this goal, this paper seeks to provide a model that can characterize the VM migration process and predict its performance, based on a comprehensive experimental analysis. The results show that, given a certain VMs migration time, it is feasible to predict the time for a VM with other configurations, as well as the time for migrating a number of VMs. The paper also shows that migration of VMs in parallel results in shorter aggregate migration times, but with higher per-VM migration latencies. Experimental results also quantify the benefits of buffering the state of migrated VMs in main memory without committing to hard disks.

IP over P2P: enabling self-configuring virtual IP networks for grid computing

Peer-to-peer (P2P) networks have mostly focused on task oriented networking, where networks are constructed for single applications, i.e. file-sharing, DNS caching, etc. In this work, we introduce IPOP, a system for creating virtual IP networks on top of a P2P overlay. IPOP enables seamless access to grid resources spanning multiple domains by aggregating them into a virtual IP network that is completely isolated from the physical network. The virtual IP network provided by IPOP supports deployment of existing IP-based protocols over a robust, self-configuring P2P overlay. We present implementation details as well as experimental measurement results taken from LAN, WAN, and Planet-Lab tests

WOW: Self-Organizing Wide Area Overlay Networks of Virtual Workstations

This paper describes WOW, a distributed system that combines virtual machine, overlay networking and peer-to-peer techniques to create scalable wide-area networks of virtual workstations for high-throughput computing. The system is architected to: facilitate the addition of nodes to a pool of resources through the use of system virtual machines (VMs) and self-organizing virtual network links; to maintain IP connectivity even if VMs migrate across network domains; and to present to end-users and applications an environment that is functionally identical to a local-area network or cluster of workstations. We describe a novel, extensible user-level decentralized technique to discover, establish and maintain overlay links to tunnel IP packets over different transports (including UDP and TCP) and across firewalls. We also report on several experiments conducted on a testbed WOW deployment with 118 P2P router nodes over PlanetLab and 33 VMware-based VM nodes distributed across six firewalled domains. Experiments show that the latency in joining a WOW network is of the order of seconds: in a set of 300 trials, 90% of the nodes self-configured P2P routes within 10 seconds, and more than 99% established direct connections to other nodes within 200 seconds. Experiments also show that the testbed delivers good performance for two unmodified, representative benchmarks drawn from the life-sciences domain. The testbed WOW achieves an overall throughput of 53 jobs/minute for PBS-scheduled executions of the MEME application (with average single-job sequential running time of 24.1s) and a parallel speedup of 13.5 for the PVM-based fastDNAml application. Experiments also demonstrate that the system is capable of seamlessly maintaining connectivity at the virtual IP layer for typical client/server applications (NFS, SSH, PBS) when VMs migrate across a WAN

Design of the FutureGrid experiment management framework

FutureGrid provides novel computing capabilities that enable reproducible experiments while simultaneously supporting dynamic provisioning. This paper describes the FutureGrid experiment management framework to create and execute large scale scientific experiments for researchers around the globe. The experiments executed are performed by the various users of FutureGrid ranging from administrators to software developers and end users. The Experiment management framework will consist of software tools that record user and system actions to generate a reproducible set of tasks and resource configurations. Additionally, the experiment management framework can be used to share not only the experiment setup, but also performance information for the specific instantiation of the experiment. This makes it possible to compare a variety of experiment setups and analyze the impact Grid and Cloud software stacks have.

Punch: web portal for running tools

This distributed network computer lets users access and run engineering tools anytime, anywhere via web browsers. It is the enabling technology for Netcare, an Internet resource that is now freely available for computer architecture education and research.

On the Design of Virtual Machine Sandboxes for Distributed Computing in Wide-area Overlays of Virtual Workstations

With recent advances in virtual computing and the revelation that compute-intensive tasks run well on system virtual machines (VMs), the ability to develop, deploy, and manage distributed systems has been ameliorated. This paper explores the design space of VM-based sandboxes where the following techniques that facilitate the deployment of secure nodes in wide-area overlays of virtual workstations (WOWs) are employed: DHCP-based virtual IP address allocation, self-configuring virtual networks supporting peer-to-peer NAT traversal, stacked file systems, and IPsec-based host authentication and end-to-end encryption of communication channels. Experiments with implementations of single-image VM sandboxes, which incorporate the above features and are easily deployable on hosted I/O VMMs, show execution time overheads of 10.6% or less for a batch- oriented CPU-intensive benchmark.

Distributed file system support for virtual machines in grid computing

This paper presents a data management solution which allows fast virtual machine (VM) instantiation and efficient run-time execution to support VMs as execution environments in grid computing. It is based on novel distributed file system virtualization techniques and is unique in that: 1) it provides on-demand access to VM state for unmodified VM monitors; 2) it supports user-level and write-back disk caches, per-application caching policies and middleware-driven consistency models; and 3) it supports the use of meta-data associated with files to expedite data transfers. The paper reports on its performance in a WAN setup using VMware-based VMs. Results show that the solution delivers performance over 30% better than native NFS and can bring application-perceived overheads below 10% relatively to a local disk setup. The solution also allows a VM with 1.6GB virtual disk and 320MB virtual memory to be cloned within 160 seconds when it is first instantiated (and within 25 seconds for subsequent clones).

Grid-computing portals and security issues

Computational grids provide computing power by sharing resources across administrative domains. This sharing, coupled with the need to execute untrusted code from arbitrary users, introduces security hazards. Grid environments are built on top of platforms that control access to resources within a single administrative domain, at the granularity of a user. In wide-area multidomain grid environments, the overhead of maintaining user accounts is prohibitive, and securing access to resources via user accountability is impractical. Typically, these issues are handled by implementing checks that guarantee the safety of applications, so that they can run in shared user accounts. This work shows that safety checks--language-based, compile-time, link-time or load-time--currently implemented in most grid environments are either inadequate or limit allowed grid users and applications. A survey of various grid systems is presented, highlighting the problems and limitations of current grid environments. A runtime process monitoring technique is also proposed. The approach allows setting-up an execution environment that supports the full legitimate use allowed by the security policy of a shared resource. For shell-based applications, performance measurements of the proposed scheme show up to 2.14 times less overheads as compared to the case where all applications including the shell are monitored.