Timothy Freeman

On the Use of Cloud Computing for Scientific Workflows

This paper explores the use of cloud computing for scientific workflows, focusing on a widely used astronomy application-Montage. The approach is to evaluate from the point of view of a scientific workflow the tradeoffs between running in a local environment, if such is available, and running in a virtual environment via remote, wide-area network resource access. Our results show that for Montage, a workflow with short job runtimes, the virtual environment can provide good compute time performance but it can suffer from resource scheduling delays and widearea communications.

Virtual workspaces: Achieving quality of service and quality of life in the Grid

By defining standardized protocols for discovering, accessing, monitoring, and managing remote computers, storage systems, networks, and other resources, Grid technologies make it possible—in principle—to allocate resources to applications dynamically, in an on-demand fashion [1]. However, while Grids offer users access to many diverse and powerful resources, they do little to ensure that once a resource is accessed, it fulfills user expectations for

Elastic Site: Using Clouds to Elastically Extend Site Resources

Infrastructure-as-a-Service (IaaS) cloud computing offers new possibilities to scientific communities. One of the most significant is the ability to elastically provision and relinquish new resources in response to changes in demand. In our work, we develop a model of an “elastic site” that efficiently adapts services provided within a site, such as batch schedulers, storage archives, or Web services to take advantage of elastically provisioned resources. We describe the system architecture along with the issues involved with elastic provisioning, such as security, privacy, and various logistical considerations. To avoid over- or under-provisioning the resources we propose three different policies to efficiently schedule resource deployment based on demand. We have implemented a resource manager, built on the Nimbus toolkit to dynamically and securely extend existing physical clusters into the cloud. Our elastic site manager interfaces directly with local resource managers, such as Torque. We have developed and evaluated policies for resource provisioning on a Nimbus-based cloud at the University of Chicago, another at Indiana University, and Amazon EC2. We demonstrate a dynamic and responsive elastic cluster, capable of responding effectively to a variety of job submission patterns. We also demonstrate that we can process 10 times faster by expanding our cluster up to 150 EC2 nodes.

Virtual Clusters for Grid Communities

A challenging issue facing Grid communities is that while Grids can provide access to many heterogeneous resources, the resources to which access is provided often do not match the needs of a specific application or service. In an environment in which both resource availability and software requirements evolve rapidly, this disconnect can lead to resource underutilization, user frustration, and much wasted effort spent on bridging the gap between applications and resources. We show here how these issues can be overcome by allowing authorized Grid clients to negotiate the creation of virtual clusters made up of virtual machines configured to suit client requirements for software environment and hardware allocation. We introduce descriptions and methods that allow us to deploy flexibly configured virtual cluster workspaces. We describe their configuration, implementation, and evaluate them in the context of a virtual cluster representing the environment in production use by the Open Science Grid. Our performance evaluation results show that virtual clusters representing current Grid production environments can be deployed and managed efficiently, and thus can provide an acceptable platform for Grid applications.

Virtual workspaces in the grid

Despite significant progress in the development of Grid infrastructure, the provisioning of a customized and controllable remote execution environment remains an open issue. This paper introduces the concept of a virtual workspace, a configurable execution environment that can be created and managed as a first-class entity to reflect client requirements. Such workspaces can be dynamically deployed on a variety of resources decoupling the notion of environment and resource. We show how virtual workspaces fit into the Grid architecture, present an example implementation using virtual machines, and discuss our initial experiences using this system in practice and with applications.

Contextualization: Providing One-Click Virtual Clusters

As virtual appliances become more prevalent, we encounter the need to stop manually adapting them to their deployment context each time they are deployed. We examine appliance contextualization needs and present architecture for secure, consistent, and dynamic contextualization, in particular for groups of appliances that must work together in a shared security context. This architecture allows for programmatic cluster creation and use, as well as mitigating potential errors and unnecessary charges during setup time. For portability across many deployment mechanisms, we introduce the concept of a standalone context broker. We describe the current implementation of the entire architecture using the virtual workspaces toolkit, showing real-life examples of dynamically contextualized Grid clusters.

A Flexible Attribute Based Access Control Method for Grid Computing

Grid systems have huge and changeable user groups, and different autonomous domains always have different security policies. The attribute based access control (ABAC) model, which is flexible and scalable, is more suitable for Grid systems. This paper describes a method of building a flexible access control mechanism that is based on ABAC and supports multiple policies for Grid computing. Firstly an attribute based multipolicy access control model ABMAC is submitted. Compared with ABAC, ABMAC can describe multiple heterogeneous policies, and each policy is encapsulated without changing its descriptions. Then by extending the authorization architecture of XACML, the paper puts forward an authorization framework that supports ABMAC and is implemented in the Globus Toolkit release 4 (GT4) (Few parts of the authorization framework described in this paper can only be found in Globus Toolkit CVS repository. A more completed authorization framework will be appeared in the Globus Toolkit release 4.2). Basing on the concept of policy encapsulation, the framework provides a flexible and scalable authorization mechanism that can support multiple existing policies in a Grid system. The design and implementation details of GT4 authorization framework are also well discussed.

A Multipolicy Authorization Framework for Grid Security

A grid system is a virtual organization that is composed of several autonomous domains. Authorization in such a system needs to be flexible and scalable to support multiple security policies. Basing on the Web services security specifications such as XACML, SAML, and the special security needs of the grid computing, we have constructed an authorization framework in the Globus Toolkit 4 that can support multiple policies. This paper describes the concepts of our design and introduces the structure and the components of the authorization framework. To show the flexibility and scalability of the framework, we introduce a new blacklist/whitelist-based authorization mechanism that can be seamlessly integrated into the framework

Improving Utilization of Infrastructure Clouds

A key advantage of infrastructure-as-a-service (IaaS) clouds is providing users on-demand access to resources. To provide on-demand access, however, cloud providers must either significantly overprovision their infrastructure (and pay a high price for operating resources with low utilization) or reject a large proportion of user requests (in which case the access is no longer on-demand). At the same time, not all users require truly on-demand access to resources. Many applications and workflows are designed for recoverable systems where interruptions in service are expected. For instance, many scientists utilize high-throughput computing (HTC)-enabled resources, such as Condor, where jobs are dispatched to available resources and terminated when the resource is no longer available. We propose a cloud infrastructure that combines on-demand allocation of resources with opportunistic provisioning of cycles from idle cloud nodes to other processes by deploying backfill virtual machines (VMs). For demonstration and experimental evaluation, we extend the Nimbus cloud computing toolkit to deploy backfill VMs on idle cloud nodes for processing an HTC workload. Initial tests show an increase in IaaS cloud utilization from 37.5% to 100% during a portion of the evaluation trace but only 6.39% overhead cost for processing the HTC workload. We demonstrate that a shared infrastructure between IaaS cloud providers and an HTC job management system can be highly beneficial to both the IaaS cloud provider and HTC users by increasing the utilization of the cloud infrastructure (thereby decreasing the overall cost) and contributing cycles that would otherwise be idle to processing HTC jobs.

Virtual workspaces for scientific applications

Scientists often face the need for more computing power than is available locally, but are constrained by the fact that even if the required resources were available remotely, their complex software stack would not be easy to port to those resources. Many applications are dependency-rich and complex, making it hard to run them on anything but a dedicated platform. Worse, even if the applications do run on another system, the results they produce may not be consistent across different runs. As part of the Center for Enabling Distributed Petascale Science (CEDPS) project we have been developing the workspace service which allows authorized clients to dynamically provision execution environments, using virtual machine technology, on remote computers. Virtual machines provide an excellent implementation of a portable environment as they allow users to configure an environment and then deploy it on a variety of platforms. This paper describes a proof-of-concept of this strategy developed for the High-Energy and Nuclear Physics (HENP) applications such as STARs. We are currently building on this work to enable production STAR runs in virtual machines.