Maciej Malawski

Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds

Large-scale applications expressed as scientific workflows are often grouped into ensembles of inter-related workflows. In this paper, we address a new and important problem concerning the efficient management of such ensembles under budget and deadline constraints on Infrastructure- as-aService (IaaS) clouds. We discuss, develop, and assess algorithms based on static and dynamic strategies for both task scheduling and resource provisioning. We perform the evaluation via simulation using a set of scientific workflow ensembles with a broad range of budget and deadline parameters, taking into account uncertainties in task runtime estimations, provisioning delays, and failures. We find that the key factor determining the performance of an algorithm is its ability to decide which workflows in an ensemble to admit or reject for execution. Our results show that an admission procedure based on workflow structure and estimates of task runtimes can significantly improve the quality of solutions.

Towards a grid management system for HLA-based interactive simulations

This paper presents the design of a system that supports execution of HLA (high level architecture) distributed interactive simulations in an unreliable grid environment. The design of the architecture is based on the OGSA (Open Grid Services Architecture) concept that allows for modularity and compatibility with grid services already being developed. First of all, we focus on the part of the system which is responsible for migration of a HLA-connected component or components of the distributed application in the grid environment. We present a runtime support library for easily plugging HLA simulations into the grid services framework. We also present the impact of execution management (namely migration) on overall system performance.

The Collage Authoring Environment

Abstract The Collage Authoring Environment is a software infrastructure which enables domain scientists to collaboratively develop and publish their work in the form of executable papers. It corresponds to the recent developments in both e-Science and computational technologies which call for a novel publishing paradigm. As part of this paradigm, static content (such as traditional scientific publications) should be supplemented with elements of interactivity, enabling reviewers and readers to reexamine the reported results by executing parts of the software on which such results are based as well as access primary scientific data. Taking into account the presented rationale we propose an environment which enables authors to seamlessly embed chunks of executable code (called assets) into scientific publications and allow repeated execution of such assets on underlying computing and data storage resources, as required by scientists who wish to build upon the presented results. The Collage Authoring Environment can be deployed on arbitrary resources, including those belonging to high performance computing centers, scientific e-Infrastructures and resources contributed by the scientists themselves. The environment provides access to static content, primary datasets (where exposed by authors) and executable assets. Execution features are provided by a dedicated engine (called the Collage Server) and embedded into an interactive view delivered to readers, resembling a traditional research publication but interactive and collaborative in its scope. Along with a textual description of the Collage environment the authors also present a prototype implementation, which supports the features described in this paper. The functionality of this prototype is discussed along with theoretical assumptions underpinning the proposed system.

Workflow composer and service registry for grid applications

Automatic composition of workflows from Web and Grid services is an important challenge in todays distributed applications. The system presented in this paper supports the user in composing an application workflow from existing Grid services. The flow composition system builds workflows on an abstract level with semantic and syntactic descriptions of services available on the Grid. Two main modules of the system are the flow composer and the distributed Grid service registry. We present motivation, the concept of the overall system architecture and the results of a feasibility study.

Exploratory programming in the virtual laboratory

GridSpace 2 is a novel virtual laboratory framework enabling researchers to conduct virtual experiments on Grid-based resources and other HPC infrastructures. GridSpace 2 facilitates exploratory development of experiments by means of scripts which can be written in a number of popular languages, including Ruby, Python and Perl. The framework supplies a repository of gems enabling scripts to interface low-level resources such as PBS queues, EGEE computing elements, scientific applications and other types of Grid resources. Moreover, GridSpace 2 provides a Web 2.0-based Experiment Workbench supporting development and execution of virtual experiments by groups of collaborating scientists. We present an overview of the most important features of the Experiment Workbench, which is the main user interface of the Virtual laboratory, and discuss a sample experiment from the computational chemistry domain.

MOCCA - towards a distributed CCA framework for metacomputing

We describe the design and implementation of MOCCA, a distributed CCA framework implemented using the H2O metacomputing system. Motivated by the quest for appropriate metasystem programming models for large scale scientific applications, MOCCA combines the advantages of component orientation with the flexible and reconfigurable H2O middleware. By exploiting unique capabilities in H2O, including client-provider separation, security, and negotiable transport protocols, enhancements to both functionality and performance could be attained. The design and implementation of MOCCA highlights the natural match between CCA components and H2O pluglets, both in structure and invocation methodology. An outline of how native CCA modules can be supported in the MOCCA framework describes the potential for future deployment of legacy codes on metacomputing systems. We also report on preliminary experiences with test applications and sample performance measurements that favorably compare MOCCA to alternative component frameworks for tightly- and loosely-coupled metacomputing systems.

Component Approach to Computational Applications on Clouds

Abstract Running computational science applications on the emerging cloud infrastructures requires appropriate programming models and tools. In this paper we investigate the applicability of the component model to developing such applications. The component model we propose takes advantages of the features of the IaaS infrastructure and offers a high-level application composition API. We describe experiments on a scientific application from the bioinformatics domain, using a hybrid cloud infrastructure which consists of a private cloud running Eucalyptus and the Amazon EC2 public cloud. The measured performance of virtual machine startup time and virtualization overhead indicate promising prospects for exploiting such infrastructures along with the proposed component-based approach.

A Framework for HLA-Based Interactive Simulations on the Grid

This article presents the design and feasibility of a system that supports execution of High Level Architecture (HLA)-distributed interactive simulations in an unreliable Grid environment. The article presents an overall architecture of a system based on experience gained from previous designs. The most important operational components of the system are presented, and actual performance issues are discussed. The design of the architecture is based on the Open Grid Services Architecture (OGSA) concept that allows for modularity and compatibility with Grid services already being developed. The issue of migration to the recently proposed Web Services Resource Framework (WSRF) is discussed as well.

How to Use Google App Engine for Free Computing

Can the Google App Engine cloud service be used, free of charge, to execute parameter study problems? That question drove this research, which is founded on the App Engines newly developed Task Queue API. The authors created a simple and extensible framework implementing the master-worker model to enable usage of the App Engine application servers as computational nodes. This article presents and discusses the results of the feasibility study, as well as compares the solution with EC2, Amazons free cloud offering.

Managing entire lifecycles of e-science applications in the gridspace2 virtual laboratory --- from motivation through idea to operable web-accessible environment built on top of PL-Grid e-infrastructure

The GridSpace2 environment, developed in the scope of the PL-Grid Polish National Grid Initiative, constitutes a comprehensive platform which supports e-science applications throughout their entire lifecycle. Application development may involve multiple phases, including writing, prototyping, testing and composing the application. Once the application attains maturity it becomes operable and capable of being executed, although it may still be subject to further development --- including actions such as sharing with collaborating researchers or making results publicly available with the use of dedicated publishing interfaces. This paper describes each of these phases in detail, showing how the GridSpace2 platform can assist the developers and publishers of computational experiments.