Bartek Palak

Approaches to Distributed Execution of Scientific Workflows in Kepler

The Kepler scientific workflow system enables creation, execution and sharing of workflows across a broad range of scientific and engineering disciplines while also facilitating remote and distributed execution of workflows. In this paper, we present and compare different approaches to distributed execution of workflows using the Kepler environment, including a distributed data-parallel framework using Hadoop and Stratosphere, and Cloud and Grid execution using Serpens, Nimrod/K and Globus actors. We also present real-life applications in computational chemistry, bioinformatics and computational physics to demonstrate the usage of different distributed computing capabilities of Kepler in executable workflows. We further analyze the differences of each approach and provide a guidance for their applications.

Design and first applications of the ITER integrated modelling & analysis suite

The ITER Integrated Modelling & Analysis Suite (IMAS) will support both plasma operation and research activities on the ITER tokamak experiment. The IMAS will be accessible to all ITER members as a key tool for the scientific exploitation of ITER. The backbone of the IMAS infrastructure is a standardized, machine-generic data model that represents simulated and experimental data with identical structures. The other outcomes of the IMAS design and prototyping phase are a set of tools to access data and design integrated modelling workflows, as well as first plasma simulators workflows and components implemented with various degrees of modularity.

Scientific workflow orchestration interoperating HTC and HPC resources

Abstract In this work we describe our developments towards the provision of a unified access method to different types of computing infrastructures at the interoperation level. For that, we have developed a middleware suite which bridges not interoperable middleware stacks used for building distributed computing infrastructures, UNICORE and gLite. Our solution allows to transparently access and operate on HPC and HTC resources from a single interface. Using Kepler as workflow manager, we provide users with the needed integration of codes to create scientific workflows accessing both types of infrastructures.

Mobile Work Environment for Grid Users

In this paper we aim to describe the project developing the Migrating Desktop infrastructure for the mobile users in a simple way. Nowadays the easy access to the grid resources is the crucial to the users’ satisfaction. Using our framework the user achieves comfortable working environment that could be customised, stored and recreated everywhere where the Internet is present...

Tools, Methods and Services Enhancing the Usage of the Kepler-based Scientific Workflow Framework

Abstract Scientific workflow systems are designed to compose and execute either a series of computational or data manipulation steps, or workflows in a scientific application. They are usually a part of a larger eScience environment. The usage of workflow systems, however very beneficial, is mostly not irrelevant for scientists. There are many requirements for additional functionalities around scientific workflows systems that need to be taken into account, like the ability of sharing workflows, provision of the user-friendly GUI tools for automation of some tasks or submission to distributed computing infrastructures, etc. In this paper we present tools developed in response to the requirements of three different scientific communities. These tools simplify and empower their work with the Kepler scientific workflow system. The usage of such tools and services is presented on Nanotechnology, Astronomy and Fusion scenarios examples.

Running simultaneous Kepler sessions for the parallelization of parametric scans and optimization studies applied to complex workflows

Abstract In this paper we present an approach taken to run multiple Kepler sessions at the same time. This kind of execution is one of the requirements for Integrated Tokamak Modelling (ITM) platform developed by the Nuclear Fusion community within the context of EUROFusion project [1] . The platform is unique and original: it entails the development of a comprehensive and completely generic tokamak simulator including both the physics and the machine, which can be applied for any fusion device. All components are linked inside workflows. This approach allows complex coupling of various algorithms while at the same time provides consistency. Workflows are composed of Kepler and Ptolemy II elements as well as set of the native libraries written in various languages (Fortran, C, C++). In addition to that, there are Python based components that are used for visualization of results as well as for pre/post processing. At the bottom of all these components there is a database layer that may vary between software releases, and require different version of access libraries. The community is using a shared virtual research environment to prepare and execute workflows. All these constraints make running multiple Kepler sessions really challenging. However, ability to run numerous sessions in parallel is a must – to reduce computation time and to make it possible to run released codes while working with new software at the same time. In this paper we present our approach to solve this issue and examples that show its correctness.

User-Friendly frameworks for accessing computational resources

The grid technology, even though it gives great computational power and greatly improves the manner of exploitation of resources, also has its disadvantages. Grid systems, due to their distribution and heterogeneity, are very complex and hard to access and oversee. Thus, one of the research fields designs tools and technologies that give easy, secure and consistent access to grid applications and resources as well as seamless interoperation between various computing environments. The lack of user-friendly tools is especially annoying in cases when users need to access computing resources of different grid infrastructures. Similar problems are experienced by grid application developers who should focus on applications themselves and not on their interoperation with different grid middleware. In this article we describe the concept of an abstract grid model and its implementation in two user-friendly frameworks --- Migrating Desktop and g-Eclipse. They both are intuitive graphical environments that provide: easy access to heterogeneous resources and seamless interoperation of underlying middleware solutions. Although the two products provide similar functionalities they are complementary to each other and target different user groups. The method of integration of scientific applications with both frameworks was also presented.

High level tools for fusion simulations workflows in distributed computing environment

This paper describes chosen aspects of the European Integrated Tokamak Modelling Task Force (ITM-TF) effort. It covers topics related to latest developments towards providing fusion scientists with a software framework for fusion modeling and simulation. The proposed framework provides researchers with an easy to use, workflow based environment for scientific computations. It allows the execution of various applications in numerous distributed computational resources directly from a user friendly, intuitive interface. We present the latest advances in the development of easy to use tools allowing the integration of the physics codes into the scientific workflows that schedule jobs to different infrastructures. The presented approach provides a simple way to access extensive computational resources and hides the complexity of the process at the same time. It allows for easy extension of existing simulation scenarios and fast prototyping. Some aspects of interactive communication between workflow environment and running jobs are discussed. Results of two practical use cases are presented in the context of the discussed workflow based solution: a parametric study and a plasma turbulence workflow.

Mobile work environment for grid users: grid applications' framework

In this article we aim to describe a project developing of the Migrating Desktop infrastructure for mobile users. This functionality refers to the work environment of the users who very often change their location. The user interface called the Migrating Desktop, or grid desktop, is a very useful environment that accomplishes an integrated set of services and real applications, which could be run on the grid. We introduce a framework for improving the grid application launching. The possibility of usage of interactive application in developed environment is presented.

Best Practices in Debugging Kepler Workflows

In this paper we present various techniques related to Kepler development, debugging, and JVM customisation. We highlight some aspects of development process that may help people to perform better while working with Kepler (especially in case they develop new components for the Kepler platform). We present knowledge and ideas that were gained over the time while working with Kepler tools throughout various projects and different applications of Kepler into existing environments. These ideas are presented for the sake of saving time and effort by other people who just start their experience with Kepler project.