Gergely Sipos

Multi-Grid, multi-user workflows in the P-GRADE Grid portal

Computational Grids connect resources and users in a complex way in order to deliver nontrivial qualities of services. According to the current trend various communities build their own Grids and due to the lack of generally accepted standards these Grids are usually not interoperable. As a result, large scale sharing of resources is prevented by the isolation of Grid systems. Similarly, people are isolated, because the collaborative work of Grid users is not supported by current environments. Each user accesses Grids as an individual person without having the possibility of organizing teams that could overcome the difficulties of application development and execution more easily. The paper describes a new workflow-oriented portal concept that solves both problems. It enables the interoperability of various Grids during the execution of workflow applications, and supports users to develop and run their Grid workflows in a collaborative way. The paper also introduces a classification model that can be used to identify workflow-oriented Grid portals based on two general features: Ability to access multiple Grids, and support for collaborative problem solving. Using the approach the different potential portal types are introduced, their unique features are discussed and the portals and Problem Solving Environments (PSE) of our days are classified. The P-GRADE Portal as a Globus-based implementation for the classification model is also presented.

Solving the grid interoperability problem by P-GRADE portal at workflow level

Grid interoperability has recently become a major issue at Grid forums. Most of the current ideas try to solve the problem at the middleware level where unfortunately too many components (information system, broker, etc.) should be made interoperable. As an alternative concept the P-GRADE portal is the first Grid portal that tries to solve the problem at the level of workflows. It means that the components of a workflow can be executed simultaneously in several Grids. In this way the user can exploit more parallelism than inside one Grid. More than that the workflow level completely hides the low level Grid details for the end-user who does not have to learn the low level Grid commands of different Grids. In this way porting workflow application between different Grids can be done with minimal user efforts. The paper describes those features and techniques that are provided and used by the P-GRADE portal to solve the Grid interoperability problem.

WS-PGRADE: Supporting parameter sweep applications in workflows

Despite majority of production level grid applications are either parameter studies or workflows some of them cannot fit into these categories and implement more complex structures. Such an application class was found in the CancerGrid project, where several parameter study simulation must be connected together into a workflow and must be executed on various production grids. The paper introduces the structure of parameter study based grid applications and describes the WS-PGRADE system which provides support for the development and execution of this category of applications. The data generator, data collector and data processor components of WS-PGRADE provide easy to understand, flexible, though very powerful mechanisms to express involved grid application structures. The grid connector layer of WS-PGRADE can distribute application components to Desktop Grids, service grids, Web Service networks and local resources, thus can find the most efficient location to execute calculations. By using WS-PGRADE, the members of the CancerGrid consortium are able to create and execute grid applications that consist of millions of jobs.

Workflow-Oriented collaborative grid portals

The paper presents how workflow-oriented, single-user Grid portals could be extended to meet the requirements of users with collaborative needs. Through collaborative Grid portals different research and engineering teams would be able to share knowledge and resources. At the same time the workflow concept assures that the shared knowledge and computational capacity is aggregated to achieve the high-level goals of the group. The paper discusses the different issues collaborative support requires from Grid portal environments during the different phases of the workflow-oriented development work. While in the design period the most important task of the portal is to provide consistent and fault tolerant data management, during the workflow execution it must act upon the security framework its back-end Grids are built on.

Brokering multi-grid workflows in the P-GRADE portal

Grid computing has gone through some generations and as a result only a few widely used middleware architectures remain. The Globus Toolkit is the most widespread middleware in most of the current production grid systems, but the LCG-2 middleware dominates in Europe. The paper describes a brokering solution that enables the interoperability of various Globus and LCG-2 based grids during the execution of workflow applications, and supports users to utilize computing and storage resources from multiple production grids by a single application. The development and execution of such applications can be managed by a Web-based Grid portal called P-GRADE Portal, and the brokering of the workflows is carried out by its integrated GTbroker and LCG-2 broker component.

Workflow-Level Parametric Study Support by MOTEUR and the P-GRADE Portal

Many large-scale scientific applications require the processing of complete data sets made of individual data segments that can be manipulated independently following a single analysis procedure. Workflow managers have been designed for describing and controlling such complex application control flows. However, when considering very data-intensive applications, there is a large potential parallelism that should be properly exploited to ensure efficient processing. Distributed systems such as Grid infrastructures are promising for handling the load resulting from parallel data analysis and manipulation. Workflow managers can help in exploiting the infrastructure parallelism, given that they are able to handle the data flow resulting from the application’s execution.

Implementation of the ABC Quantum Mechanical Reactive Scattering Program on the EGEE Grid Platform

Computer based simulation of quantum mechanical reactive scattering is a CPU intensive process. Despite the small I/O traffic a single simulation requires several thousand CPU hours. The ABC program provides an impelementation for quantum mechanical reaction simulation in such a way that the code can be efficiently ported to parallel computing platforms. The Computational Chemistry and Application Porting Support groups of the EGEE project worked together to create a grid enabled version of the ABC code. The collaboration resulted a Grid application that is capable of using several clusters and storage servers of the EGEE Grid symultaneously, achieving significant speed-up. The application has been ported to EGEE Grid as a parameter study application with the P-GRADE Grid portal. The paper describes the application porting process, the technical analysis and performance of the local and the Grid enabled ABC application.

Classification and implementations of workflow-oriented grid portals

Although several grid projects have the aim to develop portals that hide the complexity of distributed computing infrastructures, only few of these portals support workflow-based applications. The number of portals that enable collaborative workflow development or collaborative resource access is even lower. The paper introduces a model to categorize workflow-oriented grid portals based on two general features: ability to access multiple grids simultaneously and support for collaborative problem solving. The generic features of the different categories are discussed and the Globus-based implementations of the various categories are demonstrated by the P-GRADE Portal.

The collaborative P-GRADE grid portal

Grid portals are increasingly used to provide uniform access to the grid infrastructure. This paper describes how the P-GRADE Grid Portal could be used in a collaborative manner to facilitate group work and support the notion of Virtual Organisations. We describe the development issues involved in the construction of a collaborative portal, including ensuring a consistent view between participants of a collaborative workflow and management of proxy credentials to allow separate nodes of the workflow to be submitted to different grids.

Legacy code support for production grids

In order to improve reliability and to deal with the high complexity of existing middleware solutions, todays production grid systems restrict the services to be deployed on their resources. On the other hand end-users require a wide range of value added services to fully utilize these resources. This paper describes a solution how legacy code support is offered as third party service for production grids. The introduced solution, based on the grid execution management for legacy code architecture (GEMLCA), do not require the deployment of additional applications on the grid resources, or any extra effort from grid system administrators. The implemented solution was successfully connected to and demonstrated on the UK National Grid Service.