Krisztián Karóczkai

WS-PGRADE/gUSE Generic DCI Gateway Framework for a Large Variety of User Communities

The WS-PGRADE/gUSE generic DCI gateway framework has been developed to support a large variety of user communities. It provides a generic purpose, workflow-oriented graphical user interface to create and run workflows on various DCIs including clusters, Grids, desktop Grids and clouds. The framework can be used by NGIs to support small user communities who cannot afford to develop their own customized science gateway. The WS-PGRADE/gUSE framework also provides two API interfaces (Application Specific Module API and Remote API) to create application-specific science gateways according to the needs of different user communities. The paper describes in detail the workflow concept of WS-PGRADE, the DCI Bridge service that enables access to most of the popular European DCIs and the Application Specific Module and Remote API concepts to generate application-specific science gateways.

WS-PGRADE: Supporting parameter sweep applications in workflows

Despite majority of production level grid applications are either parameter studies or workflows some of them cannot fit into these categories and implement more complex structures. Such an application class was found in the CancerGrid project, where several parameter study simulation must be connected together into a workflow and must be executed on various production grids. The paper introduces the structure of parameter study based grid applications and describes the WS-PGRADE system which provides support for the development and execution of this category of applications. The data generator, data collector and data processor components of WS-PGRADE provide easy to understand, flexible, though very powerful mechanisms to express involved grid application structures. The grid connector layer of WS-PGRADE can distribute application components to Desktop Grids, service grids, Web Service networks and local resources, thus can find the most efficient location to execute calculations. By using WS-PGRADE, the members of the CancerGrid consortium are able to create and execute grid applications that consist of millions of jobs.

DCI Bridge: Executing WS-PGRADE Workflows in Distributed Computing Infrastructures

Solving distributed computing infrastructure (DCI) incompatibility issues in a generic way is a challenging and complex task. Gateways and workflow management systems are often tightly bound to some limited number of supported DCIs. To enable gateways access to many different DCIs and to solve DCI compatibility among the very different workflow management systems, we have developed a generic solution, the DCI Bridge. In this chapter we describe its internal architecture, provide usage scenario and show how the DCI Bridge resolves interoperability issues between various middleware-based DCIs. We also provide insight about the capabilities of the realized system. The generic DCI Bridge service seamlessly enables the execution of workflows (and jobs) on major DCI platforms such as ARC, Globus, gLite, UNICORE, SGE, PBS, as well as web services or clouds.

ENABLING GENERIC DISTRIBUTED COMPUTING INFRASTRUCTURE COMPATIBILITY FOR WORKFLOW MANAGEMENT SYSTEMS

Solving workflow management system’s Distributed Computing Infrastructure (DCI) incompatibility and their workflow interoperability issues are very challenging and complex tasks. Workflow management systems (and therefore their workflows, workflow developers and also their end-users) are bounded tightly to some limited number of supported DCIs, and efforts required to allow additional DCI support. In this paper we are specifying a concept how to enable generic DCI compatibility for grid workflow management systems (such as ASKALON, MOTEUR, gUSE/WS-PGRADE, etc.) on job and indirectly on workflow level. To enable DCI compatibility among the different workflow management systems we have developed the DCI Bridge software solution. In this paper we will describe its internal architecture, provide usage scenarios to show how the developed service resolve the DCI interoperability issues between various middleware types. The generic DCI Bridge service enables the execution of jobs onto the existing major DCI platforms (such as Service Grids (Globus Toolkit 2 and 4, gLite, ARC, UNICORE), Desktop Grids, Web services, or even cloud based DCIs).

Combined Health Monitoring and Emergency Management through Android Based Mobile Device for Elderly People

We have developed a combined Android based mobile data acquisition (DAQ) and emergency management solution, which can collect information remotely from patient and send the information towards to the medical data and dispatcher centre for further processing. The mobile device is capable to collect information from various sensors via Bluetooth and USB connection, and further more able to capture and forward manually initiated alarm signals in case of an emergency situation. Beside the alarm signal the system collects and sends information about the patient’s location, and it also enables two ways audio communication between the central dispatcher and the patient automatically. The developed software solution is suitable for different skilled users. Its user interface is highly configurable to support elderly persons (high contrast, huge characters, simple UI, etc.), and also provides advanced mode for the “power” users. The developed system becomes part of our testing program, which is carried out in our Hungarian Living Lab infrastructure. The combination of a mobile DAQ device and mobile emergency alarm device within a single software solution enables care givers to provide better and more effective services in elderly patient monitoring.

A Meta-Brokering Framework for Science Gateways

Recently scientific communities produce a growing number of computation-intensive applications, which calls for the interoperation of distributed infrastructures including Clouds, Grids and private clusters. The European SHIWA and ER-flow projects have enabled the combination of heterogeneous scientific workflows, and their execution in a large-scale system consisting of multiple Distributed Computing Infrastructures. One of the resource management challenges of these projects is called parameter study job scheduling. A parameter study job of a workflow generally has a large number of input files to be consumed by independent job instances. In this paper we propose a meta-brokering framework for science gateways to support the execution of such workflows. In order to cope with the high uncertainty and unpredictable load of the utilized distributed infrastructures, we introduce the so called resource priority services. These tools are capable of determining and dynamically updating priorities of the available infrastructures to be selected for job instances. Our evaluations show that this approach implies an efficient distribution of job instances among the available computing resources resulting in shorter makespan for parameter study workflows.

Dynamic workflow support in gUSE

Scientific workflow systems aim to provide user friendly, end-to-end solutions for automating and simplifying computational or data intensive tasks. A number of workflow environments have been developed in recent years to provide support for the specification and execution of scientific workflows. Normal static workflows can poorly cope with the ever changing status of the existing distributed systems. During workflow enactment unforeseen scenarios may arise, which can cause significant delays, failed executions, or improper results. Manual workflow enactment has its obvious limitations, however automatic failover mechanisms require minimum an accurate information set about the workflow tasks and about the status of the underlying processing infrastructure. Dynamism can be defined at different abstraction levels and in different phases of the workflow lifecycle. In this paper we identify the requirements of dynamic workflows in general and provide a thorough survey about gUSE/WS-PGRADEs dynamic workflow handling capabilities.

Brokering Solution for Science Gateways Using Multiple Distributed Computing Infrastructures

The ever growing number of computation-intensive applications calls for the interoperation of distributed infrastructures such as Clouds, Grids and private clusters. The European SHIWA and ER-flow projects have been initiated to enable the combination of heterogeneous scientific workflows, and to execute them in a large-scale system consisting of multiple Distributed Computing Infrastructures including Grids and Clouds. One of the resource management challenges of these projects called parameter study job scheduling. A parameter study job of a workflow has a large number of input files to be consumed by independent job instances. In this paper we propose a meta-brokering solution for these workflows to be applied in science gateways. To cope with the high uncertainty and unpredictable load of the utilized infrastructures, we rely on resource priority services. These tools are capable of determining and dynamically updating priorities of the available infrastructures to be selected for job instances.

Supporting health smart system applications in Scientific Gateway environment

Smart system application are gaining significant attention especially concerning health monitoring researches. Data captured from sensors are arriving continuously and for an efficient handling of this large volume of data stream processing would give the ideal solution [6]. Stream processing means that without intermediate storage the data should be processed as they are coming from the sensors. Storing and downloading data is a time-consuming mechanism, and during our investigation we have proved that data downloading time has significant impact on the overall execution of our health monitoring smart application. We have investigated the most common science gateway solution to examine the path that data takes from the outer storage to the executor node or the infrastructure and we have defined an execution model for smart system applications where the data movement is minimal.