Diane Lingrand

Flexible and Efficient Workflow Deployment of Data-Intensive Applications On Grids With MOTEUR

Workflows offer a powerful way to describe and deploy applications on grid infrastructures. Many workflow management systems have been proposed but there is still a lack of a system that would allow both a simple description of the dataflow of the application and an efficient execution on a grid platform. In this paper, we study the requirements of such a system, underlining the need for well-defined data composition strategies on the one hand and for a fully parallel execution on the other. As combining those features is not straightforward, we then propose algorithms to do so and we describe the design and implementation of MOTEUR, a workflow engine that fulfills those requirements. Performance results and overhead quantification are shown to evaluate MOTEUR with respect to existing comparable workflow systems on a production grid.

A Service-Oriented Architecture enabling dynamic service grouping for optimizing distributed workflow execution

In this paper, we describe a Service-Oriented Architecture allowing the optimization of the execution of service workflows. We discuss the advantages of the service-oriented approach with regard to the enactment of scientific applications on a grid infrastructure. Based on the development of a generic Web-Services wrapper, we show how the flexibility of our architecture enables dynamic service grouping for optimizing the application execution time. We demonstrate performance results on a real medical imaging application. On a production grid infrastructure, the optimization proposed introduces a significant speed-up (from 1.2 to 2.9) when compared to a traditional execution.

A Secure Grid Medical Data Manager Interfaced to the gLite Middleware

The medical community is producing and manipulating a tremendous volume of digital data for which computerized archiving, processing and analysis is needed. Grid infrastructures are promising for dealing with challenges arising in computerized medicine but the manipulation of medical data on such infrastructures faces both the problem of interconnecting medical information systems to Grid middlewares and of preserving patients’ privacy in a wide and distributed multi-user system. These constraints are often limiting the use of Grids for manipulating sensitive medical data. This paper describes our design of a medical data management system taking advantage of the advanced gLite data management services, developed in the context of the EGEE project, to fulfill the stringent needs of the medical community. It ensures medical data protection through strict data access control, anonymization and encryption. The multi-level access control provides the flexibility needed for implementing complex medical use-cases. Data anonymization prevents the exposure of most sensitive data to unauthorized users, and data encryption guarantees data protection even when it is stored at remote sites. Moreover, the developed prototype provides a Grid storage resource manager (SRM) interface to standard medical DICOM servers thereby enabling transparent access to medical data without interfering with medical practice.

Modeling user submission strategies on production grids

Production-grid users experience many system faults as well as high and variable latencies due to the scale, complexity and sharing of such infrastructures. To improve performance, they adopt different submission strategies, that are potentially aggressive for the infrastructure. This work studies the impact of three different strategies. It is based on a probabilistic modeling of these strategies which are evaluated according to their performance, measured as the reduction of the latency expectation, and the infrastructure overhead, measured as the additional number of submitted jobs. A strategy cost criterion is then derived. Experiments are performed using real workload traces collected from the EGEE production infrastructure. Under these conditions, a good balance between high performance and low overhead can be found.

Impact of the execution context on Grid job performances

In this paper, we examine how the execution context of grid jobs can help to refine submission strategies on a production grid. On this kind of infrastructure, the latency highly impacts performances. We present experiments that quantify the dependencies between the grid latency and both internal and external context parameters on the EGEE grid infrastructure. We show how job submission managers, job execution sites and the submission date can be statistically correlated to grid performances.

Data composition patterns in service-based workflows

Workflow engines are powerful tools to implement data-intensive scientific applications exploiting parallel grid resources transparently. We discuss the advantages of implementing applications as workflows of services when dealing with large data sets. We show how the graph of services associated with data composition operators enable the description of complex data flows in a very compact format. We define a strict semantics for two common composition operators and we propose an algorithm to consistently satisfy this semantics all along the workflow execution. Finally, we show how our approach enables parallel execution of the application while preserving the data flow semantics. We implemented the algorithm proposed in MOTEUR, an open source workflow engine designed to execute parallel and data-intensive applications.

Optimization of jobs submission on the EGEE production grid: modeling faults using workload

It is commonly observed that production Grids are inherently unreliable. The aim of this work is to improve Grid application performances by tuning the job submission system. A stochastic model, capturing the behavior of a complex Grid workload management system is proposed. To instantiate the model, detailed statistics are extracted from dense Grid activity traces. The model is exploited for optimizing a simple job resubmission strategy. It provides quantitative inputs to improve job submission performance and it enables the impact of faults and outliers on Grid operations to be quantified.

Analyzing the EGEE Production Grid Workload: Application to Jobs Submission Optimization

Grids reliability remains an order of magnitude below clusters on production infrastructures. This work is aims at improving grid application performances by improving the job submission system. A stochastic model, capturing the behavior of a complex grid workload management system is proposed. To instantiate the model, detailed statistics are extracted from dense grid activity traces. The model is exploited in a simple job resubmission strategy. It provides quantitative inputs to improve job submission performance and it enables quantifying the impact of faults and outliers on grid operations.

jGASW: A Service-Oriented Framework Supporting HTC and Non-functional Concerns

Although Service-Oriented principles have been widely adopted by High Throughput Computing infrastructure designers, the integration between SOA and HTC is made difficult by legacy. jGASW is a framework for wrapping legacy scientific applications as Web Services and integrating them into an intensive computing-aware SOA framework. It maps complex I/O data structures to command lines and enables dynamic allocation of computing resources; including execution on local hosts or on grid infrastructures; data transfer management and support of non-functional concerns.

A levelset based method for segmenting the heart in 3D+T gated SPECT images

Levelset methods were introduced in medical images segmentation by Malladi et al. in 1995. In this paper, we propose several improvements of the original method to speed up the algorithm convergence and to improve the quality of the segmentation in the case of cardiac gated SPECT images. We studied several evolution criterions, taking into account the dynamic property of heart image sequences. For each step of the segmentation algorithm, we have compared different solutions in order to both reduce time and improve quality. We have developed a modular segmentation tool with 3D+T visualization capabilities to experiment the proposed solutions and tune the algorithm parameters. We show segmentation results on both simulated and real SPECT images.