Ahmed Shiraz Memon

CIS: An Information Service Based on the Common Information Model

The provision and processing of information in an e- Science environment are essential tasks. For this purpose, most environments provide information services which aggregate data from different information sources and make it available to users and other services. In this paper we present CIS, an extensible information service with an underlying unified information model. Designed according to service-oriented architectural principles, CIS consumes data from sources like Ganglia, formats it according to the Common Information Model, and delivers it against XQuery requests. We realised the information service in a Web Services environment and integrated it into an implied volatility application within the NextGRID project and the UNICORE middleware.

Exploring the Potential of Using Multiple E-science Infrastructures with Emerging Open Standards-Based E-health Research Tools

E-health makes use of information and communication methods and the latest e-research tools to support the understanding of body functions. E-scientists in this field take already advantage of one single infrastructure to perform computationally-intensive investigations of the human body that tend to consider each of the constituent parts separately without taking into account the multiple important interactions between them. But these important interactions imply an increasing complexity of applications that embrace multiple physical models (i.e. multi-physics) and consider a larger range of scales (i.e. multi-scale) thus creating a steadily growing demand for interoperable infrastructures that allow for new innovative application types of jointly using different infrastructures for one application. But interoperable infrastructures are still not seamlessly provided and we argue that this is due to the absence of a realistically implementable infrastructure interoperability reference model that is based on lessons learned from e-science usage. Therefore, the goal of this paper is to explore the potential of using multiple infrastructures for one scientific goal with a particular focus on e-health. Since e-scientists gain more interest in using multiple infrastructures there is a clear demand for interoperability between them to enable a use with one e-research tool. The paper highlights work in the context of an e-Health blood flow application while the reference model is applicable to other e-science applications as well.

e-Science Infrastructure Integration Invariants to Enable HTC and HPC Interoperability Applications

During the past decade, significant international and broader interdisciplinary research is increasingly carried out by global collaborations that often share resources within a single production e-science infrastructure. More recently, increasing complexity of e-science applications embrace multiple physical models (i.e. multi-physics) and consider longer and more detailed simulation runs as well as a larger range of scales (i.e. multi-scale). This increase in complexity is creating a steadily growing demand for cross-infrastructure operations that take the advantage of multiple e-science infrastructures with a more variety of resource types. Since interoperable e-science infrastructures are still not seamlessly provided today we proposed in earlier work the Infrastructure Interoperability Reference Model (IIRM) that represents a trimmed down version of the Open Grid Service Architecture (OGSA) in terms of functionality and complexity, while on the other hand being more specifically useful for production and thus easier to implement. This contribution focuses on several important reference model invariants that are often neglected when infrastructure integration activities are being performed thus hindering seamless interoperability in many aspects. In order to indicate the relevance of our invariant definitions, we provide insights into two accompanying cross-infrastructure use cases of the bio-informatics and fusion science domain.

Lessons learned from jointly using HTC- and HPC-driven e-Science infrastructures in fusion science

The interoperability of e-Science infrastructures like DEISA/PRACE and EGEE/EGI is an increasing demand for a wide variety of cross-Grid applications, but interoperability based on common open standards adopted by Grid middleware is only starting to emerge and is not broadly provided today. In earlier work, we have shown how refined open standards form a reference model, which is based on careful academic analysis of lessons learned obtained from production cross-Grid applications that require access to both, High Throughput Computing (HTC) resources as well as High Performance Computing (HPC) resources. This paper provides insights in several concepts of this reference model with a particular focus on the finding of using HPC and HTC resources with the fusion applications BIT1 and a cross-infrastructure workflow based on the HELENA and ILSA fusion applications. Based on lessons learned over years gained with production interoperability setups and experimental interoperability work between production Grids like EGEE, DEISA, and NorduGrid, we illustrate how open Grid standards (e.g. OGSA-BES, JSDL, GLUE2, etc) can be used to overcome several limitations of the production architecture of the EUFORIA framework paving the way to a more standards-based and thus more maintainable and efficient solution.

Improvements of common open Grid standards to increase High Throughput and High Performance Computing effectiveness on large-scale Grid and e-science infrastructures

Grid and e-science infrastructure interoperability is an increasing demand for Grid applications but interoperability based on common open standards adopted by Grid middle-wares are only starting to emerge on Grid infrastructures and are not broadly provided today. In earlier work we have shown how open standards can be improved by lessons learned from cross-Grid applications that require access to both, High Throughput Computing (HTC) resources as well as High Performance Computing (HPC) resources. This paper provides more insights in several concepts with a particular focus on effectively describing Grid job descriptions in order to satisfy the demands of e-scientists and their cross-Grid applications. Based on lessons learned over years gained with interoperability setups between production Grids such as EGEE, DEISA, and NorduGrid, we illustrate how common open Grid standards (i.e. JSDL and GLUE2) can take cross-Grid application experience into account.

EMIR: An EMI Service Registry for Federated Grid Infrastructures

The European Middleware Initiative(EMI) is a European proj ect that represents a collaboration of four middlewares namely ARC, dCache, gLite, and UNICORE. All these middleware services should be easily deployable in a Grid Infrastructure. Howev er the immediate challenge is the discovery of those services in a particular infrastructure hat is typically done via so-called registries. This is a major requirement for operational system , and the middleware itself. Existing registries such as ARC Information Index or UNICORE registr y a e designed to index middleware specific services. Given the centralized nature, the sc ope of these registries can become limited when considering a federated infrastructure that r elies on service of different technology providers. Distributed Grid infrastructures such as EGI ar e be federated, therefore a unified registry should reflect this requirement. In this paper, a commo n registry EMIR is proposed, which attempts to overcome the challenges of federation, robustn es , and performance implications of ever expanding Grids.

Enhancing UNICORE storage management using hadoop distributed file system

UNICORE is a state of the art and well tested Grid middleware, designed for seamless and secure access to distributed resources, applications and data, in an easy to use fashion. A wide variety of UNICORE applications for example in bio-informatics generate and compute huge amounts of data. These large amounts of data are not easy to manage reliably and efficiently with the default UNICORE storage system which is using a standard file system. Hence, the current UNICORE does not support a scalable distributed storage system so far. We have integrated Apache Hadoop and its supported distributed storage/ file systems into the UNICORE storage management service. Thus allows to build a UNICORE storage system providing data replication, disaster recovery, durability and elasticity. In this paper we will present the architecture and operation of a prototype called UniHadoop, which provides the integration of UNICORE and the distributed storage systems (DSS) supported by Hadoop and highlight its potential in usage scenarios.

Enhanced resource management capabilities using standardized job management and data access interfaces within UNICORE Grids

Many existing Grid technologies and resource management systems lack a standardized job submission interface in Grid environments or e-Infrastructures. Even if the same language for job description is used, often the interface for job submission is also different in each of these technologies. The evolvement of the standardized Job Submission and Description Language (JSDL) as well as the OGSA - Basic Execution Services (OGSA-BES) pave the way to improve the interoperability of all these technologies enabling cross-Grid job submission and better resource management capabilities. In addition, the BytelO standards provide useful mechanisms for data access that can be used in conjunction with these improved resource management capabilities. This paper describes the integration of these standards into the recently released UNICORE 6 Grid middleware that is based on open standards such as the Web Services Resource Framework (WS-RF) and WS-Addressing (WS-A).

Federated Authentication and Credential Translation in the EUDAT Collaborative Data Infrastructure

One of the challenges in a distributed data infrastructure is how users authenticate to the infrastructure, and how their authorisations are tracked. Each user community comes with its own established practices, all different, and users are put off if they need to use new, difficult tools. From the perspective of the infrastructure project, the level of assurance must be high enough, and it should not be necessary to reimplement an authentication and authorisation infrastructure (AAI). In the EUDAT project, we chose to implement a mostly loosely coupled approach based on the outcome of the Contrail and Unicore projects. We have preferred a practical approach, combining the outcome of several projects who have contributed parts of the puzzle. The present paper aims to describe the experiences with the integration of these parts. Eventually, we aim to have a full framework which will enable us to easily integrate new user communities and new services.

Supporting interoperability of e-science infrastructures with a common information service

The heterogeneity of Grid resources (e.g. compute, storage, and network) that are geographically dispersed creates a fundamental challenge in terms of their discovery and monitoring process in production Grids today. In order to overcome these challenges, a wide variety of information models and services are developed that provide proprietary solutions in rather complex ways. Many of those services lack simplicity with respect to the setup, resource registration, maintenance, and security, thus making the information system complex and hard to use, especially in Grids driven by High Performance Computing (HPC) needs such as DEISA. In this paper we present a Common Information Service (CIS), which represents an information service, which aims to provide the functionality required to seamlessly aggregate and expose Grid resource information from various heterogeneous resources. The contribution of this paper is about using the emerging OGF GLUE2 standard information model for structuring Grid resource information in order to achieve the interoperability with other Grid technologies in general and other information systems in particular. Using this standard, our contribution provides insights of how CIS publishes and discovers resource and service information in the context of Web services based UNICORE Grids (e.g. DEISA, D-Grid) and highlights example deployments of the D-Mon project. Since the availability of up-to-date information is essential for Grid interoperability, we furthermore provide insights of how CIS can be used to contribute to cross-Grid applications based on the Infrastructure Interoperability Reference Model (IIRM).