Hiroshi Takemiya

Design, implementation and performance evaluation of GridRPC programming middleware for a large-scale computational grid

This paper reports on the design, implementation and performance evaluation of a suite of GridRPC programming middleware called Ninf-G Version 2 (Ninf-G2). Ninf-G2 is a reference implementation of the GridRPC API, a proposed GGF standard. Ninf-G2 has been designed so that it provides 1) high performance in a large-scale computational Grid, 2) the rich functionalities which are required to adapt to compensate for the heterogeneity and unreliability of a Grid environment, and 3) an API which supports easy development and execution of Grid applications. Ninf-G2 is implemented to work with basic Grid services, such as GSI, GRAM, and MDS in the Globus Toolkit version 2. The performance ofNinf-G2 was evaluated using a weather forecasting system which was developed using Ninf-G2. The experimental results indicate that high performance can be attained even in relatively fine-grained task-parallel applications on hundreds of processors in a Grid environment.

Sustainable adaptive grid supercomputing: multiscale simulation of semiconductor processing across the pacific

We propose a reservation-based sustainable adaptive grid supercomputing paradigm to enable tightly coupled computations of considerable scale (involving over 1,000 processors) and duration (over tens of continuous days) on a grid of geographically distributed parallel supercomputers. The paradigm is demonstrated for an adaptive multiscale simulation application, in which accurate but compute-intensive quantum mechanical (QM) simulations are embedded within a classical molecular dynamics (MD) simulation only when and where high fidelity is required. Key technical innovations include: 1) an embedded divide-and-conquer algorithmic framework to maximally expose data and computation localities for enhanced scalability; 2) a buffered-cluster hybridization scheme to adaptively adjust MD/QM boundaries to maintain the model accuracy; and 3) a hybrid grid remote procedure call (GridRPC) + message passing interface (MPI) grid application framework to combine flexibility (adaptive resource allocation and migration), fault tolerance (automated fault recovery), and efficiency (scalable management of large computing resources). We have achieved an automated execution of multiscale MD/QM simulation on a Grid consisting of 6 supercomputer centers in Japan and the US (in total of 150 thousand processor hours) for the dynamic simulation of implanted oxygen atoms in a silicon substrate, in which the number of processors changes dynamically on demand and resources are allocated and migrated dynamically according to both reservations and unexpected faults. The simulation results reveal a strong dependence of the oxygen penetration depth on the incident oxygen-beam position, which is useful information to further advance SIMOX (separation by implanted oxygen) technique to fabricate high speed and low power-consumption semiconductor devices

Deploying Scientific Applications to the PRAGMA Grid Testbed: Strategies and Lessons

Recent advances in grid infrastructure and middleware development have enabled various types of applications in science and engineering to be deployed on the grid. The characteristics of these applications and the diverse infrastructure and middleware solutions developed, utilized or adapted by PRAGMA member institutes are summarized. The applications include those for climate modeling, computational chemistry, bioinformatics and computational genomics, remote control of instruments, and distributed databases. Many of the applications are deployed to the PRAGMA grid testbed in routine basis experiments. Strategies for deploying applications without modifications, and those taking advantage of new programming models on the grid are explored and valuable lessons learned are reported. Comprehensive end to end solutions from PRAGMA member institutes that provide important grid middleware components and generalized models of integrating applications and instruments on the grid are also described.

Constructing Grid Applications Using Standard Grid Middleware

The effectiveness of standard Grid middleware has been evaluated through the work of “gridifying” a legacy program. As a case study, we have gridified a typical parameter survey program called barotropic S-model which aims to predict short- to middle-term weather change accurately. Ninf-G was used to gridify the system. It was found that the program could be easily gridified using Ninf-G without worrying about the complicated structure of the Grid itself. Performance was measured on the ApGrid Testbed which spreads over Pan-Pacific countries. Although large costs of initialization and termination hinder the application from attaining good performance, these costs can be reduced by optimizing the middleware as well as the application itself. Using over 100 processors, we succeeded in giving a demonstration of a weather prediction simulation at the CCGrid conference, PRAGMA workshop, and SC2003 conference. Lessons learned from the construction of the demonstration system are presented.

Grid PSE builder: a framework for building Web-based distributed PSE on grid

In order to provide a framework for building Web-based distributed PSEs (problem solving environment) on a grid, we have been designing and implementing grid PSE builder. By using grid PSE builder, PSE providers can easily build a PSE using existing large scale scientific-engineering application as PSE components, without specialized knowledge about Web security and programming. On the other hand, PSE user can access the PSE with highly secured way only by using a Web interface. We have built some Web application portals with existing applications by using grid PSE builder as a Web portal toolkit and it is possible to make existing applications into PSE components with only a small amount of work.

Dynamic Resource Allocation Based on On-the-Fly Reservation

Resource reservation is a vital issue for grid-enabled programs to execute smoothly. Instead of using metascheduling solutions to request grid resources statically, we are developing a tiny C++ library to reserve on-demand computing resources based on Globus Toolkit, local schedulers such as Maui, and SQLite database. The interface class is named CReservation in the library. At the time of preparing this paper, the only implementation available is for Maui scheduler. With such implementation classes, one can request, release, and monitor reservations of computing resources dynamically. To verify the simplicity, usability, and stability of current library, we have performed an in-depth case study with a grid enabled hybrid MD/QM (Molecular Dynamics/Quantum Mechanical) multi-scale simulation program. Our test shows that this library provides an easy and effective approach to manage resource reservations from within grid-enabled applications.

The Grid Enablement and Sustainable Simulation of Multiscale Physics Applications

The understanding of H diffusion in materials is pivotal to designing suitable processes. Though a nudged elastic band (NEB)+molecular dynamics (MD)/quantum mechanics (QM) algorithm has been developed to simulate H diffusion in materials by our group, it is often not computationally feasible for large-scale models on a conventional single system. We thus gridify the NEB+MD/QM algorithm on the top of an integrated framework developed by our group. A two days simulation on H diffusion in alumina has been successfully carried out over a Trans-Pacific Grid infrastructure consisting of supercomputers provided by TeraGrid and AIST. In this paper, we describe the NEB+MD/QM algorithm, briefly introduce the framework middleware, present the grid enablement work, and report the techniques to achieve fault-tolerance and load-balance for sustainable simulation. We believe our experience is of benefit to both middleware developers and application users.

GRPLib: A Web Service Based Framework Supporting Sustainable Execution of Large-Scale and Long-Time Grid Applications

To ensure large-scale and long-time (LSLT) applications to run smoothly in a dynamic and heterogeneous grid environment, we have designed and implemented a WSRF-based framework with which users can reserve resources and request on-demand computing resources. The framework can be architecturely divided into three tiers: the tier providing client-side reservation and allocation APIs, the tier for reservation brokerage and resource allocation, and the tier for backend services. The reservation API is implemented for making and releasing a reservation, as well as for showing available reservations. The allocation API is implemented to request, to check, and to release a resource in a convenient way. The middle tier is designed to hide the complexity of the underlying grid infrastructure, and implemented to provide several allocation algorithms. One of the main backend services is Maui-based reservation service at present. A portal to facilitate the resource management is also available. In this paper, we present the API specification, the architecture, and the implementation of this framework. We also show a detailed experimental example.

Web Service-Based Dynamic Resource and Job Managements for Large-Scale and Long-Time Grid Computations

In order to guarantee large-scale and long-time applications to execute smoothly in a dynamic and heterogeneous grid environment, we have designed and implemented a framework by which users can reserve and allocate on- demand resources, and manage their jobs from within grid- enabled applications. This framework is based on a three- tier architecture that consists of client API, centralized broker of resource and job managements, and backend services on resource reservation and job scheduling. The reservation service is currently based on Maui, and the job scheduling service is DRMAA (Distributed Resource Management Application API) compliant. In this paper, we present the API functionality, the architecture, and the implementation of this framework.