Hidemoto Nakada

Overview of GridRPC: A Remote Procedure Call API for Grid Computing

This paper discusses preliminary work on standardizing and implementing a remote procedure call (RPC) mechanism for grid computing. The GridRPC API is designed to address the lack of a standardized, portable, and simple programming interface. Our initial work on GridRPC shows that client access to existing grid computing systems such as NetSolve and Ninf can be unified via a common API, a task that has proven to be problematic in the past.

Evaluating Web services based implementations of GridRPC

GridRPC is a class of Grid middleware for scientific computing. Interoperability has been an important issue, because current GridRPC systems each employ its own protocol. Web services, where XML-based standards such as SOAP and WSDL are expected to see widespread use, could be the medium of interoperability; however it is not clear if 1) XML-based schemas have sufficient expressive power for GridRPC, and 2) whether performance could be made sufficient. Our experiments indicate that the use of such technologies are more promising. than previously reported. Although a naive implementation of SOAP-based GridRPC has severe performance overhead, application of a series of optimizations improves performance. However encoding of various features of GridRPC proved to be somewhat difficult due to WSDL limitations. The results show that GridRPC systems can be based on Web technologies, but there needs to be work to extend WSDL specifications, possibly impacting OGSA-based Grid services directions.

A Jini-Based Computing Portal System

JiPANG (A Jini-based Portal Augmenting Grids) is a portal system and a toolkit which provides uniform access interface layer to a variety of Grid systems, and is built on top of Jini distributed object technology. JiPANG performs uniform higher-level management of the computing services and resources being managed by individual Grid systems such as Ninf, NetSolve, Globus, etc. In order to give the user a uniform interface to the Grids JiPANG provides a set of simple Java APIs called the JiPANG Toolkits, and furthermore, allows the user to interact with Grid systems, again in a uniform way, using the JiPANG Browser application. With JiPANG, users need not install any client packages before-hand to interact with Grid systems, nor be concerned about updating to the latest version. Such uniform, transparent services available in a ubiquitous manner we believe is essential for the success of Grid as a viable computing platform for the next generation.

GridSpeed: a Web-based grid portal generation server

GridSpeed is a grid portal hosting server that automatically generates and publishes a customized Web interface to the grid for applications, with minimal effort required from the user. Users need only to specify information regarding their application using simple GridSpeed Web forms. With GridSpeed, users need not make any modifications to their applications nor write any glue code to publish the application on the Web, not requiring any knowledge of Perl, JSP (Java server pages) or Java servlets. Moreover, the portal generated by GridSpeed provides an application frontend as well as a set of fundamental portal services such as an information service, monitoring service, data management, single sign-on, and so forth. GridSpeed publishes a set of portals as grid services themselves generated by the system that is shamble, searchable, and accessible from others interested in using the application. This feature facilitates the reuse of application portals for specific application domains, as well as increases the number of available grid applications accessible on the Web. This work describes an overview and architecture of the GridSpeed system, and evaluates the system for two real-world scientific applications: BLAST and MCell.

The ninf portal: an automatic generation tool for grid portals

As the Grid proliferates as the next-generation computing infrastructure, a user interface in the form of Grid Portals is becoming increasingly important, especially for computational scientists and engineers. Although several Grid Portal toolkits have been proposed, portal developers still must build and deploy both the user interface and the application, which results in considerable programming efforts. We aim to ease this burden by generating a portal frontend (that constitutes of JSP and Java Servlets) from an XML document for the former, and a GridRPC system, Ninf-G for easily gridifying existing applications for the latter, and realizing their seamless integration. The resulting system, which we call the Ninf Portal, allowed concise description and easy deployment of a real Grid application with greatly small programming efforts.

Implementation of Fault-Tolerant GridRPC Applications

A task parallel application is implemented with Ninf-G, a GridRPC system. A series of experiments are conducted on the Grid testbed in Asia Pacific for three months. Through tens of long executions, typical fault patterns were collected, and instability of the network throughput was determined to be a major reason of the faults. Several important points are stressed to avoid task throughput decline due to the fault-recovery operations: Timeout minimization for fault detection, background recovery, redundant task assignments, and so on. This study also issues a steer for design of the automated fault-tolerant mechanism in an upper layer of the GridRPC framework.

Multi-Replication with Intelligent Staging in Data-Intensive Grid Applications

Existing data grid scheduling systems handle huge data I/O via replica location services coupled with simple staging, decoupled from scheduling of computing tasks. However, when the application/workflow scales, we observe considerable degradations in performance, compared to processing within a tightly-coupled cluster. For example, when numerous nodes access the same set of files simultaneously, major performance degradation occurs even if replicas are used, due to bottlenecks that manifest in the infrastructure. Instead of resorting to expensive solutions such as parallel file systems, we propose alleviating the situation by tightly coupling replica and data transfer management with computation scheduling. In particular we propose three techniques: (1) dynamic aggregation and O(1) replication of data-staging requests across multiple nodes using a multi-replication framework, (2) replica-centric scheduling - data re-use and time-to-replication as compute scheduling metrics on the grid and (3) overlapped execution of data staging and compute bound tasks. Early benchmark results implemented in our prototype Condor-like grid scheduling system demonstrate that the techniques are quite effective in eliminating much of the overhead in data transfers in many cases

Design and Implementation of NAREGI SuperScheduler Based on the OGSA Architecture

NAREGI is a 5-year Japanese National Grid Project during 2003–2007, whose chief aim is to develop a set of grid middleware to serve as a basis for future e-Science. NAREGI also aims to lead the way in standardization of grid middleware, based on the OGSA architecture. Its super-scheduler is based on the proposed OGSA-EMS Architecture, in that it becomes the first working implementation that implements the documented component relationships within the OGSA-EMS architecture document v.1.0. Through the efforts and experience in the design and implementation, it has been confirmed that the documented OGSA-EMS architecture is quite feasible, but will require significant amount of refinement and speed improvements to finalize its detailed specifications. The super-scheduler also supports co-allocation across multiple sites to support automated execution of grid-based MPIs that execute across machines. Such a resource allocation requires sophisticated interactions between the OGSA-EMS components not covered in the current OGSA-EMS architecture, some of which are non-trivial. Overall, job scheduling with OGSA-EMS has proven to not only work, but also that its job allocation and execution time is within reasonable bounds.

Parallelization of phylogenetic tree inference using grid technologies

The maximum likelihood method is considered as one of the most reliable methods for phylogenetic tree inference. However, as the number of species increases, the approach quickly loses its applicability due to explosive exponential number of trees that need to be considered. An earlier work by one of the authors [3] demonstrated that, by decomposing the trees into fragments called splits, and calculating the individual likelihood of each (small) split and combining them would result in a very close approximation of the true maximum likelihood value, as well as achieving significant reduction in computational cost. However, the cost was still significant for a practical number of species that need to be considered. To solve this problem, we further extend the algorithm so that it could be effectively parallelized in a Grid environment using Grid middleware such as Ninf and Jojo, and also applied combinatorial optimization techniques. Combined, we achieved over 64 times speedup over our previous results in a testbed of 16 nodes, with favorable speedup characteristics.