Hrabri Rajic

Standardization of an API for Distributed Resource Management Systems

Todays cluster and grid environments demand the usage of product-specific APIs and tools for developing distributed applications. We give an overview of the distributed resource management application API (DRMAA) specification, which defines a common interface for job submission, control, and monitoring. The DRMAA specification was developed by the authors at the open grid forum standardization body, and has meanwhile significant adoption in academic and commercial cluster systems. Within this paper, we describe the basic concepts of the finalized API, and explain issues and findings with the standardization of such an unified interface.

An Integrated Performance Visualizer for MPI/OpenMP Programs

As cluster computing has grown, so has its use for large scientific calculations. Recently, many researchers have experimented with using MPI between nodes of a clustered machine and OpenMP within a node, to manage the use of parallel processing. Unfortunately, very few tools are available for doing an integrated analysis of an MPI/OpenMP program. KAI Software, Pallas GmbH and the US Department of Energy have partnered together to build such a tool, VGV. VGV is designed for doing scalable performance analysis - that is, to make the performance analysis process qualitatively the same for small cluster machines as it is for the largest ASCI systems. This paper describes VGV and gives a flavor of how to find performance problems using it.

High-level performance modeling of task-based algorithms

Performing modeling and visualization of task-based parallel algorithms is challenging. Libraries such as Intel Threading Building Blocks (TBB) and Microsofts Parallel Patterns Library provide high-level algorithms that are implemented using low-level tasks. Current tools present performance at this lower level. Developers like to tune and debug at the same level as the coding abstraction, so in this paper we propose tools and a two step methodology that target this level of abstraction. In the first step, the system level metrics of utilization and overhead are collected to determine if performance is acceptable. If a problem is suspected, the second step of our methodology projects these metrics on to the algorithms contained in the application. Using these projections many common performance issues can be quickly diagnosed. We demonstrate our methodology using a prototype implementation that is integrated with the Intel Threading Building Blocks library. We show the flexibility of the approach by analyzing three applications, including a client-server benchmark that uses a parallel_for nested within a parallel pipeline.

Standardised job submission and control in cluster and grid environments

Cluster and Grid environments mostly require the use of product-specific Application Programming Interface (APIs) to submit, control and monitor computational jobs. The Open Grid Forum standardisation body therefore has developed several specifications to fill the gap and enable developers to code to few standardised APIs. This paper discusses the details of one of these specifications, the Distributed Resource Management Application API. We compare the basic concepts of the finalised API to other specifications from the same area and explain issues and findings uncovered during the standardisation process.

Speeding up NGB with distributed file streaming framework

Grid computing provides a very rich environment for scientific calculations. In addition to the challenges it provides, it also offers new opportunities for optimization. In this paper we have utilized DFS (distributed file streaming) framework to speed up NAS grid benchmark workflows. By studying I/O patterns of NGB codes we have identified program locations where it is possible to overlap computation and data workflow phases. By integrating DFS into NGB, we demonstrate a useful method of improving overall workflow efficiency by streaming the output of the current process to make an input of the following stage, reducing a workflow to a series of distributed producer consumer stages. DFS framework eliminates file transfers and in the process makes process scheduling more efficient, leading to overall performance improvements in the turnaround time for HC (helical chain) data flow graph under Globus grid environment with the embedded DFS over the original version of the benchmark