Antonio Lain

The Paradigm compiler for distributed-memory multicomputers

To harness the computational power of massively parallel distributed-memory multicomputers, users must write efficient software. This process is laborious because of the absence of global address space. The programmer must manually distribute computations and data across processors and explicitly manage communication. The Paradigm (PARAllelizing compiler for DIstributed-memory, General-purpose Multicomputers) project at the University of Illinois addresses this problem by developing automatic methods for the efficient parallelization of sequential programs. A unified approach efficiently supports regular and irregular computations using data and functional parallelism. >

The SmartFrog configuration management framework

SmartFrog is a framework for creating configuration-driven systems. It has been designed with the express purpose of making the design, deployment and management of distributed component-based systems simpler and more robust. Over the last decade it has been the focus for ongoing research into aspects of configuration management and large-scale distributed systems, providing a platform for experimentation. The paper covers the rationale for the design of the framework, details of its design, plus a description of the further research that is in progress.

Advanced compilation techniques in the PARADIGM compiler for distributed-memory multicomputers

The PARADIGM compiler project provides an automated means to parallelize programs, written in a serial programming model, for efficient execution on distributed-memory multicomputers. .A previous implementation of the compiler based on the PTD representation allowed symbolic array sizes, affine loop bounds and array subscripts, and variable number of processors, provided that arrays were single or multi-dimensionally block distributed. The techniques presented here extend the compiler to also accept multidimensional cyclic and block-cyclic distributions within a uniform symbolic framework. These extensions demand more sophisticated symbolic manipulation capabilities. A novel aspect of our approach is to meet this demand by interfacing PARADIGM with a powerful off-the-shelf symbolic package, Mathematica. This paper describes some of the Mathematica routines that performs various transformations, shows how they are invoked and used by the compiler to overcome the new challenges, and presents experimental results for code involving cyclic and block-cyclic arrays as evidence of the feasibility of the approach.

Exploiting spatial regularity in irregular iterative applications

The increasing gap between the speed of microprocessors and memory subsystems makes it imperative to exploit locality of reference in sequential irregular applications. The parallelization of such applications requires special considerations. Current RTS (Run-Time Support) for irregular computations fails to exploit the fine grain regularity present in these applications, producing unnecessary time and memory overheads. PILAR (Parallel Irregular Library with Application of Regularity) is a new RTS for irregular computations that provides a variety of internal representations of communication patterns based on their regularity; allowing for the efficient support of a wide spectrum of regularity under a common framework. Experimental results on the IBM SP-1 and Intel Paragon demonstrate the validity of our approach.<<ETX>>

Techniques to overlap computation and communication in irregular iterative applications

There are many applications in CFD and structural analysis that can be more accurately modeled using unstructured grids. Parallelization of implicit methods for unstructured grids is a difficult and important problem. This paper deals with coloring techniques to overlap computation and communication during the solution of implicit methods on message passing distributed memory multicomputers. An evaluation of coloring techniques for partitioned unstructured grids is first presented. Results show the importance of using partitioning information during coloring. It is next shown that overlapping computation and communication can be formalized as a generalized coloring problem. Modified coloring algorithms are used for this purpose. The PARTI library has been extended to support non-blocking gather-scatter operations and used in conjunction with these algorithms. Practicality issues are evaluated with experimental results on an Intel Paragon multicomputer.

Compilation of scientific programs into multithreaded and message driven computation

Many programs written in the SPMD programming model send messages asynchronously, and block when receiving messages. Multiple threads can make use of the processor while other threads wait for messages. This paper describes and evaluates two techniques for multithreading on the nodes of distributed memory message passing systems. One method is a purely runtime threads package. The second method requires the SPMD code to be systematically transformed into message driven code which can be run under a message driven model. The multithreading of scientific applications is evaluated on the iPSC2 and the CM5.<<ETX>>

Evaluation of compiler and runtime library approaches for supporting parallel regular applications

Important applications including those in computational chemistry, computational fluid dynamics, structural analysis and sparse matrix applications usually consist of a mixture of regular and irregular accesses. While current state-of-the-art run-time library support for such applications handles the irregular accesses reasonably well, the efficacy of the optimizations at run-time for the regular accesses is yet to be proven. This paper aims to find a better approach to handle the above applications in a unified compiler and run-time framework. Specifically, this paper considers only regular applications and evaluates the performance of two approaches, a run-rime approach using PILAR and a compile-time approach using a commercial HPF compiler. This study shows that using a particular representation of regular accesses, the performance of regular code using run-time libraries can come close to the performance of code generated by a compiler. It also determines the operations that usually contribute largely to the run-time overhead in case of regular accesses. Experimental results are reported for three regular applications on a 16-processor IBM SP-2.

Distributed authorization using delegation with acyclic paths

We present a new trust management scheme for distributed authorization which can be easily implemented using X.509-based certificate chains, but does not require globally unique role names. A principal proves that he has authorization for a particular action by demonstrating the existence of an acyclic chain of bindings from a specified principal to himself where the sequence of labels in the chain matches a template. This template is in an easily-computed subset of regular path expressions. Our restrictions to acyclic paths and to a subset of path expressions enable us to permit controlled delegation, relax the requirement of global agreement on role names, and provide an intuitive abstraction. We show that some useful security properties can be determined in polynomial time. Our scheme has been used in practice to secure a management framework for distributed components: we give an overview of the implementation

Compiler and run-time support for exploiting regularity within irregular applications

This paper starts from a well-known idea, that structure in irregular problems improves sequential performance, and tries to show that the same structure can also be exploited for parallelization of irregular problems on a distributed-memory multicomputer. In particular, we extend a well-known parallelization technique called run-time compilation to use structure information that is explicit on the array subscripts. This paper presents a number of internal representations suited to particular access patterns and shows how various preprocessing structures such as translation tables, trace arrays, and interprocessor communication schedules can be encoded in terms of one or more of these representations. We show how loop and index normalization are important for detection of irregularity in array references, as well as the presence of locality in such references. This paper presents methods for detection of irregularity, feasibility of inspection, and finally, placement of inspectors and interprocessor communication schedules. We show that this process can be automated through extensions to an HPF/Fortran-77 distributed-memory compiler (PARADIGM) and a new runtime support for irregular problems (PILAR) that uses a variety of internal representations of communication patterns. We devise performance measures which consider the relationship between the inspection cost, the execution cost, and the number of times the executor is invoked so that a comparison of the competing schemes can be performed independent of the number of iterations. Finally, we show experimental results on an IBM SP-2 that validate our approach. These results show that dramatic improvements in both memory requirements and execution time can be achieved by using these techniques.

Compiler support for hybrid irregular accesses on multicomputers

This paper starts from a well-known idea, that structure in irregular problems improves sequential performance, and tries to show that the same structure can also be exploited for parallelization of irregular problems on a distributedmemory multi-computer. Moreover, we will show that this process can be automated through extensions to an HPF/ Fortran-77 distributed-memory compiler and a new run-time support for irregular problems, PILAR. Finally, we will show experimental results on a real multi-computer that validate our approach.