Ming Kin Lai

Distributed parallel computing using navigational programming

Message Passing (MP) and Distributed Shared Memory (DSM) are the two most common approaches to distributed parallel computing. MP is difficult to use, whereas DSM is not scalable. Performance scalability and ease of programming can be achieved at the same time by using navigational programming (NavP). This approach combines the advantages of MP and DSM, and it balances convenience and flexibility. Similar to MP, NavP suggests to its programmers the principle of pivot-computes and hence is efficient and scalable. Like DSM, NavP supports incremental parallelization and shared variable programming and is therefore easy to use. The implementation and performance analysis of real-world algorithms, namely parallel Jacobi iteration and parallel Cholesky factorization, presented in this paper supports the claim that the NavP approach is better suited for general-purpose parallel distributed programming than either MP or DSM.

Mobile Agents - The Right Vehicle for Distributed Sequential Computing

Distributed sequential computing (DSC) is computing with distributed data using a single locus of computation. In this paper we argue that computation mobility--the ability for the locus of computation to migrate across distributed memories and continue the computation as it meets the required data--facilitated by mobile agents with strong mobility is essential for scalable distributed sequential programs that preserve the integrity of the original algorithm.

Toward Automatic Data Distribution for Migrating Computations

Program parallelization requires mapping computation and data to processing elements. Navigational programming (NavP), based on the principle of migrating computations, offers a different approach than the conventional solutions that use a SPMD model. This paper focuses on data distribution for NavP. We introduce the navigational trace graph (NTG), a mathematical structure that captures the alignment and distribution preferences of a sequential program. Graph partitioning is applied to NTGs to obtain data distribution solutions. The major advantage is that our methodology can focus exclusively on reducing communication overhead first and later determine the actual computation partition and parallelization, because NavP computations migrate freely across partitions. This is in stark contrast to SPMD, where the data partitioning imposes hard constraints on the threads because they are stationary. We present experimental results to demonstrate the effectiveness of our approach.

Mobile pipelines: parallelizing left-looking algorithms using navigational programming

We consider the class of “left-looking” sequential matrix algorithms: consumer-driven algorithms that are characterized by “lazy” propagation of data. Left-looking algorithms are difficult to parallelize using the message-passing or distributed shared memory models because they only possess pipeline parallelism. We show that these algorithms can be directly parallelized using mobile pipelines provided by the Navigational Programming methodology. We present performance data demonstrating the effectiveness of our approach.

Toward Incremental Parallelization Using Navigational Programming*The authors gratefully acknowledge the support of a U.S. Department of Education GAANN Fellowship.

The Navigational Programming (NavP) methodology is based on the principle of self-migrating computations. It is a truly incremental methodology for developing parallel programs: each step represents a functioning program, and each intermediate program is an improvement over its predecessor. The transformations are mechanical and straightforward to apply. We illustrate our methodology in the context of matrix multiplication, showing how the transformations lead from a sequential program to a fully parallel program. The NavP methodology is conducive to new ways of thinking that lead to ease of programming and high performance. Even though our parallel algorithm was derived using a sequence of mechanical transformations, it displays certain performance advantages over the classical handcrafted Gentlemans Algorithm.

Incremental parallelization using navigational programming: a case study

We show how a series of transformations can be applied to incrementally parallelize sequential programs. Our navigational programming (NavP) methodology is based on the principle of self-migrating computations and is truly incremental, in that each step represents a functioning program and every intermediate program is an improvement over its predecessor. The transformations are mechanical and straightforward to apply. We illustrate our methodology in the context of matrix multiplication. Our final stage is similar to the classical Gentlemans algorithm. The NavP methodology is conducive to new ways of thinking that lead to ease of programming and high performance.

From distributed sequential computing to distributed parallel computing

One approach to distributed parallel programming is to utilize self-migrating threads. Computations can be distributed first, and parallelized second. The first step produces a distributed sequential thread, which can be incrementally parallelized by the second step. This paper prescribes three transformations that turn distributed sequential programs into distributed parallel programs. Real-life examples and performance data are presented, and the advantages of our approach are discussed.