Holger Bischof

A Generic MPI Implementation for a Data-Parallel Skeleton: Formal Derivation and Application to FFT

We derive a provably correct, architecture-independent family of parallel implementations for a class of data-parallel algorithms, called DH (distributable homomorphisms). The implementations are well-structured SPMD programs with group-wise personalized all-to-all exchange, directly realizable in MPI. As a case study, we systematically adjust the mathematical specification of the Fast Fourier Transform (FFT) to the DH format and, thereby, obtain a generic SPMD implementation for FFT. The target program includes FFT solutions used in practice – the binary-exchange and the 2D- and SD-transpose – as special cases.

Double-Scan: Introducing and Implementing a New Data-Parallel Skeleton

We introduce a new reusable component for parallel programming, the double-scan skeleton. For this skeleton, we formulate and formally prove sufficient conditions under which the double-scan can be parallelized, and develop its efficient MPI implementation. The solution of a tridiagonal system of equations is considered as our case study. We describe how this application can be developed using the double-scan and report experimental results for both absolute performance and performance predictability of the skeleton-based solution.

DatTeL: A DATA-PARALLEL C++ TEMPLATE LIBRARY

The concept of C++ templates, realized in the Standard Template Library (STL), allows development of generic programs suitable for various concrete data structures. Our aim in this paper is to provide an opportunity for efficient execution of STL programs on parallel machines. We present DatTeL – a data-parallel library, which allows simple, efficient programming for various parallel architectures while staying within the paradigm of classical C++ template programming. We describe the principles of our approach and explain our design decisions and their implementation in the library. The presentation is illustrated with a case study — parallelization of a generic algorithm for carry-lookahead addition. We compare DatTeL to related work and report experimental results for the current version of our library.

Cost Optimality and Predictability of Parallel Programming with Skeletons

Skeletons are reusable, parameterized components with well-defined semantics and pre-packaged efficient parallel implementation. This paper develops a new, provably cost-optimal implementation of the DS (double-scan) skeleton for the divide-and-conquer paradigm. Our implementation is based on a novel data structure called plist (pointed list); implementation’s performance is estimated using an analytical model. We demonstrate the use of the DS skeleton for parallelizing a tridiagonal system solver and report experimental results for its MPI implementation on a Cray T3E and a Linux cluster: they confirm the performance improvement achieved by the cost-optimal implementation and demonstrate its good predictability by our performance model.

Foundations of data-parallel skeletons

Parallelism seeks to achieve better computation performance by simultaneously using multiple processors, either in one computer or across several computers connected in a network. Parallel computing has not yet become a routine way of solving problems faster, mostly because parallel machines are hard to program. A major reason for this unsatisfactory situation is that the convenience and correctness of parallel programming has often been neglected in favour of the foremost aim of parallelism — achieving very high absolute performance.

DATA PARALLELISM IN C++ TEMPLATE PROGRAMS: A BARNES-HUT CASE STUDY

We describe how C++ programs that use the Standard Template Library (STL) can be systematically parallelized for shared-memory machines. We present our data-parallel template library (DatTeL) and its use in introducing parallelism into sequential STL programs. As a case study, we demonstrate how an STL implementation of the Barnes-Hut algorithm for solving many-body problems can be systematically transformed into a parallel, efficient version using DatTeL. We present experimental results for the multi-threaded version of the resulting parallel program on a SunFire multiprocessor.

Design and Implementation of a Cost-Optimal Parallel Tridiagonal System Solver Using Skeletons

We address the problem of systematically designing correct parallel programs and developing their efficient implementations on parallel machines. The design process starts with an intuitive, sequential algorithm and proceeds by expressing it in terms of well-defined, pre-implemented parallel components called skeletons. We demonstrate the skeleton-based design process using the tridiagonal system solver as our example application. We develop step by step three provably correct, parallel versions of our application, and finally arrive at a cost-optimal implementation in MPI (Message Passing Interface). The performance of our solutions is demonstrated experimentally on a Cray T3E machine.