William F. McColl

Questions and Answers about BSP

Bulk Synchronous Parallelism (BSP) is a parallel programming model that abstracts from low-level program structures in favour of supersteps. A superstep consists of a set of independent local computations, followed by a global communication phase and a barrier synchronisation. Structuring programs in this way enables their costs to be accurately determined from a few simple architectural parameters, namely the permeability of the communication network to uniformly-random traffic and the time to synchronise. Although permutation routing and barrier synch ronisations are widely regarded as inherently expensive, this is not the case. As a result, the structure imposed by BSP does not reduce performance, while bringing considerable benefits for application building. This paper answers the most common questions we are asked about BSP and justifies its claim to be a major step forward in parallel programming.

Memory-efficient matrix multiplication in the BSP model

Abstract. The model of bulk-synchronous parallel (BSP) computation is an emerging paradigm of general-purpose parallel computing. Its modification, the BSPRAM model, allows one to combine the advantages of distributed and shared-memory style programming. In this paper we study the BSP memory complexity of matrix multiplication. We propose new memory-efficient BSP algorithms both for standard and for fast matrix multiplication. The BSPRAM model is used to simplify the description of the algorithms. The communication and synchronization complexity of our algorithms is slightly higher than that of known time-efficient BSP algorithms. The current time-efficient and new memory-efficient algorithms are connected by a continuous tradeoff.

Scalability, portability and predictability: the BSP approach to parallel programming

Abstract General purpose parallel computing systems come in a variety of forms. We have various kinds of distributed memory architectures, shared memory multiprocessors, and clusters of workstations. New technologies may increase this range still further. Can one hope to design portable and scalable parallel software in the face of such architectural diversity? In this paper we show that it is indeed possible to produce fully portable parallel software which will run with highly efficient, scalable and predictable performance on any general purpose parallel architecture. The approach we describe is based on the bulk synchronous parallel (BSP) model of computation. The BSP model provides a simple, unified framework for the design and programming of all kinds of general purpose parallel systems. Over the last few years, a number of important research activities in algorithms and architectures have been pursued as part of this new approach to scalable parallel computing. In this paper we give some simple BSP algorithms and show how they can be expressed as programs. We also briefly describe some of the BSP programming language developments which are now being pursued.

The Depth of All Boolean Functions

Every Boolean function of n arguments has a circuit of depth

A two-way BSP algorithm for tridiagonal systems

n + 1

The planar realization of Boolean functions

over the basis

The maximum depth of monotone formulae

\{ f|f:\{ 0,1\} ^2 \to \{ 0,1\} \}

Parallel Algorithms and Architectures

.

BSP Algorithms - Write Once, Run Anywhere

A two-way parallel recursive method is presented for solving a tridiagonal linear system. The algorithm is based on the parallel segment recursive method proposed in [9]. The computation and communication costs of the algorithm are analysed using the BSP (Bulk Synchronous Parallel) model.

A Two-Way BSP Algorithm for Tridiagonal Systems

Abstract It is shown that every Boolean function of n arguments has a planar circuit of size 61 48 × 2 n .