Norman Scaife

Comparing Parallel Functional Languages: Programming and Performance

This paper presents a practical evaluation and comparison of three state-of-the-art parallel functional languages. The evaluation is based on implementations of three typical symbolic computation programs, with performance measured on a Beowulf-class parallel architecture.We assess three mature parallel functional languages: PMLS, a system for implicitly parallel execution of ML programs; GPH, a mainly implicit parallel extension of Haskell; and Eden, a more explicit parallel extension of Haskell designed for both distributed and parallel execution. While all three languages employ a completely implicit approach to communication, each language takes a different approach to specifying and controlling parallelism, ranging from explicit identification of processes as language constructs (Eden) through annotation of potential parallelism (GPH) to automatic detection of parallel skeletons in sequential code (PMLS).We present detailed performance measurements of all three systems on a widely available parallel architecture: a Beowulf cluster of low-cost commodity workstations. We use three representative symbolic applications: a matrix multiplication algorithm, an exact linear system solver, and a simple ray-tracer. Our results show how moderate speedups can be achieved with little or no changes to the sequential code, and that parallel performance can be significantly improved even within our high-level model of parallel functional programming by controlling key aspects of the program such as load distribution and thread granularity.

Nested algorithmic skeletons from higher-order functions

Algorithmic skeletons provide a promising basis for the automatic utilisation of parallelism at sites of higher order function use through static program analysis. However, decisions about whether or not to realise particular higher order function instances as skeletons must be based on information about processing resources available at runtime In principle, nested higher order functions may be realised as nested skeletons. However, where higher order function arguments result from partially applied functions, free-variable bindings must be identified and communicated through the corresponding skeleton hierarchy to where those arguments are actually applied Here, a skeleton based parallelising compiler for Standard ML is presented. Hybrid skeletons, which can change from parallel to serial evaluation at runtime, are considered and mechanisms for their nesting are discussed. The main compilation stages are illustrated for simple examples. A nested higher order function based algorithm for multiplying matrices of arbitrary length integers is presented along with performance figures for compiled code running on a Fujitsu AP3000.

Carbon Credits for Resource-Bounded Computations Using Amortised Analysis

Bounding resource usage is important for a number of areas, notably real-time embedded systems and safety-critical systems. In this paper, we present a fully automatic static type-based analysis for inferring upper bounds on resource usage for programs involving general algebraic datatypes and full recursion. Our method can easily be used to bound any countable resource, without needing to revisit proofs. We apply the analysis to the important metrics of worst-case execution time, stack- and heap-space usage. Our results from several realistic embedded control applications demonstrate good matches between our inferred bounds and measured worst-case costs for heap and stack usage. For time usage we infer good bounds for one application. Where we obtain less tight bounds, this is due to the use of software floating-point libraries.

Prototyping a parallel vision system in Standard ML

The construction of a parallel vision system from Standard ML prototypes is presented. The system recognises 3D objects from 2D scenes through edge detection, grouping of edges into straight lines and line junction based model matching. Functional prototyping for parallelism is illustrated through the development of the straight line detection component. The assemblage of the whole system from prototyped components is then considered and its performance discussed.

A Dual Source, Parallel Architecture for Computer Vision

We present a parallel architecture for object recognition and location based on concurrent processing of depth and intensity image data. Parallel algorithms for curvature computation and segmentation of depth data into planar or curved surface patches, and edge detection and segmentation of intensity data into extended linear features, are described. Using this feature data in comparison with a CAD model, objects can be located in either depth or intensity images by a parallel pose clustering algorithm.The architecture is based on cooperating stages for low/intermediate level processing and for high level matching. Here, we discuss the use of individual components for depth and intensity data, and their realisation and integration within each parallel stage. We then present an analysis of the performance of each component, and of the system as a whole, demonstrating good parallel execution from raw image data to final pose.

Comparative Cross-Platform Performance Results from a Parallelizing SML Compiler

We have developed a parallelising compiler for Standard ML which allows instantiation of a fixed set of higher order functions with equivalent parallel algorithmic skeletons. The compiler is intended to be both portable and semi-automatic. Here we discuss the performance of the code generated by the compiler, for five exemplar programs on four MIMD parallel machines. Results show consistent cross-platform behaviour and good relative speedup for small numbers of processors but with limited scalability to larger numbers of processors.

Skeleton realisations from functional prototypes

Prototyping is an attractive general basis for experimentation with system design, where the design is refined in the light of prototype behaviour, leading in turn to prototype refinement. A prototype should be considerably cheaper to construct than the final system, and avoid commitment to fine implementation detail, while displaying the same gross input/output characteristics. Once the design has stabilised, the prototype provides a standard for the final system.

Prototyping parallel algorithms using standard ML

We have been developing techniques for deriving parallel implementations of vision algorithms from prototypes written in a functional language (SML). Initially, we analysed simple, well understood algorithms to allow the prototyping methodology to be investigated in a predictable environment. Subsequently, we have extended our approach to more difficult cases such as edge tracking which present problems for parallel system development. Here we demonstrate the power and generality of our approach to parallel algorithm development by a representative set of vision algorithms encoded in SML.

Empirical parallel performance prediction from semantics-based profiling

The PMLS parallelizing compiler for Standard ML is based upon the automatic instantiation of algorithmic skeletons at sites of higher order function use. PMLS seeks to optimise run-time parallel be- haviour by combining skeleton cost models with Structural Operational Semantics rule counts for HOF argument functions. In this paper, the formulation of a general rule count cost model as a set of over-determined linear equations is discussed, and their solution by singular value decom- position, and by a genetic algorithm, are presented.