Steffen Priebe

Comparing Parallel Functional Languages: Programming and Performance

This paper presents a practical evaluation and comparison of three state-of-the-art parallel functional languages. The evaluation is based on implementations of three typical symbolic computation programs, with performance measured on a Beowulf-class parallel architecture.We assess three mature parallel functional languages: PMLS, a system for implicitly parallel execution of ML programs; GPH, a mainly implicit parallel extension of Haskell; and Eden, a more explicit parallel extension of Haskell designed for both distributed and parallel execution. While all three languages employ a completely implicit approach to communication, each language takes a different approach to specifying and controlling parallelism, ranging from explicit identification of processes as language constructs (Eden) through annotation of potential parallelism (GPH) to automatic detection of parallel skeletons in sequential code (PMLS).We present detailed performance measurements of all three systems on a widely available parallel architecture: a Beowulf cluster of low-cost commodity workstations. We use three representative symbolic applications: a matrix multiplication algorithm, an exact linear system solver, and a simple ray-tracer. Our results show how moderate speedups can be achieved with little or no changes to the sequential code, and that parallel performance can be significantly improved even within our high-level model of parallel functional programming by controlling key aspects of the program such as load distribution and thread granularity.

Implementation Skeletons in Eden: Low-Effort Parallel Programming

Algorithmic skeletons define general patterns of computation which are useful for exposing the computational structure of a program. Being general structures they qualify as a target for parallelisation, which is most often carried out by providing specialised, non-portable, low-level parallel implementations (architectural skeletons) of each algorithmic skeleton for different platforms. In the paper we introduce an intermediate layer of implementation skeletons for the parallel functional language Eden. These are portable high-level skeletons which simplify the design of parallel programs substantially. Runtime experiments on a network of workstations and on a Beowulf cluster have shown that even on such high-latency parallel platforms good speedups can be obtained.

Parallelism abstractions in eden

Two important abstractions have contributed to create a reliable programming methodology for industrial-strength programs These are functional abstraction (which has received different names in programming languages, such as procedure, subroutine, function, etc), and data abstraction (also with different names such as abstract data type, object, package or simply module). In both abstractions two different pieces of information are distinguished:

Hierarchical master-worker skeletons

Master-worker systems are a well-known and often applicable scheme for the parallel evaluation of a pool of tasks, a work pool. The system consists of a master process managing a set of worker processes. After an initial phase with a fixed amount of tasks for each worker, further tasks are distributed in reply to results sent back by the workers. As this setup quickly leads to a bottleneck in the master process, the paper investigates techniques for hierarchically nesting the basic master-worker scheme. We present implementations of hierarchical master-worker skeletons, and how to automatically calculate parameters of the nested skeleton for good performance. Nesting master-worker systems is nontrivial especially in cases where newtasks are dynamically created from previous results (typically breadthordepth-first treesearchalgorithms).Wediscusshowtohandledynamically growing pools in a hierarchy and present a declarative implementation for nested master-worker systems with dynamic task creation. The skeletons are experimentally evaluated with two typical test programs. We analyse their runtime behaviour and the effects of different hierarchies on runtimes via trace visualisations.

Dynamic task generation and transformation within a nestable workpool skeleton

Within a classical workpool skeleton a master process employs a set of worker processes to solve tasks contained in a task pool. In contrast to the usual statically fixed task set some applications generate tasks dynamically. Additionally often the need for dynamic task pool transformation arises, for example to combine newly generated partial tasks to form full tasks. We present an extended workpool skeleton for the parallel Haskell dialect Eden which provides both features and employs careful stream-processing and a termination detection mechanism. We also show how to nest the skeleton to alleviate the bottleneck a single master presents. Furthermore we demonstrate its efficiency by its fruitful use for the parallelisation of a DNA sequence alignment algorithm.

High-Level Process Control in Eden

High-level control of parallel process behaviour simplifies the development of parallel software substantially by freeing the programmer from low-level process management and coordination details. The latter are handled by a sophisticated runtime system which controls program execution. In this paper we look behind the scenes and show how the enormous gap between high-level parallel language constructs and their low-level implementation has been bridged in the implementation of the parallel functional language Eden. The main idea has been to implement the process control in a functional language and to restrict the extensions of the low-level runtime system to a few selected primitive operations.

Preprocessing eden with template haskell

Extending a programming language by new language constructs often implies extending its compiler by additional machinery. To reduce the complex interweaving of compiler and extension implementations we present a simple and modular concept of lifting the often needed additional preprocessing out of the base compiler implementation. Avoiding the introduction of standalone tools, this preprocessor framework for extensions of Haskell is designed as a separate portable library of monadic preprocessing functions based on Template Haskell. Additional preprocessing passes expressed in this framework can then much easier be carried along the series of ever advancing base compiler versions. Taking Eden, a parallel programming extension of Haskell, as an example we show that besides achieving improved portability and reusability pass code sizes can be reduced considerably.

Functional Vs Object-Oriented Distributed Languages

Conventional distributed programming languages require the programmer to explicitly specify many aspects of distributed coordination, including resource location, task placement, communication and synchronisation. Functional languages aim to provide higher-level abstraction, and this paper investigates the effectiveness of this for distributed co-ordination. The investigation contrasts and compares contrasts Java and two Haskell-based distributed functional languages, Eden and GdH. Three distributed programs are used as case studies, and the performance and programming effort are reported.