Sergei Gorlatch

Patterns and skeletons for parallel and distributed computing

1 Foundations of Data-parallel Skeletons.- 2 SAT: A Programming Methodology with Skeletons and Collective Operations.- 3 Transforming Rapid Prototypes to Efficient Parallel Programs.- 4 Parallelism Abstractions in Eden.- 5 Skeleton Realisations from Functional Prototypes.- 6 Task and Data Parallelism in P3L.- 7 Skeleton-based Programming Environments.- 8 Applying the Quality Connector Pattern.- 9 Service Design Patterns for Computational Grids.- 10 Towards Patterns of Web Services Composition.- 11 Multi-paradigm and Design Pattern Approaches for HW/SW Design and Reuse.

SkelCL - A Portable Skeleton Library for High-Level GPU Programming

While CUDA and OpenCL made general-purpose programming for Graphics Processing Units (GPU) popular, using these programming approaches remains complex and error-prone because they lack high-level abstractions. The especially challenging systems with multiple GPU are not addressed at all by these low-level programming models. We propose SkelCL -- a library providing so-called algorithmic skeletons that capture recurring patterns of parallel computation and communication, together with an abstract vector data type and constructs for specifying data distribution. We demonstrate that SkelCL greatly simplifies programming GPU systems. We report the competitive performance results of SkelCL using both a simple Mandelbrot set computation and an industrial-strength medical imaging application. Because the library is implemented using OpenCL, it is portable across GPU hardware of different vendors.

Bandwidth-efficient collective communication for clustered wide area systems

Metacomputing infrastructures couple multiple clusters (or MPPs) via wide-area networks. A major problem in programming parallel applications for such platforms is their hierarchical network structure: latency and bandwidth of WANs often are orders of magnitude worse than those of local networks. Our goal is to optimize MPIs collective operations for such platforms. In this paper we focus on optimized utilization of the (scarce) wide-area bandwidth. We use two techniques: selecting suitable communication graph shapes, and splitting messages into multiple segments that are sent in parallel over different WAN links. To determine the best graph shape and segment size, we introduce a performance model called parameterized LogP (P-LogP), a hierarchical extension of the LogP model that covers messages of arbitrary length. With P-LogP, the optimal segment size and the best broadcast tree shape can be determined at runtime. (For conciseness, we restrict our discussion to the broadcast operation). An experimental performance evaluation shows that the new broadcast has significantly improved performance (for large messages) and that there is a close match between the theoretical model and the measured completion times.

Systematic Efficient Parallelization of Scan and Other List Homomorphisms

Homomorphisms are functions which can be parallelized by the divide-and-conquer paradigm. A class of distributable homomorphisms (DH) is introduced and an efficient parallel implementation schema for all functions of the class is derived by transformations in the Bird-Meertens formalism. The schema can be directly mapped on the hypercube with an unlimited or an arbitrary fixed number of processors, providing provable correctness and predictable performance. The popular scan-function (parallel prefix) illustrates the presentation: the systematically derived implementation for scon coincides with the practically used “folklore” algorithm for distributed-memory machines.

A grid workflow language using high-level petri nets

One approach to Grid application programming is to implement services with often-used functionality on high-performance Grid hosts. Complex applications are created by using several services and specifying the workflow between them. We discuss how the workflow of Grid applications can be described easily as a High-Level Petri Net (HLPN), in order to orchestrate and execute distributed applications on the Grid automatically. Petri Nets provide an intuitive graphical workflow description, which is easier to use than script-based descriptions and is much more expressive than directed acyclic graphs (DAG). In addition, the workflow description can be analysed for certain properties such as deadlocks and liveness, using standard algorithms for HLPNs. We propose a platform-independent, XML-based language, called Grid Workflow Description Language (GWorkflowDL), and show how it can be adapted to particular Grid platforms. As two example target platforms, we discuss Java/RMI and the current WSRF standard.

High-level development of multiserver online games

Multiplayer online games with support for high user numbers must provide mechanisms to support an increasing amount of players by using additional resources. This paper provides a comprehensive analysis of the practically proven multiserver distribution mechanisms, zoning, instancing, and replication, and the tasks for the game developer implied by them. We propose a novel, high-level development approach which integrates the three distribution mechanisms seamlessly in todays online games. As a possible base for this high-level approach, we describe the real-time framework (RTF) middleware system which liberates the developer from low-level tasks and allows him to stay at high level of design abstraction. We explain how RTF supports the implementation of single-server online games and how RTF allows to incorporate the three multiserver distribution mechanisms during the development process. Finally, we describe briefly how RTF provides manageability and maintenance functionality for online games in a grid context with dynamic resource allocation scenarios.

RTF: a real-time framework for developing scalable multiplayer online games

This paper presents a middleware system called RTF (Real-Time Framework) which aims at supporting the development of modern real-time online games. The RTF automatically distributes the game state among participating servers, and supports parallel state update computations, as well as efficient communication and synchronization between game servers and clients. The game developers access the RTF via a comfortable interface that offers a considerably higher level of abstraction than the time-consuming and error-prone programming in C++ with socket-based communication. We describe the structure of the RTF system and its current proof-of-concept implementation which supports the multi-server parallelization concepts of zoning, instancing and replication for real-time online games. The novel feature of the RTF is the combination of these three concepts which allows the developer to create large, seamless virtual worlds and to migrate zones between servers.

Optimization rules for programming with collective operations

We study how several collective operations like broadcast, reduction, scan, etc. can be composed efficiently in complex parallel programs. Our specific contributions are: (1) a formal framework for reasoning about collective operations; (2) a set of optimization rules which save communications by fusing several collective operations into one; (3) performance estimates, which guide the application of optimization rules depending on the machine characteristics; (4) a simple case study with machine experiments.

Extracting and implementing list homomorphisms in parallel program development

Abstract Homomorphisms are functions that match the divide-and-conquer pattern and are widely used in parallel programming. Two problems are studied for homomorphisms on lists: 1. (1) parallelism extraction: finding a homomorphic representation of a given function; 2. (2) parallelism implementation: deriving an efficient parallel program that computes the function. The proposed approach to parallelism extraction starts by writing two sequential programs for the function, on traditional cons lists and on dual snoc lists; the parallel program is obtained by generalizing sequential programs as terms. For almost-homomorphic functions, e.g., the maximum segment sum problem, our method provides a systematic embedding into a homomorphism. The implementation problem is addressed by introducing the class of distributable homomorphisms and deriving for them a common parallel program schema. The derivation is based on equational reasoning in the Bird-Meertens formalism, which guarantees the correctness of the parallel target program. The approach is illustrated with the function scan (parallel prefix), for which the combination of our two systematic methods yields the optimal hypercube algorithm, usually presented ad hoc in the literature.

From Grid Middleware to Grid Applications: Bridging the Gap with Hocs

This paper deals with the problem of application programming for grid systems that combine heterogeneous data and computational resources via the Internet. We argue that grid programming is still too complex because of the big gap between the currently used and anticipated grid middleware, (e.g., Globus or WSRF) and the application level. We suggest that this gap needs to be closed in future-generation grids and propose a novel approach to bridging the gap by using Higher-Order Components (HOCs) — recurring patterns of parallel behaviour that are provided to the user as program building blocks with pre-packaged implementation and middleware setup. The presentation is illustrated with a simple case study of computing fractal images. Our experiments demonstrate that HOCs can simplify grid application programming significantly, without serious performance loss.