Michele Cermele

Dynamic load balancing of distributed SPMD computations with explicit message-passing

Distributed systems have the potentiality of becoming an alternative platform for parallel computations. However, there are still many obstacles to overcome, one of the most serious is that distributed systems typically consist of shared heterogeneous components with highly variable computational power. We present a load balancing support that checks the load status and, if necessary, adapts the workload to dynamic platform conditions through data migrations from overloaded to underloaded nodes. Unlike task migration supports for task parallelism and other data migration frameworks for master/slave-based parallel applications, our support works for the entire class of SPMD regular applications with explicit communications such as linear algebra problems, partial differential equation solvers, image processing algorithms. Although we considered several variants (three activation mechanisms, three load monitoring techniques and four decision policies), we implemented only the protocols that guarantee program consistency. The efficiency of the strategies is tested in the instance of two SPMD algorithms that are based on the PVM library enriched by special-purpose primitives for data management. As additional contribution, our research keeps the entire support for dynamic load balancing transparent to the programmer. The only visible interface of our support is the activation phase.

DAME: an environment for preserving the efficiency of data-parallel computations on distributed systems

DAME (Data Migration Environment) uses transparent supports to overcome inefficiencies in data parallel programming. These supports hide irregular network topology, dynamically adapt the data distribution to platform conditions, and mask the consequent nonuniform distribution to the programmer. The authors compare DAMEs performance with that of some popular frameworks. They begin by discussing DAMEs three main design goals: efficiency, transparency, and scalability. Next, they describe the five supports that DAME gives the programmer: virtual topology, data distribution, data management, interprocess communication, and workload reconfiguration. Then, they present the results they obtained from experiments using 10 workstations that provide a hardware heterogeneous, data homogeneous, nonuniform platform. The results show that DAME provides a virtual single program, multiple data machine that overcomes most of the differences that distinguish a parallel virtual machine from an ideal SPMD machine.

Non-uniform and dynamic domain decompositions for hypercomputing

Abstract Implementing efficient parallel programs on a network-based computing platform is still a challenge. This paper proposes a new adaptive data distribution (ADD) support that avoids the complex task of managing irregular data distributions and adapting them to the variable conditions of a multi-users system. In particular, ADD provides the programmer with a data partition algorithm that fits the non-uniformity of the platform nodes at the beginning of program execution, a set of data inquiry primitives that allow the programmer to deal with a logical regular partition and a runtime support that autonomously adapts the data distribution to the node power variations occurring during computation. Several experimental results demonstrate that ADD is a very useful tool to maintain the efficiency of SPMD computations especially when the platform is highly non-uniform and variable.

Supporting Self-Adaptivity for SPMD Message-Passing Applications

Real parallel applications find little benefits from code portability that does not guarantee acceptable efficiency. In this paper, we describe the new features of a framework that allows the development of Single Program Multiple Data (SPMD) applications adaptable to different distributed-memory machines, varying from traditional parallel computers to networks of workstations. Special programming primitives providing indirect accesses to the platform and data domain guarantee code portability and open the way to runtime optimizations carried out by a scheduler and a runtime support.