Ronald Charles Moore

Combining static partitioning with dynamic distribution of threads

This paper presents a hybrid approach to automatic parallelization of computer programs which combines static extraction of threads (tasks) with dynamic scheduling for parallel and distributed execution. Fine-grain scheduling decisions are made at compile time, and coarse-grain scheduling decisions are made at run time. The approach consists of two components: compiler technology which performs the static analysis (thread extraction), and an architecture which takes over the responsibility for scheduling and distributing the threads. Each processor is augmented with a broker, whose responsibility it is to shop for tasks for the processor to perform. This approach aims to provide an adaptive run-time distribution of computation for irregular problems such as the simulation of embedded systems. Finally, this approach is general enough to allow the seamless incorporation of heterogeneous hardware, in particular including dynamically reconfigurable hardware, e.g. FPGAs.

The CDAG: a data structure for automatic parallelization for a multithreaded architecture

Despite the explosive new interest in distributed computing, bringing software-particularly legacy software-to parallel platforms remains a daunting task. The self distributing associative architecture (SDAARC) takes a two-fold approach to this problem. Seemingly sequential programs are first translated into a population of migratory threads and containers by the compiler, and then allowed to migrate to minimize communication while maximizing parallelism by a run time environment. However previous compilers for multithreaded architectures such as SDAARC did not permit the full range of control flow complexity found in programming languages such as C. Thus, we propose a new data structure, and present algorithms for its construction, which extends the familiar concepts of control flow and data flow graphs to conveniently represent the activities required of an automatically generated thread.

SDAARC: an extended cache-only memory architecture

An established data distribution paradigm - extended to include automatic distribution of instruction sequences - might help overcome the traditional difficulties associated with parallel programming. The self-distributing associative architecture (SDAARC) that we describe is based on the cache-only memory architecture concept, but extends the data migration mechanisms with migrating microthreads and a scheduling concept. We present the architecture in a top-down fashion, consider it first as a complete system consisting of compiler and runtime technology, then examine the compiler technology and the runtime system in greater detail.

Automatic scheduling for cache only memory architectures

For parallel and distributed systems to gain more acceptance than they have to date, they will need to be scalable, affordable-but most importantly, they must be made as easy to program as sequential systems. Ideally, we would like to be able to take programs written in conventional languages and recompile them for parallel architectures, thus freeing the programmer from all additional effort above and beyond that necessary to program a conventional computer. This in turn implies that either the compiler, the hardware, or both, must address the fundamental issue of distribution. This problem is two fold: both data and computation must somehow be distributed. The paper attempts to bring data distribution concepts from cache only memory architectures together with scheduling concepts from multithreaded architectures, in order to arrive at one unified, simplified, cohesive abstract model of computation. The fusion of data and computation distribution is the central principle guiding the development of a new architecture being developed by the authors, named SDAARC (Self Distributing Associative Architecture).

The SDAARC architecture

While traditional parallel computing systems are still struggling to gain a wider acceptance, the largest parallel computer that has ever been available is currently growing with the communication resource Internet. Unfortunately it is also rarely used in the parallel computation field. The reason for the rejection of parallel computers is mainly the difficulty of parallel programming. In this paper we propose the Self Distributing Associative ARChitecture (SDAARC). It has been derived from the Cache Only Memory Architecture (COMA). COMAs provide a distributed shared memory (DSM) with automatic distribution of data. We show how this paradigm of data distribution can be extended to the automatic distribution of instruction sequences (microthreads). We show how microthreads can be extracted from legacy C code to produce code that can automatically be parallelized by SDAARC at run time. We also discuss how SDAARC can be implemented on a rightly coupled multiprocessor systems on heterogenous LAN based computer networks (Intranet) and on WANs of computing resources.