Alexey Kalinov

Heterogeneous Distribution of Computations Solving Linear Algebra Problems on Networks of Heterogeneous Computers

This paper presents and analyzes two different strategies of heterogeneous distribution of computations solving dense linear algebra problems on heterogeneous networks of computers. The first strategy is based on heterogeneous distribution of processes over processors and homogeneous block cyclic distribution of data over the processes. The second is based on homogeneous distribution of processes over processors and heterogeneous block cyclic distribution of data over the processes. Both strategies were implemented in the mpC language?a dedicated parallel extension of ANSI C for efficient and portable programming of heterogeneous networks of computers. The first strategy was implemented using calls to ScaLAPACK; the second strategy was implemented with calls to LAPACK and BLAS. Cholesky factorization on a heterogeneous network of workstations is used to demonstrate that the heterogeneous distributions have an advantage over the traditional homogeneous distribution.

Heterogeneous Distribution of Computations While Solving Linear Algebra Problems on Networks of Heterogeneous Computers

The paper presents a heterogeneous distribution of computations while solving dense linear algebra problems on heterogeneous networks of computers. The distribution is based on heterogeneous block cyclic distribution which is extension of the traditional homogeneous block cyclic distribution taking into account differences in the processor performances. The mpC language, specially designed for parallel programming heterogeneous networks is briefly introduced. An mpC aplication carring out Cholesky factorization on a heterogenous network of workstations is used to demonstrate that the heterogeneous distribution have an essential advantage over the traditional homogeneous distribution.

Natural block data decomposition for heterogeneous clusters

We propose general purposes natural heuristics for static block and block-cyclic heterogeneous data decomposition over processes of parallel program mapped into multidimensional grid. This heuristics is an extension of the intuitively clear heterogeneous data distribution for one-dimensional case. It is compared to advanced heuristics for heterogeneous data decomposition proposed for solving linear algebra problems on two-dimensional process grid. We experimentally show that for typical local network (12 Windows 2000 PCs interconnected via Fast Ethernet switch) and for typical linear algebra problems these two heuristics have almost the same efficiency. We demonstrate efficiency of the proposed natural decomposition for case of three-dimensional process grid on the example of 3D modeling of supernova explosion.

Optimal mapping of a parallel application processes onto heterogeneous platform

The paper is devoted to analysis of a strategy of computation distribution on heterogeneous parallel systems. According to this strategy processes of parallel program are distributed over the processors according to their performances and data are distributed between processes evenly. The paper presents an algorithm that computes optimal number of the processes and their distribution over processors minimizing the execution time of an application. The processor performance is considered as a function of the number of processes running on the processor and the amount of the data processing by the processor.

A parallel language and its programming system for heterogeneous networks

The paper presents a new parallel language, mpC, designed specially for programming high-performance computations on heterogeneous networks of computers, as well as its supportive programming environment. The main idea underlying mpC is that an mpC application explicitly defines an abstract network and distributes data, computations and communications over the network. The mpC programming environment uses at run time this information as well as information about any real executing network in order to map the application to the real network in such a way that ensures efficient execution of the application on this real network. Experience of using mpC for solving both regular and irregular real-life problems on networks of heterogeneous computers is also presented.

A programming environment for heterogenous distributed memory machines

mpC is a medium level programming language for distributed memory machines (DMM). The language is an ANSI C superset based on the notion of a network comprising virtual processors of different types and performances connected with links of different bandwidths. It allows the user to describe a network topology, create and discard networks, and distribute data and computations over the networks. In other words, the user can specify (dynamically) the topology of his application, and the mpC programming environment will use this (topological) information in run time to ensure the efficient execution of the application on any particular DMM. The paper outlines the principal features of mpC and its programming environment which make them suitable tools to write efficient and portable parallel programs for heterogenous DMM.

Scalability analysis of matrix-matrix multiplication on heterogeneous clusters

The paper is devoted to scalability analysis of a typical linear algebra algorithm on heterogeneous clusters. We proof that traditional scalability metrics proposed for analysis of linear algebra algorithms is applicable on heterogeneous platform and investigate influence of three heterogeneous strategies of computation distribution to scalable universal matrix multiplication algorithm (SUMMA) scalability.

Coverage-driven Automated Compiler Test Suite Generation

Abstract The paper presents a novel approach to automated compiler test suite generation based on the source level specification. Several coverage criteria are introduced. The application of the proposed methodology to testing the realistic programming language is discussed.

Multidimensional Static Block Data Decomposition for Heterogeneous Clusters

We propose general static block and block-cyclic heteroge- neous decomposition of multidimensional data over processes of parallel program mapped onto multidimensional process grid. The decomposi- tion is compared with decomposition of two-dimensional data over two- dimensional process grid of Beaumont et al and with natural decompo- sition of three-dimensional data over three-dimensional process grid.

Experiments with mpC: Efficient Solving Regular Problems on Heterogeneous Networks of Computers via Irregulation

mpC is a medium-level parallel language for programming heterogeneous networks of computers. It allows to write libraries of parallel routines adaptable to peculiarities of any particular executing multiprocessor system to ensure efficient running. The adaptable routines distribute data and computations in accordance with performances of participating processors. In this case even the problems traditionally considered regular, become irregular. Advantages of mpC for efficient solving of regular problems on heterogeneous networks of computers are demonstrated with an mpC routine implementing Cholesky factorization, with efficiency of the mpC routine being compared with ScaLAPACK one.