Zeki Bozkus

Compiling Fortran 90D/HPF for distributed memory MIMD computers

Distributed memory multiprocessors are increasingly being used to provide high performance for advanced calculations with scientific applications. Distributed memory machines offer significant advantages over their shared memory counterparts in terms of cost and scalability, though it is widely accepted that they are difficult to program given the current status of software technology. Currently, distributed memory machines are programmed using a node language and a message passing library. This process is tedious and error prone because the user must perform the task of data distribution and communication for non-local data access. This thesis describes an advanced compiler that can generate efficient parallel programs when the source programming language naturally represents an applications parallelism. Fortran 90D/HPF described in this thesis is such a language. Using Fortran 90D/HPF, parallelism is represented with parallel constructs, such as array operations, where statements, forall statements, and intrinsic functions. The language provides directives for data distribution. Fortran 90D/HPF gives the programmer powerful tools to express a problem with natural data parallelism. To validate this hypothesis, a prototype of Fortran 90D/HPF was implemented. The compiler is organized around several major units: language parsing, partitioning data and computation, detecting communication and generating code. The compiler recognizes the presence of communication patterns in the computations in order to generate appropriate communication calls. Specifically, this involves a number of tests on the relationships among subscripts of various arrays in a statement. The compiler includes a specially designed algorithm to detect communications and to generate appropriate collective communication calls to execute array assignments and forall statements. The Fortran 90D/HPF compiler performs several types of communication and computation optimizations to improve the performance of the generated code. Empirical measurements show that the performance of the output of the Fortran 90D/HPF compiler is comparable to that of corresponding hand-written codes on several systems. We hope that this thesis assists in the widespread adoption of parallel computing technology and leads to a more attractive and powerful software development environment to support application parallelism that many users need.

Fortran 90D/HPF compiler for distributed memory MIMD computers: Design, implementation, and performance results

Fortran 90D/HPF is a data parallel language with special directives to enable users to specify data alignment and distributions. The authors describe the design and implementation of a Fortran 90D/HPF compiler. Techniques for data and computation partitioning, communication detection and generation, and the run-time support for the compiler are discussed. Initial performance results for the compiler are presented. It is believed that the methodology to process data distribution, computation partitioning, communication system design and the overall compiler design can be used by the implementors of HPF compilers.

Benchmarking the CM-5 multicomputer

The authors study the performance of the CM-5 multiprocessor. They provide a number of benchmarks for its communication and computation performance. Many of the operations, like scans and global reduction, can be performed using special hardware available on the CM-5. These operations have been benchmarked. The authors also describe how to embed a mesh and a hypercube on a CM-5 architecture and provide timings for some mesh and hypercube communication primitives on the CM-5.<<ETX>>

Exploiting heterogeneous parallelism with the Heterogeneous Programming Library

While recognition of the advantages of heterogeneous computing is steadily growing, the issues of programmability and portability hinder its exploitation. The introduction of the OpenCL standard was a major step forward in that it provides code portability, but its interface is even more complex than that of other approaches. In this paper, we present the Heterogeneous Programming Library (HPL), which permits the development of heterogeneous applications addressing both portability and programmability while not sacrificing high performance. This is achieved by means of an embedded language and data types provided by the library with which generic computations to be run in heterogeneous devices can be expressed. A comparison in terms of programmability and performance with OpenCL shows that both approaches offer very similar performance, while outlining the programmability advantages of HPL.

Experimental performance evaluation of the CM-5

Abstract In this paper, we present an extensive experimental performance evaluation of the communication capabilities of the CM-5. We first study the communication characteristics such as start-up time, sustainable bandwidth for simple messages as a function of message size and number of processors, and the effect of multiple messages and link contention on the communication time. We study the effect of dense communication patterns such as complete exchange and propose four algorithms for scheduling a complete exchange operation. We also consider the scheduling of irregular communication patterns and present four algorithms for the same. We have tested these algorithms on many synthetic irregular communication patterns as well as those arising in real problems such as the conjugate gradient solver and the Euler solver. Finally, we study the performance and communication aspects of scientific applications such as two-dimensional FFT and Gaussian Elimination on the CM-5.

Fortran 90D intrinsic functions on distributed memory machines: implementation and scalability

The authors are developing a Fortran 90D compiler, which converts Fortran 90D code into Fortran 77 plus message passing node programs for distributed memory machines. The authors present the implementation and performance results of Fortran 90D intrinsic functions on the Intel iPSC/860 hypercube. They have implemented several Fortran 90D intrinsic functions so that they can be called from the node programs of a distributed memory machine. The implementations are scalable, portable and architecture independent.<<ETX>>

Compiling distribution directives in a Fortran 90D compiler

Data partitioning and mapping is one of the most important steps of writing a parallel program, especially a data parallel one. Recently, Fortran D, and subsequently, High Performance Fortran (HPF) have been proposed to allow users to specify data distributions and alignments for the arrays in their programs. The paper presents the design of the data partitioning module of Fortran 90D compiler that processes the alignment and distribution directives.<<ETX>>

Big data platform development with a domain specific language for telecom industries

This paper introduces a system that offer a special big data analysis platform with Domain Specific Language for telecom industries. This platform has three main parts that suggests a new kind of domain specific system for processing and visualization of large data files for telecom organizations. These parts are Domain Specific Language (DSL), Parallel Processing/Analyzing Platform for Big Data and an Integrated Result Viewer. In addition to these main parts, Distributed File Descriptor (DFD) is designed for passing information between these modules and organizing communication. To find out benefits of this domain specific solution, standard framework of big data concept is examined carefully. Big data concept has special infrastructure and tools to perform for data storing, processing, analyzing operations. This infrastructure can be grouped as four different parts, these are infrastructure, programming models, high performance schema free databases, and processing-analyzing. Although there are lots of advantages of Big Data concept, it is still very difficult to manage these systems for many enterprises. Therefore, this study suggest a new higher level language, called as DSL which helps enterprises to process big data without writing any complex low level traditional parallel processing codes, a new kind of result viewer and this paper also presents a Big Data solution system that is called Petaminer.

Compiling HPF for Distributed Memory MIMD Computers

This paper describes the design of a High Performance Fortran (HPF/Fortran 90D) compiler, a source-to-source translator for distributed memory systems. HPF is a data parallel language with compiler directives that enable users to specify data alignment and distribution. This paper presents a systematic methodology to process HPF. Furthermore, techniques for data and computation partitioning, communication detection and generation, and the run-time support for the compiler are discussed. Finally, initial performance results for the compiler are presented which show that the code produced by the compiler is portable, yet efficient. We believe that the methodology used to process data distribution, computation partitioning, communication system design and the overall compiler design can also be used by other HPF compiler implementors.

A Compilation Approach for Fortran 90D/ HPF Compilers

This paper describes a compilation approach for a Fortran 90D/HPF compiler, a source-to-source parallel compiler for distributed memory systems. Different from Fortran 77 parallelizing compilers, a Fortran90D/HPF compiler does not parallelize sequential constructs. Only parallelism expressed by Fortran 90D/HPF parallel constructs is exploited. The methodology of compilation of Fortran 90D/HPF programs such as computation partitioning, communication detection and generation are discussed. An example of Gaussian Elimination is used to illustrate the compilation techniques with performance results.