Gabriele Jost

Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-Core SMP Nodes

Today most systems in high-performance computing (HPC) feature a hierarchical hardware design: Shared memory nodes with several multi-core CPUs are connected via a network infrastructure. Parallel programming must combine distributed memory parallelization on the node interconnect with shared memory parallelization inside each node. We describe potentials and challenges of the dominant programming models on hierarchically structured hardware: Pure MPI (Message Passing Interface), pure OpenMP (with distributed shared memory extensions) and hybrid MPI+OpenMP in several ¿avors. We pinpoint cases where a hybrid programming model can indeed be the superior solution because of reduced communication needs and memory consumption, or improved load balance. Furthermore we show that machine topology has a signi¿cant impact on performance for all parallelization strategies and that topology awareness should be built into all applications in the future. Finally we give an outlook on possible standardization goals and extensions that could make hybrid programming easier to do with performance in mind.

Toward enhancing OpenMP's work-sharing directives

OpenMP provides a portable programming interface for shared memory parallel computers (SMPs). Although this interface has proven successful for small SMPs, it requies greater flexibility in light of the steadily growing size of individual SMPs and the recent advent of multithreaded chips. In this paper, we describe two application development experiences that exposed these expressivity problems in the current OpenMP specification. We then propose mechanisms to overcome these limitations, including thread subteams and thread topologies. Thus, we identify language features that improve OpenMP application performance on emerging and large-scale platforms while preserving ease of programming.

Hybrid MPI and OpenMP parallel programming

Most HPC systems are clusters of shared memory nodes. Such systems can be PC clusters with dual or quad boards, but also “constelation” type systems with large SMP nodes. Parallel programming must combine the distributed memory parallelization on the node inter-connect with the shared memory parallelization inside of each node.

Adding new dimensions to performance analysis through user-defined objects

Performance analysis of parallel applications requires a high degree of flexibility for the user to navigate through huge amounts of performance trace data. Many different statistics need to be calculated and compared in order to derive meaningful conclusions. In this paper we propose user-defined objects as a means to customize the analysis for different hardware platforms, programming paradigms, or application areas. We describe how the concept is realized in a development version of the Sun™ Studio Performance Analyzer and demonstrate its usefulness for the analysis of a nested OpenMP application.