Tom Henderson

Experience Applying Fortran GPU Compilers to Numerical Weather Prediction

Graphics Processing Units (GPUs) have enabled significant improvements in computational performance compared to traditional CPUs in several application domains. Until recently, GPUs have been programmed using C/C++ based methods such as CUDA (NVIDIA) and OpenCL (NVIDIA and AMD). Using these approaches, Fortran Numerical Weather Prediction (NWP) codes would have to be completely re-written to take full advantage of GPU performance gains. Emerging commercial Fortran compilers allow NWP codes to take advantage of GPU processing power with much less software development effort. The Non-hydrostatic Icosahedral Model (NIM) is a prototype dynamical core for global NWP. We use NIM to examine Fortran directive-based GPU compilers, evaluating code porting effort and computational performance.

The scalable modeling system: directive-based code parallelization for distributed and shared memory computers

A directive-based parallelization tool called the Scalable Modeling System (SMS) is described. The user inserts directives in the form of comments into existing Fortran code. SMS translates the code and directives into a parallel version that runs efficiently on shared and distributed memory high-performance computing platforms including the SGI Origin, IBM SP2, Cray T3E, Sun, and Alpha and Intel clusters. Twenty directives are available to support operations including array re-declarations, inter-process communications, loop translations, and parallel I/O operations. SMS also provides tools to support incremental parallelization and debugging that significantly reduces code parallelization time from months to weeks of effort. SMS is intended for applications using regular structured grids that are solved using finite difference approximation or spectral methods. It has been used to parallelize 10 atmospheric and oceanic models, but the tool is sufficiently general that it can be applied to other structured grids codes. Recent performance comparisons demonstrate that the Eta, Hybrid Coordinate Ocean model and Regional Ocean Modeling System model, parallelized using SMS, perform as well or better than their OpenMP or Message Passing Interface counterparts.

Parallelizing Operational Weather Forecast Models for Portable and Fast Execution

This paper describes a high-level library (The Nearest Neighbor Tool, NNT) that has been used to parallelize operational weather prediction models. NNT is part of the Scalable Modeling System (SMS), developed at the Forecast Systems Laboratory (FSL). Programs written in NNT rely on SMSs run-time system and port between a wide range of computing platforms, performing well in multiprocessor systems. We show, using examples from operational weather models, how large Fortran 77 codes can be parallelized using NNT. We compare the ease of programmability of NNT and High Performance Fortran (HPF). We also discuss optimizations like data movement overlap (in interprocessor communication and I/O operations), and the minimization of data exchanges through the use of redundant computations. We show that although HPF provides a simpler programming interface, NNT allows for program optimizations that increase performance considerably and still keeps a simple user interface. These optimizations have proven essential to run weather prediction models in real time, and HPF compilers should incorporate them in order to meet operational demands. Throughout the paper, we present performance results of weather models running on a network of workstations, the Intel Paragon, and the SGI Challenge. Finally, we study the cost of programming global address space architectures with NNTs local address space paradigm.

PERFORMANCE AND PORTABILITY IN PARALLEL COMPUTING: A WEATHER FORECAST VIEW

We have developed a high level library, the Nearest Neighbor Tool (NNT), to facilitate the coding of finite difference approximation weather prediction models on parallel computers. NNT provides portability and ease of programming and at the same time optimizes performance by allowing the overlap of computation and communication to tolerate the latency of remote data moves. In this paper we describe NNT and the implementation of the Well Posed Topographic model (WPT), a finite difference approximation weather prediction model. We present a qualitative study of the performance of the code on various multiprocessors and evaluate the effectiveness of NNT.

Comparison of shared memory and distributed memory parallelisation strategies for grid-based weather forecast models

We have parallelized a grid-based weather forecast model called SEQN using two programming models: shared memory and message passing. By shared memory we mean programming in standard Fortran 77 with directives for parallelism, such as is found on the Kendall Square Research KSR1 parallel supercomputer. For message passing we used the distributed memory Intel Paragon. We have benchmarked both versions of the code on the respective machines, and have run the message passing version on the KSR1 in order to directly compare performance and evaluate the cost of portability. In addition we present first results from the KSR2.<<ETX>>

Comparing scalable programming techniques for weather prediction

In this paper we study the of issues of programmability and performance in the parallelization of weather prediction models. We compare parallelization using a high level library (the Nearest Neighbor Tool: NNT) and a high level language/directive approach (High Performance Fortran: HPF). We report on the performance of a complete weather prediction model (the Rapid Update Cycle, which is currently run operationally at the National Meteorological Center at Washington) coded using NNT. We quantify the performance effects of optimizations possible with NNT that must be performed by an HPF compiler.

THE SCALABLE MODELING SYSTEM: A HIGH-LEVEL ALTERNATIVE TO MPI

A directive-based parallelization tool called the Scalable Modeling System (SMS) is described. The user inserts directives in the form of comments into existing Fortran code. SMS translates the code and directives into a parallel version that runs efficiently on both shared and distributed memory high-performance computing platforms. SMS provides tools to support partial parallelization and debugging that significantly decreases code parallelization time. The performance of an SMS parallelized version of the Eta model is compared to the operational version running at the National Centers for Environmental Prediction (NCEP).