Alice Koniges

Multiphysics simulations: Challenges and opportunities

We consider multiphysics applications from algorithmic and architectural perspectives, where “algorithmic” includes both mathematical analysis and computational complexity, and “architectural” includes both software and hardware environments. Many diverse multiphysics applications can be reduced, en route to their computational simulation, to a common algebraic coupling paradigm. Mathematical analysis of multiphysics coupling in this form is not always practical for realistic applications, but model problems representative of applications discussed herein can provide insight. A variety of software frameworks for multiphysics applications have been constructed and refined within disciplinary communities and executed on leading-edge computer systems. We examine several of these, expose some commonalities among them, and attempt to extrapolate best practices to future systems. From our study, we summarize challenges and forecast opportunities.

MPI-IO/GPFS, an Optimized Implementation of MPI-IO on Top of GPFS

MPI-IO/GPFS is an optimized prototype implementation of the I/O chapter of the Message Passing Interface (MPI) 2 standard. It uses the IBM General Parallel File System (GPFS) Release 3 as the underlying file system. This paper describes optimization features of the prototype that take advantage of new GPFS programming interfaces. It also details how collective data access operations have been optimized by minimizing the number of messages exchanged in sparse accesses and by increasing the overlap of communication with file access. Experimental results show a performance gain. A study of the impact of varying the number of tasks running on the same node is also presented.

Performance of the IBM general parallel file system

We measure the performance and scalability of IBMs General Parallel File System (GPFS) under a variety of conditions. The measurements are based on benchmark programs that allow us to vary block sizes, access patterns, etc., and to measure aggregate throughput rates. We use the data to give performance recommendations for application development and as a guide to the improvement of parallel file systems.

Hard x-ray and hot electron environment in vacuum hohlraums at the National Ignition Facility

Time resolved hard x-ray images (hv>9keV) and time integrated hard x-ray spectra (hv=18–150keV) from vacuum hohlraums irradiated with four 351nm wavelength National Ignition Facility [J. A. Paisner, E. M. Campbell, and W. J. Hogan, Fusion Technol. 26, 755 (1994)] laser beams are presented as a function of hohlraum size, laser power, and duration. The hard x-ray images and spectra provide insight into the time evolution of the hohlraum plasma filling and the production of hot electrons. The fraction of laser energy detected as hot electrons (Fhot) shows a correlation with laser intensity and with an empirical hohlraum plasma filling model. In addition, the significance of Au K-alpha emission and Au K-shell reabsorption observed in some of the bremsstrahlung dominated spectra is discussed.

Multithreaded Global Address Space Communication Techniques for Gyrokinetic Fusion Applications on Ultra-Scale Platforms

We present novel parallel language constructs for the communication intensive part of a magnetic fusion simulation code. The focus of this work is the shift phase of charged particles of a tokamak simulation code in toroidal geometry. We introduce new hybrid PGAS/OpenMP implementations of highly optimized hybrid MPI/OpenMP based communication kernels. The hybrid PGAS implementations use an extension of standard hybrid programming techniques, enabling the distribution of high communication work loads of the underlying kernel among OpenMP threads. Building upon lightweight one-sided CAF (Fortran 2008) communication techniques, we also show the benefits of spreading out the communication over a longer period of time, resulting in a reduction of bandwidth requirements and a more sustained communication and computation overlap. Experiments on up to 130560 processors are conducted on the NERSC Hopper system, which is currently the largest HPC platform with hardware support for one-sided communication and show performance improvements of 52% at highest concurrency.

Performance evaluation of supercomputers using HPCC and IMB Benchmarks

The HPC Challenge (HPCC) Benchmark suite and the Intel MPI Benchmark (IMB) are used to compare and evaluate the combined performance of processor, memory subsystem and interconnect fabric of five leading supercomputers-SGI Altix BX2, Cray X1, Cray Opteron Cluster, Dell Xeon Cluster, and NEC SX-8. These five systems use five different networks (SGI NUMALINK4, Cray network, Myrinet, InfiniBand, and NEC IXS). The complete set of HPCC Benchmarks are run on each of these systems. Additionally, we present Intel MPI Benchmarks results to study the performance of 11 MPI communication functions on these systems.

Effective Communication and File-I/O Bandwidth Benchmarks

We describe the design and MPI implementation of two benchmarks created to characterize the balanced system performance of high-performance clusters and supercomputers: b eff, the communication-specific benchmark examines the parallel message passing performance of a system, and b e_ff_io, which characterizes the effective I/O bandwidth. Both benchmarks have two goals: a) to get a detailed insight into the performance strengths and weaknesses of different parallel communication and I/O patterns, and based on this, b) to obtain a single bandwidth number that characterizes the average performance of the system namely communication and I/O bandwidth. Both benchmarks use a time-driven approach and loop over a variety of communication and access patterns to characterize a system in an automated fashion. Results of the two benchmarks are given for several systems including IBM SPs, Cray T3E, NEC SX-5, and Hitachi SR 8000. After a redesign of b eff io, I/O bandwidth results for several compute partition sizes are achieved in an appropriate time for rapid benchmarking.

A programming model performance study using the NAS parallel benchmarks

Harnessing the power of multicore platforms is challenging due to the additional levels of parallelism present. In this paper we use the NAS Parallel Benchmarks to study three programming models, MPI, OpenMP and PGAS to understand their performance and memory usage characteristics on current multicore architectures. To understand these characteristics we use the Integrated Performance Monitoring tool and other ways to measure communication versus computation time, as well as the fraction of the run time spent in OpenMP. The benchmarks are run on two different Cray XT5 systems and an Infiniband cluster. Our results show that in general the three programming models exhibit very similar performance characteristics. In a few cases, OpenMP is significantly faster because it explicitly avoids communication. For these particular cases, we were able to re-write the UPC versions and achieve equal performance to OpenMP. Using OpenMP was also the most advantageous in terms of memory usage. Also we compare performance differences between the two Cray systems, which have quad-core and hex-core processors. We show that at scale the performance is almost always slower on the hex-core system because of increased contention for network resources.

Towards a High-Performance Implementation of MPI-IO on Top of GPFS

MPI-IO/GPFS is a prototype implementation of the I/O chapter of the Message Passing Interface (MPI) 2 standard. It uses the IBM General Parallel File System (GPFS) as the underlying file system. This paper describes the features of this prototype which support its high performance. The use of hints allows tailoring the use of the file system to the application needs.

Laser coupling to reduced-scale hohlraum targets at the Early Light Program of the National Ignition Facility

A platform for analysis of material properties under extreme conditions, where a sample is bathed in radiation with a high temperature, is under development. Depositing maximum laser energy into a small, high-Z enclosure produces this hot environment. Such targets were recently included in an experimental campaign using the first four of the 192 beams of the National Ignition Facility [J. A. Paisner, E. M. Campbell, and W. J. Hogan, Fusion Technol. 26, 755 (1994)], under construction at the University of California Lawrence Livermore National Laboratory. These targets demonstrate good laser coupling, reaching a radiation temperature of 340 eV. In addition, there is a unique wavelength dependence of the Raman backscattered light that is consistent with Brillouin backscatter of Raman forward scatter [A. B. Langdon and D. E. Hinkel, Phys. Rev. Lett. 89, 015003 (2002)]. Finally, novel diagnostic capabilities indicate that 20% of the direct backscatter from these reduced-scale targets is in the polarization or...