Vitali A. Morozov

Investigation of Catalytic Finite-Size-Effects of Platinum Metal Clusters

In this paper, we use density functional theory (DFT) calculations on highly parallel computing resources to study size-dependent changes in the chemical and electronic properties of platinum (Pt) for a number of fixed freestanding clusters ranging from 13 to 1415 atoms, or 0.7-3.5 nm in diameter. We find that the surface catalytic properties of the clusters converge to the single crystal limit for clusters with as few as 147 atoms (1.6 nm). Recently published results for gold (Au) clusters showed analogous convergence with size. However, this convergence happened at larger sizes, because the Au d-states do not contribute to the density of states around the Fermi-level, and the observed level fluctuations were not significantly damped until the cluster reached ca. 560 atoms (2.7 nm) in size.

GROPHECY: GPU performance projection from CPU code skeletons

We propose GROPHECY, a GPU performance projection framework that can estimate the performance benefit of GPU acceleration without actual GPU programming or hardware. Users need only to skeletonize pieces of CPU code that are targets for GPU acceleration. Code skeletons are automatically transformed in various ways to mimic tuned GPU codes with characteristics resembling real implementations. The synthesized characteristics are used by an existing analytical model to project GPU performance. The cost and benefit of GPU development can then be estimated according to the transformed code skeleton that yields the best projected performance. With GROPHECY, users can leap toward GPU acceleration only when the cost-benefit makes sense. The framework is validated using kernel benchmarks and data-parallel codes in legacy scientific applications. The measured performance of manually tuned codes deviates from the projected performance by 17% in geometric mean.

Topology-aware data movement and staging for I/O acceleration on Blue Gene/P supercomputing systems

There is growing concern that I/O systems will be hard pressed to satisfy the requirements of future leadership-class machines. Even current machines are found to be I/O bound for some applications. In this paper, we identify existing performance bottlenecks in data movement for I/O on the IBM Blue Gene/P (BG/P) supercomputer currently deployed at several leadership computing facilities. We improve the I/O performance by exploiting the network topology of BG/P for collective I/O, leveraging data semantics of applications and incorporating asynchronous data staging. We demonstrate the efficacy of our approaches for synthetic benchmark experiments and for application-level benchmarks at scale on leadership computing systems.

HACC: Simulating sky surveys on state-of-the-art supercomputing architectures

Abstract Current and future surveys of large-scale cosmic structure are associated with a massive and complex datastream to study, characterize, and ultimately understand the physics behind the two major components of the ‘Dark Universe’, dark energy and dark matter. In addition, the surveys also probe primordial perturbations and carry out fundamental measurements, such as determining the sum of neutrino masses. Large-scale simulations of structure formation in the Universe play a critical role in the interpretation of the data and extraction of the physics of interest. Just as survey instruments continue to grow in size and complexity, so do the supercomputers that enable these simulations. Here we report on HACC (Hardware/Hybrid Accelerated Cosmology Code), a recently developed and evolving cosmology N-body code framework, designed to run efficiently on diverse computing architectures and to scale to millions of cores and beyond. HACC can run on all current supercomputer architectures and supports a variety of programming models and algorithms. It has been demonstrated at scale on Cell- and GPU-accelerated systems, standard multi-core node clusters, and Blue Gene systems. HACC’s design allows for ease of portability, and at the same time, high levels of sustained performance on the fastest supercomputers available. We present a description of the design philosophy of HACC, the underlying algorithms and code structure, and outline implementation details for several specific architectures. We show selected accuracy and performance results from some of the largest high resolution cosmological simulations so far performed, including benchmarks evolving more than 3.6 trillion particles.

Accelerating I/O Forwarding in IBM Blue Gene/P Systems

Current leadership-class machines suffer from a significant imbalance between their computational power and their I/O bandwidth. I/O forwarding is a paradigm that attempts to bridge the increasing performance and scalability gap between the compute and I/O components of leadership-class machines to meet the requirements of data-intensive applications by shipping I/O calls from compute nodes to dedicated I/O nodes. I/O forwarding is a critical component of the I/O subsystem of the IBM Blue Gene/P supercomputer currently deployed at several leadership computing facilities. In this paper, we evaluate the performance of the existing I/O forwarding mechanisms for BG/P and identify the performance bottlenecks in the current design. We augment the I/O forwarding with two approaches: I/O scheduling using a work-queue model and asynchronous data staging. We evaluate the efficacy of our approaches using microbenchmarks and application-level benchmarks on leadership class systems.

HACC: extreme scaling and performance across diverse architectures

Supercomputing is evolving towards hybrid and accelerator-based architectures with millions of cores. The HACC (Hardware/Hybrid Accelerated Cosmology Code) framework exploits this diverse landscape at the largest scales of problem size, obtaining high scalability and sustained performance. Developed to satisfy the science requirements of cosmological surveys, HACC melds particle and grid methods using a novel algorithmic structure that flexibly maps across architectures, including CPU/GPU, multi/many-core, and Blue Gene systems. We demonstrate the success of HACC on two very different machines, the CPU/GPU system Titan and the BG/Q systems Sequoia and Mira, attaining unprecedented levels of scalable performance. We demonstrate strong and weak scaling on Titan, obtaining up to 99.2% parallel efficiency, evolving 1.1 trillion particles. On Sequoia, we reach 13.94 PFlops (69.2% of peak) and 90% parallel efficiency on 1,572,864 cores, with 3.6 trillion particles, the largest cosmological benchmark yet performed. HACC design concepts are applicable to several other supercomputer applications.

A new computational paradigm in multiscale simulations: application to brain blood flow

Interfacing atomistic-based with continuum-based simulation codes is now required in many multiscale physical and biological systems. We present the computational advances that have enabled the first multiscale simulation on 190,740 processors by coupling a high-order (spectral element) Navier-Stokes solver with a stochastic (coarse-grained) Molecular Dynamics solver based on Dissipative Particle Dynamics (DPD). The key contributions are proper interface conditions for overlapped domains, topology-aware communication, SIMDization, multiscale visualization and a new do- main partitioning for atomistic solvers. We study blood flow in a patient-specific cerebrovasculature with a brain aneurysm, and analyze the interaction of blood cells with the arterial walls endowed with a glycocalyx causing thrombus formation and eventual aneurysm rupture. The macro-scale dynamics (about 3 billion unknowns) are resolved by NεκTαr - a spectral element solver; the micro-scale flow and cell dynamics within the aneurysm are resolved by an in-house version of DPD-LAMMPS (for an equivalent of about 100 billions molecules).

Numerical Simulation of Laser-Produced Plasma Devices for EUV Lithography Using the Heights Integrated Model

ABSTRACT Laser-produced plasma (LPP) devices have been modeled as the light source for extreme ultraviolet (EUV) lithography. A key challenge for LPP is achieving sufficient brightness to support the throughput requirements of high-volume manufacturing. An integrated model (HEIGHTS) was applied to simulate the environment of EUV sources and optimize their output. The model includes plasma evolution and magnetohydrodynamic processes in a two-temperature approximation, as well as photon radiation transport determined by the Monte Carlo method. It uses the total variation diminishing scheme for the description of magnetic compression and diffusion in a cylindrical 2-D geometry for the target. Generation of the internal magnetic field with nonparallel density and temperature gradients was also considered. Preliminary results from numerical simulation in hydrodynamics and line radiation output of xenon and tin plasmas are presented for planar and droplet targets.

HEIGHTS initial simulation of discharge produced plasma hydrodynamics and radiation transport for extreme ultraviolet lithography

Discharge-produced plasma (DPP) devices have been proposed as a light source for EUV lithography. A key challenge for DPP is achieving sufficient brightness to support the throughput requirements of exposure tools for high-volume manufacturing lithography. To simulate the environment of the EUV source and optimize the output of the source, an integrated model is being developed to describe the hydrodynamic and optical processes that occur in DPP devices. The model includes both plasma evolution and magnetohydrodynamic processes as well as detailed photon radiation transport. The total variation diminishing scheme in the Lax-Friedrich formulation for the description of magnetic compression and diffusion in a cylindrical geometry is used. Several models are being developed for opacity calculations: a collisional radiation equilibrium model, a self-consistent field model with Auger processes, and a nonstationary kinetic model. Radiation transport for both continuum and lines with detailed spectral profiles are taken into account. The developed models are integrated into the HEIGHTS-EUV computer simulation package. Preliminary results of a numerical simulation of xenon gas hydrodynamics and EUV radiation output are presented for various plasma conditions.

Simulation and optimization of DPP hydrodynamics and radiation transport for EUV lithography devices

Discharge produced plasma (DPP) devices are being used as a light source for Extreme Ultraviolet (EUV) Lithography. A key challenge for DPP is achieving sufficient brightness to support the throughput requirements of exposure tools for high-volume manufacturing lithography. An integrated model is being developed to simulate the environment of the EUV source and optimize the output of the source. The model describes the hydrodynamic and optical processes that occur in DPP devices. It takes into account plasma evolution and magnetohydrodynamic processes as well as detailed photon radiation transport. The total variation diminishing scheme in the Lax-Friedrich formulation for the description of magnetic compression and diffusion in a cylindrical geometry is used. Several models are being developed for opacity calculations: a collisional radiation equilibrium model, a self-consistent field model with Auger processes, and a non-stationary kinetic model. Radiation transport for both continuum and lines with detailed spectral profiles is taken into account. The developed models are being integrated into the HEIGHTS-EUV computer simulation package. Preliminary results of a numerical simulation of xenon gas hydrodynamics and EUV radiation output are presented for various plasma conditions.