Michael L. Gittings

Predictive Performance and Scalability Modeling of a Large-Scale Application

In this work we present a predictive analytical model that encompasses the performance and scaling characteristics of an important ASCI application. SAGE (SAIC’s Adaptive Grid Eulerian hydrocode) is a multidimensional hydrodynamics code with adaptive mesh refinement. The model is validated against measurements on several systems including ASCI Blue Mountain, ASCI White, and a Compaq Alphaserver ES45 system showing high accuracy. It is parametric - basic machine performance numbers (latency, MFLOPS rate, bandwidth) and application characteristics (problem size, decomposition method, etc.) serve as input. The model is applied to add insight into the performance of current systems, to reveal bottlenecks, and to illustrate where tuning efforts can be effective. We also use the model to predict performance on future systems.

Richtmyer–Meshkov instability growth: experiment, simulation and theory

Richtmyer–Meshkov instability is investigated for negative Atwood number and two-dimensional sinusoidal perturbations by comparing experiments, numerical simulations and analytic theories. The experiments were conducted on the NOVA laser with strong radiatively driven shocks with Mach numbers greater than 10. Three different hydrodynamics codes (RAGE, PROMETHEUS and FronTier) reproduce the amplitude evolution and the gross features in the experiment while the fine-scale features differ in the different numerical techniques. Linearized theories correctly calculate the growth rates at small amplitude and early time, but fail at large amplitude and late time. A nonlinear theory using asymptotic matching between the linear theory and a potential flow model shows much better agreement with the late-time and large-amplitude growth rates found in the experiments and simulations. We vary the incident shock strength and initial perturbation amplitude to study the behaviour of the simulations and theory and to study the effects of compression and nonlinearity.

Massively Parallel Simulations with DOE?s ASCI Supercomputers: An Overview of the Los Alamos Crestone Project

The Los Alamos Crestone Project is part of the Department of Energy’s (DOE) Accelerated Strategic Computing Initiative, or ASCI Program. The main goal of this software development project is to investigate the use of continuous adaptive mesh refinement (CAMR) techniques for application to problems of interest to the Laboratory. There are many code development efforts in the Crestone Project, both unclassified and classified codes. In this overview I will discuss the unclassified SAGE and the RAGE codes. The SAGE (SAIC adaptive grid Eulerian) code is a one-, two-, and three-dimensional, multimaterial, Eulerian, massively parallel hydrodynamics code for use in solving a variety of high-deformation flow problems. The RAGE CAMR code is built from the SAGE code by adding various radiation packages, improved setup utilities, and graphics packages. It is used for problems in which radiation transport of energy is important. The goal of these massively-parallel versions of the SAGE and RAGE codes is to run extremely large problems in a reasonable amount of calendar time. Our target is scalable performance to ∼10,000 processors on a 1 billion CAMR computational cell problem that requires hundreds of variables per cell, multiple physics packages (e.g., radiation and hydrodynamics), and implicit matrix solves for each cycle. A general description of the RAGE code has been published in [1], [2], [3] and [4].

Using pulsed power for hydrodynamic code verification and validation

Summary form only given, as follows. A series of Near Term Liner Experiments (NTLX) was recently, conducted on the Shiva Star capacitor bank at the AFRL. These experiments consisted of an aluminum liner that is magnetically imploded onto a central target by self-induced radial Lorentz forces. The central target consists of an inner Lucite cylinder surrounded by an outer Sn layer. Shock propagation within the Lucite is measured to provide information for hydrodynamic code verification and validation. Target design utilized the adaptive mesh refinement (AMR) Eulerian hydrodynamics code RAGE in 2- and 3D. 1D simulations of the liner driver utilizing the RAVEN MHD code set initial liner/target interaction parameters, which are then used as initial conditions for the RAGE calculations. Both codes utilized standard SESAME equation-of-state data to close the system of equations. At liner/target impact, a convergent shock is generated that drives subsequent hydrodynamics experiments. In concentric targets, cylindrically symmetric shocks will converge on axis, characterizing the symmetry of the liner driver. By shifting the target center away from the liner symmetry axis, variations in shock propagation velocity generate off-center shock convergence. Comparison of experimentally measured and simulated shock trajectories are discussed as are convergence effects associated with cylindrical geometry. Efforts are currently underway to compare equation-of-state effects by utilizing a Gruneisen EOS instead of the original SESAME tables. Further comparisons are made to quasi-analytical solutions in cylindrical geometry [Harlow, F.H. and Amsden, A.A., Fluid Dynamics, Los Alamos National Laboratory Report LA-4700, (1971)] to examine shock acceleration due to radial convergence.

Using pulsed power for hydrodynamic code validation

As part of ongoing hydrodynamic code validation efforts, a series of Near Term Liner Experiments (NTLX) was designed for the Shiva Star capacitor bank at the Air Force Research Laboratory (AFRL). A cylindrical aluminum liner that is magnetically imploded onto a central target by self-induced radial Lorentz forces drove the experiments. The behavior of the target was simulated using the adaptive mesh refinement (AMR) Eulerian hydrodynamics code RAGE in 2- and 3-D. One-dimensional simulations of the liner driver utilizing the RAVEN MHD code were used to predict the liner density and temperature profiles as well as the velocity at impact time. At liner/target impact, a convergent shock is generated that drives subsequent hydrodynamics experiments. In concentric targets, cylindrically symmetric shocks will converge on axis. This characterizes the symmetry of the liner driver. By shifting the target center away from the liner symmetry axis, material dependencies in the shock propagation velocity generate off-center shock convergence. Both codes show excellent agreement over the majority of the shock and interface trajectories. However, a small but significant discrepancy between codes does occur during the last few millimeters of run in when convergence effects are greatest. Comparisons with experimental data show similar shock velocities being measured. However, the simulated shock arrives at a given trajectory location nearly 100 ns earlier than experimentally measured.

Calculations of Tsunamis from Submarine Landslides

Great underwater landslides like Storegga off the Norwegian coast leave massive deposits on the seafloor and probably produce enormous tsunamis. We have performed a numerical study of such landslides using the multi-material compressible hydrocode Sage in order to understand the relationship between the rheology of the slide material, the configuration of the resulting deposits on the seafloor, and the tsunami that is produced. Instabilities in the fluid-fluid mixing between slide material and seawater produce vortices and swirls with sizes that depend on the rheology of the slide material. These dynamical features of the flow may be preserved as ridges when the sliding material finally stops. Thus studying the configuration of the morphology of prehistoric slide relics on the abyssal plain may help us understand the circumstances under which the slide was initiated.

The Los Alamos Crestone Project: cluster computing applications

Summary form only given. The Los Alamos Crestone Project is part of the Department of Energys (DOE) Accelerated Strategic Computing Initiative, or ASCI Program. The main goal of this software development project is to investigate the use of continuous adaptive mesh refinement (CAMR) techniques for application to problems of interest to the Laboratory. There are many code development efforts in the Crestone Project, both unclassified and classified codes. An overview of the Crestone Project, and the SAGE and RAGE codes, has been published recently in Weaver and Gittings (2003). In This work, I will give the status of the use of these CAMR codes on commodity cluster machines. One of the most economical methods for achieving supercomputing capability is to use commodity processors connected by commodity interconnects. This was highlighted recently at Virginia Tech when Dr. Varadarajan built the third fastest supercomputer in the world by connecting 1100 dual-processor Macintosh G5 machines together (see http://www.top500.org). Most commodity clusters use a form of LINUX as the operating system. We will give an overview of the current status of using the Crestone Project codes SAGE and RAGE on commodity cluster machines. These codes are intended for general applications without tuning of algorithms or parameters. We have run a wide variety of physical applications from millimeter-scale laboratory laser experiments, to the multikilometer-scale asteroid impacts into the Pacific Ocean, to parsec-scale galaxy formation. Examples of these simulations will be shown. The goal of our effort is to avoid ad hoc models and attempt to rely on first-principles physics. In addition to the large effort on developing parallel code physics packages, a substantial effort in the project is devoted to improving the computer science and software quality engineering (SQE) of the Project codes as well as a sizable effort on the verification and validation (V&V) of the resulting codes. Examples of these efforts for our project will be discussed. Recent results of the scaling of these codes on commodity clusters will be shown.

NUMERICAL MODELING OF MUNROE JETS

Munroe jets are formed by the oblique interaction of detonation products from two explosive charges separated by an air gap. The jet consists of a high velocity jet of low density precursor gases and particles that travel faster than the primary jet which is a high pressure regular reflection. The Los Alamos PHERMEX Data Volumes contain 40 radiographs taken by Douglas Venable in the 1960s of Munroe Jets generated by Composition B explosive charges separated by 5 to 80 mm of air. In several of the experiments the Munroe jets interacted with thin Tantalum foils and with aluminum plates. The PHERMEX experiments were modeled using the AMR Eulerian reactive hydrodynamic code NOBEL. When the detonation arrives at the bottom of the gap, the detonation products expand against the air and precursor gases travel at high velocity ahead of the detonation wave in the explosive. The expanding detonation products from the explosive collide and result in a high pressure shock reflection. Interaction with a metal plate c...