Tomas Oppelstrup

Enabling strain hardening simulations with dislocation dynamics

Numerical algorithms for discrete dislocation dynamics simulations are investigated for the purpose of enabling strain hardening simulations of single crystals on massively parallel computers. The algorithms investigated include the /(N) calculation of forces, the equations of motion, time integration, adaptive mesh refinement, the treatment of dislocation core reactions, and the dynamic distribution of work on parallel computers. A simulation integrating all of these algorithmic elements using the Parallel Dislocation Simulator (ParaDiS) code is performed to understand their behavior in concert, and evaluate the overall numerical performance of dislocation dynamics simulations and their ability to accumulate percents of plastic strain.

A First-Passage Kinetic Monte Carlo algorithm for complex diffusion-reaction systems

We develop an asynchronous event-driven First-Passage Kinetic Monte Carlo (FPKMC) algorithm for continuous time and space systems involving multiple diffusing and reacting species of spherical particles in two and three dimensions. The FPKMC algorithm presented here is based on the method introduced in Oppelstrup et al. [10] and is implemented in a robust and flexible framework. Unlike standard KMC algorithms such as the n-fold algorithm, FPKMC is most efficient at low densities where it replaces the many small hops needed for reactants to find each other with large first-passage hops sampled from exact time-dependent Greens functions, without sacrificing accuracy. We describe in detail the key components of the algorithm, including the event-loop and the sampling of first-passage probability distributions, and demonstrate the accuracy of the new method. We apply the FPKMC algorithm to the challenging problem of simulation of long-term irradiation of metals, relevant to the performance and aging of nuclear materials in current and future nuclear power plants. The problem of radiation damage spans many decades of time-scales, from picosecond spikes caused by primary cascades, to years of slow damage annealing and microstructure evolution. Our implementation of the FPKMC algorithm has been able to simulate the irradiation of a metal sample for durations that are orders of magnitude longer than any previous simulations using the standard Object KMC or more recent asynchronous algorithms.

First-passage kinetic Monte Carlo method.

We present an efficient method for Monte Carlo simulations of diffusion-reaction processes. Introduced by us in a previous paper [Phys. Rev. Lett. 97, 230602 (2006)], our algorithm skips the traditional small diffusion hops and propagates the diffusing particles over long distances through a sequence of superhops, one particle at a time. By partitioning the simulation space into nonoverlapping protecting domains each containing only one or two particles, the algorithm factorizes the N -body problem of collisions among multiple Brownian particles into a set of much simpler single-body and two-body problems. Efficient propagation of particles inside their protective domains is enabled through the use of time-dependent Greens functions (propagators) obtained as solutions for the first-passage statistics of random walks. The resulting Monte Carlo algorithm is event-driven and asynchronous; each Brownian particle propagates inside its own protective domain and on its own time clock. The algorithm reproduces the statistics of the underlying Monte Carlo model exactly. Extensive numerical examples demonstrate that for an important class of diffusion-reaction models the algorithm is efficient at low particle densities, where other existing algorithms slow down severely.

Towards real-time simulation of cardiac electrophysiology in a human heart at high resolution

We have developed the capability to rapidly simulate cardiac electrophysiological phenomena in a human heart discretised at a resolution comparable with the length of a cardiac myocyte. Previous scientific investigation has generally invoked simplified geometries or coarse-resolution hearts, with simulation duration limited to 10s of heartbeats. Using state-of-the-art high-performance computing techniques coupled with one of the most powerful computers available (the 20 PFlop/s IBM BlueGene/Q at Lawrence Livermore National Laboratory), high-resolution simulation of the human heart can now be carried out over 1200 times faster compared with published results in the field. We demonstrate the utility of this capability by simulating, for the first time, the formation of transmural re-entrant waves in a 3D human heart. Such wave patterns are thought to underlie Torsades de Pointes, an arrhythmia that indicates a high risk of sudden cardiac death. Our new simulation capability has the potential to impact a multitude of applications in medicine, pharmaceuticals and implantable devices.

Probing the limits of metal plasticity with molecular dynamics simulations

Along with high strength, plasticity is what makes metals so widely usable in our material world. Both strength and plasticity properties of a metal are defined by the motion of dislocations - line defects in the crystal lattice that divide areas of atomic planes displaced relative to each other by an interatomic distance. Here we present first fully dynamic atomistic simulations of single crystal plasticity in metal tantalum predicting that above certain maximum rate of straining - the ultimate limit - the dislocations can no longer relieve mechanical loads and another mechanism, twinning, comes into play and takes over as the dominant mode of dynamic response. At straining rates below the ultimate limit, the metal attains a path-independent stationary state of plastic flow in which both flow stress and dislocation density remain constant indefinitely for as long as the straining conditions remain unchanged. In this distinct state tantalum flows like a viscous fluid while still remaining a strong and stiff metal.Ordinarily, the strength and plasticity properties of a metal are defined by dislocations—line defects in the crystal lattice whose motion results in material slippage along lattice planes. Dislocation dynamics models are usually used as mesoscale proxies for true atomistic dynamics, which are computationally expensive to perform routinely. However, atomistic simulations accurately capture every possible mechanism of material response, resolving every “jiggle and wiggle” of atomic motion, whereas dislocation dynamics models do not. Here we present fully dynamic atomistic simulations of bulk single-crystal plasticity in the body-centred-cubic metal tantalum. Our goal is to quantify the conditions under which the limits of dislocation-mediated plasticity are reached and to understand what happens to the metal beyond any such limit. In our simulations, the metal is compressed at ultrahigh strain rates along its [001] crystal axis under conditions of constant pressure, temperature and strain rate. To address the complexity of crystal plasticity processes on the length scales (85–340 nm) and timescales (1 ns–1μs) that we examine, we use recently developed methods of in situ computational microscopy to recast the enormous amount of transient trajectory data generated in our simulations into a form that can be analysed by a human. Our simulations predict that, on reaching certain limiting conditions of strain, dislocations alone can no longer relieve mechanical loads; instead, another mechanism, known as deformation twinning (the sudden re-orientation of the crystal lattice), takes over as the dominant mode of dynamic response. Below this limit, the metal assumes a strain-path-independent steady state of plastic flow in which the flow stress and the dislocation density remain constant as long as the conditions of straining thereafter remain unchanged. In this distinct state, tantalum flows like a viscous fluid while retaining its crystal lattice and remaining a strong and stiff metal.

Atomistic simulations of grain boundary pinning in CuFe alloys

The authors apply a hybrid Monte Carlo-molecular dynamics code to the study of grain boundary motion upon annealing of pure Cu and Cu with low concentrations of Fe. The hybrid simulations account for segregation and precipitation of the low solubility Fe, together with curvature driven grain boundary motion. Grain boundaries in two different systems, a {Sigma}7+U-shaped half-loop grain and a nanocrystalline sample, were found to be pinned in the presence of Fe concentrations exceeding 3%.

Reverse Code Generation for Parallel Discrete Event Simulation

Reverse computation has become a central notion in discrete event simulation over the last decade. It is not just a theoretical line of research, but an immensely practical one that is necessary to achieve high performance for large parallel discrete event simulations (PDES). The models that are implemented for PDES are of increasing complexity and size and require various language features to support abstraction, encapsulation, and composition when building a simulation model. In this paper we focus on parallel simulation models that are written in C++ and present an approach for automatically generating reverse code for C++. The strategy we have adopted for our approach is to first assure that we can correctly handle event methods that use the entire C++ language. Although a significant runtime overhead is introduced with our technique, the assurance that the reverse code is always generated fully automatically is an enormous win that can open the door to routine optimistic simulation with models that can be implemented using the entire C++ language.

Toward real-time modeling of human heart ventricles at cellular resolution: simulation of drug-induced arrhythmias

We have developed a highly efficient and scalable cardiac electrophysiology simulation capability that supports groundbreaking resolution and detail to elucidate the mechanisms of sudden cardiac death from arrhythmia. We can simulate thousands of heartbeats at a resolution of 0.1 mm, comparable to the size of cardiac cells, thereby enabling scientific inquiry not previously possible. Based on scaling results from the partially deployed Sequoia IBM Blue Gene/Q machine at Lawrence Livermore National Laboratory and planned optimizations, we estimate that by SC12 we will simulate 8 -- 10 heartbeats per minute -- a time-to-solution 400 -- 500 times faster than the state-of-the-art. Performance between 8 and 11 PFlop/s on the full 1,572,864 cores is anticipated, representing 40 -- 55 percent of peak. The power of the model is demonstrated by illuminating the subtle arrhythmogenic mechanisms of anti-arrhythmic drugs that paradoxically increase arrhythmias in some patient populations.

Massively parallel models of the human circulatory system

The potential impact of blood flow simulations on the diagnosis and treatment of patients suffering from vascular disease is tremendous. Empowering models of the full arterial tree can provide insight into diseases such as arterial hypertension and enables the study of the influence of local factors on global hemodynamics. We present a new, highly scalable implementation of the lattice Boltzmann method which addresses key challenges such as multiscale coupling, limited memory capacity and bandwidth, and robust load balancing in complex geometries. We demonstrate the strong scaling of a three-dimensional, high-resolution simulation of hemodynamics in the systemic arterial tree on 1,572,864 cores of Blue Gene/Q. Faster calculation of flow in full arterial networks enables unprecedented risk stratification on a perpatient basis. In pursuit of this goal, we have introduced computational advances that significantly reduce time-to-solution for biofluidic simulations.

Automatic Generation of Reversible C++ Code and Its Performance in a Scalable Kinetic Monte-Carlo Application

The fully automatic generation of code that establishes the reversibility of arbitrary C/C++ code has been a target of research and engineering for more than a decade as reverse computation has become a central notion in large scale parallel discrete event simulation (PDES). The simulation models that are implemented for PDES are of increasing complexity and size and require various language features to support abstraction, encapsulation, and composition when building a simulation model. In this paper we focus on parallel simulation models that are written in C++ and present an approach and an evaluation for a fully automatically generated reversible code for a kinetic Monte-Carlo application implemented in C++. Although a significant runtime overhead is introduced with our technique, the assurance that the reverse code is generated automatically and correctly, is an enormous win that allows simulation model developers to write forward event code using the entire C++ language, and have that code automatically transformed into reversible code to enable parallel execution with the Rensselaers Optimistic Simulation System (ROSS).