Peter D. Barnes

Warp speed: executing time warp on 1,966,080 cores

Time Warp is an optimistic synchronization protocol for parallel discrete event simulation that coordinates the available parallelism through its rollback and antimessage mechanisms. In this paper we present the results of a strong scaling study of the ROSS simulator running Time Warp with reverse computation and executing the well-known PHOLD benchmark on Lawrence Livermore National Laboratorys Sequoia Blue Gene/Q supercomputer. The benchmark has 251 million PHOLD logical processes and was executed in several configurations up to a peak of 7.86 million MPI tasks running on 1,966,080 cores. At the largest scale it processed 33 trillion events in 65 seconds, yielding a sustained speed of 504 billion events/second using 120 racks of Sequoia. This is by far the highest event rate reported by any parallel discrete event simulation to date, whether running PHOLD or any other benchmark. Additionally, we believe it is likely to be the largest number of MPI tasks ever used in any computation of any kind to date. ROSS exhibited a super-linear speedup throughout the strong scaling study, with more than a 97x speed improvement from scaling the number of cores by only 60x (from 32,768 to 1,966,080). We attribute this to significant cache-related performance acceleration as we moved to higher scales with fewer LPs per core. Prompted by historical performance results we propose a new, long term performance metric called Warp Speed that grows logarithmically with the PHOLD event rate. As we define it our maximum speed of 504 billion PHOLD events/sec corresponds to Warp 2.7. We suggest that the results described here are significant because they demonstrate that direct simulation of planetary-scale discrete event models are now, in principle at least, within reach.

Proton radiography as a means of material characterization

We describe how protons with energies of 800 MeV or greater can be used as radiographic probes for material characterization. A feature which distinguishes protons from x-rays is their charge, which results in multiple Coulomb scattering effects in proton radiographs. Magnetic lensing can ameliorate these effects and even allow mixed substances to be disentangled. We illustrate some of these effects using 800 MeV protons radiographs of a composite step wedge composed of Aluminum, Foam, and Graphite. We discuss how proton radiographs must be manipulated in order to use standard tomographic reconstruction algorithms. We conclude with a brief description of an upcoming experiment, which will be performed at Brookhaven National Laboratory at 25 GeV.

Large-scale parallel simulations of distributed detection algorithms for collaborative autonomous sensor networks

A key component of the Third Offset Strategy proposed by the United States Department of Defense is the use of unmanned autonomous systems to deter potential conflicts. Collaborative autonomy technologies are also being explored by the private sector, which is rapidly pushing towards the deployment of self-driving vehicles. For areas affected by disaster, autonomous drone swarms can assist with search and rescue operations by surveilling large regions quickly without exposing emergency responders to risk prematurely. A substantial amount of progress has been made in distributed sensing research over the last few years. However, simulation results for applications that require complex inter-agent communications have rarely been demonstrated at scale; these simulations are generally executed using tens or hundreds of agents rather than the thousands or tens of thousands envisioned for large autonomous swarms. We address this deficit here by presenting two contributions. First, we extend our previous work on efficient, distributed algorithms for weak radiation source detection to accommodate the use case of surveillance across a very wide area. We then demonstrate the efficacy of the proposed algorithms at scale using a parallelized version of the ns-3 discrete event simulator.

Exploratory Modeling and Simulation of the Evolutionary Dynamics of Single-Stranded RNA Virus Populations

Infection, replication and mutation govern the population dynamics of viruses and are the key mechanisms driving their evolution. In particular, RNA viruses (such as the causative agents of Ebola, Dengue, Zika, West Nile, and SARS) have the highest mutation rates which enable them to form highly diverse populations within a single host, evade immune responses and develop resistances to drugs. Understanding the complexity of virus evolution is crucial for developing reliable countermeasures. We present an exploratory simulation model to study the evolution of heterogeneous virus populations in heterogeneous cell environments. This is a unique model that operates at three scales and captures the core mechanisms of the evolutionary process. To the best of our knowledge, this is the first HPC-based simulation of its kind.