David R. Jefferson

Analyzing internet voting security

An extensive assessment of a proposed Internet-based voting system.

Warp speed: executing time warp on 1,966,080 cores

Time Warp is an optimistic synchronization protocol for parallel discrete event simulation that coordinates the available parallelism through its rollback and antimessage mechanisms. In this paper we present the results of a strong scaling study of the ROSS simulator running Time Warp with reverse computation and executing the well-known PHOLD benchmark on Lawrence Livermore National Laboratorys Sequoia Blue Gene/Q supercomputer. The benchmark has 251 million PHOLD logical processes and was executed in several configurations up to a peak of 7.86 million MPI tasks running on 1,966,080 cores. At the largest scale it processed 33 trillion events in 65 seconds, yielding a sustained speed of 504 billion events/second using 120 racks of Sequoia. This is by far the highest event rate reported by any parallel discrete event simulation to date, whether running PHOLD or any other benchmark. Additionally, we believe it is likely to be the largest number of MPI tasks ever used in any computation of any kind to date. ROSS exhibited a super-linear speedup throughout the strong scaling study, with more than a 97x speed improvement from scaling the number of cores by only 60x (from 32,768 to 1,966,080). We attribute this to significant cache-related performance acceleration as we moved to higher scales with fewer LPs per core. Prompted by historical performance results we propose a new, long term performance metric called Warp Speed that grows logarithmically with the PHOLD event rate. As we define it our maximum speed of 504 billion PHOLD events/sec corresponds to Warp 2.7. We suggest that the results described here are significant because they demonstrate that direct simulation of planetary-scale discrete event models are now, in principle at least, within reach.

Reverse Code Generation for Parallel Discrete Event Simulation

Reverse computation has become a central notion in discrete event simulation over the last decade. It is not just a theoretical line of research, but an immensely practical one that is necessary to achieve high performance for large parallel discrete event simulations (PDES). The models that are implemented for PDES are of increasing complexity and size and require various language features to support abstraction, encapsulation, and composition when building a simulation model. In this paper we focus on parallel simulation models that are written in C++ and present an approach for automatically generating reverse code for C++. The strategy we have adopted for our approach is to first assure that we can correctly handle event methods that use the entire C++ language. Although a significant runtime overhead is introduced with our technique, the assurance that the reverse code is always generated fully automatically is an enormous win that can open the door to routine optimistic simulation with models that can be implemented using the entire C++ language.

The Backstroke framework for source level reverse computation applied to parallel discrete event simulation

We introduce Backstroke, a new open source framework for the automatic generation of reverse code for functions written in C++. Backstroke enables reverse computation for optimistic parallel discrete event simulations. It is built using the ROSE open-source compiler infrastructure, and handles complex C++ features including pointers and pointer types, arrays, function and method calls, class types, inheritance, polymorphism, virtual functions, abstract classes, templated classes and containers. Backstroke also introduces new program inversion techniques based on advanced compiler analysis tools built into ROSE. We explore and illustrate some of the complex language and semantic issues that arise in generating correct reverse code for C++ functions.

A new method for program inversion

Program inversion has been successfully applied to several areas such as optimistic parallel discrete event simulation (OPDES) and reverse debugging. This paper introduces a new program inversion algorithm for imperative languages, and focuses on handling arbitrary control flows and basic operations. By building a value search graph that represents recoverability relationships between variable values, we turn the problem of recovering previous values into a graph search one. Forward and reverse code is generated according to the search results. We have implemented our algorithm as part of a compiler framework named Backstroke, a C++ source-to-source translator based on ROSE compiler. Backstroke targets optimistic simulation codes and automatically generates a reverse function to recover values modified by a target function. Experimental results show that our method is effective and produces better performance than previously proposed methods.

Automatic Generation of Reversible C++ Code and Its Performance in a Scalable Kinetic Monte-Carlo Application

The fully automatic generation of code that establishes the reversibility of arbitrary C/C++ code has been a target of research and engineering for more than a decade as reverse computation has become a central notion in large scale parallel discrete event simulation (PDES). The simulation models that are implemented for PDES are of increasing complexity and size and require various language features to support abstraction, encapsulation, and composition when building a simulation model. In this paper we focus on parallel simulation models that are written in C++ and present an approach and an evaluation for a fully automatically generated reversible code for a kinetic Monte-Carlo application implemented in C++. Although a significant runtime overhead is introduced with our technique, the assurance that the reverse code is generated automatically and correctly, is an enormous win that allows simulation model developers to write forward event code using the entire C++ language, and have that code automatically transformed into reversible code to enable parallel execution with the Rensselaers Optimistic Simulation System (ROSS).

A modular voting architecture (“frog voting”)

This paper presents a new framework–a reference architecture–for voting that we feel has many attractive features. It is not a machine design, but rather a framework that will stimulate innovation and design. It is potentially the standard architecture for all future voting equipment. The ideas expressed here are subject to improvement and further research. An early version of this paper appeared in [2]. This version of the paper is very similar, but contains a postscript (Section 8) providing commentary and discussion of perspectives on this proposal generated during the intervening years between 2001 and 2008.

Improving per processor memory use of ns-3 to enable large scale simulations

In this paper we describe enhancements to improve the scaling of the ns-3 simulator for large problem sizes. The ns-3 simulator has a parallel capability however the current implementation instantiates the entire network topology on all ranks (processors). This restricts the problem sizes that could be run. We describe an approach to removing this limitation by distributing the network topology across ranks such that each rank only holds a part of the network topology. Performance studies were conducted to investigate the scaling performance of the modified ns-3 simulator.

Generation of Reversible C++ Code for Optimistic Parallel Discrete Event Simulation

The reversible execution of C/C++ code has been a target of research and engineering for more than a decade as reversible computation has become a central notion in large-scale parallel discrete event simulation (PDES). The simulation models that are implemented for PDES are of increasing complexity and size and require various language features to support abstraction, encapsulation, and composition when building a simulation model. In this paper, we focus on parallel simulation models that are written with user-defined C++ abstractions and abstractions of the C++ Standard Library. We present an approach based on incremental state saving for establishing reversibility of C++ and an evaluation for a kinetic Monte-Carlo simulation implemented in C++. Although a significant runtime overhead is introduced with our technique, it is an enormous win that it allows using the entire C++ language, and has that code automatically transformed into reversible code to enable parallel execution with the Rensselaer’s optimistic simulation system (ROSS).

Dealing with Reversibility of Shared Libraries in PDES

State recoverability is a crucial aspect of speculative Time Warp-based Parallel Discrete Event Simulation. In the literature, we can identify three major classes of techniques to support the correct restoration of a previous simulation state upon the execution of a rollback operation: state checkpointing/restore, manual reverse computation and automatic reverse computation. The latter class has been recently supported by relying either on binary code instrumentation or on source-to-source code transformation. Nevertheless, both solutions are not intrinsically meant to support a reversible execution of third-party shared libraries, which can be pretty useful when implementing complex simulation models. In this paper, we present an architectural solution (realized as a static C library) which allows to transparently instrument at runtime any third party shared library, with no need for any modification to the models code. We also present a preliminary experimental evaluation, based on the integration of our library with the ROOT-Sim simulation engine.