Matthew Otten

An MPI/OpenACC implementation of a high-order electromagnetics solver with GPUDirect communication

We present performance results and an analysis of a message passing interface (MPI)/OpenACC implementation of an electromagnetic solver based on a spectral-element discontinuous Galerkin discretization of the time-dependent Maxwell equations. The OpenACC implementation covers all solution routines, including a highly tuned element-by-element operator evaluation and a GPUDirect gather–scatter kernel to effect nearest neighbor flux exchanges. Modifications are designed to make effective use of vectorization, streaming, and data management. Performance results using up to 16,384 graphics processing units of the Cray XK7 supercomputer Titan show more than 2.5× speedup over central processing unit-only performance on the same number of nodes (262,144 MPI ranks) for problem sizes of up to 6.9 billion grid points. We discuss performance-enhancement strategies and the overall potential of GPU-based computing for this class of problems.

Excited States of Methylene, Polyenes, and Ozone from Heat-Bath Configuration Interaction

The electronically excited states of methylene (CH2), ethylene (C2H4), butadiene (C4H6), hexatriene (C6H8), and ozone (O3) have long proven challenging due to their complex mixtures of static and dynamic correlations. The semistochastic heat-bath configuration interaction (SHCI) algorithm, which efficiently and systematically approaches the full configuration interaction (FCI) limit, is used to provide close approximations to the FCI energies in these systems. This article presents the largest FCI-level calculation to date on hexatriene, using a polarized double-ζ basis (ANO-L-pVDZ), which gives rise to a Hilbert space containing more than 1038 determinants. These calculations give vertical excitation energies of 5.58 and 5.59 eV, respectively, for the 21Ag and 11Bu states, showing that they are nearly degenerate. The same excitation energies in butadiene/ANO-L-pVDZ were found to be 6.58 and 6.45 eV. In addition to these benchmarks, our calculations strongly support the presence of a previously hypothesized ring-minimum species of ozone that lies 1.3 eV higher than the open-ring-minimum energy structure and is separated from it by a barrier of 1.11 eV.

Origins and optimization of entanglement in plasmonically coupled quantum dots

A system of two or more quantum dots interacting with a dissipative plasmonic nanostructure is investigated in detail by using a cavity quantum electrodynamics approach with a model Hamiltonian. We focus on determining and understanding system configurations that generate multiple bipartite quantum entanglements between the occupation states of the quantum dots. These configurations include allowing for the quantum dots to be asymmetrically coupled to the plasmonic system. Analytical solution of a simplified limit for an arbitrary number of quantum dots and numerical simulations and optimization for the two- and three-dot cases are used to develop guidelines for maximizing the bipartite entanglements. For any number of quantum dots, we show that through simple starting states and parameter guidelines, one quantum dot can be made to share a strong amount of bipartite entanglement with all other quantum dots in the system, while entangling all other pairs to a lesser degree.

Time-dependent linear-response variational Monte Carlo

We present the extension of variational Monte Carlo (VMC) to the calculation of electronic excitation energies and oscillator strengths using time-dependent linear-response theory. By exploiting the analogy existing between the linear method for wave-function optimisation and the generalised eigenvalue equation of linear-response theory, we formulate the equations of linear-response VMC (LR-VMC). This LR-VMC approach involves the first-and second-order derivatives of the wave function with respect to the parameters. We perform first tests of the LR-VMC method within the Tamm-Dancoff approximation using single-determinant Jastrow-Slater wave functions with different Slater basis sets on some singlet and triplet excitations of the beryllium atom. Comparison with reference experimental data and with configuration-interaction-singles (CIS) results shows that LR-VMC generally outperforms CIS for excitation energies and is thus a promising approach for calculating electronic excited-state properties of atoms and molecules.

Why is MPI so slow?: analyzing the fundamental limits in implementing MPI-3.1

This paper provides an in-depth analysis of the software overheads in the MPI performance-critical path and exposes mandatory performance overheads that are unavoidable based on the MPI-3.1 specification. We first present a highly optimized implementation of the MPI-3.1 standard in which the communication stack---all the way from the application to the low-level network communication API---takes only a few tens of instructions. We carefully study these instructions and analyze the root cause of the overheads based on specific requirements from the MPI standard that are unavoidable under the current MPI standard. We recommend potential changes to the MPI standard that can minimize these overheads. Our experimental results on a variety of network architectures and applications demonstrate significant benefits from our proposed changes.

Exposing Latent Hierarchies for Large-Scale Design and Discovery

Whitepaper submitted to the 2017 DOE ASCR Applied Math Meeting addresses the advancing in numerical optimization for large-scale scientific discovery in complex physics.