Tim Warburton

Nodal discontinuous Galerkin methods on graphics processors

Discontinuous Galerkin (DG) methods for the numerical solution of partial differential equations have enjoyed considerable success because they are both flexible and robust: They allow arbitrary unstructured geometries and easy control of accuracy without compromising simulation stability. Lately, another property of DG has been growing in importance: The majority of a DG operator is applied in an element-local way, with weak penalty-based element-to-element coupling. The resulting locality in memory access is one of the factors that enables DG to run on off-the-shelf, massively parallel graphics processors (GPUs). In addition, DGs high-order nature lets it require fewer data points per represented wavelength and hence fewer memory accesses, in exchange for higher arithmetic intensity. Both of these factors work significantly in favor of a GPU implementation of DG. Using a single US

Extreme-Scale AMR

400 Nvidia GTX 280 GPU, we accelerate a solver for Maxwells equations on a general 3D unstructured grid by a factor of around 50 relative to a serial computation on a current-generation CPU. In many cases, our algorithms exhibit full use of the devices available memory bandwidth. Example computations achieve and surpass 200gigaflops/s of net application-level floating point work. In this article, we describe and derive the techniques used to reach this level of performance. In addition, we present comprehensive data on the accuracy and runtime behavior of the method.

GPU Accelerated Adams–Bashforth Multirate Discontinuous Galerkin FEM Simulation of High-Frequency Electromagnetic Fields

Many problems are characterized by dynamics occurring on a wide range of length and time scales. One approach to overcoming the tyranny of scales is adaptive mesh refinement/coarsening (AMR), which dynamically adapts the mesh to resolve features of interest. However, the benefits of AMR are difficult to achieve in practice, particularly on the petascale computers that are essential for difficult problems. Due to the complex dynamic data structures and frequent load balancing, scaling dynamic AMR to hundreds of thousands of cores has long been considered a challenge. Another difficulty is extending parallel AMR techniques to high-order-accurate, complex-geometry-respecting methods that are favored for many classes of problems. Here we present new parallel algorithms for parallel dynamic AMR on forest-ofoctrees geometries with arbitrary-order continuous and discontinuous finite/spectral element discretizations. The implementations of these algorithms exhibit excellent weak and strong scaling to over 224,000 Cray XT5 cores for multiscale geophysics problems.

Scalability of Higher-Order Discontinuous Galerkin FEM Computations for Solving Electromagnetic Wave Propagation Problems on GPU Clusters

A multirate Adams-Bashforth (AB) scheme for simulation of electromagnetic wave propagation using the discontinuous Galerkin finite element method (DG-FEM) is presented. The algorithm is adapted such that single-instruction multiple-thread (SIMT) characteristic for the implementation on a graphics processing unit (GPU) is preserved. A domain decomposition strategy respecting the multirate classification for computation on multiple GPUs is presented. Accuracy and performance is analyzed with help of suitable benchmarks.

Viscous Shock Capturing in a Time-Explicit Discontinuous Galerkin Method

A highly parallel implementation of Maxwells equations in the time domain using a cluster of Graphics Processing Units (GPUs) is presented. The higher-order Discontinuous Galerkin Finite Element Method (DG-FEM) is used for spatial discretization since its characteristics are matching the parallelization design aspects of the NVIDIA Compute Unified Device Architecture (CUDA) programming model. Asynchronous data transfer is introduced to minimize parallelization overhead and improve parallel efficiency. The implementation is benchmarked with help of a realistic 3-D geometry of an electromagnetic compatibility problem.

A Low-Storage Curvilinear Discontinuous Galerkin Method for Wave Problems

We present a novel, cell-local shock detector for use with discontinuous Galerkin (DG) methods. The output of this detector is a reliably scaled, element-wise smoothness estimate which is suited as a control input to a shock capture mechanism. Using an artificial viscosity in the latter role, we obtain a DG scheme for the numerical solution of nonlinear systems of conservation laws. Building on work by Persson and Peraire, we thoroughly justify the detectors design and analyze its performance on a number of benchmark problems. We further explain the scaling and smoothing steps necessary to turn the output of the detector into a local, artificial viscosity. We close by providing an extensive array of numerical tests of the detector in use.

Convergence Analysis of an Adaptive Interior Penalty Discontinuous Galerkin Method

The low-storage curvilinear discontinuous Galerkin (LSC-DG) method reduces the storage requirements for solving symmetric and linear linear symmetric conservation laws in curvilinear domains when compared to the standard discontinuous Galerkin method. We perform a semidiscrete, a priori convergence analysis of LSC-DG and determine sufficient conditions on sequences of meshes to guarantee the same rate of convergence as the original upwind DG method on curvilinear domains. Computational results confirm the optimal order convergence on sample mesh sequences that satisfy the sufficiency conditions. Additional results show that the sufficient conditions for optimal order convergence of LSC-DG may also be necessary conditions.

A nodal discontinuous Galerkin method for reverse-time migration on GPU clusters

We study the convergence of an adaptive interior penalty discontinuous Galerkin (IPDG) method for a two-dimensional model second order elliptic boundary value problem. Based on a residual-type a posteriori error estimator, we prove that after each refinement step of the adaptive scheme we achieve a guaranteed reduction of the global discretization error in the mesh-dependent energy norm associated with the IPDG method. In contrast to recent work on adaptive IPDG methods [O. Karakashian and F. Pascal, Convergence of Adaptive Discontinuous Galerkin Approximations of Second-order Elliptic Problems, preprint, University of Tennessee, Knoxville, TN, 2007], the convergence analysis does not require multiple interior nodes for refined elements of the triangulation. In fact, it will be shown that bisection of the elements is sufficient. The main ingredients of the proof of the error reduction property are the reliability and a perturbed discrete local efficiency of the estimator, a bulk criterion that takes care of a proper selection of edges and elements for refinement, and a perturbed Galerkin orthogonality property with respect to the energy inner product. The results of numerical experiments are given to illustrate the performance of the adaptive method.

GPU accelerated Discontinuous Galerkin FEM for electromagnetic radio frequency problems

Improving both accuracy and computational performance of numerical tools is a major challenge for seismic imaging and generally requires specialized implementations to make full use of modern parallel architectures. We present a computational strategy for reverse-time migration (RTM) with accelerator-aided clusters. A new imaging condition computed from the pressure and velocity fields is introduced. The model solver is based on a high-order discontinuous Galerkin time-domain (DGTD) method for the pressure–velocity system with unstructured meshes and multirate local time stepping. We adopted the MPI+X approach for distributed programming where X is a threaded programming model. In this work we chose OCCA, a unified framework that makes use of major multithreading languages (e.g. CUDA and OpenCL) and offers the flexibility to run on several hardware architectures. DGTD schemes are suitable for efficient computations with accelerators thanks to localized element-to-element coupling and the dense algebraic operations required for each element. Moreover, compared to high-order finite-difference schemes, the thin halo inherent to DGTD method reduces the amount of data to be exchanged between MPI processes and storage requirements for RTM procedures. The amount of data to be recorded during simulation is reduced by storing only boundary values in memory rather than on disk and recreating the forward wavefields. Computational results are presented that indicate that these methods are strong scalable up to at least 32 GPUs for a three-dimensional RTM case.

GPU performance analysis of a nodal discontinuous Galerkin method for acoustic and elastic models

DG-FEM computations of Maxwells equations in time domain on GPU clusters were presented. DG-FEM discretization incorporates domain decomposition on the element level, which is used for a highly parallel implementation with speedups of more than one order of magnitude. Further work will include double precision accuracy on GPU as well as investigation of scalability and improved exploitation of the GPU performance.