Christopher Siefert

An Algebraic Multigrid Approach Based on a Compatible Gauge Reformulation of Maxwell's Equations

With the rise in popularity of compatible finite element, finite difference, and finite volume discretizations for the time domain eddy current equations, there has been a corresponding need for fast solvers of the resulting linear algebraic systems. However, the traits that make compatible discretizations a preferred choice for the Maxwells equations also render these linear systems essentially intractable by truly black-box techniques. We propose an algebraic reformulation of the discrete eddy current equations along with a new algebraic multigrid (AMG) technique for this reformulated problem. The reformulation process takes advantage of a discrete Hodge decomposition on cochains to replace the discrete eddy current equations by an equivalent

Spatially adaptive stochastic methods for fluid-structure interactions subject to thermal fluctuations in domains with complex geometries

2\times2

TOWARDS EXTREME-SCALE SIMULATIONS FOR LOW MACH FLUIDS WITH SECOND-GENERATION TRILINOS

block linear system whose diagonal blocks are discrete Hodge-Laplace operators acting on 1-cochains and 0-cochains, respectively. While this new AMG technique requires somewhat specialized treatment on the finest mesh, the coarser meshes can be handled using standard methods for Laplace-type problems. Our new AMG method is applicable to a wide range of compatible methods on structured and unstructured grids, including edge finite elements, mimetic finite differences, covolume methods, and Yee-like schemes. We illustrate the new technique, using edge elements in the context of smoothed aggregation AMG, and present computational results for problems in both two and three dimensions.

Design considerations for a flexible multigrid preconditioning library

We develop stochastic mixed finite element methods for spatially adaptive simulations of fluid-structure interactions when subject to thermal fluctuations. To account for thermal fluctuations, we introduce a discrete fluctuation-dissipation balance condition to develop compatible stochastic driving fields for our discretization. We perform analysis that shows our condition is sufficient to ensure results consistent with statistical mechanics. We show the Gibbs-Boltzmann distribution is invariant under the stochastic dynamics of the semi-discretization. To generate efficiently the required stochastic driving fields, we develop a Gibbs sampler based on iterative methods and multigrid to generate fields with O ( N ) computational complexity. Our stochastic methods provide an alternative to uniform discretizations on periodic domains that rely on Fast Fourier Transforms. To demonstrate in practice our stochastic computational methods, we investigate within channel geometries having internal obstacles and no-slip walls how the mobility/diffusivity of particles depends on location. Our methods extend the applicability of fluctuating hydrodynamic approaches by allowing for spatially adaptive resolution of the mechanics and for domains that have complex geometries relevant in many applications.

Reducing Communication Costs for Sparse Matrix Multiplication within Algebraic Multigrid

Trilinos is an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems. While Trilinos was originally designed for scalable solutions of large problems, the fidelity needed by many simulations is significantly greater than what one could have envisioned two decades ago. When problem sizes exceed a billion elements even scalable applications and solver stacks require a complete revision. The second-generation Trilinos employs C++ templates in order to solve arbitrarily large problems. We present a case study of the integration of Trilinos with a low Mach fluids engineering application (SIERRA low Mach module/Nalu). Through the use of improved algorithms and better software engineering practices, we demonstrate good weak scaling for up to a nine billion element large eddy simulation (LES) problem on unstructured meshes with a 27 billion row matrix on 524,288 cores of an IBM Blue Gene/Q platform.

Algebraic multigrid techniques for discontinuous Galerkin methods with varying polynomial order

MueLu is a library within the Trilinos software project [An overview of Trilinos, Technical Report SAND2003-2927, Sandia National Laboratories, 2003] and provides a framework for parallel multigrid preconditioning methods for large sparse linear systems. While providing efficient implementations of modern multigrid methods based on smoothed aggregation and energy minimization concepts, MueLu is designed to be customized and extended. This article gives an overview of design considerations for the MueLu package: user interfaces, internal design, data management, usage of modern software constructs, leveraging Trilinos capabilities, linear algebra operations and advanced application.

Towards Extreme-Scale Simulations with Next-Generation Trilinos: A Low Mach Fluid Application Case Study

We consider the sequence of sparse matrix-matrix multiplications performed during the setup phase of algebraic multigrid. In particular, we show that the most commonly used parallel algorithm is often not the most communication-efficient one for all of the matrix-matrix multiplications involved. By using an alternative algorithm, we show that the communication costs are reduced (in theory and practice), and we demonstrate the performance benefit for both model (structured) and more realistic unstructured problems on large-scale distributed-memory parallel systems. Our theoretical analysis shows that we can reduce communication by a factor of up to 5.4 for a model problem, and we observe in our empirical evaluation communication reductions of factors up to 4.7 for structured problems and 3.7 for unstructured problems. These reductions in communication translate to run-time speedups of factors up to 2.8 and 2.5, respectively.

Analysis and Computation of Compatible Least-Squares Methods for div-curl Equations

We present a parallel algebraic multigrid (AMG) algorithm for the implicit solution of the Darcy problem discretized by the discontinuous Galerkin (DG) method that scales optimally for regular and irregular meshes. The main idea centers on recasting the preconditioning problem so that existing AMG solvers for nodal lower order finite elements can be leveraged. This is accomplished by a transformation operator which maps the solution from a Lagrange basis representation to a Legendre basis representation. While this mapping function must be user supplied, we demonstrate how easily it can be constructed for somepopular finite element representations includingquadrilateral/hexahedral and triangular/tetrahedral DG formulations. Furthermore, we show that the mapping does not depend on the Jacobian transformation between reference and physical space and so it can be constructed with very limited mesh information. Parallel performance studies demonstrate the versatility of this approach.

Highly Scalable Linear Solvers on Thousands of Processors

Trilinos is an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems. While the original version of Trilinos was designed for highly scalable solutions for large problems, the need for increasingly higher fidelity simulations has pushed the problem sizes beyond what could have been envisioned two decades ago. When problem sizes exceed a billion elements even highly scalable applications and solver stacks require a complete revision. The next-generation Trilinos employs C++ templates in order to solve arbitrarily large problems and enable extreme-scale simulations. We present a case study that involves integration of Trilinos with an engineering application (Sierra low Mach module/Nalu), involving the simulation of low Mach fluid flow for problems of size up to nine billion elements. Through the use of improved algorithms and better software engineering practices, we demonstrate good weak scaling for the matrix assembly and solve for the engineering application for up to a nine billion element fluid flow large eddy simulation (LES) problem on unstructured meshes with a 27 billion row matrix on 131,072 cores of a Cray XE6 platform.

High resolution viscous fingering simulation in miscible displacement using a p-adaptive discontinuous Galerkin method with algebraic multigrid preconditioner

We develop and analyze least-squares finite element methods for two complementary div-curl elliptic boundary value problems. The first one prescribes the tangential component of the vector field on the boundary and is solved using curl-conforming elements. The second problem specifies the normal component of the vector field and is handled by div-conforming elements. We prove that both least-squares formulations are norm-equivalent with respect to suitable discrete norms, yield optimal asymptotic error estimates, and give rise to algebraic systems that can be solved by efficient algebraic multigrid methods. Numerical results that illustrate scalability of iterative solvers and optimal rates of convergence are also included.