Heidi K. Thornquist

An overview of the Trilinos project

The Trilinos Project is an effort to facilitate the design, development, integration, and ongoing support of mathematical software libraries within an object-oriented framework for the solution of large-scale, complex multiphysics engineering and scientific problems. Trilinos addresses two fundamental issues of developing software for these problems: (i) providing a streamlined process and set of tools for development of new algorithmic implementations and (ii) promoting interoperability of independently developed software.Trilinos uses a two-level software structure designed around collections of packages. A Trilinos package is an integral unit usually developed by a small team of experts in a particular algorithms area such as algebraic preconditioners, nonlinear solvers, etc. Packages exist underneath the Trilinos top level, which provides a common look-and-feel, including configuration, documentation, licensing, and bug-tracking.Here we present the overall Trilinos design, describing our use of abstract interfaces and default concrete implementations. We discuss the services that Trilinos provides to a prospective package and how these services are used by various packages. We also illustrate how packages can be combined to rapidly develop new algorithms. Finally, we discuss how Trilinos facilitates high-quality software engineering practices that are increasingly required from simulation software.

Stable Galerkin reduced order models for linearized compressible flow

The Galerkin projection procedure for construction of reduced order models of compressible flow is examined as an alternative discretization of the governing differential equations. The numerical stability of Galerkin models is shown to depend on the choice of inner product for the projection. For the linearized Euler equations, a symmetry transformation leads to a stable formulation for the inner product. Boundary conditions for compressible flow that preserve stability of the reduced order model are constructed. Preservation of stability for the discrete implementation of the Galerkin projection is made possible using a piecewise-smooth finite element basis. Stability of the reduced order model using this approach is demonstrated on several model problems, where a suitable approximation basis is generated using proper orthogonal decomposition of a transient computational fluid dynamics simulation.

Anasazi software for the numerical solution of large-scale eigenvalue problems

Anasazi is a package within the Trilinos software project that provides a framework for the iterative, numerical solution of large-scale eigenvalue problems. Anasazi is written in ANSI C++ and exploits modern software paradigms to enable the research and development of eigensolver algorithms. Furthermore, Anasazi provides implementations for some of the most recent eigensolver methods. The purpose of our article is to describe the design and development of the Anasazi framework. A performance comparison of Anasazi and the popular FORTRAN 77 code ARPACK is given.

Amesos2 and Belos: Direct and iterative solvers for large sparse linear systems

Solvers for large sparse linear systems come in two categories: direct and iterative. Amesos2, a package in the Trilinos software project, provides direct methods, and Belos, another Trilinos package, provides iterative methods. Amesos2 offers a common interface to many different sparse matrix factorization codes, and can handle any implementation of sparse matrices and vectors, via an easy-to-extend C++ traits interface. It can also factor matrices whose entries have arbitrary “Scalar” type, enabling extended-precision and mixed-precision algorithms. Belos includes many different iterative methods for solving large sparse linear systems and least-squares problems. Unlike competing iterative solver libraries, Belos completely decouples the algorithms from the implementations of the underlying linear algebra objects. This lets Belos exploit the latest hardware without changes to the code. Belos favors algorithms that solve higher-level problems, such as multiple simultaneous linear systems and sequences of related linear systems, faster than standard algorithms. The package also supports extended-precision and mixed-precision algorithms. Together, Amesos2 and Belos form a complete suite of sparse linear solvers.

A parallel preconditioning strategy for efficient transistor-level circuit simulation

We describe a parallel computing approach for large-scale SPICE-accurate circuit simulation, which is based on a new strategy for the parallel preconditioned iterative solution of circuit matrices. This strategy consists of several steps, including singleton removal, block triangular form (BTF) reordering, hypergraph partitioning, and a block Jacobi pre-conditioner. Our parallel implementation makes use of a mixed load balance, employing a different parallel partition for the matrix load and solve. Based on message-passing, our circuit simulation code was originally designed for large parallel computers, but for the purposes of this paper we demonstrate that it also gives good parallel speedup in modern multi-core environments. We show that our new parallel solver outperforms a serial direct solver, a parallel direct solver and an alternative iterative solver on a set of circuit test problems.

Parallel Transistor-Level Circuit Simulation

With the advent of multi-core technology, inexpensive large-scale parallel platforms are now widely available. While this presents new opportunities for the EDA community, traditional transistor-level, SPICE-style circuit simulation has unique parallel simulation challenges. Here the Xyce Parallel Circuit Simulator is described, which has been designed from the “from-the-ground-up” to be distributed memory-parallel. Xyce has demonstrated scalable circuit simulation on hundreds of processors, but doing so required a comprehensive parallel strategy. This included the development of new solver technologies, including novel preconditioned iterative solvers, as well as attention to other aspects of the simulation such as parallel file I/O, and efficient load balancing of device evaluations and linear systems. Xyce relies primarily upon a message-passing (MPI-based) implementation, but optimal scalability on multi-core platforms can require a combination of message-passing and threading. To accommodate future parallel platforms, software abstractions allowing adaptation to other parallel paradigms are part of the Xyce design.

Galerkin reduced order models for compressible flow with structural interaction

The Galerkin projection procedure for construction of reduced order models of compressible flow is examined as an alternative discretization of the governing differential equations. The numerical stability of Galerkin models is shown to depend on the choice of inner product for the projection. For the linearized Euler equations, a symmetry transform leads to a stable formulation for the inner product. Boundary conditions for compressible flow that preserve stability of the reduced order model are constructed. Coupling with a linearized structural dynamics model is made possible through the solid wall boundary condition. Preservation of stability for the discrete implementation of the Galerkin projection is made possible using piecewise-smooth finite element bases. Stability of the coupled fluid/structure system is examined for the case of uniform flow past a thin plate. Stability of the reduced order model for the fluid is demonstrated on several model problems, where a suitable approximation basis is generated using proper orthogonal decomposition of a transient computational fluid dynamics simulation.

Developing a dynamic model of cascading failure for high performance computing using trilinos

This paper describes work-in-progress toward the development of a dynamic model of cascading failure in power systems that is suitable for High Performance Computing simulation environments. Doing so involves simulating a power grid as a set of differential, algebraic and discrete equations. We describe the general form of the algorithm in use for this simulation and provide details about the implementation using the Trilinos software libraries. Several computational tests illustrate how the proposed approach can be leveraged to optimize the computational efficiency of cascading failure simulation.

Enabling next-generation parallel circuit simulation with trilinos

The Xyce Parallel Circuit Simulator, which has demonstrated scalable circuit simulation on hundreds of processors, heavily leverages the high-performance scientific libraries provided by Trilinos. With the move towards multi-core CPUs and GPU technology, retaining this scalability on future parallel architectures will be a challenge. This paper will discuss how Trilinos is an enabling technology that will optimize the trade-off between effort and impact for application codes, like Xyce, in their transition to becoming next-generation simulation tools.

Basker: A Threaded Sparse LU Factorization Utilizing Hierarchical Parallelism and Data Layouts

Scalable sparse LU factorization is critical for efficient numerical simulation of circuits and electrical power grids. In this work, we present a new scalable sparse direct solver called Basker. Basker introduces a new algorithm to parallelize the Gilbert-Peierls algorithm for sparse LU factorization. As architectures evolve, there exists a need for algorithms that are hierarchical in nature to match the hierarchy in thread teams, individual threads, and vector level parallelism. Basker is designed to map well to this hierarchy in architectures. There is also a need for data layouts to match multiple levels of hierarchy in memory. Basker uses a two-dimensional hierarchical structure of sparse matrices that maps to the hierarchy in the memory architectures and to the hierarchy in parallelism. We present performance evaluations of Basker on the Intel SandyBridge and Xeon Phi platforms using circuit and power grid matrices taken from the University of Florida sparse matrix collection and from Xyce circuit simulations. Basker achieves a geometric mean speedup of 5.91× on CPU (16 cores) and 7.4× on Xeon Phi (32 cores) relative to KLU. Basker outperforms Intel MKL Pardiso (PMKL) by as much as 30× on CPU (16 cores) and 7.5× on Xeon Phi (32 cores) for low fill-in circuit matrices. Furthermore, Basker provides 5.4× speedup on a challenging matrix sequence taken from an actual Xyce simulation.