Patrick K. Notz

Graph-Based Software Design for Managing Complexity and Enabling Concurrency in Multiphysics PDE Software

Multiphysics simulation software is plagued by complexity stemming from nonlinearly coupled systems of Partial Differential Equations (PDEs). Such software typically supports many models, which may require different transport equations, constitutive laws, and equations of state. Strong coupling and a multiplicity of models leads to complex algorithms (i.e., the properly ordered sequence of steps to assemble a discretized set of coupled PDEs) and rigid software. This work presents a design strategy that shifts focus away from high-level algorithmic concerns to low-level data dependencies. Mathematical expressions are represented as software objects that directly expose data dependencies. The entire system of expressions forms a directed acyclic graph and the high-level assembly algorithm is generated automatically through standard graph algorithms. This approach makes problems with complex dependencies entirely tractable, and removes virtually all logic from the algorithm itself. Changes are highly localized, allowing developers to implement models without detailed understanding of any algorithms (i.e., the overall assembly process). Furthermore, this approach complements existing MPI-based frameworks and can be implemented within them easily. Finally, this approach enables algorithmic parallelization via threads. By exposing dependencies in the algorithm explicitly, thread-based parallelism is implemented through algorithm decomposition, providing a basis for exploiting parallelism independent from domain decomposition approaches.

Aria 1.5 : user manual.

Aria is a Galerkin finite element based program for solving coupled-physics problems described by systems of PDEs and is capable of solving nonlinear, implicit, transient and direct-to-steady state problems in two and three dimensions on parallel architectures. The suite of physics currently supported by Aria includes the incompressible Navier-Stokes equations, energy transport equation, species transport equations, nonlinear elastic solid mechanics, and electrostatics as well as generalized scalar, vector and tensor transport equations. Additionally, Aria includes support for arbitrary Lagrangian-Eulerian (ALE) and level set based free and moving boundary tracking. Coupled physics problems are solved in several ways including fully-coupled Newtons method with analytic or numerical sensitivities, fully-coupled Newton-Krylov methods, fully-coupled Picards method, and a loosely-coupled nonlinear iteration about subsets of the system that are solved using combinations of the aforementioned methods. Error estimation, uniform and dynamic h-adaptivity and dynamic load balancing are some of Arias more advanced capabilities. Aria is based on the Sierra Framework.

Computational thermal, chemical, fluid, and solid mechanics for geosystems management.

This document summarizes research performed under the SNL LDRD entitled - Computational Mechanics for Geosystems Management to Support the Energy and Natural Resources Mission. The main accomplishment was development of a foundational SNL capability for computational thermal, chemical, fluid, and solid mechanics analysis of geosystems. The code was developed within the SNL Sierra software system. This report summarizes the capabilities of the simulation code and the supporting research and development conducted under this LDRD. The main goal of this project was the development of a foundational capability for coupled thermal, hydrological, mechanical, chemical (THMC) simulation of heterogeneous geosystems utilizing massively parallel processing. To solve these complex issues, this project integrated research in numerical mathematics and algorithms for chemically reactive multiphase systems with computer science research in adaptive coupled solution control and framework architecture. This report summarizes and demonstrates the capabilities that were developed together with the supporting research underlying the models. Key accomplishments are: (1) General capability for modeling nonisothermal, multiphase, multicomponent flow in heterogeneous porous geologic materials; (2) General capability to model multiphase reactive transport of species in heterogeneous porous media; (3) Constitutive models for describing real, general geomaterials under multiphase conditions utilizing laboratory data; (4) General capability to couple nonisothermal reactive flow with geomechanics (THMC); (5) Phase behavior thermodynamics for the CO2-H2O-NaCl system. General implementation enables modeling of other fluid mixtures. Adaptive look-up tables enable thermodynamic capability to other simulators; (6) Capability for statistical modeling of heterogeneity in geologic materials; and (7) Simulator utilizes unstructured grids on parallel processing computers.

Solution-Verified Reliability Analysis and Design of Compliant Micro-Electro-Mechanical Systems

An important component of verification and validation of computational models is solution verification, which focuses on the convergence of the desired solution quantities as one refines the spatial and temporal discretizations and iterative controls. Uncertainty analyses often treat solution verification as a separate issue, hopefully through the use of a priori grid convergence studies and selection of models with acceptable discretization errors. In this paper, a tighter connection between solution verification and uncertainty quantification is investigated. In particular, error estimation techniques, using global norm and quantity of interest error estimators, are applied to the nonlinear structural analysis of microelectromechanical systems (MEMS). Two primary approaches for uncertainty quantification are then developed: an error-corrected approach, in which simulation results are directly corrected for discretization errors, and an error-controlled approach, in which estimators are used to drive adaptive h-refinement of mesh discretizations. The former requires quantity of interest error estimates that are quantitatively accurate, whereas the latter can employ any estimator that is qualitatively accurate. Combinations of these error-corrected and error-controlled approaches are also explored. Each of these techniques treats solution verification and uncertainty analysis as a coupled problem, recognizing that the simulation errors may be influenced by, for example, conditions present in the tails of input probability distributions. The most effective and affordable of these approaches are carried forward in probabilistic design studies for robust and reliable operation of a bistable MEMS device. Computational results show that on-line and parameter-adaptive solution verification can lead to uncertainty quantification and design under uncertainty studies that are more accurate, efficient, reliable, and convenient.

Solution-verified reliability analysis and design of bistable MEMS using error estimation and adaptivity.

This report documents the results for an FY06 ASC Algorithms Level 2 milestone combining error estimation and adaptivity, uncertainty quantification, and probabilistic design capabilities applied to the analysis and design of bistable MEMS. Through the use of error estimation and adaptive mesh refinement, solution verification can be performed in an automated and parameter-adaptive manner. The resulting uncertainty analysis and probabilistic design studies are shown to be more accurate, efficient, reliable, and convenient.

Large deformation solid-fluid interaction via a level set approach.

Solidification and blood flow seemingly have little in common, but each involves a fluid in contact with a deformable solid. In these systems, the solid-fluid interface moves as the solid advects and deforms, often traversing the entire domain of interest. Currently, these problems cannot be simulated without innumerable expensive remeshing steps, mesh manipulations or decoupling the solid and fluid motion. Despite the wealth of progress recently made in mechanics modeling, this glaring inadequacy persists. We propose a new technique that tracks the interface implicitly and circumvents the need for remeshing and remapping the solution onto the new mesh. The solid-fluid boundary is tracked with a level set algorithm that changes the equation type dynamically depending on the phases present. This novel approach to coupled mechanics problems promises to give accurate stresses, displacements and velocities in both phases, simultaneously.

SIERRA Multimechanics Module: Aria User Manual – Version 4.40

Aria is a Galerkin finite element based program for solving coupled-physics problems described by systems of PDEs and is capable of solving nonlinear, implicit, transient and direct-to-steady state problems in two and three dimensions on parallel architectures. The suite of physics currently supported by Aria includes thermal energy transport, species transport, and electrostatics as well as generalized scalar, vector and tensor transport equations. Additionally, Aria includes support for manufacturing process flows via the incompressible Navier-Stokes equations specialized to a low Reynolds number (Re < 1) regime. Enhanced modeling support of manufacturing processing is made possible through use of either arbitrary LagrangianEulerian (ALE) and level set based free and moving boundary tracking in conjunction with quasi-static nonlinear elastic solid mechanics for mesh control. Coupled physics problems are solved in several ways including fully-coupled Newton’s method with analytic or numerical sensitivities, fully-coupled NewtonKrylov methods and a loosely-coupled nonlinear iteration about subsets of the system that are solved using combinations of the aforementioned methods. Error estimation, uniform and dynamic h-adaptivity and dynamic load balancing are some of Aria’s more advanced capabilities. Aria is based upon the Sierra Framework.