Scott A. Hutchinson

Efficient parallel computation of unstructured finite element reacting flow solutions

Abstract A parallel unstructured finite element (FE) reacting flow solver designed for message passing MIMD computers is described. This implementation employs automated partitioning algorithms for load balancing unstructured grids, a distributed sparse matrix representation of the global FE equations, and parallel Krylov subspace iterative solvers. In this paper, a number of issues related to the efficient implementation of parallel unstructured mesh applications are presented. These issues include the differences between structured and unstructured mesh parallel applications, major communication kernels for unstructured Krylov iterative solvers, automatic mesh partitioning algorithms, and the influence of mesh partitioning metrics and single-node CPU performance on parallel performance. Results are presented for example FE heat transfer, fluid flow and full reacting flow applications on a 1024 processor nCUBE 2 hypercube and a 1904 processor Intel Paragon. Results indicate that very high computational rates and high scaled efficiencies can be achieved for large problems despite the use of sparse matrix data structures and the required unstructured data communication.

Parallel sparse matrix vector multiply software for matrices with data locality

In this paper we describe general software utilities for performing unstructured sparse matrix–vector multiplications on distributed-memory message-passing computers. The matrix–vector multiply comprises an important kernel in the solution of large sparse linear systems by iterative methods. Our focus is to present the data structures and communication parameters required by these utilities for general sparse unstructured matrices with data locality. These types of matrices are commonly produced by finite difference and finite element approximations to systems of partial differential equations. In this discussion we also present representative examples and timings which demonstrate the utility and performance of the software.

Analysis of gallium arsenide deposition in a horizontal chemical vapor deposition reactor using massively parallel computations

Abstract A numerical analysis of the deposition of gallium arsenide from trimethylgallium (TMG) and arsine in a horizontal CVD reactor with tilted susceptor and a 3″ diameter rotating substrate is performed. The three-dimensional model includes complete coupling between fluid mechanics, heat transfer, and species transport, and is solved using an unstructured finite element discretization on a massively parallel computer. A reaction mechanism consisting of three surface and two bulk species, four surface reactions, and four gas phase species was used to model the deposition. The effects of three operating parameters (the disk rotation rate, inlet TMG fraction, and inlet velocity) and two design parameters (the tilt angle of the reactor base and the reactor width) on the growth rate and uniformity are presented. The nonlinear dependence of the growth rate uniformity on the key operating parameters is discussed in detail. Efficient and robust algorithms for massively parallel reacting flow simulations, as incorporated into our analysis code MPSalsa, make detailed analysis of this complicated system feasible.

Redesigning the WARPED simulation kernel for analysis and application development

WARPED is a publicly available time warp simulation kernel. The kernel defines a standard interface to the application developer and is designed to provide a highly configurable environment for the integration of time warp optimizations. It is written in C++, uses the MPI message passing standard, and executes on a variety of parallel and distributed processing platforms. Version 2.0 of WARPED described here is distributed with several applications and the configuration can be set so that a sequential kernel implementation can be instantiated The kernel supports LP clustering, various GVT algorithms, and numerous optimizations to adaptively adjust simulation parameters at runtime.

Robust, stable time-domain methods for solving MPDEs of fast/slow systems

We explore the stability properties of time-domain numerical methods for multitime partial differential equations (MPDEs) in detail. We demonstrate that simple techniques for numerical discretization can lead easily to instability. By investigating the underlying eigenstructure of several discretization techniques along different artificial time scales, we show that not all combinations of techniques are stable. We identify choices of discretization method and step size, along fast and slow time scales, that lead to robust, stable time-domain integration methods for the MPDE. One of our results is that applying overstable methods along one time-scale can compensate for unstable discretization along others. Our novel integration schemes bring robustness to time-domain MPDE solution methods, as we demonstrate with examples.

Computational Algorithms for Device-Circuit Coupling

Circuit simulation tools (e.g., SPICE) have become invaluable in the development and design of electronic circuits. Similarly, device-scale simulation tools (e.g., DaVinci) are commonly used in the design of individual semiconductor components. Some problems, such as single-event upset (SEU), require the fidelity of a mesh-based device simulator but are only meaningful when dynamically coupled with an external circuit. For such problems a mixed-level simulator is desirable, but the two types of simulation generally have different (sometimes conflicting) numerical requirements. To address these considerations, we have investigated variations of the two-level Newton algorithm, which preserves tight coupling between the circuit and the partial differential equations (PDE) device, while optimizing the numerics for both.

Integrating multiple parallel simulation engines for mixed-technology parallel simulation

The emergence of mixed-signal (analog and digital) integrated circuits motivates the need for CAD tools supporting mixed-signal design and analysis. Furthermore, the presence of a large body of existing models in existing modeling language and the need for modeling mixed-signal (analog and digital) circuits motivates the need for a single unified simulation framework into which different parallel simulation subsystems can be easily connected. In this paper we have review the design of a light-weight simulation backplane for integrating simulators from different domains. Of particular focus for this paper is the integration of a parallel SPICE (analog circuit) simulator called Xyce/sup TM/ with a parallel VHDL (digital circuit) simulator called SAVANT.

Electrical defibrillation optimization: an automated, iterative parallel finite-element approach

To date, optimization of electrode systems for electrical defibrillation has been limited to hand-selected electrode configurations. Here, the authors present an automated approach which combines detailed, three-dimensional (3-D) finite-element torso models with optimization techniques to provide a flexible analysis and design tool for electrical defibrillation optimization. Specifically, a parallel direct search (PDS) optimization technique is used with a representative objective function to find an electrode configuration which corresponds to the satisfaction of a postulated defibrillation criterion with a minimum amount of power and a low possibility of myocardium damage. For adequate representation of the thoracic inhomogeneities, 3-D finite-element torso models are used in the objective function computations. The CPU-intensive finite-element calculations required for the objective function evaluation have been implemented on a message-passing parallel computer in order to complete the optimization calculations in a timely manner. To illustrate the optimization procedure, it has been applied to a representative electrode configuration for transmyocardial defibrillation, namely the subcutaneous patch-right ventricular catheter (SP-RVC) system. Sensitivity of the optimal solutions to various tissue conductivities has been studied. Results for the optimization of defibrillation systems are presented which demonstrate the feasibility of the approach.

Numerical analysis of electrical defibrillation: The parallel approach

Numerical modeling offers a viable tool for studying electrical defibrillation, allowing the behavior of field quantities to be observed easily as the different system parameters are varied. One numerical technique, namely the finite-element method, has been found particularly effective for modeling complex thoracic anatomies. However, an accurate finite-element model of the thorax often requires a large number of elements and nodes, leading to a large set of equations that cannot be solved effectively with the computational power of conventional computers. This is especially true if many finite-element solutions need to be achieved within a reasonable time period (eg, electrode configuration optimization). In this study, the use of massively parallel computers to provide the memory and reduction in solution time for solving these large finite-element problems is discussed. Both the uniform and unstructured grid approaches are considered. Algorithms that allow efficient mapping of uniform and unstructured grids to data-parallel and message-passing parallel computers are discussed. An automatic iterative procedure for electrode configuration optimization is presented. The procedure is based on the minimization of an objective function using the parallel direct search technique. Computational performance results are presented together with simulation results.

High Performance MP Unstructured Finite Element Simulation of Chemically Reacting Flows

We describe the performance of MPSalsa, a MP code that simulates complex systems with strongly coupled fluid flow, thermal energy transfer, mass transfer and non-equilibrium chemical reactions. MPSalsa uses 3D unstructured finite element methods, fully implicit time integration, and general gas-phase and surface-species chemical kinetics to solve the coupled nonlinear PDEs on complex domains. It is designed around general kernels for domain partitioning, unstructured message passing, distributed sparse-block matrix representation of the fully summed global finite element equations, and preconditioned Krylov iterative solvers. Using these techniques, we obtained sustained rates of 210+ Gflop/s for a 3-D chemically reacting flow problem.