Manoj Bhardwaj

Application of the FETI method to ASCI problems—scalability results on 1000 processors and discussion of highly heterogeneous problems

We report on the application of the one-level FETI method to the solution of a class of structural problems associated with the Department of Energys Accelerated Strategic Computing Initiative (ASCI). We focus on numerical and parallel scalability issues,and discuss the treatment by FETI of severe structural heterogeneities. We also report on preliminary performance results obtained on the ASCI Option Red supercomputer configured with as many as one thousand processors, for problems with as many as 5 million degrees of freedom.

Salinas: A Scalable Software for High-Performance Structural and Solid Mechanics Simulations

We present Salinas, a scalable implicit software application for the finite element static and dynamic analysis of complex structural real-world systems. This relatively complete engineering software with more than 100,000 lines of C++ code and a long list of users sustains 292.5 Gflop/s on 2,940 ASCI Red processors, and 1.16 Tflop/s on 3,375 ASCI White processors.

Salinas - User's Notes

Salinas provides a massively parallel implementation of structural dynamics finite element analysis, required for high fidelity, validated models used in modal, vibration, static and shock analysis of weapons systems. This document provides a users guide to the input for Salinas. Details of input specifications for the different solution types, output options, element types and parameters are included. The appendices contain detailed examples, and instructions for running the software on parallel platforms.

Experiences with FETI-DP in a Production Level Finite Element Application

We discuss application of the FETI-DP linear solver within the Salinas finite element application. An overview of Salinas and of the FETI-DP solver is presented. We discuss scalability of the software on ASCI-red, Cplant and ASCI-white. Options for solution of the coarse grid problem that results from the FETI problem are evaluated. The finite element software and solver are seen to be numerically and cpu scalable on each of these platforms. In addition, the software is very robust and can be used on a large variety of finite element models.

Salinas-Theory Manual

Salinas provides a massively parallel implementation of structural dynamics finite element analysis, required for high fidelity, validated models used in modal, vibration, static and shock analysis of structural systems. This manual describes the theory behind many of the constructs in Salinas. For a more detailed description of how to use Salinas, we refer the reader to Salinas, Users Notes. Many of the constructs in Salinas are pulled directly from published material. Where possible, these materials are referenced herein. However, certain functions in Salinas are specific to our implementation. We try to be far more complete in those areas. The theory manual was developed from several sources including general notes, a programmer notes manual, the users notes and of course the material in the open literature.

One-Way Coupled Fluid Structure Simulations of Stores in Weapons Bays

A one-way coupled fluid-structure interaction framework for the simulations of stores in weapons bays is described. The coupling as used here can be described as loose and one-way. The term loose coupling refers to the fact that the fluid and structural equation systems are only coupled through the boundary conditions and solved as separate systems. This is done primarily for efficiency and software maintainability reasons. The term one-way refers to the fact that the fluid loads are transferred to the structural solver, but the structural deflections are not coupled back to the fluid solver. The deflections of the structures are assumed to be small (even though the accelerations may be large), so that this assumption is valid. The framework is used to compute the flow field over a model rectangular weapons bay with a model store in it. examined the effects of these loads on the structural response of the stores carried in the weapons bays. Studies of the effects of the loads on stores have largely been restricted to separation trajectories. In these studies, experiments and simulations have been carried out to examine the effects of the unsteady flow field on the store trajectory once ejected. However, if the loads are large enough and contain energy in the right frequency ranges, it is possible that the store and its components can be dramatically affected. This effect can be an important consideration that must be included in the store design and qualification process. In this paper, we present a method to couple a fluid dynamics solver with a structural dynamics solver in order to be able to study the effects of the unsteady loads on the store structure and its components. The fluid dynamics solver used in this work is SIGMA CFD - an in-house block structured implicit code that solves the compressible flow governing equations. The structural solver used is Salinas - a massively parallel implementation of structural dynamics finite element analysis that uses a variant of the Newmark-Beta time integrator called the generalized alpha method for linear and nonlinear transient dynamics. As will be described in detail below, the coupling between the two codes is done through boundary conditions only - the two equation systems are still solved separately. In addition to this, the deflections computed by the structural dynamics solver are not passed back to the CFD code - the CFD code assumes rigid structures in its computations. These assumptions and their implications are discussed in greater detail below. This paper is organized as follows. Sections II and III describe the CFD and structural dynamics codes, respectively. The coupling approach is detailed in Section IV. Section V discusses the model problem the framework is applied to. Accuracy of the load transfers are evaluated in Section VI, mesh refinement results are presented in Section VII and finally, a model application is presented in Section VIII.

Mechanism free domain decomposition

The simulation of three-dimensional (3D) structural dynamics on massively parallel platforms places stringent requirements on the existing software infrastructure. A constrained and nonlinear graph partitioning problem that arises in scalable iterative substructuring methods, such as finite element tearing and interconnecting (FETI) methods, is identified. New sufficient criteria on a partition are presented that ensure the applicability of FETI methods, and improve the associated preconditioner. One-dimensional finite elements in 3D structures are treated by an encapsulation method. The techniques are demonstrated on complex finite element model problems.

Application of the FETI Method to ASCI Problems: Scalability Results on One Thousand Processors and Discussion of Highly Heterogeneous Problems

We report on the application of the one-level FETI method to the solution of a class of substructural problems associated with the Department of Energys Accelerated Strategic Computing Initiative (ASCI). We focus on numerical and parallel scalability issues, and on preliminary performance results obtained on the ASCI Option Red supercomputer configured with as many as one thousand processors, for problems with as many as 5 million degrees of freedom.

A COMPARISON OF TRANSIENT INFINITE ELEMENTS AND TRANSIENT KIRCHHOFF INTEGRAL METHODS FOR FAR FIELD ACOUSTIC ANALYSIS

Finite element analysis of transient acoustic phenomena on unbounded exterior domains is very common in engineering analysis. In these problems there is a common need to compute the acoustic pressure at points outside of the acoustic mesh, since meshing to points of interest is impractical in many scenarios. In aeroacoustic calculations, for example, the acoustic pressure may be required at tens or hundreds of meters from the structure. In these cases, a method is needed for post-processing the acoustic results to compute the response at far-field points. In this paper, we compare two methods for computing far-field acoustic pressures, one derived directly from the infinite element solution, and the other from the transient version of the Kirchhoff integral. We show that the infinite element approach alleviates the large storage requirements that are typical of Kirchhoff integral and related procedures, and also does not suffer from loss of accuracy that is an inherent part of computing numerical derivatives in the Kirchhoff integral. In order to further speed up and streamline the process of computing the acoustic response at points outside of the mesh, we also address the nonlinear iterative procedure needed for locating parametric coordinates within the host infinite element of far-field points, the parallelization of the overall process, linear solver requirements, and system stability considerations.

Multilevel parallel optimization using massively parallel structural dynamics

A large scale optimization of an electronics package has been completed using a massively parallel structural dynamics code. The optimization goals were to maximize safety margins for stress and acceleration resulting from transient impulse loads, while remaining within strict mass limits. The optimization process utilized nongradient, gradient, and approximate optimization methods in succession to modify shell thickness and foam density values within the electronics package. This combination of optimization methods was successful in improving the performance from an infeasible design which violated stress allowables by a factor of two to a completely feasible design with positive design margins, while remaining within the mass limits. In addition, a tradeoff curve of mass versus safety margin was developed to facilitate the design decision process. These studies employed the ASCI Red supercomputer and utilized multiple levels of parallelism on up to 2560 processors. In one portion of this optimization study, a series of calculations were performed on ASCI Red in four days, where an equivalent calculation on a single desktop computer would have taken greater than 10 years to complete.