Edward A. Luke

Loci: a rule-based framework for parallel multi-disciplinary simulation synthesis

We present a rule-based framework for the development of scalable parallel high performance simulations for a broad class of scientific applications (with particular emphasis on continuum mechanics). We take a pragmatic approach to our programming abstractions by implementing structures that are used frequently and have common high performance implementations on distributed memory architectures. The resulting framework borrows heavily from rule-based systems for relational database models, however limiting the scope to those parts that have obvious high performance implementation. Using our approach, we demonstrate predictable performance behavior and efficient utilization of large scale distributed memory architectures on problems of significant complexity involving multiple disciplines.

Exploring Traditional and Emerging Parallel Programming Models Using a Proxy Application

Parallel machines are becoming more complex with increasing core counts and more heterogeneous architectures. However, the commonly used parallel programming models, C/C++ with MPI and/or OpenMP, make it difficult to write source code that is easily tuned for many targets. Newer language approaches attempt to ease this burden by providing optimization features such as automatic load balancing, overlap of computation and communication, message-driven execution, and implicit data layout optimizations. In this paper, we compare several implementations of LULESH, a proxy application for shock hydrodynamics, to determine strengths and weaknesses of different programming models for parallel computation. We focus on four traditional (OpenMP, MPI, MPI+OpenMP, CUDA) and four emerging (Chapel, Charm++, Liszt, Loci) programming models. In evaluating these models, we focus on programmer productivity, performance and ease of applying optimizations.

A fast mesh deformation method using explicit interpolation

A novel mesh deformation algorithm for unstructured polyhedral meshes is developed utilizing a tree-code optimization of a simple direct interpolation method. The algorithm is shown to provide mesh quality that is competitive with radial basis function based methods, with markedly better performance in preserving boundary layer orthogonality in viscous meshes. The parallelization of the algorithm is described, and the algorithm cost is demonstrated to be O(nlogn). The parallel implementation was used to deform meshes of 100 million nodes on nearly 200 processors demonstrating that the method scales to large mesh sizes. Results are provided for a simulation of a high Reynolds number fluid-structure interaction case using this technique.

Coupling Heat Transfer and Fluid Flow Solvers for Multidisciplinary Simulations

The feasibility of multidisciplinary simulations for realistic geometries involving detailed physical models is demonstrated. Specifically, a three-dimensional chemically reacting fluid flow solver is coupled with a solid-phase heat transfer solver that includes cooling channels. Both fluid- and solid-phase models employ the integral, conservative form of the governing equations and are discretized by means of two finite volume numerical schemes. To keep the heat flux consistent, a special algorithm is developed at the interface between the solid and fluid regions. Physical and thermal properties of the solid materials can be temperature dependent, and different materials can be used in different parts of the domains due to a multiblock gridding strategy. The cooling channel model is developed by using conservation laws of mass, momentum, and energy, taking into account the effects of heat transfer and friction. The coupling of the models (solid and fluid, solid and cooling channels) is detailed. A hot-air nozzle test case is examined

Loci: A Deductive Framework for Graph-Based Algorithms

Modern computational fluid dynamics (CFD) software is complex. Often CFD simulations require complex geometries, flexible boundary conditions, multiple integrated computational models (for example, heat conduction, structural deformations, gas dynamics, etc.), as well as grid adaptation. As a result of this complexity, the correct implementation of numerical simulation components is actually less challenging than guaranteeing the correct coordination of complex component interactions. If one is to consider the development of CFD applications that reliably incorporate a broad selection of numerical models, then one must consider technologies that simplify, automate, and validate the numerical model coordination mechanisms. The Loci system presented here addresses these issues by introducing a deductive framework for the coordination of numerical value classes constructed in C++.

A Multiphysics Simulation Capability using the SIMULIA Co-Simulation Engine

A multiphysics simulation capability suitable for fluid-structure interaction is presented that uses the Abaqus nonlinear structural dynamics solver and the Loci/CHEM computational fluid dynamics solver. The coupling is achieved using SIMULIA’s Co-Simulation Engine technology. The Co-Simulation Engine is a software framework that allows the coupling of multiple simulation domains by coupling solvers in a synchronized manner.

Comprehensive Code Verification for an Unstructured Finite Volume CFD Code

A detailed code verification study of an unstructured finite volume CFD code is presented. The Method of Manufactured Solutions is used to generate exact solutions to the Euler and Navier-Stokes equations to verify the order of accuracy of the code. Testing is performed on different grid types including triangular and quadrilateral elements in 2D and tetrahedral, prismatic and hexahedral elements in 3D. The requirements for systematic mesh refinement are discussed, particularly in regards to unstructured meshes. Different code options verified include the baseline steady-state governing equations, transport models, turbulence models, boundary conditions and unsteady flows.

Verification of the Loci-CHEM CFD Code using the Method of Manufactured Solutions

Loci CHEM is a Computational Fluid Dynamics (CFD) code developed using the Loci Framework that can simulate three-dimensional flows of chemically reacting mixtures of thermally perfect gases. It is a library of Loci rules that consists of reusable rules that can be dynamically reconfigured to solve a variety of problems. The present study involves the verification of Euler and Navier-Stokes equations and their associated sub-models in LociCHEM on different mesh types using the Method of Manufactured Solutions (MMS). MMS is the most general approach for ensuring that there are no mistakes (i.e. bugs) in the computer code and that the algorithms are consistent and convergent. The different meshes include both 2D and 3D meshes. The observed order of accuracy is calculated and compared with the formal order of accuracy of the chosen numerical method. The code is verified when the observed order of accuracy matches the formal order in the limit as the mesh is refined.

Comprehensive Numerical Study of Jet-Flow Impingement over Flat Plates

Thepresentstudyattemptsanumericalinvestigationofthecomplexe owe eldthatoccurswhenanunderexpanded jet collides against a solid surface. Numerous examples of this problem can be found in the aerospace industry (e.g., rocket test stands, multistage separation ). A simplie ed geometry, already employed in previous experimental inquiries, was chosen as a test case: an underexpanded, axisymmetric, air jet impinging on a e at plate at varied angles. The three-dimensional Navier ‐Stokes equations were solved by means of a second-order-accurate Roetype algorithm with a generalized grid formulation. The computational domain includes theconvergent ‐divergent nozzle and the external e eld. The numerical results show various jet-shock and shock-shock interactions and compare very well with experimental data, including shadowgraph pictures and both location and values of the peak pressures on the inclined plate. This investigation focused on performing a thorough comparison between experiments and simulations, thereby establishing some level of cone dence in the accuracy and reliability of the numerical tool developed, CHEM. CHEM can accommodate more complicated and realistic geometries and physical conditions than those encountered in this study: with further ree nement and validation it can be used for rocket plume and plume/solid surface interaction simulations, both on the ground and in e ight.

Verification of RANS Turbulence Models in Loci-CHEM using the Method of Manufactured Solutions

An approach for verifying Reynolds-Averaged Navier-Stokes (RANS) turbulence models in Computational Fluid Dynamics (CFD) codes is presented. This approach relies on smooth, non-physical Manufactured Solutions which are chosen so as to provide contributions from all of the terms in the turbulence transport equations including convection, diffusion, production, and destruction. The Loci-CHEM CFD code is employed to solve the steady-state, compressible RANS equations in two-dimensions. The turbulence model verified is the baseline version of Menters k-ω model. Special attention is paid to the blending function which allows the model to switch between a k-ω and a transformed k-e model. Results are presented for the observed order of accuracy on families of Cartesian structured grids, skewed curvilinear structured grids, and unstructured triangular grids. The Manufactured Solutions clearly identify problems on skewed meshes which cause the diffusion operator to become inconsistent (i.e., the discretization error does not decrease with mesh refinement). An alternative formulation for the diffusion operator is implemented and the Manufactured Solutions are shown to converge in a second-order manner with mesh refinement for the structured grids. Preliminary investigations on the unstructured meshes indicate that while the code is still consistent, the observed order of accuracy is reduced to first order. Nomenclature ω k CD cross-diffusion term in the turbulence frequency transport equation p c specific heat at constant pressure v c specific heat at constant volume k DE discretization error on mesh level k; k=1,2,3,… d normal distance to the nearest no-slip wall e Favre-averaged internal energy t e Favre-averaged total energy F Menter blending function 1 F Menter blending function for the baseline model exact f exact solution to the partial differential equations k f numerical solution to the partial differential equations on mesh level k; k=1,2,3,… h Favre-averaged enthalpy f h heat of formation