Jayanarayanan Sitaraman

Parallel domain connectivity algorithm for unsteady flow computations using overlapping and adaptive grids

This paper describes the algorithms and functionality of a new module developed to support overset grid assembly associated with performing time-dependent and adaptive moving body calculations of external aerodynamic flows using a multi-solver paradigm (i.e. different CFD solvers in different parts of the computational domain). We use the term domain connectivity in this paper to denote all the procedures that are involved in an overset grid assembly, and the module developed is referred henceforth as the domain-connectivity module. The domain-connectivity module coordinates the data transfer between different solvers applied in different parts of the computational domain - body fitted structured or unstructured to capture viscous near-wall effects, and Cartesian adaptive mesh refinement to capture effects away from the wall. The execution of the CFD solvers and the domain-connectivity module are orchestrated by a Python-based computational infrastructure. The domain-connectivity module is fully parallel and performs all its operations (identification of grid overlaps and determination of data interpolation strategy) on the partitioned grid data. In addition, the domain connectivity procedures are completely automated such that no user intervention or manual input is necessary. The capabilities and performance of the package are presented for several test problems, including flow over a NACA 0015 wing and an AGARD A2 slotted airfoil, hover simulation of a scaled V-22 rotor, and dynamic simulation of a UH-60A rotor in forward flight. A modification to the algorithm for improved domain connectivity solutions in problems with tight tolerances as well as heterogeneous grid clustering is also presented.

Robust and efficient overset grid assembly for partitioned unstructured meshes

This paper presents a method to perform efficient and automated Overset Grid Assembly (OGA) on a system of overlapping unstructured meshes in a parallel computing environment where all meshes are partitioned into multiple mesh-blocks and processed on multiple cores. The main task of the overset grid assembler is to identify, in parallel, among all points in the overlapping mesh system, at which points the flow solution should be computed (field points), interpolated (receptor points), or ignored (hole points). Point containment search or donor search, an algorithm to efficiently determine the cell that contains a given point, is the core procedure necessary for accomplishing this task. Donor search is particularly challenging for partitioned unstructured meshes because of the complex irregular boundaries that are often created during partitioning. Another challenge arises because of the large variation in the type of mesh-block overlap and the resulting large load imbalance on multiple processors. Desirable traits for the grid assembly method are efficiency (requiring only a small fraction of the solver time), robustness (correct identification of all point types), and full automation (no user input required other than the mesh system). Additionally, the method should be scalable, which is an important challenge due to the inherent load imbalance. This paper describes a fully-automated grid assembly method, which can use two different donor search algorithms. One is based on the use of auxiliary grids and Exact Inverse Maps (EIM), and the other is based on the use of Alternating Digital Trees (ADT). The EIM method is demonstrated to be more efficient than the ADT method, while retaining robustness. An adaptive load re-balance algorithm is also designed and implemented, which considerably improves the scalability of the method.

Time dependent adjoint-based optimization for coupled fluid-structure problems

A formulation for sensitivity analysis of fully coupled time-dependent aeroelastic problems is given in this paper. Both forward sensitivity and adjoint sensitivity formulations are derived that correspond to analogues of the fully coupled non-linear aeroelastic analysis problem. Both sensitivity analysis formulations make use of the same iterative disciplinary solution techniques used for analysis, and make use of an analogous coupling strategy. The information passed between fluid and structural solvers is dimensionally equivalent in all cases, enabling the use of the same data structures for analysis, forward and adjoint problems. The fully coupled adjoint formulation is then used to perform rotor blade design optimization for a four bladed HART2 rotor in hover conditions started impulsively from rest. The effect of time step size and mesh resolution on optimization results is investigated. A fully coupled unsteady aeroelastic sensitivity analysis formulation is given.Forward and adjoint sensitivity formulations are analogues of analysis formulation.Fluid-structure interface information is dimensionally equivalent in all solvers.A four bladed HART2 rotor shape is optimized in hover using the adjoint platform.The effect of time step size and mesh resolution on optimization results is investigated.

Wall distance search algorithm using voxelized marching spheres

Minimum distance to a solid wall is a commonly used parameter in turbulence closure formulations associated with the Reynolds Averaged form of the Navier Stokes Equations (RANS). This paper presents a new approach to efficiently compute the minimum distance between a set of points and a surface. The method is based on sphere voxelization, and uses fast integer arithmetic algorithms from the field of computer graphics. Using a simple test case where the number of points (Np) and surface elements (Nb) can be independently specified, the present method is empirically estimated to be O(Np^0^.^8Nb^0^.^5). An unstructured grid around an aircraft configuration (DLR-F6) is chosen as the test case for demonstration and validation. Multi-processor computations (up to 256 processors) are conducted to study efficiency and scalability. Encouraging results are obtained, with the sphere voxelization algorithm demonstrated to be more efficient than all of the alternate methods for computing minimum distances. However, a load imbalance does exist, which negatively impacts the scalability for large number of cores. A simple method for load re-balancing is formulated and tested, which results in significant improvements in both efficiency and scalability.

CU++: an object oriented framework for computational fluid dynamics applications using graphics processing units

The application of graphics processing units (GPU) to solve partial differential equations is gaining popularity with the advent of improved computer hardware. Various lower level interfaces exist that allow the user to access GPU specific functions. One such interface is NVIDIA’s Compute Unified Device Architecture (CUDA) library. However, porting existing codes to run on the GPU requires the user to write kernels that execute on multiple cores, in the form of Single Instruction Multiple Data (SIMD). In the present work, a higher level framework, termed CU++, has been developed that uses object oriented programming techniques available in C++ such as polymorphism, operator overloading, and template meta programming. Using this approach, CUDA kernels can be generated automatically during compile time. Briefly, CU++ allows a code developer with just C/C++ knowledge to write computer programs that will execute on the GPU without any knowledge of specific programming techniques in CUDA. This approach is tremendously beneficial for Computational Fluid Dynamics (CFD) code development because it mitigates the necessity of creating hundreds of GPU kernels for various purposes. In its current form, CU++ provides a framework for parallel array arithmetic, simplified data structures to interface with the GPU, and smart array indexing. An implementation of heterogeneous parallelism, i.e., utilizing multiple GPUs to simultaneously process a partitioned grid system with communication at the interfaces using Message Passing Interface (MPI) has been developed and tested.

A GPU-based incompressible Navier–Stokes solver on moving overset grids

In pursuit of obtaining high fidelity solutions to the fluid flow equations in a short span of time, graphics processing units (GPUs) which were originally intended for gaming applications are currently being used to accelerate computational fluid dynamics (CFD) codes. With a high peak throughput of about 1 TFLOPS on a PC, GPUs seem to be favourable for many high-resolution computations. One such computation that involves a lot of number crunching is computing time accurate flow solutions past moving bodies. The aim of the present paper is thus to discuss the development of a flow solver on unstructured and overset grids and its implementation on GPUs. In its present form, the flow solver solves the incompressible fluid flow equations on unstructured/hybrid/overset grids using a fully implicit projection method. The resulting discretised equations are solved using a matrix-free Krylov solver using several GPU kernels such as gradient, Laplacian and reduction. Some of the simple arithmetic vector calculations are implemented using the CU++: An Object Oriented Framework for Computational Fluid Dynamics Applications using Graphics Processing Units, Journal of Supercomputing, 2013, doi:10.1007/s11227-013-0985-9 approach where GPU kernels are automatically generated at compile time. Results are presented for two- and three-dimensional computations on static and moving grids.

Out-of-core visualization of time-varying hybrid-grid volume data

Traditional computational fluid dynamics (CFD) solvers are usually written for a single gridding paradigm such as structured-Cartesian, structured-body-fitted, or unstructured grids. Each type of mesh paradigms has inherent advantages and disadvantages. Thus, the methods of coupling multiple mesh paradigms have been developed to facilitate the use of different solvers in different part of the computational domain. However, the complex hybrid gridding paradigm poses challenges to rendering calculations for visualizing the data. This paper describes a volume visualization system for time-varying adaptive moving-body CFD datasets, where the grid system consists of unstructured grids near the body surface, coupled with Structured Adaptive Mesh Refinement (SAMR) grid in the off-body domain. We present two approaches to the hybrid-grid volume ray casting: a KD-tree based single-pass algorithm, and a multi-pass algorithm using the depth peeling technique. The system has a three-level memory hierarchy: GPU memory, main memory, and a solid state drive (SSD). Through data caching and prefetching within the memory hierarchy, the latency of time-step swapping can be hidden. Experimental results show that our system allows interactive volume exploration on single-GPU commodity PCs.

On the integral constraint of the pressure Poisson equation for incompressible flows

We illustrate, using analytical and numerical proofs, how a conservative discretisation of the pressure Poisson equation arising out of the discretisation of the incompressible Navier–Stokes equations (on a two-dimensional unstructured non-staggered grid) satisfies the integral constraint on the pressure boundary condition without any additional treatment. When discretised in a non-conservative manner, it is seen that the integral constraint is not exactly satisfied, but only to an order , where is an appropriate velocity scale. When solved using an iterative method, such as the Bi-Conjugate Stabilised method, it is proved that the vanishing sum of residuals on all points inclusive of the boundary is a consequence of this integral constraint. This result can then be used as a tool to identify whether the discrete integral constraint has been satisfied or not, especially when the pressure is solved as a Neumann problem.

A code-independent generalized actuator line model for wind farm aerodynamics over simple and complex terrain

Abstract Actuator line model representations of wind turbines reduce the simulation cost of wind farm aerodynamics relative to blade geometry resolving simulations. However, most implementations are code specific and may not have proper load balancing when run in parallel. Here, a generalized actuator line model (GALM) is developed to overcome this implementation drawback of existing approaches. The GALM can be coupled to any existing microscale solver with minimal or no modifications. The coupling of our GALM model with the open-source microscale code CgWind is used to demonstrate both its ease of use and fidelity. The results obtained from the coupled-model simulations are validated for both isolated turbine in wind tunnel experiments and measurements in offshore wind farm. Finally, an operational wind farm over complex terrain is demonstrated. The simulations show that wake meandering and power production are strongly influenced by terrain impacts. Access information for obtaining the code is provided.

Simulation of Attached and Separated Flows using Realizable Linear and Non-Linear Hybrid RANS-LES Models

Hybrid modeling approach, employing a Reynolds averaged Navier-Stokes (RANS) model in the near-wall region and large eddy simulation(LES) away from the wall is gaining popularity for the simulation of engineering flows as it reduces the computational expense of LES. However, there are a number of issues with the existing hybrid models which are yet to be resolved fully. To propose a possible solution, a unified RANS-LES model derived using stochastic analysis is investigated for the simulation of separated flows for the first time. Flow past a NACA 0012 airfoil over a wide range of angles of attack is chosen as the test case for the study. To provide a comparison with existing models, simulations were also performed using detached eddy simulation (DES). The results show that the unified and DES are able to predict pre-stall lift and drag accurately when the simulations were performed in three-dimensions. Post-stall lift and drag predictions from both the models were unsatisfactory. Further suggestions for improving the performance of the unified model are presented.