Paul Fischer

A Parareal in Time Semi-implicit Approximation of the Navier-Stokes Equations

The “parareal in time” algorithm introduced in Lions et al. [2001] enables parallel computation using a decomposition of the interval of time integration. In this paper, we adapt this algorithm to solve the challenging Navier-Stokes problem. The coarse solver, based on a larger timestep, may also involve a coarser discretization in space. This helps to preserve stability and provides for more significant savings.

Roughness-Induced Transient Growth

Numerical simulations are used to model the boundary layer disturbance field due to a spanwise periodic array of circular disks at the surface. Our earlier computations (Fischer and Choudhari 2004) reproduced some of the major trends associated with roughness induced transient growth as measured by White and Ergin (2003), confirming the accuracy of their data reduction technique. The present paper explores the effects of roughness height, shape, and size on the transient growth along the wake of the roughness array. Results indicate the range of roughness heights over which the energy levels of the dominant stationary disturbance are consistent with the empirically determined Rek 2 scaling of White and Ergin. Computations further reveal the intricate effects of disturbance nonlinearity on transient growth characteristics. At sufficiently large roughness heights, simulations indicate spontaneous vortex shedding behind the roughness array, which has been identified in previous experiments as a precursor to premature transition (i.e., tripping) of the laminar boundary layer. The effect of stationary, suboptimal transient growth disturbances on the amplification of Tollmien-Schlichting instability waves in the boundary layer is also assessed in the light of the recent findings concerning the stabilizing influence of optimal growth streaks (Cossu and Brandt 2002).

Hydrodynamic and Longitudinal Impedance Analysis of Cerebrospinal Fluid Dynamics at the Craniovertebral Junction in Type I Chiari Malformation

Elevated or reduced velocity of cerebrospinal fluid (CSF) at the craniovertebral junction (CVJ) has been associated with type I Chiari malformation (CMI). Thus, quantification of hydrodynamic parameters that describe the CSF dynamics could help assess disease severity and surgical outcome. In this study, we describe the methodology to quantify CSF hydrodynamic parameters near the CVJ and upper cervical spine utilizing subject-specific computational fluid dynamics (CFD) simulations based on in vivo MRI measurements of flow and geometry. Hydrodynamic parameters were computed for a healthy subject and two CMI patients both pre- and post-decompression surgery to determine the differences between cases. For the first time, we present the methods to quantify longitudinal impedance (LI) to CSF motion, a subject-specific hydrodynamic parameter that may have value to help quantify the CSF flow blockage severity in CMI. In addition, the following hydrodynamic parameters were quantified for each case: maximum velocity in systole and diastole, Reynolds and Womersley number, and peak pressure drop during the CSF cardiac flow cycle. The following geometric parameters were quantified: cross-sectional area and hydraulic diameter of the spinal subarachnoid space (SAS). The mean values of the geometric parameters increased post-surgically for the CMI models, but remained smaller than the healthy volunteer. All hydrodynamic parameters, except pressure drop, decreased post-surgically for the CMI patients, but remained greater than in the healthy case. Peak pressure drop alterations were mixed. To our knowledge this study represents the first subject-specific CFD simulation of CMI decompression surgery and quantification of LI in the CSF space. Further study in a larger patient and control group is needed to determine if the presented geometric and/or hydrodynamic parameters are helpful for surgical planning.

Aspect ratio effects in turbulent duct flows studied through direct numerical simulation

Three-dimensional effects in turbulent duct flows, i.e., sidewall boundary layers and secondary motions, are studied by means of direct numerical simulation (DNS). The spectral element code Nek5000 is used to compute turbulent duct flows with aspect ratios 1–7 (at Reb, c = 2800, Reτ, c ≃ 180) and aspect ratio 1 (at Reb, c = 5600, Reτ, c ≃ 330), in streamwise-periodic boxes of length 25h. The total number of grid points ranges from 28 to 145 million, and the pressure gradient is adjusted iteratively in order to keep the same bulk Reynolds number in the centreplane with changing aspect ratio. Turbulence is initiated via a trip forcing active during the initial stages of the simulation, and the statistical convergence of the data is discussed both in terms of transient approach and averaging period. Spanwise variations in wall shear, mean-flow profiles, and turbulence statistics are analysed as a function of aspect ratio, and also compared with the spanwise-periodic channel (as idealisation of an infinite as...

Hybrid Schwarz-Multigrid Methods for the Spectral Element Method: Extensions to Navier-Stokes

The performance of multigrid methods for the standard Poisson problem and for the consistent Poisson problem arising in spectral element discretizations of the Navier-Stokes equations is investigated. It is demonstrated that overlapping additive Schwarz methods are effective smoothers, provided that the solution in the overlap region is weighted by the inverse counting matrix. It is also shown that spectral element based smoothers are superior to those based upon finite element discretizations. Results for several large 3D Navier-Stokes applications are presented.

OpenACC acceleration of the Nek5000 spectral element code

We present a case study of porting NekBone, a skeleton version of the Nek5000 code, to a parallel GPU-accelerated system. Nek5000 is a computational fluid dynamics code based on the spectral element method used for the simulation of incompressible flow. The original NekBone Fortran source code has been used as the base and enhanced by OpenACC directives. The profiling of NekBone provided an assessment of the suitability of the code for GPU systems, and indicated possible kernel optimizations. To port NekBone to GPU systems required little effort and a small number of additional lines of code (approximately one OpenACC directive per 1000 lines of code). The naïve implementation using OpenACC leads to little performance improvement: on a single node, from 16 Gflops obtained with the version without OpenACC, we reached 20 Gflops with the naïve OpenACC implementation. An optimized NekBone version leads to a 43 Gflop performance on a single node. In addition, we ported and optimized NekBone to parallel GPU systems, reaching a parallel efficiency of 79.9% on 1024 GPUs of the Titan XK7 supercomputer at the Oak Ridge National Laboratory.

High-resolution coupled physics solvers for analysing fine-scale nuclear reactor design problems

An integrated multi-physics simulation capability for the design and analysis of current and future nuclear reactor models is being investigated, to tightly couple neutron transport and thermal-hydraulics physics under the SHARP framework. Over several years, high-fidelity, validated mono-physics solvers with proven scalability on petascale architectures have been developed independently. Based on a unified component-based architecture, these existing codes can be coupled with a mesh-data backplane and a flexible coupling-strategy-based driver suite to produce a viable tool for analysts. The goal of the SHARP framework is to perform fully resolved coupled physics analysis of a reactor on heterogeneous geometry, in order to reduce the overall numerical uncertainty while leveraging available computational resources. The coupling methodology and software interfaces of the framework are presented, along with verification studies on two representative fast sodium-cooled reactor demonstration problems to prove the usability of the SHARP framework.

An MPI/OpenACC implementation of a high-order electromagnetics solver with GPUDirect communication

We present performance results and an analysis of a message passing interface (MPI)/OpenACC implementation of an electromagnetic solver based on a spectral-element discontinuous Galerkin discretization of the time-dependent Maxwell equations. The OpenACC implementation covers all solution routines, including a highly tuned element-by-element operator evaluation and a GPUDirect gather–scatter kernel to effect nearest neighbor flux exchanges. Modifications are designed to make effective use of vectorization, streaming, and data management. Performance results using up to 16,384 graphics processing units of the Cray XK7 supercomputer Titan show more than 2.5× speedup over central processing unit-only performance on the same number of nodes (262,144 MPI ranks) for problem sizes of up to 6.9 billion grid points. We discuss performance-enhancement strategies and the overall potential of GPU-based computing for this class of problems.

Stabilization of the Spectral Element Method in Convection Dominated Flows by Recovery of Skew-Symmetry

We investigate stability properties of the spectral element method for advection dominated incompressible flows. In particular, properties of the widely used convective form of the nonlinear term are studied. We remark that problems which are usually associated with the nonlinearity of the governing Navier–Stokes equations also arise in linear scalar transport problems, which implicates advection rather than nonlinearity as a source of difficulty. Thus, errors arising from insufficient quadrature of the convective term, commonly referred to as ‘aliasing errors’, destroy the skew-symmetric properties of the convection operator. Recovery of skew-symmetry can be efficiently achieved by the use of over-integration. Moreover, we demonstrate that the stability problems are not simply connected to underresolution. We combine theory with analysis of the linear advection-diffusion equation in 2D and simulations of the incompressible Navier–Stokes equations in 2D of thin shear layers at a very high Reynolds number and in 3D of turbulent and transitional channel flow at moderate Reynolds number. For the Navier–Stokes equations, where the divergence-free constraint needs to be enforced iteratively to a certain accuracy, small divergence errors can be detrimental to the stability of the method and it is therefore advised to use additional stabilization (e.g. so-called filter-based stabilization, spectral vanishing viscosity or entropy viscosity) in order to assure a stable spectral element method.

A spectrally accurate method for overlapping grid solution of incompressible Navier-Stokes equations

An overlapping mesh methodology that is spectrally accurate in space and up to third-order accurate in time is developed for solution of unsteady incompressible flow equations in three-dimensional domains. The ability to decompose a global domain into separate, but overlapping, subdomains eases mesh generation procedures and increases flexibility of modeling flows with complex geometries. The methodology employs implicit spectral element discretization of equations in each subdomain and explicit treatment of subdomain interfaces with spectrally-accurate spatial interpolation and high-order accurate temporal extrapolation, and requires few, if any, iterations, yet maintains the global accuracy and stability of the underlying flow solver. The overlapping mesh methodology is thoroughly validated using two-dimensional and three-dimensional benchmark problems in laminar and turbulent flows. The spatial and temporal convergence is documented and is in agreement with the nominal order of accuracy of the solver. The influence of long integration times, as well as inflow-outflow global boundary conditions on the performance of the overlapping grid solver is assessed. In a turbulent benchmark of fully-developed turbulent pipe flow, the turbulent statistics with the overlapping grids is validated against published available experimental and other computation data. Scaling tests are presented that show near linear strong scaling, even for moderately large processor counts. A novel overlapping grid method for incompressible flow equations is developed.The overlapping grid method is based on spectral-element discretization.The method maintains global spectral accuracy with polynomial refinement.An explicit interface extrapolation scheme for temporal coupling is proposed.The scheme allows for high-order temporal accuracy with only a few iterations.