Nicholas Malaya

Estimating uncertainties in statistics computed from direct numerical simulation

Rigorous assessment of uncertainty is crucial to the utility of direct numerical simulation (DNS) results. Uncertainties in the computed statistics arise from two sources: finite statistical sampling and the discretization of the Navier–Stokes equations. Due to the presence of non-trivial sampling error, standard techniques for estimating discretization error (such as Richardson extrapolation) fail or are unreliable. This work provides a systematic and unified approach for estimating these errors. First, a sampling error estimator that accounts for correlation in the input data is developed. Then, this sampling error estimate is used as part of a Bayesian extension of Richardson extrapolation in order to characterize the discretization error. These methods are tested using the Lorenz equations and are shown to perform well. These techniques are then used to investigate the sampling and discretization errors in the DNS of a wall-bounded turbulent flow at Reτ ≈ 180. Both small (Lx/δ × Lz/δ = 4π × 2π) and la...Rigorous assessment of uncertainty is crucial to the utility of DNS results. Uncertainties in the computed statistics arise from two sources: finite statistical sampling and the discretization of the Navier-Stokes equations. Due to the presence of non-trivial sampling error, standard techniques for estimating discretization error (such as Richardson extrapolation) fail or are unreliable. This work provides a systematic and unified approach for estimating these errors. First, a sampling error estimator that accounts for correlation in the input data is developed. Then, this sampling error estimate is used as part of a Bayesian extension of Richardson extrapolation in order to characterize the discretization error. These methods are tested using the Lorenz equations and are shown to perform well. These techniques are then used to investigate the sampling and discretization errors in the DNS of a wall-bounded turbulent flow. For both cases, it is found that while the sampling uncertainty is large enough to make the order of accuracy difficult to determine, the estimated discretization errors are quite small. This indicates that the commonly used heuristics provide ad- equate resolution for this class of problems. However, it is also found that, for some quantities, the discretization error is not small relative to sampling error, indicating that the conventional wisdom that sampling error dominates discretization error for this class of simulations needs to be reevaluated.

Petascale direct numerical simulation of turbulent channel flow on up to 786K cores

We present results of performance optimization for direct numerical simulation (DNS) of wall bounded turbulent flow (channel flow). DNS is a technique in which the fluid flow equations are solved without subgrid modeling. Of particular interest are high Reynolds number (Re) turbulent flows over walls, because of their importance in technological applications. Simulating high Re turbulence is a challenging computational problem, due to the high spatial and temporal resolution requirements.

A Web services accessible database of turbulent channel flow and its use for testing a new integral wall model for LES

abstract The output from a direct numerical simulation (DNS) of turbulent channel flow at Reτ ≈ 1000 is used to construct a publicly and Web services accessible, spatio-temporal database for this flow. The simulated channel has a size of 8πh × 2h × 3πh, where h is the channel half-height. Data are stored at 2048 × 512 × 1536 spatial grid points for a total of 4000 time samples every 5 time steps of the DNS. These cover an entire channel flow-through time, i.e. the time it takes to traverse the entire channel length 8πh at the mean velocity of the bulk flow. Users can access the database through an interface that is based on the Web services model and perform numerical experiments on the slightly over 100 terabytes (TB) DNS data on their remote platforms, such as laptops or local desktops. Additional technical details about the pressure calculation, database interpolation, and differentiation tools are provided in several appendices. As a sample application of the channel flow database, we use it to conduct an a-priori test of a recently introduced integral wall model for large eddy simulation of wall-bounded turbulent flow. The results are compared with those of the equilibrium wall model, showing the strengths of the integral wall model as compared to the equilibrium model.

Theoretically based optimal large-eddy simulation

Large eddy simulation (LES), in which the large scales of turbulence are simulated while the effects of the small scales are modeled, is an attractive approach for predicting the behavior of turbulent flows. However, there are a number of modeling and formulation challenges that need to be addressed for LES to become a robust and reliable engineering analysis tool. Optimal LES is a LES modeling approach developed to address these challenges. It requires multipoint correlation data as input to the modeling, and to date these data have been obtained from direct numerical simulations (DNSs). If optimal LES is to be generally useful, this need for DNS statistical data must be overcome. In this paper, it is shown that the Kolmogorov inertial range theory, along with an assumption of small-scale isotropy, the application of the quasinormal approximation and a mild modeling assumption regarding the three-point third-order correlation are sufficient to determine all the correlation data required for optimal LES m...

MASA: a library for verification using manufactured and analytical solutions

In this paper we introduce the Manufactured Analytical Solution Abstraction (MASA) library for applying the method of manufactured solutions to the verification of software used for solving a large class of problems stemming from numerical methods in mathematical physics including nonlinear equations, systems of algebraic equations, and ordinary and partial differential equations. We discuss the process of scientific software verification, manufactured solution generation using symbolic manipulation with computer algebra systems such as Maple™ or SymPy, and automatic differentiation for forcing function evaluation. We discuss a hierarchic methodology that can be used to alleviate the combinatorial complexity in generating symbolic manufactured solutions for systems of equations based on complex physics. Finally, we detail the essential features and examples of the Application Programming Interface behind MASA, an open source library designed to act as a central repository for manufactured and analytical solutions over a diverse range of problems.

Experiences from Leadership Computing in Simulations of Turbulent Fluid Flows

We performed a direct numerical simulation (DNS) of high Reynolds number turbulent channel flow to expand our understanding of wall-bounded turbulence. The resolution requirements inherent to realistic turbulent flows necessitate leadership computing systems. Using Mira at Argonne Leadership Computing Facility, Argonne National Laboratory, we were able to achieve DNS at Re_tau = 5,200, and generated approximately 140 Tbytes of data. To use Mira efficiently, we developed a new DNS code, PoongBack, including a new parallel 3D FFT kernel. The new code shows excellent scalability up to 786,432 cores. Here, we summarize our code development and production simulation efforts, including parallel I/O.

Manufactured Solutions for the Favre-Averaged Navier-Stokes Equations with Eddy-Viscosity Turbulence Models

The Method of Manufactured Solutions is applied to verify the implementation of eddy viscosity turbulence models for closure of the Favre-averaged Navier-Stokes equations. In the Method of Manufactured Solutions, the governing equations are modied by the addition of source terms such that the exact solution|i.e., the manufactured solution|is known a priori. Given the exact solution, order of accuracy studies are conducted to verify that the discrete solution converges to the exact solution at the expected rate. The goal of this work is to verify the implementation of turbulence models in the Fully-Implicit NavierStokes ow solver. The turbulence model of interest for this work is the Spalart-Allmaras one-equation model, and two manufactured solutions have been examined. The rst solution is based on trigonometric functions, as commonly used in manufactured solution literature. This solution is appropriate for use in unbounded ows, enabling verication of the implementation of the free shear ow form of the model. The second solution has been newly developed in this work and is intended for use in wall-bounded ows. While other manufactured solutions for the Spalart-Allmaras model for wall-bounded ow have appeared in the literature, these solutions are shown to have features that make them illsuited to verication. To avoid such features, the wall-bounded solution developed here is loosely based on the behavior of the model solution in the inner portion of a zero-pressure boundary layer. Results obtained using both solutions show that the Fully-Implicit NavierStokes ow solver is achieving the expected second-order accuracy.

Petascale I/O using HDF-5

Our work is focused on performing Petascale Direct Numerical Simulations (DNS) of turbulent flows. An essential performance component of these simulations are the restart files, as a single petascale simulation will write on the order of a petabyte. Petascale I/O requires both performance and dataset maintainability, for archival and post processing of the velocity fields for statistics. This paper presents benchmarks and comparisons between a single shared file written using the HDF-5 library and a POSIX compliant I/O library on several top-10 machines and filesystems. It is shown that a properly tuned HDF-5 routine provides strong I/O performance, which coupled with the metadata handling and portability available to the file format, indicates that the lower performance provides a worthy tradeoff. The benchmarks presented were provided from real turbulence simulations and required extensive tuning for different platforms, and in particular, between different file systems (GPFS and Lustre).

Modeling Multi-point Correlations in Wall-Bounded Turbulence

In large eddy simulation (LES), one is generally not interested in the large-scale or filtered quantities computed in the simulation, but rather the corresponding characteristics of the underlying real turbulence. One approach to reconstructing the statistics of turbulence from the filtered statistics of an LES is to employ models for the small separation multi-point velocity correlations, which can be parameterized using the statistics of the LES. This has been employed to good effect in isotropic turbulence, but to employ this technique for near-wall turbulent shear flows requires a model for the anisotropy and inhomogeneity in the correlations. Here we explore the use of multi-point correlation models in LES modeling and reconstruction, and propose a anisotropy/inhomogeneity model for the two-point second-order correlation.