Ken’ichi Itakura

Energy dissipation rate and energy spectrum in high resolution direct numerical simulations of turbulence in a periodic box

High-resolution direct numerical simulations (DNSs) of incompressible homogeneous turbulence in a periodic box with up to 40963 grid points were performed on the Earth Simulator computing system. DNS databases, including the present results, suggest that the normalized mean energy dissipation rate per unit mass tends to a constant, independent of the fluid kinematic viscosity ν as ν→0. The DNS results also suggest that the energy spectrum in the inertial subrange almost follows the Kolmogorov k−5/3 scaling law, where k is the wavenumber, but the exponent is steeper than −5/3 by about 0.1.

Small-scale statistics in high-resolution direct numerical simulation of turbulence : Reynolds number dependence of one-point velocity gradient statistics

One-point statistics of velocity gradients and Eulerian and Lagrangian accelerations are studied by analysing the data from high-resolution direct numerical simulations (DNS) of turbulence in a periodic box, with up to 4096 3 grid points. The DNS consist of two series of runs; one is with k max η∼ 1 (Series 1) and the other is with k max η∼2 (Series 2), where k max is the maximum wavenumber and η the Kolmogorov length scale. The maximum Taylor-microscale Reynolds number R λ in Series 1 is about 1130, and it is about 675 in Series 2. Particular attention is paid to the possible Reynolds number (Re) dependence of the statistics. The visualization of the intense vorticity regions shows that the turbulence field at high Re consists of clusters of small intense vorticity regions, and their structure is to be distinguished from those of small eddies. The possible dependence on Re of the probability distribution functions of velocity gradients is analysed through the dependence on R λ of the skewness and flatness factors (S and F). The DNS data suggest that the R λ dependence of S and F of the longitudinal velocity gradients fit well with a simple power law: S∼-0.32R λ 0.11 and F∼1.14R λ 0.34 , in fairly good agreement with previous experimental data. They also suggest that all the fourth-order moments of velocity gradients scale with R λ similarly to each other at R λ >00, in contrast to R λ < 100. Regarding the statistics of time derivatives, the second-order time derivatives of turbulent velocities are more intermittent than the first-order ones for both the Eulerian and Lagrangian velocities, and the Lagrangian time derivatives of turbulent velocities are more intermittent than the Eulerian time derivatives, as would be expected. The flatness factor of the Lagrangian acceleration is as large as 90 at R λ ≈430. The flatness factors of the Eulerian and Lagrangian accelerations increase with R λ approximately proportional to R λ αE and R λ αL , respectively, where α E ≈0.5 and α L ≈1.0, while those of the second-order time derivatives of the Eulerian and Lagrangian velocities increases approximately proportional to R λ βE and R λ βL , respectively, where β E ≈1.5 and β L ≈3.0.

16.4-Tflops Direct Numerical Simulation of Turbulence by a Fourier Spectral Method on the Earth Simulator

The high-resolution direct numerical simulations (DNSs) of incompressible turbulence with numbers of grid points up to 40963 have been executed on the Earth Simulator (ES). The DNSs are based on the Fourier spectral method, so that the equation for mass conservation is accurately solved. In DNS based on the spectral method, most of the computation time is consumed in calculating the three-dimensional (3D) Fast Fourier Transform (FFT), which requires huge-scale global data transfer and has been the major stumbling block that has prevented truly high-performance computing. By implementing new methods to efficiently perform the 3D-FFT on the ES, we have achieved DNS at 16.4 Tflops on 20483 grid points. The DNS yields an energy spectrum exhibiting a wide inertial subrange, in contrast to previous DNSs with lower resolutions, and therefore provides valuable data for the study of the universal features of turbulence at large Reynolds number.

Performance evaluation of NEC SX-9 using real science and engineering applications

This paper describes a new-generation vector parallel supercomputer, NEC SX-9 system. The SX-9 processor has an outstanding core to achieve over 100Gflop/s, and a software-controllable on-chip cache to keep the high ratio of the memory bandwidth to the floating-point operation rate. Moreover, its large SMP nodes of 16 vector processors with 1.6Tflop/s performance and 1TB memory are connected with dedicated network switches, which can achieve inter-node communication at 128GB/s per direction. The sustained performance of the SX-9 processor is evaluated using six practical applications in comparison with conventional vector processors and the latest scalar processor such as Nehalem-EP. Based on the results, this paper discusses the performance tuning strategies for new-generation vector systems. An SX-9 system of 16 nodes is also evaluated by using the HPC challenge benchmark suite and a CFD code. Those evaluation results clarify the highest sustained performance and scalability of the SX-9 system.

Statistics of Energy Transfer in High-Resolution Direct Numerical Simulation of Turbulence in a Periodic Box

The statistics of energy transfer is studied by using the data of a series of high-resolution direct numerical simulations of incompressible homogeneous turbulence in a periodic box with the Taylor micro-scale Reynolds number R λ and grid points up to approximately 1130 and 4096 3 , respectively. The data show that the energy transfer T across the wave number k is highly intermittent and the skewness S and flatness F of T increase with k approximately as S ∝( k L ) α S , F ∝( k L ) α F in the inertial subrange, where α S ∼2/3, α F ∼1 and L the characteristic length scale of energy containing eddies. The comparison between the statistics of T , the energy dissipation rate e and its average e r over a domain of scale r shows that T is less intermittent than e, while there is a certain similarity between the probability distribution functions of T and e r .

Spectra of Energy Dissipation, Enstrophy and Pressure by High-Resolution Direct Numerical Simulations of Turbulence in a Periodic Box

The spectra of the squares of velocity quadratics including the energy dissipation rate e per unit mass, the enstrophy ω 2 and the pressure p were measured using the data obtained from direct numerical simulations (DNSs) of incompressible turbulence in a periodic box with number of grid points up to 2048 3 . These simulations were performed using the Earth Simulator computing system. The spectra for e, ω 2 and p exhibited a wave number range in which the spectra scaled with the wave number k as ∝ k - a . Exponent a for p was about 1.81, which is in good agreement with the value obtained by assuming the joint probability distribution of the velocity field to be Gaussian, while a values for e and ω 2 were about 2/3, and very different from the Gaussian approximation values.

Energy Spectrum in the Near Dissipation Range of High Resolution Direct Numerical Simulation of Turbulence

The energy spectrum in the near dissipation range of turbulence is studied by analyzing the data of a series of high-resolution direct numerical simulations of incompressible homogeneous turbulence...

World-highest resolution global atmospheric model and its performance on the Earth Simulator

Mechanisms of interactions among different scale phenomena play important roles for forecasting of weather and climate. Multi-scale Simulator for the Geoenvironment (MSSG), which deals with multi-scale multi-physics phenomena, is a coupled non-hydrostatic atmosphere-ocean model designed to be run efficiently on the Earth Simulator. We present its simulation results with the world-highest 1.9km horizontal resolution for the entire globe. To gain high performance by exploiting the system capabilities, we propose novel performance evaluation metrics that incorporate the effects of the data caching mechanism between CPU and memory. A potentially attainable computational performance is also introduced by evaluating both computational and memory intensities. With the useful code optimization guideline based on such metrics, we demonstrate that MSSG can achieve an excellent peak performance ratio of 32.2% on the Earth Simulator with the single-core performance found to be a key to reduced time-to-solution.

Scalability of hybrid programming for a CFD code on the earth simulator

The Earth Simulator (ES) is an SMP cluster system. There are two types of parallel programming models available on the ES. One is a flat programming model, in which a parallel program is implemented by MPI interfaces only, both within an SMP node and among nodes. The other is a hybrid programming model, in which a parallel program is written by using thread programming within an SMP node and MPI programming among nodes simultaneously. It is generally known that it is difficult to obtain the same high level of performance using the hybrid programming model as can be achieved with the flat programming model.In this paper, we have evaluated scalability of the code for direct numerical simulation of the Navier-Stokes equations on the ES. The hybrid programming model achieves the sustained performance of 346.9 Gflop/s, while the flat programming model achieves 296.4 Gflop/s with 16 PNs of the ES for a DNS problem size of 2563. For small scale problems, however, the hybrid programming model is not as efficient because of microtasking overhead. It is shown that there is an advantage for the hybrid programming model on the ES for the larger size problems.

Migration of an Atmospheric Simulation Code to an OpenACC Platform Using the Xevolver Framework

As the diversity of HPC systems increases, even legacy HPC applications often need to use accelerators for higher performance. To migrate large-scale legacy HPC applications to modern HPC systems including accelerators, OpenACC is a promising approach because its directive-based approach can prevent drastic code modifications. This paper shows a case study of the migration of a large-scale atmospheric simulation code to an OpenACC platform by keeping the maintainability of the original code. Although OpenACC enables an application to use accelerators by adding a small number of directives, it requires modifying the original code to achieve a high performance in most cases, and tends to degrade the maintainability and/or portability. To avoid such code modifications, this paper adopts a code transformation framework, Xevolver. Instead of directly modifying the code, custom code transformation rules and custom directives are defined using the Xevolver framework. This paper first shows that just inserting OpenACC directives does not lead to high performance and non-trivial code modifications are required in practice. Then, the direct code modification can be avoided by using externally defined transformation rules and directives to keep the original code unchanged as much as possible. Finally, the performance evaluation shows that the code modifications can improve the performance of the OpenACC code.