Simone Marras

Stabilized high-order Galerkin methods based on a parameter-free dynamic SGS model for LES

The high order spectral element approximation of the Euler equations is stabilized via a dynamic sub-grid scale model (Dyn-SGS). This model was originally designed for linear finite elements to solve compressible flows at large Mach numbers. We extend its application to high-order spectral elements to solve the Euler equations of low Mach number stratified flows. The major justification of this work is twofold: stabilization and large eddy simulation are achieved via one scheme only.Because the diffusion coefficients of the regularization stresses obtained via Dyn-SGS are residual-based, the effect of the artificial diffusion is minimal in the regions where the solution is smooth. The direct consequence is that the nominal convergence rate of the high-order solution of smooth problems is not degraded. To our knowledge, this is the first application in atmospheric modeling of a spectral element model stabilized by an eddy viscosity scheme that, by construction, may fulfill stabilization requirements, can model turbulence via LES, and is completely free of a user-tunable parameter.From its derivation, it will be immediately clear that Dyn-SGS is independent of the numerical method; it could be implemented in a discontinuous Galerkin, finite volume, or other environments alike. Preliminary discontinuous Galerkin results are reported as well. The straightforward extension to non-linear scalar problems is also described. A suite of 1D, 2D, and 3D test cases is used to assess the method, with some comparison against the results obtained with the most known Lilly-Smagorinsky SGS model.

Strong scaling for numerical weather prediction at petascale with the atmospheric model NUMA

Numerical weather prediction (NWP) has proven to be computationally challenging due to its inherent multiscale nature. Currently, the highest resolution global NWP models use a horizontal resolution of 9 km. At this resolution, many important processes in the atmosphere are not resolved. Needless to say, this introduces errors. In order to increase the resolution of NWP models, highly scalable atmospheric models are needed. The non-hydrostatic unified model of the atmosphere (NUMA), developed by the authors at the Naval Postgraduate School, was designed to achieve this purpose. NUMA is used by the Naval Research Laboratory, Monterey as the engine inside its next generation weather prediction system NEPTUNE. NUMA solves the fully compressible Navier–Stokes equations by means of high-order Galerkin methods (both spectral element as well as discontinuous Galerkin methods can be used). NUMA is capable of running middle and upper atmosphere simulations since it does not make use of the shallow-atmosphere approximation. This article presents the performance analysis and optimization of the spectral element version of NUMA. The performance at different optimization stages is analyzed using a theoretical performance model as well as measurements via hardware counters. Machine-independent optimization is compared to machine-specific optimization using Blue Gene (BG)/Q vector intrinsics. The best portable version of the main computations was found to be about two times slower than the best non-portable version. By using vector intrinsics, the main computations reach 1.2 PFlops on the entire IBM Blue Gene supercomputer Mira (12% of the theoretical peak performance). The article also presents scalability studies for two idealized test cases that are relevant for NWP applications. The atmospheric model NUMA delivers an excellent strong scaling efficiency of 99% on the entire supercomputer Mira using a mesh with 1.8 billion grid points. This allows running a global forecast of a baroclinic wave test case at a 3-km uniform horizontal resolution and double precision within the time frame required for operational weather prediction.

A parameter-free dynamic alternative to hyper-viscosity for coupled transport equations

The stabilization of high order spectral elements to solve the transport equations for tracers in the atmosphere remains an active topic of research among atmospheric modelers. This paper builds on our previous work on variational multiscale stabilization (VMS) and discontinuity capturing (DC) (Marras et al. (2012) 7]) and shows the applicability of VMS+DC to realistic atmospheric problems that involve physics coupling with phase change in the simulation of 3D deep convection. We show that the VMS+DC approach is a robust technique that can damp the high order modes characterizing the spectral element solution of complex coupled transport problems. The method has important properties that techniques of more common use often lack: 1) it is free of a user-defined parameter, 2) it is anisotropic in that it only acts along the flow direction, 3) it is numerically consistent, and 4) it can improve the monotonicity of high-order spectral elements. The proposed method is assessed by comparing the results against those obtained with a fourth-order hyper-viscosity programmed in the same code. The main conclusion that arises is that tuning can be fully avoided without loss of accuracy if the dissipative scheme is properly designed. Finally, the cost of parallel communication is that of a second order operator which means that fewer communications are required by VMS+DC than by a hyper-viscosity method; fewer communications translate into a faster and more scalable code, which is of vital importance as we approach the exascale range of computing.

An unstructured CFD approach for numerical weather prediction

The aim of this paper is twofold. First, it is difficult for a newcomer in the Numerical Weather Prediction (NWP) community to find similarities with Computational Fluid Dynamics (CFD) techniques as far as the numerical methods of the dynamical cores in NWP are concerned. Different variables than the CFD traditional conservative one are used and seemingly different discretization techniques have been developed, whereas the very same Euler equations are being solved in both cases. So the first aim is to compare and contrast the main numerical elements used in both communities. The second aim consists in validating a CFD solver adapted to NWP to a set of traditional NWP benchmarks on fully nonstructured three dimensional configurations. It is shown that it produces accurate and low diffusive results. The main advantages of this approach are the same than compared to CFD finite difference solvers, namely scalability, adaptivity for localized phenomena and geometrical flexibility. Pole singularities are trivially removed.

Linear and nonlinear ultrasound simulations using the discontinuous Galerkin method

A nodal discontinuous Galerkin (DG) code based on the nonlinear wave equation is developed to simulate transient ultrasound propagation. The DG method has high-order accuracy, geometric flexibility, low dispersion error, and excellent scalability, so DG is an ideal choice for solving this problem. A nonlinear acoustic wave equation is written in a first-order flux form and discretized using nodal DG. A dynamic sub-grid scale stabilization method for reducing Gibbs oscillations in acoustic shock waves is then established. Linear and nonlinear numerical results from a two-dimensional axisymmetric DG code are presented and compared to numerical solutions obtained from linear and Khokhlov-Zabolotskaya-Kuznetsov-based simulations in FOCUS. The numerical results indicate that these nodal DG simulations capture nonlinearity, thermoviscous absorption, and diffraction for both flat and focused pistons in homogeneous media.

Physics-based Stabilization of Spectral Elements for the 3D Euler Equations of Moist Atmospheric Convection

In the context of stabilization of high order spectral elements, we introduce a dissipative scheme based on the solution of the compressible Euler equations that are regularized through the addition of a residual-based stress tensor. Because this stress tensor is proportional to the residual of the unperturbed equations, its effect is close to none where the solution is sufficiently smooth, whereas it increases elsewhere. This paper represents a first extension of the work by Nazarov and Hoffman (Int J Numer Methods Fluids 71:339–357, 2013) to high-order spectral elements in the context of low Mach number atmospheric dynamics. The simulations show that the method is reliable and robust for problems with important stratification and thermal processes such as the case of moist convection. The results are partially compared against a Smagorinsky solution. With this work we mean to make a step forward in the implementation of a stabilized, high order, spectral element large eddy simulation (LES) model within the Nonhydrostatic Unified Model of the Atmosphere, NUMA.

Application of a Galerkin Finite Element Scheme to Atmospheric Buoyant and Gravity Driven Flows

The application of a new finite element (FE) technique for the solution of stratified, non-hydrostatic, low-Mach number flows is introduced in the context of mesoscale atmospheric modeling. In this framework, a Compressible Variational Multiscale (VMS-C) finite element algorithm to solve the conservative form of the Euler equations coupled to the conservation of potential temperature was developed. This methodology is new in the fields of Computational Fluid Dynamics for compressible flows and in Numerical Weather Prediction (NWP), and we mean to show its ability to maintain stability in the solution of thermal, gravity-driven flows in a stratified environment. This effort is justified by the advantages offered by a Galerkin finite element algorithm when massive parallel efficiency is a constraint, which is indeed becoming the paradigm for both CFD and NWP practitioners. The algorithm is validated against the standard test cases specifically designed to test the dynamical core of new atmospheric models. In the context of buoyant and gravity flows three tests are selected among those presented in the literature: the warm rising smooth anomaly, and two versions of the density current evolution from a cold disturbance defined by different initial conditions. The reference quantitative and qualitative values are taken from the literature and from the output obtained with the Weather Research and Forecasting model (WRF-ARW), a state-of-the-art research NWP model.

Nonlinear ultrasound simulations using a time-explicit discontinuous Galerkin (DG) method

Histotripsy with ultrasound is an emerging noninvasive therapeutic modality that uses cavitation to precisely destroy diseased soft tissue. Accurate simulations of histotripsy are needed for treatment planning and device design. These simulations are performed in the time-domain, span hundreds of wavelengths, and must handle strong shocks and discontinuities between materials, such as the brain and the skull. The discontinuous Galerkin (DG) method is an outstanding candidate for such simulations. DG methods possess the following qualities: 1) high order accuracy, 2) geometric flexibility, 3) excellent dissipation properties, and 4) excellent scalability on massively parallel machines. The objective of this work is to develop a massively parallel DG method for histotripsy simulations in the brain.

The Impacts of Dry Dynamic Cores on Asymmetric Hurricane Intensification

AbstractThe fundamental pathways for tropical cyclone (TC) intensification are explored by considering axisymmetric and asymmetric impulsive thermal perturbations to balanced, TC-like vortices using the dynamic cores of three different nonlinear numerical models. Attempts at reproducing the results of previous work, which used the community WRF Model, revealed a discrepancy with the impacts of purely asymmetric thermal forcing. The current study finds that thermal asymmetries can have an important, largely positive role on the vortex intensification, whereas other studies find that asymmetric impacts are negligible.Analysis of the spectral energetics of each numerical model indicates that the vortex response to asymmetric thermal perturbations is significantly damped in WRF relative to the other models. Spectral kinetic energy budgets show that this anomalous damping is primarily due to the increased removal of kinetic energy from the vertical divergence of the vertical pressure flux, which is related to ...