Hans Johansen

A Fourth-Order Accurate Finite-Volume Method with Structured Adaptive Mesh Refinement for Solving the Advection-Diffusion Equation

We present a fourth-order accurate algorithm for solving Poissons equation, the heat equation, and the advection-diffusion equation on a hierarchy of block-structured, adaptively refined grids. For spatial discretization, finite-volume stencils are derived for the divergence operator and Laplacian operator in the context of structured adaptive mesh refinement and a variety of boundary conditions; the resulting linear system is solved with a multigrid algorithm. For time integration, we couple the elliptic solver to a fourth-order accurate Runge-Kutta method, introduced by Kennedy and Carpenter [Appl. Numer. Math., 44 (2003), pp. 139-181], which enables us to treat the nonstiff advection term explicitly and the stiff diffusion term implicitly. We demonstrate the spatial and temporal accuracy by comparing results with analytical solutions. Because of the general formulation of the approach, the algorithm is easily extensible to more complex physical systems.

Arbitrary-Order Conservative and Consistent Remapping and a Theory of Linear Maps: Part II

AbstractThis paper extends on the first part of this series by describing four examples of 2D linear maps that can be constructed in accordance with the theory of the earlier work. The focus is again on spherical geometry, although these techniques can be readily extended to arbitrary manifolds. The four maps include conservative, consistent, and (optionally) monotone linear maps (i) between two finite-volume meshes, (ii) from finite-volume to finite-element meshes using a projection-type approach, (iii) from finite-volume to finite-element meshes using volumetric integration, and (iv) between two finite-element meshes. Arbitrary order of accuracy is supported for each of the described nonmonotone maps.

Toward Exascale Earthquake Ground Motion Simulations for Near-Fault Engineering Analysis

Modernizing SW4 for massively parallel time-domain simulations of earthquake ground motions in 3D earth models increases resolution and provides ground motion estimates for critical infrastructure risk evaluations. Simulations of ground motions from large (M ≥ 7.0) earthquakes require domains on the order of 100 to500 km and spatial granularity on the order of 1 to5 m resulting in hundreds of billions of grid points. Surface-focused structured mesh refinement (SMR) allows for more constant grid point per wavelength scaling in typical Earth models, where wavespeeds increase with depth. In fact, MR allows for simulations to double the frequency content relative to a fixed grid calculation on a given resource. The authors report improvements to the SW4 algorithm developed while porting the code to the Cori Phase 2 (Intel Xeon Phi) systems at the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory. Investigations of the performance of the innermost loop of the calculations found that reorganizing the order of operations can improve performance for massive problems.

Analyzing the Adaptive Mesh Refinement (AMR) Characteristics of a High-Order 2D Cubed-Sphere Shallow-Water Model

AbstractAdaptive mesh refinement (AMR) is a technique that has been featured only sporadically in atmospheric science literature. This paper aims to demonstrate the utility of AMR for simulating atmospheric flows. Several test cases are implemented in a 2D shallow-water model on the sphere using the Chombo-AMR dynamical core. This high-order finite-volume model implements adaptive refinement in both space and time on a cubed-sphere grid using a mapped-multiblock mesh technique. The tests consist of the passive advection of a tracer around moving vortices, a steady-state geostrophic flow, an unsteady solid-body rotation, a gravity wave impinging on a mountain, and the interaction of binary vortices. Both static and dynamic refinements are analyzed to determine the strengths and weaknesses of AMR in both complex flows with small-scale features and large-scale smooth flows. The different test cases required different AMR criteria, such as vorticity or height-gradient based thresholds, in order to achieve the...

Parallel In Situ Detection of Connected Components in Adaptive Mesh Refinement Data

Adaptive Mesh Refinement (AMR) represents a significant advance for scientific simulation codes, greatly reducing memory and compute requirements by dynamically varying simulation resolution over space and time. As simulation codes transition to AMR, existing analysis algorithms must also make this transition. One such algorithm, connected component detection, is of vital importance in many simulation and analysis contexts, with some simulation codes even relying on parallel, in situ connected component detection for correctness. Yet, current detection algorithms designed for uniform meshes are not applicable to hierarchical, non-uniform AMR, and to the best of our knowledge, AMR connected component detection has not been explored in the literature. Therefore, in this paper, we formally define the general problem of connected component detection for AMR, and present a general solution. Beyond solving the general detection problem, achieving viable in situ detection performance is even more challenging. The core issue is the conflict between the communication-intensive nature of connected component detection (in general, and especially for AMR data) and the requirement that in situ processes incur minimal performance impact on the co-located simulation. We address this challenge by presenting the first connected component detection methodology for structured AMR that is applicable in a parallel, in situ context. Our key strategy is the incorporation of an multi-phase AMR-aware communication pattern that synchronizes connectivity information across the AMR hierarchy. In addition, we distil our methodology to a generic framework within the Combo AMR infrastructure, making connected component detection services available for many existing applications. We demonstrate our methods efficacy by showing its ability to detect ice calving events in real time within the real-world BISICLES ice sheet modelling code. Results show up to a 6.8x speedup of our algorithm over the existing specialized BISICLES algorithm. We also show scalability results for our method up to 4,096 cores using a parallel Combo-based benchmark.

Progress in fast, accurate multi-scale climate simulations

Abstract We present a survey of physical and computational techniques that have the potential to contribute to the next generation of high-fidelity, multi-scale climate simulations. Examples of the climate science problems that can be investigated with more depth with these computational improvements include the capture of remote forcings of localized hydrological extreme events, an accurate representation of cloud features over a range of spatial and temporal scales, and parallel, large ensembles of simulations to more effectively explore model sensitivities and un- certainties. Numerical techniques, such as adaptive mesh refinement, implicit time integration, and separate treatment of fast physical time scales are enabling improved accuracy and fidelity in simulation of dynamics and allowing more complete representations of climate features at the global scale. At the same time, partnerships with computer science teams have focused on taking advantage of evolving computer architectures such as many-core processors and GPUs. As a result, approaches which were previously considered prohibitively costly have become both more efficient and scalable. In combination, progress in these three critical areas is poised to transform climate modeling in the coming decades. These topics have been presented within a workshop titled, “Numerical and Computational Developments to Advance Multiscale Earth System Models (MSESM ‘15),” as part of the International Conference on Computational Sciences, Reykjavik, Iceland, June 1-3, 2015.

A moving control volume approach to computing hydrodynamic forces and torques on immersed bodies

Abstract We present a moving control volume (CV) approach to computing hydrodynamic forces and torques on complex geometries. The method requires surface and volumetric integrals over a simple and regular Cartesian box that moves with an arbitrary velocity to enclose the body at all times. The moving box is aligned with Cartesian grid faces, which makes the integral evaluation straightforward in an immersed boundary (IB) framework. Discontinuous and noisy derivatives of velocity and pressure at the fluid–structure interface are avoided and far-field (smooth) velocity and pressure information is used. We re-visit the approach to compute hydrodynamic forces and torques through force/torque balance equations in a Lagrangian frame that some of us took in a prior work (Bhalla et al., 2013 [13] ). We prove the equivalence of the two approaches for IB methods, thanks to the use of Peskins delta functions. Both approaches are able to suppress spurious force oscillations and are in excellent agreement, as expected theoretically. Test cases ranging from Stokes to high Reynolds number regimes are considered. We discuss regridding issues for the moving CV method in an adaptive mesh refinement (AMR) context. The proposed moving CV method is not limited to a specific IB method and can also be used, for example, with embedded boundary methods.

An intercomparison of GCM and RCM dynamical downscaling for characterizing the hydroclimatology of California and Nevada

© 2018 American Meteorological Society. Dynamical downscaling is a widely used technique to properly capture regional surface heterogeneities that shape the local hydroclimatology. However, in the context of dynamical downscaling, the impacts on simulation fidelity have not been comprehensively evaluated across many user-specified factors, including the refinements of model horizontal resolution, large-scale forcing datasets, and dynamical cores. Two global-to-regional downscaling methods are used to assess these: specifically, the variable-resolution Community Earth System Model (VR-CESM) and the Weather Research and Forecasting (WRF) Model with horizontal resolutions of 28, 14, and 7 km. The modeling strategies are assessed by comparing the VR-CESM and WRF simulations with consistent physical parameterizations and grid domains. Two groups of WRF Models are driven by either the NCEP reanalysis dataset (WRF_NCEP) or VR-CESM7 results (WRF_VRCESM) to evaluate the effects of large-scale forcing datasets. The simulated hydroclimatologies are compared with reference datasets for key properties including total precipitation, snow cover, snow water equivalent (SWE), and surface temperature. The large-scale forcing datasets are critical to the WRF simulations of total precipitation but not surface temperature, controlled by the wind field and atmospheric moisture transport at the ocean boundary. No significant benefit is found in the regional average simulated hydroclimatology by increasing horizontal resolution refinement from 28 to 7 km, probably due to the systematic biases from the diagnostic treatment of rainfall and snowfall in the microphysics scheme. The choice of dynamical core has little impact on total precipitation but significantly determines simulated surface temperature, which is affected by the snow-albedo feedback in winter and soil moisture estimations in summer.

Sensitivity of Mountain Hydroclimate Simulations in Variable‐Resolution CESM to Microphysics and Horizontal Resolution

Author(s): Rhoades, AM; Ullrich, PA; Zarzycki, CM; Johansen, H; Margulis, SA; Morrison, H; Xu, Z; Collins, WD | Abstract:

Recent progress and review of issues related to Physics Dynamics Coupling in geophysical models

AbstractNumerical weather, climate, or Earth system models involve the coupling of components. At a broad level, these components can be classified as the resolved fluid dynamics, unresolved fluid ...Geophysical models of the atmosphere and ocean invariably involve parameterizations. These represent two distinct areas: a) Subgrid processes which the model cannot (yet) resolve, due to its discrete resolution, and b) sources in the equation, due to radiation for example. Hence coupling between these physics parameterizations and the resolved fluid dynamics and also between the dynamics of the different fluids in the system (air and water) is necessary. This coupling is an important aspect of geophysical models. However, often model development is strictly segregated into either physics or dynamics. Hence, this area has many more unanswered questions than in-depth understanding. Furthermore, recent developments in the design of dynamical cores (e.g. significant increase of resolution, move to non-hydrostatic equation sets etc), extended process physics (e.g. prognostic micro physics, 3D turbulence, non-vertical radiation etc) and predicted future changes of the computational infrastructure (e.g. Exascale with its need for task parallelism, data locality and asynchronous time stepping for example) is adding even more complexity and new questions. This paper reviews the state-of-the-art of the physics-dynamics coupling in geophysical models, surveys the analysis techniques, and points out the open questions in this research field.