Björn Gmeiner

Parallel multigrid on hierarchical hybrid grids: a performance study on current high performance computing clusters

This article studies the performance and scalability of a geometric multigrid solver implemented within the hierarchical hybrid grids (HHG) software package on current high performance computing clusters up to nearly 300,000 cores. HHG is based on unstructured tetrahedral finite elements that are regularly refined to obtain a block‐structured computational grid. One challenge is the parallel mesh generation from an unstructured input grid that roughly approximates a human head within a 3D magnetic resonance imaging data set. This grid is then regularly refined to create the HHG grid hierarchy. As test platforms, a BlueGene/P cluster located at Jülich supercomputing center and an Intel Xeon 5650 cluster located at the local computing center in Erlangen are chosen. To estimate the quality of our implementation and to predict runtime for the multigrid solver, a detailed performance and communication model is developed and used to evaluate the measured single node performance, as well as weak and strong scaling experiments on both clusters. Thus, for a given problem size, one can predict the number of compute nodes that minimize the overall runtime of the multigrid solver. Overall, HHG scales up to the full machines, where the biggest linear system solved on Jugene had more than one trillion unknowns. Copyright © 2012 John Wiley & Sons, Ltd.

Performance and Scalability of Hierarchical Hybrid Multigrid Solvers for Stokes Systems

In many applications involving incompressible fluid flow, the Stokes system plays an important role. Complex flow problems may require extremely fine resolutions, easily resulting in saddle-point problems with more than a trillion (

Optimization of the multigrid-convergence rate on semi-structured meshes by local Fourier analysis

10^{12}

Scheduling Massively Parallel Multigrid for Multilevel Monte Carlo Methods

) unknowns. Even on the most advanced supercomputers, the fast solution of such systems of equations is a highly nontrivial and challenging task. In this work we consider a realization of an iterative saddle-point solver which is based mathematically on the Schur-complement formulation of the pressure and algorithmically on the abstract concept of hierarchical hybrid grids. The design of our fast multigrid solver is guided by an innovative performance analysis for the computational kernels in combination with a quantification of the communication overhead. Excellent node performance and good scalability to almost a million parallel threads are demonstrated on different characteristic types of modern supercomputers.

LOCAL MASS-CORRECTIONS FOR CONTINUOUS PRESSURE APPROXIMATIONS OF INCOMPRESSIBLE FLOW ∗

In this paper a local Fourier analysis for multigrid methods on tetrahedral grids is presented. Different smoothers for the discretization of the Laplace operator by linear finite elements on such grids are analyzed. A four-color smoother is presented as an efficient choice for regular tetrahedral grids, whereas line and plane relaxations are needed for poorly shaped tetrahedra. A novel partitioning of the Fourier space is proposed to analyze the four-color smoother. Numerical test calculations validate the theoretical predictions. A multigrid method is constructed in a block-wise form, by using different smoothers and different numbers of pre- and post-smoothing steps in each tetrahedron of the coarsest grid of the domain. Some numerical experiments are presented to illustrate the efficiency of this multigrid algorithm.

Resilience for Massively Parallel Multigrid Solvers

The computational complexity of naive, sampling-based uncertainty quantification for 3D partial differential equations is extremely high. Multilevel approaches, such as multilevel Monte Carlo (MLMC), can reduce the complexity significantly when they are combined with a fast multigrid solver, but to exploit them fully in a parallel environment, sophisticated scheduling strategies are needed. We optimize the concurrent execution across the three layers of the MLMC method: parallelization across levels, across samples, and across the spatial grid. In a series of numerical tests, the influence on the overall performance of the “scalability window” of the multigrid solver (i.e., the range of processor numbers over which good parallel efficiency can be maintained) is illustrated. Different homogeneous and heterogeneous scheduling strategies are proposed and discussed. Finally, large 3D scaling experiments are carried out, including adaptivity.

Electromagnetic analysis of conductor track surface roughnesses from 1 GHz to 110 GHz

In this work, we discuss a family of finite element discretizations for the incompressible Stokes problem using continuous pressure approximations on simplicial meshes. We show that after a simple and cheap correction, the mass-fluxes obtained by the considered schemes preserve local conservation on dual cells without reducing the convergence order. This allows the direct coupling to vertex-centered finite volume discretizations of transport equations. Further, we can postprocess the mass fluxes independently for each dual box to obtain an elementwise conservative velocity approximation of optimal order that can be used in cell-centered finite volume or discontinuous Galerkin schemes. Numerical examples for stable and stabilized methods are given to support our theoretical findings. Moreover, we demonstrate the coupling to vertex- and cell-centered finite volume methods for advective transport.

Peta-Scale Hierarchical Hybrid Multigrid Using Hybrid Parallelization

Fault tolerant massively parallel multigrid methods for elliptic partial differential equations are a step towards resilient solvers. Here, we combine domain partitioning with geometric multigrid methods to obtain fast and fault-robust solvers for three-dimensional problems. The recovery strategy is based on the redundant storage of ghost values, as they are commonly used in distributed memory parallel programs. In the case of a fault, the redundant interface values can be easily recovered, while the lost inner unknowns are recomputed approximately with recovery algorithms using multigrid cycles for solving a local Dirichlet problem. Different strategies are compared and evaluated with respect to performance, computational cost, and speedup. Especially effective are asynchronous strategies combining global solves with accelerated local recovery. By this, multiple faults can be fully compensated with respect to both the number of iterations and run-time. For illustration, we use a state-of-the-art petascal...

Hybrid Parallel Multigrid Methods for Geodynamical Simulations

Conductor tracks comprise a frequency dependent attenuation of electromagnetic waves, since with increasing frequency the current flow is displaced to the near surface region due to the skin effect. Therefore, the effective length of the conductor is increased by the surface roughness, while its effective cross-section is decreased by current displacement, both leading to higher metallization loss. In this paper, surface topographies of typical conductor materials were recorded by confocal microscopy and rebuilt as 3D CAD models. Subsequent electromagnetic simulations reveal the influence due to roughness on high frequency characteristics for physical vapor deposited, thick film and photochemically etched microstrips.

Electromagnetic analysis of fringed microstrip lines on porosified LTCC

In this article we present a performance study of our finite element package Hierarchical Hybrid Grids (HHG) on current European supercomputers. HHG is designed to close the gap between the flexibility of finite elements and the efficiency of geometric multigrid by using a compromise between structured and unstructured grids. A coarse input finite element mesh is refined in a structured way, resulting in semi-structured meshes. Within this article we compare and analyze the efficiencies of the stencil-based code on those clusters.