Wim Vanroose

Hiding Global Communication Latency in the GMRES Algorithm on Massively Parallel Machines

In the generalized minimal residual method (GMRES), the global all-to-all communication required in each iteration for orthogonalization and normalization of the Krylov base vectors is becoming a performance bottleneck on massively parallel machines. Long latencies, system noise, and load imbalance cause these global reductions to become very costly global synchronizations. In this work, we propose the use of nonblocking or asynchronous global reductions to hide these global communication latencies by overlapping them with other communications and calculations. A pipelined variation of GMRES is presented in which the result of a global reduction is used only one or more iterations after the communication phase has started. This way, global synchronization is relaxed and scalability is much improved at the expense of some extra computations. The numerical instabilities that inevitably arise due to the typical monomial basis by powering the matrix are reduced and often annihilated by using Newton or Chebysh...

Hiding global synchronization latency in the preconditioned Conjugate Gradient algorithm

Scalability of Krylov subspace methods suffers from costly global synchronization steps that arise in dot-products and norm calculations on parallel machines. In this work, a modified preconditioned Conjugate Gradient (CG) method is presented that removes the costly global synchronization steps from the standard CG algorithm by only performing a single non-blocking reduction per iteration. This global communication phase can be overlapped by the matrix-vector product, which typically only requires local communication. The resulting algorithm will be referred to as pipelined CG. An alternative pipelined method, mathematically equivalent to the Conjugate Residual (CR) method that makes different trade-offs with regard to scalability and serial runtime is also considered. These methods are compared to a recently proposed asynchronous CG algorithm by Gropp. Extensive numerical experiments demonstrate the numerical stability of the methods. Moreover, it is shown that hiding the global synchronization step improves scalability on distributed memory machines using the message passing paradigm and leads to significant speedups compared to standard preconditioned CG.

Complete Photo-Induced Breakup of the H2 Molecule as a Probe of Molecular Electron Correlation

Despite decades of progress in quantum mechanics, electron correlation effects are still only partially understood. Experiments in which both electrons are ejected from an oriented hydrogen molecule by absorption of a single photon have recently demonstrated a puzzling phenomenon: The ejection pattern of the electrons depends sensitively on the bond distance between the two nuclei as they vibrate in their ground state. Here, we report a complete numerical solution of the Schrödinger equation for the double photoionization of H2. The results suggest that the distribution of photoelectrons emitted from aligned molecules reflects electron correlation effects that are purely molecular in origin.

Accuracy of Hybrid Lattice Boltzmann/Finite Difference Schemes for Reaction-Diffusion Systems

In this article we construct a hybrid model by spatially coupling a lattice Boltzmann model (LBM) to a finite difference discretization of the partial differential equation (PDE) for reaction-diffusion systems. Because the LBM has more variables (the particle distribution functions) than the PDE (only the particle density), we have a one-to-many mapping problem from the PDE to the LBM domain at the interface. We perform this mapping using either results from the Chapman–Enskog expansion or a pointwise iterative scheme that approximates these analytical relations numerically. Most importantly, we show that the global spatial discretization error of the hybrid model is one order less accurate than the local error made at the interface. We derive closed expressions for the spatial discretization error at steady state and verify them numerically for several examples on the one-dimensional domain.

Local Fourier analysis of the complex shifted Laplacian preconditioner for Helmholtz problems

SUMMARY In this paper, we solve the Helmholtz equation with multigrid preconditioned Krylov subspace methods. The class of shifted Laplacian preconditioners is known to significantly speed up Krylov convergence. However, these preconditioners have a parameter β∈R, a measure of the complex shift. Because of contradictory requirements for the multigrid and Krylov convergence, the choice of this shift parameter can be a bottleneck in applying the method. In this paper, we propose a wavenumber-dependent minimal complex shift parameter, which is predicted by a rigorous k-grid local Fourier analysis (LFA) of the multigrid scheme. We claim that, given any (regionally constant) wavenumber, this minimal complex shift parameter provides the reader with a parameter choice that leads to efficient Krylov convergence. Numerical experiments in one and two spatial dimensions validate the theoretical results. It appears that the proposed complex shift is both the minimal requirement for a multigrid V-cycle to converge and being near optimal in terms of Krylov iteration count. Copyright

Towards mechanistic models of plant organ growth

Modelling and simulation are increasingly used as tools in the study of plant growth and developmental processes. By formulating experimentally obtained knowledge as a system of interacting mathematical equations, it becomes feasible for biologists to gain a mechanistic understanding of the complex behaviour of biological systems. In this review, the modelling tools that are currently available and the progress that has been made to model plant development, based on experimental knowledge, are described. In terms of implementation, it is argued that, for the modelling of plant organ growth, the cellular level should form the cornerstone. It integrates the output of molecular regulatory networks to two processes, cell division and cell expansion, that drive growth and development of the organ. In turn, these cellular processes are controlled at the molecular level by hormone signalling. Therefore, combining a cellular modelling framework with regulatory modules for the regulation of cell division, expansion, and hormone signalling could form the basis of a functional organ growth simulation model. The current state of progress towards this aim is that the regulation of the cell cycle and hormone transport have been modelled extensively and these modules could be integrated. However, much less progress has been made on the modelling of cell expansion, which urgently needs to be addressed. A limitation of the current generation models is that they are largely qualitative. The possibilities to characterize existing and future models more quantitatively will be discussed. Together with experimental methods to measure crucial model parameters, these modelling techniques provide a basis to develop a Systems Biology approach to gain a fundamental insight into the relationship between gene function and whole organ behaviour.

On the indefinite Helmholtz equation: Complex stretched absorbing boundary layers, iterative analysis, and preconditioning

This paper studies and analyzes a preconditioned Krylov solver for Helmholtz problems that are formulated with absorbing boundary layers based on complex coordinate stretching. The preconditioner problem is a Helmholtz problem where not only the coordinates in the absorbing layer have an imaginary part, but also the coordinates in the interior region. This results into a preconditioner problem that is invertible with a multigrid cycle. We give a numerical analysis based on the eigenvalues and evaluate the performance with several numerical experiments. The method is an alternative to the complex shifted Laplacian and it gives a comparable performance for the studied model problems.

Mesoscale analysis of the equation-free constrained runs initialization scheme

In this article, we analyze the stability, convergence, and accuracy of the constrained runs initialization scheme for a mesoscale lattice Boltzmann model (LBM). This type of initialization scheme was proposed by Gear and Kevrekidis in [J. Sci. Comput., 25 (2005), pp. 17–28] in the context of both singularly perturbed ordinary differential equations and equation-free computing. It maps the given macroscopic initial variables to the higher-dimensional space of microscopic/mesoscopic variables. The scheme performs short runs with the microscopic/mesoscopic simulator and resets the macroscopic variables (typically the lower order moments of the microscopic/mesoscopic variables), while leaving the higher order moments unchanged. We use the LBM Bhatnagar–Gross–Krook (BGK) model for one-dimensional reaction-diffusion systems as the microscopic/mesoscopic model. For such systems, we prove that the constrained runs scheme is unconditionally stable and that it converges to an approximation of the slaved state, i.e...

Improving the arithmetic intensity of multigrid with the help of polynomial smoothers

SUMMARY The basic building blocks of a classic multigrid algorithm, which are essentially stencil computations, all have a low ratio of executed floating point operations per byte fetched from memory. This important ratio can be identified as the arithmetic intensity. Applications with a low arithmetic intensity are typically bounded by memory traffic and achieve only a small percentage of the theoretical peak performance of the underlying hardware. We propose a polynomial Chebyshev smoother, which we implement using cache-aware tiling, to increase the arithmetic intensity of a multigrid V-cycle. This tiling approach involves a trade-off between redundant computations and cache misses. Unlike common conception, we observe optimal performance for higher degrees of the smoother. The higher-degree polynomial Chebyshev smoother can be used to smooth more than just the upper half of the error frequencies, leading to better V-cycle convergence rates. Smoothing more than the upper half of the error spectrum allows a more aggressive coarsening approach where some levels in the multigrid hierarchy are skipped. Copyright

Analyzing the wave number dependency of the convergence rate of a multigrid preconditioned Krylov method for the Helmholtz equation with an absorbing layer

This paper analyzes the Krylov convergence rate of a Helmholtz problem preconditioned with Multigrid. The multigrid method is applied to the Helmholtz problem formulated on a complex contour and uses GMRES as a smoother substitute at each level. A one-dimensional model is analyzed both in a continuous and discrete way. It is shown that the Krylov convergence rate of the continuous problem is independent of the wave number. The discrete problem, however, can deviate significantly from this bound due to a pitchfork in the spectrum. It is further shown in numerical experiments that the convergence rate of the Krylov method approaches the continuous bound as the grid distance