Jennifer Pestana

Natural preconditioning and iterative methods for saddle point systems

The solution of quadratic or locally quadratic extremum problems subject to linear(ized) constraints gives rise to linear systems in saddle point form. This is true whether in the continuous or the discrete setting, so saddle point systems arising from the discretization of partial differential equation problems, such as those describing electromagnetic problems or incompressible flow, lead to equations with this structure, as do, for example, interior point methods and the sequential quadratic programming approach to nonlinear optimization. This survey concerns iterative solution methods for these problems and, in particular, shows how the problem formulation leads to natural preconditioners which guarantee a fast rate of convergence of the relevant iterative methods. These preconditioners are related to the original extremum problem and their effectiveness---in terms of rapidity of convergence---is established here via a proof of general bounds on the eigenvalues of the preconditioned saddle point matri...

Fast multipole preconditioners for sparse matrices arising from elliptic equations

Among optimal hierarchical algorithms for the computational solution of elliptic problems, the fast multipole method (FMM) stands out for its adaptability to emerging architectures, having high arithmetic intensity, tunable accuracy, and relaxable global synchronization requirements. We demonstrate that, beyond its traditional use as a solver in problems for which explicit free-space kernel representations are available, the FMM has applicability as a preconditioner in finite domain elliptic boundary value problems, by equipping it with boundary integral capability for satisfying conditions at finite boundaries and by wrapping it in a Krylov method for extensibility to more general operators. Here, we do not discuss the well developed applications of FMM to implement matrix-vector multiplications within Krylov solvers of boundary element methods. Instead, we propose using FMM for the volume-to-volume contribution of inhomogeneous Poisson-like problems, where the boundary integral is a small part of the overall computation. Our method may be used to precondition sparse matrices arising from finite difference/element discretizations, and can handle a broader range of scientific applications. It is capable of algebraic convergence rates down to the truncation error of the discretized PDE comparable to those of multigrid methods, and it offers potentially superior multicore and distributed memory scalability properties on commodity architecture supercomputers. Compared with other methods exploiting the low-rank character of off-diagonal blocks of the dense resolvent operator, FMM-preconditioned Krylov iteration may reduce the amount of communication because it is matrix-free and exploits the tree structure of FMM. We describe our tests in reproducible detail with freely available codes and outline directions for further extensibility.

A preconditioned MINRES method for nonsymmetric Toeplitz matrices

Circulant preconditioning for symmetric Toeplitz linear systems is well established; theoretical guarantees of fast convergence for the conjugate gradient method are descriptive of the convergence seen in computations. This has led to robust and highly efficient solvers based on use of the fast Fourier transform exactly as originally envisaged in [G. Strang, Stud. Appl. Math., 74 (1986), pp. 171--176]. For nonsymmetric systems, the lack of generally descriptive convergence theory for most iterative methods of Krylov type has provided a barrier to such a comprehensive guarantee, though several methods have been proposed and some analysis of performance with the normal equations is available. In this paper, by the simple device of reordering, we rigorously establish a circulant preconditioned short recurrence Krylov subspace iterative method of minimum residual type for nonsymmetric (and possibly highly nonnormal) Toeplitz systems. Convergence estimates similar to those in the symmetric case are established.

Some observations on weighted GMRES

We investigate the convergence of the weighted GMRES method for solving linear systems. Two different weighting variants are compared with unweighted GMRES for three model problems, giving a phenomenological explanation of cases where weighting improves convergence, and a case where weighting has no effect on the convergence. We also present a new alternative implementation of the weighted Arnoldi algorithm which under known circumstances will be favourable in terms of computational complexity. These implementations of weighted GMRES are compared for a large number of examples. We find that weighted GMRES may outperform unweighted GMRES for some problems, but more often this method is not competitive with other Krylov subspace methods like GMRES with deflated restarting or BICGSTAB, in particular when a preconditioner is used.

On the eigenvalues and eigenvectors of block triangular preconditioned block matrices

Block lower triangular matrices and block upper triangular matrices are popular preconditioners for

On the choice of preconditioner for minimum residual methods for non-Hermitian matrices

2\times 2

Combination preconditioning of saddle point systems for positive definiteness

block matrices. In this note we show that a block lower triangular preconditioner gives the same spectrum as a block upper triangular preconditioner and that the eigenvectors of the two preconditioned matrices are related.

Null-Space Preconditioners for Saddle Point Systems

Abstract We consider the solution of left preconditioned linear systems P − 1 C x = P − 1 c , where P , C ∈ C n × n are non-Hermitian, c ∈ C n , and C , P , and P − 1 C are diagonalisable with spectra symmetric about the real line. We prove that, when P and C are self-adjoint with respect to the same Hermitian sesquilinear form, the convergence of a minimum residual method in a particular nonstandard inner product applied to the preconditioned linear system is bounded by a term that depends only on the spectrum of P − 1 C . The inner product is related to the spectral decomposition of P . When P is self-adjoint with respect to a nearby Hermitian sesquilinear form to C , the convergence of a minimum residual method in this nonstandard inner product applied to the preconditioned linear system is bounded by a term involving the eigenvalues of P − 1 C and a constant factor. The size of this factor is related to the nearness of the Hermitian sesquilinear forms. Numerical experiments indicate that for certain matrices eigenvalue-dependent convergence is observed both for the nonstandard method and for standard GMRES.

Block preconditioners for linear systems arising from multiscale collocation with compactly supported RBFs

Amongst recent contributions to preconditioning methods for saddle point systems, standard iterative methods in nonstandard inner products have been usefully employed. Krzyzanowski ( ˙ Numer. Linear Algebra Appl. 2011; 18:123–140) identified a two-parameter family of preconditioners in this context and Stoll and Wathen (SIAM J. Matrix Anal. Appl. 2008; 30:582–608) introduced combination preconditioning, where two preconditioners, self-adjoint with respect to different inner products, can lead to further preconditioners and associated bilinear forms or inner products. Preconditioners that render the preconditioned saddle point matrix nonsymmetric but self-adjoint with respect to a nonstandard inner product always allow a MINREStype method (W-PMINRES) to be applied in the relevant inner product. If the preconditioned matrix is also positive definite with respect to the inner product a more efficient CG-like method (W-PCG) can be reliably used. We establish eigenvalue expressions for Krzyzanowski preconditioners and show that for a specific ˙ choice of parameters, although the Krzyzanowski preconditioned saddle point matrix is self-adjoint with ˙ respect to an inner product, it is never positive definite. We provide explicit expressions for the combination of certain preconditioners and prove the rather counterintuitive result that the combination of two specific preconditioners for which only W-PMINRES can be reliably used leads to a preconditioner for which, for certain parameter choices, W-PCG is reliably applicable. That is, combining two indefinite preconditioners can lead to a positive definite preconditioner. This combination preconditioner outperforms either of the two preconditioners from which it is formed for a number of test problems.

EFFICIENT BLOCK PRECONDITIONING FOR A C 1 FINITE ELEMENT DISCRETIZATION OF THE DIRICHLET BIHARMONIC PROBLEM

The null-space method is a technique that has been used for many years to reduce a saddle point system to a smaller, easier to solve, symmetric positive definite system. This method can be understood as a block factorization of the system. Here we explore the use of preconditioners based on incomplete versions of a particular null-space factorization and compare their performance with the equivalent Schur complement based preconditioners. We also describe how to apply the nonsymmetric preconditioners proposed using the conjugate gradient method (CG) with a nonstandard inner product. This requires an exact solve with the (1,1) block, and the resulting algorithm is applicable in other cases where Bramble--Pasciak CG is used. We verify the efficiency of the newly proposed preconditioners on a number of test cases from a range of applications.