Henk A. van der Vorst

Templates for the solution of algebraic eigenvalue problems: a practical guide

List of symbols and acronyms List of iterative algorithm templates List of direct algorithms List of figures List of tables 1: Introduction 2: A brief tour of Eigenproblems 3: An introduction to iterative projection methods 4: Hermitian Eigenvalue problems 5: Generalized Hermitian Eigenvalue problems 6: Singular Value Decomposition 7: Non-Hermitian Eigenvalue problems 8: Generalized Non-Hermitian Eigenvalue problems 9: Nonlinear Eigenvalue problems 10: Common issues 11: Preconditioning techniques Appendix: of things not treated Bibliography Index .

Solving Linear Systems on Vector and Shared Memory Computers

Vector and parallel processing overview of current high-performance computers implementation details and overhead performance - analysis, modeling and measurements building blocks in linear algebra direct solution of sparse linear systems iterative solution of sparse linear systems. Appendices: acquiring mathematical software information on various high-performance computers level 1,2, and 3 BLAS quick reference operation counts for various BLAS and decompositions.

Approximate solutions and eigenvalue bounds from Krylov subspaces

Approximations to the solution of a large sparse symmetric system of equations are considered. The conjugate gradient and minimum residual approximations are studied without reference to their computation. Several different bases for the associated Krylov subspace are used, including the usual Lanczos basis. The zeros of the iteration polynomial for the minimum residual approximation (harmonic Ritz values) are characterized in several ways and, in addition, attractive convergence properties are established. The connection of these harmonic Ritz values to Lehmanns optimal intervals for eigenvalues of the original matrix appears to be new.

Eigenvalue computation in the 20th century

This paper sketches the main research developments in the area of computational methods for eigenvalue problems during the 20th century. The earliest of such methods dates back to work of Jacobi in the middle of the 19th century. Since computing eigenvalues and vectors is essentially more complicated than solving linear systems, it is not surprising that highly significant developments in this area started with the introduction of electronic computers around 1950. In the early decades of this century, however, important theoretical developments had been made from which computational techniques could grow. Research in this area of numerical linear algebra is very active, since there is a heavy demand for solving complicated problems associated with stability and perturbation analysis for practical applications. For standard problems, powerful tools are available, but there still remain many open problems. It is the intention of this contribution to sketch the main developments of this century, especially as they relate to one another, and to give an impression of the state of the art at the turn of our century.

Parallel Numerical Linear Algebra

We survey general techniques and open problems in numerical linear algebra on parallel architectures. We first discuss basic principles of parallel processing, describing the costs of basic operations on parallel machines, including general principles for constructing efficient algorithms. We illustrate these principles using current architectures and software systems, and by showing how one would implement matrix multiplication. Then, we present direct and iterative algorithms for solving linear systems of equations, linear least squares problems, the symmetric eigenvalue problem, the nonsymmetric eigenvalue problem, the singular value decomposition, and generalizations of these to two matrices. We consider dense, band and sparse matrices.

Developments and trends in the parallel solution of linear systems

Abstract In this review paper, we consider some important developments and trends in algorithm design for the solution of linear systems concentrating on aspects that involve the exploitation of parallelism. We briefly discuss the solution of dense linear systems, before studying the solution of sparse equations by direct and iterative methods. We consider preconditioning techniques for iterative solvers and discuss some of the present research issues in this field.

Maintaining convergence properties of BiCGstab methods in finite precision arithmetic

It is well-known that Bi-CG can be adapted so that hybrid methods with computational complexity almost similar to Bi-CG can be constructed, in which it is attempted to further improve the convergence behavior. In this paper we will study the class of BiCGstab methods.In many applications, the speed of convergence of these methods appears to be determined mainly by the incorporated Bi-CG process, and the problem is that the Bi-CG iteration coefficients have to be determined from the BiCGstab process. We will focus our attention to the accuracy of these Bi-CG coefficients, and how rounding errors may affect the speed of convergence of the BiCGstab methods. We will propose a strategy for a more stable determination of the Bi-CG iteration coefficients and by experiments we will show that this indeed may lead to faster convergence.

Approximate and Incomplete Factorizations

In this chapter, we give a brief overview of a particular class of preconditioners known as incomplete factorizations. They can be thought of as approximating the exact LU factorization of a given matrix A (e.g., computed via Gaussian elimination) by disallowing certain fill-ins. As opposed to other PDE-based preconditioners such as multigrid and domain decomposition, this class of preconditioners is primarily algebraic in nature and can in principle be applied to general sparse matrices. When applied to PDE problems, they are usually not optimal in the sense that the condition number of the preconditioned system grows as the mesh size h is reduced, although usually at a slower rate than for the unpreconditioned system. On the other hand, they are often quite robust with respect to other more algebraic features of the problem such as rough and anisotropic coefficients and strong convection terms.

Generalized conjugate gradient squared

The Conjugate Gradient Squared (CGS) is an iterative method for solving nonsymmetric linear systems of equations. However, during the iteration large residual norms may appear, which may lead to inaccurate approximate solutions or may even deteriorate the convergence rate. Instead of squaring the Bi-CG polynomial as in CGS, we propose to consider products of two nearby Bi-CG polynomials which leads to generalized CGS methods, of which CGS is just a particular case. This approach allows the construction of methods that converge less irregularly than CGS and that improve on other convergence properties as well. Here, we are interested in a property that got less attention in literature: we concentrate on retaining the excellent approximation qualities of CGS with respect to components of the solution in the direction of eigenvectors associated with extreme eigenvalues. This property seems to be important in connection with Newtons scheme for nonlinear equations: our numerical experiments show that the number of Newton steps may decrease significantly when using a generalized CGS method as linear solver for the Newton correction equations.

Parallel incomplete factorizations with pseudo­overlapped subdomains

Abstract We address the hard question of efficient use on parallel platforms, of incomplete factorization preconditioning techniques for solving large and sparse linear systems by Krylov subspace methods. A novel parallelization strategy based on pseudo-overlapped subdomains is explored. This results in efficient parallelizable preconditioners. Numerical results give evidence that high performance can be achieved.