Christopher C. Paige

LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares

An iterative method is given for solving Ax ~ffi b and minU Ax b 112, where the matrix A is large and sparse. The method is based on the bidiagonalization procedure of Golub and Kahan. It is analytically equivalent to the standard method of conjugate gradients, but possesses more favorable numerical properties. Reliable stopping criteria are derived, along with estimates of standard errors for x and the condition number of A. These are used in the FORTRAN implementation of the method, subroutine LSQR. Numerical tests are described comparing I~QR with several other conjugate-gradient algorithms, indicating that I~QR is the most reliable algorithm when A is ill-conditioned.

Solution of Sparse Indefinite Systems of Linear Equations

The method of conjugate gradients for solving systems of linear equations with a symmetric positive definite matrix A is given as a logical development of the Lanczos algorithm for tridiagonalizing...

Towards a Generalized Singular Value Decomposition

We suggest a form for, and give a constructive derivation of, the generalized singular value decomposition of any two matrices having the same number of columns. We outline its desirable characteristics and compare it to an earlier suggestion by Van Loan [SIAM J. Numer. Anal., 13 (1976), pp. 76–83]. The present form largely follows from the work of Van Loan, but is slightly more general and computationally more amenable than that in the paper cited. We also prove a useful extension of a theorem of Stewart [SIAM Rev. 19 (1977), pp. 634–662] on unitary decompositions of submatrices of a unitary matrix.

Properties of numerical algorithms related to computing controllability

The numerical properties of some methods for computing controllability are used in an expository way to motivate a wider understanding of numerical computations. In particular, the numerical rank of a matrix, the numerical stability of algorithms, the sensitivity of problems, and the scaling of problems are discussed. A numerically stable algorithm is given for computing controllability, but it is pointed out that a measure of the distance of the given system from the nearest uncontrollable system would be more useful, and this appears to be an open computational problem.

Approximate solutions and eigenvalue bounds from Krylov subspaces

Approximations to the solution of a large sparse symmetric system of equations are considered. The conjugate gradient and minimum residual approximations are studied without reference to their computation. Several different bases for the associated Krylov subspace are used, including the usual Lanczos basis. The zeros of the iteration polynomial for the minimum residual approximation (harmonic Ritz values) are characterized in several ways and, in addition, attractive convergence properties are established. The connection of these harmonic Ritz values to Lehmanns optimal intervals for eigenvalues of the original matrix appears to be new.

A Schur decomposition for Hamiltonian matrices

Abstract A Schur-type decomposition for Hamiltonian matrices is given that relies on unitary symplectic similarity transformations. These transformations preserve the Hamiltonian structure and are numerically stable, making them ideal for analysis and computation. Using this decomposition and a special singular-value decomposition for unitary symplectic matrices, a canonical reduction of the algebraic Riccati equation is obtained which sheds light on the sensitivity of the nonnegative definite solution. After presenting some real decompositions for real Hamiltonian matrices, we look into the possibility of an orthogonal symplectic version of the QR algorithm suitable for Hamiltonian matrices. A finite-step initial reduction to a Hessenberg-type canonical form is presented. However, no extension of the Francis implicit-shift technique was found, and reasons for the difficulty are given.

Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem

Eigenvalues and eigenvectors of a large sparse symmetric matrix A can be found accurately and often very quickly using the Lanczos algorithm without reorthogonalization. The algorithm gives essentially correct information on the eigensystem of A, although it does not necessarily give the correct multiplicity of multiple, or even single, eigenvalues. It is straightforward to determine a useful bound on the accuracy of every eigenvalue given by the algorithm. The initial behavior of the algorithm is surprisingly good: it produces vectors spanning the Krylov subspace of a matrix very close to A until this subspace contains an exact eigenvector of a matrix very close to A, and up to this point the effective behavior of the algorithm for the eigenproblem is very like that of the Lanczos algorithm using full reorthogonalization. This helps to explain the remarkable behavior of the basic Lanczos algorithm.

History and generality of the CS decomposition

Abstract It is almost a quarter of a century since Chandler Davis and William Kahan brought together the key ideas of what Stewart later completed and defined to be the CS decomposition (CSD) of a partitioned unitary matrix. This paper outlines some germane points in the history of the CSD, pointing out the contributions of Jordan, of Davis and Kahan, and of Stewart, and the relationship of the CSD to the “direct rotation” of Davis and Kato. The paper provides an easy to memorize, constructive proof of the CSD, reviews one of its important uses, and suggests a motivation for the CSD which emphasizes how generally useful it is. It shows the relation between the CSD and generalized singular value decompositions, and points out some useful nullity properties one form of the CSD trivially reveals. Finally it shows how, via the QR factorization, the CSD can be used to obtain interesting results for partitioned nonsingular matrices. We suggest the CSD be taught in its most general form with no restrictions on the two by two partition, and initially with no mention of angles between subspaces.

Loss and recapture of orthogonality in the modified Gram-Schmidt algorithm

This paper arose from a fascinating observation, apparently by Charles Sheffield, and relayed to us by Gene Golub, that the QR factorization of an

Editorial: Total Least Squares and Errors-in-variables Modeling

m \times n