Per-Gunnar Martinsson

Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets. This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed—either explicitly or implicitly—to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, robustness, and/or speed. These claims are supported by extensive numerical experiments and a detailed error analysis. The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the

Randomized algorithms for the low-rank approximation of matrices

k

On the Compression of Low Rank Matrices

dominant components of the singular value decomposition of an

Fast direct solvers for integral equations in complex three-dimensional domains

m \times n

An Algorithm for the Principal Component Analysis of Large Data Sets

matrix. (i) For a dense input matrix, randomized algorithms require

A Fast Randomized Algorithm for Computing a Hierarchically Semiseparable Representation of a Matrix

\bigO(mn \log(k))

High-order accurate methods for Nyström discretization of integral equations on smooth curves in the plane

floating-point operations (flops) in contrast to

A direct solver for variable coefficient elliptic PDEs discretized via a composite spectral collocation method

\bigO(mnk)

A Direct Solver with

for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to

O(N)

\bigO(k)