Erich Kaltofen

On Wiedemann's Method of Solving Sparse Linear Systems

Douglas Wiedemann’s (1986) landmark approach to solving sparse linear systems over finite fields provides the symbolic counterpart to non-combinatorial numerical methods for solving sparse linear systems, such as the Lanczos or conjugate gradient method (see Golub and van Loan (1983)). The problem is to solve a sparse linear system, when the individual entries lie in a generic field, and the only operations possible are field arithmetic; the solution is to be exact. Such is the situation, for instance, if one works in a finite field. Wiedemann bases his approach on Krylov subspaces, but projects further to a sequence of individual field elements. By making a link to the Berlekamp/Massey problem from coding theory — the coordinate recurrences — and by using randomization an algorithm is obtained with the following property. On input of an n×n coefficient matrix A given by a so-called black box, which is a program that can multiply the matrix by a vector (see Figure 1), and of a vector b, the algorithm finds, with high probability in case the system is solvable, a random solution vector x with Ax = b. It is assumed that the field has sufficiently many elements, say no less than 50n log(n), otherwise one goes to a finite algebraic extension. The complexity of the method is in the general singular case O(n log(n)) calls to the black box for A and an additional O(n log(n)) field arithmetic operations. Note that the black box model for matrix sparsity is a significant abstraction. For a matrix that has an abundance of zero entries, multiplying the matrix by a vector may cost no more than O(n) field operations, in which case the algorithm becomes almost quadratic. However, the model also applies to structured matrices with few or no zero entries, such as Toeplitzand Vandermonde-like matrices, or matrices that correspond to resultants (Canny

Computing with polynomials given byblack boxes for their evaluations: Greatest common divisors, factorization, separation of numerators and denominators

Algorithms are developed that adopt a novel implicit representation for multivariate polynomials and rational functions with rational coefficients, that of black boxes for their evaluation. We show that within this representation the polynomial greatest common divisor and factorization problems, as well as the problem of extracting the numerator and denominator of a rational function, can all be solved in random polynomial-time. Since we can convert black boxes efficiently to sparse format, problems with sparse solutions, e.g., sparse polynomial factorization and sparse multivariate rational function interpolation, are also in random polynomial time. Moreover, the black box representation is one of the most space efficient implicit representations that we know. Therefore, the output programs can be easily distributed over a network of processors for further manipulation, such as sparse interpolation.

Improved Sparse Multivariate Polynomial Interpolation Algorithms

We consider the problem of interpolating sparse multivariate polynomials from their values. We discuss two algorithms for sparse interpolation, one due to Ben-Or and Tiwari (1988) and the other due to Zippel (1988). We present efficient algorithms for finding the rank of certain special Toeplitz systems arising in the Ben-Or and Tiwari algorithm and for solving transposed Vandermonde systems of equations, the use of which greatly improves the time complexities of the two interpolation algorithms.

Polynomial-Time Reductions from Multivariate to Bi- and Univariate Integral Polynomial Factorization

Consider a polynomial f with an arbitrary but fixed number of variables and with integral coefficients. We present an algorithm which reduces the problem of finding the irreducible factors of f in polynomial-time in the total degree of f and the coefficient lengths of f to factoring a univariate integral polynomial. Together with A. Lenstra’s,.H. Lenstra’s and L. Lovasz’ polynomial-time factorization algorithm for univariate integral polynomials [Math. Ann., 261 (1982), pp. 515–534] this algorithm implies the following theorem. Factoring an integral polynomial with a fixed number of variables into irreducibles, except for the constant factors, can be accomplished in deterministic polynomial-time in the total degree and the size of its coefficients. Our algorithm can be generalized to factoring multivariate polynomials with coefficients in algebraic number fields and finite fields in polynomial-time. We also present a different algorithm, based on an effective version of a Hilbert Irreducibility Theorem, w...

Analysis of Coppersmith's block Wiedemann algorithm for the parallel solution of sparse linear systems

By using projections by a block of vectors in place of a single vector it is possible to parallelize the outer loop of iterative methods for solving sparse linear systems. We analyze such a scheme proposed by Coppersmith for Wiedemanns coordinate recurrence algorithm, which is based in part on the Krylov subspace approach. We prove that by use of certain randomizations on the input system the parallel speed up is roughly by the number of vectors in the blocks when using as many processors. Our analysis is valid for fields of entries that have sufficiently large cardinality. Our analysis also deals with an arising subproblem of solving a singular block Toeplitz system by use of the theory of Toeplitz-like matrices

Polynomial Factorization 1987-1991

Algorithms invented in the past 25 years make it possible on a computer to efficiently factor a polynomial in one, several, or many variables with coefficients from a certain field, such as a finite field or the rational, real, or complex numbers. I have surveyed work up to 1986 in the papers (Kaltofen 1982 and 1990a). This article discusses important developments of the past five years; I also take a fresh perspective of some older results. Although a conscientious effort has been made to cover (at least by citation) the significant contributions of that period, omissions are likely, which I ask to be kindly brought to my attention. Three parameters partition the factorization problem: first, the mathematical nature and computational representation of the coefficient domains of the input polynomial, second, that of the irreducible factors, and, third, the representation of the input polynomial and the sought irreducible factors, which depends not only on the degree and number of variables but also on properties such as sparsity. Say, for instance, that a bivariate polynomial with rational coefficients is to be factored into irreducible polynomials with real coefficients. The input polynomial as well as the factors may be represented by lists of monomials, that is terms and their corresponding non-zero coefficients. For the rational input the coefficients can be just fractions of two long integers, but the representation of the real coefficients for the factors is less standardized. One choice represents a real algebraic number by its rational minimum polynomial and an isolating interval with rational boundaries (Collins 1975), while another uses a rational linear relation of powers of a complex algebraic number that is universal for all coefficients of a single factor (Kaltofen 1990b). The organization of this survey is governed by these distinguishing problem specifications. We first discuss the “classical univariate problems” of factoring a polynomial

Greatest common divisors of polynomials given by straight-line programs

Algorithms on multivariate polynomials represented by straight-line programs are developed. First, it is shown that most algebraic algorithms can be probabilistically applied to data that are given by a straight-line computation. Testing such rational numeric data for zero, for instance, is facilitated by random evaluations modulo random prime numbers. Then, auxiliary algorithms that determine the coefficients of a multivariate polynomial in a single variable are constructed. The first main result is an algorithm that produces the greatest common divisor of the input polynomials, all in straight-line representation. The second result shows how to find a straight-line program for the reduced numerator and denominator from one for the corresponding rational function. Both the algorithm for that construction and the greatest common divisor algorithm are in random polynomial time for the usual coefficient fields and output a straight-line program, which with controllably high probability correctly determines the requested answer. The running times are polynomial functions in the binary input size, the input degrees as unary numbers, and the logarithm of the inverse of the failure probability. The algorithm for straight-line programs for the numerators and denominators of rational functions implies that every degree-bounded rational function can be computed fast in parallel, that is, in polynomial size and polylogarithmic depth.

Fast parallel absolute irreducibility testing

We present a fast parallel deterministic algorithm for testing multivariate integral polynomials for absolute irreducibility, that is irreducibility over the complex numbers. More precisely, we establish that the set of absolutely irreducible integral polynomials belongs to the complexity class NC of Boolean circuits of polynomial size and logarithmic depth. Therefore it also belongs to the class of sequentially polynomial-time problems. Our algorithm can be extended to compute in parallel one irreducible complex factor of a multivariate integral polynomial. However, the coeffieients of the computed factor are only represented modulo a not necessarily irreducible polynomial specifying a splitting field. A consequence of our algorithm is that multivariate polynomials over finite fields can be tested for absolute irreducibility in deterministic sequential polynomial time in the size of the input. We also obtain a sharp bound for the last prime p for which, when taking an absolute irreducible integral polynomial modulo p, the polynomials irreducibility in the algebraic closure of the finite field of order p is not preserved.

Processor efficient parallel solution of linear systems over an abstract field

Parallel randomized algorithms are presented that solve n-dimensional systems of linear equations and compute inverses of n × n non-singular matrices over a field in O((log n)) time, where each time unit represents an arithmetic operation in the field generated by the matrix entries. The algorithms utilize within a O(log n) factor as many processors as are needed to multiply two n × n matrices. The algorithms avoid zero divisions with controllably high probability provided the O(n) random elements used are selected uniformly from a sufficiently large set. For fields of small positive characteristic, the processor count measures of our solutions are somewhat higher.

LINBOX: A GENERIC LIBRARY FOR EXACT LINEAR ALGEBRA

Black box techniques [12] are enabling exact linear algebra computations of a scale well beyond anything previously possible. The development of new and interesting algorithms has proceeded apace for the past two decades. It is time for the dissemination of these algorithms in an easily used software library so that the mathematical community may readily take advantage of their power. LinBox is that library. In this paper, we describe the design of this generic library, sketch its current range of capabilities, and give several examples of its use. The examples include a solution of Trefethen’s “Hundred Digit Challenge” problem #7 [14] and the computation of all the homology groups of simplicial complexes using the Smith normal form [8]. Exact black box methods are currently successful on sparse matrices with hundreds of thousands of rows and columns and having several million nonzero entries. The main reason large problems can be solved by black box methods is that they require much less memory in general than traditional eliminationbased methods do. This fact is widely used in the numerical computation area. We refer for instance to the templates for linear system solution and eigenvalue problems [2,1]. This has also led the computer algebra community to a considerable interest in black box methods. Since Wiedemann’s seminal paper [16], many developments have been proposed especially to adapt Krylov or Lanczos methods to fast exact algorithms. We refer to [5] and references therein for a review of problems and solutions. LinBox supplies efficient black box solutions for a variety of problems including linear equations and matrix normal forms with the guiding design principle of re-usability. The most essential and driving design criterion for LinBox is that it is generic with respect to the domain of computation. This is because there are many and various representations of finite fields each of which is advantageous to use for some algorithm under some circumstance. The integral and rational number capabilities depend heavily on modular