Claude Brezinski

Extrapolation methods : theory and practice

Introduction to the Theory. First Steps. What is an Extrapolation Method? What is an Extrapolation Algorithm? Quasi-linear Sequence Transformations. Sequence Transformations as Ratios of Determinants. Triangular Recursive Schemes. Normal Forms of the Algorithms. Progressive Forms of the Algorithms. Particular Rules of the Algorithms. Accelerability and Non-accelerability. Optimality. Asymptotic Behaviour of Sequences. Scalar Extrapolation Algorithms. The E-algorithm. Richardson Extrapolation Process. The -algorithm. The G-transformation. Rational Extrapolation. Generalizations of the -algorithm. Levins Transformations. Overholts Process. -type Algorithms. The Iterated 2 Process. Miscellaneous Algorithms. Special Devices. Error Estimates and Acceleration. Convergence Tests and Acceleration. Construction of Asymptotic Expansions. Construction of Extrapolation Processes. Extraction Procedures. Automatic Selection. Composite Sequence Transformations. Error Control. Contractive Sequence Transformations. Least Squares Extrapolation. Vector Extrapolation Algorithms. The Vector -algorithm. The Topological -algorithm. The Vector E-algorithm. The Recursive Projection Algorithm. The H-algorithm. The Ford-Sidi Algorithms. Miscellaneous Algorithms. Continuous Prediction Algorithms. The Taylor Expansion. Confluent Overholts process. Confluent -algorithms. Confluent -algorithm. Confluent G-transform. Confluent E-algorithm. -type Confluent Algorithms. Applications. Sequences and Series: Simple Sequences, Double Sequences, Chebyshev and Fourier Series, Continued Fractions, Vector Sequences. Systems of Equations: Linear Systems, Projection Methods, Regularization and Penalty Techniques, Nonlinear Equations, Continuation Methods. Eigenelements: Eigenvalues and eigenvectors, Derivatives of Eigensystems. Integral and Differential Equations: Implicit Runge-Kutta Methods, Boundary Value Problems, Nonlinear Methods, Laplace Transform Inversion, Partial Differential Equations. Interpolation and Approximation. Statistics: The Jackknife, ARMA Models, Monte-Carlo Methods. Integration and Differentiation: Acceleration of Quadrature Formulae, Nonlinear Quadrature Formulae, Cauchys Principal Values, Infinite Integrals, Multiple Integrals, Numerical Differentiation. Prediction. Software. Programming the Algorithms. Computer Arithmetic. Programs. Bibliography. Index.

A general extrapolation algorithm

SummaryIn this paper a general formalism for linear and rational extrapolation processes is developped. This formalism includes most of the sequence transformations actually used for convergence acceleration. A general recursive algorithm for implementing the method is given. Convergence results and convergence acceleration results are proved. The vector case and some other extensions are also studied.

A breakdown-free Lanczos type algorithm for solving linear systems

SummaryLanczos type algorithms for solving systems of linear equations have their foundations in the theory of formal orthogonal polynomials and the method of moments which leads to a determinantal formula for their iterates. The various Lanczos type algorithms mainly differ by the way of computing the coefficients entering into the recurrence formulae. If the denominator in the formula for one of these coefficients is zero, then a breakdown occurs in the algorithm, and it must be stopped. Such a breakdown is in fact due to the non-existence of some orthogonal polynomial. In this paper we show how to jump over such a singularity by computing the next existing orthogonal polynomial by the block bordering method. The resulting algorithm, called MRZ, is equivalent to the nongeneric BIODIR algorithm (which is a look-ahead Lanczos type algorithm), but our derivation is much simpler.

Avoiding breakdown and near-breakdown in Lanczos type algorithms

Lanczos type algorithms form a wide and interesting class of iterative methods for solving systems of linear equations. One of their main interest is that they provide the exact answer in at mostn steps wheren is the dimension of the system. However a breakdown can occur in these algorithms due to a division by a zero scalar product. After recalling the so-called method of recursive zoom (MRZ) which allows to jump over such breakdown we propose two new variants. Then the method and its variants are extended to treat the case of a near-breakdown due to a division by a scalar product whose absolute value is small which is the reason for an important propagation of rounding errors in the method. Programming the various algorithms is then analyzed and explained. Numerical results illustrating the processes are discussed. The subroutines corresponding to the algorithms described can be obtained vianetlib.

Multi-parameter regularization techniques for ill-conditioned linear systems

Summary. When a system of linear equations is ill-conditioned, regularization techniques provide a quite useful tool for trying to overcome the numerical inherent difficulties: the ill-conditioned system is replaced by another one whose solution depends on a regularization term formed by a scalar and a matrix which are to be chosen. In this paper, we consider the case of several regularizations terms added simultaneously, thus overcoming the problem of the best choice of the regularization matrix. The error of this procedure is analyzed and numerical results prove its efficiency.

Other Manifestations of the Schur Complement

Abstract Many series and sequence transformations used in numerical analysis are defined as ratios of determinants. They are implemented by recursive algorithms based on determinantal identities. The aim of this paper is to study the connections of these transformations and algorithms with the Schur complement of a matrix. The Schurs formula is extended to the vector case, thus providing the same treatment for vector sequence transformations and the corresponding recursive algorithms. Thanks to these connections, particular rules for avoiding division by zero or numerical instability are obtained for these algorithms. Some fixed-point methods and continued fractions also fit in this framework.

Rational approximation to formal power series

Abstract A general method for obtaining rational approximations to formal power series is defined and studied. This method is based on approximate quadrature formulas. Newton-Cotes and Gauss quadrature methods are used. It is shown that Pade approximants and the ϵ-algorithm are related to Gaussian formulas while linear summation processes are related to Newton-Cotes formulas. An example is exhibited which shows that Pade approximation is not always optimal. An application to e − t is studied and a method for Laplace transform inversion is proposed.

Lanczos-type algorithms for solving systems of linear equations

Abstract Brezinski, C. and H. Sadok, Lanczos-type algorithms for solving systems of linear equations, Applied Numerical Mathematics 11 (1993) 443–473. In this paper, a synthesis of the various Lanczos-type algorithms for solving systems of linear equations is given. It is based on formal orthogonal polynomials and the various algorithms consist in using various recurrence relations for computing these orthogonal polynomials. Moreover new algorithms are easily obtained from the theory. New formulae and a new interpretation of the conjugate gradient squared (CGS) algorithm are also derived and a new formula for the second topological e-algorithm. The theory of orthogonal polynomials also enables us to avoid breakdown in Lanczos-type methods and in the CGS. The case of near-breakdown can be treated similarly.

Recursive interpolation, extrapolation and projection

Abstract The recursive projection algorithm derived in a previous paper is related to several well-known methods of numerical analysis such as the conjugate gradient method, Rosens method and Henricis. It is connected with the general interpolation problem, with extrapolation methods, with orthogonal projection on a subspace and with Fourier expansions. Several other connections and applications are presented.

Extrapolation algorithms and Pade´ approximations: a historical survey

Abstract In numerical analysis there are many methods producing sequences. Such is the case of iterative methods, of methods involving series expansions, of discretization methods, that is methods depending on a parameter such that the approximate solution tends to the exact one when the parameter tends to zero, of perturbation methods, etc. Sometimes, the convergence of these sequences is so slow that their effective use is quite limited. The aim of extrapolation methods is to construct a new sequence converging to the same limit faster than the initial one. Among these methods, the most well known are certainly Richardsons extrapolation algorithm and Aitkens gD2 process. In many branches of applied sciences, the solution of a given problem is often obtained as a power series expansion. The question is then trying to approximate the function from its series expansion. A possible answer is to construct a rational function whose series expansion matches the original one as far as possible. Such rational functions are called Pade approximants. These two subjects, which have some connections, go quite deep and far into the history of mathematics. They are related to continued fractions (a field which goes back to the Greek antiquity), orthogonal polynomials, the moment problem, etc., they played an important role in the development of mathematics (such as the transcendence of e and π) and they have many applications. This paper will give a short historical overview of these two subjects. Of course, we do not pretend to be exhaustive nor even to quote every important contribution. We refer the interested reader to the literature and, in particular, to the recent books [5,22,29,24,38,46,48,68,78,131]. For an extensive bibliography, see [23].