Emilio Spedicato

A class of direct methods for linear systems

SummaryA class of methods of direct type for solving determined or underdetermined, full rank or deficient rank linear systems is presented and theoretically analyzed. The class can be considered as a generalization of the methods of Brent and Brown as restricted to linear systems and implicitly contains orthogonal,LU andLLT factorization methods.

Numerical experience with Newton-like methods for nonlinear algebraic systems

In this paper we present an extensive computational experience with several Newton-like methods, namely Newton’s method, the ABS Huang method, the ABS row update method and six Quasi-Newton methods. The methods are first tested on 31 families of problems with dimensionsn=10, 50, 100 and two starting points. Newton’s method appears to be the best in terms of number of solved problems, followed closely by the ABS Huang method. Broyden’s “bad” method and Greenstadt’s second method show a very poor performance. The other four Quasi-Newton methods perform similarly, strongly suggesting that Greenstadt’s first method and Martínez’ column update method are locally and superlinearly convergent, a result that has yet to be proven theoretically. Thomas’ method appears to be marginally more robust and fast and provides moreover a better approximation to the Jacobian. An interesting and somewhat unexpected observation is that the number of iterations for satisfying the convergence test increases very little with the dimension of the problem. In a second set of experiments we look at the structure of the regions of convergence/nonconvergence by starting the methods from all nodes of a regular grid and assigning to each node a number according to the outcome of the iteration. The obtained regions have clearly a fractal type structure, which, on the two tested problems, is much simpler for Newton’s method than for the other methods. Newton’s method also is the one with the smallest nonconvergence region. Among the Quasi-Newton methods Thomas’ method shows a definitely smaller nonconvergence region.ZusammenfassungWir berichten über umfangreiche rechnerische Erfahrungen mit verschiedenen Newtonartigen Verfahren, nämlich dem klassischen Newton-Verfahren, dem ABS-Huang-Verfahren, dem AMS Zeilen-Update-Verfahren und 6 verschiedenen Quasi-Newton-Verfahren. Die Verfahren werden zuerst on 31 Problemfamilien mit Dimension 10, 50, 100, und zwei Startwerten getestet. Das Newton-Verfahren erweist sich am besten in Bezug auf die Anzahl gelöster Probleme, dicht gefolgt vom ABS-Huang-Verfahren. Broydens “schlechtes” Verfahren und Greenstadts zweites Verfahren erweisen sich als ungünstig. Die anderen vier Quasi-Newton-Verfahren sind gleichwertig, wobei sich Greenstadts erstes Verfahren und das Spalten-Update-Verfahren von Martínez als lokal superlinear konvergent herausstellen, ein theoretisch noch nicht bewiesenes Resultat. Das Verfahren von Thomas scheint ein wenig robuster zu sein und die Jacobimatrix besser zu approximieren. Interessant und etwas unerwartet ist die Beobachtung, daß die Iterationszahl bis zur Erfüllung des Konvergenztests nur schwach mit der Dimension des Systems steigt. In einer zweiten Versuchsreihe beobachten wir die Struktur der Konvergenz/Nichtkonvergenz-Bereiche bei Start von Punkten eines regulären Gitters. Die erhaltenen Bereiche haben eine fraktale Struktur, die — bei den beiden verwendeten Problemen — für das Newton-Verfahren viel einfacher ist als für die anderen Verfahren; es hat auch den kleinsten Nichtkonvergenz-Bereich. Unter den Quasi-Newton-Verfahren zeigt das Verfahren von Thomas klar einen kleineren Nichtkonvergenz-Bereich.

Variable metric methods for unconstrainted optimization and nonlinear least squares

Variable metric or quasi-Newton methods are well known and commonly used in connection with unconstrained optimization, since they have good theoretical and practical convergence properties. Although these methods were originally developed for small- and moderate-size dense problems, their modifications based either on sparse, partitioned or limited-memory updates are very efficient on large-scale sparse problems. Very significant applications of these methods also appear in nonlinear least-squares approximation and nonsmooth optimization. In this contribution, we give an extensive review of variable metric methods and their use in various optimization fields.

The local convergence of ABS methods for nonlinear algebraic equations

SummaryIn this paper we consider an extension to nonlinear algebraic systems of the class of algorithms recently proposed by Abaffy, Broyden and Spedicato for general linear systems. We analyze the convergence properties, showing that under the usual assumptions on the function and some mild assumptions on the free parameters available in the class, the algorithm is locally convergent and has a superlinear rate of convergence (per major iteration, which is computationally comparable to a single Newtons step). Some particular algorithms satisfying the conditions on the free parameters are considered.

ABS methods and ABSPACK for linear systems and optimization: A review

Abstract.ABS methods are a large class of methods, based upon the Egervary rank reducing algebraic process, first introduced in 1984 by Abaffy, Broyden and Spedicato for solving linear algebraic systems, and later extended to nonlinear algebraic equations, to optimization problems and other fields; software based upon ABS methods is now under development. Current ABS literature consists of about 400 papers. ABS methods provide a unification of several classes of classical algorithms and more efficient new solvers for a number of problems. In this paper we review ABS methods for linear systems and optimization, from both the point of view of theory and the numerical performance of ABSPACK.

A class of ABS algorithms for Diophantine linear systems

Summary. Systems of integer linear (Diophantine) equations arise from various applications. In this paper we present an approach, based upon the ABS methods, to solve a general system of linear Diophantine equations. This approach determines if the system has a solution, generalizing the classical fundamental theorem of the single linear Diophantine equation. If so, a solution is found along with an integer Abaffian (rank deficient) matrix such that the integer combinations of its rows span the integer null space of the cofficient matrix, implying that every integer solution is obtained by the sum of a single solution and an integer combination of the rows of the Abaffian. We show by a counterexample that, in general, it is not true that any set of linearly independent rows of the Abaffian forms an integer basis for the null space, contrary to a statement by Egervary. Finally we show how to compute the Hermite normal form for an integer matrix in the ABS framework.

A variable-metric method for function minimization derived from invariancy to nonlinear scaling

The effect of nonlinearly scaling the objective function on the variable-metric method is investigated, and Broydens update is modified so that a property of invariancy to the scaling is satisfied. A new three-parameter class of updates is generated, and criteria for an optimal choice of the parameters are given. Numerical experiments compare the performance of a number of algorithms of the resulting class.

On some classes of variationally derived Quasi-Newton methods for systems of nonlinear algebraic equations

SummaryWe consider the problem of solving systems of nonlinear algebraic equations by Quasi-Newton methods which are variationally obtainable. Properties of termination and of optimal conditioning of this class are studied. Extensive numerical experiments compare particular algorithms and show the superiority of two recently proposed methods.

THE IMPLICIT LX METHOD OF THE ABS CLASS

We describe an algorithm of the ABS class, which solves a general qonsingular linear system in n 3/3 + 0(n 2) multiplications without the assumption that the coefficient matrix be regular. The method can be viewed as a variation of the implicit LU algorithm of the ABS class, whose associated factorization contains a factor which is not triangular (but can be reduced to triangular form after suitable row permutations). We describe king the Abaffan properties of the method, including in particular an efficient way of upd matrix after column interchanges. Such a problem arises in the application to the simplex algorithm, where the implicit LX algorithm provides a faster technique than the standard LU factorization for the pivoting operation if the number of equality constraints m is greater than n/2

A class of rank-one positive definite qnasi-newton updates for unconstrained minimization 2

We introduce variationally a class of rank-one Quasi-Newton updates, which is a subset of the generalized Huang-Oren class. We show that this class contains two disjoint subclasses of positive definite updates. One optimally conditioned update in the sense of Oren and Spedicato is shown to exist in each of the two subclasses. Some criteria for selection of the remaining degrees of freedom and some numerical experiments are discussed.