Paolo Zellini

A new class of quasi-Newtonian methods for optimal learning in MLP-networks

In this paper, we present a new class of quasi-Newton methods for an effective learning in large multilayer perceptron (MLP)-networks. The algorithms introduced in this work, named LQN, utilize an iterative scheme of a generalized BFGS-type method, involving a suitable family of matrix algebras L. The main advantages of these innovative methods are based upon the fact that they have an O(nlogn) complexity per step and that they require O(n) memory allocations. Numerical experiences, performed on a set of standard benchmarks of MLP-networks, show the competitivity of the LQN methods, especially for large values of n.

Matrix algebras in Quasi-Newton methods for unconstrained minimization

Summary. In this paper a new class of quasi-Newton methods, named ℒQN, is introduced in order to solve unconstrained minimization problems. The novel approach, which generalizes classical BFGS methods, is based on a Hessian updating formula involving an algebra ℒ of matrices simultaneously diagonalized by a fast unitary transform. The complexity per step of ℒQN methods is O(n log n), thereby improving considerably BFGS computational efficiency. Moreover, since ℒQNs iterative scheme utilizes single-indexed arrays, only O(n) memory allocations are required. Global convergence properties are investigated. In particular a global convergence result is obtained under suitable assumptions on f. Numerical experiences [7] confirm that ℒQN methods are particularly recommended for large scale problems.

MATRIX DECOMPOSITIONS USING DISPLACEMENT RANK AND CLASSES OF COMMUTATIVE MATRIX ALGEBRAS

Using the notion of displacement rank, we look for a unifying approach to representations of a matrix A as sums of products of matrices belonging to commutative matrix algebras. These representations are then considered in case A is the inverse of a Toeplitz or a Toeplitz plus Hankel matrix. Some well-known decomposition formulas for A (Gohberg-Semencul or Kailath et al., Gader, Bini-Pan, and Gohberg-Olshevsky) turn out to be special cases of the above representations. New formulas for A in terms of algebras of symmetric matrices are studied, and their computational aspects are discussed.

On the optimal computation of a set of symmetric and persymmetric bilinear forms

Abstract A special class T n of n × n matrices is described, which has tensor rank n over the real field. A tensor base for general symmetric, persymmetric, both symmetric and persymmetric matrices and Toeplitz symmetric matrices can be defined in terms of the tensor bases of T l for some different values of l . It is proved that both symmetric and persymmetric n × n matrices and Toeplitz symmetric n × n matrices have tensor rank [ (n+1) 2 4 ] and 2 n −2, respectively, in the real field.

Matrix algebras in optimal preconditioning

The theory and the practice of optimal preconditioning in solving a linear system by iterative processes is founded on some theoretical facts understandable in terms of a class V of spaces of matrices including diagonal algebras and group matrix algebras. The V-structure lets us extend some known crucial results of preconditioning theory and obtain some useful information on the computability and on the efficiency of new preconditioners. Three preconditioners not yet considered in literature, belonging to three corresponding algebras of V, are analyzed in detail. Some experimental results are included.

On some properties of circulant matrices

Abstract A class Σ of matrices is studied which contains, as special subclasses, p -circulant matrices ( p ⩾ 1), Toeplitz symmetric matrices and the inverses of some special tridiagonal matrices. We give a necessary and sufficient condition in order that matrices of Σ commute with each other and are closed with respect to matrix product.

Low-complexity minimization algorithms

SUMMARY Structured matrix algebras L and a generalized BFGS-type iterative scheme have been recently investigated to introduce low-complexity quasi-Newton methods, named LQN, for solving general (non-structured) minimization problems. In this paper we introduce the L k QN methods, which exploit

Matrix displacement decompositions and applications to Toeplitz linear systems

Using the approach of Bozzo, Di Fiore, and Zellini, new matrix displacement decomposition formulas are introduced. It is shown how an arbitrary square matrix A can be expressed as sums of products of Hessenberg algebra matrices and high level (block) matrices whose submatrices are Hessenberg algebra matrices and have variable sizes. In most cases these block factors are block-diagonal matrices. Then these formulas are used in sequential and parallel solution of Toeplitz systems.

Hartley-type algebras in displacement and optimization strategies

Abstract The Hartley-type ( Ht ) algebras are used to face efficiently the solution of structured linear systems and to define low complexity methods for solving general (nonstructured) nonlinear problems. Displacement formulas for the inverse of a symmetric Toeplitz matrix in terms of Ht transforms are compared with the well known Ammar–Gader formula. The L QN unconstrained optimization methods, which define Hessian approximations by updating n × n matrices from an algebra L , can be implemented for L =Ht with an O( n ) amount of memory allocations and O( n log n ) arithmetic operations per step. The L QN methods with the lowest experimental rate of convergence are shown to be linearly convergent.

Closure, commutativity and minimal complexity of some spaces of matrices

The paper is concerned with some algebraic and computational properties of spaces spanned by a set of (0, 1) matrices Jk , with particular reference to the closure, the commutativity and the tensor rank of . Both group matrices and other algebras of matrices which are not related to a group structure are involved in a unifying approach.