Marco Protasi

Suspiciousness of loading problems

We introduce the notion of suspect families of loading problems in the attempt of formalizing situations in which classical learning algorithms based on local optimization are likely to fail (because of local minima or numerical precision problems). We show that any loading problem belonging to a nonsuspect family can be solved with optimal complexity by a canonical form of gradient descent with forced dynamics (i.e., for this class of problems no algorithm exhibits a better computational complexity than a slightly modified form of backpropagation). The analyses of this paper suggest intriguing links between the shape of the error surface attached to parametrical learning systems (like neural networks) and the computational complexity of the corresponding optimization problem.

Non-suspiciousness: a generalisation of convexity in the frame of foundations of numerical analysis and learning

The effectiveness of connectionist models in emulating intelligent behaviour is strictly related to the capability of the learning algorithms to find optimal or near-optimal solutions. In this paper, a canonical reduction of gradient descent dynamics is proposed, allowing the formulation of the neural network learning as a finite continuous optimisation problem, under some nonsuspiciousness conditions. In the linear case, the nonsuspect nature of the problem guarantees the implementation of an iterative method with O(n/sup 2/) as computational complexity. Finally, since nonsuspiciousness is a generalisation of the concept of convexity, it is possible to apply this theory to the resolution of nonlinear problems.

An efficient algorithm for the binary classification of patterns using MLP-networks

An alternative to the back propagation algorithm for the effective training of multi-layer perceptron (MLP)-networks with binary output is described. The algorithm determines the optimal set of weights by an iterative scheme based on the singular value decomposition (SVD) method and the Fletcher and Reeves version of the conjugate gradient method.<<ETX>>

Unimodal loading problems

This paper deals with optimal learning and provides a unified viewpoint of most significant results in the field. The focus is on the problem of local minima in the cost function that is likely to affect more or less any learning algorithm. We give some intriguing links between optimal learning and the computational complexity of loading problems. We exhibit a computational model such that the solution of all loading problems giving rise to unimodal error functions require the same time, thus suggesting that they belong to the same computational class.

Computational Experiences of New Direct Methods for the On-line Training of MLP-Networks with Binary Outputs

In this paper the authors consider some recent” direct methods” for the training of a three-layered feedforward neural network with thresholds: the Algorithm II and III (Barmann eal., 1992), here named FBFBK the Least Squares Backpropagation (LSB) Algorithm (Barmann et al., 1993), and their innovative method called Iterative Conjugate Gradient Singular Value Decomposition (ICGSV D) Algorithm (Di Martino et al., 1993).