I. Guyon

High-Order Neural Networks: Information Storage without Errors

A new learning rule is derived, which allows the perfect storage and the retrieval of information and sequences, in neural networks exhibiting high-order interactions between some or all neurons. Such interactions increase the storage capacity of the networks and allow to solve a class of problems which were intractable with standard networks. We show that it is possible to restrict the amount of high-order interactions while improving the attractivity of the stored patterns.

Neural Network Design for Efficient Information Retrieval

The ability of neural networks to store and retrieve information has been investigated for many years. A renewed interest has been triggered by the analogy between neural networks and spin glasses which was pointed out by W.A. Little et al.1 and J. Hopfield2. Such systems would be potentially useful autoassociative memories “if any prescribed set of states could be made the stable states of the system”2; however, the storage prescription (derived from Hebb’s lav/) which was used by both authors did not meet this requirement, so that the information retrieval properties of neural networks based on this law were not fully satisfactory. In the present paper, a generalization of Hebb’s law is derived so as to guarantee, under fairly general conditions, the retrieval of the stored information (autoassociative memory). Illustrative examples are presented.

Neural Networks for Associative Memory Design

The recent wave of interest in cellular structures known as neural networks is due, in large part, to the advent of a new model [1] which turned out to be amenable to analytical results with the tools of statistical mechanics [2]. In a surprisingly short period of time, one has gone a very long way from the initial model; in the present volume, H. Gutfreund presents a review of the basic concepts and of the most recent developments in this very fast-growing field. Apart from the theoretical interest involved in modeling the brain - or at least some functions of the brain - there is also a considerable interest from the point of view of applications to data processing. The present paper will focus essentially on the latter aspect of artificial neural networks. Associative memory is the basic function that can be performed by these systems; therefore, we shall present a general discussion of the concepts related to associative memory, applied to pattern recognition and error correction; various illustrative examples will be shown, and we shall discuss the basic issues in this context. We shall also mention recent developments in the storage and retrieval of sequences of pieces of information.