W. K. Theumann

Generalization and chaos in a layered neural network

The generalization performance of a multi-state and a graded response layered attractor neural network trained with examples of low activity is established exactly for monotonic and non-monotonic input/output functions. Complex behaviour is found which goes from fixed-point attractors to chaos through a cascade of bifurcations, depending on an appropriate threshold or cut-off parameter. The effect of the irregular behaviour on the generalization curves is explicitly demonstrated and phase diagrams for the recognition ratio of concepts in terms of the threshold/cut-off exhibit ordered (generalization), disordered (paramagnetic or self-sustained activity) and chaotic phases.

Retrieval behavior and thermodynamic properties of symmetrically diluted Q-Ising neural networks.

The retrieval behavior and thermodynamic properties of symmetrically diluted Q-Ising neural networks are derived and studied in replica-symmetric mean-field theory generalizing earlier works on either the fully connected or the symmetrical extremely diluted network. Capacity-gain parameter phase diagrams are obtained for the Q=3, Q=4, and Q=infinity state networks with uniformly distributed patterns of low activity in order to search for the effects of a gradual dilution of the synapses. It is shown that enlarged regions of continuous changeover into a region of optimal performance are obtained for finite stochastic noise and small but finite connectivity. The de Almeida-Thouless lines of stability are obtained for arbitrary connectivity, and the resulting phase diagrams are used to draw conclusions on the behavior of symmetrically diluted networks with other pattern distributions of either high or low activity.

Mean-field dynamics of sequence processing neural networks with finite connectivity

A recent dynamic mean-field theory for sequence processing in fully connected neural networks of Hopfield-type is extended and analyzed here for a symmetrically diluted network with finite connectivity near saturation. Equations for the dynamics and the stationary states are obtained for the macroscopic observables and the precise equivalence is established with the single-pattern retrieval problem in a layered feed-forward network with finite connectivity.

Symmetric sequence processing in a recurrent neural network model with a synchronous dynamics

The synchronous dynamics and the stationary states of a recurrent attractor neural network model with competing synapses between symmetric sequence processing and Hebbian pattern reconstruction are studied in this work allowing for the presence of a self-interaction for each unit. Phase diagrams of stationary states are obtained exhibiting phases of retrieval, symmetric and period-two cyclic states as well as correlated and frozen-in states, in the absence of noise. The frozen-in states are destabilized by synaptic noise and well-separated regions of correlated and cyclic states are obtained. Excitatory or inhibitory self-interactions yield enlarged phases of fixed-point or cyclic behaviour.

Flow Diagrams of the Quadratic Neural Network

The macroscopic dynamics of an extremely diluted threestate neural network based on mutual information and mean-field theory arguments is studied in order to establish the stability of the stationary states. Results are presented in terms of the pattern-recognition overlap, the neural activity, and the activity-overlap. It is shown that the presence of synaptic noise is essential for the stability of states that recognize only the active patterns when the full structure of the patterns is not recognizable. Basins of attraction of considerable size are obtained in all cases for a not too large storage ratio of patterns.

The spherical-model limit in a random field

The spherical-model limitn → ∞ of then-vector model in a random field, with either a statistically independent distribution or with long-range correlated random fields, is studied to demonstrate the correctness of the replica method in which then → ∞ and replica limits limits are interchanged, provided the replica and thermodynamic limits are taken in the right order, in the case of long-range correlated random fields. A scaling form for the two-point correlation function relevant to the first-order phase transition below the lower critical dimensionality of the random system is also obtained.

Instability of frozen-in states in synchronous Hebbian neural networks

The full dynamics of a synchronous recurrent neural network model with Ising binary units and a Hebbian learning rule with a finite self-interaction is studied in order to determine the stability to synaptic and stochastic noise of frozen-in states that appear in the absence of both kinds of noise. Both the numerical simulation procedure of Eissfeller and Opper and a new alternative procedure that allows us to follow the dynamics over larger time scales have been used in this work. It is shown that synaptic noise destabilizes the frozen-in states and yields either retrieval or paramagnetic states for not too large stochastic noise. The indications are that the same results may follow in the absence of synaptic noise, for low stochastic noise.

The three-state layered neural network with finite dilution

The dynamics and the stationary states of an exactly solvable three-state layered feed-forward neural network model with asymmetric synaptic connections, finite dilution and low pattern activity are studied in extension of a recent work on a recurrent network. Detailed phase diagrams are obtained for the stationary states and for the time evolution of the retrieval overlap with a single pattern. It is shown that in spite of instabilities for low thresholds there is a gradual improvement in network performance with increasing threshold up to an optimal stage. The robustness to synaptic noise is checked and the effects of dilution and of variable threshold on the information content of the network are also established.

MIXTURE STATES AND STORAGE WITH CORRELATED PATTERNS IN HOPFIELD'S MODEL

The Hopfield model of associative memory with the Hebb learning rule is studied for a finite number p of correlated patterns. The storage capacity α = P/N is considered in a network with further P − p embedded uncorrelated patterns, and the corresponding phase diagrams are exhibited. Numerical simulations are carried out to discuss the retrieval quality and the basins of attraction of the network.

Period-two cycles in a feedforward layered neural network model with symmetric sequence processing.

The effects of dominant sequential interactions are investigated in an exactly solvable feedforward layered neural network model of binary units and patterns near saturation in which the interaction consists of a Hebbian part and a symmetric sequential term. Phase diagrams of stationary states are obtained and a phase of cyclic correlated states of period two is found for a weak Hebbian term, independently of the number of condensed patterns c.