Masatoshi Shiino

Self-consistent signal-to-noise analysis and its application to analogue neural networks with asymmetric connections

A new systematic method is proposed for the analysis of the storage capacity of analogue neural networks with general input-output relations. It is based on the self-consistent signal-to-noise analysis in which renormalization of the signal part in the local field is properly performed. A remarkable feature of the present recipe, which in the case of symmetric analogue networks yields the same result as obtained by replica calculations, is the capability of dealing with asymmetric networks of analogue neurons. The theory is applied for the asymmetric network in which each neuron is loaded with biased patterns while some neurons are assigned only to extend inhibitory synaptic couplings free of learning.

Free energies based on generalized entropies and H-theorems for nonlinear Fokker–Planck equations

The relationship between H-theorems and free energies is studied on the basis of generalized entropies. Two kinds of nonlinear Fokker–Planck equations with different nonlinear diffusion terms that exhibit the power-law-type equilibrium distributions of Tsallis thermostatistics are investigated from the viewpoint of nonequilibrium free energies and stability analysis of their solutions. Using the generalized entropies Liapunov functions are constructed to show H-theorems, which ensure uniqueness of and convergence to the equilibrium distributions of the nonlinear Fokker–Planck equations.

H-Theorem with Generalized Relative Entropies and the Tsallis Statistics

Generalized relative entropies with a real parameter q are proposed for Markovian stochastic processes, and the associated H-theorem is proved. The proposed entropies play the role of a Lyapunov function as the standard relative entropy known as the Kullback-Leibler divergence does. The H-theorem can be applied to Tsallis statistics, and it is shown that the nonequilibrium free energy, consistently defined based on the generalized entropy in the framework of Tsallis statistics, monotonically decreases to attain its equilibrium value.

Replica-symmetric theory of nonlinear analogue neural networks

On the basis of the theory of the naive mean field model of spin glasses, analogue neural networks of the Hopfield type are investigated in the saturation limit. The saddle-point equations for the order parameters describing retrieval and spin glass phases of the networks are obtained by means of a statistical mechanical analysis within the framework of the replica-symmetric theory and are shown to undergo a modification to that of AGS theory, due to the absence of the Onsager reaction term in the TAP equation. Based on the equations, the memory storage capacity of the networks is analysed as a function of the analogue gain beta . A small increase in the critical storage capacity is found for finite values of beta , compared with that of the Ising model networks with corresponding inverse temperature, although the qualitative nature of the phase diagram is unchanged.

Associative memory storing an extensive number of patterns based on a network of oscillators with distributed natural frequencies in the presence of external white noise.

We study associative memory based on temporal coding in which successful retrieval is realized as an entrainment in a network of simple phase oscillators with distributed natural frequencies under the influence of white noise. The memory patterns are assumed to be given by uniformly distributed random numbers on [0, 2 pi) so that the patterns encode the phase differences of the oscillators. To derive the macroscopic order parameter equations for the network with an extensive number of stored patterns, we introduce an effective transfer function by assuming a fixed-point equation of the form of the Thouless-Anderson-Palmer equation, which describes the time-averaged output as a function of the effective time-averaged local field. Properties of the networks associated with synchronization phenomena for a discrete symmetric natural frequency distribution with three frequency components are studied based on the order parameter equations, and are shown to be in good agreement with the results of numerical simulations. Two types of retrieval states are found to occur with respect to the degree of synchronization, when the size of the width of the natural frequency distribution is changed.

H-theorem and stability analysis for mean-field models of non-equilibrium phase transitions in stochastic systems

Abstract The construction of an H -functional for stochastic systems exhibiting phase transitions of mean-field type is described and an H -theorem is proved. By calculating the second order variation of the H -functional around the stationary distribution functions, we obtain conditions for the stability of the stationary states of the systems. We find a simple criterion, expressed in terms of static correlations, for determining the change in stability of the stationary states of the systems with changes in the control parameters.

Onset of 'super retrieval phase' and enhancement of the storage capacity in neural networks of nonmonotonic neurons

Analogue neural networks of associative memory with continuous time dynamics are studied for nonmonotonic transfer functions using the method of self-consistent signal-to-noise analysis. The Hebb learning rule with unbiased random patterns is assumed for the synaptic couplings. A novel phenomenon is found to occur as a result of a phase transition concerning the property of the local field distribution. In retrieval states of the newly found phase which the authors refer to as the super retrieval phase, noise in the local field vanishes and the memory retrieval without errors ensures even for an extensive number of memory patterns stored under the local learning rule. The storage capacity is obtained as a function of the parameter representing the degree of nonmonotonicity of the transfer functions, with the result that an enhancement of the storage capacity can also occur.

Nonequilibrium phase transition in systems of stochastically perturbed oscillators

Abstract It is shown that a system of infinitely many coupled nonlinear oscillators under the influence of external noise, undergoes a phase transition as the noise power is varied. The phase transition is an analogue of the usual second kind one in thermodynamic systems. The method of the nonlinear Fokker-Planck equation is successfully used to make self-consistent analyses on the critical behavior of the system.

Nonlinear Fokker–Planck equation exhibiting bifurcation phenomena and generalized thermostatistics

A nonlinear Fokker–Planck equation exhibiting bifurcation phenomena is proposed within the framework of generalized thermostatistics. The nonlinearity responsible for the occurrence of bifurcation of solutions is assumed to be of the form appearing in the standard mean field model. A Liapunov function is defined that takes the form of free energy involving generalized entropies of Tsallis and an H-theorem is proved to show that the free energy, which is bounded below, continues to decrease until the system approaches one of the equilibrium distributions. The H-theorem ensures, instead of uniqueness of the equilibrium distribution, global stability of the system in that either one of multisolutions must be approached for large times. Local stability analysis is conducted and the second-order variation of the Liapunov function is computed to find its relevant part whose sign governs stability of the equilibrium distribution of the system. The case with a bistable potential is investigated, as an example of ...

OSCILLATOR NEURAL NETWORK MODEL WITH DISTRIBUTED NATIVE FREQUENCIES

We study the associative memory of an oscillator neural network with distributed native frequencies. The model is based on the use of the Hebb learning rule with random patterns (iµ = ±1), and the distribution function of native frequencies is assumed to be symmetric with respect to its average. Although the system with an extensive number of stored patterns is not allowed to become entirely synchronized, long time behaviours of the macroscopic order parameters describing partial synchronization phenomena can be obtained by discarding the contribution from the desynchronized part of the system. The oscillator network is shown to work as associative memory accompanied by synchronized oscillations. A phase diagram representing properties of memory retrieval is presented in terms of the parameters characterizing the native frequency distribution. Our analytical calculations based on the self-consistent signal-to-noise analysis are shown to be in excellent agreement with numerical simulations, confirming the validity of our theoretical treatment.