Giuseppe Genovese

Equilibrium statistical mechanics of bipartite spin systems

The aim of this paper is to give an extensive treatment of bipartite mean field spin systems, pure and disordered. At first, bipartite ferromagnets are investigated, and an explicit expression for the free energy is achieved through a new minimax variational principle. Then, via the Hamilton?Jacobi technique, the same structure of the free energy is obtained together with the existence of its thermodynamic limit and the minimax principle is connected to a standard max one. The same is investigated for bipartite spin-glasses. By the Borel?Cantelli lemma we obtain the control of the high temperature regime, while via the double stochastic stability technique we also obtain the explicit expression of the free energy in the replica symmetric approximation, uniquely defined by a minimax variational principle again. We also obtain a general result that states that the free energies of these systems are convex linear combinations of their independent one-party model counterparts. For the sake of completeness, we show further that at zero temperature the replica symmetric entropy becomes negative and, consequently, such a symmetry must be broken. The treatment of the fully broken replica symmetry case is deferred to a forthcoming paper. As a first step in this direction, we start deriving the linear and quadratic constraints to overlap fluctuations.

The Replica Symmetric Approximation of the Analogical Neural Network

In this paper we continue our investigation of the analogical neural network, by introducing and studying its replica symmetric approximation in the absence of external fields. Bridging the neural network to a bipartite spin-glass, we introduce and apply a new interpolation scheme to its free energy, that naturally extends the interpolation via cavity fields or stochastic perturbations from the usual spin glass case to these models.While our methods allow the formulation of a fully broken replica symmetry scheme, in this paper we limit ourselves to the replica symmetric case, in order to give the basic essence of our interpolation method. The order parameters in this case are given by the assumed averages of the overlaps for the original spin variables, and for the new Gaussian variables. As a result, we obtain the free energy of the system as a sum rule, which, at least at the replica symmetric level, can be solved exactly, through a self-consistent mini-max variational principle.The so gained replica symmetric approximation turns out to be exactly correct in the ergodic region, where it coincides with the annealed expression for the free energy, and in the low density limit of stored patterns. Moreover, in the spin glass limit it gives the correct expression for the replica symmetric approximation in this case. We calculate also the entropy density in the low temperature region, where we find that it becomes negative, as expected for this kind of approximation. Interestingly, in contrast with the case where the stored patterns are digital, no phase transition is found in the low temperature limit, as a function of the density of stored patterns.

How glassy are neural networks

In this paper we continue our investigation on the high storage regime of a neural network with Gaussian patterns. Through an exact mapping between its partition function and one of a bipartite spin glass (whose parties consist of Ising and Gaussian spins respectively), we give a complete control of the whole annealed region. The strategy explored is based on an interpolation between the bipartite system and two independent spin glasses built respectively by dichotomic and Gaussian spins: critical line, behavior of the principal thermodynamic observables and their fluctuations as well as overlap fluctuations are obtained and discussed. Then, we move further, extending such an equivalence beyond the critical line, to explore the broken ergodicity phase under the assumption of replica symmetry and show that the quenched free energy of this (analogical) Hopfield model can be described as a linear combination of the two quenched spin glass free energies even in the replica symmetric framework.

A mechanical approach to mean field spin models

Inspired by the bridge pioneered by Guerra among statistical mechanics on lattice and analytical mechanics on 1+1 continuous Euclidean space-time, we built a self-consistent method to solve for the thermodynamics of mean-field models defined on lattice, whose order parameters self average. We show the whole procedure by analyzing in full details the simplest test case, namely the Curie-Weiss model. Further we report some applications also to models whose order parameters do not self-average, by using the Sherrington-Kirkpatrick spin glass as a guide.

Phase transitions in restricted Boltzmann machines with generic priors

We study generalized restricted Boltzmann machines with generic priors for units and weights, interpolating between Boolean and Gaussian variables. We present a complete analysis of the replica symmetric phase diagram of these systems, which can be regarded as generalized Hopfield models. We underline the role of the retrieval phase for both inference and learning processes and we show that retrieval is robust for a large class of weight and unit priors, beyond the standard Hopfield scenario. Furthermore, we show how the paramagnetic phase boundary is directly related to the optimal size of the training set necessary for good generalization in a teacher-student scenario of unsupervised learning.

Legendre Duality of Spherical and Gaussian Spin Glasses

The classical result of concentration of the Gaussian measure on the sphere in the limit of large dimension induces a natural duality between Gaussian and spherical models of spin glass. We analyse the Legendre variational structure linking the free energies of these two systems, in the spirit of the equivalence of ensembles of statistical mechanics. Our analysis, combined with the previous work (Barra et al., J. Phys. A: Math. Theor. 47, 155002, 2014), shows that such models are replica symmetric. Lastly, we briefly discuss an application of our result to the study of the Gaussian Hopfield model.

Non-Convex Multipartite Ferromagnets

We investigate a multipartite ferromagnetic model without self-interactions between spins of the same party, so that the Hamiltonian is not a definite quadratic form of the magnetisations. We find the free energy and study the phase transition for all zero external fields. Moreover in the bipartite case we analyse the fluctuations of the rescaled magnetisations, below and at the critical point, and we study the phase transitions with non-zero magnetic fields.

Phase diagram of restricted Boltzmann machines and generalized Hopfield networks with arbitrary priors

Restricted Boltzmann machines are described by the Gibbs measure of a bipartite spin glass, which in turn can be seen as a generalized Hopfield network. This equivalence allows us to characterize the state of these systems in terms of their retrieval capabilities, both at low and high load, of pure states. We study the paramagnetic-spin glass and the spin glass-retrieval phase transitions, as the pattern (i.e., weight) distribution and spin (i.e., unit) priors vary smoothly from Gaussian real variables to Boolean discrete variables. Our analysis shows that the presence of a retrieval phase is robust and not peculiar to the standard Hopfield model with Boolean patterns. The retrieval region becomes larger when the pattern entries and retrieval units get more peaked and, conversely, when the hidden units acquire a broader prior and therefore have a stronger response to high fields. Moreover, at low load retrieval always exists below some critical temperature, for every pattern distribution ranging from the Boolean to the Gaussian case.

Universality in bipartite mean field spin glasses

In this work, we give a proof of universality with respect to the choice of the statistical distribution of the quenched noise, for mean field bipartite spin glasses. We use mainly techniques of spin glasses theory, as Guerras interpolation and the cavity approach.

Overlap Synchronisation in Multipartite Random Energy Models

In a multipartite random energy model, made of a number of coupled generalised random energy models (GREMs), we determine the joint law of the overlaps in terms of the ones of the single GREMs. This provides the simplest example of the so-called overlap synchronisation.