J P L Hatchett

Analytic solution of attractor neural networks on scale-free graphs

We study the influence of network topology on retrieval properties of recurrent neural networks, using replica techniques for dilute systems. The theory is presented for a network with an arbitrary degree distribution p(k) and applied to power-law distributions p(k) ∼ k −γ , i.e. to neural networks on scale-free graphs. A bifurcation analysis identifies phase boundaries between the paramagnetic phase and either a retrieval phase or a spin-glass phase. Using a population dynamics algorithm, the retrieval overlap and spin-glass order parameters may be calculated throughout the phase diagram. It is shown that there is an enhancement of the retrieval properties compared with a Poissonian random graph. We compare our findings with simulations.

Parallel dynamics of disordered Ising spin systems on finitely connected random graphs

We study the dynamics of bond-disordered Ising spin systems on random graphs with finite connectivity, using generating functional analysis. Rather than disorder-averaged correlation and response functions (as for fully connected systems), the dynamic order parameter is here a measure which represents the disorder-averaged single-spin path probabilities, given external perturbation field paths. In the limit of completely asymmetric graphs, our macroscopic laws close already in terms of the single-spin path probabilities at zero external field. For the general case of arbitrary graph symmetry, we calculate the first few time steps of the dynamics exactly, and we work out (numerical and analytical) procedures for constructing approximate stationary solutions of our equations. Simulation results support our theoretical predictions.

Dynamical Replica Analysis of Disordered Ising Spin Systems on Finitely Connected Random Graphs

We study the dynamics of macroscopic observables such as the magnetization and the energy per degree of freedom in Ising spin models on random graphs of finite connectivity, with random bonds and/or heterogeneous degree distributions. To do so, we generalize existing versions of dynamical replica theory and cavity field techniques to systems with strongly disordered and locally treelike interactions. We illustrate our results via application to, e.g., +/-J spin glasses on random graphs and of the overlap in finite connectivity Sourlas codes. All results are tested against Monte Carlo simulations.

Finitely connected vector spin systems with random matrix interactions

We use finite connectivity equilibrium replica theory to solve models of finitely connected unit-length vectorial spins, with random pair-interactions which are of the orthogonal matrix type. Finitely connected spin models, although still of a mean-field nature, can be regarded as a convenient level of description in between fully connected and finite-dimensional ones. Since the spins are continuous and the connectivity c remains finite in the thermodynamic limit, the replica-symmetric order parameter is a functional. The general theory is developed for arbitrary values of the dimension d of the spins, and arbitrary choices of the ensemble of random orthogonal matrices. We calculate phase diagrams and the values of moments of the order parameter explicitly for d = 2 (finitely connected XY spins with random chiral interactions) and for d = 3 (finitely connected classical Heisenberg spins with random chiral interactions). Numerical simulations are shown to support our predictions quite satisfactorily.

Dynamic rewiring in small world networks

We investigate equilibrium properties of small world networks, in which both connectivity and spin variables are dynamic, using replicated transfer matrices within the replica symmetric approximation. Population dynamics techniques allow us to examine the order parameters of our system at total equilibrium, probing both spin and graph statistics. Of these, interestingly, the degree distribution is found to acquire a Poisson-like form (both within and outside the ordered phase). Comparison with Glauber simulations confirms our results satisfactorily.

Asymmetrically extremely dilute neural networks with Langevin dynamics and unconventional results

We study graded response attractor neural networks with asymmetrically extremely dilute interactions and Langevin dynamics. We solve our model in the thermodynamic limit using generating functional analysis, and find (in contrast to the binary neurons case) that even in statics, for T> 0 or large α, one cannot eliminate the non-persistent order parameters, atypically for recurrent neural network models. The macroscopic dynamics is driven by the (non-trivial) joint distribution of neurons and fields, rather than just the (Gaussian) field distribution. We calculate phase transition lines and find, as may be expected for this asymmetric model, that there is no spin-glass phase, only recall and paramagnetic phases. We present simulation results in support of our theory.

Dynamical replica theoretic analysis of CDMA detection dynamics

We investigate the detection dynamics of the Gibbs sampler for code-division multiple access (CDMA) multiuser detection. Our approach is based upon dynamical replica theory which allows an analytic approximation to the dynamics. We use this tool to investigate the basins of attraction when phase coexistence occurs and conclude that good decoding past the spinodal point is not practically possible with this algorithm. We examine the efficacy of our method by doing a comparison with Monte Carlo simulations.