Marc Mézard

Spin glass theory and beyond

This book contains a detailed and self-contained presentation of the replica theory of infinite range spin glasses. The authors also explain recent theoretical developments, paying particular attention to new applications in the study of optimization theory and neural networks. About two-thirds of the book are a collection of the most interesting and pedagogical articles on the subject.

Learning in feedforward layered networks: the tiling algorithm

The authors propose a new algorithm which builds a feedforward layered network in order to learn any Boolean function of N Boolean units. The number of layers and the number of hidden units in each layer are not prescribed in advance: they are outputs of the algorithm. It is an algorithm for growth of the network, which adds layers, and units inside a layer, at will until convergence. The convergence is guaranteed and numerical tests of this strategy look promising.

Wealth condensation in a simple model of economy

We introduce a simple model of economy, where the time evolution is described by an equation capturing both exchange between individuals and random speculative trading, in such a way that the fundamental symmetry of the economy under an arbitrary change of monetary units is insured. We investigate a mean-field limit of this equation and show that the distribution of wealth is of the Pareto (power-law) type. The Pareto behaviour of the tails of this distribution appears to be robust for finite range models, as shown using both a mapping to the random ‘directed polymer’ problem, as well as numerical simulations. In this context, a phase transition between an economy dominated by a few individuals and a situation where the wealth is more evenly spread out, is found. An interesting outcome is that the distribution of wealth tends to be very broadly distributed when exchanges are limited, either in amplitude or topologically. Favouring exchanges (and, less surprisingly, increasing taxes) seems to be an efficient way to reduce inequalities.

Learning algorithms with optimal stability in neural networks

To ensure large basins of attraction in spin-glass-like neural networks of two-state elements xi imu =+or-1. The authors propose to study learning rules with optimal stability Delta , where delta is the largest number satisfying Delta <or=( Sigma j Jij xi jmu ) xi imu ; mu =1. . . . .p: i=1. . . . .N (where N is the number of neurons and p is the number of patterns). They motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix (Jij). In addition, numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns.

The Bethe lattice spin glass revisited

Abstract:So far the problem of a spin glass on a Bethe lattice has been solved only at the replica symmetric level, which is wrong in the spin glass phase. Because of some technical difficulties, attempts at deriving a replica symmetry breaking solution have been confined to some perturbative regimes, high connectivity lattices or temperature close to the critical temperature. Using the cavity method, we propose a general non perturbative solution of the Bethe lattice spin glass problem at a level of approximation which is equivalent to a one step replica symmetry breaking solution. The results compare well with numerical simulations. The method can be used for many finite connectivity problems appearing in combinatorial optimization.

THE SIMPLEST SPIN GLASS

We study a system of Ismg spins with quenched random mfimte ranged p-spin interactions For p -* co, we can solve thts model exactly either by a direct mlcrocanomcal argument, or through the introduction of rephcas and Pansls ultrametnc ansatz for rephca symmetry breaking, or by means of TAP mean field equat,ons Although the model is extremely simple it retains the charactertstlc features of a spin glass We use ~t to confirm the methods that have been apphed in more comphcated sxtuatlons and to exphcltly exhibit the structure of the spin glass phase

The Cavity Method at Zero Temperature

In this note we explain the use of the cavity method directly at zero temperature, in the case of the spin glass on a lattice with a local tree like structure, which is the proper generalization of the usual Bethe lattice to frustrated problems. The computation is done explicitly in the formalism equivalent to “one step replica symmetry breaking;” we compute the energy of the global ground state, as well as the complexity of equilibrium states at a given energy. Full results are presented for a Bethe lattice with connectivity equal to three. The main assumptions underlying the one step cavity approach, namely the existence of many local ground states, are explicitely stated and discussed: some of the main obstacles towards a rigorous study of the problem with the cavity method are outlined.

Statistical-Physics-Based Reconstruction in Compressed Sensing

Compressed sensing is triggering a major evolution in signal acquisition. It consists in sampling a sparse signal at low rate and later using computational power for its exact reconstruction, so that only the necessary information is measured. Currently used reconstruction techniques are, however, limited to acquisition rates larger than the true density of the signal. We design a new procedure which is able to reconstruct exactly the signal with a number of measurements that approaches the theoretical limit in the limit of large systems. It is based on the joint use of three essential ingredients: a probabilistic approach to signal reconstruction, a message-passing algorithm adapted from belief propagation, and a careful design of the measurement matrix inspired from the theory of crystal nucleation. The performance of this new algorithm is analyzed by statistical physics methods. The obtained improvement is confirmed by numerical studies of several cases.

Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices

Compressed sensing is a signal processing method that acquires data directly in a compressed form. This allows one to make fewer measurements than were considered necessary to record a signal, enabling faster or more precise measurement protocols in a wide range of applications. Using an interdisciplinary approach, we have recently proposed in Krzakala et?al (2012 Phys. Rev. X 2 021005) a strategy that allows compressed sensing to be performed at acquisition rates approaching the theoretical optimal limits. In this paper, we give a more thorough presentation of our approach, and introduce many new results. We present the probabilistic approach to reconstruction and discuss its optimality and robustness. We detail the derivation of the message passing algorithm for reconstruction and expectation maximization learning of signal-model parameters. We further develop the asymptotic analysis of the corresponding phase diagrams with and without measurement noise, for different distributions of signals, and discuss the best possible reconstruction performances regardless of the algorithm. We also present new efficient seeding matrices, test them on synthetic data and analyze their performance asymptotically.

Random K-satisfiability problem: From an analytic solution to an efficient algorithm

We study the problem of satisfiability of randomly chosen clauses, each with K Boolean variables. Using the cavity method at zero temperature, we find the phase diagram for the K=3 case. We show the existence of an intermediate phase in the satisfiable region, where the proliferation of metastable states is at the origin of the slowdown of search algorithms. The fundamental order parameter introduced in the cavity method, which consists of surveys of local magnetic fields in the various possible states of the system, can be computed for one given sample. These surveys can be used to invent new types of algorithms for solving hard combinatorial optimizations problems. One such algorithm is shown here for the K=3 satisfiability problem, with very good performances.