Alejandro Lage-Castellanos

The marriage problem: From the bar of appointments to the agency

We study the stable marriage problem from different points of view. We proposed a microscopic dynamic that led the system to a stationary state that we are able to characterize analytically. Then, we derive a thermodynamical description of the Nash equilibrium states of the system that agree very well with the results of Monte Carlo simulations. Finally, through large-scale numerical simulations we compare the global optimum of the society with the stable marriage of lower energy. We showed that both states are strongly correlated and that the selfish attitude results in a benefit for most of the practitioners belonging to blocking pairs in the global optimum of the society.

Characterizing and improving generalized belief propagation algorithms on the 2D Edwards-Anderson model

We study the performance of different message passing algorithms in the two-dimensional Edwards–Anderson model. We show that the standard belief propagation (BP) algorithm converges only at high temperature to a paramagnetic solution. Then, we test a generalized belief propagation (GBP) algorithm, derived from a cluster variational method (CVM) at the plaquette level. We compare its performance with BP and with other algorithms derived under the same approximation: double loop (DL) and a two-way message passing algorithm (HAK). The plaquette-CVM approximation improves BP in at least three ways: the quality of the paramagnetic solution at high temperatures, a better estimate (lower) for the critical temperature, and the fact that the GBP message passing algorithm converges also to nonparamagnetic solutions. The lack of convergence of the standard GBP message passing algorithm at low temperatures seems to be related to the implementation details and not to the appearance of long range order. In fact, we prove that a gauge invariance of the constrained CVM free energy can be exploited to derive a new message passing algorithm which converges at even lower temperatures. In all its region of convergence this new algorithm is faster than HAK and DL by some orders of magnitude.

Replica Cluster Variational Method

We present a general formalism to make the Replica-Symmetric and Replica-Symmetry-Breaking ansatz in the context of Kikuchi’s Cluster Variational Method (CVM). Using replicas and the message-passing formulation of CVM we obtain a variational expression of the replicated free energy of a system with quenched disorder, both averaged and on a single sample, and make the hierarchical ansatz using functionals of functions of fields to represent the messages. We obtain a set of integral equations for the message functionals. The main difference with the Bethe case is that the functionals appear in the equations in implicit form and are not positive definite, thus standard iterative population dynamic algorithms cannot be used to determine them. In the simplest cases the solution could be obtained iteratively using Fourier transforms.We begin to study the method considering the plaquette approximation to the averaged free energy of the Edwards-Anderson model in the paramagnetic Replica-Symmetric phase. In two dimensions we find that the spurious spin-glass phase transition of the Bethe approximation disappears and the paramagnetic phase is stable down to zero temperature on the square lattice for different random interactions. The quantitative estimates of the free energy and of various other quantities improve those of the Bethe approximation. The plaquette approximation fails to predict a second-order spin-glass phase transition on the cubic 3D lattice but yields good results in dimension four and higher. We provide the physical interpretation of the beliefs in the replica-symmetric phase as disorder distributions of the local Hamiltonian. The messages instead do not admit such an interpretation and indeed they cannot be represented as populations in the spin-glass phase at variance with the Bethe approximation.The approach can be used in principle to study the phase diagram of a wide range of disordered systems and it is also possible that it can be used to get quantitative predictions on single samples. These further developments present however great technical challenges.

Message passing and Monte Carlo algorithms: Connecting fixed points with metastable states

Mean field-like approximations (including naive mean-field, Bethe and Kikuchi and more general cluster variational methods) are known to stabilize ordered phases at temperatures higher than the thermodynamical transition. For example, in the Edwards-Anderson model in 2 dimensions these approximations predict a spin glass transition at finite T. Here we show that the spin glass solutions of the Cluster Variational Method (CVM) at plaquette level do describe well the actual metastable states of the system. Moreover, we prove that these states can be used to predict non-trivial statistical quantities, like the distribution of the overlap between two replicas. Our results support the idea that message passing algorithms can be helpful to accelerate Monte Carlo simulations in finite-dimensional systems.

Replica Cluster Variational Method: the Replica Symmetric solution for the 2D random bond Ising model

We present and solve the replica symmetric equations in the context of the replica cluster variational method for the 2D random bond Ising model (including the 2D Edwards–Anderson spin-glass model). First, we solve a linearized version of these equations to obtain the phase diagrams of the model on the square and triangular lattices. In both cases, the spin-glass transition temperatures and the multicritical point estimations improve largely over the Bethe predictions. Moreover, we show that this phase diagram is consistent with the behavior of inference algorithms on single instances of the problem. Finally, we present a method to consistently find approximate solutions to the equations in the glassy phase. The method is applied to the triangular lattice down to T = 0, also in the presence of an external field.

Zero temperature solutions of the Edwards-Anderson model in random Husimi lattices

We solve the Edwards-Anderson model (EA) in different Husimi lattices using the cavity method at replica symmetric (RS) and 1-step of replica symmetry breaking (1RSB) levels. We show that, at T = 0, the structure of the solution space depends on the parity of the loop sizes. Husimi lattices with odd loop sizes may have a trivial paramagnetic solution thermodynamically relevant for highly frustrated systems while, in Husimi lattices with even loop sizes, this solution is absent. The range of stability under 1RSB perturbations of this and other RS solutions is computed analytically (when possible) or numerically. We also study the transition from 1RSB solutions to paramagnetic and ferromagnetic RS solutions. Finally we compare the solutions of the EA model in Husimi lattices with that on the (short loops free) Bethe lattices, showing that already for loop sizes of order 8 both models behave similarly.

Inference algorithm for finite-dimensional spin glasses: belief propagation on the dual lattice.

Starting from a cluster variational method, and inspired by the correctness of the paramagnetic ansatz [at high temperatures in general, and at any temperature in the two-dimensional (2D) Edwards-Anderson (EA) model] we propose a message-passing algorithm--the dual algorithm--to estimate the marginal probabilities of spin glasses on finite-dimensional lattices. We use the EA models in 2D and 3D as benchmarks. The dual algorithm improves the Bethe approximation, and we show that in a wide range of temperatures (compared to the Bethe critical temperature) our algorithm compares very well with Monte Carlo simulations, with the double-loop algorithm, and with exact calculation of the ground state of 2D systems with bimodal and Gaussian interactions. Moreover, it is usually 100 times faster than other provably convergent methods, as the double-loop algorithm. In 2D and 3D the quality of the inference deteriorates only where the correlation length becomes very large, i.e., at low temperatures in 2D and close to the critical temperature in 3D.

Statistical mechanics of sparse generalization and graphical model selection

One of the crucial tasks in many inference problems is the extraction of an underlying sparse graphical model from a given number of high-dimensional measurements. In machine learning, this is frequently achieved using, as a penalty term, the Lp norm of the model parameters, with p≤1 for efficient dilution. Here we propose a statistical mechanics analysis of the problem in the setting of perceptron memorization and generalization. Using a replica approach, we are able to evaluate the relative performance of naive dilution (obtained by learning without dilution, following by applying a threshold to the model parameters), L1 dilution (which is frequently used in convex optimization) and L0 dilution (which is optimal but computationally hard to implement). Whereas both Lp diluted approaches clearly outperform the naive approach, we find a small region where L0 works almost perfectly and strongly outperforms the simpler to implement L1 dilution.

Random field Ising model in two dimensions: Bethe approximation, cluster variational method and message passing algorithms

We study two free energy approximations (Bethe and plaquette-CVM) for the Random Field Ising Model in two dimensions. We compare results obtained by these two methods in single instances of the model on the square grid, showing the difficulties arising in defining a robust critical line. We also attempt average case calculations using a replica-symmetric ansatz, and compare the results with single instances. Both, Bethe and plaquette-CVM approximations present a similar panorama in the phase space, predicting long range order at low temperatures and fields. We show that plaquette-CVM is more precise, in the sense that predicts a lower critical line (the truth being no line at all). Furthermore, we give some insight on the non-trivial structure of the fixed points of different message passing algorithms.

The importance of dilution in the inference of biological networks

One of the crucial tasks in many inference problems is the extraction of an underlying sparse graphical model from a given number of high-dimensional measurements. In machine learning, this is frequently achieved using, as a penalty term, the Lp norm of the model parameters, with p ≤ 1 for efficient dilution. Here we propose a statistical-mechanics analysis of the problem in the setting of perceptron memorization and generalization. Using a replica approach, we are able to evaluate the relative performance of naive dilution (obtained by learning without dilution, following by applying a threshold to the model parameters), L1 dilution (which is frequently used in convex optimization) and L0 dilution (which is optimal but computationally hard to implement). Whereas both Lp diluted approaches clearly outperform the naive approach, we find a small region where L0 works almost perfectly and strongly outperforms the simpler to implement L1 dilution. In the second part we propose an efficient message-passing strategy in the simpler case of discrete classification vectors, where the norm L0 norm coincides with the L1. Some examples are discussed.