11 Constraint optimisation and landscapes.
Florent Krzakala and Jorge Kurchan ,(1) PCT-ESPCI CNRS UMR Gulliver 7083 and (2) PMMH-ESPCI, CNRS UMR 7636,10 rue Vauquelin, 75005 Paris, FRANCEWe describe an effective landscape introduced in [1] for the analysis of Constraint Sat-isfaction problems, such as Sphere Packing, K-SAT and Graph Coloring. This geometricconstruction reexpresses these problems in the more familiar terms of optimisation in ruggedenergy landscapes. In particular, it allows one to understand the puzzling fact that unsophis-ticated programs are successful well beyond what was considered to be the ‘hard’ transition,and suggests an algorithm defining a new, higher, easy-hard frontier.PACS Numbers : 75.10.Nr, 02.50.-r,64.70.Pf, 81.05.Rm a r X i v : . [ qu a n t - ph ] S e p Amongst glassy systems, the particular class of ‘Constraint Optimisation’ has receivedconstant attention [2, 3]. These are problems in which we are given a set of constraints thatmust be satisfied, and our task is to optimize the conditions without violating them. Thetypical example is packing: we are asked to put as many objects (spheres, say) in a givenvolume, without violating the constraint that they should not overlap. Another examplethat has been widely studied by computer scientists is K-SAT, where we have N Booleanvariables, and αN logic clauses: our task is to add more and more clauses while still findingsome set of variables that satisfy them. One last example is the q -coloring problem: wehave a graph with N nodes and αN links and our task is to color each vertex with one of q colours, with the condition that linked vertices have different colours. If we consider asequence of graphs as a set of nodes and a predefined list of links, then adding links one byone makes the problem harder and harder.What motivated our interest in these problems was what we percieved as a confusingsituation in the literature. Consider first sphere packing. In Fig. 1 we show different volumefractions that are often quoted in the literature. In particular ‘Random Close Packing’ (asdefined empirically), the ‘optimal random packing’ (zero-temperature glass state) and the so-called ‘J-point’, are sometimes used as synonyms and sometimes not. The ’J-point’ deserves random loose packing mode−coupling random close packingcrystalJ−point procedure Ideal glass state?? φ FIG. 1: The various transition densities for the sphere-packing problem. some explanation. It can be defined as follows [4]: one starts from small spheres in randompositions, and ‘inflates’ them gradually (in the computer, of course), only displacing them theleast neccessary to avoid overlaps [6]. At some point the system blocks and the procedurestops: this is the J-point. It was studied extensively by the Chicago group [4, 5], whoproposed that it be identified with Random Close Packing. On the other hand, RandomClose Packing is often associated with the zero-temperature ideal glass state. The twoidentifications seem hardly compatible, as they would imply that the fast algorithm describedabove allows to find rapidly the ideal glass state, contrary to all our prejudices.Let us now turn to the SAT and Colouring problems. Carrying over the knowledgefrom mean-field glasses, it was concluded that the set of solutions evolves, as the difficultyis increased, in the following manner: for low α the set of solutions is connected. As α is increased there is a well defined ‘dynamic’ or ‘clustering’ point α d at which the set ofsatisfied solutions breaks into many comparable disconnected pieces [11]. At a larger value α K the volume becomes dominated by a few regions, and finally, at some α c , there are nomore solutions [7].Beyond the clustering transition α d the problem was thought to become hard . And yet,as it turned out, even very simple programs[15] manage to find solutions well beyond thishard transition!
The situation is showed in Fig. 2. This is another puzzle we set out toclarify.A first observation one can make is that the ‘J-point’ procedure can be generalised to allof these problems: one just has in all cases to increase the difficulty gradually, and keep thesystem satisfied by minimal changes each time. For example, for the Colouring problem,one adds one link at a time, and corrects any miscoloring generated by such an addition.The number of colour flips needed each time to correct the miscoloring grows and it divergeswith a well-defined, reproducible power law (see Fig. 3) at a value α ∗ , by definition the limitreached by the program.Second, and most important, we introduce a (pseudo) energy landscape as follows (seeFig. 4). As the difficulty in the problem is increased – by increasing the radius, or addingclauses, or adding links – the set of satisfied configurations becomes a subset of the previousone. This allows to construct a single-valued envelope function (Fig. 4): the pseudo-energy. It is easy to see that the J-point procedure is just a zero-temperature descent on this landscape .We can now carry over everything we know from energy landscapes. For the J-point inthe context of sphere packings, we conclude that: α d α sp αα d αα sp αα asat c c (3.86 , 4.21 , 4.245 , 4.267)(2.0 , 2.275, 2.3 , 2.345) * FIG. 2: Why is it so easy to go beyond α d , the putative ‘hard’ limit? Values of the parameter: i) α d the ‘clustering’ transition, ii) α ASAT for ASAT, α ∗ for our algorithm, iii) α SP the performanceof a Survey Propagation implementation, and iv) α c the optimum [8]. • The J-point, being the result of a gradient from a random configuration, cannot bethe optimal amourphous solution. It is just the analogue of the infinite temperatureinherent structures. • It is in general more compact than the clustering (Mode Coupling) point, since it gainsfrom ‘falling to the bottom of one cluster’. • It may be more or less compact than the Kauzmann ( α K ) point itself, depending onthe dimension, polydispersity, shape, etc.For problems such as SAT and Coloring, we have now a recursive incremental algorithm,in which one increases the difficulty at small steps, at the same time correcting the configu-ration minimally in order to stay satisfied [12]. This algorithm finds solutions in polynomialtime up to a α ∗ ≥ α d . Once α d is reached, the current solution remains ‘trapped’ within onecluster. On increasing further α (for example, in the Coloring problem, by adding furtherlinks), the cluster of solution contracts until it finally dissappears at α ∗ . As one can see inFig. 2, one can go quite a long way beyond α d . How much so depends on how fast thecluster dissapears: gradually for small q and K (in Coloring and SAT, respectively), and t i m e / N α q=3 q=4 α d = α K α uncol α d α K α uncol N=10 N=2.10 N=4.10 FIG. 3: Integrated number of colour flips needed to avoid miscolourings, per unit size, for the threeand four colouring problem. The clustering transition and glass transitions are no obstacle. radiusnumber of clauses, number of links, ...
FIG. 4: Pseudo-energy (conjugated to pressure) landscape for constraint optimisation problems.The sets of satisfied configurations at increasing levels of difficulty are included in the previous:this allows for the definition of a well-defined envelope. essentially immediately for large q, K and for problems that have variables whose value isfrozen within a cluster.Our conjecture is that unsophisticated programs will not do better than α ∗ , or rather,than its ‘slow annealing’ version as above [12]. Comparison with message-passing algorithmssuch as Belief and Survey Propagation is complicated by the fact that neither our versionof the Recursive Incremental program, nor the published implementations of Survey Prop-agation have been pushed to their optimum [13]. With this caveat, the Survey Propagationalgorithm seems to do better in the Coloring problem. On the other hand, Braunstein andZecchina have recently shown that a message-passing algorithm does well in the BinaryPerceptron model [14] – a problem with single-configuration states – and this suggests thatthese algorithms go beyond α ∗ in that case. At any rate, it would be very interesting toexplore along these lines the K = 3 SAT problem, a much better studied case.Perhaps the greatest promise of this approach comes from the fact that, as we haveindicated in Ref. [1], α ∗ defined by the Recursive Incremental algorithm lends itself, due toits simple geometric definition, to an analytic computation. [1] F. Krzakala and J. Kurchan, Phys. Rev. E 76 , 021122 (2007)[2] G. Parisi, Lectures of the Varenna summer school, arXiv:cs/0312011.[3] M. Garey and D. S. Johnson,
Computers and Intractability: a Guide to the theory and NP-completeness (Freeman, San Francisco, 1979); C.H. Papadimitriou,
Computational Complexity (Addison-Wesley, 1994).[4] C. S. O’Hern, L. E. Silbert, A. J. Liu, S. R. Nagel, Phys. Rev.
E 68 , 011306 (2003)[5] See also M. Wyart, Annales de Physique, Vol. 30 No. 3 (2005).[6] An infinitely fast version of: B. D. Lubachevsky and F. H. Stillinger, J. Stat. Phys. , 561(1990).[7] See, for a recent discussion: F. Krzakala, A. Montanari, F. Ricci-Tersenghi, G. Semerjian andL. Zdeborov´a, cond-mat/0612365. Proc. Natl. Acad. Sci. 104, 10318 (2007).[8] For the 3-SAT problem transition points see A. Montanari, G. Parisi, F. Ricci-Tersenghi J.Phys. A 37 , 2073 (2004), and refs [7, 9]. The value of α W S is a variant of Walk SAT, see:John Ardelius and Erik Aurell, Phys. Rev. E 74, 037702 (2006). For a Survey Propagationimplementation for the SAT problem see Ref. [9] and J Chavas, C Furtlehner, M Mezard andR Zecchina, J. Stat. Mech. (2005) P11016For the 3-coloring transition points see: L. Zdeborov´a and F. Krzakala,
Phase transitions in the coloring of random graphs , arXiv:0704.1269v1 (2007). The value of α ∗ is the one obtainedby the algorithm described here (see Ref. [1]). The value for a survey propagation program istaken from [10].[9] M´ezard M, Zecchina R, Phys. Rev. E 66
812 (2002)[10] Mulet R, Pagnani A, Weigt M, et al. , Phys. Rev. Lett. , 23 (2006).[12] One can also envisage a further improvement, in analogy with the Lubachevsky-Stillinger [6]algorithm: one can follow each increase in α with a number of steps of diffusion betweensatisfied solutions.[13] Our program is not optimal because we used a Walk-Col routine to find the finite number ofrecolorings at each step. An exhaustive enumeration would have found the minimal numberof recolorings, and this may diverge at a larger value of α , thus yielding a larger value of α ∗ .On the other hand, the Survey Propagation routines have a large freedom in the method ofgoing from fields to configurations, and this has not been fully exploited yet.[14] Braunstein A, Zecchina R, Phys. Rev. Lett.96