Maximally flexible solutions of a random K -satisfiability formula
MMaximally flexible solutions of a random K -satisfiability formula Han Zhao , ∗ and Hai-Jun Zhou , † CAS Key Laboratory for Theoretical Physics, Institute of Theoretical Physics,Chinese Academy of Sciences, Beijing 100190, China School of Physical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China (Dated: June 15, 2020)Random K -satisfiability ( K -SAT) is a paradigmatic model system for studying phase transitions inconstraint satisfaction problems and for developing empirical algorithms. The statistical propertiesof the random K -SAT solution space have been extensively investigated, but most earlier effortsfocused on solutions that are typical. Here we consider maximally flexible solutions which satisfyall the constraints only using the minimum number of variables. Such atypical solutions have highinternal entropy because they contain a maximum number of null variables which are completely freeto choose their states. Each maximally flexible solution indicates a dense region of the solution space.We estimate the maximum fraction of null variables by the replica-symmetric cavity method, andimplement message-passing algorithms to construct maximally flexible solutions for single K -SATinstances. I. INTRODUCTION
The random K -satisfiability ( K -SAT) problem is aparadigmatic model system of theoretical computerscience [1]. It has been widely adopted to under-stand the typical-case computational complexity of non-deterministic polynomial complete (NP-complete) opti-mization problems. It also serves as a convenient testground for various empirical search algorithms. Thereare only two parameters: the number K of variables in-volved in each constraint, and the ratio α between thenumber of constraints and the number of variables (theclause density). Phase transitions in this system hasbeen extensively investigated in the statistical physicscommunity following the initial empirical observations ofCheeseman, Kirkpatrick, and colleagues [2, 3] and thetheoretical attempts of Monasson and Zecchina [4, 5].Deep insights have been achieved on the statisticalproperties of the random K -SAT solution space overthe last two decades [6–19]. It is now widely ac-cepted that random K -SAT will experience a satisfia-bility phase transition as the clause density α exceedscertain critical value, α s . The numerical value of α s as a function of K can be computed with high pre-cision by the zero-temperature limit of the first-stepreplica-symmetry-breaking (1RSB) mean field theory ofstatistical physics [6, 7, 11]. Before the satisfiabilityphase transition occurs at α s , the random K -SAT prob-lem will first experiences several other interesting phasetransitions as the clause density α increases, such asthe emergence of solution communities [17, 18, 20], thebreaking of ergodicity in the solution space (the clus-tering or dynamical transition) [9, 10, 12, 15, 16], andthe dominance of a sub-exponential number of solutionclusters (the condensation transition) [12–14]. Powerful ∗ Electronic address: [email protected] † Electronic address: [email protected] message-passing algorithms, such as belief-propagationand survey-propagation, have been developed for solvinghard random K -SAT problem instances [6, 21–23].Most of the statistical physics studies in the litera-ture consider solutions (satisfying configurations) thatare picked uniformly at random from the whole solutionspace; in other words, every solution is assigned the samestatistical weight and it has the same contribution to thepartition function. This uniform statistical ensemble isappropriate for investigating typical configurations in thesolution space, but it will completely miss all the differenttypes of atypical solutions. A recent surprising theoreti-cal finding by Huang and co-authors was that the typicalequilibrium solutions may be widely separated in the so-lution space as α becomes close to α s , and then it shouldbe very difficult to reach any of them by local dynami-cal processes [24, 25]. But empirical algorithms such assurvey propagation did indeed succeed at α very closeto α s [6, 21]. Some recent reports suggested that atyp-ical solutions are very important for understanding theperformance of empirical K -SAT algorithms [20, 26–31].Especially, Baldassi and co-authors found that there aresub-dominant and dense clusters in the K -SAT solutionspace, and biasing the search process towards such clus-ters can greatly increase the chance of solving a given K -SAT problem instance [29, 30].We follow this research line on atypical solutions inthe present work, and discuss the issue of maximallyflexible solutions. A maximally flexible solution has theproperty that it satisfies all the constraints of a K -SATinstance only using the minimum number of variables.An extensive number N ρ of the N variables in such aconfiguration can be deleted while the formula is stillsatisfied. These insignificant variables are referred toas the null variables and ρ is the fraction of null vari-ables. Such atypical solutions have high internal entropy(at least of order 2 Nρ ), because the null variables arecompletely free to choose their states. Therefore, eachmaximally flexible solution is associated with a dense re-gion of the solution space. We estimate the maximum a r X i v : . [ c ond - m a t . d i s - nn ] J un raction of null variables by the replica-symmetric (RS)cavity method [32], and our theoretical results suggestthat the maximum fraction ρ max0 of null variables is pos-itive at the satisfiability phase transition point α s . Weimplement two message-passing algorithms to constructmaximally flexible solutions for single K -SAT instances.More work needs to be done to extend the theoreticaland algorithmic investigations to the 1RSB level.The paper is organized as follows: In Sec. II we intro-duce a three-state model and write down the RS meanfield equations. We then describe in Sec. III and Sec. IVtwo message-passing algorithms and discuss the numeri-cal results. Finally we conclude this work in Sec. V. II. REPLICA-SYMMETRIC MEAN FIELDTHEORY
There are N variables in a K -SAT formula (in-stance) and these variables are subject to M constraints(clauses). The clause density α is defined as α ≡ MN . (1)A simple K -SAT formula instance with K = 3 is shownin Fig. 1, which means( x ∨ x ∨ x ) ∧ ( x ∨ x ∨ x ) , where x i ∈ { TRUE , FALSE } is the Boolean state of variable i , x i denotes the Boolean negation of x i ; and ∨ and ∧ denote the Boolean OR and AND operators. There are N = 4 variables and M = 2 clauses, so the clause densityis α = 2.In the original K -SAT problem each variable i canonly take two states σ i = − FALSE ) and σ i = +1 (corresponding to Boolean TRUE ),similar to the Ising model, so the total number of pos-sible microscopic configurations is 2 N [4]. Each clause a involves K variables (we consider K = 3 and K = 4in this paper), and its energy E a is either zero (clausesatisfied) or unity (clause unsatisfied): E a = (cid:89) i ∈ ∂a − J ia σ i , (2)where J ia ∈ {− , +1 } is the quenched binary couplingconstant between clause a and variable i ( J ia = − i is negated in clause a , otherwise J ia = +1), and ∂a denotes the set of variables participating in clause a (the cardinality | ∂a | = K ). The clause energy E a willbe zero if any of the variables i ∈ ∂a takes the state σ i = J ia . A factor-graph representation for the K -SATproblem is shown in Fig. 1, where each clause is linkedto K variables.In the random K -SAT problem the K variables of eachclause a are chosen uniformly at random from the N a b FIG. 1: Factor graph representation for a K -SAT formulacomposed of N = 4 variables and M = 2 clauses. The vari-ables and clauses are denoted by circular and square nodes,respectively. Each clause has K = 3 attached links in thisexample. Each variable i has three states ( s i ∈ {− , , +1 } )in our generalized model. The energy E a of a clause a is ei-ther zero or unity. A solid link ( i, a ) between variable i andclause a means a positive coupling constant ( J ia = 1) andthat E a = 0 if s i = 1, while a dashed link means a negativecoupling constant ( J ia = −
1) and that E a = 0 if s i = −
1. Ifvariable i takes the null state s i = 0, it does not contributeto satisfying any of the attached clauses. variables, and the link coupling constants J ia are inde-pendently assigned the value − Kα .In this work we extend the random K -SAT problemby allowing each variable i the possibility of a null state.The state of variable i in the modified system is denotedas s i and it can be +1, − s i = 0then variable i does not contribute to satisfying any of theattached clauses, and these clauses need to be satisfiedby its other connected variables. The modified energy˜ E a of clause a is then˜ E a = (cid:89) i ∈ ∂a (cid:0) − δ J ia s i (cid:1) , (3)where δ nm is the Kronecker symbol, δ nm = 1 if m = n and δ nm = 0 if m (cid:54) = n .A microscopic configuration of the extended K -SATsystem is denoted as s ≡ ( s , s , ..., s N ). There are atotal number 3 N of possible configurations, but in thiswork we only allow those which satisfy the given K -SATformula, that is, the global constraint is the zero-total-energy condition M (cid:88) a =1 ˜ E a = 0 . (4)The partition function for the system is then Z ( β ) = (cid:88) s N (cid:89) i =1 e βδ si M (cid:89) a =1 (cid:104) − (cid:89) j ∈ ∂a (cid:0) − δ J ja s j (cid:1)(cid:105) . (5)Here a positive inverse temperature β is introduced toencourage more null variables.2ollowing the replica-symmetric (RS) cavity methodof statistical mechanics, which assumes that the variablesparticipating in a given clause a will become mutually in-dependent if this clause is deleted from this system (i.e.,the Bethe-Peierls approximation for a locally tree-likefactor graph [1]), we write down the belief-propagation(BP) equation for the partition function (5) as q i → a ( s i ) = e βδ si (cid:81) b ∈ ∂i \ a p b → i ( s i ) (cid:80) s (cid:48) i e βδ s (cid:48) i (cid:81) b ∈ ∂i \ a p b → i ( s (cid:48) i ) , (6) p a → i ( s i ) = 1 − (1 − δ J ia s i ) (cid:81) j ∈ ∂a \ i [1 − q j → a ( J ja )]3 − (cid:81) j ∈ ∂a \ i [1 − q j → a ( J ja )] . (7)Here q i → a ( s i ) is the cavity probability that variable i would take state s i in the absence of clause a ; p a → i ( s i ) isthe cavity probability that variable i would take state s i if it only participates in clause a ; the set ∂i contains allthe clauses to which variable i are linked, and ∂i \ a meansexcluding clause a from this set ∂i (and similarly, ∂a \ i means excluding variable i from the variable set ∂a ).The marginal probability q i ( s i ) of variable i being instate s i is then evaluated as q i ( s i ) = e βδ si (cid:81) a ∈ ∂i p a → i ( s i ) (cid:80) s (cid:48) i e βδ s (cid:48) i (cid:81) a ∈ ∂i p a → i ( s (cid:48) i ) . (8)The mean fraction ρ of null variables at a given value ofinverse temperature β is computed through ρ = 1 N N (cid:88) i =1 q i (0) . (9)We are aiming at constructing K -SAT configurationswhich contain a maximum number of null variables. Toestimate the maximum fraction ρ max0 of null variablesachievable for a given problem instance, we will compute ρ as a function of inverse temperature β using the aboveexpression.To determine the maximal value of β that is physicallymeaningful, we also need to compute the free energy andentropy of the system. The total free energy F ( β ) isdefined as F ( β ) ≡ − β ln Z ( β ) . (10)In the RS mean field theory F ( β ) can be decomposedinto two parts, the contributions f i + ∂i of the variables i and the associated clauses, and the contributions f a ofsingle clauses a [1, 32]: F ( β ) = N (cid:88) i =1 f i + ∂i − M (cid:88) a =1 ( K − f a . (11) The minus sign before the second summation in the aboveexpression can be understood as follows: each clause a isconsidered K times by the contributions f i + ∂i of its K attached variables i , so there should be a correction term( K − f a . The expressions f i + ∂i and f a are, respectively, f i + ∂i = − β ln (cid:110) e β (cid:89) a ∈ ∂i (cid:104) − (cid:89) j ∈ ∂a \ i [1 − q j → a ( J ja )] (cid:105) + (cid:88) σ i = ± (cid:89) a ∈ ∂i (cid:104) − − σ i J ia (cid:89) j ∈ ∂a \ i [1 − q j → a ( J ja )] (cid:105)(cid:111) , (12) f a = − β ln (cid:104) − (cid:89) j ∈ ∂a [1 − q j → a ( J ja )] (cid:105) . (13)The free energy density is then f ≡ F ( β ) /N , and theentropy density s of the system is computed through s = − ( ρ + f ) β . (14)The entropy density s measure the abundance of satis-fying configurations with null variable fraction ρ . Thevalue of s may saturate to a positive value as β increases(and ρ also saturates to the limiting value ρ max0 ), or itmay become negative as β exceeds certain threshold value β max . In the latter case we take the computed value of ρ at β max as the maximum null variable fraction ρ max0 .We run BP iterations on single K -SAT instances to ob-tain the values of ρ as a function of β . As initial condi-tions all the cavity distributions q i → a ( s i ) are assumed tobe the uniform distribution over the three states. Some ofthe numerical results are shown in Fig. 2. As expected,the null fraction ρ increases with β for each value ofclause density α . At α = 1 BP is always convergent(both for K = 3 and K = 4) and ρ reaches a limitingvalue ρ max0 as β becomes large. When α increases, how-ever, we find that the BP iteration is convergent only forsufficiently small β values (e.g., up to β ≈ .
46 for the 3-SAT instances at α = 2 and up to β ≈ . α = 5). This non-convergent behavior of BPindicates that ergodicity is broken at high β values andthe system enters into the spin glass phase as the nullfraction ρ exceeds certain threshold value [8, 13, 32].To compute the maximum null fraction ρ max0 within theRS mean field theory, we perform population dynamicssimulations on the BP equations following the standardmethod in the literature [1], and get ensemble-averagedresults on ρ and the entropy density. Again all the cav-ity distributions q i → a ( s i ) are initialized to be the uni-form distribution over s i ∈ {− , , } in this populationdynamics simulation. The ensemble-averaged results of ρ for the random 3-SAT cases are shown in Fig. 2a, andthey are in good agreement with the single-instance re-sults. The relationship between entropy density s and ρ is shown in Fig. 3. The maximum entropy value of en-tropy density decreases with clause density α . For exam-ple at α = 1 the most probable fraction of null variablesis ρ ≈ . K = 3), while this most probable fractiondecreases to ρ ≈ .
08 as the clause density increases tobe α = 3.3 N=10 N=10 N=10 N=3 10 N=3 10 N=3 10 =2=3 =1 (a) =1=5=8 (b) FIG. 2: Fraction ρ of null variables versus the inverse tem-perature β , as estimated by BP iteration computations on asingle K -SAT instance of size N and clause density α (sym-bols), or through RS population dynamics simulations at theensemble level (lines). (a) K = 3, N = 10 or 3 × , and α = 1 , , α s ≈ . K = 4, N = 10 , and α = 1 , , α s ≈ . α the BPiteration becomes non-convergent as β exceeds certain criticalvalue. At these higher β values we can still estimate ρ byaveraging over many non-convergent BP iteration steps (datanot included here). The maximum fraction ρ max0 of null variables also de-creases with α , as is shown in Fig. 4. This shrinkingtrend is fully expected. As the density of clauses in-creases, more and more variables need to actively par-ticipate in satisfying these clauses and so fewer variablescan be spared. The true interesting feature of Fig. 4is that ρ max0 is positive even at the satisfiability phasetransition point α s . This indicates that there are stillan exponential number ( ∼ exp[ N ρ max0 ( α s )]) of satisfyingIsing configurations σ right at the transition point α s .These solutions probably form a dense cluster and theyare quite atypical configurations in the conventional two- s =1=2=3 (a) s =1=5=8 (b) FIG. 3: K -SAT entropy density s versus the fraction ofnull variables ρ , as predicted by the RS population dynamicssimulations. (a) K = 3, and the clause density is α = 1 , , K = 4, and α = 1 , , ρ max0 of null variables istaken to be the value of ρ at which the entropy density s reaches zero. state K -SAT model [29–31]. Our three-state model offersa simple way of emphasizing these solution clusters.In Table I we compare the values of ρ max0 as predictedby the RS mean field theory and the corresponding valuesachieved by the BPD message-passing algorithm of thenext section, for the random 3-SAT problem with clausedensities α close to the satisfiability threshold α s ≈ . ρ pretty close tothe theoretical ρ max0 values. This table also indicates theRS theoretical predictions are quite reasonable.4 ABLE I: Comparison of the predicted value of maximum null-variable fraction ρ max0 by the RS mean field theory and thenull-variable fraction ρ of solutions constructed by the BPD algorithm, for the random 3-SAT problem at clause densities α ∈ [4 . , . β ≈ .
0. Each BPD data is the average value over ten single random 3-SATinstances of size N = 10 . α .
00 4 .
02 4 .
04 4 .
06 4 .
08 4 .
10 4 .
12 4 .
14 4 .
16 4 .
18 4 . . . . . . . . . . . . . . . . . . . . . . . m a x K=3=4.267 (a) m a x K=4=9.930 (b)
FIG. 4: The maximum fraction ρ max0 of null variables ver-sus the clause density α of the random K -SAT problem, aspredicted by the RS population dynamics simulations. (a) K = 3; (b) K = 4. The vertical dashed lines mark the pre-dicted satisfiability phase transition point α s ( ≈ .
267 for K = 3 and ≈ .
930 for K = 4 [6, 7, 11]). III. BELIEF-PROPAGATION GUIDEDDECIMATION
Various local-search heuristic algorithms and message-passing algorithms have been proposed for the K -SATproblem (see, e.g., Refs. [6, 21, 22, 33–35]. Here weadopt the belief-propagation guided decimation (BPD)algorithm, widely used in the K -SAT problem and other optimization problems such as minimum feedback vertexset and minimum vertex covers [22, 36–38], to constructsatisfying configurations with close-to-maximum numberof null variables.The marginal state probabilities q i ( s i ) for all the vari-ables of a given problem instance are estimated by BPiterations at certain fixed value of inverse temperature β , and the algorithm then fix a small fraction r of thevariables i to +1 and − r of the remain-ing variables are fixed to +1 or − i to be fixedin one round of the BPD decimation process are thosewhose marginal probability values q i (+1) or q i ( −
1) arethe largest among all the variables not yet fixed. For sucha variable i , we set its spin to σ i = +1 if q i (+1) > q i ( − σ i = −
1. After variable i isfixed we then simplify the formula by deleting the clausesthat are satisfied by variable i and fix some additionalvariables j if they are required to be fixed by some otherclauses.Some of the results obtained by BPD for single ran-dom 3-SAT instances are shown in Fig. 5. At each valueof clause density α we find that the density ρ of nullvariables obtained by BPD is quite close to the theoret-ically predicted maximum value ρ max0 . This fact is alsodemonstrated in Table I. Another interesting feature isthat the BPD performance is only slightly dependent onthe value of the inverse temperature β .When the clause density α is approaching the satisfi-ability threshold α s , it become more and more difficultfor the BPD algorithm to construct satisfying configura-tions. This happens both for the conventional BPD algo-rithm with only two states [22] and the present one withthree states. We notice that the failure of conventionaltwo-state BPD occurs more abruptly than the presentthree-state BPD algorithm (Fig. 6). For example, whenapplying the two-state BPD algorithm on a 3-SAT in-stance of size N = 10 and clause density α = 4 . =2=3=4=4.1 FIG. 5: The mean fraction ρ of null variables in satisfyingconfigurations of random 3-SAT instances, obtained by theBPD algorithm with fixed inverse temperature β . A singleproblem instance of size N = 10 is used and its clause densityis α = 2 , , , . ρ max0 of nullvariables as predicted by the RS mean field theory. i . If thisfixation does not force any additional spin fixation weaccept it; otherwise we may accept it with some smallprobability, favoring fixation choices which have the leasteffects to the remaining variables. IV. BELIEF-PROPAGATION GUIDEDREINFORCEMENT
We also implement a slightly different message-passing algorithm, belief-propagation guided reinforce-ment (BPR), for constructing maximally flexible satis-fying configurations. The main idea of BPR is the sameas that of BPD but it allows each variable state s i to bemodified multiple times during the search process, so itis much more robust [39]. The most important featureof BPR is the memory effect which is realized by an ex-ternal reinforcement vector ψ i ≡ ( ψ − i , ψ i , ψ +1 i ) on everyvariable i . With this reinforcement factor ψ i , the BPequation (6) is slightly modified as q i → a ( s i ) = ψ s i i e βδ si (cid:81) b ∈ ∂i \ a p b → i ( s i ) (cid:80) s (cid:48) i ψ s (cid:48) i i e βδ s (cid:48) i (cid:81) b ∈ ∂i \ a p b → i ( s (cid:48) i ) . (15) s u cc e ss p r obab ili t y BPD with two states
N=10 N=10 N=10 (a) s u cc e ss p r obab ili t y BPD with three states
N=10 N=10 N=10 (b) FIG. 6: Comparison on the performance of the BPD algo-rithm with two states (a) or three states (b). Three random3-SAT formulas of size N ∈ { , , } are used in thiscomparison, and the first αN clauses of these instances areconsidered. Each data point shows the probability of success-fully constructing a solution by the BPD algorithm among1000 independent trials. The BPD algorithms are not sensi-tive to the inverse temperature β (see Fig. 5), so we simplyset it to a moderate value, e.g., β = 5 . The expression (8) for the marginal probability q i ( s i ) isalso revised to be q i ( s i ) = ψ s i i e βδ si (cid:81) a ∈ ∂i p a → i ( s i ) (cid:80) s (cid:48) i ψ s (cid:48) i i e βδ s (cid:48) i (cid:81) a ∈ ∂i p a → i ( s (cid:48) i ) . (16)We implement BPR as follows. First, the memory vec-tor of each variable i is initialized as ψ i = (1 , , ψ s i i = 1 for s i ∈ {− , , } . Then at each BPR time step t : (1) we first iterate equations (15) and (7) on all thelinks of the factor graph a number of repeats (e.g., tentimes) and; (2) then compute the marginal probabilitydistributions q i ( s i ) following Eq. (16); and (3) then up-date the memory vectors ψ i of all the variables according6 t g ( t ) BPR with three states =4.2=4.14=4.1=4.0=3.8=3.6
FIG. 7: The performance of the BPR algorithm on singlerandom 3-SAT instances of size N = 10 and different fixedclause densities α ranging from 3 . .
2. Each data set(symbols linked by lines) shows the evolution of the number g ( t ) of unsatisfied clauses with the evolution time t of BPR.The inverse temperature is β ≈ . to q i ( s i ). We take the following reinforcement rule: If themost likely state of variable i is s ∗ i ∈ {− , , +1 } , that is,if p i ( s ∗ i ) is larger than the other two probabilities, thena small amount η is added to ψ i ( s ∗ i ) while the other twoelements of ψ i are not changed (we set η = 0 . t we also assign an instanta-neous spin value σ i ( t ) ∈ {− , +1 } to each variable i . Ifthe most likely state of i is predicted to be +1 (or − q i ( s i ), then weset σ i ( t ) = +1 (and respectively, σ i ( t ) = − s i = 0 is the most likely state of i , we set σ i ( t ) = 1 (respectively, σ i ( t ) = −
1) if this variable hasmore links of positive (respectively, negative) couplingconstants. After an instantaneous spin configuration isobtained by this way, we then count the total number ofunsatisfied clauses and denote this number as g ( t ).Some evolution trajectories of g ( t ) with BPR time t are shown in Fig. 7 for a single random 3-SAT formula.When the clause density α is less than 4 .
15 we find that g ( t ) reaches zero in less than 10 steps. When α = 4 . α s ), however, we find that g ( t ) saturatesto a small positive value, meaning the algorithm fails toreach a satisfying solution. V. CONCLUSION
In summary, we designed a three-state spin glass modelto explore the atypical and maximally flexible solutionsof the random K -SAT problem. We succeeded to con-struct satisfying solutions with a close-to-maximum frac-tion ρ of null variables for single random K -SAT in-stances, when the clause density α of these instances arenot too close to the satisfiability phase transition point α s . Our replica-symmetric mean field theoretical resultssuggested that even at α = α s the maximum fraction ρ max0 of null variables is still positive.The RS mean field theory adopted in this work isclearly inadequate when the inverse temperature β be-comes large and the clause density α is approaching α s .The non-convergence of the BP iterations might havecontributed to the failure of the BPD and BPR algo-rithms at the hard region of α ≈ α s . We need to extendthe mean field theory to the 1RSB level and get refinedpredictions on the maximum null-variable fraction ρ max0 and improved message-passing algorithms. It would bevery interesting to see whether the 1RSB theory still pre-dicts a positive ρ max0 at α s . We will address these issuesin a follow-up report. Acknowledgments
One of authors(Han Zhao) thanks Yi-Zhi Xu forhelpful discussions. This work was supported bythe National Natural Science Foundation of ChinaGrants No.11975295 and No.11947302, and the ChineseAcademy of Sciences Grant No.QYZDJ-SSW-SYS018.Numerical simulations were carried out at the HPC clus-ter of ITP-CAS. [1] M. M´ezard and A. Montanari,
Information, physics, andcomputation (Oxford University Press, 2009).[2] P. Cheeseman, B. Kanefsky, and W. Taylor, Where thereally hard problems are, in
Proceedings 12th Int. JointConf. on Artificial Intelligence , IJCAI’91, Vol. 1 (Mor-gan Kaufmann Publishers Inc., San Francisco, CA, USA,1991) pp. 163–169.[3] S. Kirkpatrick and B. Selman, Critical behavior in thesatisfiability of random boolean expressions, Science ,1297 (1994).[4] R. Monasson and R. Zecchina, Entropy of the k-satisfiability problem, Physical Review Letters , 3881(1996).[5] R. Monasson and R. Zecchina, Statistical mechanics of the random k-satisfiability model, Phys. Rev. E , 1357(1996).[6] M. M´ezard, G. Parisi, and R. Zecchina, Analytic and al-gorithmic solution of random satisfiability problems, Sci-ence , 812 (2002).[7] M. M´ezard and R. Zecchina, Random k-satisfiabilityproblem: From an analytic solution to an efficient al-gorithm, Physical Review. E , 056126 (2002).[8] A. Montanari, G. Parisi, and F. Ricci-Tersenghi, Insta-bility of one-step replica-symmetry-broken phase in satis-fiability problems, Journal of Physics A General Physics , 2073 (2003).[9] D. Achlioptas, A. Naor, and Y. Peres, Rigorous locationof phase transitions in hard optimization problems, Na- ure , 759 (2005).[10] M. M´ezard, T. Mora, and R. Zecchina, Clustering of so-lutions in the random satisfiability problem, Physical Re-view Letters , 197205.1 (2005).[11] S. Mertens, M. Mezard, and R. Zecchina, Threshold val-ues of random k-sat from the cavity method, RandomStructures and Algorithms , 340 (2006).[12] F. Krzaka(cid:32)la, A. Montanari, F. Ricci-Tersenghi, G. Semer-jian, and L. Zdeborov´a, Gibbs states and the set of so-lutions of random constraint satisfaction problems, Pro-ceedings of the National Academy of Sciences , 10318(2007).[13] A. Montanari, F. Ricci-Tersenghi, and G. Semerjian,Clusters of solutions and replica symmetry breaking inrandom k-satisfiability, Journal of Statistical MechanicsTheory and Experiment (2008).[14] H.-J. Zhou, t → , 066102 (2008).[15] D. Achlioptas, Solution clustering in random satisfiabil-ity, European Physical Journal B , 395 (2008).[16] E. Maneva and A. Sinclair, On the satisfiability thresh-old and clustering of solutions of random 3-sat formulas,Theoretical Computer Science , 359 369 (2008).[17] H. Zhou and H. Ma, Communities of solutions in singlesolution clusters of a random K -satisfiability formula,Physical Review E (2009).[18] H.-J. Zhou and C. Wang, Ground-state configurationspace heterogeneity of random finite-connectivity spinglasses and random constraint satisfaction problems, J.Stat. Mech.: Theor. Exp P10010 (2010).[19] S. H. Lee, M. Ha, C. Jeon, and H. Jeong, Finite-sizescaling in random k -satisfiability problems, Phys. Rev. E , 061109 (2010).[20] K. Li, H. Ma, and H. Zhou, From one solution of a 3-satisfiability formula to a solution cluster: Frozen vari-ables and entropy, Physical Review E , 031102 (2009).[21] A. Braunstein, M. M´ezard, and R. Zecchina, Surveypropagation: An algorithm for satisfiability, RandomStructures and Algorithms , 201 (2005).[22] A. Montanari, F. Ricci-Tersenghi, and G. Semerjian,Solving constraint satisfaction problems through beliefpropagation-guided decimation, in Proceedings of 45thAnnual Allerton Conference on Communication, Con-trol, and Computing (Curran Associates, Inc., New York,2007) pp. 352–359.[23] R. Marino, G. Parisi, and F. Ricci-Tersenghi, The back-tracking survey propagation algorithm for solving ran-dom k -sat problems, Nature Commun. , 12996 (2016).[24] H. Huang, K. Y. M. Wong, and Y. Kabashima, Entropylandscape of solutions in the binary perceptron problem,J. Phys. A: Math. Theor. , 375002 (2013).[25] H. Huang and Y. Kabashima, Origin of the computa-tional hardness for learning with binary synapses, Phys. Rev. E , 052813 (2014).[26] L. Dall’asta, A. Ramezanpour, and R. Zecchina, Entropylandscape and non-gibbs solutions in constraint satisfac-tion problems, Physical Review E , 031118 (2008).[27] Y. Zeng and H.-J. Zhou, Solution space coupling in therandom k -satisfiability problem, Comm. Theor. Phys. ,363 (2013).[28] F. Krzakala, M. M´ezard, and L. Zdeborov´a, Reweightedbelief propagation and quiet planting for random k -sat,Journal on Satisfiability, Boolean Modeling and Compu-tation , 149 (2014).[29] C. Baldassi, A. Ingrosso, C. Lucibello, L. Saglietti, andR. Zecchina, Subdominant dense clusters allow for simplelearning and high computational performance in neuralnetworks with discrete synapses, Physical Review Letters , 128101.1 (2015).[30] C. Baldassi, A. Ingrosso, C. Lucibello, L. Saglietti, andR. Zecchina, Local entropy as a measure for samplingsolutions in constraint satisfaction problems, Journal ofStatistical Mechanics: Theory and Experiment ,023301 (2016).[31] L. Budzynski, F. Ricci-Tersenghi, and G. Semerjian, Bi-ased landscapes for random constraint satisfaction prob-lems, Journal of Statistical Mechanics: Theory and Ex-periment , 023302 (2019).[32] M. Mzard and G. Parisi, The bethe lattice spin glass re-visited, European Physical Journal B Condensed Matterand Complex Systems , 217 (2001).[33] M. Alava, J. Ardelius, E. Aurell, P. Kaski, S. Krishna-murthy, P. Orponen, and S. Seitz, Circumspect descentprevails in solving random constraint satisfaction prob-lems, Proc. Natl. Acad. Sci. USA , 15253 (2008).[34] C. P. Gomes, H. Kautz, A. Sabharwal, and B. Selman,Satisfiability solvers, in Handbook of Knowledge Repre-sentation , edited by F. van Harmelen, V. Lifschitz, andB. Porter (Elsevier Science, Amsterdam, 2008) Chap. 2,pp. 89–134.[35] F. Baader, I. Horrocks, and U. Sattler, Description log-ics, in
Handbook of Knowledge Representation , edited byF. van Harmelen, V. Lifschitz, and B. Porter (ElsevierScience, Amsterdam, 2008) Chap. 3, pp. 135–180.[36] S. Mugisha and H.-J. Zhou, Identifying optimal targetsof network attack by belief propagation, Phys. Rev. E , 012305 (2016).[37] H.-J. Zhou, Spin glass approach to the feedback vertexset problem, Eur. Phys. J. B , 455 (2013).[38] J.-H. Zhao and H.-J. Zhou, Statistical physics of hardcombinatorial optimization: Vertex cover problem, Chin.Phys. B , 078901 (2014).[39] A. Braunstein and R. Zecchina, Learning by messagepassing in networks of discrete synapses, Phys. Rev. Lett. , 030201 (2006)., 030201 (2006).