A Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries
AA Dichotomy for the Generalized Model Counting Problem forUnions of Conjunctive Queries
Batya Kenig
University of Washington
Dan Suciu
University of Washington
ABSTRACT
We study the generalized model counting problem, defined as fol-lows: given a database, and a set of deterministic tuples, countthe number of subsets of the database that include all determinis-tic tuples and satisfy the query. This problem is computationallyequivalent to the evaluation of the query over a tuple-independentprobabilistic database where all tuples have probabilities in { , , } .Previous work has established a dichotomy for Unions of Conjunc-tive Queries (UCQ) when the probabilities are arbitrary rationalnumbers, showing that, for each query, its complexity is either inpolynomial time or safe in the firstcase, and unsafe in the second case. Here, we strengthen the hard-ness proof, by proving that an unsafe UCQ query remains { , , } . This requires acomplete redesign of the hardness proof, using new techniques. Arelated problem is the model counting problem , which asks for theprobability of the query when the input probabilities are restrictedto { , } . While our result does not extend to model counting forall unsafe UCQs, we prove that model counting is KEYWORDS
Model counting, Tuple-Independent Databases,
ACM Reference Format:
Batya Kenig and Dan Suciu. 2020. A Dichotomy for the Generalized ModelCounting Problem for Unions of Conjunctive Queries. In
Proceedings ofACM Conference (Conference’17).
ACM, New York, NY, USA, 50 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn
Fix a First Order (FO) sentence Q . The generalized model countingproblem for Q is the following: given a database DB , and a subset oftuples D ⊆ DB , count the number of models of Q that are subsetsof DB and include all tuples in D . In the model counting problem ,we set D = ∅ , and the problem is to count the number of modelsof Q that are subsets of DB .An equivalent formulation to the generalized model countingproblem is to state it as a special case of the query evaluationproblem on Tuple-Independent probabilistic Database (TID) [10]. Inthat setting, each tuple in the domain has an associated probability, Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected].
Conference’17, July 2017, Washington, DC, USA © 2020 Association for Computing Machinery.ACM ISBN 978-x-xxxx-xxxx-x/YY/MM...$15.00https://doi.org/10.1145/nnnnnnn.nnnnnnn and the problem is to compute the probability that a query Q istrue over a possible world obtained by including randomly andindependently each tuple in the domain. It is not hard to see thatthe generalized model counting problem corresponds to restrictingthe probabilities to { , / , } , while the model counting problemcorresponds to the restricting them to { , / } .In this paper we will restrict the query Q to be a Union of Con-junctive Queries or, equivalently, to a dual of a UCQ (see below).The following dichotomy is known [4]: either Q is computable inPTIME over any TID, or the query evaluation problem is provably safe , in the second case it is called unsafe . Moreover, one can decideby static analysis over the expression of the query Q whether it issafe or unsafe. In this paper we strengthen that result by proving adichotomy theorem for the generalized model counting problem:for any UCQ Q , if Q is safe then the generalized model countingproblem is in PTIME, and if Q is unsafe then the problem is Q is safe, then itcan be evaluated in PTIME over any TID, even if all probabilitiesare in { , / , } . This paper is about the other side of the proof:if Q is unsafe, then we show that the query evaluation problemis { , / , } . As we explain below, some parts of the hardness proofin [4] (namely Sections 6 and 7) continue to hold even when theprobabilities are restricted to { , / , } , but the most difficult part(Section 8) requires an entirely new proof. The reason is that theprevious proof [4] relies on multiple distinct probabilities in ( , ) ,a number that depends on the size of the database. In our paper wedevelop entirely new proof techniques for this most critical pieceof the hardness proof. Before we present the technical material, wecomment on several aspects of our contributions. If Q is an unsafe query, then the evaluation problem over arbitraryTIDs is any query Q can be evaluated in PTIME if the input TID has boundedtree-width, and Van den Broeck et al. [3, 5] prove that any queryin FO can be evaluated in PTIME if the TID is symmetric . Thisleads to a natural question: could the query evaluation problembecome easier if we restrict what probabilities can appear in theTID? Our result answers this negatively: query evaluation remainshard even if the probabilities are restricted to { , / , } . In fact,it remains hard even if the probabilities are restricted to { , c , } ,where c ∈ ( , ) is any fixed constant. The only property needed inour proof is the following simple fact: a r X i v : . [ c s . D B ] A ug onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Lemma 1.1.
Let f ( x , . . . , x n ) be a multivariate polynomial, notidentically 0, where each variable has degree ≤ . Let c , c , c ∈ R be three distinct constants. Then there exists an assignment θ of thevariables x , . . . , x n with values in { c , c , c } , such that f [ θ ] (cid:44) . The proof is by induction on n : if f = дx n + hx n + k , where д , h , k are multi-variate polynomials in the other variables, at leastone not identically 0, then there exists an assignment θ s.t. at leastone of д [ θ ] , h [ θ ] , k [ θ ] is (cid:44)
0; since a degree 2 polynomial in x n has at most 2 roots, there exists a “non-root” c i ∈ { c , c , c } , thus f [ θ ; x n : = c i ] (cid:44)
0. While our hardness proof is complex, the onlyprobabilities that we need to set in the proof are either 0 or 1 ornon-roots of a degree 2 multivariate polynomial, where we useLemma 1.1. Thus, if Q is unsafe, then it remains { , c , } for some fixed c ∈ ( , ) . This ruins anyhope of improving query evaluation by restricting the probabilityvalues. Our result states that, if a query is unsafe, then the evaluation prob-lem over TID’s with probabilities in { , / , } is { , / } ? This corresponds tothe model counting problem, and is a natural setting in probabilisticdatabases, where tuples not present in the database have probability0, while those in the database could be associated with probability1 /
2. Amarilli and Kimelfeld [2] study precisely this problem andprove a dichotomy for the model counting problem for conjunctivequeries without self-joins. We also prove final, type I queries. This complements the result proven by Amarilliand Kimelfeld. We leave open the question whether any UCQ ishard for model counting. Thus, with the exception of forbidden typeI queries, in this paper we study the generalized model countingproblem for a query Q , which we denote by GFOMC ( Q ) . We arguenext that this is, in fact, a more natural problem than the modelcounting problem. The dual of a First Order query Q is obtained by switching ∃ and ∀ , and switching ∨ and ∧ . Many problems over Boolean formulasare closed under duals, for example the satisfiability for a class ofBoolean formulas is in PTIME iff the validity for the class of duals isalso in PTIME; similarly, model counting has the same complexityfor a class of Boolean formulas and for its dual. The same propertyholds for GFOMC . The query evaluation problem for Q on a TID isessentially the same as the evaluation problem of its dual Q ′ on theTID where each probability p is replaced by p ′ def = − p , becausePr ( Q ) = − Pr ′ ( Q ′ ) . Thus, GFOMC ( Q ) and GFOMC ( Q ′ ) have the samecomplexity. However, duality does not preserve the complexity formodel counting: if Q is hard on TIDs with probabilities { , / } , itdoesn’t follow that is dual Q ′ is also hard on TIDs with probabilitiesin { , / } . For that reason, we find the generalized model countingproblem a more robust notion than the model counting problem,and will focus on it in this paper.Throughout the paper we will discuss duals of UCQs insteadof UCQs. We denote the class of sentences that are duals of UCQsby ∀ CNF . Since model counting for UCQs correspond to restricting probabilities to { , / } , we define model couning for ∀ CNF to meanrestricting the probabilities to { / , } . The hardness proofs in [4] follows a simple high level structure.First, show that if Q is unsafe, then it can be rewritten to a simplerquery Q ′ , which is still unsafe, such the computation problem forPr ( Q ) can be reduced in polynomial time to that for Pr ( Q ′ ) ; inparticular, hardness of Q ′ implies hardness of Q . Second, providea direct Q ′ in the simplerclass. Usually, the polynomial time reduction consists of addingto the database tuples with probabilities 0 or 1, never differentprobability values (see [4, Definition 4.13]). For example, to provethat ∀ x ∀ y ( R ( x ) ∨ S ( x , y ) ∨ T ( y ) ∨ A ( x )) ∧ ∀ yB ( y ) is hard it sufficesto set all probabilities in A to 0 and all probabilities in B to 1, andobtain the query ∀ x ∀ y ( R ( x ) ∨ S ( x , y ) ∨ T ( y )) then prove that thelatter is hard. A forbidden query is an unsafe query where no morerewriting to a simpler unsafe query is possible.The first step of the proof in [4] also applies to GFOMC . Thus,in order to prove that
GFOMC ( Q ) is hard for any unsafe query Q ,it suffices to prove it for “forbidden” queries Q . The definition offorbidden queries in [4] is quite technical. In this paper we avoidit, and instead prove hardness directly for a slightly larger class ofqueries Q , called final queries. (We will discuss forbidden queries inSec. B.) More precisely, the queries we consider are called bipartitequeries , they have only two variables, x , y , and three kinds of atoms:two unary atoms R ( x ) , T ( y ) , and an arbitrary number of binaryatoms S j ( x , y ) , j = , , . . . , and they are unsafe iff the two unaryatoms are connected by the clauses of Q . If seeting any atom to 0 orto 1 makes the query safe, then we call it final . Our main result inthis paper consists of proving that, for every final query, GFOMC ( Q ) is GFOMC
A positive 2CNF formula is Φ = (cid:211) ( i , j )∈ E ( X i ∨ X j ) . The modelcounting problem, denoted E is bipartite,in which case the problem is denoted GFOMC ( Q ) .Our reduction is a polynomial-time reduction, also called Cook-reduction (as opposed to the many-one polynomial time reduc-tion, or
Karp-reduction ). Specifically, we construct a sequence ofdatabases DB , DB , . . . with probabilities in { , / , } , and showhow to compute Φ from Pr ( Q ) , Pr ( Q ) , . . . To compute Φ , weneed to solve a linear system of equations, and the crux of thehardness proof consist of showing that the matrix of this systemis non-singular. We call this matrix the big matrix , since its size ispolynomial in the size of Φ .This is the place where our proof diverges from that in [4]. There,the databases DB , DB , . . . were isomorphic, but used differentprobabilities, hence the need to use very many distinct probabilityvalues. In contrast, we construct databases whose probabilities areonly among { , / , } , and, instead, we vary the number of tuples.Each database DB i consists of disjoint paths (formal definition in Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Sec. 3) of lengths that depends on i . Instead of varying the proba-bility values in ( , ) as in [4], here we fix the probability values in { , / , } and vary the length of the paths. We show that the non-singularity of the big matrix follows byproving that a certain “small matrix”, A , is non-singular. The smallmatrix is a 2 × i in DB i we gain more informationabout Φ from probability Pr ( Q ) on DB i . The small matrix dependsonly on the query Q , more precisely, on its the arithmetizationof its lineage Y on one link of the path. The arithmetization ofa Boolean formula Y is a multilinear polynomial y that agreeswith Y at all points in { , } n . For example, if Q = ∀ x , ∀ y ( R ( x ) ∨ S ( x , y )) ∧ ( S ( x , y ) ∨ T ( y )) , then the lineage is Y = ( R ∨ S ) ∧ ( S ∨ T ) ,and its arithmetization is y ( r , s , t ) = rt + s − rst . Equivalently, thearithmetization is the formula for the probability of Y ; if Pr ( R ) = Pr ( S ) = Pr ( T ) = /
2, then Pr ( Y ) = r ( / , / , / ) = /
8. Thesmall matrix A is (cid:32) y [ r = , s = ] y [ r = , s = ] y [ r = , s = ] y [ r = , s = ] (cid:33) . At its core,our hardness proof relies on the following connection between logicand algebra: the small matrix is non-singular iff the bipartite query Q connects the atoms R and S . We state here the formal lemma thatcaptures this connection.Lemma 1.2. Let y be the arithmetization of a Boolean formula Y ,and let R , T be two Boolean variables. Denote the following matrix ofpolynomials: y def = (cid:34) y y y y (cid:35) (1) where y def = y [ r : = t : = ] is obtained by setting r , t to 0,and similarly for y , y , y . Then the following are equivalent:(1) Y disconnects R , T , i.e. Y ≡ F ∧ G such that R ∈ Vars ( F ) , T ∈ Vars ( G ) , Vars ( F ) ∩ Vars ( G ) = ∅ . (2) det ( y ) ≡ . The direction (1) ⇒ (2) is immediate, because y factorizes as y = f · д where r occurs only in f and t only in д , thus y ij = f i · д j and it follows that det ( y ) ≡
0. For (2) ⇒ (1) assume that y · y ≡ y · y . Assume w.l.o.g. that gcd ( y , y , y , y ) =
1, whichalso implies gcd ( y , y , y ) = y is a multi-linearpolynomial . Then define f i def = gcd ( y i , y i ) and д j = gcd ( y j , y j ) ,and we have y ij = f i · д j for all i , j = ,
2, because gcd ( f , д ) = gcd ( y , y , y ) =
1, and similarly for gcd ( f i , д j ) =
1. The claimfollows from y = ( − r )( − t ) y + ( − r ) ty + r ( − t ) y + rty = (( − r ) f + r f )(( − t ) д + tд ) def = f · д , which implies condition (1) bydefining F , G the Boolean formulas associated to f , д respectively.Finally, our proof falls into place by the following argument. Let Q be a bipartite query. If Q connects the atoms R ( x ) , T ( y ) , thenits lineage Y is connected, and thus det ( y ) (cid:46)
0. By Lemma 1.1, If h def = gcd ( y , y , y , y ) (cid:44) , then we define y ′ ij def = y ij / h , obtain y ′ ij = f ′ i · д j ,and define f i def = f ′ i h . Assuming p = gcd ( y , y , y ) then p | y · y ≡ y · y , but p ̸ | y , implying p | y . there exist probability values in { , / , } that ensure that thesmall matrix A is non-singular. Then, the big matrix is non-singular,and we can compute in polynomial time Φ from the probabilitiesPr ( Q ) , Pr ( Q ) , . . . , completing the reduction. When a bipartite query Q contains the atoms R ( x ) , T ( y ) , then we callit a type I query. Otherwise, we call it a type II query. Our discussionso far has been restricted to type I queries; we prove their hardnessin main body of the paper. We discuss type II queries, and provetheir hardness in the appendix. The proof for Type II queries ismore complex than for Type I queries. While the two proofs sharemany common techniques, they are sufficiently distinct to justifya completely separate proof for Type II queries. In particular, theproof for type I queries is by reduction from In Section 2 we provide background on final queries, and formallystate the problem and main result. We prove the hardness for queriesof type I in Section 3, and defer the proof for type II queries to theAppendix.
Fix a finite domain
Dom and let
Tup ( Dom ) denote the set of groundtuples consisting of relation names from some fixed vocabulary,and constants from Dom . With some abuse, we write
Tup insteadof
Tup ( Dom ) when the domain is clear from the context.A tuple-indepepndent probabilistic database is a pair ∆ = ( Dom , p ) where p : Tup → [ , ] associates a probability to each tuple.With some abuse, we simply say that ∆ is a probabilistic database .Given a Boolean query Q , its probability is defined as Pr ( Q ) def = (cid:205) W ⊆ Tup : W | = Q Pr ( W ) , where the probability of a world W is Pr ( W ) def = (cid:206) t ∈ W p ( t )× (cid:206) t ∈ Tup − W ( − p ( t )) . The probabilistic query evaluationproblem , PQE ( Q ) , is the following: given a probabilistic database ( Dom , p ) , where p takes rational values, compute Pr ( Q ) . The fol-lowing dichotomy theorem was shown in [4]:Theorem 2.1. [4] Let Q be any UCQ, then one of the followingholds: either PQE ( Q ) is in PTIME, or PQE ( Q ) is Q called safety suchthat PQE ( Q ) is in PTIME when Q is safe, and is Q isunsafe. For the purpose of this paper we do not need the general defini-tion of safety, and will omit it, except for the special case of bipartitequeries defined below. If Q is not safe, then we call it unsafe .The generalized model counting problem for Q , denote GFOMC ( Q ) ,is the following restriction: given a probabilistic database ( Dom , p ) where p ( t ) ∈ { , / , } for every tuple t , compute Pr ( Q ) . Obviously,if PQE ( Q ) is in PTIME, then so is GFOMC ( Q ) . We prove that theconverse holds too:Theorem 2.2. For any unsafe UCQ Q , GFOMC ( Q ) is / is replaced by someconstant probability c ∈ ( , ) . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu The high level structure of the proof is similar to that in [4]. Start-ing with an unsafe query Q , first simplify it using simple rewriterules, as long as the query is still unsafe, until one reaches an unsafequery where every further simplification is safe. Second, provethat each such simplified query Q (called in [4] a forbidden query), GFOMC ( Q ) is ∀ CNF. We briefly review ∀ CNF here. A clause is a disjunction of atoms, C ≡ R ( x )∨ R ( x )∨· · · ; we note that thisis the dual notion of a conjunctive query. A homomorphism C → C ′ is a function Vars ( C ) → Vars ( C ′ ) that maps every atom in C to anatom in C ′ . We say that C is minimized if every homomorphism C → C is a bijection. It is known that every clause is equivalent toa minimized clause . A ∀ CNF formula is a conjunction of clauses, Q ≡ C ∧ C ∧ · · · ; this is the dual of a UCQ. If there exists ahomomorphism C i → C j then C j is redundant, and can be removedfrom Q . In this paper we always assume, unless otherwise stated,that all clauses are minimized, and there is no redundant clause.All logical variables are universally quantified, and we will freelyswitch between prenex normal form of the entire sentence, or ofeach clause individually, e.g. ∀ x ∀ y ( C ∧ C ) v.s. ∀ x ∀ yC ∧ ∀ x ∀ yC .The lineage of a ∀ CNF query Q on a tuple independent database ∆ = ( Dom , p ) is the propositional formula Φ ∆ ( Q ) computed asusual, by induction on the structure of Q . We assume that eachtuple in the domain Dom is associated with a Boolean variable,thus Φ ∆ ( Q ) is a Boolean function over the tuples. We remark that Φ ∆ ( Q ) is a Boolean CNF whose size is polynomial in the size of thedomain Dom .We consider a restricted vocabulary consisting of two unarysymbols R , T , and one or more binary symbols S , S , . . . . We call aprobabilistic database ∆ = ( Dom , p ) bipartite if Dom is the disjointunion
Dom = U ∪ V , and every tuple t with probability p ( t ) (cid:44) t = R ( u ) or t = S j ( u , v ) or t = T ( v ) for u ∈ U and v ∈ V . Wedenote by GFOMC bi ( Q ) the GFOMC problem where the probabilisticdatabase is restricted to be bipartite.We define next a bipartite query. It has only two variables x , y ,its atoms are restricted to be R ( x ) , or T ( y ) , or S j ( x , y ) for some j = , , . . . Denote by S J ( x , y ) def = (cid:212) j ∈ J S j ( x , y ) . Definition 2.3.
We define the following types of sentences: • A left clause of Type I is ∀ x ∀ y ( R ( x ) ∨ S J ( x , y )) , where J (cid:44) ∅ . • A left clause of Type II is ∀ x (cid:16)(cid:212) m ℓ = ( ∀ yS J ℓ ( x , y )) (cid:17) where ∅ (cid:44) J , . . . , J m ⊆ [ p ] , and m > • A middle clause is ∀ x ∀ yS J ( x , y ) for J (cid:44) ∅ . • A right clause of Type I is ∀ y ∀ x ( S J ( x , y ) ∨ T ( y )) , where J (cid:44) ∅ . This follows from the same property for conjunctive queries. Φ ∆ ( ∀ xQ ) def = (cid:219) u ∈ Dom Φ ∆ ( Q [ u / x ]) Φ ∆ ( Q ∧ Q ) def = Φ ∆ ( Q )∧ Φ ∆ ( Q ) Φ ∆ ( Q ∨ Q ) def = Φ ∆ ( Q )∨ Φ ∆ ( Q ) • A right clause of Type II is ∀ y (cid:16)(cid:212) n ℓ = ( ∀ xS J ℓ ( x , y )) (cid:17) where ∅ (cid:44) J , . . . , J n ⊆ [ p ] , and n > A , B ∈ { I , II } , we define a bipartite query of type A − B to be a ∀ CNF query Q def = Q left ∧ Q middle ∧ Q right where Q left is aconjunction of left clauses of type A , Q middle is a conjunction ofmiddle clauses, and Q right is a conjunction of right clauses of type B . An example of a left clause of type I is ∀ x ∀ y ( R ( x ) ∨ S ( x , y ) ∨ S ( x , y )) . An example of a left clause of type II is ∀ x ( ∀ yS ( x , y ) ∨ ∀ yS ( x , y )) . For Type II clauses the term “clause” is used with someabuse, since this sentence is not in prenex normal form; the prenexnormal form of our example is ∀ x ∀ y ∀ y ( S ( x , y ) ∨ S ( x , y )) .Definition 4.14 in [4] calls a UCQ query safe if it satisfies a certainsyntactic condition. We review below the safety definition only forthe special case of bipartite queries and, for the intuition behindthis definition, we make two observations. First, let Q = Q left ∧ Q middle be a bipartite query without any right clauses. Then one cancompute Pr ( Q ) in polynomial time on a probabilistic database ∆ = ( Dom , p ) , in other words Q is safe . Indeed, Q ≡ (cid:211) a ∈ Dom Q [ a / x ] , x occurs in each atom of Q , hence the events Q [ a / x ] , Q [ a / x ] , . . . are independent, and therefore Pr ( Q ) = (cid:206) a ∈ Dom Pr ( Q [ a / x ]) . Itis then easy to check that each Pr ( Q [ a / x ]) can be computed inpolynomial time. Second, suppose Q is a bipartite query that canbe partitioned into Q ≡ Q ′ ∧ Q ′′ , where Symb ( Q ′ ) ∩ Symb ( Q ′′ ) = ∅ , Q ′ has no right clauses, and Q ′′ has no left clauses. Then Q is againsafe, because Pr ( Q ) = Pr ( Q ′ ) · Pr ( Q ′′ ) . This justifies: Definition 2.4. A bipartite query is unsafe if it contains a leftclause C and a right clause C k that are connected by a path, i.e.there exists a sequence C , C , . . . , C k , called a left-to-right path ,such that every two consecutive clauses share an atom. The length of the unsafe query is the minimal k for which there exists a left-to-right path of length k .Let A , B be two decision problems. A polynomial time many-onereduction from A to B , in notation A ≤ Pm B , is a polynomial timecomputable function f such that, for every instance x of A , f ( x ) is an instance of B s.t. x holds iff f ( x ) holds. A polynomial timereduction from A to B , in notation A ≤ P B , is a polynomial timealgorithm for solving instances of A given an oracle for solvinginstances of B . The first part of the proof of Theorem 2.2 is given bythe following theorem, where H = ∀ x ∀ y ( R ( x ) ∨ S ( x , y ) ∨ T ( y )) .Theorem 2.5. Let Q be a ∀ CNF query that is unsafe (according toDefinition 4.14 in [4]). Then, either
GFOMC bi ( H ) ≤ Pm GFOMC bi ( Q ) , orthere exists a bipartite, unsafe query Q ′ such that GFOMC bi ( Q ′ ) ≤ Pm GFOMC bi ( Q ) . Proof. (Sketch) The proof follows directly from [4], more pre-cisely from Theorems 6.3 and 7.3; note that a bipartite query iscalled a two-leveled query in [4]. The only tool used in thoseproofs is Lemma 4.17, continues to hold if all probabilities are in { , / , } . □ It has only unary atoms.
Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Thus, in order to prove Theorem 2.2, it suffices to show that
GFOMC bi ( H ) is GFOMC bi ( Q ) is Q . Hardness for H is already shown in [4], be-cause that proof only uses probabilities in { , / , } ; furthermore,Amarilli and Kimelfeld [2] strengthened this result by showing thatmodel counting for H is H any further in this paper; we only consider bipartite queries.Next, we need a technical lemma, whose proof is in the appendix.Lemma 2.6. Let Q be a bipartite, unsafe query of type A − B , with A , B ∈ { I , II } , and of length k . Then there exists a bipartite, unsafequery Q ′ of type A − A of length ≥ k such that GFOMC bi ( Q ′ ) ≤ Pm GFOMC bi ( Q ) . We introduce now two simplification rules for queries (these aresubsets of the rules in Definition 4.13 [4]). Fix a bipartite query Q ,and let S be any relational symbol. We denote by Q [ S : = false ] the query obtained by replacing every occurrence of the atom S by false . Similarly, Q [ S : = true ] is obtained by replacing S by true .We sometimes abbreviate these rewritings by Q [ S : = ] , Q [ S : = ] .As discussed earlier, we always assume that the rewritten query isreduced, by minimizing its clauses and removing redundant clauses.Lemma 2.7. Let Q be a bipartite query, and let Q ′ be either Q [ S : = ] or Q [ S : = ] . Then (1) GFOMC bi ( Q ′ ) ≤ Pm GFOMC bi ( Q ) , (2) Q and Q ′ have the same types (I or II), (3) if Q ′ is unsafe, then so is Q (but theconverse does not hold in general), and (4) the length of Q ′ is ≥ thelength of Q . The proof is immediate and omitted. This justifies:
Definition 2.8. A final query is a bipartite, unsafe query Q withthe following property: for any symbol S of Q , both Q [ S : = ] and Q [ S : = ] are safe queries.Intuitively, if we want to prove that an unsafe query Q is Q ′ = Q [ S : = ] or Q ′ = Q [ S : = ] andprove that Q ′ is final query is one where no furthersimplifications are possible. A “forbidden query” (Definition 7.2in [4], and reviewed here in Sec. B) is defined similarly, but considerssome additional simplifications, thus, every forbidden query is final,but the converse does not hold. In this paper we only discuss finalqueries.We say that Q is disconnected if Q ≡ Q ′ ∧ Q ′′ , where Q ′ , Q ′′ use disjoint sets of symbols. Every final query is connected. Indeed,assuming the contrary, one of Q ′ , Q ′′ must must be unsafe, and,assuming Q ′ is unsafe, we can set to true all symbols S occur-ring in Q ′′ and obtain the simpler query Q ′ which is still unsafe,contradicting the fact that Q is final.Finally, we can state now the main technical result of this pa-per, which immediately implies Theorem 2.2. Recall that the modelcounting problem for UCQ’s is defined as PQE ( Q ) where the prob-abilities are restricted to { , / } . Since our discuss is for ∀ CNFrather than UCQ’s, we define the model counting problem , FOMC ( Q ) ,as PQE ( Q ) where the probabilities are restricted to { / , } . In thispaper we prove:Theorem 2.9. (1) If Q is a final query of type I, then FOMC bi ( Q ) is Q is a final query of type II of length ≥ , then GFOMC bi ( Q ) is In the rest of the paper we prove Theorem 2.9. We end this sectionby showing how this theorem proves our main result, Theorem 2.2,which we prove for ∀ CNF queries rather than UCQs. Let Q beany unsafe ∀ CNF query. By Theorem 2.5, either
GFOMC bi ( H ) ≤ Pm GFOMC ( Q ) , in which case the theorem follows from the fact that GFOMC bi ( H ) is GFOMC bi ( Q ′ ) ≤ Pm GFOMC ( Q ) for somebipartite, unsafe query Q ′ of some type A − B . If A is I, then byLemma 2.6 GFOMC bi ( Q ′′ ) ≤ Pm GFOMC bi ( Q ′ ) for some bipartite, unsafequery Q ′′ of type I-I: w.l.o.g. we may assume that Q ′′ is final (byLemma 2.7), then the result follows from Theorem 2.9 (1). If A is II,then we apply Lemma 2.6 three times, to obtain a bipartite, unsafequery Q ′′ of type II and of length ≥
8, such that
GFOMC bi ( Q ′′ ) ≤ Pm GFOMC bi ( Q ′ ) . As before we can assume w.l.o.g. that Q ′′ is final, hencethe rsult follows from Theorem 2.9 (2). In this section we prove the first item of Theorem 2.9. A Positive2CNF, or P2CNF, is a formula Φ = (cid:211) ( i , j )∈ E ( X i ∨ X j ) with n variablesand | E | = m clauses. The problem “given a P2CNF Φ , compute thenumber of satisfying assignments Φ ” is denoted For every final query Q of type I, ≤ P FOMC bi ( Q ) . Let θ : { X , . . . , X n } → { , } n be an assignment to Φ ’s vari-ables. We define its signature to be the mapping k ( θ ) : { , } →{ , . . . , m } : k ( θ ) def = ( k ( θ ) , k ( θ ) , k ( θ ) , k ( θ )) , where k ab ( θ ) def = |{( i , j ) ∈ E : θ ( X i ) = a , θ ( X j ) = b }| ab ∈ { , } Thus, k is the number of clauses where both X i , X j are false ,and k the number of clauses where both are true . We assumethat E is a directed graph, and contains at most one of ( i , j ) or ( j , i ) for all i (cid:44) j . Thus, k and k may be different, and their sum k ,
10 def = k + k represents the number of clauses with exactly1 variable set to true , and we write k ′ ( θ ) def = ( k , k , , k ) forthe undirected signature. For any vector k = ( k , k , k , k ) , its count , k , is the number of assignments θ : { X , . . . , X n } → { , } n with signature k , and k ′ is the similar undirected counts : ∀ k ∈ { , . . . , m } : k def = |{ θ : k ( θ ) = k }| (2) ∀ k ′ ∈ { , . . . , m } : k ′ def = |{ θ : k ′ ( θ ) = k ′ }| (3)Thus, there are ( m + ) counts, and ( m + ) undirected counts, ofwhich at most ( m + ) are non-zero, because, for any signature, k + k , + k = m .To prove Theorem 3.1, we start from a P2CNF Φ with m clauses,and construct ( m + ) bipartite TIDs ∆ . Then, we describe an al-gorithm that, using all probabilities Pr ∆ ( Q ) , computes all ( m + ) consistent undirected counts k ′ (the others are = Φ = (cid:205) k ′ : k ′ = k ′ . This strategy requires computing a polynomial num-ber of counts. The crux of the proof consists in constructing thedatabases ∆ to make this computation possible. Each such data-base is a union of blocks ; we describe next how to compute Pr ∆ ( Q ) when the TID ∆ is a union of blocks. In Section 3.2 we present the onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu reduction from ∆ ( Q ) allow us to compute the counts k . Throughout this section we fix a final query Q of type-I: Q = ∀ x ∀ y ( ℓ (cid:219) i = ( R ( x )∨ S J i ( x , y )))∧ C ( x , y )∧( r (cid:219) k = ( T ( y )∨ S J k ( x , y ))) (4)Its vocabulary is R = { R ( x ) , S ( x , y ) , . . . , S p ( x , y ) , T ( y )} . Recallthat a bipartite TID ∆ = ( Dom , p ) has a bipartite domain Dom = U ∪ V . Definition 3.2. A block B ( u , v ) is a bipartite TID with two distin-guished constants u , v . We call u , v the end points of B ( u , v ) , andcall any other constant occurring in B ( u , v ) an internal point .In this section we assume that both end-points u , v of a block arein its left domain, i.e. there are atoms R ( u ) , R ( v ) but not T ( u ) , T ( v ) (more precisely, the latter have probabilities 1); in the appendixwe allow u , v to be on the left or right. Two blocks B ( u , v ) and B ( u ′ , v ′ ) are called disjoint if they share at most their end points: Dom ( B ( u , v ))∩ Dom ( B ( u ′ , v ′ )) ⊆ { u , v }∩{ u ′ , v ′ } . Therefore, whenthe blocks are disjoint, they can only share tuples R ( u ) or R ( v ) respectively. Definition 3.3.
We denote by Y ( u , v ) the lineage of Q over thebipartite TID B ( u , v ) : Y ( u , v ) def = Φ B ( u , v ) ( Q ) . For any pair ( a , b ) ∈{ , } we define Y ab ( u , v ) def = Y ( u , v )[ R ( u ) : − a , R ( v ) : − b ] (5)In other words, Y ab is the lineage Y ( u , v ) where we substitutethe Boolean variables R ( u ) , R ( v ) with the values a , b respectively.Theorem 3.4. Let U be a domain of size n , and let ∆ = (cid:208) u , v ∈ U B ( u , v ) be a bipartite TID that is a disjoint union of blocks. Assume its bipar-tite domain is V ∪ V s.t. U ⊆ V , and that all tuples { R ( u ) | u ∈ U } have probability / . Then: Pr ∆ ( Q ) = n (cid:213) θ : U →{ , } (cid:214) u , v ∈ U Pr ( Y θ ( u ) θ ( v ) ( u , v )) (6)Proof. Since Φ ∆ ( Q ) = (cid:211) u , v ∈ U Y ( u , v ) and any two Booleanformulas Y ( u , v ) , Y ( u ′ , v ′ ) share at most the boolean variables R ( u ) or R ( v ) (when u = u ′ or v = v ′ ), we have:Pr ∆ ( Q ) = Pr ( Φ ∆ ( Q )) = Pr ( (cid:219) u , v ∈ U Y ( u , v )) = n (cid:213) θ : U →{ , } n Pr ( (cid:219) u , v ∈ U Y ( u , v )[ R ( u ) : − θ ( u ) , R ( v ) : − θ ( v )]) = n (cid:213) θ : U →{ , } n Pr ( (cid:219) u , v ∈ U Y θ ( u ) θ ( v ) ( u , v )) = n (cid:213) θ : U →{ , } n (cid:214) u , v ∈ U Pr ( Y θ ( u ) θ ( v ) ( u , v )) (7)where (7) follows because the Boolean functions Y θ ( u ) θ ( v ) ( u , v ) and Y θ ( u ′ ) θ ( v ′ ) ( u ′ , v ′ ) are are over disjoint sets of Boolean variables,hence they are independent. □ Block TID associated with a graph.
Let G ( U , E ) be a directed graphwhere | U | = n and, for each u (cid:44) v at most one of ( u , v ) , ( v , u ) is in E . We associate every edge ( u , v ) ∈ E with a block B ( u , v ) , and de-fine the block-disjoint TID associated with G : ∆ = (cid:208) u , v ∈ U B ( u , v ) ,where for every non-edge ( a , b ) (cid:60) E we define B ( a , b ) to be thetrivial block consisting of all tuples S ( a , b ) , . . . , S p ( a , b ) with prob-ability 1. In this setting, by (6) we have:Pr ∆ ( Q ) = n (cid:213) θ : U →{ , } n (cid:214) ( u , v )∈ E Pr ( Y θ ( u ) θ ( v ) ( u , v )) (8) FOMC ( Q ) In this section we show that using an oracle to
FOMC ( Q ) over a blockdisjoint TID ∆ allows us to construct a system of linear equations M whose solution allows us to solve ∆ , which guarantee that M is non-singular, and thus has a unique solution. In section 3.3 we showhow to construct the blocks such that these conditions hold.Fix a final query Q of type-I, and an instance of Φ = (cid:211) ( i , j )∈ E ( X i ∨ X j ) . We let U = { , . . . , n } and | E | = m . We create m blocks B ( u , v ) , for all ( u , v ) ∈ E and define ∆ = (cid:208) ( u , v )∈ E B ( u , v ) .The blocks will be isomorphic, and therefore, the following quanti-ties do not depend on u , v : y ab def = Pr ( Y ab ( u , v )) ( u , v ) ∈ E and ab ∈ { , } A block B ( u , v ) is symmetric if Pr ( Y ab ( u , v )) = Pr ( Y ba ( u , v )) . In ourreduction, we construct symmetric blocks, and thus can assumethat y ab = y ba .Consider an assignment θ occuring the sum (cid:205) θ of (8), and let k ( θ ) = { k ( θ ) , . . . , k ( θ )} be its signature. The factor y ab occursprecisely k ab times in the product, hence its exponent is k ab , i.e. thenumber of directed edges ( u , v ) ∈ E where θ ( u ) = a , and θ ( v ) = b .Therefore, expression (8) becomes:Pr ∆ ( Q ) = n (cid:213) k + k + k + k = m k (cid:0) y (cid:1) k (cid:0) y (cid:1) k (cid:0) y (cid:1) k (cid:0) y (cid:1) k (9)Our unknowns are k ; there is one unknown for every signature k .Since the blocks are symmetric, then y = y , and we may write:Pr ∆ ( Q ) = n (cid:213) k + k + k + k = m k (cid:0) y (cid:1) k (cid:0) y (cid:1) k + k (cid:0) y (cid:1) k = n (cid:213) k + k + k = m k ′ (cid:0) y (cid:1) k (cid:0) y (cid:1) k , (cid:0) y (cid:1) k (10)where k , def = k + k . Eq. (10) has ( m + ) unknowns k ′ , withcoefficients (cid:0) y (cid:1) k (cid:0) y (cid:1) k (cid:0) y (cid:1) k , . Intuitively, in order to con-struct a system of linear equations that will allow us to solve forthe ( m + ) unknowns k ′ , we need to create ( m + ) linearlyindependent coefficients y k y k , y k . We do this by constructing ( m + ) distinct, block-disjoint TIDs. The TIDs we construct in-duce probabilities y , y , y that meet three important conditions.These conditions then allow us to prove that the resulting systemof linear equations, denoted M , is invertible, and thus has a singlesolution.In the rest of this section we present the three conditions on theprobabilities y ab , and prove that if they are met then the resulting Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA system of ( m + ) linear equations is non-singular. Since the sizeof the system is polynomial in the size of the data, then this allowsus to solve for the counts k ′ in PTIME, thus proving hardness. Infact, we prove something more general that we also need in thehardness proof of Type-II queries.Let h ≥ λ , λ ∈ R , and a i , b i , i = , . . . , h be real numbers satisfying the following conditions: λ (cid:44) ± λ and λ (cid:44) , λ (cid:44) b i (cid:44) ∀ i = , h (12) a i b j (cid:44) a j b i i (cid:44) j (13)Let p = { p , . . . , p h } be h natural numbers where p i ≥
1. For i = , h we define: y i ( p ) = h (cid:214) j = (cid:16) a i λ p j + b i λ p j (cid:17) ∀ i ∈ { , . . . , h } (14)Finally, define the following ( m + ) h × ( m + ) h matrix M : M p , k def = (cid:214) i = , h y k i i ( p ) , p ∈{ , , . . . , m + } h k ∈{ , , . . . , m } h k + . . . + k h = m Example 3.5.
For a simple example, assume h = m =
2, then: y i ( p , p ) = ( a λ p + b λ p )( a λ p + b λ p ) i = , , M = y ( , ) y ( , ) y ( , ) . . . y ( , ) y ( , ) y ( , ) y ( , ) . . . y ( , ) . . . y ( , ) y ( , ) y ( , ) . . . y ( , ) Each row of M has all products y k y k y k where k + k + k = The matrix M is non-singular
Before we prove the theorem, we show how to use it to proveTheorem 3.1. We set h =
2. Given the P2CNF formula defined by thegraph ( U , E ) , we will construct a TID obtained as a disjoint unionof blocks ∆ = (cid:208) ( u , v )∈ E B ( u , v ) . The probability of Q is given byEq. (10). Fix two numbers p , p ≥
1. We will describe in the nextsection how to construct a symmetric block B ( u , v ) such that itsprobabilites are given by expresions similar to (14), more precisely: y = (cid:214) j = , ( a λ p i + b λ p i ) y = (cid:214) j = , ( a λ p i + b λ p i ) y = (cid:214) j = , ( a λ p i + b λ p i ) where the coefficients satsify conditions (11)-(13). We use repeatedlythe oracle for Pr ( ∆ ) , once for each ( m + ) combination of values p , p ∈ { , , . . . , m + } , and obtain a system of linear equationswith unknowns k ′ . By Theorem 3.6 this system has a uniquesolution, which can be computed in polynomial time using Gaussianelimination. This gives us all the undirected counts k ′ , from whichwe extract Φ = (cid:205) k ′ : k = k ′ .In the rest of this section we prove Theorem 3.6, by proving aseries of lemmas in calculus, of possible independent interest. Lemma 3.7. For each k = ( k , . . . , k h )∈{ , . . . , m } h , define the fol-lowing polynomial in variables y = ( y , . . . , y h ) : д k ( y ) def = y k · · · y k h h .Then, the polynomials д k , k ∈ { , . . . , m } h are linearly independent. Proof. Assume the contrary, that there exists coefficients a k ,..., k h such that, denoting f ( y ) def = (cid:205) k a k (cid:206) i y k i i , the polynomial f is iden-tically 0, f ≡
0. For each i = , h , let S i def = { v i , , . . . , v i , m } be aset of m + A i the Vandermondematrix defined by the set S i : ( A i ) k ℓ = v ki ,ℓ , for 0 ≤ k , ℓ ≤ m . Forany values u i ∈ S i , the polynomial f is 0, which implies: (cid:213) k a k (cid:214) i u k i i = ( m + ) h combinations of values u , . . . , u h , we obtaina linear system of ( m + ) h unknowns a k ,..., k h , whose matrix isthe Kronecker product A def = A ⊗ A ⊗ · · · ⊗ A h . Since det ( A i ) (cid:44) i , it follows that det ( A ) (cid:44)
0. This implies that the system hasa single solution, a k ,..., k h = k , . . . , k h , proving that thepolynomials are linearly independent. □ Lemma 3.8.
Let f ( z , . . . , z h ) be a polynomial of degree ≤ m inevery z i , such that f (cid:46) . For all i = , h , let A i ⊆ R be a set of m + distinct, real values. Then there exists values u i ∈ A i , for i = , h ,such that f ( u , . . . , u h ) (cid:44) . Proof. By induction on h . When h =
1, then f has at most m roots, hence there is at least one value u ∈ A such that f ( u ) (cid:44) h >
1, consider any value a ∈ A h , and set z h = a . If f [ z h : = a ] (cid:46)
0, then by induction on h we get values u ∈ A , . . . , u h − ∈ A h − for the other h − f [ z : = u , . . . , z h − : = u h − , z h : = a ] (cid:44)
0. If f [ z h : = a ] ≡ m + a ∈ A h ,then f can be divided by the polynomial (cid:206) a ∈ A h ( z h − a ) , whichhas degree m +
1, contradiction. □ Lemma 3.9.
Let д , д , . . . , д N be linearly independent multivari-ate polynomials, in h variables y , . . . , y h . Let H : A (⊆ R h ) → R h be a differentiable function such that its Jacobian J ( H ) (cid:44) at someinterior point in A , and define f i ( z , . . . , z h ) def = д i ( H ( z , . . . , z h )) forevery i . Then the functions f , . . . , f N are linearly independent. Proof. Assume, by contradiction, that there exist constants a , . . . , a N not all of which are 0 such that: F = a f + a f + · · · + a N f N ≡
0. We prove that G = a д + a д + · · · + a N д N ≡ v = ( v , . . . , v h ) ∈ A be a point where J ( H ) (cid:44)
0. By the inverse function theorem, the function H is in-vertible in some neighborhood of H ( v ) . Assume w.l.o.g. that thisneighborhood is a product of open intervals, (cid:206) i ( α i , β i ) , where α i < β i , and let A i ⊆ ( α i , β i ) be any finite set with m + i = , h . Since H is invertible on A × · · · × A h , forany combination of values u ∈ A × · · · × A h there exists w ∈ R h such that H ( w ) = u . By assumption F ( w ) =
0, and this implies G ( u ) = G ( H ( w )) = F ( w ) =
0. Lemma 3.8 implies that G ≡
0, whichis a contradiction. □ onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Lemma 3.10.
Let c , . . . , c h be distinct real values, and let H : R h → R h be the following function: H ( z ) = y , where: y i def = (cid:214) j = , h ( c i + z j ) (15) Let u = ( u , . . . , u h ) be any point with distinct coordinates, i.e. u i (cid:44) u j , such that u i + c j (cid:44) for all i , j . Then the Jacobian of H at u isnonzero, J ( H )( u ) (cid:44) . Proof. Recall the definition of the Jacobian:
J ( H ) def = ∂ y ∂ z · · · ∂ y ∂ z h . . . . . . . . . ∂ y h ∂ z · · · ∂ y h ∂ z h Since y i = (cid:206) hj = ( c i + z j ) then ∂ y i ∂ z k = (cid:206) j (cid:44) k ( c i + z j ) . For each row i ∈ [ , h ] , we factor out the product (cid:206) hj = ( c i + z j ) . This results inthe following matrix: J ′ def = c + z · · · c + z h . . . . . . . . . c h + z · · · c h + z h . We note thatdet (J ( H )) = det (J ′ )· (cid:206) hi = (cid:206) hj = ( c i + z j ) . Therefore, if det (J ′ ) (cid:44) (J ) (cid:44)
0. The expression for det (J ′ ) has a closed form [7]:det (J ′ ) = (cid:206) ≤ i < j ≤ h ( c i − c j )( z i − z j ) (cid:206) ≤ i < j ≤ h ( c i + z j ) (16)When z j = u j for all j , then this value is (cid:44) c i ’s aredistinct, and the u i ’s are distinct. □ For the next two statements we fix c , . . . , c h to distinct realvalues and, for each k , . . . , k h ∈{ , . . . , m } , we define the followingpolynomial in variables z = ( z , . . . , z h ) : f k ,..., k h ( z ) def = h (cid:214) i = h (cid:214) j = ( c i + z j ) k i (17)Corollary 3.11. The polynomials f k ,..., k h of (17) are linearlyindependent. Proof. By Lemma 3.7, the polynomials д k ,..., k h ( y ) def = (cid:206) hi = y k i i are linearly independent. By Lemma 3.10, the transformation z (cid:55)→ y given by (15) has a non-zero Jacobian (at some point). Then, byLemma 3.9 it follows that the polynomials f k ,..., k h ( z ) in (17) arealso linearly independent. □ Lemma 3.12.
For each i = , h , let A i ⊆ R be a set of m + distinct,real values. Consider the following ( m + ) h × ( m + ) h matrix, whoserows are indexed by u ∈ A , . . . , A h , and whose columns are indexedby k ∈ { , , , · · · , m } h : M u , k def = h (cid:214) i = h (cid:214) j = ( c i + u j ) k i Then M is non-singular. Proof. We notice that each row of the matrix M consists of thepolynomials f k ,..., k h ( z ) applied to some point in u . We constructa matrix M ′ such that det ( M ′ ) (cid:44) M ′ differs from M onlyby permutations of rows and columns. We construct, by inductionon k , a k × ( m + ) h matrix M k such that each row consists ofthe values of the polynomials f k ,..., k h ( z ) applied to some point in u ∈ A × · · · × A h , and such that the k × k minor consisting of thefirst k columns in M k is non-singular. When k = u ∈ A × · · · × A h , and the statement holds because M u , ( , ,..., ) = k = · · · = k h = M k + by extending M k . First, extend M k with a row consisting of the polynomials f k ,..., k h ( z ) . Let D be the ( k + ) × ( k + ) minor M k + defined by the first k + D is a linear combination of these polynomials, where the coefficientof each polynomial is the k × k minor consisting of the first k + k columns isnon-zero. By Corollary 3.11, D ( z ) (viewed as a polynomial in z ) isnon-zero, hence by Lemma 3.8 there exists u ∈ A × · · · × A h suchthat D ( u ) (cid:44)
0, proving the claim. Thus, we obtain M ′ def = M ( m + ) n ,a matrix that is non-singular. Since the matrix is non-singular, notwo rows in M ′ can use the same value u ∈ A × · · · × A h , andsince both the number of rows in M ′ and the cardinality of the set A × · · · × A h are the same, ( m + ) h , it follows that M ′ containsprecisely the same rows as M , up to a permutation, proving thelemma. □ Finally, we can prove now Theorem 3.6. For that we use the factthat k = m − ( k + · · · + k h ) and write: M p , k = y m ( p ) (cid:214) i = , h (cid:18) y i ( p ) y ( p ) (cid:19) k i Since every row p in M has the same factor y m ( p ) , it suffices toprove that the matrix M ′ without this factor is non-singular: M ′ p , k = (cid:214) i = , h (cid:18) y i ( p ) y ( p ) (cid:19) k i = (cid:214) i = , h (cid:214) j = , h (cid:169)(cid:173)(cid:171) a i λ p j + b i λ p j a λ p j + b λ p j (cid:170)(cid:174)(cid:172) k i = (cid:214) i = , h (cid:214) j = , h (cid:169)(cid:173)(cid:173)(cid:173)(cid:171) a i + b i (cid:16) λ λ (cid:17) p j a + b (cid:16) λ λ (cid:17) p j (cid:170)(cid:174)(cid:174)(cid:174)(cid:172) k i = (cid:214) i = , h (cid:214) j = , h (cid:169)(cid:173)(cid:173)(cid:173)(cid:171) b i b + a i − a b i b a + b (cid:16) λ λ (cid:17) p j (cid:170)(cid:174)(cid:174)(cid:174)(cid:172) k j def = (cid:214) i = , h (cid:214) j = , h (cid:169)(cid:173)(cid:173)(cid:173)(cid:171) c i + d i a + b (cid:16) λ λ (cid:17) p j (cid:170)(cid:174)(cid:174)(cid:174)(cid:172) k j We now use Lemma 3.12. Since λ / λ (cid:60) {− , , + } , the function p j (cid:55)→ z j def = d i a + b (cid:16) λ λ (cid:17) pj is injective, therefore the m + p j = , , , . . . , m + m + z j . By Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Lemma 3.12, the matrix M ′ is non-singular, completing the proofof Theorem 3.6. B ( u , v ) In this section, we design the block TID B ( u , v ) such that the prob-ability of the lineage of Q over B ( u , v ) can be expressed as in (14),and that it meets the conditions of (11)-(13).We design our blocks to prove hardness for final type-I queries,where every endpoint (i.e., u and v ) is assigned one of two values { , } . Since we design symmetric blocks, then h = { , , } . Inour construction, each block is parameterized by a pair p = { p , p } .The main focus of this section is a block that is parameterizedby a single value p , denoted B p ( u , v ) . We show how two blocks B p ( u , v ) , B p ( u , v ) can be combined in parallel to create a block B p ( u , v ) that has the desired form and properties. The Block B p ( u , v ) . The bipartite domain of B p ( u , v ) is V ( p )∪ V ( p ) ,and is defined as follows: V ( p ) = { u , v } ∪ { r k : k ∈ [ , p − ]} (18) V ( p ) = { t k : k ∈ [ , p ]} (19)where the r k and t k are fresh constants. The tuple probabilities of B p ( u , v ) are: • Pr ( R ( u )) = (cid:40) u ∈ V ( p ) • Pr ( T ( v )) = (cid:40) v ∈ V ( p ) S ∈ R the probabilities are: • Pr ( S ( u , t )) = . • Pr ( S ( v , t p )) = . • Pr ( S ( r k , t k )) = for all k ∈ [ , p − ] . • Pr ( S ( r k , t k + )) = for all k ∈ [ , p − ] . • Otherwise, Pr ( S ( a , b )) = Example 3.13.
The graphical representation of B p ( u , v ) is thepath: u def = r − t − r − · · · − r p − − t p − r p def = v Observe that the binary atoms corresponding to the edges of thepath have a probability of , and atoms corresponding to non-edges(e.g., S ( t , t ) , S ( r , t ) ) have a probability of 1. Also note that r def = u ,and r p def = v .Let p ≥ Q over theblock TID B p ( u , v ) is denoted Y ( p ) ( u , v ) , and its arithmetization y ( p ) . That is, y ( p ) is the multilinear polynomial representing theprobability Pr ( Y ( p ) ( u , v )) . We associate the endpoints u and v withthe unary atoms R ( u ) and R ( v ) respectively. Thus Y ( p ) ab ( u , v ) def = Y ( p ) ( u , v )[ R ( u ) : − a , R ( v ) : − b ] , and y ab ( p ) def = y ( p )[ R ( u ) : − a , R ( v ) : − b ] isdefined accordingly.We define z ab ( p ) to be the valuation of y ab ( p ) when all of theRVs in Y ( p ) ab ( u , v ) have a probability of . That is, for any natural p ≥ z ab ( p ) def = y ab ( p )[ , . . . , ] (20) In the rest of this section we design the block B p ( u , v ) and provethat it meets the following conditions.Theorem 3.14. Let B p ( u , v ) be a block TID, parameterized by aparameter p ≥ . For every i ∈ { , , } it holds that: z i ( p ) = ( a i λ p + b i λ p ) ∀ i ∈ { , , } (21) where λ , λ and a i , b i are constants, independent of p , satisfying: λ (cid:44) ± λ and λ (cid:44) , λ (cid:44) b i (cid:44) ∀ i ∈ { , , } (23) a i b j (cid:44) a j b i i (cid:44) j (24)Let p = { p , p } . We now describe how to create the block B p ( u , v ) that has probability y i ( p ) (as in (14)) for every i ∈ { , , } .To do so, we construct two disjoint blocks between nodes u and v with parameters p , p respectively (see Figure 1). The lineage ofthis block is Y ( p ) i ( u , v ) ∧ Y ( p ) i ( u , v ) . Since the blocks are disjoint,then var ( Y ( p ) ( u , v )) ∩ var ( Y ( p ) ( u , v )) = { R ( u ) , R ( v )} . In particular,the Boolean functions Y ( p ) i ( u , v ) and Y ( p ) i ( u , v ) are disjoint forevery i ∈ { , , } . So, we get that: y ab ( p ) = Pr ( (cid:219) j = Y ( p j ) ab ( u , v )) = (cid:214) j = Pr ( Y ( p j ) ab ( u , v )) = y ab ( p ) y ab ( p ) (25)Consequently, once we prove Theorem 3.14, the conditions of (11)-(13) follow from (25).To prove Theorem 3.14 we first show that Y ( p ) ( u , v ) is a con-nected Boolean function. Consequently, we can apply Lemma 1.2to argue that the determinant of the “small matrix” associated with y ( p ) (see (1)) is not identically zero, and, by Lemma 1.1, there areprobability values in { , , } for which the small matrix is non-singular. In fact, we prove something stronger: if Q is final, thenthe matrix remains nonsingular even if we set all probabilities to1 /
2. Some further analysis of this matrix, along with two simpleproperties of the lineage Y ( p ) ( u , v ) allow us to prove Theorem 3.14.We recall that Q ( s , t ) is the Boolean formula that results frommapping x (cid:55)→ s and y (cid:55)→ t . We note that Q ( s , t ) is identically true whenever ( s , t ) (cid:60) {( u , t ) , ( v , t p ) , ( r k , t k ) , ( r k , t k + ) | k ∈[ , p − ]} . There-fore: Y ( p ) ( u , v ) = p (cid:219) i = Q ( r i − , t i ) ∧ Q ( r i , t i ) (26)where r = u and r p = v . In particular, when p = Y ( ) ( u , v ) = Q ( u , t ) ∧ Q ( v , t ) .We say that a Boolean function f is disconnected if f = f ∧ f where f and f are non-constant, disjoint Boolean functions (i.e.,var ( f )∩ var ( f ) = ∅ ). Otherwise, we say that f is connected .Lemma 3.15. If Q is an unsafe type-I query, then Y ( p ) ( u , v ) isconnected. Proof. By induction on p . Since Q is unsafe, it is connected.Therefore, it must hold that h def = Q ( u , t ) and h def = Q ( v , t ) areconnected. Observe that when p = Y ( ) ( u , v ) = h ∧ h .Since T ( t ) ∈ var ( h )∩ var ( h ) then h and h are not disjoint.Therefore, Y ( ) ( u , v ) is disconnected only if there is some homo-morphism C ( u , t ) → C R ( u , t ) (or C ( v , t ) → C R ( v , t ) ) where C onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Figure 1: Block B p ( u , v ) where p = { p , p } . is a clause in Q left ∧ Q middle and C R is a right clause. We note that C (cid:60) Q left because all left clauses, and only left clauses, contain theunary symbol R . If C ∈ Q middle , it means that C R is a redundantclause, which is a contradiction (we assume that all clauses of Q are non-redundant). Symmetrically, for any k ∈ [ , p − ] it holdsthat Q ( r k , t k )∧ Q ( r k , t k + ) are connected via the unary atom R ( r k ) ,and thus connected.Let p >
1. Observe that Y ( p ) ( u , v ) = Y ( p − ) ( u , r p − ) ∧ Y ( ) ( r p − , v ) where Y ( ) ( r p − , v ) = Q ( r p − , t p )∧ Q ( v , t p ) . By induction, Y ( p − ) ( u , v ) and Y ( ) ( r p − , v ) are both connected. In addition, Y ( p − ) ( u , v ) and Y ( ) ( r p − , v ) are connected via the unary left atom R ( r p − ) that ap-pears in both Boolean formulas. By the previous reasoning none ofthe clauses containing R ( r p − ) become redundant. This completesthe proof. □ Note, in particular, that Lemma 3.15 holds for final queries.Let N by the number of RVs in y ab ( ) . We define the matrix ofpolynomials: A ( ) def = (cid:34) y ( ) y ( ) y ( ) y ( ) (cid:35) (27)and the polynomial f A : [ , ] N →[ , ] associated with the determi-nant of A ( ) : f A ( u , . . . , u N ) def = det ( A ( ) ) = y ( ) y ( )− y ( ) y ( ) (28)and observe that f A is a degree-2 polynomial. The previous lemma,and Lemma 1.2 from the introduction, imply that f A (cid:46)
0. Therefore,by Lemma 1.1, there exists an assignment u of the variables of f A with probabilities in { , / , } that f A ( u ) (cid:44)
0. Next, we show astronger result: if Q is a final query, then f A ( u ) (cid:44) any assign-ment u : { u , . . . , u N } → ( , ) N , and in particular, f A ( , . . . , ) (cid:44)
0. Theorem 3.16. If u i ∈( , ) for all i ∈[ , N ] then f A ( u , . . . , u N ) (cid:44) . To prove Theorem 3.16, we use the fact that Q is final, and prove: Lemma 3.17. Let Q be a final Type-I query, and let X be a BooleanRV in Y ( p ) ( u , v ) corresponding to any atom other than R ( u ) and R ( v ) .Then Y ( p ) [ X : − a ] is disconnected for a ∈ { , } . Proof. For i ≥
1, we define: f ( i ) def = Q ( r i − , t i )∧ Q ( r i , t i ) .Case 1: X = R ( r k ) for some k ∈[ , p − ] (we let r = u , and r p = v ). Wenote that Y ( p ) = h ∧ h where h def = (cid:211) ki = f ( i ) and h def = (cid:211) pi = k + f ( i ) .Since var ( h )∩ var ( h ) = { x } , the result follows.Case 2: X = T ( t k ) for some k ∈[ , p ] . We express Y ( p ) = h ∧ h where h def = (cid:211) k − i = f ( i )∧ Q ( r k − , t k ) and h def = Q ( r k , t k ) (cid:211) pi = k + f ( i ) . The re-sult follows from noting that var ( h )∩ var ( h ) = { x } .Case 3: X = S ( r k , t k ) for some binary symbol S ∈ Symb ( Q ) where k ∈[ , p ] . Consider the Boolean function Q ( r k , t k ) . Since Q is fi-nal, then setting a value to S ( r k , t k ) makes it safe. Hence, it doesone of the following: (1) Makes Q ( r k , t k ) disconnected, (2) Makes Q left ( r k , t k ) redundant or, (3) Makes Q right ( r k , t k ) redundant. If(1) then we are done because if Q ( r k , t k )[ S ( r k , t k )] is disconnected,then clearly so is Y ( p ) . Otherwise, since (2) is equivalent to setting R ( r k )← T ( t k )← □ Corollary 3.18.
There exists some constant c (cid:44) such that: f A = c (cid:206) Ni = u i ( − u i ) Proof. Since both f A [ u i : = ] = f A [ u i : = ] =
0, it follows that f A is divisible by u i ( − u i ) , for every i . Hence f A = c (cid:206) Ni = u i ( − u i ) and, since each variable has degree ≤ f A , it follows that c is aconstant. □ Theorem 3.16 follows from Corollary 3.18 because f A ( u ) (cid:44) u : var ( f A ) → ( , ) N . Thus, we have establishedthat f A ( , . . . , ) (cid:44)
0. That is, det ( A ( ) ) (cid:44) y ( ) have a uniform value of . In other words, we have shownthat the matrix: A ( ) def = (cid:34) z ( ) z ( ) z ( ) z ( ) (cid:35) (29)is invertible.So far, we have discussed only the “small” matrix A ( ) that cor-responds to one step of the zig-zag block B p ( u , v ) ; in other words,this is the matrix that we have for B ( u , v ) , when p =
1. Next, weshow how to compute A ( p ) . We define: A ( p ) def = (cid:104) z ( p ) z ( p ) z ( p ) z ( p ) (cid:105) (30)Lemma 3.19. A ( p ) = (cid:104) z ( ) z ( ) z ( ) z ( ) (cid:105) p = p − (cid:16) A ( ) (cid:17) p Proof. The proof is by induction on p . The base case for p = p − p . Y ( p ) ( u , v ) def = (cid:219) ( s , t )∈ B p ( u , v ) Q ( s , t ) = (cid:219) ( s , t )∈ B p − ( r , r p − ) Q ( s , t ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) Y ( p − ) ( r , r p − ) (cid:219) ( s , t )∈ B ( r p − , r p ) Q ( s , t ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) Y ( ) ( r p − , r p ) Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
We note that the only atom common to both Y ( p − ) ( r , r p − ) and Y ( ) ( r p − , r p ) is R ( r p − ) . Since Pr ( R ( r p − ) = ) = , we have that: z ab ( p ) = (cid:0) z a ( p − ) (cid:1) (cid:0) z ( ) b (cid:1) + (cid:0) z ( p − ) a (cid:1) (cid:0) z ( ) b (cid:1) (31)Applying the induction hypothesis. A ( p ) = (cid:104) z ( p − ) z ( p − ) z ( p − ) z ( p − ) (cid:105) (cid:104) z ( ) z ( ) z ( ) z ( ) (cid:105) = p − (cid:104) z ( ) z ( ) z ( ) z ( ) (cid:105) p □ Let λ and λ be the eigenvalues of A ( ) (defined in (29)). Weprove now condition (22), which is the first of the three condi-tions that we need to establish. We require the following simpleproposition.Proposition 3.20. The following hold: (1) z ( ) < z ( ) = z ( ) < z ( ) , and (2) < z ab ( ) ≤ for all a , b ∈ { , } Proof. We note that Y ( ) ( u , v ) and Y ( ) ( v , u ) are identical. Con-sequently, Y ( ) ab ≡ Y ( ) ba up to the renaming of the variables. There-fore, z ( ) = z ( ) . Since Q does not contain negations, then itslineage is a monotonic Boolean function. Further, since the Booleanfunction Y ( ) depends on both atoms R ( u ) and R ( v ) , then (1) follows.Item (2) follows by noting that Y ( ) is satisfiable for any final type-Iquery, thus z ( ) > □ Lemma 3.21.
Let λ , λ be the eigenvalues of A ( ) . Then: λ (cid:44) , λ (cid:44) , and λ (cid:44) ± λ . Proof. This follows immediately from λ + λ = Tr ( A ( ) ) = z + z > λ λ = det ( A ( ) ) (cid:44)
0, and the fact that a symmetricmatrix where all eigenvalues are equal is a diagonal matrix (which A ( ) obviously is not). For a more elementary argument, recallthat the characteristic polynomial of A ( ) (see (29)) is (we drop theparameter ( ) ):det ( λI − A ( ) ) = λ − λ ( z + z ) + ( z z − z z ) (32)From (32) we see that λ = ( A ( ) ) = z z − z z =
0. By Theorem 3.16, this cannot bethe case. Therefore, λ , λ (cid:44)
0. Also from (32) we get that the tworoots of det ( λI − A ( ) ) are: λ = ( z + z ) ± (cid:112) ( z − z ) + z z . Since z = z > λ (cid:44) λ . Since z + z > λ (cid:44) ± λ . □ Since A ( ) has two distinct, non-zero eigenvalues, then it hastwo linearly independent eigenvectors. In other words, A ( ) is diagonizable , and thus A ( p ) = (cid:16) A ( ) (cid:17) p = PD k P − where P isthe matrix whose columns are the eigenvectors of A ( ) , and D isthe diagonal matrix of its eigenvalues. Let v = ( c , c ) T , and v = ( c , c ) T be the two linearly independent eigenvectors corre-sponding to eigenvalues λ and λ respectively. So, by Lemma 3.19, A ( p ) = ( A ( ) ) p will have the following form. A ( p ) = PD p P − = (cid:2) c c c c (cid:3) (cid:20) λ p λ p (cid:21) (cid:104) b b b b (cid:105) = (cid:20) c b λ p + c b λ p c b λ p + c b λ p c b λ p + c b λ p c b λ p + c b λ p (cid:21) (33) = (cid:20) a λ p + b λ p a λ p + b λ p a λ p + b λ p a λ p + b λ p (cid:21) (34) = (cid:20) a λ p + b λ p a λ p + b λ p a λ p + b λ p a λ p + b λ p (cid:21) (35)where the transition from (33) to (34) is by defining a def = c b , b def = c b etc. The transition from (34) to (35) follows from the factthat A ( ) is a symmetric matrix (Proposition 3.20). Notice that thisestablishes Eq.(21), which we need to prove as part of Theorem 3.14.In particular, we have that: A ( ) = (cid:104) a λ + b λ a λ + b λ a λ + b λ a λ + b λ (cid:105) (36) Proof of Theorem 3.14.
From (35) it follows that z i ( p ) can bewritten in the form of (21) for all i ∈ { , , } . In Lemma 3.21,we have shown that λ , λ (cid:44) λ (cid:44) ± λ , thus provingcondition (22).By Lemma 3.19 we have that A ( ) = ( A ( ) ) = I where I is theidentity matrix. Therefore, from (35), we get the following threeequations: a + b = a + b = a + b = b i (cid:44) i ∈ { , , } . If b = a = − b it follows that z ( ) =
0, which is a contradiction(Proposition 3.20). Thus, b (cid:44)
0. Assume that b =
0. This meansthat a =
1, and that for any p ≥
1, we have that z ( p ) = λ p (see (35)). Recall that z ( p ) = z ( p − ) z ( ) + z ( p − ) z ( ) .By proposition 3.20 we have that z ( p − ) > z ( p − ) , and that z ( ) > z ( ) . Therefore, z ( p ) > z ( p − ) z ( ) = λ p , and wearrive at a contradiction. Similarly, if b = a =
1, and z ( p ) = λ p . Now, since z ( p ) = z ( p − ) z ( ) + z ( p − ) z ( ) ,then due to monotonicity, we have that z ( p ) < λ p , which is acontradiction.Finally, we show that a i b j (cid:44) a j b i for i (cid:44) j . Assume, by con-tradiction, that a b = a b . Substituting b = ( − a ) and b = ( − a ) , this implies that a = a and thus b = b .But, by (36), this means that z ( ) = z ( ) which, by Proposi-tion 3.20, is a contradiction. Now, assume, by contradiction, that a b = a b . From (37) we have that − a = b . Substituting,this gives us that a b = − b ( − a ) or, that b =
0. But then z ( ) =
0, which, by Proposition 3.20, is a contradiction. Sym-metrically, it is shown that a b (cid:44) a b . This completes theproof. One can think of the model counting problem as: given a set oftuples to exclude , compute the number of models of a sentence thatdo not use any of the excluded tuples. In this paper we studiedthe generalized model counting problem, where we are also givena set of tuples to include , and need to count only those modelsthat contain all these tuples, and none of the excluded ones. We onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu have established a dichotomy for Unions of Conjunctive Queriesor, equivalently, for ∀ CNF formulas. For a special case, called finalqueries of type I we have also established a dichotomy for the modelcounting problem; this complements a result by Amarilli et al. [1]that prove a dichotomy for model counting for conjunctive querieswithout self-joins. We leave open the question whether UCQs admita dichotomy for model counting.
REFERENCES [1] Antoine Amarilli, Pierre Bourhis, and Pierre Senellart. Provenance circuits fortrees and treelike instances. In
Automata, Languages, and Programming - 42ndInternational Colloquium, ICALP 2015, Kyoto, Japan, July 6-10, 2015, Proceedings,Part II , pages 56–68, 2015. URL: https://doi.org/10.1007/978-3-662-47666-6_5, doi:10.1007/978-3-662-47666-6\_5 .[2] Antoine Amarilli and Benny Kimelfeld. Model counting for conjunctive querieswithout self-joins.
CoRR , abs/1908.07093, 2019. URL: http://arxiv.org/abs/1908.07093, arXiv:1908.07093 .[3] Paul Beame, Guy Van den Broeck, Eric Gribkoff, and Dan Suciu. Symmetricweighted first-order model counting. In
Proceedings of the 34th ACM Symposiumon Principles of Database Systems, PODS 2015, Melbourne, Victoria, Australia, May31 - June 4, 2015 , pages 313–328, 2015. URL: https://doi.org/10.1145/2745754.2745760, doi:10.1145/2745754.2745760 .[4] Nilesh N. Dalvi and Dan Suciu. The dichotomy of probabilistic inference forunions of conjunctive queries.
J. ACM , 59(6):30:1–30:87, 2012. URL: https://doi.org/10.1145/2395116.2395119, doi:10.1145/2395116.2395119 .[5] Guy Van den Broeck, Wannes Meert, and Adnan Darwiche. Skolemization forweighted first-order model counting. In
Principles of Knowledge Representationand Reasoning: Proceedings of the Fourteenth International Conference, KR 2014,Vienna, Austria, July 20-24, 2014
Ann. Math. Artif. Intell. , 2:165–178,1990. URL: https://doi.org/10.1007/BF01531004, doi:10.1007/BF01531004 .[7] C. Krattenthaler. Advanced determinant calculus. In Dominique Foata andGuo-Niu Han, editors,
The Andrews Festschrift , pages 349–426, Berlin, Heidelberg,2001. Springer Berlin Heidelberg.[8] J. Scott Provan and Michael O. Ball. The complexity of counting cuts and ofcomputing the probability that a graph is connected.
SIAM J. Comput. , 12(4):777–788, 1983. URL: https://doi.org/10.1137/0212053, doi:10.1137/0212053 .[9] Richard P. Stanley.
Enumerative Combinatorics: Volume 1 . Cambridge UniversityPress, USA, 2nd edition, 2011.[10] Dan Suciu, Dan Olteanu, Christopher Ré, and Christoph Koch.
Probabilis-tic Databases . Synthesis Lectures on Data Management. Morgan & ClaypoolPublishers, 2011. URL: https://doi.org/10.2200/S00362ED1V01Y201105DTM016, doi:10.2200/S00362ED1V01Y201105DTM016 . Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA Δ uv 𝑅 ($) (𝑢)𝑅 (’) (𝑣)𝑅 ) 𝑢, 𝑣 , … , 𝑅 ’,$ (𝑢, 𝑣)𝑆 . $ 𝑢, 𝑣 , … , 𝑆 . ’ 𝑢, 𝑣 zg( Δ) uv 𝑅(𝑢)𝑅(𝑣) 𝑒 𝑓 𝑓 … 𝑅(𝑓 )𝑅(𝑓 ) 𝑆 . (𝑢, 𝑒 )𝑆 . (𝑣, 𝑒 )𝑆 . (𝑓 , 𝑒 )𝑆 . (𝑓 , 𝑒 ) Figure 2: Illustration for a reduction
PQE ( zg ( Q )) ≤ Pm PQE ( Q ) where Q is a Type I-II query. Given a query Q we construct the query zg ( Q ) . Then, given the database ∆ for zg ( Q ) on the left, we construct the database zg ( ∆ ) for Q on the right. Here n is one plusthe longest right clause of Q ; for example if Q right = ∀ y ( ∀ xS ( x , y ) ∨ ∀ xS ( x , y ) , then n = and there is a single node f ( ) uv , so thatin total there are three edges incoming to e uv . APPENDIXA PROOF OF LEMMA 2.6
Let Q be an unsafe, bipartite query of length k , and let C , C , . . . , C k be a minimal left-right path. Let R denote its vocabulary. We describe(a) a new unsafe, bipartite query zg ( Q ) of length 2 k , over a new vocabulary zg (R) and (b) a polynomial-time mapping that takes a bipartiteTID ∆ over the vocabulary zg (R) and returns bipartite TID zg ( ∆ ) over the vocabulary R such that Pr ∆ ( Q ) = Pr zg ( ∆ ) ( zg ( Q )) and zg ( Q ) islong. The probabilities values in ∆ and zg ( ∆ )) are the same, which proves that GFOMC bi ( zg ( Q )) ≤ Pm GFOMC bi ( Q ) . Moreover, if Q is of type A − B , then zg ( Q ) is of type A − A . These properties prove Lemma 2.6.Define a number n as follows. If Q right is of Type I, then n =
2. Otherwise, recall from Def. 2.3 that the right clauses of type II have theform ∀ y (cid:16)(cid:212) ℓ i = ( ∀ xS J ℓ ( x , y )) (cid:17) . We define n to be the maximum between 3, and the largest value of ℓ of all right clauses. Thus, by definition n ≥ The vocabulary zg (R) . We start by describing the mapping zg between the vocabularies. The vocabulary R contains a set of binarysymbols S , . . . , S t . In addition, it may contain a single unary symbol R , and a single unary symbol T . The new vocabulary zg (R) consists of m + n disjoint copies of R , as follows: • If R ∈ R then zg (R) contains symbols R ( ) , R ( ) , . . . , R ( n ) , where the first and last one are unary, and all others are binary. • For every binary symbol S ∈ R , zg (R) contains the following binary symbols: S ( ) , . . . , S ( n ) . • If T ∈ R , then zg (R) contains a binary symbol T ( ) .Notice that the only unary symbols are R ( ) and R ( n ) ; the symbol T became a binary symbol T ( ) . These will be the unary symbols “ R and T ” of the new query zg ( Q ) . Thus, if Q was of type I-I or I-II then zg ( Q ) will be of type I-I because it has the two unary symbols R ( ) and R ( n ) .If Q was of type II-I or II-II, then it has no R -symbol, hence zg ( Q ) has no unary symbols, i.e. it will be of type II-II. The database zg ( ∆ ) . Next, given a bipartite probabilistic database ∆ = ( Dom , p ) over vocabulary zg (R) , we describe how to construct ( zg ( ∆ ) , p ′ ) . Since ∆ is bipartite, is domain is Dom = V ∪ V and the only tuples t with p ( t ) (cid:44) S j ( u , v ) , u ∈ V , v ∈ V .Define the following bipartite database, zg ( ∆ ) = ( Dom ′ , p ′ ) over the vocabulary R . Its domain Dom ′ = U ∪ U consists of the followingconstants: • For every u ∈ V , u is in U . • For every v ∈ V , v is in U . • For every pair u ∈ V , v ∈ V , there is a fresh element denoted e uv , which is in U . • For every pair u ∈ V , v ∈ V , there are n − f ( i ) uv , i = , . . . , n −
1, all of which are in U .To define the tuple probabilities p ′ in ∆ , we establish a 1-to-1 correspondence between the tuples in ∆ and those in zg ( ∆ ) , which, in turn,defines p ′ in terms of p : • p ′ ( R ( u )) def = p ( R ( ) ( u )) for all u ∈ V . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu • p ′ ( R ( f ( i ) uv )) = p ( R ( i ) ( u , v )) for all u ∈ V , v ∈ V , i ∈ [ , n − ] . • p ′ ( R ( v )) def = p ( R ( n ) ( v )) for all v ∈ V . • p ′ ( S ( u , e uv )) def = p ( S ( ) ( u , v )) for all u ∈ V , v ∈ V . • p ′ ( S ( f ( i ) uv , e uv )) def = p ( S ( i ) ( u , v )) for all u ∈ V , v ∈ V , i = , . . . , n − • p ′ ( S ( v , e uv )) def = p ( S ( n ) ( u , v )) for all u ∈ V , v ∈ V . • p ′ ( T ( e uv )) def = p ( T ( ) ( u , v )) All other tuples have probability p ′ ( t ) = The querry zg ( Q ) . Finally, we define the zig-zag query zg ( Q ) . • For every left clause C in Q there are n clauses in zg ( Q ) , of which the first is a left clause, the last is a right clause, and the rest aremiddle clauses. More precisely, if C is of type I clause, C = R ( x ) ∨ S J ( x , y ) then zg ( Q ) contains the following clauses: ∀ x ∀ y ( R ( ) ( x )∨ S ( ) J ( x , y )) (38) ∀ x ∀ y ( R ( i ) ( x , y )∨ S ( i ) J ( x , y )) i = , . . . , n − ∀ x ∀ y ( R ( n ) ( y )∨ S ( n ) J ( x , y )) (39)If C is of type II, C = ∀ x (cid:16)(cid:212) mi = ∀ y ( S J i ( x , y )) (cid:17) then zg ( Q ) contains the following clauses: ∀ x (cid:169)(cid:173)(cid:171) m (cid:220) i = ∀ y ( S ( ) J i ( x , y )) (cid:170)(cid:174)(cid:172) (40) ∀ x ∀ y (cid:169)(cid:173)(cid:171) m (cid:220) i = S ( j ) J i ( x , y ) (cid:170)(cid:174)(cid:172) i = , . . . , n − ∀ y (cid:169)(cid:173)(cid:171) m (cid:220) i = ∀ x ( S ( n ) J i ( x , y )) (cid:170)(cid:174)(cid:172) (41) • Every middle clause C = S J ( x , y ) in Q becomes n middle clauses in zg ( Q ) : ∀ x ∀ yS ( i ) J ( x , y ) i = , . . . , n (42) • Every right clause becomes several middle clauses. There are two cases. If the right part of Q is of Type I, then every right clause hasthe form C = ∀ x ∀ y ( S J ( x , y ) ∨ T ( y )) . In that case recall that n = zg ( Q ) : ∀ x ∀ y ( S ( ) J ( x , y ) ∨ T ( ) ( x , y )) (43) ∀ x ∀ y ( S ( ) J ( x , y ) ∨ T ( ) ( x , y )) (44)If the right part of Q is of Type II, then every right clause has the form C = ∀ y ( (cid:212) ℓ i = ∀ xS J i ( x , y )) . In this case, we create n ℓ middleclauses in zg ( Q ) , as follows. For every function ϕ : [ ℓ ] → [ n ] there will be one middle clause (for a total of n ℓ middle clauses): C ( ϕ ) def = ∀ x ∀ y (cid:16) S ( ϕ ( )) J ( x , y ) ∨ · · · ∨ S ( ϕ ( l )) J ℓ ( x , y ) (cid:17) (45) zg ( Q ) is defined as the conjunction of all clauses above. Recall that, by assumption, we minimize every ∀ CNF expression, that means thatthe clauses described above need to be minimized, and redundant ones need to be removed.We prove several properties of zg ( Q ) :Lemma A.1. Pr ∆ ( zg ( Q )) = Pr zg ( ∆ ) ( zg ( Q )) Proof. We prove a stronger statement: the lineage of Q on zg ( ∆ ) is equivalent to the lineage of zg ( Q ) on ∆ , up to the 1-to-1 correspondencebetween the tuples described above. For that, we will show that for each clause C of Q , it’s lineage is equivalent to the conjunction of lineagesof clauses derived from C in zg ( Q ) . There are several cases. In all cases we exploit the fact that x in C can be mapped only to constants of theform u , f ( ) uv , . . . , f ( n − ) uv , v , and y can be mapped only to constants of the form e uv , where u ∈ V , v ∈ V : • Suppose C is a left clause. If it is of Type I, C = R ( x ) ∨ S J ( x , y ) , then its lineage (in zg ( ∆ ) ) is the conjunction (over u ∈ V and v ∈ V )of: (cid:16) R ( u ) ∨ S J ( u , e uv ) (cid:17) ∧ n − (cid:219) i = ( R ( f ( i ) uv ) ∨ S J ( f ( i ) uv , e uv )) ∧ (cid:16) R ( v ) ∨ S J ( v , e uv ) (cid:17) Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
This is precisely the lineage of the clauses (38)-(39). If C is of type II, C = (cid:212) ℓ i = ∀ yS J i ( x , y ) then its lineage is the conjunction of thefollowing Boolean formulas: (cid:219) u ∈ V (cid:169)(cid:173)(cid:173)(cid:171) ℓ (cid:220) i = (cid:169)(cid:173)(cid:171) (cid:219) v ∈ V S J i ( u , e uv ) (cid:170)(cid:174)(cid:172)(cid:170)(cid:174)(cid:174)(cid:172)(cid:219) v ∈ V (cid:169)(cid:173)(cid:173)(cid:171) ℓ (cid:220) i = (cid:169)(cid:173)(cid:171) (cid:219) u ∈ V S J i ( v , e uv ) (cid:170)(cid:174)(cid:172)(cid:170)(cid:174)(cid:174)(cid:172) n − (cid:219) j = (cid:169)(cid:173)(cid:171) ℓ (cid:220) i = S J i ( f ( j ) uv , e uv ) (cid:170)(cid:174)(cid:172) which is precisely the lineage of the clauses (40)-(41). • If C is a middle clause, C = S J ( x , y ) , then it’s lineages in zg ( ∆ ) is the Boolean expression: S J ( u , e uv ) ∧ (cid:219) j ∈[ , n − ] S J ( f ( j ) uv , e uv ) ∧ S J ( v , e uv ) This is precisely the lineage of clause (42). • If C is a right clause, then we distinguish two cases. If it is of Type I, C = S J ( x , y ) ∨ T ( y ) , then recall that n =
2, and its lineage is theconjunction over all u , v of the expression: ( S J ( u , e uv ) ∨ T ( e uv )) ∧ ( S J ( v , e uv ) ∨ T ( e uv )) which is precisely the lineage of the expressions (43)-(44). If it is of Type II, C = ∀ y (cid:16)(cid:212) ℓ i = ∀ x ( S J i ( x , y )) (cid:17) , then we note that y must bemapped to some value e uv . Fixing y to e uv implies exactly n possibilities for each x . Namely, { u , v , f ( ) uv , . . . , f ( n ) uv } . The lineage of C isthe conjunction, over all u ∈ U , v ∈ V , of the Boolean formula: ( S J ( u , e uv ) ∧ S J ( f ( ) uv , e uv ) ∧ · · · S J ( f ( n − ) uv , e uv ) ∧ S J ( v , e uv ))· · ·( S J ℓ ( u , e uv ) ∧ S J ℓ ( f ( ) uv , e uv ) ∧ · · · S J ℓ ( f ( n − ) uv , e uv ) ∧ S J ℓ ( v , e uv )) which is equivalent to: ( (cid:219) i ∈ ℓ S ( i ) J ( u , v ) ∨ · · · ∨ (cid:219) i ∈ ℓ S ( i ) J ℓ ( u , v )) We apply the distributivity law of ∨ over ∧ to convert this expression into a CNF expression, and obtain the conjunction of all lineagesof clauses (45).This completes the proof. □ Lemma A.2. If Q is an unsafe query, then zg ( Q ) is also unsafe. Proof. We start by observing that the following clauses in zg ( Q ) are both minimized and non-redundant: • Every left clause of the form (38) or (40) is minimized and non-redundant. • If C is a middle clause in Q , then the middle clause C ( i ) in zg ( Q ) is minimized and non-redundant. • If C = ∀ y (cid:212) ℓ i = ∀ xS J i ( x , y ) is a right clause in Q , then every middle clause C ( ϕ ) of the form (45) where ϕ is injective (i.e. the indices ϕ ( ) , ϕ ( ) , . . . , ϕ ( ℓ ) are mapped to distinct elements of [ n ] ) is minimized and non-redundant. Indeed, if there was some homomorhism C ′ → ( S ( ϕ ( )) J ( x , y ) ∨ S ( ϕ ( )) J ( x , y ) ∨ · · · S ( ϕ ( ℓ )) J ℓ ( x , y )) , where C ′ is a clause in zg ( Q ) derived from some clause C in Q , then we canconstruct a homorphism C → ( ∀ xS J ( x , y ) ∨ ∀ xS J ( x , y ) ∨ . . . ) , implying that C was redundant in Q , which is a contradiction. Tosee why, note that such a homomorphism would imply that Symb ( C ) ⊆ J i for some i ∈ [ ℓ ] . We remark here that, if ϕ is not injective,then C ( ϕ ) may be redundant. Since we have chosen n such that n ≥ ℓ , for for every right clause C there exists some; injective ϕ , andin that case C ( ϕ ) is non-redundant. onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Example A.3.
We justify here the reason for introducing the “dead end” branches f ( i ) uv . Consider zg ( Q ) where Q is the following query: Q = ∀ x ∀ y ( R ( x ) ∨ S ( x , y ))∧ ∀ x ∀ y ( S ( x , y )∨ S ( x , y )) ∧ ( S ( x , y )∨ S ( x , y )∨ S ( x , y )) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = D ∧ ∀ y ( ∀ x ( U ( x , y )∨ S ( x , y )∨ S ( x , y ))∨ ∀ x ( U ( x , y )∨ S ( x , y )∨ S ( x , y ))∨ ∀ x ( U ( x , y )∨ S ( x , y )∨ S ( x , y ))) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = C The middle clause D generates two clauses in zg ( Q ) : D ( ) def = ( S ( ) ( x , y ) ∨ S ( ) ( x , y ) ∨ S ( ) ( x , y )) D ( ) def = ( S ( ) ( x , y ) ∨ S ( ) ( x , y ) ∨ S ( ) ( x , y )) Assuming n = f ( i ) uv ), the right clause C is mapped to the conjunction of 2 = C ( ϕ ) and all becomeredundant. For example, when ϕ ( ) = ϕ ( ) = ϕ ( ) = C ( ϕ ) = ∀ x ∀ y ( U ( ) ( x , y ) ∨ S ( ) ( x , y ) ∨ S ( ) ( x , y ) ∨ S ( ) ( x , y ) ∨ U ( ) ( x , y ) ∨ S ( ) ( x , y ) ∨ S ( ) ( x , y )) And this is redundant because of D ( ) . It is easy to check that all 8 clauses C ( ϕ ) are made redundant by either D ( ) or D ( ) . We notice that thequery Q is even a forbidden query , to be discussed in Sec. B, where U is called a ubiquitous symbol . Thus, the dead end branches cannot beavoided even if Q were guaranteed to be a forbidden query.Next, we prove that zg ( Q ) is an unsafe bipartite query, by showing the existence of a left-to-right path. By assumption Q is an unsafe,bipartite query of length k , hence there exists a left-to-right path C , C , . . . , C k in Q . We define a left-to-right path in zg ( Q ) as follows. Thepath starts with: C ( ) (cid:124)(cid:123)(cid:122)(cid:125) (38) or (40) , C ( ) , . . . , C ( ) k − (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) (42)that is, we start with the translation of C into a left clause C ( ) using expression (38) if C is of type I, or (40) if C is of type II, then continuewith the translations of the middle clauses, all using branch i =
1. The path ends with the following: C ( n ) k − , . . . , C ( n ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) (42) , C ( n ) (cid:124)(cid:123)(cid:122)(cid:125) (39) or (41)That is, it ends at the translation of C into a right clause C ( n ) , as per (39) or (41). So far, all clauses we have used are non-redundant. Itremains to connect C ( ) k − with C ( n ) k − , and for that we use the right clause C k . Assume first that C k is of type I, i.e. C k = ∀ x ∀ y ( S J ( x , y ) ∨ T ( y )) ;in that case n =
2. By assumption C k − and C k have some common symbol, call it S ; obviously S must be among the symbols S J , since itcannot be T (because C k − is not a right clause). Define C ( ) k , C ( ) k to be the middle clauses (43) and (44) respectively. The left-to-right path in zg ( Q ) is: C ( ) , C ( ) , . . . , C ( ) k − , C ( ) k , C ( ) k , C ( ) k − , . . . , C ( ) The clauses C ( ) k − , C ( ) k share the common symbol S ( ) ; the clauses C ( ) k , C ( ) k share the common symbol T ( ) , and the clauses C ( ) k , C ( ) k − share the common symbol S ( ) . Thus, the path above is a left-right path of length 2 k + zg ( Q ) . Assume now that C k is of type II, i.e. C k = ∀ xS J ( x , y ) ∨ · · · ∀ xS J ℓ ( x , y ) and let S ∈ Symb ( C k − ) ∩ Symb ( C k ) ; assume wlog that S ∈ Symb ( S J ) . We consider two derived middleclause C ( ϕ ) k , C ( ϕ ) k of the form (45), as follows. Both ϕ , ϕ will be injective, ensuring that both clauses are non-redundant, and are defined asfollows: ϕ ( ) def = ϕ ( ) def = ϕ : { , . . . , ℓ } → { , . . . , n } any injective function ϕ ( ) def = n ϕ ( ) def = ϕ : { , . . . , ℓ } → { , , . . . , n − } any injective functionRecall that we have defined n ≥
3, henc ϕ is injective. Thus, C ( ϕ ) k = ( S ( ) J ∨ S ( ) J ∨ · · · ) and C ( ϕ ) k = ( S ( n ) J ∨ S ( ) J ∨ · · · ) . Now we observe that C ( ) k − , C ( ϕ ) k share the common symbol S ( ) , the clauses C ( ϕ ) k , C ( ϕ ) k share all common symbols in S ( ) J , and C ( ϕ ) k , C ( ) k − share the commonsymbols S ( n ) . Thus, the following is a left-to-right path in zg ( Q ) : C ( ) , C ( ) , . . . , C ( ) k − , C ( ϕ ) k , C ( ϕ ) k , C ( n ) k − , . . . , C ( n ) Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA □ This proves that zg ( Q ) is an unsafe, bipartite query of length ≤ k + zg ( Q ) has length ≥ k , where k is the length of the unsafe query Q . Considerany left-ot-right path in zg ( Q ) : C ′ , C ′ , . . . , C ′ m . Since C ′ is a left clause it must be of the form (38) or (40), hence all its symbols are from thebranch 1, like S ( ) . Let C ′ ℓ be the first clause that contains an index of a branch other than 1, i.e. either some S ( i ) , i > T ( ) . C ′ ℓ must alsohave a symbol on branch 1, because it shares a symbol with C ′ ℓ − whose symbols are all on branch 1, hence C ′ ℓ can only be of the form (43)or (45). Therefore, the fragment C ′ , C ′ , . . . , C ′ ℓ of the path in zg ( Q ) corresponds to a left-to-right path C , C , . . . , C ℓ in Q , namely consistingof the clauses in Q that generated the clauses C ′ , C ′ , . . . , C ′ ℓ in zg ( Q ) . Since k is the length of the shortest left-right path in Q , we have ℓ ≥ k .We reason similarly about the suffix of the path. Since C ′ m is a right clause, it is of the form (39) or (41) (note that it originates from a left clause C m ), hence all its symbols are on branch n , i.e. S ( n ) . Let C ′ p be the last clause that has some symbol on a branch other than n . Usingthe same argument as above, we conclude that C ′ p originates from a right clause C p , hence the fragment C ′ p , C ′ p + , . . . , C ′ m of the path in zg ( Q ) corresponds to a left-to-right path in Q (in reverse order): C m , C m − , . . . , C p + , C p . Therefore, m − p ≥ k . Since ℓ ≤ p we concludethat m ≥ ℓ + ( m − p ) ≥ k . B BACKGROUND ON FACTORIZATION AND INDEPENDENCE
Every multivariate polynomial f admits a unique decomposition into irreducible factors: f = д k д k · · · д k m m . In this paper we use repeatedlythe following:Theorem B.1. Let f , f , f , f be four multivariate polynomials, such that the following determinant is identically zero: det (cid:32) f f f f (cid:33) ≡ Then, there exists polynomials д , д and h , h such that the following identity holds: (cid:34) д д (cid:35) · (cid:104) h h (cid:105) ≡ (cid:34) f f f f (cid:35) Proof. By induction on the total degree of f f . Let p be any irreducible factor of f f ≡ f f . Assume that p | f and p | f (theother three cases are similar and omitted). Then the polynomials f / p , f / p , f , f also satisfy the condition of the theorem, therefore, byinduction hypothesis, there exists д , д , h , h such that: (cid:34) д д (cid:35) · (cid:104) h h (cid:105) ≡ (cid:34) f / p f f / p f (cid:35) Then the polynomials д , д , ph , h satisfy the condition of the theorem: (cid:34) д д (cid:35) · (cid:104) ph h (cid:105) ≡ (cid:34) f f f f (cid:35) □ The next two results concern variations on Lemma 1.1. We are given four monotone Boolean functions F , G , H , K over n variables, witharithmetizations f , д , h , k , and seek a valuation φ ∈ { , / , } n such that, on one hand f [ φ ] д [ φ ] (cid:44) h [ φ ] k [ φ ] , on the other hand φ avoids thevalue 0 as much as possible. We start with a simple case, when φ avoids the value 0 completely.If θ is a truth assignment of the Boolean variables of F , then we denote by F θ the Boolean function obtained from F by setting X : = θ ( X ) =
1, and leaving the other variables free. For example, if F ( X , X , X , X ) and θ ( X ) = θ ( X ) = θ ( X ) = θ ( X ) = F θ def = F ( , X , , X ) . We denote by F θ the number of models of the residual formula F θ .Lemma B.2. Let F , G , H , K be four monotone Boolean functions with n variables and f , д , h , k their arithmetization. If F ∧ G (cid:46) H ∧ K thenthere exists φ ∈ { / , } n such that f [ φ ] д [ φ ] (cid:44) h [ φ ] k [ φ ] . Proof. Let θ be an assignment that distinguishes the two. Assume w.l.o.g. that: F [ θ ] = , G [ θ ] ∈ { , } , H [ θ ] = K [ θ ] = m is the number of variables in F θ , G θ , H θ , K θ , then F θ < m because F [ θ ] =
0, G θ ≤ m , and H θ = K θ = m , which implies H θ K θ − F θ G θ = m − F θ G θ > □ Corollary B.3.
With the notations in Lemma B.2, if there exists any assignment φ ∈ { , / , } such that f [ φ ] д [ φ ] (cid:44) h [ φ ] k [ φ ] , then thereexists an assignment φ ′ ∈ { , / , } such that (cid:44) f [ φ ′ ] д [ φ ′ ] (cid:44) h [ φ ′ ] k [ φ ′ ] (cid:44) . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Proof. If f [ φ ] (cid:44) , д [ φ ] (cid:44)
0, and h [ φ ] , k [ φ ] (cid:44)
0, then we are done. Otherwise, assume w.l.o.g. that f [ φ ] = h [ φ ] , k [ φ ] (cid:44)
0. Let θ bethe following assignment: θ ( X ) = φ ( X ) =
0, and θ ( X ) =
1. We claim that F [ θ ] =
0. To see this, let θ ′ be θ except the variables that areassigned 1 / φ remain unassigned. We now observe that F θ ′ ( / , . . . , / ) = c · F θ ′ where c >
0. Since F θ ′ = F θ ′ satisfies it, including the assignment that sets all of its variables to 1.So, we have that F [ θ ] =
0, and H [ θ ] = K [ θ ] =
1, implying that F ∧ G (cid:46) H ∧ K , and we can use Lemma B.2. □ When F , G are equivalent to H , K then obviously no φ exists that satisfies Lemma B.2. But even when F , G are different from H , K , such a φ may still not exists, as illustrated by the following: Example B.4.
Consider the following four functions: F = ( X ∨ X ) ∨ ( Y ∨ Y ) ∨ T H = ( X ∨ X ) ∨ ( Y ∨ Y ) ∨ TG = ( X ∨ X ) ∨ ( Y ∨ Y ) ∨ T K = ( X ∨ X ) ∨ ( Y ∨ Y ) ∨ TT = ( Y ∨ Y ∨ Y )( X ∨ X ∨ X ) They are distinct, yet for any φ ∈ { / , } , f [ φ ] д [ φ ] = h [ φ ] k [ φ ] . Indeed, if we set any variable X i to 1 then F = H and G = K , if we set any Y j to 1 then F = K and G = H . Finally, if assign probabilities 1 / F = G = H = K bysymmetry.Lemma B.5. Let F , G , H , K be monotone Boolean functions with n variables, and let U , . . . , U k be some Boolean variables with the property that F [ U i : = ] ≡ G [ U i : = ] ≡ H [ U i : = ] ≡ K [ U i : = ] , for all i = , k . Assume that there exists φ ∈ { , / , } such that f [ φ ] д [ φ ] (cid:44) h [ φ ] k [ φ ] .Then there exists φ ∈ { , / , } such that (cid:44) f [ φ ] д [ φ ] (cid:44) h [ φ ] k [ φ ] (cid:44) and φ ( U i ) ∈ { / , } for all i = , k . Proof. The multilinear polynomial f admits the following expansion in terms of the variables u , . . . , u k : f = (cid:213) τ ∈{ , } k u τ f [ τ ] where u τ def = (cid:206) i : τ ( u i ) = ( − u i ) · (cid:206) i : τ ( u i ) = u i . By assumption, for every τ (cid:44) ( , , . . . , ) , we have f [ τ ] = д [ τ ] = h [ τ ] = k [ τ ] , thus we canwrite: f = (cid:214) i ( − u i ) f ( , . . . , ) + ∆ h = (cid:214) i ( − u i ) h ( , . . . , ) + ∆ д = (cid:214) i ( − u i ) д ( , . . . , ) + ∆ k = (cid:214) i ( − u i ) k ( , . . . , ) + ∆ where ∆ is the same quantity for all four polynomials.We can assume w.l.o.g. that F ∧ G ≡ H ∧ K , otherwise the claim follows immediately from Lemma B.2. We consider two cases. First, if F ∨ G (cid:46) H ∨ K . In that case we can assume w.l.o.g. that there exists a truth assignment θ such that F [ θ ] = G [ θ ] = H [ θ ] = K [ θ ] = θ ( U i ) = U i (otherwise F [ θ ] = K [ θ ] ). Define φ as follow: φ ( U i ) = / φ ( X ) = θ ( X ) when X (cid:60) { U , . . . , U k } . We firstnote that since K is monotonic, then k [ φ ] =
1. We compute f [ φ ] using the formula above, noting that for τ = ( , , . . . , ) , f =
0, and for anyother τ , f [ τ ] = k [ τ ] = K [ θ ] =
1. Therefore, v def = (cid:206) ( − u i ) = / k and ∆ = ( k − )/ k = − v . We obtain: f [ φ ] = ( − v ) > h [ φ ] = ( − v ) > д [ φ ] = ( − v ) > k [ φ ] = h [ φ ] k [ φ ] − f [ φ ] д [ φ ] > ( − v ) − ( − v ) > F ∧ G ≡ H ∧ K and F ∨ G ≡ H ∨ K . Then f + д ≡ h + k because:Pr ( F ∨ G ) = f + д − Pr ( F ∧ G ) = Pr ( H ∨ K ) = h + k − Pr ( H ∧ K ) Let φ be an assignment such that f [ φ ] д [ φ ] (cid:44) h [ φ ] k [ φ ] . Assume w.l.o.g. that φ ( U i ) = i ; otherwise, if φ ( U i ) = / U i from the list U , . . . , U k and decrease k . We define φ as follows: φ ( U i ) = / U i , and φ ( X ) = φ ( X ) for all other variables. To prove the claim of the lemma, let f , д , h , k be the polynomials in the variables u , . . . , u k obtainedby substituting all other variables X with the value φ ( X ) . Thus, f ( , , . . . , ) = f [ φ ] , and similarly for д , h , k , therefore, k [ φ ] h [ φ ] − f [ φ ] д [ φ ] = ( k h ( , . . . , ) + ∆ )( k k ( , . . . , ) + ∆ ) − ( k f ( , . . . , ) + ∆ )( k д ( , . . . , ) + ∆ ) = k (cid:0) h [ φ ] k [ φ ] − f [ φ ] д [ φ ] (cid:1) + ∆ k (cid:0) h [ φ ] + k [ φ ] − f [ φ ] − д [ φ ] (cid:1) = k (cid:0) h [ φ ] k [ φ ] − f [ φ ] д [ φ ] (cid:1) (cid:44) f + д = h + k . Also, ∆ >
0, because at least one of the four values f [ φ ] , д [ φ ] , h [ φ ] , k [ φ ] is >
0, hence setting any u i : = f [ φ ] , д [ φ ] , h [ φ ] , k [ φ ] are (cid:44) □ Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Next, we discuss tight connections between Boolean formulas, their arithmetizations, and conditional independence statements, of possibleindependent interest.
Definition B.6.
Fix a Boolean formula F . • We say that F is connected if, whenever F ≡ F ∧ F where F , F do not share any common Boolean variables, then either F ≡ true or F ≡ true . Otherwise we say that F decomposes into F , F . • We say that F disconnects two sets of variables U , V , if F ≡ F ∧ F where F , F do not share any common Boolean variables, V ∩ Vars ( F ) = ∅ , U ∩ Vars ( F ) = ∅ . Otherwise, we say that U , V are connected. • A Boolean variable X in F disconnects U , V if both F [ X : = ] and F [ X : = ] disconnect U , V .Notice that, if F does not depend on U , then it trivially disconnects U , V , by writing F = true ∧ F . We describe now the equivalent notionson multi-variate polynomials f . Definition B.7.
Let f be a multivariate polynomial. • We say that f is irreducible if, whenever f ≡ f · f , then either f or f is a constant polynomial. • We say that f disconnects two sets of variables u , v if f ≡ f · f and v ∩ Vars ( f ) = u ∩ Vars ( f ) = ∅ . • We say that x disconnects u , v if both f [ x : = ] and f [ x : = ] disconnect u , v .We prove that, if the arithmetization f factorizes f = д · h , then the associated Boolean function decomposes.Lemma B.8. Let F be a Boolean formula, and assume its arithmetization factorizes as f = д · h , where both д , h are non-constant multi-linearpolynomials. Then, there exists two Boolean formulas G , H such that F = G ∧ H and д , h are, up to a constant, the arithmetization of G , H ; inother words, there exists some constant c (cid:44) such that Pr ( G ) = c · д and Pr ( H ) = h / c . Proof. We can assume w.l.o.g. that , F (cid:46) false , and let θ be any assignment such that F [ θ ] = true . Then f [ θ ] = д [ θ ] h [ θ ] = θ , θ are the restrictions of θ to Vars ( д ) and Vars ( h ) respectively. Denote c = h [ θ ] (cid:44)
0, then д [ θ ] = / c . Define G , H as follows. Vars ( G ) = Vars ( д ) , Vars ( H ) = Vars ( h ) , and for any assignment θ ′ of Vars ( G ) , define G [ θ ′ ] def = c · д [ θ ′ ] (similarly for H ). We claim that this iscorrect, i.e. c · д [ θ ′ ] ∈ { , } . Indeed, consider the full assignment θ ′ = θ ′ ∪ θ . If F [ θ ′ ] = false then 0 = f [ θ ′ ] = д [ θ ′ ] h [ θ ] = c · д [ θ ′ ] , andif F [ θ ′ ] = true then 1 = f [ θ ′ ] = д [ θ ′ ] h [ θ ] = c · д [ θ ′ ] , proving the claim. Thus, Pr ( G ) = c · д and similarly Pr ( H ) = h / c . □ The following are easy to check:Lemma B.9.
Let F be a Boolean formula and f be the multilinear polynomial representing its arithmetization. Then: • F is connected iff f is irreducible. • F disconnects U , V iff f disconnects u , v . • X disconnects U , V iff x disconnects u , v . In this paper we are concerned only with monotone Boolean formulas F , which admit a unique, canonical CNF representation, where noclause is redundant (i.e. is not a superset of some other clause). Then, connectedness can be viewed as a graph-theoretic property, since it isequivalent to saying that there exists clauses C , C , . . . , C k such that U ∩ Vars ( C ) (cid:44) ∅ , V ∩ Vars ( C k ) (cid:44) ∅ and Vars ( C i − ) ∩ Vars ( C i ) (cid:44) ∅ for i = , k . Define the distance , d ( U , V ) , to be the minimum such k . Notice that we may have d ( U , V ) = U , V are single variables.Connectedness is also an algebraic property, and related to polynomial factorization. We will make use repeatedly of these equivalentformulations.Fix a set of variables U . For any number m ≥
0, define the ball B ( U , m ) def = { Z | d ( U , Z ) ≤ m } . The following is easy to check:Lemma B.10. Fix a monotone Boolean formula F .(1) If X disconnects U , V and X also disconnects U , V , then it disconnects U ∪ U , V .(2) If X disconnects U , V and m = d ( U , X ) , n = d ( V , X ) , then X disconnects B ( U , m − ) and B ( V , n − ) . A third characterization uses conditional independence. Let Pr (−) denote the distribution where each random variable X is set to true independently, with probability Pr ( X ) = x . For a fixed Boolean formula F , define Pr F the probability space Pr F def = Pr (−| F ) : that is, itsoutcomes are assignments that satisfy F . We write U ⊥ F V when U , V are independent in the probability space Pr F . We prove the following:Lemma B.11. A Boolean variable X disconnects U , V , iff U ⊥ F V | X . Proof. We start by establishing the connection between Pr F and the arithmetization f . Consider some partial assignment θ of thevariables in F with values in { , } . Then Pr F ( θ ) = f [ θ ]/ f , where the polynomial f is evaluated over values x = Pr ( X = ) , y = Pr ( Y = ) , . . . For example, if F = X ∨ Y , then Pr ( F ) = f ( x , y ) = x + y − xy and Pr F ( X = ) = Pr ( X = | F ) = f ( , y )/ f ( x , y ) = /( x + y − xy ) .Assume first that X disconnects U , V . Then F = G ∧ H and f = д · h , where u ⊆ Vars ( д ) , v ⊆ Vars ( h ) . For any assignments θ , θ ofthe variables U , V respectively, with values { , } we write U = θ for the event that the outcomes of U are those given by θ . Then wehave Pr F ( U = θ , V = θ ) = f [ U = θ , V = θ ]/ f = д [ U = θ ] · h [ V = θ ]/( д · h ) = Pr G ( U = θ ) · Pr H ( V = θ ) . Conversely, assume onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Pr F ( U = θ , V = θ ) = Pr F ( U = θ ) Pr F ( V = θ ) . Then f [ θ , θ ]/ f = ( f [ θ ]/ f ) · ( f [ θ ]/ f ) , or, equivalently, f [ θ , θ ] · f = f [ θ ] f [ θ ] . ByTheorem B.1 there exists polynomials д , д , h , h such that: f ≡ д h f [ θ ] ≡ д h f [ θ ] ≡ д h f [ θ θ ] ≡ д h Since f is multi-linear, Vars ( д ) ∩ Vars ( h ) = ∅ . From the first two identities we conclude that u ⊆ Vars ( д ) (since, recall, θ assigns thevariables u to 0 , v ⊆ Vars ( h ) . Thus, the factorization f ≡ д h disconnects u , v ,which proves the lemma. □ Recall that, for every joint distribution of random variables, if U U ⊥ V | X then U ⊥ V | X and U ⊥ V | X . The converse does not hold ingeneral, but it holds for ⊥ F , by Lemma B.10. Definition B.12.
Let F be a monotone Boolean formula, where X disconnects U , V , and let Y be any other variable in F . If F does notdisconnect either U Y , V , nor U , V Y , then we say that Y is a migrating variable w.r.t. X , U , V .Suppose X disconnects U , V in F , in other words F [ X = ] = G ∧ H and F [ X = ] = G ∧ H , where Vars ( G ) ∩ Vars ( H ) = ∅ , Vars ( G ) ∩ Vars ( H ) = ∅ and U occurs only in G , G while V occurs only in H , H . Consider where Y occurs. If it occurs in G , G then X separates Y and V ; if it occurs in H , H then X separates U and Y . If none of these hold, then we say that it is migrating.Lemma B.13. Assuming m = d ( U , X ) , n = d ( V , X ) , if Y migrates w.r.t. X , U , V , then d ( U , Y ) ≥ m − , and d ( V , Y ) ≥ n − . Proof. Follows from Lemma B.10, item 2. □ Example B.14.
Consider the following Boolean function: F = ( U ∨ Z ) ∧ def = C (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123) ( Z ∨ Z ∨ Z ∨ Z ) ∧ def = C (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123) ( Z ∨ X ∨ Y ) ∧ def = C (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123) ( X ∨ Y ∨ Z ) ∧( X ∨ Z ) ∧ ( Y ∨ Z ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = C ∧( Z ∨ V ) X disconnects U , V , and we have: F [ X : = ] = def = G (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123) ( U ∨ Z ) ∧ Z ∧ def = H (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123) ( Z ∨ Y ) ∧ ( Y ∨ Z ) ∧ ( Y ∨ Z ) ∧ ( Z ∨ V ) F [ X : = ] = ( U ∨ Z ) ∧ ( Z ∨ Z ∨ Z ∨ Z ) ∧ ( Y ∨ Z ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = G ∧ ( Z ∨ V ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = H Here Y , Z , and Z migrate from the right to the left.As we shall see, migrating variables add complexity to our proof. However, we prove that the migration property is symmetric: if X causes Y to migrate, then Y causes X to migrate. To prove this we use the following result of independent interest.Lemma B.15. Let X , Y , U , V be jointly distributed random variable, and assume that Y is binary (i.e. it has only two outcomes). Then, thefollowing implication holds: if ( U ⊥ V | X ) and ( U X ⊥ V | Y ) then either ( V ⊥ Y ) or ( U ⊥ Y | X ) . This implication does not hold in general, but it holds when Y is binary. A similar, but different example is given by Geiger and Pearl [6],in Corollary 8.Proof. Let Pr (−) denote the joint distribution. As usual we write Pr ( X ) or Pr ( XY ) etc, for the marginal distribution. The first condition, ( U ⊥ V | X ) says that Pr ( U X ) · Pr ( V X ) − Pr ( UV X ) · Pr ( X ) =
0. We use the fact that Y has only two outcomes, and expand each probability,using the formula Pr (−) = Pr (−| Y = )( − y ) + Pr (−| Y = ) y , where y def = Pr ( Y = ) , and further abbreviate p (−) = Pr (−| Y = ) and p (−) = Pr (−| Y = ) , thus, for example Pr ( U ) = p ( U )( − y ) + p ( U ) y . We also use the second condition, ( U X ⊥ V | Y ) , which impliesPr ( U X V | Y ) = Pr ( U X | Y ) · Pr ( V | Y ) and Pr ( V X | Y ) = Pr ( V | Y ) · Pr ( X | Y ) , and derive:0 = Pr ( U X ) · Pr ( V X ) − Pr ( UV X ) · Pr ( X ) = (cid:0) p ( U X )( − y ) + p ( U X ) y (cid:1) · (cid:0) p ( V X )( − y ) + p ( V X ) y (cid:1) − (cid:0) p ( UV X )( − y ) + p ( UV X ) y (cid:1) · (cid:0) p ( X )( − y ) + p ( X ) y (cid:1) = (cid:0) p ( U X )( − y ) + p ( U X ) y (cid:1) · (cid:0) p ( V ) p ( X )( − y ) + p ( V ) p ( X ) y (cid:1) − (cid:0) p ( U X ) p ( V )( − y ) + p ( U X ) p ( V ) y (cid:1) · (cid:0) p ( X )( − y ) + p ( X ) y (cid:1) We multiply out both products. The term p ( U X ) p ( V ) p ( X )( − y ) occurs in both products, and cancels out, and similarly for the term p ( U X ) p ( V ) p ( X ) y , thus we obtain:0 = (cid:0) p ( U X ) p ( V ) p ( X ) y ( − y ) + p ( U X ) p ( V ) p ( X ) y ( − y ) (cid:1) − (cid:0) p ( U X ) p ( V ) p ( X ) y ( − y ) + p ( U X ) p ( V ) p ( X ) y ( − y ) (cid:1) = (cid:0) p ( U X ) p ( X ) − p ( U X ) p ( X ) (cid:1) · (cid:0) p ( V ) − p ( V ) (cid:1) y ( − y ) Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
We can assume w.l.o.g. that y ( − y ) (cid:44)
0. If ( p ( V ) − p ( V )) = V ⊥ Y . Suppose p ( U X ) p ( X ) − p ( U X ) p ( X ) =
0, then: p ( U X ) p ( X ) = p ( U X ) p ( X ) This is equivalent to Pr ( U | X , Y = ) = Pr ( U | X , Y = ) , or U ⊥ Y | X . □ We prove:Corollary B.16.
Let F be a monotone, connected Boolean formula. Suppose X disconnects U , V , and Y also disconnects U , V . Then Y ismigrating w.r.t. X , U , V iff X is migrating w.r.t. Y , U , V . Proof. We prove the counterpositive: if X does not migrate w.r.t. Y , U , V , then Y does not migrate w.r.t. X , U , V . Since X disconnects U , V , then, by Lemma B.11, we have U ⊥ F V | X . If X does not migrate w.r.t. Y , U , V , then Y disconnects either U X , V or U , V X . Assuming theformer, we have U X ⊥ F V | Y , Lemma B.15 implies that either ( V ⊥ F Y ) or ( U ⊥ F Y | X ) holds. The first is not possible because F is connected,hence we have ( U ⊥ F Y | X ) . Then we also have ( U ⊥ F V Y | X ) , proving that Y does not migrate. □ C PROOF OF THEOREM 2.9 (2)
In this section we prove Theorem 2.9 (2): if Q is a bipartite, unsafe query of type II of length ≥
5, then PP CN F ≤ P GFOMC bi ( Q ) . For type Iqueries we proved that all probabilities in a block can be set to 1/2, and therefore we obtained a symmetry (more precisely, we obtained y = y ), allowing us to reduce from , / , Q in Def. 2.3. In this section Q is of type II-II, and here we simply call it of type II. Recall thedefinition of a left-rigth path from Def. 2.4: it is a sequence of clauses C , C , . . . , C k such that C is a left clause, C k is a right clause, andevery consecutive clauses share a common relational symbol. We will use repeatedly this simple fact:Lemma C.1. Fix a left-to-right path C , . . . , C k . If Q is a final query, then every symbol occurring in Q must also occur in the left-to-right path. Indeed, if a symbol S does not occur on the path, then the query Q [ S : = ] still contains the path C , . . . , C k , since none of these clausescontained S , and therefore Q [ S : = ] is an unsafe query, contradicting the assumption that it is final. C.1 The Coloring Count Problem
For Type I queries, our reduction was not directly from m ≥ , n ≥
2. For every bipartite graph G = ( U , V , E ⊆ U × V ) , a coloring is a pair of functions σ : U → [ m ] and τ : V → [ n ] . The coloring associates a color to each node. Let M def = max (| U | , | V | , | E |) . Given a coloring σ , τ , we denote by k α β ( σ , τ ) thenumber of edges whose endpoints are colored with α and β respectively; denote by k α , ˆ1 ( σ , τ ) the number of nodes u colored α , and by k ˆ1 , β ( σ , τ ) the number of nodes colored β . (The choice for the notation ˆ1 will be come clear in the next section.) All these numbers are ≤ M .Taking together, these numbers form the signature of a coloring σ , τ , which is a mapping k ( σ , τ ) : ([ m ]∪{ ˆ1 })×([ n ]∪{ ˆ1 }) → { , . . . , M } where k ˆ1 , ˆ1 ( σ , τ ) def =
0; equivalently, the signature is a vector with ( m + )( n + ) dimensions and with values in [ , M ] , i.e. ∈ { , . . . , M } ( m + )( n + ) .Formally: ∀ α ∈ [ m ] , ∀ β ∈ [ n ] : k α β ( σ , τ ) def = |{( u , v ) ∈ E | σ ( u ) = α , τ ( v ) = β }| k α ˆ1 ( σ , τ ) def = |{ u ∈ U | σ ( u ) = α }| k ˆ1 , β ( σ , τ ) def = |{ v ∈ V | τ ( v ) = β }| Signature of σ , τ : k ( σ , τ ) def = ( k α , β ( σ , τ )) α ∈[ m ]∪{ ˆ1 } , β ∈[ n ]∪{ ˆ1 } Conversely, given such a vector k : ([ m ] ∪ { ˆ1 }) × ([ n ] ∪ { ˆ1 }) → { , . . . , M } , its coloring count , k , is the number of colorings σ , τ withsignature k : ∀ k ∈ { , . . . , M } ( m + )( n + ) : k def = |{( σ , τ ) | k ( σ , τ ) = k }| Definition C.2 (Coloring Counting Problem).
We assume m ≥ , n ≥ CCP ( m , n ) , is thefollowing: given a bipartite graph ( U , V , E ⊆ U × V ) , compute all coloring counts: { k | k : ([ m ] ∪ { ˆ1 }) × ([ n ] ∪ { ˆ1 }) → { , . . . , M }} , where M = max (| U | , | V | , | E |) Notice that the number of coloring counts is ( M + ) ( m + )( n + ) , hence, under the assumption that m and n are constant (in other words, m , n = O ( ) ), then the size of the output is polynomial in the size of the graph. onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Theorem C.3.
For all m , n ≥ , CCP ( m , n ) is Proof. Assuming we have an oracle for
CCP ( m , n ) , we describe a PTIME algorithm for computing Φ = (cid:211) ( u , v )∈ E ( X u ∨ Y v ) be a PP2CNF, where E ⊆ U × V , and let M = | E | . Use the oracle to compute all coloring counts k , for all vectors k . Call a coloring σ , τ valid if it uses only two colors, i.e. σ ( U ) ⊆ { , } and τ ( V ) ⊆ { , } , and call a signature k valid if k α β = α > β >
2. Avalid coloring corresponds to a truth assignment, e.g. by associating false to color 1 and true to color 2. The assignment satisfies Φ iff itssignature satisfies k =
0. Thus, Φ is the sum of k over all valid, satisfying signatures k . □ For example, assume we have 3 colors for the left, a , b , c and 3 colors for the right, u , v , w . Then one can think of a vector k as a 4 × k = k au k av k aw k a ˆ1 k bu k bv k bw k b ˆ1 k cu k cv k cw k c ˆ1 k ˆ1 u k ˆ1 v k ˆ1 w k ˆ1ˆ1 Given a bipartite graph with M = | E | edges (we assume no isolated vertices), there are M matrices k . An oracle for the CCP ( , ) willcompute all M numbers k . To answer the a , b on the left, and u , v on theright, e.g. k cu = k cv = . . . =
0. Using the counts k for these matrices, we can obtain if Q is a bipartite, unsafe query of type II-II of length ≥ , then there exists numbers m , n that depend only on Q such that CCP ( m , n ) ≤ P GFOMC bi ( Q ) Theorem C.3 and Theorem C.4 prove Theorem 2.9 (2). In the rest of this appendix we prove Theorem C.4.
C.2 Möbius Inversion formula for Type II Queries
The principle behind the formula for Type I queries was the Shannon expansion formula: Pr ( F ) = Pr ( F [ X : = ])( − p ) + Pr ( F [ X : = ]) p ,where X is a boolean variable and p = Pr ( X ) . Essentially, we applied this formula repeatedly, once for each Boolean variable associated to aunary atom R ( u ) . For a Type II query, we no longer have unary atoms. Instead, we will use a different principle: the inclusion/exclusionformula, Pr ( F ∨ G ) = Pr ( F ) + Pr ( G ) − Pr ( F ∧ G ) . We start by expressing Q as a union, so we can enable the inclusion exclusion formula.Recall the definition of a bipartite query Q in Def. 2.3. Q is of type II-II, and here we simply call it of type II. We will rewrite Q as follows: Q = Q left ∧ Q middle ∧ Q right (46) Q left = ∀ x ( ∀ yG ( x , y ) ∨ . . . ∨ ∀ yG m ( x , y )) def = ∀ xG ( x ) (47) Q middle = ∀ x ∀ yC ( x , y ) (48) Q right = ∀ y ( ∀ xH ( x , y ) ∨ . . . ∨ ∀ xH n ( x , y )) def = ∀ yH ( y ) (49)where m ≥ , n ≥
2, and G i ( x , y ) , C ( x , y ) , H j ( x , y ) are CNF formulas, i.e. conjunctions of one or more clauses. Indeed, each type II querycan be written this way, because Q left is a conjunction of left clauses, each of the form ∀ x (cid:16)(cid:212) m ℓ = ∀ yS J ℓ ( x , y ) (cid:17) , hence we obtain (47) bydistributing ∧ over ∨ , in other words converting from CNF to DNF. We apply similar reason to Q right and obtain (49) Example C.5.
We illustrate this transformation on a Q left with two left clauses: Q left = ∀ x ( ∀ y ( S ( x , y ) ∨ S ( x , y )) ∨ ( ∀ y ( S ( x , y ) ∨ S ( x , y )))) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) left clause 1 ∧ ∀ x (( ∀ y ( S ( x , y ))) ∨ ∀ y ( S ( x , y ) ∨ S ( x , y ))) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) left clause 2 = ∀ x ( ∀ y ( S ( x , y ) (cid:124) (cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32) (cid:125) def = G ( x , y ) ) ∨ ∀ y (( S ( x , y ) ∨ S ( x , y )) ∧ ( S ( x , y )∨ S ( x , y )) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = G ( x , y ) ) ∨ ∀ y (( S ( x , y ) ∨ S ( x , y )) ∧ ( S ( x , y ) ∨ S ( x , y )) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = G ( x , y ) )) = ∀ x ( ∀ yG ( x , y ) ∨ ∀ yG ( x , y ) ∨ ∀ yG ( x , y )) Thus, we can write Q as: Q = ( ∀ x (cid:220) i ∀ yG i ) ∧ ( ∀ x ∀ yC ) ∧ ( ∀ y (cid:220) j ∀ xH j ) (50)which gets us closer to using the inclusion/exclusion formula. Here, each G i ( x , y ) and every H j ( x , y ) is a CNF formula, i.e. a conjunction ofclauses. At this point we observe that some of the terms in the inclusion/exclusion formula can be logically equivalent. For example, giventhree Boolean formulas F , F , F , the expansion of Pr ( F ∨ F ∨ F ) has 7 terms, but some may be logically equivalent, e.g. we may have F ∧ F ≡ F ∧ F ∧ F . In our proof it is important to ensure that all terms are logically in-equivalent (we make this precise in Lemma C.16below), and for that reason we consider next the lattice consisting of all logically inequivalent conjunctions: Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Definition C.6.
Let F = { F , . . . , F m } be a set of formulas; For each set α ⊆ [ m ] denote by F α def = (cid:211) i ∈ α F i . The closure of α is: ¯ α def = { i | F α ⇒ F i } . A set α is closed if α = ¯ α . The lattice associated with F denoted ˆ L ( F ) consists of all closed sets ordered by reverse set inclusion¯ α ≤ ¯ β if ¯ β ⊆ ¯ α . We denote by ˆ1 the top element of ˆ L ( F ) (that is, ˆ1 = ∅ ), and define F ˆ1 def = F ∨ . . . ∨ F m . The Möbius function µ : ˆ L ( F ) → Z isdefined as µ ( ˆ1 ) = µ ( α ) = − (cid:205) β > α µ ( β ) . The support of F is L ( F ) def = { α ∈ ˆ L ( F ) | µ ( α ) (cid:44) } .Intuitively, the lattice is obtained as follows. Compute all 2 m conjunctions F α , then group them into equivalence classes based on logicalequivalence. The lattice consists of all equivalence classes. In each class there exists a maximal α such that F α is in that class, and this α is closed; we use it as representative for the class. By convention, the top element of the lattice is defined as F ˆ1 = F ∨ · · · ∨ F m ; this isconsistent to what we need in the rest of this section, and also standard in the context of the Möbius function, see [9, pp.117]. The support L ( F ) is obtained by removing all elements α where µ ( α ) = Y = { Y , . . . , Y m } is aset of m formulas, then, Möbius’ inversion formula is:Pr ( Y ∨ . . . ∨ Y m ) = − (cid:213) α < ˆ1 µ ( α ) Pr ( Y α ) Obviously it suffices to sum only over the support, less ˆ1, i.e. α ∈ L ( F ) − { ˆ1 } , a fact that we will exploit later. As before, we write y for thearithmetization of Y , i.e. the probability above expressed in terms of the probabilities of the Boolean variables, z , z , . . . Then:Pr ( y ∨ . . . ∨ y m ) = − (cid:213) α < ˆ1 µ ( α ) · y α Example C.7.
Consider the following set Y = { Y , Y , Y } : Y = Z Z Y = Z Z Y = Z Z where Z Z means Z ∧ Z . We notice that Y = Y = Y = Y = Z Z Z and therefore ˆ L ( Y ) = {∅ , , , , } . The Möbius function is µ (∅) = µ ( ) = µ ( ) = µ ( ) = − µ ( ) =
2, thus the support is {∅ , , , , } , and we obtain:Pr ( Z Z ∨ Z Z ∨ Z Z ) = Pr ( Y ) + Pr ( Y ) + Pr ( Y ) − ( Y Y Y ) For another example, consider Y = { Y , Y , Y } where: Y = Z Z Y = Z Z Y = Z Z Then ˆ L ( Y ) = {∅ , , , , , , } , and µ (∅) = µ ( ) = µ ( ) = µ ( ) = − µ ( ) = µ ( ) = µ ( ) =
0. The support consists of ∅ , , , , ,
23 and thus: Pr ( Y ∨ Y ∨ Y ) = Pr ( Y ) + Pr ( Y ) + Pr ( Y ) − Pr ( Y Y ) − Pr ( Y Y ) Given a bipartite query Q of type II, we denote the following sets of formulas: G def = { G ( x , y ) ∧ C ( x , y ) , . . . , G m ( x , y ) ∧ C ( x , y )} H def = { C ( x , y ) ∧ H ( x , y ) , . . . , C ( x , y ) ∧ H n ( x , y )} where G , . . . , G m , H , . . . , H n are the CNF formulas that occur in (47), (48), and (49) respectively. We define two lattices: Definition C.8.
Fix a bipartite, unsafe query query Q of type II, as in Eq.(46). The left and right lattice supports of Q are L ( G ) and L ( H ) , where G and H are the sets defined above. The strict supports are L ( G ) def = L ( G ) − { ˆ1 } and L ( H ) def = L ( H ) − { ˆ1 } , and we denote by ¯ m = | L ( G )| and¯ n = | L ( H )| their sizes. Notice that 3 ≤ ¯ m ≤ m − ≤ ¯ n ≤ n − α ∈ L ( G ) and β ∈ L ( H ) : G α ( x ) def = ∀ yG α ( x , y ) (51) H β ( y ) def = ∀ xH β ( x , y ) (52) Q α β ( x , y ) def = G α ( x ) ∧ Q ∧ H β ( y ) (53)We notice that, if α , β (cid:44) ˆ1, then ∀ x ∀ yQ α β ( x , y ) is equivalent to the following expression (recall that Q middle = ∀ x ∀ yC ( x , y ) ). ∀ x ∀ yQ α β ( x , y ) = ∀ x ∀ y ( G α ( x , y ) ∧ C ( x , y ) ∧ H β ( x , y )) (54)because, for every α (cid:44) ˆ1, ∀ xG α ( x ) ⇒ Q left . Indeed, each CNF expression G i in (50) is a conjunction of subclauses S J k , one from eachleft clause. Therefore, for any left clause ∀ x ( (cid:212) k ∀ yS J k ( x , y )) in Q left , the logical implication G i ( x ) ⇒ (cid:212) k ∀ yS J k ( x , y ) holds, and therefore, ∀ xG α ( x ) ∧ Q left ≡ ∀ xG α ( x ) (since α (cid:44) ∅ ). On the other hand, if α = ˆ1, then: ∀ x ∀ yQ ˆ1 β ( x , y ) = Q ∧ ∀ yH β ( y ) ∀ x ∀ yQ α ˆ1 ( x , y ) = ∀ xG α ( x ) ∧ Q (55)because Q left ⇒ ∀ xG ˆ1 ( x ) and Q right ⇒ ∀ yH ˆ1 ( y ) . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Example C.9.
Consider: Q = ∀ x ( ∀ yS ( x , y ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) G ( x , y ) ∨ ∀ yS ( x , y ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) G ( x , y ) ) ∧ ∀ x ∀ y ( S ( x , y ) ∨ S ( x , y )) ∧ ∀ y ( ∀ xS ( x , y ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) H ( x , y ) ∨ ∀ xS ( x , y ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) H ( x , y ) ) Then: G ( x ) = ∀ yS ( x , y ) G ( x ) = ∀ yS ( x , y ) G ( x ) = ∀ y ( S ( x , y ) ∧ S ( x , y )) G ˆ1 ( x ) = ∀ y ( S ( x , y ) ∨ S ( x , y )) H ( y ) = ∀ xS ( x , y ) H ( y ) = ∀ xS ( x , y ) H ( x ) = ∀ x ( S ( x , y ) ∧ S ( x , y )) H ˆ1 ( y ) = ∀ x ( S ( x , y ) ∨ S ( x , y )) We show now a few examples of Q α β : ∀ x ∀ yQ , ( x , y ) = ∀ x ∀ y (cid:0) S ( x , y ) ∧ S ( x , y ) (cid:1) ∀ x ∀ yQ , ( x , y ) = ∀ x ∀ y (cid:0) S ( x , y ) ∧ S ( x , y ) (cid:1) ∀ x ∀ yQ , ( x , y ) = ∀ x ∀ y (cid:0) S ( x , y ) ∧ S ( x , y ) ∧ S ( x , y ) (cid:1) ∀ x ∀ yQ , ( x , y ) = ∀ x ∀ y (cid:0) S ( x , y ) ∧ ( S ( x , y ) ∨ S ( x , y )) ∧ S ( x , y ) (cid:1) Notice that the middle clause became redudant in all queries except Q , . C.3 Forbidden Queries of Type II
In our hardness proof we need all queries Q α β to be connected: in example C.9 none of these queries is connected. To ensure this property,we need to restrict our queries to a strict subclass of final queries of Type II, which are called forbidden queries in [4]. Every final query oftype II can be simplified to a forbidden query, hence it suffices to prove hardness for forbidden queries. In this section we give the formaldefinition of forbidden queries, prove the connectedness property, then prove that every final query of type II can be simplified to a forbiddenquery. The key results in this section are based on [4]. Here we expand those results and give a complete characterization of the forbiddenqueries. Our setting here is slightly simpler than that in [4], because we assume that the bipartite query Q is long, i.e. the shortest left-rightpath has length k ≥ C ( x ) = ∀ yS J ( x , y ) ∨ ∀ yS J ( x , y ) ∨ · · · A binary symbol U ( x , y ) is C -ubiquitous if it occursin all its subclauses S J , S J , . . . A binary symbol U ( x , y ) is left ubiquitous if it is C -ubiquitous for all left clauses C . We define similarly rightubiquitous symbols, and denote them with V . Definition C.10.
Let Q be a query of type II. Q is called a forbidden query if it is a final query, and, for every left-right path C , C , . . . , C k ofminimal length, every symbol in C is either ubiquitous, or occurs in C ; similarly, every symbol in C k is either ubiquitous or occurs in C k − .The following was shown in [4], and we included its proof here, later in this section:Theorem C.11. Let Q be a final query of type II of length k , where k ≥ . Then there exists a query Q ′ such that GFOMC bi ( Q ′ ) ≤ Pm GFOMC bi ( Q ) and Q ′ is either of type I, or Q ′ is a forbidden query of type II of length ≥ k . We also give a complete syntactic characterization of the forbidden queries, which is novel:Theorem C.12.
Let Q = Q left ∧ Q middle ∧ Q right be a forbidden query, and let U = { U , . . . , U r } , V = { V , . . . , V t } denote the sets ofleft-ubiquitous and right-ubiquitous symbols respectively, and let C , C , . . . , C k be a left-to-right path of minimal length. Then, every clause in Q has one of the following forms: • A left clause, (cid:212) i ∀ yS J i ( x , y ) , where each subclause S J i is: U ( x , y ) ∨ · · · ∨ U r ( x , y ) ∨ S J ( x , y ) ∨ S J ( x , y ) ∨ · · · where S J , S J , . . . ⊆ Symb ( C ) . Notice that the left clause C contains all left ubiquitous symbols. • The middle clause C does not contain any left-ubiquitous symbol. • If C is any middle clause that contains a left ubiquitous symbol, then Symb ( C ) ⊆ Symb ( C ) ∪ Symb ( C ) . • A middle clause without any ubiquitous symbols, of the form S j ( x , y ) ∨ S j ( x , y ) ∨ · · · . • Symmetric clauses on the right.Furthermore, if r > , then for each i = , . . . , r there exists at least one middle clause that contains U i and no other left ubiquitous symbol.Similarly, on the right.Example C.13. We illustrate here a simple forbidden query: ∀ x (cid:0) ∀ y ( U ( x , y ) ∨ S ( x , y )) ∨ ∀ y ( U ( x , y ) ∨ S ( x , y )) (cid:1) ∧ ∀ x ∀ y (cid:0) S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ) (cid:1) ∧ ∀ y (cid:0) ∀ x ( V ( x , y ) ∨ S ( x , y )) ∨ ∀ x ( V ( x , y ) ∨ S ( x , y )) (cid:1) Here U is a left-ubiquitous symbol, and V is a right-ubiquitous symbol. Example C.14.
The ubiquitous symbols need not be unique, and may occur in middle clauses. Consider: Q = ∀ x ( ∀ y ( U ( x , y ) ∨ U ′ ( x , y ) ∨ S ( x , y ) ∨ S ( x , y )) ∨ ∀ y ( U ( x , y ) ∨ U ′ ( x , y ) ∨ S ( x , y ) ∨ S ( x , y )) ∨ ∀ y ( U ( x , y ) ∨ U ′ ( x , y ) ∨ S ( x , y ) ∨ S ( x , y )))∧ ∀ x ∀ y ( S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ))∧ ∀ y ( ∀ x ( V ( x , y ) ∨ S ( x , y )) ∨ ∀ x ( V ( x , y ) ∨ S ( x , y )))∧ ∀ x ∀ y ( U ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y )) ∧ ∀ x ∀ y ( U ′ ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y )) Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Before we prove the two theorems, we show the two consequences that will need later in this paper.Lemma C.15 (Connected).
Let Q be a forbidden query. Then ∀ α ∈ L ( G ) , ∀ β ∈ L ( H ) , the queries ∀ x ∀ yQ α β ( x , y ) are connected, and dependon all relational symbols in Q . Proof. Fix a left-to-right path C , C , . . . , C k of minimal length; by Lemma C.1, all symbols in Q occur on this path. Referring to theexpressions in Eq. (54), all clauses C , . . . , C k − occur in C ( x , y ) . Assume first that α , β (cid:44) ˆ1, then ∀ x ∀ yQ α β ( x , y ) = ∀ x ∀ y ( G α ( x , y ) ∧ C ( x , y ) ∧ H β ( x , y )) , and we prove that none of the clauses C , . . . , C k − becomes redundant. Recall that G α ( x , y ) is a conjunction CNF expressions G i ( x , y ) , each of which is a conjunction of subclauses S J k ( x , y ) of some left clause of Q (see Example C.5). Therefore every clause S J k ( x , y ) of G α ( x , y ) contains all ubiquitous symbols, while none of the clauses C , . . . , C k − contains all ubiquitous symbols, proving that none of themiddle clauses on the minimal-length left-to-right path are not redundant. (In fact, by Theorem C.12, the middle clause C does not contain any ubiquitous symbol. Since the path is minimal then none of the middle clauses contain any ubiquitous symbol.) Some clauses S J k ( x , y ) of G α ( x , y ) may become redundant, but the only homomorphisms C ′ → S J k must be from some other clause C ′ of G α ( x , y ) : otherwise, if C ′ isa middle clause, then we obtain a homomorphism C ′ → C ′ , where C ′ is the left clause that contains S J k . Thus, at least one subclause of G α ( x , y ) has to be non-redundant, proving that Q α β depends on all relational symbols in Q . It remains to consider the cases α = ˆ1 or β = ˆ1;assuming α = ˆ1, by Eq. (55) ∀ x ∀ yQ ˆ1 α ( x , y ) = Q ∧ ∀ yH β ( y ) and the argument is similar.x □ Lemma C.16 (Invertible).
The mapping ( α , β ) (cid:55)→ Q α β ( x , y ) is invertible. More precisely: if the logical implication ∀ x ∀ yQ α β ( x , y ) ⇒ ∀ x ∀ yQ α β ( x , y ) holds, then α ≤ α and β ≤ β , in the lattices ˆ L ( G ) and ˆ L ( H ) respectively. Proof. We expand ∀ x ∀ yQ α β ⇒ ∀ x ∀ yQ α β and drop the quantifiers, to obtain: G α ( x , y ) ∧ C ( x , y ) ∧ H β ( x , y ) ⇒ G α ( x , y ) ∧ C ( x , y ) ∧ H β ( x , y ) Let V be all right ubiquitous symbols. If we set them to V : =
1, then H β [ V : = ] = β , therefore we obtain: G α ( x , y ) ∧ C [ V : = ]( x , y ) ⇒ G α ( x , y ) ∧ C [ V : = ]( x , y ) We conjoin both terms with C ( x , y ) and notice that C [ V : = ] ∧ C ≡ C and therefore we obtain: G α ( x , y ) ∧ C ( x , y ) ⇒ G α ( x , y ) ∧ C ( x , y ) By definition this means α ≤ α . We prove similarly that β ≤ β . □ We will now give the proof of Theorem C.11, but first illustrate the basic idea on an example.
Example C.17.
The query Q in Example C.9 is not forbidden; we repeat it here: Q = ∀ x ( ∀ yS ( x , y ) ∨ ∀ yS ( x , y )) ∧ ∀ x ∀ y ( S ( x , y ) ∨ S ( x , y )) ∧ ∀ y ( ∀ xS ( x , y ) ∨ ∀ xS ( x , y )) Notice that S does not occur in the middle clause. Define the following query, obtained by replacing S with a unary symbol R ( x ) : Q ′ = ∀ x ∀ y ( S ′ ( x , y ) ∨ R ( x )) ∧ ∀ x ∀ y ( S ′ ( x , y ) ∨ S ′ ( x , y )) ∧ ∀ y ( ∀ xS ′ ( x , y ) ∨ ∀ xS ′ ( x , y )) We claim that
GFOMC bi ( Q ′ ) ≤ Pm GFOMC bi ( Q ) ; since Q ′ is a query of Type I-II, we have already shown that GFOMC ( Q ′ ) is GFOMC ( Q ) is also ∆ ′ = ( Dom ′ , p ′ ) for Q ′ . Define the followingprobabilistic database ∆ = ( Dom , p ) for Q , where Dom def = Dom ′ ∪ { b } for a fresh constant b , and where the probabilities are defined asfollows, for all a , b ∈ Dom : p ( S ( a , b )) def = p ′ ( R ( a )) p ( S ( a , b )) def = j = , , p ( S j ( a , b )) def = p ( S j ( a , b )) def = p ′ ( S ′ j ( a , b )) In Q , we have ∀ yS ( x , y ) ≡ ( ∀ y (cid:44) b S ( x , y )) ∧ S ( x , b ) ≡ ∀ yS ′ ( x , y ) , where here the variable y in ∀ yS ′ ( x , y ) ranges over Dom ′ , i.e. without b . Similarly, ∀ yS ( x , y ) ≡ ( ∀ y (cid:44) b S ( x , y )) ∧ ( S ( x , b )) ≡ R ( x ) , etc, and the query Q becomes: Q = ∀ x ( ∀ yS ( x , y ) ∨ ∀ yS ( x , y ))∧ ∀ x ∀ y ( S ( x , y ) ∨ S ( x , y ))∧ ∀ y ( ∀ xS ( x , y ) ∨ ∀ xS ( x , y ))≡ ∀ x ( R ( x ) ∨ ∀ yS ′ ( x , y ))∧ ∀ x ∀ y ( S ′ ( x , y ) ∨ S ′ ( x , y ))∧ ∀ y ( ∀ xS ′ ( x , y ) ∨ ∀ xS ′ ( x , y )) which is equivalent to Q ′ , proving Pr ( Q ) = Pr ′ ( Q ′ ) .Proof. (of Theorem C.11) Fix a left-right path in Q , not necessarily of minimal length, denote it C , C , . . . , C k , and recall that C is aunion of of subclauses C = ∀ x ( ∀ yS J ( x , y ) ∨ ∀ yS J ( x , y ) ∨ · · · ) . We start with the following:Claim 1 (Variant of Lemma 8.36 in [4]). Suppose S is a symbol that occurs in both C , C , S is a symbol that occurs in C and does notoccur in C , C , . . . , C k . Then, if there exists a subclause S J ( x , y ) of C that contains S but not S , then there exists an unsafe query Q ′ withstrictly fewer binary symbols such that GFOMC bi ( Q ′ ) ≤ Pm GFOMC bi ( Q ) . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu The intuition is that, since S only occurs in C , it should be a ubiquitous symbol, but fails to be one; then we can simplify Q to Q ′ . Thequery Q ′ will have left clauses that are slightly more general than those introduced in Definition 2.3: it may contain left clauses of the form: ∀ x (cid:0) R ( x ) ∨ R ( x ) ∨ · · · ∨ ∀ yS J ( x , y ) ∨ ∀ yS J ( x , y ) ∨ · · · (cid:1) Its middle and right clauses are as given in Definition 2.3. That is, its left clauses may contain multiple unary symbols and/or multiplesubclauses. Such left clauses can be further simplified to either Type I left clauses, or Type II leff clauses, see Propositions 8.6 and 8.7 in [4].Proof. (Of Claim 1) Let S J , . . . , S J t be all the subclauses of C that contain S ; by assumption there exists at least one other sub-clausethat contains S and not S . Define Q ′ the query obtained from Q as follows. The vocabulary consists of (a) for every symbol S j in Q otherthan S , there is a fresh binary symbol S ′ j ( x , y ) in Q ′ . (b) for every q = , . . . , t and every symbol S j that occurs in the subclause S J q (including S ) there is a fresh unary symbol denoted R ( q ) j ( x ) . Notice that Q ′ has one less binary symbol, since there is no S ′ ( x , y ) . For any subclause S J ( x , y ) = S j ( x , y ) ∨ S j ( x , y ) ∨ · · · ∨ S j m ( x , y ) that occurs anywhere in Q , we denote the following expressions: S ′ J def = S ′ j ( x , y ) ∨ · · · ∨ S ′ j m ( x , y ) R ( q ) J = R ( q ) j ( x ) ∨ · · · ∨ R ( q ) j m ( x ) if Symb ( S J ) ⊆ Symb ( S J q ) true otherwiseWe construct the query Q ′ from Q by replacing each subclause S J ( x , y ) with a new expression, according to the following two cases (thejustification will become clear below, when we describe the mapping from ∆ to ∆ ′ ): Case 1: S ∈ Symb ( S J ) . Then replace S J ( x , y ) with (cid:211) q = , t R ( q ) J ( x ) . (Note that this might be true .) Case 2: S (cid:60) Symb ( S J ) . Then replace S J ( x , y ) with (cid:211) q = , t R ( q ) J ( x ) ∧ S ′ J ( x , y ) We show now that
GFOMC bi ( Q ′ ) ≤ Pm GFOMC bi ( Q ) . Given a database ∆ ′ = ( Dom ′ , p ′ ) , we define ( Dom def = Dom ∪ { b , b , . . . , b t } , p ) , where b , . . . , b t are fresh constants, and define: ∀ q = , t : p ( S ( a , b q )) def = p ′ ( R ( q ) ( a )) p ( S ( a , b )) def = ∀ q = , t : ∀ S j ∈ Symb ( S J q ) − { S } p ( S j ( a , b q )) def = p ′ ( R ( q ) j ( a )) p ( S j ( a , b )) def = p ′ ( S ′ j ( a , b )) ∀ q = , t : ∀ S j ∈ Symb ( Q ) − Symb ( S J q ) p ( S j ( a , b q )) def = p ( S j ( a , b )) def = p ′ ( S ′ j ( a , b )) We check that the lineage of Q on ∆ is the same as that of Q ′ on ∆ ′ . Consider any subclause of Q and assume first that it contains S : S J ( x , y ) = S ( x , y ) ∨ S j ( x , y ) ∨ · · · When we substitute y : = b q , then S ( x , b q ) ≡ R ( q ) ( x ) , and for every other symbol S j i ( x , b q ) is either R ( q ) j i ( x ) or true , hence the expression is equivalent to R ( q ) J ; when we substitute y : = b for some b ∈ Dom ′ , then S ( x , b ) ≡ true and entireexpression vanishes. This justifies Case 1 above. Case 2 is justified similarly and ommitted.It remains to prove that Q ′ is an unsafe query. For that we prove that, once converted into CNF, Q ′ has a left-right path of clauses that isnon-redundant. We start by observing that every clause C , C , . . . , C k on our path (other than C ) is converted into an isomorphic clause,where each symbol S j ( x , y ) is replaced by S ′ j ( x , y ) . Indeed, by assumption, no subclause S J ( x , y ) on this path contains S , hence they areconverted according to case 2. Moreover, R ( q ) J ≡ true , because there must exist a symbol S j in S J that does not occur in C : indeed, when S J is any middle clause C i , i = , . . . , k −
1, and
Symb ( C i ) = Symb ( S J ) ⊆ Symb ( S J ) then there is a homomorphism C i → S J → C , contradiction.Since the length of Q is ≥ S J has no common symbols with the right clause C k . Consider any other clause C of Q . Each of its subclauses S J is converted to a conjunction of up to t + (cid:211) q = , t R ( q ) J ( x ) or (cid:211) q = , t R ( q ) J ( x ) ∧ S ′ J ( x , y ) ; notice that some expressions R ( q ) J may be ≡ true . We convert the resulting expression into CNF, thus from C we obtain a conjunction C ′ ∧ C ′′ ∧ · · · , each obtained bymaking one choice out of the up to t + C . Considering now C , we define C ′ to be obtained as follows: (a)for each subclause S J q ( x , y ) of C , choose R ( q ) J ( x ) (actually one can check that no other choice exists here) (b) for each other subclause S J ,choose S ′ J . That is, C looks like this: R ( ) J ( x ) ∨ R ( ) J ( x ) ∨ · · · ∨ R ( t ) J t ( x ) ∨ ∀ yS ′ J ( x , y ) ∨ · · · By assumption there exists at least one subclause S J other than S J , . . . , S J t , hence C ′ is a left clause, i.e. it does not degenerate to (cid:212) q R ( q ) J q ( x ) . (If we applied this construction to Example C.13trying to remove U ( x , y ) , then the left clause degenerates to R ( ) ( x ) ∨ R ( ) ( x ) ∨ R ( ) ( x ) ∨ R ( ) ( x ) .) Furthermore, the symbol S ′ is common in C ′ and C ′ , hence C ′ , C ′ , . . . , C ′ k is a left-right path. It remains to prove that it is not redundant. For that, assume the contrary, that thereexists a homomorphism f : C ′ → C ′ i , for some clause C ′ (cid:46) true . Let C be the original clause from which we derived C ′ (thus C became C ′ ∧ C ′′ ∧ · · · ), and consider any subclause S J ( x , y ) of C . We will construct a homomorphism д : S J ( x , y ) → C i ; by taking their union weobtain a homomorphism C → C i , contradicting the fact that Q has no redundant clauses. To construct д , we consider the two cases above.Case 1: S ∈ Symb ( S J ) , then C ′ must contain some expression R ( q ) J ( x ) for some choice of q = , . . . , t , and since R ( q ) J ( x ) (cid:46) true (otherwise C ′ ≡ true ), we have that Symb ( S J ) ⊆ Symb ( S J q ) . Given the homomorphism f : C ′ → C ′ i , we have that C ′ i contains R ( q ) J ( x ) , hence it must be Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA that i =
0. We simply define д to map S J to the subclause S J q of C . Case 2: S (cid:60) Symb ( S J ) . Its translation is (cid:211) q = , t R ( q ) J ( x ) ∧ S ′ J ( x , y ) , hence C ′ must contain either some R ( q ) J ( x ) , in which case we argue as in Case 1, or contains S ′ J ( x , y ) . In that case we use the homomorphism f : itmaps S ′ J ( x , y ) to C ′ i , we simply define д similarly from S ′ J ( x , y ) to C i .This concludes our proof of Claim 1. □ Example C.18.
We illustrate with an example showing some of the subtleties of the proof of Claim 1. Consider two clauses C , C thatform the beginning of a left-right path. We apply Claim 1 to the symbol S in C , noting that it does not co-occur with S , and show theirconversions to C ′ , C ′ below: C = ∀ x (cid:0) ∀ yS ( x , y ) ∨ ∀ y ( S ( x , y ) ∨ S ( x , y )) ∨ ∀ y ( S ( x , y ) ∨ S ( x , y )) (cid:1) C = ∀ x ∀ y (cid:0) S ( x , y ) ∨ S ( x , y ) ∨ S ( x , y ) (cid:1) C ′ = ∀ x (cid:0) ∀ yS ′ ( x , y ) ∨ R ( ) ( x ) ∨ R ( ) ( x ) ∨ R ( ) ( x ) ∨ R ( ) ( x ) (cid:1) C ′ = ∀ x ∀ y (cid:0) S ′ ( x , y ) ∨ S ′ ( x , y ) ∨ S ′ ( x , y ) (cid:1) There are two subclauses containing S . Thus, there are two unary symbols R ( ) , R ( ) because S occurs in both, and only one symbol R ( ) andone symbol R ( ) because S , S occur only in one subclause respectively. We illustrate why C ′ , C ’ do not become redundant, by consideringseveral other clauses, which are not part of the left right path, and thus we denote them D , D , . . . : D = ∀ x (cid:0) ∀ y ( S ( x , y ) ∨ S ( x , y )) ∨ ∀ yS ( x , y ) (cid:1) D ′ = ∀ x (cid:0) true ∨ (cid:0) R ( ) ( x ) ∧ ∀ yS ′ ( x , y ) (cid:1)(cid:1) ≡ true D = ∀ x (cid:0) ∀ y ( S ( x , y ) ∨ S ( x , y )) ∨ ∀ yS ( x , y ) (cid:1) D ′ = ∀ x (cid:0) ∀ y ( S ′ ( x , y ) ∨ S ′ ( x , y )) ∨ (cid:0) R ( ) ( x ) ∧ ∀ yS ′ ( x , y ) (cid:1)(cid:1) D = ∀ x (cid:0) ∀ yS ( x , y ) ∨ ∀ y ( S ( x , y ) ∨ S ( x , y )) (cid:1) D ′ = ∀ x (cid:0) ∀ yS ′ ( x , y ) ∨ ∀ y ( S ′ ( x , y ) ∨ S ′ ( x , y )) (cid:1) Clause D rewrites to true hence does not lead to any redundancies. Clause D rewrites to the expression D ′ , which further rewrites to twoclauses: ∀ x ( ∀ y ( S ′ ( x , y ) ∨ S ′ ( x , y )) ∨ R ( ) ( x )) and ∀ x ( ∀ y ( S ′ ( x , y ) ∨ S ′ ( x , y )) ∨ ∀ yS ′ ( x , y )) . Considering the first clause, we notice that thereexists a homomorphism from R ( ) ( x ) to C ′ , but this is because there was a homomorphism from ∀ yS ( x , y ) to ∀ y ( S ( x , y ) ∨ S ( x , y )) ; thisdoes not extend to the entire clause (otherwise there would be a homomorphism D → C ). Finally, in D we observe that the subclause S J def = S ( x , y ) ∨ S ( x , y ) rewrites to S ′ ( x , y ) ∨ S ′ ( x , y ) , because R ( ) J ( x ) ≡ R ( ) J ( x ) ≡ true since S , S occur in separate clauses with S ; onesubtlety of the proof of the claim is precisely to ensure that and expression like this does not rewrite to R ( ) ( x ) ∧ R ( ) ( x ) ∧ ∀ yS ′ ( x , y ) , becauseafter converting to CNF it will render C ′ redundant.Continuing the proof of Theorem C.11, we can now assume w.l.o.g. that for every S common to C , C and for every S occurring only in C , every subclause of C containing S must also contain S . Next we show:Claim 2. Every subclause S J ( x , y ) of C has some common symbol with C (this is Lemma 8.38 in [4]). Indeed, if S ′ J is any other subclause of C that does have some common symbol S with C , then all symbols S of S J must occur in S ′ J ,therefore ∀ yS J ( x , y ) ∨ ∀ yS ′ J ( x , y ) ≡ ∀ yS J ( x , y ) , contradicting the assumption that C has non redundant subclauses, proving claim 2.At this point we will restrict the left-right path C , C , . . . , C k to be of minimal length.Claim 3. Every symbol in C is either C -ubiquitous, or occurs in C . Proof. Let S ∈ Symb ( C ) − Symb ( C ) . By minimality, S does not occur in C , C , . . . , C k . We prove that S is C -ubiquitous. If it is not,then there exists a subclause S J that does not contain S . By Claim 2, S J contains some symbol S common with C , thus, by Claim 1 S J must contain S , contradiction. □ Claim 4.
There exists at least one C -ubiquitous symbol. Indeed, otherwise, all symbols of C occur in C , which implies that there exists a homomorphism C → C (because C is a middleclause), contradicting the fact that Q has no redundant clauses; this proves the claim. Finally, we prove:Claim 5. If C , C ′ are two left clauses then the C -ubiquitous symbols are the same as the C ′ -ubiquitous symbols. We first prove that C ′ shares some common symbols with C . Assuming otherwise, since all symbols in Q must occur on the path C , C , . . . , C k , and this path has minimal length, it follows that all symbols of C ′ occur in C , none occur in C (by the assumption inthe claim), thus they are C -ubiquitous. In particular, Symb ( C ′ ) ⊆ Symb ( S J ) for any subclause S J of C , which implies that there existshomomorphisms C ′ → S J → C contradicting the assumption that C is non-redundant. Thus, C ′ shares some common symbols with C ,and therefore C ′ , C , C , . . . , C k is also a left-right path of minimal length. Since the previous argument applies to this path as well, symbolsin C ′ are also partitioned into those common with C and symbols ubiquitous in C ′ . Consider a symbol ubiquitous in C ′ : it must occuron the path C , C , . . . , C k and, since it doesn’t occur in C , it must occur in C (since the path is of minimal length), implying that it isubiquitous in C . By symmetry, every ubiquitous symbol in C is also a ubiquitous symbol in C ′ . This proves the claim. □ Finally, we prove Theorem C.12, which follows from the next technical lemma. onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu
Lemma C.19.
Let Q be a forbidden query and let C , C , . . . , C k be a left-right path of minimal length. Then (1) There exists at least oneleft ubiquitous symbol that does not occur in C ; in particular, the query Q has at least one left ubiquitous symbol U . (2) More: no ubiquitoussymbol occurs in C . (3) for every left clause C = ∀ yS J ( x , y ) ∨ ∀ yS J ( x , y ) ∨ · · · , every subclause S J i has some common symbol with C . (4)If there are more than one left ubiquitous symbols, then each of them occurs in some middle clause. (5) For any clause C other than C , if Symb ( C ) ⊆ Symb ( C ) ∪ Symb ( C ) then C contains some ubiquitous symbol. (6) If C is a middle clause containing some left ubiquitous symbol U ,then Symb ( C ) ∩ Symb ( C ) = ∅ ; equivalently, Symb ( C ) ⊆ Symb ( C ) ∪ Symb ( C ) . Proof. (1) Assuming otherwise, then by Claim 3, every symbol of C must occur in C , Symb ( C ) ⊆ Symb ( C ) which implies that thereexists a homomorphism C → C . (2) We strengthen the claim by showing that no ubiquitous symbol can occur in C . Let U occur in C . We claim that Q [ U : = ] is an unsafe query, contradicting the assumption that Q is final. For that we prove that none of the clauses C [ U : = ] , C [ U : = ] , C , . . . , C k is redundant in Q [ U : = ] . It is easy to see that there is no homomorphism C [ U : = ] → C i [ U : = ] for i = ,
1, because both C , C contain U and that would imply the existence of a homomorphism C → C i . Assuming there exists ahomomorphism C [ U : = ] → C i for i ≥
2; we must have i =
2, because the path has minimal length. Let S be a non-ubiquitous symbolin Symb ( C ) ∩ Symb ( C ) , thus S (cid:60) Symb ( C ) since the path is of minimal length. Consider now the left-right path C , C , C , C , . . . , C k in Q . This is also of minimal length, hence by the definition of a forbidden query, S must occur in C , but this contradicts the existence of ahomomorphism C [ U : = ] → C . (3) From Claim 5, we know that the C -ubiquitous symbols are the same as the C -ubiquitous symbols.Since C , C , . . . , C k is of minimal length and contains all relational symbols of Q , then Symb ( C ) ⊆ Symb ( C ) ∪ Symb ( C ) . Now, suppose that S J i is a sub-clause of C that does not contain any symbol from C . This means that S J i contains only left ubiquitous symbols. Since thequery is minimized, then S J i is the only subclause of C . But then, C → C because every subclause of C contains all ubiquitous symbols (i.e., Symb ( S J i ) ), which is a contradiction.(4) Suppose there are at least two left ubiquitous symbols U , U , and suppose U does not occur in any middle clause. We claim that Q [ U : = ] is an unsafe query, contradicting the fact that Q is final. To prove the claim, we show that none of the clauses C [ U : = ] , C , . . . , C k in Q [ U : = ] is redundant. Assume that there exists a homomorphism C [ U : = ] → C i for i ≥
1; then C must contain U (otherwise thereexists a homomorphism C → C i ), hence C is a left clause, but in that case it also contains U , hence the homomorphism is not possible.Similarly, if there exists a homomorphism C [ U : = ] → C [ U : = ] , then there exists a homomorphism C → C , because C contains U inevery subclause. (5) Suppose Symb ( C ) ⊆ Symb ( C ) ∪ Symb ( C ) . If C has no ubiquitous symbols, then we claim that Symb ( C ) ⊆ Symb ( C ) , butthis implies that there exists a homomorphism C → C which is a contradiction. To prove the claim, let S be any symbol in C . Since S is notubiquitous, if S ∈ Symb ( C ) then it must also belong to C , proving the claim. (6) Assume otherwise, then C , C , C , . . . , C k is also a left-rightpath of minimal length, hence by item (2) C cannot contain any ubiquitous symbol, contradiction. □ C.4 Computing Q on a Block-database Recall that Φ ∆ ( Q ) denotes the lineage of Q on the TID ∆ . As before, a block B ( u , v ) is a bipartite TID with two distinguished constants u , v .We denote by U ( B ) , V ( B ) the two partitions of the domain of B , that is Dom ( B ) = U ( B ) ∪ V ( B ) . By definition, u ∈ U ( B ) , v ∈ V ( B ) . We define: Y α β ( u , v ) def = Φ B ( u , v ) ( Q α β ( u , v )) (56)where Q α β was introduced in Eq. (53). We prove the following theorem, which is the analogous of Theorem 3.4 for type II queries: Recallthat L ( G ) , L ( H ) are the supports of the left lattice, and the right lattice of the query, and this includes ˆ1. In this section we want to remove ˆ1,and denote L ( G ) def = L ( G ) − { ˆ1 } , L ( H ) def = L ( H ) − { ˆ1 } .Theorem C.20. Let U , V be two disjoint sets, and let ∆ = (cid:208) u ∈ U , v ∈ V B ( u , v ) be a disjoint union of blocks (that is, no two blocks share anytuple or any constant, expect for endpoints u , v ). Then: Pr ( Q ) = (− ) | U | + | V | (cid:213) σ : U → L ( G ) , τ : V → L ( H ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V µ ( τ ( v )) (cid:170)(cid:174)(cid:172) · (cid:214) u ∈ U , v ∈ V Pr ( Y σ ( u ) τ ( v ) ( u , v )) (57)The sum above has exponentially many terms, namely 2 | L ( G )| + | L ( H )| ; we will show later how to use this formula.Proof. The TID ∆ is bipartite, hence its domain is partitioned into Dom ( ∆ ) = ¯ U ∪ ¯ V . We have U ⊆ ¯ U , V ⊆ ¯ V , and the inclusions may bestrict since each block B ( u , v ) may have its own left and right constants, other than u , v .By definition, Φ ∆ ( Q ) = Φ ∆ ( Q left ) ∧ Φ ∆ ( Q middle ) ∧ Φ ∆ ( Q right ) . Since Q left = ∀ x ( (cid:212) i = , m ∀ yG i ( x , y )) , its lineage is given by Φ ∆ ( Q left ) = (cid:211) a ∈ ¯ U Φ ∆ ( (cid:212) i = , m ∀ yG i ( a , y )) . When a ∈ ¯ U − U , then the lineage Φ ∆ ( (cid:212) i = , m ∀ yG i ( a , y )) is equal to the lineage restricted to the uniqueblock B ( u , v ) that contains a , i.e. Φ ∆ ( (cid:212) i = , m ∀ yG i ( a , y )) = Φ B ( u , v ) ( (cid:212) i = , m ∀ yG i ( a , y )) , because, by construction, a can only be connectedto constants in the same block. When a ∈ U , then the lineage may span multiple blocks. Using this observation, and repeating it for Q right , Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA we derive the following expressions for the lineages: Φ ∆ ( Q left ) = (cid:219) a ∈ ¯ U Φ ∆ ( (cid:220) i ∀ y ( G i ( a , y ))) = (cid:169)(cid:173)(cid:171) (cid:219) u ∈ U Φ ∆ ( (cid:220) i ∀ y ( G i ( u , y ))) (cid:170)(cid:174)(cid:172) ∧ (cid:169)(cid:173)(cid:171) (cid:219) a ∈ ¯ U − U Φ ∆ ( (cid:220) i ∀ y ( G i ( a , y ))) (cid:170)(cid:174)(cid:172) = (cid:169)(cid:173)(cid:171) (cid:219) u ∈ U Φ ∆ ( (cid:220) i ∀ y ( G i ( u , y ))) (cid:170)(cid:174)(cid:172) ∧ (cid:169)(cid:173)(cid:171) (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( Q left ) (cid:170)(cid:174)(cid:172) Φ ∆ ( Q right ) = (cid:169)(cid:173)(cid:171) (cid:219) v ∈ V Φ ∆ ( (cid:220) j ∀ x ( H j ( x , v ))) (cid:170)(cid:174)(cid:172) ∧ (cid:169)(cid:173)(cid:171) (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( Q right ) (cid:170)(cid:174)(cid:172) Φ ∆ ( Q middle ) = (cid:220) u ∈ U , v ∈ V Φ B ( u , v ) ( Q middle ) Φ ∆ ( Q ) = Φ ∆ ( Q left ) ∧ Φ ∆ ( Q middle ) ∧ Φ ∆ ( Q right ) = (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) (cid:219) u ∈ U Φ ∆ ( (cid:220) i ∀ y ( G i ( u , y ) ∧ C ( u , y ) (cid:124) (cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32) (cid:125) part of Q )) (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) ∧ (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( Q ) ∧ (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) (cid:219) v ∈ V Φ ∆ ( (cid:220) j ∀ x ( H j ( x , v ) ∧ C ( x , v ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32) (cid:125) part of Q )) (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) (58)In the last line we added the redundant terms C ( u , y ) and C ( x , v ) (recall that Q middle = ∀ x ∀ yC ( x , y ) ). Now we will apply Möbius’ inversionformula on the expression for Φ ∆ ( Q ) above, repeatedly, once for each u ∈ U , then once for each v ∈ V . We show how to do it for one fixedconstant u ∈ U . First, we separate u from the conjunction (cid:211) u ∈ U above: Φ ∆ ( Q ) = Φ ∆ ( (cid:220) i ∀ y ( G i ( u , y ) ∧ C ( u , y )))∧ (cid:169)(cid:173)(cid:173)(cid:171) (cid:219) u ∈ U −{ u } Φ ∆ ( (cid:220) i ∀ y ( G i ( u , y ) ∧ C ( u , y ))) (cid:170)(cid:174)(cid:174)(cid:172) ∧ (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( Q ) ∧ (cid:169)(cid:173)(cid:171) (cid:219) v ∈ V Φ ∆ ( (cid:220) j ∀ x ( H j ( x , v ) ∧ C ( x , v ))) (cid:170)(cid:174)(cid:172)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) def = REST = Φ ∆ ( (cid:220) i ∀ y ( G i ( u , y ) ∧ C ( u , y ))) ∧ REST = (cid:220) i (cid:0) Φ ∆ ( ∀ yG i ( u , y ) ∧ C ( u , y )) ∧ REST (cid:1) Pr ( Q ) = Pr ( Φ ∆ ( Q )) = − (cid:213) α ∈ L ( G ) µ ( α ) Pr (cid:0) Φ ∆ ( ∀ y ( G α ( u , y ) ∧ C ( u , y ))) ∧ REST (cid:1)
Recall that the lattice ˆ L ( G ) was defined by the formulas G ∧ C , G ∧ C , . . . , and not by G , G , . . . We introduced earlier the term C ( u , y ) in order to be able to apply the Möbius formula for this lattice. Next, we consider formula REST, which contains the conjunction (cid:211) u ∈ U −{ u } .We separate a second u ∈ U − { u } , and repeat this argument for u , u , . . . ∈ U , reducing the REST formula, until we arrive at:Pr ( Q ) = (− ) | U | (cid:213) σ : U → L ( G ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) Pr ( (cid:219) u ∈ U Φ ∆ ( ∀ y ( G σ ( u ) ( u , y ) ∧ C ( u , y ))) ∧ REST ) where REST def = (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( Q ) ∧ (cid:169)(cid:173)(cid:171) (cid:219) v ∈ V Φ ∆ ( (cid:220) j ∀ x ( H j ( x , v ) ∧ C ( x , v ))) (cid:170)(cid:174)(cid:172) (59)We repeat the same process on the right clauses (cid:211) v ∈ V Φ ∆ (cid:16)(cid:212) j ∀ x ( H j ( x , v ) ∧ C ( x , v )) (cid:17) , and obtain:Pr ( Q ) = (− ) | U | + | V | (cid:213) σ : U → L ( G ) , τ : V → L ( H ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V µ ( τ ( v )) (cid:170)(cid:174)(cid:172) Pr ( (cid:219) u ∈ U Φ ∆ ( ∀ y ( G σ ( u ) ( u , y ) ∧ C ( u , y ))) ∧ (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( Q ) ∧ (cid:219) v ∈ V Φ ∆ ( ∀ x ( H τ ( v ) ( x , v ) ∧ C ( x , v )))) (60) onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Next, we observe that: Φ ∆ ( ∀ y ( G σ ( u ) ( u , y ) ∧ C ( u , y ))) = (cid:219) v ∈ V Φ B ( u , v ) ( ∀ y ( G σ ( u ) ( u , y ) ∧ C ( u , y ))) (61) Φ ∆ ( ∀ x ( H τ ( v ) ( x , v ) ∧ C ( x , v ))) = (cid:219) u ∈ U Φ B ( u , v ) ( ∀ x ( H τ ( v ) ( x , v ) ∧ C ( x , v ))) Since C is the middle part of Q , we have Φ B ( u , v ) ( C ( u , y ) ∧ Q ) ≡ Φ B ( u , v ) ( Q ) and therefore we can eliminate C ( u , v ) and obtain:Pr ( Q ) = (− ) | U | + | V | (cid:213) σ : U → L ( G ) , τ : V → L ( H ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V µ ( τ ( v )) (cid:170)(cid:174)(cid:172) Pr ( (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( ∀ y ( G σ ( u ) ( u , y ))) ∧ Φ B ( u , v ) ( Q ) ∧ Φ B ( u , v ) ( ∀ x ( H τ ( v ) ( x , v )))) = (− ) | U | + | V | (cid:213) σ : U → L ( G ) , τ : V → L ( H ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V µ ( τ ( v )) (cid:170)(cid:174)(cid:172) Pr ( (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( ∀ y ( G σ ( u ) ( u , y )) ∧ Q ∧ ∀ x ( H τ ( v ) ( x , v )))) (62) = (− ) | U | + | V | (cid:213) σ : U → L ( G ) , τ : V → L ( H ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V µ ( τ ( v )) (cid:170)(cid:174)(cid:172) (cid:214) u ∈ U , v ∈ V Pr ( Φ B ( u , v ) ( ∀ y ( G σ ( u ) ( u , y )) ∧ Q ∧ ∀ x ( H τ ( v ) ( x , v )))) This completes the proof of Theorem C.20. □ Let G ( U , V , E ) be a bipartite graph, defining a CCP problem. As for Type I queries (Eq. (8)) we construct a TID that is a union of disjointblocks (cid:208) u ∈ U , v ∈ V B ( u , v ) , where for every non-edge ( a , b ) (cid:60) E we define B ( a , b ) to be the trivial block where all tuples have probability 1.However, unlike the formula (8) for type I queries, now we need to cope the products of Möbius functions in Eq. (57). To remove those,we extend the graph by adding, for each node u ∈ U , one outgoing edge ( u , u ′ ) where u ′ is a fresh node with no other incoming edges;similarly, we add edges ( v ′ , v ) , one for each node v ∈ V . More precisely, the new graph is ( U ∪ V ′ , V ∪ U ′ , E ∪ E ′ ) , where V ′ = { v ′ | v ∈ V } , U ′ = { u ′ | u ∈ U } and E ′ = {( u , u ′ ) | u ∈ U } ∪ {( v , v ′ ) | v ∈ V } . Then, we have:Corollary C.21. Given the notations above: Pr ( Q ) = (cid:213) σ : U → L ( G ) , τ : V → L ( H ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V µ ( τ ( v )) (cid:170)(cid:174)(cid:172) · (cid:214) ( u , v )∈ E Pr ( Y σ ( u ) τ ( v ) ( u , v )) · (cid:214) u ∈ U Pr ( Y σ ( u ) , ˆ1 ( u , u ′ )) · (cid:214) v ∈ V Pr ( Y ˆ1 τ ( v ) ( v ′ , v )) where Y α β ( u , v ) def = Φ B ( u , v ) ( G α ( u ) ∧ Q ∧ H α ( v )) Y α ˆ1 ( u , u ′ ) def = Φ B ( u , u ′ ) ( G α ( u ) ∧ Q ) = Φ B ( u , u ′ ) ( G α ( u ) ∧ Q ∧ H ˆ1 ( u ′ )) see Eq. (55) Y ˆ1 β ( v ′ , v ) def = Φ B ( v ′ , v ) ( Q ∧ H α ( v )) = Φ B ( v ′ , v ) ( G ˆ1 ( v ′ ) ∧ Q ∧ H α ( v )) Proof. (Sketch) The proof consists of a straightforward extension of the proof of Theorem C.20. We begin the proof similarly, but inEq.(58) we replace the sets U and V by U ∪ V ′ and V ∪ U ′ respectively. Next, we apply the Möbius inversion formula repeatedly, once toeach u ∈ U as before, but we do not apply it to any v ′ ∈ V ′ . Therefore, there are two changes to the expression REST in (59). First, it willhave the following residual: (cid:219) v ′ ∈ V ′ Φ ∆ ( (cid:220) i ∀ yG i ( v ′ , y )) Since v ′ has a single outgoing edge ( v ′ , v ) , this residual is restricted to the lineage in the block B ( v ′ , v ) , i.e. it is equivalent to: (cid:219) v ∈ V Φ B ( v ′ , v ) ( (cid:220) i ∀ yG i ( v ′ , y )) (63)The second change in (59) is that we need to expand the expressions (cid:211) u ∈ U , v ∈ V Φ B ( u , v ) ( Q ) with (cid:211) u ∈ U Φ B ( u , u ′ ) ( Q ) and (cid:211) v ∈ V Φ B ( v ′ , v ) ( Q ) .The latter absorbs the residual (63), because ( (cid:212) i ∀ yG i ( x , y )) is implied by Q (i.e. it is one of the conjuncts of Q ): ∀ v ∈ V : Φ B ( v ′ , v ) ( (cid:220) i ∀ yG i ( v ′ , y )) ∧ Φ B ( v ′ , v ) ( Q ) ≡ Φ B ( v ′ , v ) ( Q ) Similarly, when we apply Möbius inversion formula to each v ∈ V we do not apply it to any u ′ ∈ U ′ , and are left with a similar residualon the right, which also gets absorbed. Thus, the only change to Eq. (60) is the addition of the boolean formulas (cid:211) u ∈ U Φ B ( u , u ′ ) ( Q ) and (cid:211) v ∈ V Φ B ( v ′ , v ) ( Q ) . Next, we need to modify Eq. (61) from (cid:211) v ∈ V to (cid:211) v ∈( V ∪{ u ′ }) , in effect adding the conjunct (cid:211) u ∈ U Φ B ( u , u ′ ) ( ∀ yG σ ( u ) ( u , y )) ,and similarly for the line below Eq. (61). The Boolean formula under Pr (· · · ) in Eq. (62) becomes: (cid:219) u ∈ U , v ∈ V Φ B ( u , v ) ( ∀ y ( G σ ( u ) ( u , y )) ∧ Q ∧ ∀ x ( H τ ( v ) ( x , v ))) ∧ (cid:219) u ∈ U Φ B ( u , u ′ ) ( ∀ y ( G σ ( u ) ( u , y )) ∧ Q ) ∧ (cid:219) v ∈ V Φ B ( v ′ , v ) (∧ Q ∧ ∀ x ( H τ ( v ) ( x , v ))) Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Since distinct blocks do not share any tuples, the conjuncts above are independent, hence, as before, the probability is their product, Pr ( Q ) = (cid:214) u ∈ U , v ∈ V Pr ( Φ B ( u , v ) ( ∀ y ( G σ ( u ) ( u , y )) ∧ Q ∧ ∀ x ( H τ ( v ) ( x , v )))) · (cid:214) u ∈ U Pr ( Φ B ( u , u ′ ) ( ∀ y ( G σ ( u ) ( u , y )) ∧ Q )) · (cid:214) v ∈ V Pr ( Φ B ( v ′ , v ) (∧ Q ∧ ∀ x ( H τ ( v ) ( x , v )))) Finally, we notice that u ∈ U , v ∈ V in the first product can be replaced by ( u , v ) ∈ E , because all non-edges are trivially true. □ We can now describe the reduction from
CCP to GFOMC ( Q ) . Let Q be any bipartite, unsafe type II query (meaning: type II-II), and let ¯ m , ¯ n be the sizes of its lattice supports L ( G ) and L ( H ) . Since Q is unsafe, we have ¯ m , ¯ n ≥
3. Our goal is to prove
CCP ( ¯ m , ¯ n ) ≤ P GFOMC ( Q ) . Fix aninstance of a CCP ( ¯ m , ¯ n ) problem ( U , V , E ) . We extend the graph as before to ( U ∪ V ′ , V ∪ U ′ , E ∪ E ′ ) . We define the TID ∆ to be a union ofblocks B ( u , v ) for all ( u , v ) ∈ E ∪ E ′ ; as before, B ( a , b ) is trivially true when ( a , b ) is not an edge. The blocks (to be defined in the next section)will be isomorphic, and therefore, the following quantities do not depend on u , v : y α β def = Pr ( Y α , β ( u , v )) u ∈ U , v ∈ V (64) y α ∗ def = Pr ( Y α , ˆ1 ( u , u ′ )) u ∈ Uy ˆ1 β def = Pr ( Y ˆ1 β ( v ′ , v )) v ∈ V By Corollary C.21, we obtain:Pr ( Q ) = (cid:213) σ : U → L ( G ) τ : V → L ( H ) (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U µ ( σ ( u )) (cid:170)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V µ ( τ ( v )) (cid:170)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:173)(cid:171) (cid:214) ( u , v )∈ E y σ ( u ) , τ ( v ) (cid:170)(cid:174)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:171) (cid:214) u ∈ U y σ ( u ) , ˆ1 (cid:170)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:171) (cid:214) v ∈ V y ˆ1 τ ( v ) (cid:170)(cid:174)(cid:172) Recall that L ( G ) = L ( G ) − { ˆ1 } is the strict support of the left lattice, and ¯ m is its size. Consider now the coloring counting problem givenby the the graph ( U , V , E ) . Every pair σ , τ defines a coloring of the graph ( U , V , E ) . Denoting k def = { k α , β , k α , ˆ1 , k ˆ1 , β | α ⊆ [ ¯ m ] , β ⊆ [ ¯ n ]} itssignature, the factor y α β will occur precisely k α β times, i.e. with the exponent k α β where k α β is the number of edges in the bipartite graphwhose edges are colored α and β respectively. The factor y ˆ1 , β will occur k ˆ1 , β times where k ˆ1 , β represents the number of nodes in V colored β . Likewise, k α , ˆ1 represents the number of nodes in U colored α . Therefore:Pr ( Q ) = (cid:213) k k (cid:32)(cid:214) α ( µ ( α )) k α , ˆ1 (cid:33) · (cid:169)(cid:173)(cid:173)(cid:171)(cid:214) β ( µ ( β )) k ˆ1 , β (cid:170)(cid:174)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:173)(cid:171)(cid:214) α , β y k α β α β (cid:170)(cid:174)(cid:174)(cid:172) · (cid:169)(cid:173)(cid:173)(cid:171)(cid:214) β y k ˆ1 , β ˆ1 , β (cid:170)(cid:174)(cid:174)(cid:172) · (cid:32)(cid:214) α y k α , ˆ1 α , ˆ1 (cid:33) (65)where α , β range over the strict supports L ( G ) , L ( H ) (i.e. are (cid:44) ˆ1). Our unknowns are k ; there is one unknown for every coloring signature k . We introduce new variables x k , indexed by the signatures k , as follows: x k def = k · (cid:32)(cid:214) α ( µ ( α )) k α , ˆ1 (cid:33) · (cid:169)(cid:173)(cid:173)(cid:171)(cid:214) β ( µ ( β )) k ˆ1 , β (cid:170)(cid:174)(cid:174)(cid:172) Thus, one call to the oracle for Pr ( Q ) computes the following linear combination of the unknowns x k :Pr ( Q ) = (cid:213) k (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) (cid:214) α ∈ L ( G ) β ∈ L ( H )( α , β ) (cid:44) ( ˆ1 , ˆ1 ) y k α β α β (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) x k Notice that here α , β range over the entire support, i.e. including ˆ1, except of the combination α = ˆ1 , β = ˆ1, because that does not occur inPr ( Q ) , Eq. (65).Let h def = ( ¯ m + )( ¯ n + ) = O ( ) . The equation above has ( M + ) h unknowns x k because for every pair α ∈ L ( G ) and β ∈ L ( H ) there can bebetween 0 and M blocks with the αβ configuration. Accordingly, there are ( M + ) h coefficients y k α β α β . To simplify the notation, lets denotethe pair αβ by a single index i , where i = , . . . , h . The equation becomes:Pr ( Q ) = (cid:213) k ,..., k h ∈{ ,..., M } (cid:169)(cid:173)(cid:171) (cid:214) i = , h y k i i (cid:170)(cid:174)(cid:172) x k k ··· k h (66) onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Let p = { p , . . . , p h } be a set of h natural numbers where p i ≥
1. We will show in the next several sections how to construct a block B ( u , v ) that depends on p , hence we denote it B ( p ) ( u , v ) , where all tuples have probabilities in { , / , } , such that: y i = h (cid:214) j = y ( p j ) i where y ( p j ) i def = a i λ p j + b i λ p j ∀ i ∈ [ , h ] (67)where λ , λ and a i , b i , i ∈ [ , h ] are constants independent of p , . . . , p h satisfying the following. λ (cid:44) ± λ and λ (cid:44) , λ (cid:44) b i (cid:44) ∀ i ∈ [ h ] (69) a i b j (cid:44) a j b i i (cid:44) j (70)By Theorem 3.6, if we set the values of p , . . . , p h independently to 1 , , . . . , ( M + ) , then we obtain a system with ( M + ) h equationswhose matrix is non-singuar, from which we can compute the unknowns k in polynomial time (by Gaussian elimination), and thus sholvethe instance of the CCP ( ¯ m , ¯ n ) problem. In the next section we describe how to construct the block B p ( u , v ) to ensure that the probability (67)is given by an expression of the type (67). We notice that B p ( u , v ) has the same structure for all u , v ; in what follows we only discuss a singleblock B p ( u , v ) . C.5 Designing the Block B ( p ) ( u , v ) Fix p = ( p , . . . , p h ) a vector of h natural numbers ≥
1. We describe here the block B ( p ) ( u , v ) ; its tuples and probabilities are the samefor all choices of u , v , thus our discussion below does not depend on u , v . The block B ( p ) ( u , v ) will consists of a union of h blocks, eachcorresponding to one of the parameters p , . . . , p h : B ( p ) ( u , v ) = (cid:216) j = , h B p j j ( u , v ) Since the blocks B p , . . . , B p h h have disjoint sets of tuples, we have: ∀ α ∈ L ( G ) , β ∈ L ( H ) : Y ( p ) α β ( u , v ) def = Φ B ( p ) ( u , v ) ( G α ( u ) ∧ Q ∧ H β ( v )) = (cid:219) j = , h Φ B pjj ( u , v ) ( G α ( u ) ∧ Q ∧ H β ( v )) def = (cid:219) j = , h Y ( p j ) α β , j Indeed, each clause of the lineage of G α ( u ) lies entirely within one block B p j j , because G α ( u ) = ∀ yG α ( u , y ) has a single variable ∀ y (seeEq. (51)). Consider now the query Q . The only clauses whose lineage may span multiple blocks are those in Q left ≡ ∀ x (cid:16)(cid:212) j = , m ∀ yG j ( x , y ) (cid:17) ,and only those clauses obtained by mapping x to u . But that formula is absorbed by G α ( u ) , in other words: ( ∀ yG α ( u , y )) ∧ ( (cid:220) j = , m ∀ yG j ( u , y )) ≡ ∀ yG α ( u , y ) Therefore, the probability y α β = Pr ( Y α β ( u , v )) is a product of probabilities one for each block: ∀ α ∈ L ( G ) , β ∈ L ( H ) : y ( p ) α β = (cid:214) j = , h Pr ( Φ B ( pj ) j ( u , v ) ( G α ( u ) ∧ Q ∧ H β ( v ))) def = (cid:214) j = , h y ( p j ) α β , j We will design the blocks B ( p j ) j ( u , v ) to be similar, and differ only in their parameter p j . To simplify the notation, we drop the index j : thus,the expressions p j , B ( p j ) j ( u , v ) , y α β , j , Y ( p j ) α β , j , etc become p , B ( p ) ( u , v ) , y α β , Y ( p ) α β . Our goal is to design the block B ( p ) ( u , v ) , where p ≥ α , β (including ˆ1), we have: y ( p ) α β = Pr ( Y α β ) = a α β λ p + b α β λ p (71)where the parameters λ , λ , a α β , a α , ˆ1 , etc, are independent of p and satisfy the conditions (68)-(70).We describe now the block B ( p ) ( u , v ) , and will refer to Fig. 3. Let m be the maximum number of subclauses in any left or right clause;notice that m ≥
2. An elementary block B ( a , b ) is the set of tuples B ( a , b ) def = { S ( a , b ) , S ( a , b ) , . . . } , i.e. there is exactly one tuple S ( a , b ) foreach binary symbol S ∈ R . Definition C.22.
The block B ( p ) ( u , v ) is the disjoint union of the following blocks: • A prefix block B pref ( u , r ) , which, in turn, is the disjoint union of r parallel blocks: B pref ( u , r ) = (cid:216) i = , r ( B ( u , t pref , i ) ∪ B ( r , t pref , i )) where B ( u , t pref , i ) , B ( r , t pref , i ) are elementary blocks. The number r will be chosen later. Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA • A zig-zag part, which is a union of 2 p + B ( r , t ) ∪ B ( r , t ) ∪ B ( r , t ) ∪ . . . ∪ B ( r p , t p − ) ∪ B ( r p , t p )• A suffix block B suff ( t p , v ) , which is the union of r parallel blocks (same r as for the prefix): B suff ( t p , v ) = (cid:216) i = , r ( B ( r suff , i , t p ) ∪ B ( r suff , i , v )) where B ( r suff , i , t p ) , B ( r suff , i , v ) are elementary blocks. • For each constant r i (including r i , suff ) introduced above there are m − B ( r i , e ( ) i ) ∪ . . . ∪ B ( r i , e ( m − ) i ) . • For each constant t i introduced above (including t pref , i ) there are m − B ( f ( ) i , t i ) ∪ . . . ∪ B ( f ( m − ) i , t i ) . • For any other pairs of constants a , b not explicitly mentioned above, there is a trivial elementary block B ( a , b ) where all tuples haveprobability 1.We denote by B ( r , t p ) the zig-zag portion of the block: B ( r , t p ) = B ( r , t ) ∪ (cid:216) i = , p (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) B ( r i , t i − ) ∪ B ( r i , t i ) ∪ (cid:216) j ( B ( r i , e ( j ) i ) ∪ B ( f ( j ) i − , t i − )) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) dead ends (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) (72)Therefore the entire block is: B ( p ) ( u , v ) = B pref ( u , r ) ∪ (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171)(cid:216) j B ( r , e ( j ) ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) dead end (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) ∪ B ( r , t p ) ∪ (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171)(cid:216) j B ( f ( j ) i , t p ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) dead end (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) C.6 Two Properties of Y ( p ) α β In this section we prove two properties of Y ( p ) α β . First, the mapping ( α , β ) (cid:55)→ Y ( p ) α β is invertible (see Lemma C.16). Second, if Q is a forbiddenquery, then Y ( p ) α β is a connected Boolean formula, for every α ∈ L ( G ) , β ∈ L ( H ) (see Lemma C.15). To simplify the discussion, we consider Y ( p ) α β to be the lineage on the zig-zag block B ( p ) ( r , t p ) (72), and will not consider the prefix and the suffix blocks of B ( p ) ( u , v ) ; the proofs of bothproperties extend immediately to the complete block B ( p ) ( u , v ) . Thus, Y ( p ) α β means the the lineage restricted to the zig-zag block B ( p ) ( r , t p ) : Y ( p ) α β def = Φ B ( r , t p ) ( G α ( r ) ∧ Q ∧ H β ( t p )) y ( p ) α β def = Pr ( Y ( p ) α β ) (73)As usual, each tuple in B ( p ) ( r , t p ) becomes a Boolean variable.Lemma C.23. The mapping ( α , β ) (cid:55)→ Y ( p ) α β is invertible. More precisely, if the logical implication ∀ x ∀ yY ( p ) α β ( x , y ) ⇒ ∀ x ∀ yY ( p ) α β ( x , y ) holds,then α ≤ α and β ≤ β , in the lattices ˆ L ( G ) and ˆ L ( H ) respectively. Proof. The proof is identical to that of Lemma C.16 and omitted. □ Lemma C.24.
Let Q be a forbidden query of type II. Then, for all α ∈ L ( G ) , β ∈ L ( H ) , the Boolean formulas Y ( p ) α β are connected, and depend onall Boolean variables in the block B ( p ) ( r , t ) . Proof. The proof extends the proof idea in lemma C.15, and it is slightly simpler because Q is forbidden. Let C , C , . . . , C k be a left-to-right path in Q , of minimal length; we will use it to construct a long path of clauses in Y ( p ) α β that goes through all zig-zag blocks. Westart with one block in the zig-zag fragment, say B ( r i , t i − ) . The middle clauses C , . . . , C k − have only two logical variables x , y , e.g. ∀ x ∀ y ( S ( x , y ) ∨ S ( x , y ) ∨ · · · ) , hence their groundings C ( r i , t i − ) , C ( r i , t i − ) , . . . C p − ( r i , t i − ) are non-redundant clauses (since they werenon-redundant in Q ) and are connected. Similarly, on the next block in the zig-zag, C ( r i , t i ) , C ( r i , t i ) , . . . C k − ( r i , t i ) are non-redundantand connected. It remains to connect C ( r i , t i − ) and C ( r i , t i ) and for that we use the left clause C = ∀ x (cid:212) k = ,ℓ ∀ yS J k ( x , y ) . Consider thefirst two sub-clauses S J , S J in C : since Q is forbidden (see Def. C.10), each has a common symbol with C . We consider the grounding of onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu 𝑟 " 𝑡 " 𝑟 $ 𝑡 $ 𝑟 % 𝑡 % 𝑟 & 𝑡 & 𝑡 &’$ 𝑠 " 𝑠 $ 𝑠 % 𝑠 & 𝜃 " 𝜃 " 𝒖 𝜶 𝒗 𝜷 𝜃 𝑟 " 𝑡 " 𝑟 $ 𝑡 $ 𝑟 % 𝑡 % 𝑟 & 𝑡 & 𝑡 &’$ 𝜃𝑢 𝑣 𝑟 " 𝑡 " 𝑟 $ 𝑡 $ 𝑟 % 𝑡 % 𝑟 & 𝑡 & 𝑡 &’$ 𝑢 𝑣𝜃′𝜃′ 𝜃𝜃 , 𝜃 , … (a)Choose θ ⊆ θ such that:0 < | λ | < λ ∀ α , β : b α β > ( α , β ) (cid:44) ( α , β ) .Choose θ ′ such that: a α β b α β (cid:44) a α β b α β (b)Parallel branches of θ ′ such that:For all ( α , β ) (cid:44) ( α , β ) : a α β b α β (cid:44) a α β b α β Figure 3: Illustration of the Block B ( p ) ( u , v ) and the progression of assignments θ , θ , θ ′ that, together, satisfy conditions (68) , (69) ,and (70) . Fig. (a) shows only the zig-zag part B ( p ) ( r , t p ) , where we choose θ to satisfy conditions (68) and (69) . The assignment θ is independent of α , β and leaves unassigned some Boolean variables at the beginning and that the end ( u α , v β ). Fig. (b) extendsthe block B ( p ) ( r , t p ) with a prefix and a suffix consisting of a single branch, thus B ( p ) ( u , v ) is isomorphic to B ( p + ) ( r , t p + ) . Here θ ′ s extends the assignment θ to the remaining variables in the prefix and suffix in order to ensure one condition a α β b α β (cid:44) a α β b α β . Fig. (c) extends the prefix/suffix with parallel branches in order to satisfy all conditions a α β b α β (cid:44) a α β b α β . C that maps S J to the block B ( r i , t i − ) , maps S J to the block B ( r i , t i ) , and maps each all other subclauses to distinct dead-end branches B ( r i , e ( j ) i ) . This is possible because there are m − m ≥ ℓ , and it is also necessary, in order to ensure that the groundedclause is not redundant, see Example A.3 in Appendix A. Therefore, this grounding is a clause in Y ( p ) α β that is connected to C ( r i , t i − ) via S J ( r i , t i − ) and is also connected to C ( r i , t i ) via S J ( r i , t i ) . By repeating this for all links of the zig-zag chain, we obtain a long sequenceof connected clauses that start with C ( r , t ) , end with C k − ( r p , t p ) . With the same argument we can extend this connected componentalong all dead-end branches, i.e extend it with C ( r i , e ( j ) i ) , . . . , C k − ( r i , e ( j ) i ) for all i , j , and similarly with C k − ( f ( j ) i , t i ) , . . . , C ( f ( j ) i , t i ) . Thus,this giant connected component has a zig-zag, with many spikes. It remains to prove that the clauses at the end of the zig-zag and at theend of the spikes are also connected. Now consider the beginning of the zig-zag, which is a grounding of G α ( r , t ) . When α (cid:44) ˆ1, then,as we argued in the proof of Lemma C.15, no clause of G α ( r , t ) can make C ( r , t ) redundant, no middle clause can make any clause in G α ( r , t ) redundant, and, while some clauses within G α ( r , t ) might have homomorphisms to others, any remaining non-redundant clauseof G α ( r , t ) contains all ubiquitous symbols. The case when α = ˆ1 differs a little from Lemma C.15, and here we consider two cases. Ifsome ubiquitous symbol of Q occurs in a middle clause C , then all ubiquitous symbols of Q occur in some middle clauses by Lemma C.19(4). Since each such middle clauses is non-redundant in C ( r , t ) , and is connected to the path C , C , . . . (otherwise it consists only ofubiquitous symbols, hence there exists a homomorphism C → C ), the entire Boolean formula is connected and has all Boolean variables. Ifno ubiquitous symbol occurs in a middle clause, then every clause in G ˆ1 ( r , t ) is non-redundant, because if there were a homomorphismfrom some middle clause C ( r , t ) , then C must contain only non-ubiquitous symbols, hence all are in C , by the Definition C.10 of forbiddenqueries, which implies a homomorphism C → C , contradiction. □ Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Recall from Section B that the distance of two Boolean variables in a monotone formula is the smallest number of clauses that connectsthem. Fix a left-right path C , C , . . . , C k in Q , of minimal length. Let U be any left-ubiquitous symbol in C ; recall that, by Lemma C.19, U does not occur in C . Similarly, let V be a right-ubiquitous symbol in C k and note that that it does not occur in C k − . We think of theBoolean variables (tuples) in the zig-zag block B ( p ) ( r , t p ) as being partially ordered: U ( r , t ) is the very “first” tuple, and X comes before Y if d ( U ( r , t ) , X ) < d ( U ( r , t ) , Y ) in the Boolean formula Y ( p ) α β . This is only a pre-order, because we may have equal distances, it helpsunderstanding the structure of the block B ( p ) ( r , t p ) . The next lemma shows that this pre-order is independent of the choice of α , β .Lemma C.25. Let X denote the Boolean variable associated to any tuple of the zig-zag block B ( r , t p ) . Then the distance between U ( r , t ) and X is the same in all formulas Y ( p ) α β , i.e. it does not depend on α , β . Similarly, the distance from X to V ( r p , t p ) is the same. Proof. Every clause in Y ( p ) α β that contains U ( r , t ) is connected to the clause C ( r , t ) (and is not connected to C ( r , t ) ). From there,the distance to any variable X is the same, regardless of the choice of α , β . The same argument applies to Y ( p ) α ˆ1 and Y ( p ) ˆ1 β . □ Lemma C.26.
Let X be the Boolean variable associated to a tuple on the main branch of the zig-zag block B ( r , t p ) ; that is, X has the form X = S ( r i , t i ) or X = S ( r i , t i − ) , but not on a dead-end branch like S ( r i , e ( j ) i ) . Assume that d ( U ( r , t ) , X ) ≥ and d ( X , V ( r p , t p )) ≥ . Then X disconnects U , V in Y ( p ) α β , for all α , β (see Definition B.6). Proof. Let S be the relational symbol of the tuple X , and assume that X = S ( r i , t i ) ; the case X = S ( r i , t i − ) is similar. Since Q is final,both Q [ S : = ] and Q [ S : = ] are safe, and this can happen in one of two ways. The first is that the left clause C and right clause C k remainleft/right clauses in Q [ S : = ] (or Q [ S : = ] ), but the path C , . . . , C k − is disconnected, i.e. one of the clauses becomes true or becomesredundant. In that case setting X : = X : = U ( r i , t i ) from V ( r i , t i ) . The second case is when Q [ S : = ] or Q [ S : = ] has no left clause, or no right clause: this happens either because they become true , or redundant, or they degenerateto middle clauses (e.g. setting S : = ∀ y ( U ( x , y ) ∨ S ( x , y )) ∨ ∀ y ( U ( x , y ) ∨ S ( x , y )) ). In that case the connection between C ( r i , t i − ) and C ( r i , t i ) is broken, again disconnecting U ( r , t ) from V ( r p , t p ) . □ C.7 Consistent Assignments
We want to assign the same probabilities to similar tuples in similar blocks, e.g. the grounding of S should have the same probability inall blocks B ( r , t ) , B ( r , t ) , B ( r , t ) , . . . We make this precise below. We denote with lower case s , s ′ , . . . the real variables representingthe probabilities of the ground tuples, e.g. S ( r i , t i ) , S ′ ( r i , t i ) , . . . We group the Boolean variables and their associated real variables intoequivalence classes defined follows: • For each symbol S ∈ R the odd equivalence class is the set of Boolean variables corresponding to its grounding in the odd-numberedzig-zag blocks: S ( r , t ) , S ( r , t ) , S ( r , t ) , . . . , S ( r p , t p ) . • For each symbol S ∈ R the even equivalence class is the set of real variables corresponding to its grounding in the even-numberedzig-zag blocks: S ( r , t ) , S ( r , t ) , S ( r , t ) , . . . , S ( r p , t p − ) . • For each symbol S ∈ R the j ’th left dead-end equivalence class is the set of variables corresponding to the S ( r , e ( j ) ) , . . . , S ( r p , e ( j ) p ) ;similarly for the j ’th right dead-end equivalence class, S ( f ( j ) , t ) , S ( f ( j ) , t ) , . . . , S ( f ( j ) p − , t p ) . • We will write . . . , S i − , S i , S i + , . . . for variables in the same equivalence class derived from some symbol S . We denote similarly thereal variables, . . . , s i − , s i , s i + , . . . This notation is with some abuse, because does not specify whether we mean the odd equivalenceclass, or the even class, or one of the dead end classes. Depending on the type of class, it may contain either p or p + • Finally, we notice that the distance between two consecutive variables in the same odd equivalence class or the same even equivalenceclass is exactly 2 k , i.e. d ( S i − , S i ) = k , ∀ i . The distance between two consecutive variables in a dead-end equivalence class is ≥ k ,but in general depends on the symbol S . Definition C.27.
Let θ be any partial assignment from the Boolean variables in the zig-zag block B ( r , t p ) to { , / , } . We say that θ is consistent if, for every equivalence class E the following holds. If X , Y are two variables in that equivalence class and θ ( X ) is defined, theneither θ ( Y ) is also defined as θ ( X ) = θ ( Y ) , or θ ( Y ) is undefined and Y = S ( r , t ) or Y = S ( r p , t p ) for some symbol S .In other words, θ must act in the same way on the entire equivalence class, with only exception that it may leave some variables in thefirst block B ( r , t ) and the last block B ( r p , t p ) undefined.We define now a partial, consistent assignment θ as follows, see also Fig. 3 (a). Let X be a tuple on a dead-end branch, e.g. X = S ( r i , e ( j ) i ) .Setting X : = X : = U ( r , t ) from V ( r p , t p ) , for example, if X occurs in one grounding of C that connects C ( r i , t i − ) with C ( r i , t i ) , then setting X : = X : = U ( r , t ) to V ( r p , t p ) . If setting X : = X : = U ( r , t ) from V ( r p , t p ) , then we define θ ( X ) : =
0, or 1 respectively, and do this for all tuples the equivalence class of X ; it holdsthat the tuples U ( r , t ) and V ( r p , t p ) remain connected in all Boolean formulas Y ( p ) α β [ θ ] , for all α , β . We proceed similarly with the dead-end onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu tuples on the right, i.e. we set an entire equivalence class to 0 or to 1 if U ( r , t ) and V ( r p , t p ) remain connected. After this process, θ hasthe following property: Definition C.28.
Let U , V be the left- and right-ubiquitous symbols introduced above. We say that the consistent partial assigning θ is final if, forall α , β , the Boolean function Y ( p ) α β [ θ ] is connected, and, for any tuple X in B ( r , t p ) , setting X : = X : = U ( r , t ) from V ( r p , t p ) .We will fix θ from now on. C.8 The Eigenvalues of the Zig-zag Block
We will now give a closed form formula for the probability of Y ( p ) α β , which, recall, we are using temporarily to denote the lineage on the zig-zagblock B ( r , t p ) , see Eq. (73), i.e. without the prefix/suffix blocks. Since α ∈ L ( G ) and β ∈ L ( H ) , the probabilities y ( p ) α β form a ( ¯ m + ) × ( ¯ n + ) matrix. However, we show that this matrix has rank 2, hence it can be described by a 2 × y ( p ) α β is given interms of two eigenvalues: y ( p ) α β = a α β λ p + b α β λ p for λ , λ (cid:44) λ (cid:44) ± λ .Let C , C , . . . , C k be a left-right path in Q , of minimal length, and recall that U , V are two ubiquitous symbols that do not occur in C , C k − . Fix an index j such that 3 ≤ j ≤ k −
5, and fix a symbol S ∈ Symb ( C j ) ∩ Symb ( C j + ) . Since Q is final, both Q [ S : = ] and Q [ S : = ] are safe queries, which implies that S disconnects Q left from Q right in both Q [ S : = ] and Q [ S : = ] . We will call S an articulation symbol .We denote by S , S , . . . , S p the Boolean variables forming the odd equivalence class S ( r , t ) , . . . , S ( r p , t p ) , and denote by s , s , . . . , s p theirassociated real variables. In Fig. 3 (a) the articulation variables are shown with an x .Denote by B ( r , r i ) and B ( t i , t p ) the following subsets of the zig-zag block B ( r , t p ) : B ( r , r i ) def = (cid:216) ℓ = , i (cid:169)(cid:173)(cid:171) B ( r ℓ − , t ℓ − ) ∪ B ( r ℓ , t ℓ − ) ∪ (cid:216) j (cid:16) B ( r ℓ , e ( j ) ℓ ) ∪ B ( f ( j ) ℓ − , t ℓ − ) (cid:17)(cid:170)(cid:174)(cid:172) B ( t i , t p ) def = (cid:216) ℓ = i + , p (cid:169)(cid:173)(cid:171) B ( r ℓ , t ℓ − ) ∪ B ( r ℓ , t ℓ ) ∪ (cid:216) j (cid:16) B ( f ( j ) ℓ − , t ℓ − ) ∪ B ( r ℓ , e ( j ) ℓ ) (cid:17)(cid:170)(cid:174)(cid:172) These are precisely the two connected components of B ( r , t p ) after we remove the single block B ( r i , t i ) .Lemma C.29. The Boolean variable S i disconnects B ( r , r i ) and B ( t i , t p ) (see Def. B.6) in the Boolean function Y ( p ) α β , for any α , β . In other words, we have: Y ( p ) α β [ S i : = ] = A ∧ B Y ( p ) α β [ S i : = ] = A ∧ B (74)where all Boolean variables from B ( r , r i ) are in A , A , and all Boolean variables from B ( t i , t p ) are in B , B . The proof follows immediatelyfrom the fact that S disconnects Q left from Q right in both Q [ S : = ] and S [ S : = ] .Since S ℓ is in A , A for ℓ < i , we can repeat this process and use S ℓ to disconnect A , A , etc. More generally, consider any values v , v , . . . , v p ∈ { , } . Then, assigning these values to the variables of the articulation symbol splits Y ( p ) α β into p + Y ( p ) α β [ S : = v , S : = v , · · · , S p : = v p ] = U ( v ) α ∧ Z ( v v ) ∧ · · · ∧ Z ( v p − v p ) p ∧ V ( v p ) β (75)The only expressions that depend on the choice of α , β are U ( v ) α and V ( v p ) β . Since all Boolean expressions on the RHS in (75) have disjointBoolean variables, their probabilities are independent, hence their arithmetization is a product of multilinear polynomials: y ( p ) α β [ s : = v , s : = v , · · · , s p : = v p ] = u ( v ) α · z ( v v ) · · · z ( v p − v p ) p · v ( v p ) β We express the result in matrix form, where diag ( a , b ) , denotes the diagonal matrix (cid:34) a b (cid:35) . This is an important expression which wedefine formally: Definition C.30.
The polynomial y ( p ) α β is defined as follows: y ( p ) α , β = (cid:104) u ( ) α u ( ) α (cid:105) · diag ( − s , s ) · (cid:34) z ( ) z ( ) z ( ) z ( ) (cid:35) · diag ( − s , s ) · (cid:34) z ( ) z ( ) z ( ) z ( ) (cid:35) · · · z ( ) p z ( ) p z ( ) p z ( ) p · diag ( − s p , s p ) v ( ) β v ( ) β Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Denote by z i the 2 × z i def = (cid:34) z ( ) i z ( ) i z ( ) i z ( ) i (cid:35) We view it as matrix of multilinear polynomials, where the variables represent the (yet unchosen) probabilities of the tuples in all our blocks.Our next task is to define a consistent assignment θ that extends θ from Sec. C.6, such that det ( z i [ θ ]) (cid:44)
0. For that we will use Lemma 1.2in the introduction. However, we have a problem: the matrix z i may contain two variables from the same equivalence class, and theassignment θ from Lemma 1.2 might assign them different values, making θ is inconsistent. This happens whenever the decompositionin Eq. (74) has migrating variables. Since we chose the articulation variable S in Symb ( C j ) ∩ Symb ( C j + ) , the Boolean variable S i = S ( r i , t i ) disconnects U i def = U ( r i , t i ) , V i def = V ( r i , t i ) , i.e. the left-ubiquitous symbol and the right-ubiquitous symbol in block B ( r i , t i ) . The distancefrom U i to S i is j , and the distance from S i to V i is k − j −
1, hence, by Lemma B.10 item 2, S i also separates all the symbols in C , . . . , C j − from C j + , . . . , C k (more precisely: their groundings in the block B ( r i , t i ) ), hence the only possible migrating variables are of the form S ′ i def = S ′ ( r i , t i ) , with S ′ ∈ Symb ( C j − ) ∪ · · · ∪ Symb ( C j + ) . In other words, d ( S ′ i , S i ) ≤
1. Suppose that S ′ i migrates from left to right, i.e. itoccurs in A and in B in Eq. (74). Then: • s ′ i occurs in the first column of z i , because this is part of A . • s ′ i occurs in the second row of z i + , because this is part of B ; then it’s equivalent variable s ′ i − occurs in the second row of z i .If S ′ i migrates from right to left, i.e. from B to A , then: • s ′ i occurs in the first row of z i + , because this is part of B ; hence s ′ i − occurs in the first row of z i . • s ′ i occurs in the second column of z i , because this is part of A .Pictorially, these two cases are illustrated as follows: z i = (cid:34) z ( ) i ( s ′ i ) z ( ) i z ( ) i ( s ′ i − , s ′ i ) z ( ) i ( s ′ i − ) (cid:35) z i = (cid:34) z ( ) i ( s ′ i − ) z ( ) i ( s ′ i − , s ′ i ) z ( ) i z ( ) i ( s ′ i ) (cid:35) (76)We seek a consistent assignment θ , hence we need θ ( s ′ i − ) = θ ( s ′ i ) , and for that we will simply substitute the real variable s ′ i − with thevariable s ′ i . To do that, we need the following technical lemma.Lemma C.31. Let F be a connected Boolean function, and A = ( A , . . . , A k ) , B = ( B , . . . , B k ) be two tuples of k Boolean variables each. Let X be a variable that disconnects A , B , such that d ( A , X ) ≥ and d ( B , X ) ≥ . Let f be the arithmetization of F . Then the polynomial f [ b : = a ] ,where each variable b i is substituted by the variable a i , is irreducible. Notice that, while f is a multilinear polynomial, each variable a i has degree 2 in the polynomial f [ b : = a ] . In general, if f is irreducible,then equating variables does not keep it irreducible. For example, if F = A ∨ B then f = a + b − ab and f [ b : = a ] = a − a factorizes as a ( − a ) . The lemma gives sufficient conditions for which f [ B : = A ] is irreducible.Proof. By Lemma B.10 (2) X disconnects a ball around of A from a ball around B . Let C , D be two variables s.t. d ( A , C ) = d ( B , D ) = X disconnects A C , B D . Decompose the multilinear polynomial according to X : f [ x : = ] = д ( a , c ) · h ( b , d ) f [ x : = ] = д ( a , c ) · h ( b , d ) f [ b : = a , x : = ] = д ( a , c ) · h ( a , d ) f [ b : = a , x : = ] = д ( a , c ) · h ( a , d ) where we indicated that a , c occurs only in д , д , while b , d occurs only in h , h . While д ( a , c ) may further factorize, all variables a , c are inthe same irreducible factor, because their clauses are connected in F [ X : = ] ; similarly for the other three expressions. Assume now that f [ b : = a ] factorizes; since the degree of X in f [ b : = a ] is ≤
1, there exists an irreducible factor k of f [ b : = a ] that does not contain X . The k divides both expressions in the second line above. We consider three cases. Case 1: If k contains the variable c , then it must divide both д ( a , c ) and д ( a , c ) . It follows that k divides f = д · h · ( − x ) + д · h · x .This is a contradiction because f is irreducible. Case 2: If k contains the variable d , the it must divide h ( a , d ) and h ( a , d ) . In particular k contains the variables a , and the degrees of all a -variables is 1, because their degree in h ( a , d ) , h ( a , d ) is 1. Substitute b for a in k , we a conclude that k [ a : = a ] divides both h ( b , d ) and h ( b , d ) . Hence it divides f by the argument in Case 1, and we reached a contradiction. Case 3 If k contains neither c nor d , then by the argument above it cannot contain any of a . Since k divides д ( a , c ) · h ( a , d ) , it alsodivides д ( a , c ) · h ( b , d ) , and it similarly divides д ( a , c ) · h ( b , d ) . This, again, implies that k divides f , which is a contradiction. □ We will now prove the existence of a consistent assignment θ that extends θ such that det ( z i [ θ ]) (cid:44)
0. Recall that θ , introduced inthe previous section, has the property that all Boolean functions Y ( p ) α β [ θ ] are connected, and every variable X disconnects U ( r , t ) from V ( r p , t p ) . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Theorem C.32.
Let V def = (cid:208) i Vars ( z i )∪ { s , s , . . . , s p } , and define θ the following assignment: if θ ( x ) is defined then θ ( x ) def = θ ( x ) , otherwise θ ( x ) def = / . Then, for every i , det ( z i [ θ ]) (cid:44) . Proof. Choose arbitrary α , β , and recall that the polynomial y ( p ) α β [ θ ] is irreducible (Lemma C.24). Fix i = , . . . , p : we will first definesome consistent θ that extends θ such that det ( z i [ θ ]) (cid:44)
0. Denote by a the set of migrating variables s ′ i , and denote by b the set of thepredecessor equivalent variables s ′ i − ; as we have seen, both s ′ i and s ′ i − belong to z i . Let A , B their corresponding Boolean variables. Toapply Lemma C.31, we need to find some variable X that disconnects A , B and that is far from both. Using the left-right path C , C , . . . , C k ,choose any symbol S ′′ ∈ C j for any 1 < j < k −
1, and define X def = S ′′ ( r i , t i − ) . Since all B -variables are in the block B ( r i − , t i − ) and all A -variables are in B ( r i , t i ) , we have d ( B , X ) ≥ d ( A , X ) ≥
2. Therefore, by the lemma, the polynomial y ( p ) α β [ b : = a , θ ] is irreducible.Claim 6. det ( z i [ b : = a , θ ]) (cid:46) . Proof. Assume the contrary, that det ( z i [ b : = a , θ ]) ≡
0. Then, by Theorem B.1, the matrix y i [ b : = a , θ ] has rank 1, more precisely thereexists polynomials д , д , h , h , such that: (cid:34) д д (cid:35) · (cid:104) h h (cid:105) ≡ z i [ b : = a , θ ] Substituting this expression in y ( p ) α β [ b : = a , θ ] we obtain a factorization: y ( p ) α β [ b : = a , θ ] = (cid:104) u ( ) α u ( ) α (cid:105) · · · (cid:34) д д (cid:35)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) factor 1 · (cid:104) h h (cid:105) · · · v ( ) β v ( ) β (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) factor 2This contradicts the fact that y ( p ) α β [ b : = a , θ ] is irreducible. This completes the proof of the claim. □ Claim 7.
There exists an assignment θ i of the variables of z i [ b : = a ] with values in { , / , } s.t. θ i extends θ and det ( z i [ b :: = a , θ i ]) (cid:44) . Proof. We use Lemma 1.2 in the introduction. To apply it we must verify that det ( z i [ b : = a , θ ]) is a polynomial of degree ≤ s ′ i migrates from left to right then the variable s ′ i = s ′ i − has degree 2 in y ( ) i and degree 0 in y ( ) i , hence it has degree 2 in det ( z i ) , and similarly for variables that migrate from right to left. Therefore, by Lemma 1.2,there exists an assignment θ i of the variables in z i [ b : = a , θ ] (hence: an extension of θ ) such that det ( z i [ b : = a , θ i ]) (cid:44) □ From here we derive immediately:Claim 8.
There exists a consistent assignment θ of the variables V that extends θ and det ( z i [ θ ]) (cid:44) for all i = , p . Proof. Take θ = θ ∪ θ ∪ · · · ∪ θ p . While θ i − , θ i are defined on some common variables (the migrating variables) they have the samevalues. Finally, assign θ ( s ) = θ ( s ) = · · · = θ ( s p ) = / z i ). □ Finally, we prove that θ assigns the value 1 / θ .Claim 9. Assume p ≥ , and let θ be any consistent assignment of the variables V that extends θ . Suppose that there exists some variable X (cid:60) Dom ( θ ) , such θ ( X ) = or θ ( X ) = . Then there exists i such that det ( z i [ θ ]) = . Proof. Let X be any variable (cid:60) Dom ( θ ) . Consider the product of matrices in Def. C.30 that defines the polynomial y ( p ) α , β ; then y ( p ) α , β [ θ ] isthe same product, where θ is applied to each matrix, i.e. z i [ θ ] . X is a variable that occurs in either one, or at most two consecutive matrices(when it migrates). Assuming it occurs in z i , z i + , and we split the expression in Def. C.30 into three parts y ( p ) α , β [ X : = , θ ] = a · diag ( − s i − , s i − ) · w · diag ( − s i + , s i + ) · b Where: a def = u · (cid:214) ℓ = , i − diag ( − s ℓ − , s ℓ − ) · z ℓ [ θ ] w def = z i [ X : = , θ ] · diag ( − s i ) · z i + [ X : = , θ ] b def = (cid:214) ℓ = i + , p z ℓ [ θ ] · diag ( − s ℓ , s ℓ ) · v Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
We prove that, if y ( p ) α , β [ X : = , θ ] factorizes, then det ( w ) ≡
0. This implies that either det ( z i [ X : = , θ ]) ≡ ( z i + [ X : = , θ ]) ≡
0. Wewill denote s def = s i − and s ′ def = s i + to reduce clutter, thus: y ( p ) α , β [ X : = , θ ] = a b w ( − s )( − s ′ ) + a b w ( − s ) s ′ + a b w s ( − s ′ ) + a b w ss ′ Since y ( p ) α , β [ X : = , θ ] is reducible, let f be an irreducible factor f that contains the variable U ( r , t ) , where U is a left ubiquitous symbol.That is, f contains a “far left” variable, which only occurs in a and a . In particular, it does not share any variables with b , b , because X separates them in the Boolean formula Y ( p ) α , β [ θ ] . We have: f (cid:12)(cid:12) a b w ( − s )( − s ′ ) + a b w ( − s ) s ′ + a b w s ( − s ′ ) + a b w s ′ Consider now the variables s , s ′ : f may contain neither, or just s , or both s , s ′ (since U ( r , t ) is closer to S i − than to S i + ) in the latter casewe switch the roles of a , b , i.e. start with some irreducible factor that contains some “far right” variable V ( r p , t p ) . Hence we will assumew.l.o.g. that f does not contain s ′ . Then, can set separately s ′ = s ′ = f (cid:12)(cid:12) ( a w ( − s ) + a w s ) b (77) f (cid:12)(cid:12) ( a w ( − s ) + a w s ) b Consider first the case when f does not contain s either. In that case we repeat the argument, and obtain: f (cid:12)(cid:12) a b w f (cid:12)(cid:12) a b w f (cid:12)(cid:12) a b w f (cid:12)(cid:12) a b w It follows that f | a and f | a , which implies f | y ( p ) α , β [ θ ] (without setting X : = f contains s . Notice that we can remove b , b from (77), since f shares no variables with them. We eliminate a by multiply the firstline by w and the second by − w , then eliminate a similarly, and obtain: f (cid:12)(cid:12) a ( − s )( w w − w w ) f (cid:12)(cid:12) a s ( w w − w w ) If ( w w − w w ) (cid:44)
0, then f divides both a ( − s ) and a s , again implying that f | y ( p ) α , β [ θ ] , which is a contradiction. □□ In summary, we have ∀ i : z i [ θ ] def = z = (cid:34) z z z z (cid:35) (78)Lemma C.33. z , z , z , z > . Proof. Consider z : this is the probability of the Boolean expression Z ( ) i [ θ ] in (75), for any choice of i . We claim that Z ( ) i [ θ ] (cid:46) false .By definition, Z ( ) i [ θ ] is the formula obtained by factorizing: Y ( p ) α β [ S i − : = , S i : = , θ ] = A ∧ Z ( ) i [ θ ] ∧ B We claim that this expression is not ≡ false . Indeed, by definition of θ , Y ( p ) α β [ θ ] is a connected monotone Boolean function, and by ourchoice of the variables S i , their distance is d ( S i − , S i ) = k , hence they are neither prime implicants, nor do they occur together in a clause. Itfollows that by setting both to 0 we not make Y ( p ) α β [ S i − : = , S i : = , θ ] ≡ false . Since all Boolean variables in Z ( ) i [ θ ] have been assignedprobability 1 /
2, it follows that its probability is >
0, proving z >
0. The other cases are similar and omitted. □ Denote by λ , λ the eigenvalues of the matrix z . We also assign probabilities 1 / s = s = · · · = s p = / y ( p ) α β [ θ ] = p + (cid:104) u ( ) α u ( ) α (cid:105) · (cid:34) z z z z (cid:35) p · v ( ) β v ( ) β = (cid:16) a α β ( λ / ) p + b α β ( λ / ) p (cid:17) (79)where a α β , b α β are coefficients that are independent of p . We prove now Condition (68):Theorem C.34. The following hold: < | λ | < λ . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Proof. det ( z ) (cid:44) λ , λ (cid:44) λ (cid:44) λ , because both z and z are (cid:44)
0, since they represent probabilities. λ + λ >
0, becausethe trace of the matrix is z + z > □ Next, prove condition (69).Lemma C.35.
Assume λ < λ , then, for all α , β , b α β > . Proof. We first expand the power p of the matrix z : z p = (cid:34) a λ p + a λ p b λ p + b λ p c λ p + c λ p d λ p + d λ p (cid:35) We prove that a , b , c , d >
0. We start by observing that λ , λ are the solutions to: λ − ( z + z ) λ + ( z z − z z ) = z , z , z , z represent proabilities, hence they are >
0. It follows that λ + λ = tr ( z ) = z + z >
0. Since λ λ = det ( z ) = ( z z − z z ) , we cannot have λ = z , because then λ = tr ( z ) − λ = z and z z = λ λ = det ( z ) = z z − z z implying z = z =
0, which is impossible by Lemma C.33. Thus, λ , (cid:44) z , z .Next, since z is the identity matrix, we obtain: a + a = b + b = c + c = d + d = b = − b and c = − c . Similarly, we examine z = z , and deduce a λ + a λ = z , d λ + d λ = z . If a = a = − a = λ = z , contradiction. Thus, a , a , d , d (cid:44)
0. Finally, we consider the expression for z p : z p = (cid:34) a λ p + a λ p b ( λ p − λ p ) c ( λ p − λ p ) d λ p + d λ p (cid:35) We notice that all entries in z p are >
0, because they are probabilities. When p → ∞ then all terms above are dominated by λ p , whichimplies a , b , c , d >
0. Finally, the lemma follows from computing y ( p ) α β using (79), and obtain: y ( p ) α β = (cid:16) ( u ( ) α a v ( ) β + u ( ) α b v ( ) β + u ( ) α c v ( ) β + u ( ) α d v ( ) β ) λ p + ( u ( ) α a v ( ) β + u ( ) α b v ( ) β + u ( ) α c v ( ) β + u ( ) α d v ( ) β ) λ p (cid:17) and the lemma follows by observing that the factor of λ p is > □ Finally, it remains to prove condition-(70): a α β b α β (cid:44) a α β b α β . To enforce this condition we need to design carefully the prefix andsuffix blocks. We do this in the next sections. C.9 One Condition a α β b α β (cid:44) a α β b α β Our end goal is to define the block B ( p ) ( u , v ) , and its probabilities, such as to satisfy all three conditions (68), (69), and (70). In the previoussections we have denoted by Y ( p ) α β and y ( p ) α β the lineage on the zig-zag block only (73), and satisfied the first two conditions; in this and thenext section we extend it with the prefix/suffix, and satisfy the third condition, (70). Recall that the complete block B ( p ) ( u , v ) is: B ( p ) ( u , v ) = B ( u , r ) ∪ (cid:169)(cid:173)(cid:171)(cid:216) j B ( r , e ( j ) ) (cid:170)(cid:174)(cid:172) ∪ B ( r , t p ) ∪ (cid:169)(cid:173)(cid:171)(cid:216) j B ( f ( j ) p , t p ) (cid:170)(cid:174)(cid:172) ∪ B ( t p , v ) (80)We will denote by: Y ( p ) α β ( r , t p ) def = Φ B ( r , t p ) ( G α ( r ) ∧ Q ∧ H β ( t p )) Y ( p ) α β ( u , v ) def = Φ B ( p ) ( u , v ) ( G α ( u ) ∧ Q ∧ H β ( v )) and similarly for their probabilities, y ( p ) α β ( r , t p ) , y ( p ) α β ( u , v ) . We have already defined a partial assignment θ of Boolean variables in B ( p ) ( r , t p ) to probabilities in { , / , } to satisfy Condition (68) (Condition (69) came almost for free). Now we will extend θ to a total assignment, i.e.to all variables in the block B ( p ) ( u , v ) , to also satisfy condition (70).More precisely, let V ⊂ B ( r , t p ) be the set of all Boolean variables that occur in the matrices z , . . . , z p . Add to it the articulation variables, V ′ def = V ∪ { s , s , . . . , s p } . Then θ is the assignment of the variables V ′ given by Theorem C.32. We have: Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA y ( p ) α β ( r , t p )[ θ ] = p (cid:104) u ( ) α ( r , t )[ θ ] u ( ) α ( r , t )[ θ ] (cid:105) · (cid:34) z z z z (cid:35) p · v ( ) β ( r p , t p )[ θ ] v ( ) β ( r p , t p )[ θ ] = a α β ( r , t p )( λ / ) p + b α β ( r , t p )( λ / ) p (81) y ( p ) α β ( u , v )[ θ ] = p (cid:104) u ( ) α ( u , t )[ θ ] u ( ) α ( u , t )[ θ ] (cid:105) · (cid:34) z z z z (cid:35) p · v ( ) β ( r p , v )[ θ ] v ( ) β ( r p , v )[ θ ] = a α β ( t p , v )( λ / ) p + b α β ( t p , v )( λ / ) p (82)We are interested in the latter expression, where we haven’t yet defined the structure and/or probabilities of the two polynomial matrices u α , v β . Notice that, whatever our choice, conditions (68), (69) continue to apply, since we proved them for any polynomials u α , v β . Now, wewill design the prefix/suffix blocks and assign probabilities to the variables in u α , v β to also satisfy condition (70). We will do this in twosteps. The first step, described in this section, will satisfy the condition for one pair ( α , β ) (cid:44) ( α , β ) . For that we need a single branch inthe prefix and a single branch in the suffix, thus B ( p ) ( u , v ) will be like B ( p + ) ( r , t p + ) , as illustrated in Fig. 3 (b). The second step, describedin the next sections, uses multiple parallel branches to satisfy the condition for all pairs, illustrated in Fig. 3 (c).We will start by satisfying a single condition: for a fixed pair ( α , β ) (cid:44) ( α , β ) ,our goal is to satisfy. a α β b α β (cid:44) a α β b α β . We will usea single branch for the prefix/suffix blocks, hence B ( p ) ( u , v ) is isomorphic to B ( p + ) ( r , t p + ) , and, y ( p ) α β is given by (81). It suffices to provehow complete the assignment θ to all variables in B ( p ) ( r , t p ) in order to satisfy a α β b α β (cid:44) a α β b α β . Our construction is independentof p because, on one hand, the condition that we need to satisfy, a α β b α β (cid:44) a α β b α β , is independent of p , on the other hand we canassume w.l.o.g. that the polynomials v β ( r p , t p )[ θ ] and v β ( r p + , t p + )[ θ ] have the same variables, since the blocks B ( r p , t p ) and B ( r p + , t p + ) are isomorphic. In other words, we assume that the variables of the polynomials y ( p ) α β ( r , t p )[ θ ] are the same for all choices of p (since θ already assigns values to all variables in the zig-zag portion of B ( p ) ( r , t p ) , which depends on p ).When there are no migrating variables, then the polynomials u ( ) α , u ( ) α , v ( ) β , v ( ) β in (81) do not contain any variables with V ′ , thus we candrop the argument θ ; then we are free to assign their probabilities as we need. However, if a symbol S ′ migrates from right to left, thenvariables from its equivalence class occur in both u α and v β . The reason is that θ assigns the same value to all variables in an equivalenceclass, hence all the tuples S ′ ( r , t ) , S ′ ( r , t ) , . . . , S ′ ( r p , t p ) are associated with the same real variable s ′ . Assuming S ′ migrates from right toleft, then S ′ ( r p , t p ) appears in v ( ) β (on the “right”) and S ′ ( r , t ) appears in u ( ) α (on the “left”). Similarly, u ( ) α and v ( ) β may share commonvariables. The notation u ( ) α [ θ ] , . . . , v ( ) β [ θ ] indicates that we apply θ to all migrating variables s ′ , and recall that θ ( s ′ ) = /
2. Recall that θ assigns probabilities 1 / θ ′ we must do it consistently with this assignment.We start by restating the condition a α β b α β (cid:44) a α β b α β .Lemma C.36. Let z be a × matrix with eigenvalues (cid:44) λ (cid:44) λ (cid:44) , and let u i , v i , i = , be four vectors. Define the following twosequences, y ( p ) i , p ≥ , i = , : y ( p ) def = u · z p · v def = a ( λ ) p + b ( λ ) p y ( p ) def = u · z p · v def = a ( λ ) p + b ( λ ) p For any p ≥ , consider the following matrix: D ( p ) def = (cid:34) y ( p ) y ( p ) y ( p + ) y ( p + ) (cid:35) Then the following statements are equivalent: • a b (cid:44) a b , • there exists p ≥ such that det ( D ( p ) ) (cid:44) , • for all p ≥ , det ( D ( p ) ) (cid:44) . Proof. We compute det ( D ) directly:det (cid:34) a λ p + b λ p a λ p + b λ p a λ p + + b λ p + a λ p + + b λ p + (cid:35) = λ p λ p ( λ − λ )( a b − a b ) and using the fact that 0 (cid:44) λ (cid:44) λ (cid:44) □ Therefore, in order to find an assignment θ ′ that satisfies a α β b α β (cid:44) a α β b α β , we need to construct two blocks, B ( p ) ( r , t p ) and B ( p + ) ( r , t p + ) : θ already assigns probabilities to the variables in their zig-zag part, hence we only need to extend it with θ ′ that assigns onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu probabilities to the remaining variables (which we assumed are the same in B ( p ) ( r , t p ) and B ( p + ) ( r , t p + ) ) such that det ( D ( p ) α β α β )[ θ ′ ] (cid:44) D ( p ) α β α β def = y ( p ) α β [ θ ] y ( p ) α , β [ θ ] y ( p + ) α β [ θ ] y ( p + ) α , β [ θ ] (83)Furthermore, by lemma C.36 we can check the condition for any p , so we choose conveniently to check it for p =
0, in other words usethe blocks B ( ) ( r , t ) and B ( ) ( r , t ) . To find θ ′ such that det ( D ( p ) α β α β )[ θ ′ ] (cid:44)
0, we proceed as usual: we first prove that the polynomialdet ( D ( ) α β α β ) (cid:46)
0, then use this to argue that there exists an assignment θ ′ of its variables such that det ( D ( ) α β α β [ θ ′ ]) (cid:44) If ( α , β ) (cid:44) ( α , β ) , then det ( D ( ) α β α β ) (cid:46) . Proof. For arbitrary α , β , denote the following matrices of polynomials (appearing in Eq. (81)): u α = (cid:34) u ( ) α u ( ) α (cid:35) v β = v ( ) β v ( ) β Notice that these are polynomials that include the migrating variables. By Eq. (81), we have: y ( ) α β = u Tα v β y ( ) α β = u Tα zv β For the values α , β , α , β given in the lemma, define the following matrix: D def = y ( ) α β y ( ) α , β y ( ) α β y ( ) α , β = (cid:34) u Tα v β u Tα v β u Tα zv β u Tα zv β (cid:35) Then D [ θ ] = D ( ) α β α β , i.e. D is obtained by exposing the migrating variables, which in D ( ) α β α β are assigned by θ (all are assigned thevalue 1 / ( D [ θ ]) (cid:46) T the non-singular matrix that diagonalizes the matrix z , that is z = T − ΛT , where Λ = diag ( λ , λ ) , and define: x Tα def = u α T y β def = T − v β x Tα def = u α T y β def = T − v β The matrix T is a matrix of numbers, while x α , . . . , y β are vectors of multilinear polynomials. We denote the components of the vector x α by x ( ) α and x ( ) α , similar to the components of the vector u α . Notice that Vars ( x ( ) α ) ⊆ Vars ( u ( ) α ) ∪ Vars ( u ( ) α ) and Vars ( x ( ) α ) ⊆ Vars ( u ( ) α ) ∪ Vars ( u ( ) α ) . These variables include the migrating variables. Similarly for the other three vectors. Therefore, we obtain:det ( D ) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) u Tα v β u Tα v β u Tα zv β u Tα zv β (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) x Tα y β x Tα y β x Tα Λy β x Tα Λy β (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) x ( ) α y ( ) β + x ( ) α y ( ) β x ( ) α y ( ) β + x ( ) α y ( ) β x ( ) α λ y ( ) β + x ( ) α λ y ( ) β x ( ) α λ y ( ) β + x ( ) α λ y ( ) β (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:16) x ( ) α y ( ) β x ( ) α y ( ) β − x ( ) α y ( ) β x ( ) α y ( ) β (cid:17) · ( λ − λ ) def = f · ( λ − λ ) Since λ (cid:44) λ , in order to prove det ( D [ θ ]) (cid:46)
0, we need to show that f [ θ ] (cid:46)
0. Assuming the contrary, we have the following identity ofpolynomials: x ( ) α [ θ ] x ( ) α [ θ ] (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) Vars ( u α [ θ ] , u α [ θ ]) y ( ) β [ θ ] y ( ) β [ θ ] (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) Vars ( v β [ θ ] , v β [ θ ]) ≡ x ( ) α [ θ ] x ( ) α [ θ ] (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) Vars ( u α [ θ ] , u α [ θ ]) y ( ) β [ θ ] y ( ) β [ θ ] (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) Vars ( v β [ θ ] , v β [ θ ]) (84)We have indicate above the set of variables that occur in these multilinear polynomials. At this point we observe that Vars ( u α i [ θ ]) and Vars ( v β j [ θ ]) are disjoint sets of variables, for any i , j = ,
2. Indeed, the only variables shared by u α i and v β j are the migrating variables, butthese have been replaced by the constant 1 / θ . Therefore, assuming the identity (84) holds, then both the following identities hold too: x ( ) α [ θ ] x ( ) α [ θ ] ≡ x ( ) α [ θ ] x ( ) α [ θ ] y ( ) β [ θ ] y ( ) β [ θ ] ≡ y ( ) β [ θ ] y ( ) β [ θ ] (85) Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Now we return to the definition of the vectors x α i , y β j in terms of u α i , v β j and the non-singular matrix T . Assuming the entries in T are a , b , c , d , where ad − bc (cid:44)
0, we have: x ( ) α i = au ( ) α i + cu ( ) α i x ( ) α i = bu ( ) α i + du ( ) α i and we obtain: ( x ( ) α x ( ) α − x ( ) α x ( ) α ) = (cid:16) ( au ( ) α + cu ( ) α )( bu ( ) α + du ( ) α ) − ( bu ( ) α + du ( ) α )( au ( ) α + cu ( ) α ) (cid:17) = ( ad − bc )( u ( ) α u ( ) α − u ( ) α u ( ) α ) We apply a similar change of base from y β i to v β i , and the identities (85) become: u ( ) α [ θ ] u ( ) α [ θ ] ≡ u ( ) α [ θ ] u ( ) α [ θ ] v ( ) β [ θ ] v ( ) β [ θ ] ≡ v ( ) β [ θ ] v ( ) β [ θ ] (86)We have shown that, if det ( D [ θ ]) ≡
0, then both identities (86) hold. We prove that this is a contradiction. For that we show that there existsa total assignment θ ′ of all variables in the polynomials u α i , v β j that (1) extends θ , and (2) make at least one of the quantities in (86) (cid:44) .Notice that when α = α then the first identity does hold, but in that case β (cid:44) β and then we show that the second identity implies acontradiction. When both α = α and β = β then both identities (86) hold, but we have assumed that ( α , β ) (cid:44) ( α , β ) .To prove our claim, we remove the assignment θ and start from the polynomials u α i , v β j . We claim that the following non-indentitieshold: α (cid:44) α ⇒ u ( ) α u ( ) α (cid:46) u ( ) α u ( ) α β (cid:44) β ⇒ v ( ) β v ( ) β (cid:46) v ( ) β v ( ) β (87)Indeed, suppose the first identity holds. Then by Theorem B.1 we can factorize the polynomials as follows: (cid:34) u ( ) α u ( ) α u ( ) α u ( ) α (cid:35) ≡ (cid:34) f · h f · kд · h д · k (cid:35) If f is not a constant polynomial, then f divides both u ( ) α and u ( ) α , and therefore it divides y ( ) α β = u ( ) α v ( ) β ( − s ) + u ( ) α v ( ) β s (where s isthe articulation variable), which contradicts the fact that y ( ) α β is irreducible. This proves that f must be a constant. Similarly, д must be aconstant, by the same argument. It follows that u ( ) α ≡ cu ( ) α for some constant c . Since both u ( ) α and u ( ) α are arithmetizations of monotoneBoolean functions, when all variables are set to 1 then both quantities are =
1, hence the constant is c =
1. Thus, u ( ) α ≡ u ( ) α , and by the sameargument , u ( ) α ≡ u ( ) α which implies that the two polynomials are identical, u α ≡ u α , which implies α = α by Lemma C.23.At this point we need to treat separately the cases when α (cid:44) α and β (cid:44) β and when one of them is equal. Assume first that both aredifferent, hence both Eq. (87) hold. In that case the following product of polynomials is not identically zero: f def = (cid:16) u ( ) α u ( ) α − u ( ) α u ( ) α (cid:17) · (cid:16) v ( ) β v ( ) β − v ( ) β v ( ) β (cid:17) (cid:46) f has degree ≤
2. Indeed, the only variables common in both factors are the migrating variables. Let s ′ be amigrating variable, and assume it migrates from right to left. Then it occurs only in v ( ) β , v ( ) β , u ( ) α , u ( ) α , and therefore it has degree 1 in eachof the factors, hence it total degree in f is 2. Therefore, by Lemma 1.1, there exists an assignment θ ′ with values in { , / , } such that f [ θ ′ ] (cid:44)
0. In other words, we have proven that both the following inequalities hold: u ( ) α [ θ ′ ] u ( ) α [ θ ′ ] (cid:44) u ( ) α [ θ ′ ] u ( ) α [ θ ′ ] v ( ) β [ θ ′ ] v ( ) β [ θ ′ ] (cid:44) v ( ) β [ θ ′ ] v ( ) β [ θ ′ ] (89)It remains to prove that θ ′ assigns 1 / s ′ .Let s ′ be a variable migrating from right to left. We claim that either the following two identities hold: u ( ) α [ s ′ : = ] u ( ) α [ s ′ : = ] = u ( ) α [ s ′ : = ] u ( ) α [ s ′ : = ] (90) v ( ) β [ s ′ : = ] v ( ) β [ s ′ : = ] = v ( ) β [ s ′ : = ] v ( ) β [ s ′ : = ] (91)or the following two identities hold (obtained by switching s ′ : = s ′ : = u ( ) α [ s ′ : = ] u ( ) α [ s ′ : = ] = u ( ) α [ s ′ : = ] u ( ) α [ s ′ : = ] (92) v ( ) β [ s ′ : = ] v ( ) β [ s ′ : = ] = v ( ) β [ s ′ : = ] v ( ) β [ s ′ : = ] (93)The claim completes the proof, because, assuming the first two equalities hold, (90) and (91), then, if θ ′ ( s ′ ) = θ ′ ( s ′ ) =
1, then the right inequality in (89) becomes an equality, which is a contradiction because we havechosen θ ′ such that both (89) hold. Similarly for the case when (92) and (93) hold. In either case, θ ′ ( s ′ ) cannot be either 0 or 1, hence it mustbe 1 / s ′ was an arbitrary migrating variable, we have that θ ′ is an extension of θ . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Thus, it remains to prove the claim. Recall that, for any α , β , y ( ) α β is final , meaning that for every symbol s ′ , setting it to 0 or to 1 decomposesthe polynomial. Thus, for any α , β : y ( ) α β [ s ′ : = ] = a α · b β (94)where a α and b β are polynomials that depend only on α and β respectively. By assumption, s , causes s ′ to migrate, hence, by Corollary B.16, s ′ will cause s to migrate. Assume that s migrates from right to left: that is s occurs in b β , and when we decompose y ( ) α β [ s ′ : = ] then itoccurs on the left. Then, we apply (94) to y ( ) α β and y ( ) α β where α , α are the values given by the lemma, and β is arbitrary, and obtain: y ( ) α β [ s ′ : = , s : = ] = u ( ) α [ s ′ : = ] (cid:122) (cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32) (cid:123) a α · b · v ( ) β [ s ′ : = ] (cid:122)(cid:125)(cid:124)(cid:123) c β y ( ) α β [ s ′ : = , s : = ] = u ( ) α [ s ′ : = ] (cid:122) (cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32) (cid:123) a α · b · v ( ) β [ s ′ : = ] (cid:122)(cid:125)(cid:124)(cid:123) c β y ( ) α β [ s ′ : = , s : = ] = a α · b (cid:124) (cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32) (cid:125) u ( ) α [ s ′ : = ] · c β (cid:124)(cid:123)(cid:122)(cid:125) v ( ) β [ s ′ : = ] y ( ) α β [ s ′ : = , s : = ] = a α · b (cid:124) (cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32) (cid:125) u ( ) α [ s ′ : = ] · c β (cid:124)(cid:123)(cid:122)(cid:125) v ( ) β [ s ′ : = ] and both sides of (90) become equal to a α a α b b , thus we have proven the identity (90). Applying the same reasoning to the decomposition y ( ) α β [ s ′ : = ] (where s occurs on the left) we deduce the identity (91). Thus, when s migrates from right to left, then both identities (90)and (91) hold. Similarly, when it migrates from right to left then (90) and (91) hold, proving the claim.Next, assume that α (cid:44) α , β = β . In that case only the first condition in (87) holds, thus we have: f def = u ( ) α u ( ) α − u ( ) α u ( ) α (cid:46) f [ θ ] (cid:46)
0, where θ ( s ′ ) = / f has degree 1,because it occurs either only in u ( ) α and u ( ) α or only in u ( ) α and u ( ) α . We prove that either f [ s ′ : = ] ≡ f [ s ′ : = ] ≡
0. Using the sameargument as before, we derive that either (90) holds or (92) holds; equations (91) and (93) hold vacuously because β = β . In the first case,when (90) holds, then f [ s ′ : = ] ≡
0; in the second case f [ s ′ : = ] ≡
0. Thus, f is divisible by either s ′ or by 1 − s ′ . It follows that f is aproduct of the form s ′ ( − s ′′ ) s ′′′ · · · i.e. there is one factor for each migrating variable s ′ , and that factor is either s ′ or 1 − s ′ . It follows that,if θ assigns values 1 / f [ θ ] (cid:46)
0, completing the proof. □ Corollary C.38. If ( α , β ) (cid:44) ( α , β ) then there exists an assignment θ ′ to all variables in D ( ) α β α β such that det ( D ( ) α β α β [ θ ′ ]) (cid:44) .Notice that θ ′ depends on the choices of α , β , α , β . Proof. This is an immediate consequence of the previous Lemma C.37 and of Lemma 1.1, because det ( D ( ) α β α β ) is a degree-2 multivariatepolynomial that is (cid:46) □ This completes our goal for this subsection. For any fixed ( α , β ) (cid:44) ( α , β ) , we can construct a block B ( p ) ( u , v ) isomorphic to B ( p + ) ( r , t p + ) and define an assignment θ ′ of its variables such that the polynomials y ( p ) α β ( u , v ) satisfy condition-(70) for the given pair.Importantly, while θ ′ depends on α , β , α , β , its restriction to the zig-zag block B ( p ) ( r , t p ) agrees with θ , and is thus independent on α , β , α , β , see Fig. 3 (b). C.10 All Conditions a α β b α β (cid:44) a α β b α β In the second part, we show that, if we modify the prefix/suffix blocks by constructing many parallel branches, in order to satisfy condition-(70)for all pairs ( α , β ) (cid:44) ( α , β ) .Consider a block B ( p ) ( u , v ) defined by Eq. (80). Let V pref , V suff be the set of variables (tuples) in the prefix block B ( u , r ) and suffix block B ( t , v ) respectively. These sets will depend on how many branches we choose for these blocks. Let V zigzag be the remaining variables, in thezig-zag block B ( p ) ( r , t p ) and the two remaining sets of dead-end branches at r and t p respectively. We will fix the following assignment θ on V zigzag . Consider a prefix with a single branch, and a suffix with a single branch, thus B ( p ) ( u , v ) is isomorphic to B ( p + ) ( r , t p + ) , then applyCorollary C.38. This gives us an assignment θ ′ to all variables in B ( p ) ( u , v ) such that det ( D ( p ) α β α β )[ θ ′ ] (cid:44)
0, where D ( p ) α β α β is definedby Eq. (83) w.r.t. the entire block B ( p ) ( u , v ) . While θ ′ depends on the choices of α β α β , its restriction to B ( p ) ( r , t p ) is independent of α β α β . Let θ be that restriction. See Fig. 3 (c) for an illustration. There is no need for partial assignment θ in Sec. C.7 because when p = then θ is empty; in fact, when p = then y ( ) α β is isomorphic to Q α β , and it is final because Q is final. Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
With the assignment to V zigzag fixed, we recompute the probabilities y ( p ) α β , by separating the quantities that depend on the prefix/suffixfrom the rest. For that we use Möbius inversion formula applied to the points r and t p , which separate the two sets of blocks, and obtain: y ( p ) α β ( u , v ) = (cid:213) γ ∈ L ( G ) , δ ∈ L ( H ) p αγ c ( p ) γ δ q δ β (95)where: p αγ def = Pr ( Y αγ ( u , r )) c ( p ) γ δ def = µ ( γ ) µ ( δ ) y ( p ) γ δ ( r , t p ) (cid:214) j y ( ) γ ˆ1 ( r , e ( j ) ) (cid:214) j y ( ) ˆ1 δ ( f ( j ) , t ) q δ β def = Pr ( Y δ β ( t p , v )) (96)We think of these quantities as follows. The values c γ δ are constants, since they are defined by the assignment θ to V zigzag , and do notdepend on the pair ( α , β ) (cid:44) ( α , β ) . The quantities p αγ are defined by the prefix block B ( u , r ) . The value p αγ is defined exactly like y α β ,the only difference is that it goes from left-to-left, thus α , γ come from the same lattice L ( G ) , with the only restriction that γ (cid:44) ˆ1. Similarly,the values q δ β are defined by the suffix block.We compute det ( D α β α β ) in terms of these new variables p αγ and q δ β , and denote it by f α β α β : f α β α β = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) y ( ) α β y ( ) α , β y ( ) α β y ( ) α , β (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (cid:205) γ ∈ L ( G ) , δ ∈ L ( H ) p α γ c ( ) γ δ q δ β (cid:205) γ ∈ L ( G ) , δ ∈ L ( H ) p α γ c ( ) γ δ q δ β (cid:205) γ ∈ L ( G ) , δ ∈ L ( H ) p α γ c ( ) γ δ q δ β (cid:205) γ ∈ L ( G ) , δ ∈ L ( H ) p α γ c ( ) γ δ q δ β (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:213) γ , δ ,..., γ , δ (cid:16) c ( ) γ δ c ( ) γ δ p α γ q δ β p α γ q δ β − c ( ) γ δ c ( ) γ δ p α γ q δ β p α γ q δ β (cid:17) = (cid:213) γ , δ ,..., γ , δ (cid:16) c ( ) γ δ c ( ) γ δ p α γ q δ β p α γ q δ β (cid:17) − (cid:213) γ , δ ,..., γ , δ (cid:16) c ( ) γ δ c ( ) γ δ p α γ q δ β p α γ q δ β (cid:17) = ¯ m ¯ n (cid:213) γ , δ , γ , δ (cid:16) c ( ) γ δ c ( ) γ δ p α γ q δ β p α γ q δ β (cid:17) − ¯ m ¯ n (cid:213) γ , δ , γ , δ (cid:16) c ( ) γ δ c ( ) γ δ p α γ q δ β p α γ q δ β (cid:17) = ¯ m ¯ n (cid:213) γ , δ , γ , δ (cid:16) c ( ) γ δ c ( ) γ δ − c ( ) γ δ c ( ) γ δ (cid:17) p α γ p α γ q δ β q δ β def = ¯ m ¯ n (cid:213) γ , δ , γ , δ (cid:16) Γ γ γ δ δ (cid:17) p α γ p α γ q δ β q δ β (97)We make the following observations about this polynomial. • The quantities p αγ and q δ β are defined in (96), and thus depend on the assignment θ on the variables V pref ∪ V suff . • On the other hand, we can view the quantities p αγ and q δ β as independent variables, and thus, f α β α β in Eq. (97) is a multivariatepolynomial in these variables. • The polynomial f α β α β is not identically 0. This follows from Corollary C.38 since we can construct prefix/suffix blocks consistingof a single branch and extend θ to a total assignment θ ′ such that f α β α β = det ( D α β α β )[ θ ′ ] (cid:44) • The coefficients of the polynomial f α β α β are Γ γ γ δ δ . The total number of variables is 2 ¯ m + n , where ¯ m = | L ( G )| , ¯ n = | L ( H )| . • When α = α = α , β (cid:44) β , then the polynomial has only ¯ m + n variables, and it simplifies to:¯ m ¯ n (cid:213) γ , δ , δ (cid:16) c ( ) γ δ c ( ) γ δ − c ( ) γ δ c ( ) γ δ (cid:17) p αγ q δ β q δ β The reason is that in Eq. (97), the monomials p αγ p αγ and p αγ p αγ are of course the same, but the coefficients given by γ , γ and γ , γ cancel out, except when γ = γ . This polynomial is also not identically zero, det ( D α β α β )[ θ ′ ] (cid:44)
0, by the same argument. • Similarly, when α (cid:44) α , β = β = β the polynomial has only 2 ¯ m + ¯ n variables, simplifies similarly, and is not ≡ • If both α = α and β = β , then there are only ¯ m + ¯ n variables, but the polynomial is ≡
0. We are not interested in this case. • Suppose we have built two separate blocks B ( p ) ( u , v ) and B ( p ) ( u , v ) , with the same p , but different prefixes/suffixes, i.e. differentnumber of parallel branches, and different probability assignments. The values of the quantities (96) are p ( ) αγ and q ( ) δ β in the first block,and p ( ) αγ and q ( ) δ β in the second block; the coefficients c ( p ) γ δ are the same, since we use the same assignment θ for the variables V zigzag in blocks. Construct a new block B ( p ) ( u , v ) whose prefix consists of the union of all parallel branches in B ( p ) ( u , v ) and B ( p ) ( u , v ) , andsimilarly its suffix consists of the union of all parallel branches of the suffices of the two blocks; see Fig. 3 (c). Then the quantities (96)of the new block are given by the products, i.e. p ( ) αγ p ( ) αγ and q ( ) δ β q ( ) δ β respectively. This follows immediately from the fact that, in theMöbius expansion, the formulas for the parallel branches in the suffix/postfix become independent; this is in fact a special case ofTheorem C.20. onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Next, we state a theorem, of possible independent interest, which we prove in the next section.Theorem C.39.
Let f , . . . , f m be multivariate polynomials in the variables x = ( x , . . . , x n ) . Suppose that, for each i = , m , there exists aset of values v i = ( v i , . . . , v in ) ∈ R n , v ij > , such that f i ( v i ) (cid:44) . Then there exists natural numbers k , k , . . . , k n ≥ such that, setting v = ( v , v , . . . , v n ) where v j def = v k i v k i · · · v k n in we have f ( v ) (cid:44) , . . . , f m ( v ) (cid:44) . If v , v are two vectors, then we write v v for their element-wise product. Thus, the theorem states that, if f i ( v i ) (cid:44)
0, then there existsexponents such that, defining v def = (cid:206) i v k i i , then for all i = , . . . , m , f i ( v ) (cid:44)
0. We prove the theorem in the next section.We use this theorem as follows. We need to satisfy several conditions a α β b α β (cid:44) a α β b α β . Let m be the number of such conditions, wewill refer to them using an index i = , , . . . , m . For each condition i we know that we can satisfy that condition, using a single prefix/suffixbranch and some assignment θ i of the variables V pref , V suff ; this follows from Corollary C.38. The assignment θ i results in numerical values p αγ [ θ i ] , q δ β [ θ i ] , and, in particular, on these values, f α β α β (cid:44)
0. Next, construct parallel branches of the prefix/suffix. In the new block,the quantities p αγ are products (cid:206) i ( p αγ [ θ i ]) k i where k i is the number of copies of the branch with assignment θ i , and similarly for q δ β . Toapply the Theorem C.39 we need to prove that p αγ [ θ i ] > α , γ , θ i . Corollary C.38 seems insufficient for that. Instead, we will usethe corollary only to prove that the polynomials f α β α β are not identically 0, then prove that we can satisfy each condition i with anassignment that guarantees p αγ [ θ i ] > q δγ [ θ i ] > α , β , γ , δ .Lemma C.40. Consider two distinct monomials p α γ p α γ q δ β q δ β and p α γ p α γ q δ β q δ β occurring in the polynomial f α α β β (97) .Then there exists an assignment θ to the variables in V pref ∪ V suff such that:(a) The following inequality holds p α γ [ θ ] p α γ [ θ ] q δ β [ θ ] q δ β [ θ ] (cid:44) p α γ [ θ ] p α γ [ θ ] q δ β [ θ ] q δ β [ θ ] (b) For every α , γ , p αγ [ θ ] > and for every δ , β , q δ β [ θ ] > . Proof. We view the two monomials as multilinear polynomials over the variables V pref ∪ V suff . Consider any polynomials p αγ , q δ β ,defined by Eq. (96). The Boolean formula Y αγ is similar to Y α β in Eq. (56), the only difference is that we are now moving from left to right toleft. In particular, p αγ is an irreducible polynomial. If U is a left-ubiquitous symbol in Q , then we write U ′ , U ′′ for the Boolean variables U ′ = U ( u , t pref ) and U ′′ = U ( r , t pref ) . We say that U ′ is a ubiquitous variable at the “start” and U ′′ is at the “end”, since their distance in Y αγ is 2 k . We write Y αγ as: Y αγ = G ′ α ∧ Y ∧ G ′′ γ where G ′ α consists of clauses that contain some ubiquitous variable U ′ at the start, G ′′ γ consists of clauses with some ubiquitous variable U ′′ at the end, and Y contains all clauses without ubiquitous symbols. Notice that Y is the same formula for all choices of α and γ .We will use the following simple fact. If F is a monotone Boolean function in CNF with n variables, and f is its arithmetization, then forany θ ∈ [ , ] n , if f [ θ ] = X ∨ X ∨ · · · in F such that θ ( X ) = θ ( X ) = . . . =
0. In this case, with some abuse, wewrite F [ θ ] =
0, although F [ θ ] is not a well defined value since θ assigns real values to Boolean variables.To prove the lemma, we will prove that there exists an assignment θ that satisfies item (a) of the lemma, and also satisfies the followingproperties: (1) Y [ θ ] (cid:44) U , θ ( U ) (cid:44)
0. This implies that, for any α , γ , p αγ [ θ ] (cid:44)
0, because theclauses Y will be the same, while each clause in G ′ α or G ′′ γ has at least one ubiquitous symbol, which is not assigned the value 0. We enforcesimilar properties on Y δ β , and therefore these imply condition (b) of the lemma.We start by proving that the two monomials, when viewed as multilinear polynomials over the variables V pref ∪ V suff are not identicalpolynomials. Suppose otherwise, then: p α γ p α γ q δ β q δ β ≡ p α γ p α γ q δ β q δ β (98)Since the polynomials p αγ depend only on variables from the prefix while q δ β depend only on variables from the suffix, we obtain that bothtwo identities below must hold: p α γ p α γ ≡ p α γ p α γ q δ β q δ β ≡ q δ β q δ β (99)We claim that if the first identity holds, then, when viewed as monomial terms in f α β α β , the monomials p α γ p α γ and p α γ p α γ arethe same. If the second identity holds as well, then the other two monomials q δ β q δ β and q δ β q δ β are also identical, but both cannothold by the assumption of the lemma. To prove the claim, assume that the first identity above holds. Since each of the four polynomialsis irreducible, there are two cases. The first is when p α γ ≡ p α γ and p α γ ≡ p α γ . In that case, using the argument in Lemma C.16, weconclude that γ = γ and γ = γ , meaning that monomials p α γ p α γ and p α γ p α γ are the same. The second case is p α γ ≡ p α γ and p α γ ≡ p α γ , in which case α = α and γ = γ and γ = γ , and, again, the two monomials are identical.Therefore, at least one of the identities in (99) does not hold. Assume w.l.o.g. that it is the first identity. We will construct below a certainassignment θ on V pref such that: p α γ [ θ ] p α γ [ θ ] (cid:44) p α γ [ θ ] p α γ [ θ ] (100)and satisfies conditions (1) and (2) above. Then we will extend it by setting θ ( X ) = X in V suff ; this assignment satisfies bothconditions (a) and (b) of the lemma. We distinguish two cases: Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
Case 1: ( γ , γ ) = ( γ , γ ) . By Lemma 1.1 in the introduction there exists that θ ∈ { , / , } that satisfies: p α γ [ θ ] p α γ [ θ ] (cid:44) p α γ [ θ ] p α γ [ θ ] (101)By Corollary B.3, we may assume w.l.o.g. that neither side =
0, which implies that Y [ θ ] (cid:44)
0. We prove that for any ubiquitous symbol U ′ at the start, θ ( U ′ ) ∈ { / , } . Since the query is final, Q [ U ′ : = ] is a safe query. Since all left clauses remain intact in Q [ U ′ : = ] (none becomes redundant and no subclause becomes redundant), it follows that in Q left [ U : = ] all left clauses are disconnected fromthe right clauses; referring to a left-to-right path C , C , . . . , C k of minimal length, the clause C becomes redundant in Q [ U : = ] . Itfollows that, for any α , γ , Y αγ [ U ′ : = ] = G ′ α [ U ′ : = ] ∧ Y ∧ G ′′ γ Vars ( G ′ α [ U ′ : = ]) ∩ Vars ( Y ∧ G ′′ γ ) = ∅ where Y are clauses of Y that are not redundant in Y αγ [ U ′ : = ] . Therefore, the polynomials p αγ factorize as p αγ [ U ′ : − ] = f α · д γ ,and the following equivalence holds p α γ [ U ′ : − ] p α γ [ U ′ : − ] ≡ p α γ [ U ′ : − ] p α γ [ U ′ : − ] . Thus, if θ ( U ′ ) = U ′′ is a ubiquitous symbol at the end, then we prove similarly that θ ( U ′′ ) (cid:44)
0. This completes theproof of Case 1.
Case 2: ( γ , γ ) (cid:44) ( γ , γ ) ; by our earlier discussion we also have ( γ , γ ) (cid:44) ( γ , γ ) . Here we will define θ ( U ′ ) = F γ def = Y α γ [ U ′ : = ] = Y ∧ G ′′ γ F γ def = Y α γ [ U ′ : = ] = Y ∧ G ′′ γ F γ def = Y α γ [ U ′ : = ] = Y ∧ G ′′ γ F γ def = Y α γ [ U ′ : = ] = Y ∧ G ′′ γ Since all these Boolean formulas are connected, their arithmetizations are irreducible polynomials. It follows that f γ f γ (cid:46) f γ f γ ,since otherwise we have ( γ , γ ) = ( γ , γ ) or ( γ , γ ) = ( γ , γ ) . By Lemma 1.1 there exists an assignment θ in { , / , } such that f γ [ θ ] f γ [ θ ] (cid:46) f γ [ θ ] f γ [ θ ] . We check that these four functions, together with ubiquitous symbols at the end, U ′′ , . . . , U ′′ m , satisfy theconditions in Lemma B.5. Indeed, by setting any U ′′ i : =
1, all the subclausess in G ′′ γ that depend on the choice of γ become 1, sincethey contain all ubiquitous symbols, and therefore G ′′ γ [ U ′′ i : = ] ≡ G ′′ γ [ U ′′ i : = ] ≡ G ′′ γ [ U ′′ i : = ] ≡ G ′′ γ [ U ′′ i : = ] . Therefore, we canapply Lemma B.5 and obtain an assignment θ such that f γ [ θ ] f γ [ θ ] (cid:46) f γ [ θ ] f γ [ θ ] and θ ( U ′′ ) ∈ { / , } for all U ′′ . This concludesthe proof. □ We now proceed to re-prove Corollary C.38.Lemma C.41.
Fix any ( α , β ) (cid:44) ( α , β ) , and consider a prefix/suffix block with a single branch. Then there exists an assignment θ of thevariables in V pref ∪ V suff such that (1) f α , β , α , β [ θ ] (cid:44) and (2) for any α , γ , p αγ [ θ ] > and for any β , δ , q δ β [ θ ] > . Proof. Denote by m , m , . . . , m n the monomials occurring in all polynomials f α , β , α , β . That, each m i has the form m i = p α γ p α γ q δ β q δ β for some choice of γ , γ , δ , δ , and we can write f α , β , α , β = (cid:205) i Γ i m i , where Γ i is the coefficient of the i ’th monomial. Recall that thepolynomial is not identically 0 (this follows from Corollary C.38). For each i (cid:44) j , let θ ij be the assignment given by Lemma C.40 for this pairof monomials. Denote by p ij be the vector consisting of all values p αγ [ θ ij ] and q δ β [ θ ij ] : many do not occur in the polynomial f α , β , α , β ,but we include all of them in these vectors. Notice that all components of all these vectors are >
0. By Theorem C.39, we can constructprefix/suffix blocks consisting of multiple parallel copies of these blocks, such that (1) the new vectors p are element-wise products of thevectors p ij , and similarly for q ; in particular all their components are >
0, and (2) all pairs of monomials in f α , β , α , β are distinct: m i (cid:44) m j .Call this block the starting block; notice that it has several parallel branches in both the prefix and the suffix (the same number of branchesin the prefix and suffix). Thus, we have: f α , β , α , β ( p , q ) = (cid:213) i = , n Γ i m i Next, we make k parallel copies of the starting block; on this even larger block, the vector p is replaced by p k , i.e. each component is raisedto the power k , and similarly for q . Thus, in the new block, the polynomial is: f α , β , α , β ( p k , q k ) = (cid:213) i = , n Γ i m ki If this value is = k = , , . . . , n +
1, then Γ = · · · = Γ n =
0, because the values m i are distinct and thus the matrix of the system oflinear equation is non-singular (it is a Vandermonde matrix). But that implies that the polynomial is identically 0, which is a contradiction.Thus, there exists k such that the value of this polynomial is (cid:44)
0. Since all components of p k , q k are >
0, this proves the claim. □ Finally, we pove:Corollary C.42.
There exists a choice of the prefix/suffix blocks such that, for every pair ( α , β ) (cid:44) ( α , β ) , the polynomial (97) is (cid:44) . onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu Proof. By the previous lemma we can construct a prefix/suffix block that satisfies one polynomial (cid:44)
0. By Theorem C.39, we can constructparallel branches of these prefix/suffix blocks to satisfy all polynomials (cid:44)
0, as required. □ The corollary completes the proof: with this choice of prefix/suffix blocks we have a α β b α β (cid:44) a α β b α β for all ( α , β ) (cid:44) ( α , β ) , andthis we proved condition-(70).It remains to prove Theorem C.39. C.11 Proof of Theorem C.39
Thus, in the rest of this section it remains to prove Theorem C.39. Here we will refer to the polynomials f , f , . . . as p , p , . . . We write a multivariate polynomial as p ( x , . . . , x n ) = p ( x ) = (cid:213) e : e ≤ d a e x e (102)Here e = ( e , . . . , e n ) ∈ N n denotes a vector of exponents, x e def = (cid:206) i x e i i , and e ≤ d means e i ≤ d for all i ; in other words, each variable x i has degree ≤ d . We assume that the coefficients a e are real numbers.We will consider vectors of values v = ( v , . . . , v n ) ∈ R n + , where v i > i = , n . We denote by v k def = ( v k , . . . , v kn ) , and denoteby vw def = ( v w , . . . , v n w n ) , where w = ( w , . . . , w n ) . Then, Theorem C.39 says that: if p ( v ) (cid:44) . . . , p m ( v m ) (cid:44)
0, then there exists u def = v k · · · v k m m s.t. p ( u ) (cid:44) . . . , p m ( u ) (cid:44) p ( x ) be a multivariate polynomial in n variables, of degree d , and let v ∈ R n + bea vector of non-zero values. To compute p ( v k ) we will group the terms of (102) as follows. Let: U def = { v e | e ≤ d } ⊆ R + That is, U is the set of all distinct values v e that will occur in the expansion of p ( v ) . Assume U has m distinct values, U = { u , . . . , u m } . Forall u i ∈ U , define: E v , i def = { e | e ≤ d , v e = u i } Thus, E v , ∪ E v , ∪ . . . ∪ E v , m forms a partition of the set of all exponents occurring in p ( x ) , and we can write it as a sum of m polynomials: p ( x ) = (cid:213) i = , m ( (cid:213) e ∈ E v , i a e x e ) def = (cid:213) i = , m p v , i ( x ) Then, for all i , p v , i ( v k ) = u ki p v , i ( ) , because all terms ( v k ) e in p v , i ( v k ) are equal to u ki . Therefore, p ( v k ) = (cid:213) i = , m p v , i ( v k ) = (cid:213) i = , m u ki p v , i ( ) (103)Notice that p v , i ( ) = p v , i ( , , . . . , ) are just the sum of all coefficients of the polynomial p v , i ( x ) .Let’s call a polynomial p ( x ) balanced if p ( ) =
0; otherwise it is imbalanced . We prove:Lemma C.43.
Let v ∈ R n + be such that p ( v ) (cid:44) . Then there exists i such that p v , i ( x ) is imbalanced. Proof. We prove the converse: if each p v , i ( x ) is balanced, then p ( v ) =
0. This follows immediately from Eq. (103). □ Lemma C.44.
If at least one of the polynomials p v , i ( x ) is imbalanced, then there exists k ≥ such that forall k ≥ k , p ( v k ) (cid:44) . In otherwords, p ( v k ) (cid:44) , for all k that are “large enough”. Proof. Assume w.l.o.g. that m > p v , i ( x ) are imbalanced (otherwise we simply remove the balanced polynomialsand corresponding values u i ). Also assume u > u > · · · > u m . Then Eq. (103) becomes: p ( v k ) = (cid:213) i = , m u ki p v , i ( ) = u k (cid:169)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:173)(cid:171) p v , ( ) + (cid:213) i = , m (cid:18) u i u (cid:19) k p v , i ( ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) → (cid:170)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:174)(cid:172) When k → ∞ , then the expression under (cid:205) i = , m goes to 0, hence when k is large enough, p v , ( ) + (cid:205) i = , n (· · · ) (cid:44) □ Dichotomy for the Generalized Model Counting Problem for Unions of Conjunctive Queries Conference’17, July 2017, Washington, DC, USA
For a simple example, consider f ( x , x , x ) = x x − x , and assume v = ( , , ) . Then f ( , , ) = · − =
0. However, the readermay verify that, for every k ≥ f ( k , k , k ) > v s w t for natural numbers s , t ≥
1. We will require some simple inequality constraints ofthese pairs s , t , which we define next. Definition C.45. An inequality constraint is a pair of real numbers ( α , β ) , such that ( α , β ) (cid:44) ( , ) (i.e. not both α , β can be 0). We say thattwo natural numbers s , t ∈ N satisfy the constraint if s , t ≥ sα + tβ (cid:44)
0. If Γ is a set of inequality constraints, then we write Γ | = ( s , t ) when s , t satisfy every constraint in Γ .The intuition behind an inequality constraint is the following. We have two numbers v , w , and want to find exponents s , t such that v s (cid:44) w t . Any pair ( s , t ) satisfying the constraint ( log v , − log w ) will also satisfy the inequality v s (cid:44) w t . We need two very simple facts:Lemma C.46. If Γ is a finite set of inequality constraints, then there exists infinitely many pairs of natural numbers s , t s.t. s ≥ , t ≥ thatsatisfy all constraints in Γ . Proof. Let Γ = {( α , β ) , . . . , ( α m , β m )} , and define the set S def = {− α i / β i | ( α i , β i ) ∈ Γ , β i (cid:44) } . This is a finite set of real numbers. Then,any pair of natural numbers s , t such that s , t ≥ t / s ∈ Q − S satisfies all constraints in Γ : indeed, if β i (cid:44) t / s (cid:44) − α i / β i implies sα i + tβ i (cid:44)
0, and if β i = sα i + tβ i = sα i (cid:44) s > □ Lemma C.47.
Let α > α > · · · > α m and β > β > · · · > β n be two sequences of distinct real values. Then there exists a finite set ofinequality constraints Γ such that, for any numbers s , t , if Γ | = ( s , t ) , then the m · n values qα i + tβ j , i = , m , j = , n , are distinct. Proof. For all tuples i , i , j , j such that 1 ≤ i ≤ i ≤ m and 1 ≤ j ≤ j ≤ n and ( i , j ) (cid:44) ( i , j ) , define γ i i j j = α i − α i and δ i i j j = β j − β j . Let Γ be the set of constraints ( γ i i j j , δ i i j j ) . If Γ | = ( s , t ) , then sγ i i j j + tδ i i j j (cid:44) i , i , j , j , whichimplies sα i + tβ j (cid:44) sα i + tβ j . □ Let p ( x , y ) = (cid:205) e , f a e , f x e y f be a polynomial in two sets of variables, and let v , w be two sequences of positive real numbers. We define: U def = { v e | e ≤ d } = { u > u > · · · > u m ( > )} Z def = { w e | e ≤ d } = { z > z > · · · > z s ( > )} E v , w , i , j def = {( e , f ) | e ≤ d , f ≤ d , v e = u i , w f = z j } As before, for any two sequences of positive real numbers v , w , the sets E v , w , i , j partition the set of exponents occurring in p , and we canwrite: p ( x , y ) = (cid:213) i , j (cid:169)(cid:173)(cid:173)(cid:171) (cid:213) ( e , f )∈ E v , w , i , j a e , f x e y f (cid:170)(cid:174)(cid:174)(cid:172) def = (cid:213) i = , m ; j = , s p v , w , i , j ( x , y ) Lemma C.48.
Fix p ( x , y ) , and let v ∈ R n + be a vector of values > , such that p ( v , ) (cid:44) . Let w ∈ R n + be any other vector of values > .Then there exists a finite set of inequality constraints Γ , such that, forall s , t , if Γ | = ( s , t ) then there exists k ≥ such that forall k ≥ k , p ( u kq , v kt ) (cid:44) . Proof. Let U , Z , E v , w , i , j , p v , w , i , j ( x , y ) be defined as above. We start by noticing that, for any numbers a , b ≥
0, we have p v , w , i , j ( v a , w b ) = u ai z bj p v , w , i , j ( , ) . For any three numbers q , t , k ≥
0, we have: p ( v kq , w kt ) = (cid:213) i = , m ; j = , s p v , w , i , j ( v kq , w kt ) = (cid:213) i = , m ; j = , s u kqi z ktj p v , w , i , j ( , ) = (cid:213) i = , m ; j = , s r kij p v , w , i , j ( , ) where r ij def = u qi z tj . We notice that there exists i , j such that p v , w , i , j ( , ) (cid:44)
0. Indeed, if we choose k = q = , t =
0, then the quantity abovebecomes p ( v kq , w kt ) = p ( v , ) , which, by assumption of the lemma is (cid:44)
0, proving that at least one quantity p v , w , i , j ( , ) (cid:44) onference’17, July 2017, Washington, DC, USA Batya Kenig and Dan Suciu We will define a set of constraints Γ such that Γ | = ( q , t ) implies that all values r ij = u qi z tj are distinct or, equivalently, the quantities q log u i + t log z j are distinct. To obtain such a Γ , we apply Lemma C.47 to the sequences log u i and log z j respectively. Considering onlythose values r ij for which p v , w , i , j ( , ) (cid:44)
0, let r i j be the largest number. Then we have: p ( v kq , w kt ) = (cid:213) i = , m ; j = , s r kij p v , w , i , j ( , ) = r i j (cid:169)(cid:173)(cid:173)(cid:171) p v , w , i , j ( , ) + (cid:213) ( i , j ) (cid:44) ( i , j ) (cid:32) r ij r i j (cid:33) k p v , w , i , j ( , ) (cid:170)(cid:174)(cid:174)(cid:172) Since lim k →∞ (cid:16) r ij r i j (cid:17) k = k large enough, p ( v kq , w kt ) (cid:44)
0, as required. □ Finally, we can prove Theorem C.39.Proof. (Of Theorem C.39) We proceed by induction on m . When m = m ≥
2, and denote p ( x ) def = p ( x ) · p ( x ) · · · p m − ( x ) . By induction hypothesis, there exists v = v k · · · v k m − m − such that p ( v ) (cid:44)
0, and there exists w such that p m ( w ) (cid:44) f ( x , y ) and д ( x , y ) : f ( x , . . . , x n , y , . . . , y n ) def = p ( x y , . . . , x n y n ) д ( x , . . . , x n , y , . . . , y n ) def = p m ( x y , . . . , x n y n ) We apply Lemma C.48 to the polynomial f and the sequences v , w : the assumption f ( v , ) (cid:44) f ( v , ) = p ( v ) (cid:44)
0. Therefore,we obtain a finite set of constraints Γ s.t. for all s , t , if Γ | = ( s , t ) , then f ( v ks , w kt ) (cid:44) k “large enough”.Similarly, we apply Lemma C.48 to the polynomial д and the same sequences v , w . The condition in the lemma holds, because д ( , w ) = p m ( w ) (cid:44)
0. Therefore, there exists a finite set of constraints Γ s.t. for all s , t , if Γ | = ( s , t ) then д ( v ks , w kt ) (cid:44) k “large enough”.Let ( s , t ) be any pair that satisfies both Γ and Γ : such a pair exists by Lemma C.46, because Γ ∪ Γ is a finite set. Therefore, if k is largeenough, then, denoting u def = v ks w kt , we notice that this has the required form of the theorem, i.e. u = (cid:206) i v k i i for appropriate exponents k i , and we prove that it satisfies the conditions of the theorem. Indeed, on one hand p ( u ) = p ( v ks w kt ) = f ( v ks , w kt ) (cid:44) p i ( u ) (cid:44) i = , m −
1, and on the other hand p m ( u ) = p m ( v ks w kt ) = д ( v ks , w kt ) (cid:44)
0, proving the theorem.0, proving the theorem.