HyperBench: A Benchmark and Tool for Hypergraphs and Empirical Findings
Wolfgang Fischl, Georg Gottlob, Davide M. Longo, Reinhard Pichler
HHyperBench: A Benchmark and Tool for Hypergraphsand Empirical Findings
Wolfgang Fischl , Georg Gottlob , , Davide M. Longo , , and Reinhard Pichler TU Wien, Università della Calabria, and University of Oxford
ABSTRACT
To cope with the intractability of answering Conjunctive Queries(CQs) and solving Constraint Satisfaction Problems (CSPs), severalnotions of hypergraph decompositions have been proposed – givingrise to different notions of width, noticeably, plain, generalized, andfractional hypertree width (hw, ghw, and fhw). Given the increasinginterest in using such decomposition methods in practice, a publiclyaccessible repository of decomposition software, as well as a largeset of benchmarks, and a web-accessible workbench for inserting,analysing, and retrieving hypergraphs are called for.We address this need by providing (i) concrete implementationsof hypergraph decompositions (including new practical algorithms),(ii) a new, comprehensive benchmark of hypergraphs stemmingfrom disparate CQ and CSP collections, and (iii) HyperBench, ournew web-interface for accessing the benchmark and the results ofour analyses. In addition, we describe a number of actual experi-ments we carried out with this new infrastructure.
In this work we study computational problems on hypergraph de-compositions which are designed to speed up the evaluation ofConjunctive Queries (CQs) and the solution of Constraint Satisfac-tion Problems (CSPs). Hypergraph decompositions have meanwhilefound their way into commercial database systems such as Log-icBlox [6, 10, 34, 35, 42] and advanced research prototypes such asEmptyHeaded [1, 3, 43, 50]. Hypergraph decompositions have alsobeen successfully used in the CSP area [5, 31, 36]. In theory, thepros and cons of various notions of decompositions and widths arewell understood (see [25] for a survey). However, from a practicalpoint of view, many questions have remained open.We want to analyse the hypertree width ( hw ) of hypergraphsfrom different application contexts. The investigation of millionsof CQs [14, 44] posed at various SPARQL endpoints suggests thatthese real-world CQs with atoms of arity ≤ hw : theoverwhelming majority is acyclic; almost all of the rest has hw = hw ≤
5. Ghionna et al. [23] gave apositive answer to this question for a small set of TPC-H benchmarkqueries. We significantly extend their collection of CQs.Answering CQs and solving CSPs are fundamental tasks in Com-puter Science. Formally, they are the same problem, since bothcorrespond to the evaluation of first-order formulae over a finitestructure, such that the formulae only use { ∃ , ∧} as connectives butnot { ∀ , ∨ , ¬} . Both problems, answering CQs and solving CSPs, are NP -complete [16]. Consequently, the search for tractable fragmentsof these problems has been an active research area in the databaseand artificial intelligence communities for several decades.The most powerful methods known to date for defining tractablefragments are based on various decompositions of the hypergraph structure underlying a given CQ or CSP. The most importantforms of decompositions are hypertree decompositions (HDs) [26], generalized hypertree decompositions (GHDs) [26], and fractionalhypertree decompositions (FHDs) [29]. These decomposition methodsgive rise to three notions of width of a hypergraph H : the hypertreewidth hw ( H ) , generalized hypertree width ghw ( H ) , and fractionalhypertree width fhw ( H ) , where, fhw ( H ) ≤ ghw ( H ) ≤ hw ( H ) holdsfor every hypergraph H . For definitions, see Section 2.Both, answering CQs and solving CSPs, become tractable if theunderlying hypergraphs have bounded hw , ghw , or, fhw and anappropriate decomposition is given. This gives rise to the problemof recognizing if a given CQ or CSP has hw , ghw , or, fhw boundedby some constant k . Formally, for decomposition ∈ { HD, GHD, FHD } and k ≥
1, we consider the following family of problems:Check( decomposition , k ) input hypergraph H = ( V , E ) ; output decomposition of H of width ≤ k if it exists andanswer ‘no’ otherwise.Clearly, bounded fhw defines the largest tractable class whilebounded hw defines the smallest one. On the other hand, the prob-lem Check(HD, k ) is feasible in polynomial time [26] while theproblems Check(GHD, k ) [27] and Check(FHD, k ) [20] are NP -complete even for k = k ) problem exist [28, 46]. Incontrast, for the problems Check(GHD, k ) and Check(FHD, k ),apart from exhaustive search over possible decomposition trees(which only works for small hypergraphs), no implementationshave been reported yet [1] – with one exception: very recently, aninteresting approach is presented in [19], where SMT-solving isapplied to the Check(FHD, k ) problem. In [28], tests of theCheck(HD, k ) system are presented. However, a benchmark forsystematically evaluating systems for the Check( decomposition , k )problem with decomposition ∈ { HD, GHD, FHD } and k ≥ Goal 1:
Create a comprehensive, easily extensible benchmarkof hypergraphs corresponding to CQs or CSPs for the analysisof hypergraph decomposition algorithms.
Goal 2:
Use the benchmark from Goal 1 to find out if the hy-pertree width is, in general, small enough (say ≤
5) to allow forefficient evaluation of CQs of arbitrary arity and of CSPs.Recently, in [20], the authors have identified classes of CQs forwhich the Check(GHD, k ) and Check(FHD, k ) problems becometractable (from now on, we only speak about CQs; of course, allresults apply equally to CSPs). To this end, the Bounded IntersectionProperty (BIP) and, more generally, the Bounded Multi-IntersectionProperty (BMIP) have been introduced. The maximum number i of attributes shared by two (resp. c ) atoms is referred to as theintersection width (resp. c -multi-intersection width) of the CQ,which is similar to the notion of cutset width from the CSP literature a r X i v : . [ c s . D B ] N ov c of) query atoms is bounded by some constant i .A related property is that of bounded degree, i.e., each attributeonly occurs in a constant number of query atoms. Clearly, theBMIP is an immediate consequence of bounded degree. It has beenshown in [20] that Check(GHD, k ) is solvable in polynomial timefor CQs whose underlying hypergraphs satisfy the BMIP. For CQs,the BMIP and bounded degree seem natural restrictions. For CSPs,the situation is not so clear. This yields the following research goals. Goal 3:
Use the hypergraph benchmark from Goal 1 to analysehow realistic the restrictions to low (multi-)intersection width,or low degree of CQs and CSPs are.
Goal 4:
Verify that the tractable fragment of the Check(GHD, k )problem given by hypergraphs of low intersection width indeedallows for efficient algorithms that work well in practice.The tractability results for Check(FHD, k ) [20, 21] are signifi-cantly weaker than for Check(GHD, k ): they involve a factor whichis at least double-exponential in some “constant” (namely k , thebound d on the degree and/or the bound i on the intersection-width).Hence, we want to investigate if (generalized) hypertree decom-positions could be “fractionally improved” by taking the integraledge cover at each node in the HD or GHD and replacing it by afractional edge cover. We will thus introduce the notion of fraction-ally improved HD which checks if there exists an HD of width ≤ k ,such that replacing each integral cover by a fractional cover yieldsan FHD of width ≤ k ′ for given bounds k , k ′ with 0 < k ′ < k . Goal 5:
Explore the potential of fractionally improved HDs, i.e.,investigate if the improvements achieved are significant.In cases where Check(GHD, k ) and Check(FHD, k ) are intractable,we may have to settle for good approximations of ghw and fhw .For GHDs, we may thus use the inequality ghw ( H ) ≤ · hw ( H ) + H [4]. In contrast, for FHDs, thebest known general, polynomial-time approximation is cubic. Moreprecisely, in [40], a polynomial-time algorithm is presented which,given a hypergraph H with fhw ( H ) = k , computes an FHD of width O( k ) . In [20], it is shown that a polynomial-time approximationup to a logarithmic factor is possible for any class of hypergraphswith bounded Vapnik–Chervonenkis dimension (VC-dimension;see Section 2 for a precise definition). The problem of efficientlyapproximating the ghw and/or fhw leads us to the following goals. Goal 6:
Use the benchmark from Goal 1 to analyse if, in practice, hw and ghw indeed differ by factor 3 or, if hw is typically muchcloser to ghw than this worst-case bound. Goal 7:
Use the benchmark from Goal 1 to analyse how realisticthe restriction to small VC-dimension of CQs and CSPs is.
Results.
By pursuing these goals, we obtain the following results: • We provide
HyperBench , a comprehensive hypergraph bench-mark of initially over 3,000 hypergraphs (see Section 3). This bench-mark is exposed by a web interface, which allows the user to retrievethe hypergraphs or groups of hypergraphs together with a broadspectrum of properties of these hypergraphs, such as lower/upperbounds on hw and ghw , (multi-)intersection width, degree, etc. • We extend the software for HD computation from [28] to alsosolve the Check(GHD, k ) problem. For a given hypergraph H , oursystem first computes the intersection width of H and then applies the ghw -algorithm from [20], which is parameterized by the inter-section width. We implement several improvements and we furtherextend the system to compute also “fractionally improved” HDs. • We carry out an empirical analysis of the hypergraphs in theHyperBench benchmark. This analysis demonstrates, especiallyfor real-world instances, that the restrictions to BIP, BMIP, boun-ded degree, and bounded VC-dimension are astonishingly realistic.Moreover, on all hypergraphs in the HyperBench benchmark, werun our hw - and ghw -systems to identify (or at least bound) their hw and ghw . An interesting observation of our empirical studyis that apart from the CQs also a significant portion of CSPs inourbenchmark has small hypertree width (all non-random CQshave hw ≤ hw ≤ hw ≤
5, in all of the cases where the ghw -computation terminates, hw and ghw have identical values. • In our study of the ghw of the hypergraphs in the HyperBenchbenchmark, we observed that a straightforward implementation ofthe algorithm from [20] for hypergraphs of low intersection widthis too slow in many cases. We therefore present a new approach(based on so-called “balanced separators”) with promising experi-mental results. It is interesting to note that the new approach worksparticularly well in those situations which are particularly hardfor the straightforward implementation, namely hypergraphs H where the test if ghw ≤ k for given k gives a “no”-answer. Hence,combining the different approaches is very effective. Structure.
This paper is structured as follows: In Section 2, werecall some basic definitions and results. In Section 3, we present oursystem and test environment as well as our hypergraph benchmarkHyperBench. First results of our empirical study of the hypergraphsin this benchmark are presented in Section 4. In Section 5, wedescribe our algorithms for solving the Check(GHD, k ) problem.A further extension of the system to allow for the computationof fractionally improved HDs is described in Section 6. Finally, inSection 7 we summarize related work and conclude in Section 8 byhighlighting the most important lessons learned from our empiricalstudy and by identifying some appealing directions for future work.Due to lack of space, some of the statistics presented in the mainbody contain aggregated values (for instance, for different classesof CSPs). Figures and tables with more fine-grained results (forinstance, distinguishing the 3 classes of CSPs to be presented inSection 4) are provided in the appendix and will be made publicallyavailable in a full version of this paper in CoRR. Let ϕ be a CQ or CSP (i.e., an FO-formula with connectives { ∃ , ∧} ).The hypergraph corresponding to ϕ is defined as H = ( V ( H ) , E ( H )) ,where the set of vertices V ( H ) is defined as the set of variables in ϕ and the set of edges E ( H ) is defined as E ( H ) = { e | ϕ contains anatom A , s.t. e equals the set of variables occurring in A } . Hypergraph decompositions and width measures.
We con-sider here three notions of hypergraph decompositions with asso-ciated notions of width. To this end, we first need to introduce thenotion of (fractional) edge covers:Let H = ( V ( H ) , E ( H )) be a hypergraph and consider a function γ : E ( H ) → [ , ] . Then, we define the set B ( γ ) of all vertices covered y γ and the weight of γ as B ( γ ) = v ∈ V ( H ) | (cid:213) e ∈ E ( H ) , v ∈ e γ ( e ) ≥ , weight ( γ ) = (cid:213) e ∈ E ( H ) γ ( e ) . The special case of a function with values restricted to { , } , willbe denoted by λ , i.e., λ : E ( H ) → { , } . Following [26], we can alsotreat λ as a set with λ ⊆ E ( H ) (namely, the set of edges e with λ ( e ) =
1) and the weight as the cardinality of such a set of edges.We now introduce three notions of hypergraph decompositions.
Definition 2.1. A generalized hypertree decomposition (GHD) of ahypergraph H = ( V ( H ) , E ( H )) is a tuple (cid:10) T , ( B u ) u ∈ N ( T ) , ( λ u ) u ∈ N ( T ) (cid:11) , such that T = ⟨ N ( T ) , E ( T )⟩ is a rooted tree and thefollowing conditions hold: (1 ) ∀ e ∈ E ( H ) : there exists a node u ∈ N ( T ) with e ⊆ B u ; (2 ) ∀ v ∈ V ( H ) : the set { u ∈ N ( T ) | v ∈ B u } is connected in T ; (3 ) ∀ u ∈ N ( T ) : λ u is defined as λ u : E ( H ) → { , } with B u ⊆ B ( λ u ) .We use the following notational conventions throughout thispaper. To avoid confusion, we will consequently refer to the ele-ments in V ( H ) as vertices of the hypergraph and to the elementsin N ( T ) as the nodes of the decomposition. For a node u in T , wewrite T u to denote the subtree of T rooted at u . By slight abuse ofnotation, we will often write u ′ ∈ T u to denote that u ′ is a node inthe subtree T u of T . Finally, we define V ( T u ) : = (cid:208) u ′ ∈ T u B u ′ . Definition 2.2. A hypertree decomposition (HD) of a hypergraph H = ( V ( H ) , E ( H )) is a GHD, which in addition also satisfies thefollowing condition:(4) ∀ u ∈ N ( T ) : V ( T u ) ∩ B ( λ u ) ⊆ B u Definition 2.3. A fractional hypertree decomposition (FHD) [29]of a hypergraph H = ( V ( H ) , E ( H )) is a tuple (cid:10) T , ( B u ) u ∈ N ( T ) , ( γ u ) u ∈ N ( T ) (cid:11) , where conditions (1) and (2) of Definition 2.1 plus thefollowing condition (3’) hold:(3’) ∀ u ∈ N ( T ) : γ u is defined as γ u : E ( H ) → [ , ] with B u ⊆ B ( γ u ) .The width of a GHD, HD, or FHD is the maximum weight of thefunctions λ u or γ u , over all nodes u in T . The generalized hyper-tree width, hypertree width, and fractional hypertree width of H (denoted ghw ( H ) , hw ( H ) , fhw ( H ) ) is the minimum width over allGHDs, HDs, and FHDs of H , respectively. Condition (2) is calledthe “connectedness condition”, and condition (4) is referred to as“special condition” [26]. The set B u is often referred to as the “bag”at node u . The functions λ u and γ u are referred to as the λ - label and γ - label of node u . Strictly speaking, only HDs require that the un-derlying tree T be rooted. We assume that also the tree underlyinga GHD or an FHD is rooted where the root is arbitrarily chosen. Favourable properties of hypergraphs.
In [20], the followingproperties of hypergraphs were identified to allow for the defi-nition of tractable classes of Check(GHD, k ) and for an efficientapproximation of Check(FHD, k ), respectively. Definition 2.4.
The intersection width iwidth ( H ) of a hypergraph H is the maximum cardinality of any intersection e ∩ e of twoedges e (cid:44) e of H . We say that a hypergraph H has the i -bounded intersection property ( i -BIP) if iwidth ( H ) ≤ i . A class C of hyper-graphs has the bounded intersection property (BIP) if there existssome constant i such that every hypergraph H in C has the i -BIP. Definition 2.5.
For positive integer c , the c -multi-intersectionwidth c -miwidth( H ) of a hypergraph H is the maximum cardinalityof any intersection e ∩ · · · ∩ e c of c distinct edges e , . . . , e c of H .We say that a hypergraph H has the i -bounded c -multi-intersectionproperty ( ic -BMIP) if c -miwidth( H ) ≤ i holds. We say that a class C of hypergraphs has the bounded multi-intersection property (BMIP) if there exist constants c and i such that every hypergraph H in C has the ic -BMIP.There are two more relevant properties of (classes of) hyper-graphs: bounded degree and bounded Vapnik–Chervonenkis dimen-sion (VC-dimension). It is easy to verify [20] that bounded degreeimplies the BMIP, which in turn implies bounded VC-dimension. Definition 2.6.
The degree deg ( H ) of a hypergraph H is definedas the maximum number d of hyperedges in which a vertex occurs,i.e., d = max v ∈ V ( H ) |{ e ∈ E ( H ) | v ∈ E ( H )}| . We say that a class C of hypergraphs has bounded degree, if there exists d ≥
1, such thatevery hypergraph H ∈ C has degree ≤ d . Definition 2.7 ([51]).
Let H = ( V ( H ) , E ( H )) be a hypergraph, and X ⊆ V a set of vertices. Denote by E ( H )| X = { X ∩ e | e ∈ E ( H )} . X is called shattered if E ( H )| X = X . The Vapnik-Chervonenkisdimension (VC dimension) of H is the maximum cardinality of ashattered subset of V . We say that a class C of hypergraphs hasbounded VC-dimension, if there exists v ≥
1, such that everyhypergraph H ∈ C has VC-dimension ≤ v .The above four properties help to solve or approximate theCheck(GHD, k ) and Check(FHD, k ) problems as follows:Theorem 2.8 ([20, 21]). Let C be a class of hypergraphs. • If C has the BMIP, then the Check(GHD, k ) problem is solvable inpolynomial time for arbitrary k ≥ . Consequently, this tractabil-ity holds if C has bounded degree or the BIP (which each implythe BMIP) [20]. • If C has bounded degree, then the Check(FHD, k ) problem issolvable in polynomial time for arbitrary k ≥ [21]. • If C has bounded VC-dimension, then the fhw can be approxi-mated in polynomial time up to a logarithmic factor [20]. In this section, we introduce our system, test environment, andHyperBench – our new hypergraph benchmark and web tool.
System and Test Environment.
In [28], an implementation (called
DetKDecomp ) of the hypertree decomposition algorithm from [26]was presented. We have extended this implementation and builtour new library (called
NewDetKDecomp ) upon it. This library in-cludes the original hw -algorithm from [28], the tool hg-stats todetermine properties described in Section 4 and the algorithmsto be presented in Sections 5 and 6. The library is written in C++and comprises around 8,500 lines of code. The code is available inGitHub at http://github.com/TUfischl/newdetkdecomp.All the experiments reported in this paper were performed ona cluster of 10 workstations each running Ubuntu 16.04. Everyworkstation has the same specification and is equipped with twoIntel Xeon E5-2650 (v4) processors each having 12 cores and 256-GBmain memory. Since all algorithms are single-threaded, we were llowed to compute several instances in parallel. For all upcomingruns of our algorithms we set a timeout of 3600s. CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 1−1011−2021−3031−4041−50>50
Vertices
CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 1−1011−2021−3031−4041−50>50
Edges
CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 1−56−1011−1516−20>20
Arity
Figure 1: Hypergraph SizesHypergraph benchmark.
Our benchmark contains 3,070 hyper-graphs, which have been converted from CQs and CSPs collectedfrom various sources. Out of these 3,070 hypergraphs, 2,918 hyper-graphs have never been used in a hypertree width analysis before.The hypertree width of 70 CQs and of 82 CSPs has been analysedin [28], [13], and/or [14]. An overview of all instances of CQs andCSPs is given in Table 1. They have been collected from variouspublically available benchmarks and repositories of CQs and CSPs.In the first column, the names of each collection of CQs and CSPsare given together with references where they were first published.In the second column we display the number of hypergraphs ex-tracted from each collection. The hw of the CQs and CSPs in ourbenchmark will be discussed in detail in Section 4. To get a firstfeeling of the hw of the various sources, we mention the numberof cyclic hypergraphs (i.e., those with hw ≥
2) in the last column.
Table 1: Overview of benchmark instances
Benchmark No. instances hw ≥ C Q s SPARQL[14] 70 (out of 26,157,880) 70LUBM[12, 30] 14 2iBench[7, 12] 40 0Doctors[12, 22] 14 0Deep[12] 41 0JOB (IMDB) [38] 33 7TPC-H [11, 49] 33 1SQLShare [32] 290 (out of 15,170) 1Random [45] 500 464 C S P s Application [9] 1,090 1,090Random [9] 863 863Other [13, 28] 82 82
Total: 3,070 2,580
When gathering the CQs, we proceeded as follows: of the hugebenchmark reported in [14], we have only included CQs, whichwere detected as having hw ≥ • CQs : Our benchmark contains 535 CQs from four main sources[11, 12, 14, 32] and a set of 500 randomly generated queries usingthe query generator of [45]. In the sequel, we shall refer to theformer queries as
CQ Application , and to the latter as
CQ Random .The CQs analysed in [14] constitute by far the biggest repositoryof CQs – namely 26,157,880 CQs stemming from SPARQL queries.The queries come from real-users of SPARQL endpoints and theirhypertree width was already determined in [14]. Almost all of theseCQs were shown to be acyclic. Our analysis comprises 70 CQs from[14], which (apart from few exceptions) are essentially the ones in[14] with hw ≥
2. In particular, we have analysed all 8 CQs withhighest hw among the CQs analysed in [14] (namely, hw = hw ≥ ased on the TPC-H benchmark [49], 13 queries were handcraftedand 20 randomly generated. The TPC-H benchmark has been widelyused to assess multiple aspects of the capabilities of RDBMS to pro-cess queries. They reflect common workloads in decision supportsystems and were chosen to have broad industry-wide relevance.From SQLShare [32], a multi-year SQL-as-a-service experimentwith a large set of real-world queries, we extracted 15,170 queries byconsidering all CQs (in particular, no nested SELECTs). After elimi-nating trivial queries (i.e., queries with ≤ CQ Application hypergraphs. However, even though these sizevalues have been chosen similarly, the structural properties of thehypergraphs in two groups
CQ Application and
CQ Random differsignificantly, as will become clear from our analysis in Section 4. • CSPs : In total, our benchmark currently contains 2,035 hy-pergraphs from CSP instances, out of which 1,953 instances wereobtained from xcsp.org (see also [9]). We have selected all CSPinstances from xcsp.org with less than 100 constraints such thatall constraints are extensional. These instances are divided intoCSPs from concrete applications, called
CSP Application in the se-quel (1,090 instances), and randomly generated CSPs, called
CSPRandom other CSPs .Our HyperBench benchmark consists of these instances con-verted to hypergraphs. In Figure 1, we show the number of vertices,the number of edges and the arity (i.e., the maximum size of theedges) as three important metrics of the size of each hypergraph.The smallest are those coming from
CQ Application (at most 10edges), while the hypergraphs coming from CSPs can be signifi-cantly larger (up to 2993 edges). Although some hypergraphs arevery big, more than 50% of all hypergraphs have maximum arityless than 5. In Figure 1 we can easily compare the different types ofhypergraphs, e.g. hypergraphs of arity greater than 20 only exist inthe
CSP Application class; the other CSPs class contains the highestportion of hypergraphs with a big number of vertices and edges,etc.The hypergraphs and the results of their analysis can be accessedthrough our web tool, available at http://hyperbench.dbai.tuwien.ac.at.
In this section, we present first empirical results obtained withthe HyperBench benchmark. On the one hand, we want to get anoverview of the hypertree width of the various types of hypergraphsin our benchmark (cf. Goal 2 in Section 1). On the other hand, wewant to find out how realistic the restriction to low values forcertain hypergraph invariants is (cf. Goal 3 stated in Section 1).
Table 2: Hypergraph properties of all benchmark instances
CQ Application i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 0 118 173 101 2 421 348 302 3932 176 85 59 50 1323 137 7 5 5 04 87 5 5 5 05 35 17 0 0 06 98 0 0 0 0
CQ Random i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 1 16 49 01 1 17 77 125 202 15 53 90 120 1333 38 62 103 74 2404 31 63 62 42 1065 33 71 47 28 16 382 233 105 62 0
CSP Application & Other i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 0 597 603 01 0 1037 495 525 02 597 95 57 23 11153 6 29 21 21 524 20 10 2 0 05 6 0 0 0 0 > CSP Random i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 0 0 0 01 0 200 200 238 02 0 224 312 407 2203 0 76 147 95 5154 12 181 161 97 575 8 99 14 1 71 > Hypergraph Properties.
In [20, 21], several invariants of hyper-graphs were used to make the Check(GHD, k ) and Check(FHD, k )problems tractable or, at least, easier to approximate. We thus in-vestigate the following properties (cf. Definitions 2.4 – 2.7): • Deg : the degree of the underlying hypergraph • BIP : the intersection width • c - BMIP : the c -multi-intersection width for c ∈ { , }• VC-dim : the VC-dimensionThe results obtained from computing
Deg , BIP , 3-
BMIP , 4-
BMIP ,and
VC-dim for the hypergraphs in the HyperBench benchmarkare shown in Table 2.Table 2 has to be read as follows: In the first column, we distin-guish different values of the various hypergraph metrics. In thecolumns labelled “Deg“, “BIP“, etc., we indicate for how many in-stances each metric has a particular value. For instance, by the s0s 0s0s 0s k N o . o f i n s t a n ces yes no timeout CQ Application k N o . o f i n s t a n ces yes no timeout CQ Random
0s 7s0s 988s0s 1156s21s 0s30s 0s 0s 1s 0s 0s k N o . o f i n s t a n ces yes no timeout CSP
Figure 2: HW analysis (labels are average runtimes in s) last row in the second column, only 98 non-random CQs havedegree >
5. Actually, for most CQs, the degree is less than 10.Moreover, for the BMIP, already with intersections of 3 edges, weget 3 -miwidth( H ) ≤ CSP Application and
CSP Other , 543 (46%) hypergraphs have a high degree ( > > -miwidth( H ) ≤
4. For 5 instances the computationof the VC-dimension timed out. For all others, the VC-dimensionis ≤ > Hypertree Width.
We have systematically applied the hw -compu-tation from [28] to all hypergraphs in the benchmark. The resultsare summarized in Figure 2. In our experiments, we proceeded asfollows. We distinguish between CQ Application , CQ Random , andall three groups of CSPs taken together. For every hypergraph H ,we first tried to solve the Check(HD, k ) problem for k =
1. In caseof
CQ Application , we thus got 454 yes-answers and 81 no-answers.The number in each bar indicates the average runtime to find theseyes- and no-instances, respectively. Here, the average runtime was“0” (i.e., less than 1 second) in both cases. For
CQ Random we got 36yes- and 464 no-instances with an average runtime below 1 second.For all CSP-instances, we only got no-answers.In the second round, we tried to solve the Check(HD, k ) problemfor k = k = CQ Application yielded a yes-answer in less than 1 second. Forthe hypergraphs stemming from
CQ Random (resp. CSPs), only 68(resp. 95) instances yielded a yes-answer (in less than 1 second onaverage), while 396 (resp. 1932) instances yielded a no-answer inless than 7 seconds on average and 8 CSP instances led to a timeout(i.e., the program did not terminate within 3,600 seconds).This procedure is iterated by incrementing k and running the hw -computation for all instances, that either yielded a no-answeror a timeout in the previous round. For instance, for queries from CQ Application , one further round is needed after the second round.In other words, we confirm the observation of low hw , which wasalready made for CQs of arity ≤ CQ Random (resp. CSPs), 396 (resp. 1940 )instancesare left in the third round, of which 70 (resp. 232) yield a yes-answerin less than 1 second on average, 326 (resp. 1415) instances yielda no-answer in 32 (resp. 988) seconds on average and no (resp.293) instances yield a timeout. Note that, as we increase k , theaverage runtime and the percentage of timeouts first increase upto a certain point and then they decrease. This is due to the factthat, as we increase k , the number of combinations of edges tobe considered in each λ -label (i.e., the function λ u at each node u of the decomposition) increases. In principle, we have to test O( n k ) combinations, where n is the number of edges. However, if k increases beyond a certain point, then it gets easier to “guess” a λ -label since an increasing portion of the O( n k ) possible combinationsleads to a solution (i.e., an HD of desired width).To answer the question in Goal 2 , it is indeed the case that for abig number of instances, the hypertree width is small enough toallow for efficient evaluation of CQs or CSPs: all instances of non-random CQs have hw ≤ hw ≤
5. Intotal, including random CQs, 1,849 (60%) out of 3,070 instances have hw ≤
5, for which we could determine the exact hypertree widthfor 1,453 instances; the others may even have lower hw . Correlation Analysis.
Finally, we have analysed the pairwise cor-relation between all properties. Of course, the different intersectionwidths (BIP, 3-BMIP, 4-BMIP) are highly correlated. Other thanthat, we only observe quite a high correlation of the arity with thenumber of vertices and the hypertree width and of the number ofvertices with the arity and the hypertree width. Clearly, the corre-lation between arity and hypertree width is mainly due to the CSPinstances and the random CQs since, for non-random CQs, the hw never increases beyond 3, independently of the arity. graphical presentation of all pairwise correlations is givenin Figure 3. Here, large, dark circles indicate a high correlation,while small, light circles stand for low correlation. Blue circlesindicate a positive correlation while red circles stand for a negativecorrelation. In [20], we have argued that Deg, BIP, 3-BMIP, 4-BMIPand VC-dim are non-trivial restrictions to achieve tractability. Itis interesting to note that, according to the correlations shown inFigure 3, these properties have almost no impact on the hypertreewidth of our hypergraphs. This underlines the usefulness of theserestrictions in the sense that (a) they make the GHD computationand FHD approximation easier [20] but (b) low values of degree,(multi-)intersection-width, or VC-dimension do not pre-determinelow values of the widths. l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l −1−0.8−0.6−0.4−0.200.20.40.60.81 V e r t i c e s E dge s A r i t y D eg r ee B I P − B M I P − B M I P V C − D i m H W VerticesEdgesArityDegreeBIP3−BMIP4−BMIPVC−DimHW
Figure 3: Correlation analysis.
In this section, we report on new algorithms and implementationsto solve the Check(GHD, k ) problem and on new empirical results. Background.
In [20], it is shown that the Check(GHD, k ) problembecomes tractable for fixed k ≥
1, if we restrict ourselves to a classof hypergraphs enjoying the BIP. As our first empirical analysis withthe HyperBench has shown (see Section 4), it is indeed realisticto assume that the intersection width of a given hypergraph issmall. We have therefore extended the hw -computation from [28]by an implementation of the Check(GHD, k ) algorithm from [20],which will be referred to as the “ ghw -algorithm” in the sequel. Thisalgorithm is parameterized, so to speak, by two integers: k (thedesired width of a GHD) and i (the intersection width of H ).The key idea of the ghw -algorithm is to add a polynomial-timecomputable set f ( H , k ) of subedges of edges in E ( H ) to the hy-pergraph H , such that ghw ( H ) = k iff hw ( H ′ ) = k with H = ( V ( H ) , E ( H )) and H ′ = ( V ( H ) , E ( H ) ∪ f ( H , k )) . Tractability ofCheck(GHD, k ) follows immediately from the tractability of the Table 3: Comparison of GHW algorithms w. avg. runtime (s) hw → GlobalBIP LocalBIP BalSep ghw total yes no yes no yes no → → → → k ) problem. The set f ( H , k ) is defined as f ( H , k ) = (cid:216) e ∈ E ( H ) (cid:16) (cid:216) e ,..., e j ∈( E ( H )\{ e }) , j ≤ k ( e ∩( e ∪···∪ e j )) (cid:17) , i.e., f ( H , k ) contains all subsets of intersections of edges e ∈ E ( H ) with unions of ≤ k edges of H different from e . By the BIP, theintersection e ∩ ( e ∪ · · · ∪ e j ) has at most i · k elements. Hence, forfixed constants i and k , | f ( H , k )| is polynomially bounded. “Global” implementation. In a straightforward implementationof this algorithm, we compute f ( H , k ) and from this H ′ and call the hw -computation from [28] for the Check(HD, k ) problem as a “blackbox”. A coarse-grained overview of the results is given in Table 3 inthe column labelled as ‘GlobalBIP”. We call this implementation ofthe ghw algorithm of [20] “global” to indicate that the set f ( H , k ) iscomputed “globally”, once and for all, for the entire hypergraph. Wehave run the program on each hypergraph from the HyperBench upto hypertree width 6, trying to get a smaller ghw than hw . We havethus run the ghw -algorithm with the following parameters: for allhypergraphs H with hw ( H ) = k (or hw ≤ k and, due to timeouts,we do not know if hw ≤ k − k ∈ { , , , } , tryto solve the Check(GHD, k −
1) problem. In other words, we justtried to improve the width by 1. Clearly, for hw ( H ) ∈ { , } , noimprovement is possible since, in this case, hw ( H ) = ghw ( H ) holds.In Table 3, we report on the number of “successful” attemptsto solve the Check(GHD, k −
1) problem for hypergraphs with hw = k . Here “successful” means that the program terminatedwithin 1 hour. For instance, for the 310 hypergraphs with hw = hw =
4, the “global” computation terminatedin 137 cases (i.e., 35%) with average runtime 2809 when trying tosolve the Check(GHD, 3) problem. For the 886 hypergraphs with hw ∈ { , } , the “global” computation only terminated in 13 cases(i.e., 1.4%). Overall, it turns out that the set f ( H , k ) may be very big(even though it is polynomial if k and i are constants). Hence, H ′ can become considerably bigger than H . This explains the frequenttimeouts in the GlobalBIP column in Table 3. “Local” implementation. Looking for ways to improve the ghw -algorithm, we closely inspect the role played by the set f ( H , k ) inthe tractability proof in [20]. The definition of this set is motivatedby the problem that, in the top down construction of a GHD, wemay want to choose at some node u the bag B u such that x (cid:60) B u for some variable x ∈ B ( λ u ) ∩ V ( T u ) . This violates condition (4) ofDefinition 2.2 (the “special condition”) and is therefore forbiddenin an HD. In particular, there exists an edge e with x ∈ e and λ u ( e ) =
1. The crux of the ghw -algorithm in [20] is that for everysuch “missing” variable x , the set f ( H , k ) contains a subedge e ′ ⊆ e ith x (cid:60) e ′ . Hence, replacing e by e ′ in λ u (i.e., setting λ u ( e ) = λ u ( e ′ ) = λ u unchanged elsewhere) eliminatesthe special condition violation. By the connectedness condition, itsuffices to consider the intersections of e with unions of edges thatmay possibly occur in bags of T u rather than with arbitrary edgesin E ( H ) . In other words, for each node u in the decomposition, wemay restrict f ( H , k ) to an appropriate subset f u ( H , k ) ⊆ f ( H , k ) .The results obtained with this enhanced version of the ghw -computation are shown in Table 3 in the column labelled “LocalBIP”.We call this implementation of ghw -computation “local” becausethe set f u ( H , k ) of subedges of H to be added to the hypergraph iscomputed separately for each node u of the decomposition. Recallthat in this table, the “successful” calls of the program are recorded.Interestingly, for the hypergraphs with hw =
3, the “local” computa-tion performs significantly better (namely 63% solved with averageruntime 162 seconds rather than 41% with average runtime 537seconds). In contrast, for the hypergraphs with hw =
4, the “global”computation is significantly more successful. For hw ∈ { , } , the“global” and “local” computations are equally bad. A possible expla-nation for the reverse behaviour of “global” and “local” computationin case of hw = hw = f ( H , k ) of subedges to the “local” set f u ( H , k ) at eachnode u seems to be quite effective for the hypergraphs with hw = f u ( H , k ) ateach node u becomes counter-productive, when the set of subedgesthus eliminated is not significant. It is interesting to note that thesets of solved instances of the global computation and the localcomputation are incomparable, i.e., in some cases one method isbetter, while in other cases the other method is better. New alternative approach: “balanced separators”.
We now pro-pose a completely new approach, based on so-called “balanced sep-arators”. The latter are a familiar concept in graph theory [18, 48] –denoting a set S of vertices of a graph G , such that the subgraph G ′ induced by V ( G ) \ S has no connected component larger than somegiven size, e.g., α · | V | for some given α ∈ ( , ) . In our setting, wemay consider the label λ u at some node u in a GHD as separator in the sense that we can consider connected components of thesubhypergraph H ′ of H induced by V ( H ) \ B u . Clearly, in a GHD,we may consider any node as the root. So suppose that u is theroot of some GHD. Moreover, as is shown in [20] in the proof oftractability of Check(GHD, k ) in case of the BIP, we may choose λ u such that B ( λ u ) = B u if the subedges in f ( H , k ) have been addedto the hypergraph.By the HD-algorithm from [26], we know that an HD of H ′ (and,hence, a GHD of H ) can be constructed in such a way that everysubtree rooted at a child node u i of u contains only one connectedcomponent C i of the subhypergraph of H ′ induced by V ( H ) \ B u .For our purposes, it is convenient to define the size of a component C i as the number of edges that have to be covered at some node inthe subtree rooted at u i in the GHD. We thus call a separator λ u “balanced”, if the size of each component C i is at most | E ( H ′ )|/ In every GHD, there exists a node u (which wemay choose as the root) such that λ u is a balanced separator. This property allows us to design the algorithm sketched inFigure 4 to compute a GHD of H ′ . Actually, as will become clearbelow, we assume that the input to this recursive algorithm consistsof a hypergraph plus a set Sp of “special edges” and we request ALGORITHM
Find_GHD_via_balancedSeparators// high-level description
Input: hypergraph H ′ , integer k ≥ Output: a GHD ⟨ T , B u , λ u ⟩ of width ≤ k if exists,“Reject”, otherwise. Procedure
Find_GHD ( H : Hypergraph, Sp : Set of special edges) begin
1. Base Case: if there are only special edges left and | Sp | ≤ λ : E ( H ′ ) → { , } check if λ u is a balanced separator for H ;if none is found then return Reject.3. Split H into connected components C , . . . , C ℓ w.r.t. λ u : C i ⊆ V ( H ) \ B ( λ u ) for every i and C i is connected in H and each C i is maximal with this property.4. Build the pair (cid:10) H i , Sp i (cid:11) (the subhypergraph based on C i andthe special edges in C i ) for each connected component C i ;add B ( λ u ) as one more special edge to each set Sp i .5. Call Find_GHD( H i , Sp i ) for each pair (cid:10) H i , Sp i (cid:11) ;each successful call returns a GHD T i for H i if one call returns Reject then return Reject.6. Create and return a new GHD for H having λ u as root:each T i has one leaf node labelled B ( λ u ) ;the new GHD is obtained by gluing together all subtrees T i at the node with label B ( λ u ) . endbegin (* Main *) return Find_GHD ( H ′ , ∅ ); end Figure 4: Recursive GHD-algorithm via balanced separators that the GHD to be constructed contains “special nodes”, which (a)have to be leaf nodes in the decomposition and (b) the λ -label ofsuch a leaf node consists of a single special edge only. Each specialedge contains the set of vertices B u of some balanced separator λ u further up in the hierarchy of recursive calls of the decompositionalgorithm. The special edges are propagated to the recursive calls forsubhypergraphs in order to determine how to assemble the overallGHD from the GHDs of the subhypergraphs. This will becomeclearer in the proof sketch of Theorem 5.2.Theorem 5.2. Let H be a hypergraph, let k ≥ , and let H ′ beobtained from H by adding the subedges in f ( H , k ) to E ( H ) . Thenthe algorithm Find_GHD_via_balancedSeparators given in Figure 4outputs a GHD of width ≤ k if one exists and rejects otherwise. Proof Sketch. Steps 1 –5 of the algorithm in Figure 4 essen-tially correspond to the computation of λ u and B u for the root node u in the HD-computation of [26]. The most significant modifica-tions here are due to the handling of “special edges” in parameter Sp . A crucial property of the construction in [26] and also of ourconstruction here is that each subtree below node u in the decom-position only contains vertices from a single connected component C i w.r.t. V ( H ) \ B u (see Steps 3 and 4). Since the special edges comefrom such bags B u , special edges can never be used as separatorsin recursive calls below. Hence, the base case (in Step 1) is reachedfor E ( H ) = ∅ and | Sp | ≤
2. Indeed, | Sp | ≥ able 4: GHW of instances with average runtime in s hw → ghw yes no timeout3 → → → → H i with set Sp i of special edges arecorrect, i.e., they yield for each hypergraph H i a GHD D i such thateach special edge s in Sp i is indeed covered by a leaf node in D i whose λ -label consists of s only. In particular, since s = B ( λ u ) is aspecial edge contained in Sp i for each i , there exists a leaf node t i in D i with λ t i = { s } . In a GHD, any node can be taken as the root. Wethus choose t i as the root node in each GHD D i . By construction, wehave B u = B t = · · · = B t ℓ . Moreover, any two subhypergraphs H i , H j contain the vertices from two different connected components.Hence, apart from the vertices contained in the special edge s , anytwo GHDs D i , D j with i (cid:44) j have no vertices in common. We cantherefore construct a GHD D of H ′ by deleting the root node t i from each GHD D i and by appending the child nodes of each t i directly as child nodes of u . Clearly, the connectedness condition issatisfied in the resulting decomposition. □ If we look at the number of solved instances in Table 3, we seethat the recursive algorithm via balanced separators (reported in thelast column labelled
BalSep ) has the least number of timeouts dueto the fast identification of negative instances (i.e., those with no-answer), where it often detects quite fast that a given hypergraphdoes not have a balanced separator of desired width. As k increases,the performance of the balanced separators approach deteriorates.This is due to k in the exponent of the running time of our algorithm,i.e. we need to check for each of the possible O( n k + ) combinationsof ≤ k edges if it constitutes a balanced separator. Empirical results.
We now look at Table 4, where we report forall hypergraphs with hw ≤ k and k ∈ { , , , } , whether ghw ≤ k − hw to ghw : in 98% of the solved cases and 57% of all instances with hw ≤ hw and ghw have identical values. Actually, we thinkthat the high percentage of the solved cases gives a more realisticpicture than the percentage of all cases for the following reason:our algorithms (in particular, the “global” and “local” computations)need particularly long time for negative instances. This is due tothe fact that in a negative case, “all” possible choices of λ -labelsfor a node u in the GHD have to be tested before we can be surethat no GHD of H (or, equivalently, no HD of H ′ ) of desired width exists. Hence, it seems plausible that the timeouts are mainly due tonegative instances. This also explains why our new GHD algorithmin Figure 4, which is particularly well suited for negative instances,has the least number of timeouts.We conclude this section with a final observation: in Figure 2,we had many cases, for which only some upper bound k on the hw could be determined, namely those cases, where the attempt tosolve Check(HD, k ) yields a yes-answer and the attempt to solveCheck(HD, k −
1) gives a timeout. In several such cases, we couldget (with the balanced separator approach) a no-answer for theCheck(GHD, k −
1) problem, which implicitly gives a no-answer forthe problem Check(HD, k − ghw -computation is also profitable for the hw -computation:for 827 instances with hw ≤
6, we were not able to determine theexact hypertree width. Using our new ghw -algorithm, we closedthis gap for 297 instances; for these instances hw = ghw holds.To sum up, we now have a total of 1,778 (58%) instances forwhich we determined the exact hypertree width and a total of 1,406instances (46%) for which we determined the exact generalizedhypertree width. Out of these, 1,390 instances had identical valuesfor hw and ghw . In 16 cases, we found an improvement of thewidth by 1 when moving from hw to ghw , namely from hw = ghw =
5. In 2 further cases, we could show hw ≤ ghw ≤ hw = ghw = Goal 6 , hw is equal to ghw in 45% of the cases if weconsider all instances and in 60% of the cases (1,390 of 2,308) withsmall width ( hw ≤ hw and ghw ), then hw and ghw coincide in 99% of the cases (1,390 of 1,406). The algorithms proposed in the literature for computing FHDsare very expensive. For instance, even the algorithm used for thetractability result in [21] for hypergraphs of low degree is problem-atical since it involves a double-exponential factor in the degree.Therefore, we investigate the potential of a simplified method tocompute approximated FHDs. Below, we present two algorithms forsuch approximated FHD computations – with a trade-off betweencomputational cost and quality of the approximation. • The simplest way to obtain a fractionally improved (G)HD isto take either a GHD or HD as input and compute a fractionallyimproved (G)HD. To this end, an algorithm (which we refer to as
SimpleImproveHD ) visits each node u of a given GHD or HD andcomputes an optimal fractional edge cover γ u for the set B u ofvertices. This algorithm is simple and computationally inexpensive,provided that we can start off with a GHD or HD that was computedbefore. In our case, we simply took the HD resulting from the hw -computation reported in Figure 2. Clearly, this approach is rathernaive and the dependence on a concrete HD is unsatisfactory. Wetherefore move to a more sophisticated algorithm described next. • The algorithm
FracImproveHD has as input a hypergraph H and numbers k , k ′ ≥
1, where k is an upper bound on the hw and k ′ the desired fractionally improved hw . We search for an FHD D ′ with D ′ = SimpleImproveHD (D) for some HD D of H with width (D) ≤ k and width (D ′ ) ≤ k ′ . In other words, this algorithmsearches for the best fractionally improved HD over all HDs ofwidth ≤ k . Hence, the result is independent of any concrete HD. able 5: Instances solved with SimpleImproveHD hw ≥ [ . , ) [ . , . ) no timeout2 0 41 25 172 03 12 104 25 169 04 9 55 11 311 05 20 14 11 382 06 12 60 80 309 0 Table 6: Instances solved with
FracImproveHD hw ≥ [ . , ) [ . , . ) no timeout2 0 46 29 160 13 14 116 21 135 244 11 81 2 8 2845 18 126 59 2 2226 28 149 95 4 183The experimental results with these algorithms for computingfractionally improved HDs are summarized in Table 5 and Table 6.We have applied these algorithms to all hypergraphs for which hw ≤ k with k ∈ { , , , } is known from Figure 2. The variouscolumns of the Tables 5 and 6 are as follows: the first column(labelled hw ) refers to the (upper bound on the) hw according toFigure 2. The next 3 columns, labelled ≥ [ . , ) , and [ . , . ) tell us, by how much the width can be improved (if at all) if wecompute an FHD by one of the two algorithms. We thus distinguishthe 3 cases if, for a hypergraph of hw ≤ k , we manage to constructan FHD of width k − c for c ≥ c ∈ [ . , ) , or c ∈ [ . , . ) . Thecolumn with label “no” refers to the cases where no improvementat all or at least no improvement by c ≥ . hw -computation of Figure 2) out of 238 hypergraphs with hw =
2, no improvement was possible in 172 cases. In the remaining66 cases, an improvement to a width of at most 2 − . k − c with c ∈ [ . , . ) waspossible in 41 cases. For the hypergraphs with hw = c ∈ [ . , ) and 12 even byat least 1. The improvements achieved for the hypergraphs with hw ≤ hw ≤ k ′ ≤ k with k ∈ { , } holds. In the other cases, the results obtained with thenaive SimpleImproveHD algorithm are not much worse than withthe more sophisticated FracImproveHD algorithm. We distinguish several types of works that are highly relevant toours. The works most closely related are the descriptions of HD,GHD and FHD algorithms in [20, 26] and the implementation ofHD computation by the
DetKDecomp program reported in [28]. Wehave extended these works in several ways. Above all, we have incorporated our analysis tool (reported in Sections 3 and 4) andthe GHD and FHD computations (reported in Sections 5 and 6) intothe
DetKDecomp program – resulting in our
NewDetKDecomp library,which is openly available on GitHub. For the GHD computation,we have added heuristics to speed up the basic algorithm from[20]. Moreover, we have proposed a novel approach via balancedseparators, which allowed us to significantly extend the range ofinstances for which the GHD computation terminates in reasonabletime. We have also introduced a new form of decomposition method:the fractionally improved decompositions (see Section 6), whichallow for a practical, lightweight form of FHDs.The second important input to our work comes from the varioussources [7, 11–13, 22, 28, 32, 37, 49] which we took our CQs andCSPs from. Note that our main goal was not to add further CQsand/or CSPs to these benchmarks. Instead, we have aimed at takingand combining existing, openly accessible benchmarks of CQs andCSPs, convert them into hypergraphs, which are then thoroughlyanalysed. Finally, the hypergraphs and the analysis results are madeopenly accessible again.The third kind of works highly relevant to ours are previousanalyses of CQs and CSPs. To the best of our knowledge, Ghionnaet al. [23] presented the first systematic study of HDs of benchmarkCQs from TPC-H. However, Ghionna et al. pursued a research goaldifferent from ours in that they primarily wanted to find out towhat extent HDs can actually speed up query evaluation. Theyachieved very positive results in this respect, which have recentlybeen confirmed by the work of Perelman et al. [43], Tu et al. [50]and Aberger et al. [1, 2] on query evaluation using FHDs. As aside result, Ghionna et al. also detected that CQs tend to have lowhypertree width (a finding which was later confirmed in [14, 44]and also in our study). In a pioneering effort, Bonifati, Martens,and Timm [14] have recently analysed an unprecedented, massiveamount of queries: they investigated 180,653,910 queries from (notopenly available) query logs of several popular SPARQL endpoints.After elimination of duplicate queries, there were still 56,164,661queries left, out of which 26,157,880 queries were in fact CQs. Theauthors thus significantly extend previous work by Picalausa andVansummeren [44], who analysed 3,130,177 SPARQL queries posedby humans and software robots at the DBPedia SPARQL endpoint.The focus in [44] is on structural properties of SPARQL queries suchas keywords used and variable structure in optional patterns. Thereis one paragraph devoted to CQs, where it is noted that 99.99% ofca. 2 million CQs considered in [44] are acyclic.Many of the CQs (over 15 million) analysed in [14] have ar-ity 2 (here we consider the maximum arity of all atoms in a CQas the arity of the query), which means that all triples in such aSPARQL query have a constant at the predicate-position. Bonifatiet al. made several interesting observations concerning the shapeof these graph-like queries. For instance, they detected that exactlyone of these queries has tw =
3, while all others have tw ≤ hw ≤ DetKDecomp program from [28].Out of 6,959,510 CQs of arity 3, only 86 (i.e. 0.01%(cid:24)) turned out tohave hw = hw =
3, while all other CQs of arity3 are acyclic. Our analysis confirms that, also for non-random CQsof arity >
3, the hypertree width indeed tends to be low, with themajority of queries being even acyclic. or the analysis of CSPs, much less work has been done. Al-though it has been shown that exploiting (hyper-) tree decomposi-tions may significantly improve the performance of CSP solving[5, 31, 33, 36], a systematic study on the (generalized) hypertreewidth of CSP instances has only been carried out by few works[28, 36, 47]. To the best of our knowledge, we are the first to analysethe hw , ghw , and fhw of ca. 2,000 CSP instances, where most ofthese instances have not been studied in this respect before.It should be noted that the focus of our work is different from theabove mentioned previous works: above all, we wanted to test thepractical feasibility of various algorithms for HD, GHD, and FHDcomputation (including both, previously presented algorithms andnew ones developed as part of this work). As far as our repositoryof hypergraphs (obtained from CQs and CSPs) is concerned, weemphasize open accessibility. Thus, users can analyse their CQs andCSPs (with our implementations of HD, GHD, and FHD algorithms)or they can analyse new decomposition algorithms (with our hy-pergraphs, which cover quite a broad range of characteristics). Infact, in the recent work on FHD computation via SMT solving [19],the Hyperbench benchmark has already been used for the exper-imental evaluation. In [19] a novel approach to fhw computationvia an efficient encoding of the check-problem for FHDs to SMT(SAT modulo Theory) is presented. The tests were carried out with2,191 hypergraphs from the initial version of the HyperBench. Forall of these hypergraphs we have established at least some upperbound on the fhw either by our hw -computation or by one of ournew algorithms presented in Sections 5 and 6. In contrast, the exactalgorithm in [19] found FHDs only for 1.449 instances (66%). In 852cases, both our algorithms and the algorithm in [19] found FHDs ofthe same width; in 560 cases, an FHD of lower width was found in[19]. By using the same benchmark for the tests, the results in [19]and ours are comparable and have thus provided valuable input forfuture improvements of the algorithms by combining the differentstrengths and weaknesses of the two approaches.The use of the same benchmark has also allowed us to providefeedback to the authors of [19] for debugging their system: in 9 outof 2,191 cases, the “optimal” value for the fhw computed in [19]was apparently erroneous, since it was higher than the hw foundout by our analysis; note that upper bounds on the width are, ingeneral, more reliable than lower bounds since it is easy to verify ifa given decomposition indeed has the desired properties, whereasruling out the existence of a decomposition of a certain width is acomplex and error-prone task. In this work, we have presented HyperBench, a new and compre-hensive benchmark of hypergraphs derived from CQs and CSPsfrom various areas, together with the results of extensive empiricalanalyses with this benchmark.
Lessons learned.
The empirical study has brought many insights.Below, we summarize the most important lessons from our studies. • The finding of [14, 44] that non-random CQs have low hyper-tree width has been confirmed by our analysis, even if (in contrastto SPARQL queries) the arity of the CQs is not bounded by 3. Forrandom CQs and CSPs, we have detected a correlation betweenthe arity and the hypertree width, although also in this case, theincrease of the hw with increased arity is not dramatic. • In [20], several hypergraph invariants were identified, whichmake the computation of GHDs and the approximation of FHDs tractable. We have seen that, at least for non-random instances,these invariants indeed have low values. • The reduction of the ghw -computation problem to the hw -computation problem in case of low intersection width turned out tobe more problematical than the theoretical tractability results from[20] had suggested. Even the improvement by “local” computationof the additional subedges did not help much. However, we wereable to improve this significantly by presenting a new algorithmbased on “balanced separators”. In particular for negative instances(i.e., those with a no-answer), this approach proved very effective. • An additional benefit of the new ghw -algorithm based on“balanced separators” is that it allowed us to also fill gaps in the hw -computation. Indeed, in several cases, we managed to verify hw ≤ k for some k but we could not show hw ≰ k −
1, due to atimeout for Check(HD, k − ghw ≰ k − hw ≰ k −
1. Thisallowed us to compute the exact hw of many further hypergraphs. • Most surprisingly, the discrepancy between hw and ghw ismuch lower than expected. Theoretically, only the upper bound hw ≤ · ghw + hw ≤
6, we could show that in 53% of all cases, hw and ghw are simply identical. Moreover, in all cases when one ofour implementations of ghw -computation terminated on instanceswith hw ≤
5, we got identical values for hw and ghw . Future work.
Our empirical study has also given us many hintsfor future directions of research. We find the following tasks partic-ularly urgent and/or rewarding. • So far, we have only implemented the ghw -computation in caseof low intersection width. In [20], tractability of the Check(GHD, k )problem was also proved for the more relaxed bounded multi-intersection width. Our empirical results in Figure 6 show that, apartfrom the random CQs and random CSPs, the 3-multi-intersection is ≤ • The three approaches for ghw -computation presented hereturned out to have complementary strengths and weaknesses. Thiswas profitable when running all three algorithms in parallel andtaking the result of the first one that terminates (see Table 4). In thefuture, we also want to implement a more sophisticated combina-tion of the various approaches: for instance, one could try to applyour new “balanced separator” algorithm recursively only down toa certain recursion depth (say depth 2 or 3) to split a big givenhypergraph into smaller subhypergraphs and then continue withthe “global” or “local” computation from Section 5. • Our new approach to ghw -computation via “balanced separa-tors” proved quite effective in our experiments. However, furthertheoretical underpinning of this approach is missing. The empiricalresults obtained for our new GHD algorithm via balanced separatorssuggest that the number of balanced separators is often drasticallysmaller than the number of arbitrary separators. We want to deter-mine a realistic upper bound on the number of balanced separatorsin terms of n (the number of edges) and k (an upper bound on thewidth). This will then allow us to compute also a realistic upperbound on the runtime of this new algorithm. • Finally, we want to further extend the HyperBench benchmarkand tool in several directions. We will thus incorporate further im-plementations of decomposition algorithms from the literature suchas the GHD- and FHD computation in [41] or the polynomial-time HD computation for hypergraphs of bounded degree in [21]. More-over, we will continue to fill in hypergraphs from further sourcesof CSPs and CQs. For instance, in [1, 15, 23, 24] a collection of CQsfor the experimental evaluations in those papers is mentioned. Wewill invite the authors to disclose these CQs and incorporate theminto the HyperBench benchmark. • Very recently, a new, huge, publically available query log hasbeen reported in [39]. It contains over 200 million SPARQL querieson Wikidata. In the paper, the anonymisation and publication ofthe query logs is mentioned as future work. However, on their website, the authors have meanwhile made these queries available. Atfirst glance, these queries seem to display a similar behaviour asthe SPARQL queries collected by Bonifatti et al. [14]: there is a bignumber of single-atom queries and again, the vast majority of thequeries is acyclic. A detailed analysis of the query log in the styleof [14] constitutes an important goal for future research.
Acknowledgements
We would like to thank Angela Bonifati, Wim Martens, and ThomasTimm for sharing most of the hypergraphs with hw ≥ REFERENCES [1] Christopher R. Aberger, Andrew Lamb, Susan Tu, Andres Nötzli, Kunle Oluko-tun, and Christopher Ré. 2017. EmptyHeaded: A Relational Engine for GraphProcessing.
ACM Trans. Database Syst.
42, 4 (2017), 20:1–20:44.[2] Christopher R. Aberger, Susan Tu, Kunle Olukotun, and Christopher Ré. 2016.EmptyHeaded: A Relational Engine for Graph Processing. In
Proc. SIGMOD 2016 .ACM, 431–446.[3] Christopher R. Aberger, Susan Tu, Kunle Olukotun, and Christopher Ré. 2016.Old Techniques for New Join Algorithms: A Case Study in RDF Processing.
CoRR abs/1602.03557 (2016). http://arxiv.org/abs/1602.03557[4] Isolde Adler, Georg Gottlob, and Martin Grohe. 2007. Hypertree width and relatedhypergraph invariants.
Eur. J. Comb.
28, 8 (2007), 2167–2181.[5] Kamal Amroun, Zineb Habbas, and Wassila Aggoune-Mtalaa. 2016. A compressedGeneralized Hypertree Decomposition-based solving technique for non-binaryConstraint Satisfaction Problems.
AI Commun.
29, 2 (2016), 371–392.[6] Molham Aref, Balder ten Cate, Todd J. Green, Benny Kimelfeld, Dan Olteanu,Emir Pasalic, Todd L. Veldhuizen, and Geoffrey Washburn. 2015. Design andImplementation of the LogicBlox System. In
Proc. SIGMOD 2015 . ACM.[7] Patricia C. Arocena, Boris Glavic, Radu Ciucanu, and Renée J. Miller. 2015. TheiBench Integration Metadata Generator.
Proc. VLDB Endow.
9, 3 (Nov. 2015),108–119.[8] Albert Atserias, Martin Grohe, and Dániel Marx. 2013. Size Bounds and QueryPlans for Relational Joins.
SIAM J. Comput.
42, 4 (2013), 1737–1767.[9] Gilles Audemard, Frédéric Boussemart, Christoph Lecoutre, and Cédric Piette.2016. XCSP3: an XML-based format designed to represent combinatorial con-strained problems. http://xcsp.org. (2016).[10] Nurzhan Bakibayev, Tomás Kociský, Dan Olteanu, and Jakub Závodný. 2013.Aggregation and Ordering in Factorised Databases.
PVLDB
6, 14 (2013).[11] Michael Benedikt. 2017. CQ benchmarks. (2017). Personal Communication.[12] Michael Benedikt, George Konstantinidis, Giansalvatore Mecca, Boris Motik,Paolo Papotti, Donatello Santoro, and Efthymia Tsamoura. 2017. Benchmarkingthe Chase. In
Proc. PODS 2017 . ACM, 37–52.[13] Jeremias Berg, Neha Lodha, Matti Järvisalo, and Stefan Szeider. 2017. MaxSATBenchmarks based on Determining Generalized Hypertree-width.
MaxSAT Eval-uation 2017 (2017), 22.[14] Angela Bonifati, Wim Martens, and Thomas Timm. 2017. An Analytical Study ofLarge SPARQL Query Logs.
PVLDB
Proc. PODS 2017 . ACM, 273–287.[16] Ashok K. Chandra and Philip M. Merlin. 1977. Optimal Implementation ofConjunctive Queries in Relational Data Bases. In
Proc. STOC 1977 . ACM, 77–90.[17] Rina Dechter. 2003.
Constraint Processing .[18] Uriel Feige and Mohammad Mahdian. 2006. Finding small balanced separators.In
Proc. STOC 2006 . ACM, 375–384. https://doi.org/10.1145/1132516.1132573[19] Johannes K. Fichte, Markus Hecher, Neha Lodha, and Stefan Szeider. 2018. AnSMT Approach to Fractional Hypertree Width. In
Proc. CP 2018 (LNCS) , Vol. 11008.Springer, 109–127. [20] Wolfgang Fischl, Georg Gottlob, and Reinhard Pichler. [n. d.]. General andFractional Hypertree Decompositions: Hard and Easy Cases. In
Proc. PODS 2018 .[21] Wolfgang Fischl, Georg Gottlob, and Reinhard Pichler. 2017. Tractable Cases forRecognizing Low Fractional Hypertree Width. viXra.org e-prints viXra:1708.0373(2017). http://vixra.org/abs/1708.0373[22] Floris Geerts, Giansalvatore Mecca, Paolo Papotti, and Donatello Santoro. 2014.Mapping and cleaning. In
Proc. ICDE 2014 . IEEE, 232–243.[23] Lucantonio Ghionna, Luigi Granata, Gianluigi Greco, and Francesco Scarcello.2007. Hypertree Decompositions for Query Optimization. In
Proc. ICDE 2007 .IEEE Computer Society, 36–45. https://doi.org/10.1109/ICDE.2007.367849[24] Lucantonio Ghionna, Gianluigi Greco, and Francesco Scarcello. 2011. H-DB: ahybrid quantitative-structural sql optimizer. In
Proc. CIKM 2011 . ACM, 2573–2576.[25] Georg Gottlob, Gianluigi Greco, Nicola Leone, and Francesco Scarcello. 2016.Hypertree Decompositions: Questions and Answers. In
Proc. PODS 2016 . ACM,57–74.[26] Georg Gottlob, Nicola Leone, and Francesco Scarcello. 2002. Hypertree Decom-positions and Tractable Queries.
J. Comput. Syst. Sci.
64, 3 (2002), 579–627.[27] Georg Gottlob, Zoltán Miklós, and Thomas Schwentick. 2009. Generalized Hyper-tree Decompositions: NP-hardness and Tractable Variants.
J. ACM
56, 6 (2009),30:1–30:32.[28] Georg Gottlob and Marko Samer. 2008. A backtracking-based algorithm forhypertree decomposition.
ACM Journal of Experimental Algorithmics
13 (2008).[29] Martin Grohe and Dániel Marx. 2014. Constraint Solving via Fractional EdgeCovers.
ACM Trans. Algorithms
11, 1 (2014), 4:1–4:20.[30] Yuanbo Guo, Zhengxiang Pan, and Jeff Heflin. 2005. LUBM: A benchmark forOWL knowledge base systems.
J. Web Sem.
3, 2-3 (2005), 158–182. https://doi.org/10.1016/j.websem.2005.06.005[31] Zineb Habbas, Kamal Amroun, and Daniel Singer. 2015. A Forward-Checkingalgorithm based on a Generalised Hypertree Decomposition for solving non-binary constraint satisfaction problems.
J. Exp. Theor. Artif. Intell.
27, 5 (2015),649–671. https://doi.org/10.1080/0952813X.2014.993507[32] Shrainik Jain, Dominik Moritz, Daniel Halperin, Bill Howe, and Ed Lazowska.2016. SQLShare: Results from a Multi-Year SQL-as-a-Service Experiment. In
Proceedings of the 2016 International Conference on Management of Data (SIGMOD’16) . ACM, New York, NY, USA, 281–293. https://doi.org/10.1145/2882903.2882957[33] Shant Karakashian, Robert J. Woodward, and Berthe Y. Choueiry. 2011. Reformu-lating R(*, m)C with Tree Decomposition. In
SARA . AAAI.[34] Mahmoud Abo Khamis, Hung Q. Ngo, Christopher Ré, and Atri Rudra. 2015.Joins via Geometric Resolutions: Worst-case and Beyond. In
Proc. PODS 2015 .[35] Mahmoud Abo Khamis, Hung Q. Ngo, and Atri Rudra. 2016. FAQ: QuestionsAsked Frequently. In
Proc. PODS 2016 . 13–28.[36] Mohammed Lalou, Zineb Habbas, and Kamal Amroun. 2009. Solving HypertreeStructured CSP: Sequential and Parallel Approaches. In
Proc. RCRA@AI*IA 2009 .[37] Viktor Leis, Andrey Gubichev, Atanas Mirchev, Peter Boncz, Alfons Kemper, andThomas Neumann. 2015. How Good Are Query Optimizers, Really?
PVLDB
9, 3(Nov. 2015), 204–215. https://doi.org/10.14778/2850583.2850594[38] Viktor Leis, Bernhard Radke, Andrey Gubichev, Atanas Mirchev, Peter Boncz,Alfons Kemper, and Thomas Neumann. 2017. Query optimization through thelooking glass, and what we found running the Join Order Benchmark.
The VLDBJournal (18 Sep 2017). https://doi.org/10.1007/s00778-017-0480-7[39] Stanislav Malyshev, Markus Krötzsch, Larry González, Julius Gonsior, and AdrianBielefeldt. 2018. Getting the Most out of Wikidata: Semantic Technology Usagein Wikipedia’s Knowledge Graph. In
Proc. ISWC 2018 . To appear.[40] Dániel Marx. 2010. Approximating Fractional Hypertree Width.
ACM Trans.Algorithms
6, 2, Article 29 (2010), 29:1–29:17 pages.[41] Lukas Moll, Siamak Tazari, and Marc Thurley. 2012. Computing hypergraphwidth measures exactly.
Inf. Process. Lett.
ACM Trans. Database Syst.
40, 1 (2015), 2.[43] Adam Perelman and Christopher Ré. 2015. DunceCap: Compiling Worst-CaseOptimal Query Plans. In
Proc. SIGMOD 2015 . ACM, 2075–2076.[44] François Picalausa and Stijn Vansummeren. 2011. What are real SPARQL querieslike?. In
Proc. SWIM 2011 . ACM, 7. https://doi.org/10.1145/1999299.1999306[45] Rachel Pottinger and Alon Halevy. 2001. MiniCon: A Scalable Algorithm forAnswering Queries Using Views.
The VLDB Journal
10, 2-3 (Sept. 2001), 182–198.[46] Francesco Scarcello, Gianluigi Greco, and Nicola Leone. 2007. Weighted hypertreedecompositions and optimal query plans.
J. Comput. Syst. Sci.
73, 3 (2007).[47] Werner Schafhauser. 2006. New heuristic methods for tree decompositions andgeneralized hypertree decompositions. (2006). Master Thesis, TU Wien.[48] Aaron Schild and Christian Sommer. 2015. On Balanced Separators in RoadNetworks. In
Proc. SEA 2015 (LNCS)
Proc. SIGMOD 2015 . ACM, 2077–2078.[51] Vladimir Vapnik and Alexey Chervonenkis. 1971. On the uniform convergenceof relative frequencies of events to their probabilities.
Theory Probab. Appl. igure 5: HyperBench web tool: available at http://hyperbench.dbai.tuwien.ac.at Appendix
In this appendix, we present further details of our analyses. First, wewill present the web tool to browse and discover the hypergraphsthat we have used. In addition, above all, we provide additionalfigures and tables to allow for a more fine-grained view on theCSP instances. Recall from Section 3 that we are dealing with 3classes of CSP instance here:
CSP Application and
CSP Random ,which are both taken from xcsp.org [9], and
CSP Other , which havealready been analysed w.r.t. hw in previous works [13, 28]. Dueto lack of space, the main body of the text contains only figuresand tables with aggregated values for all CSPs. Below, additionaldetails are provided for each figure and table from the main bodyby distinguishing the two classes of CQs and three classes of CSPs. A WEB TOOL
The hypergraphs in the benchmark and the results of the analysesof these hypergraphs can be accessed via a web tool, which isavailable at http://hyperbench.dbai.tuwien.ac.at. There we haveuploaded 3,070 hypergraphs together with over 5,518 HDs andthe output of over 16,585 further algorithm runs, where no HDof desired width was found (either because a lower bound on thewidth was established or the algorithm timed out). For example, inthe screenshot in Figure 5 the results for the CSP instance “Kakuro-easy-015-ext.xml.hg” are displayed. The hypertree width of theinstance is calculated according to the list of HDs at the bottom ofthe screenshot. For this instance we have several algorithm runs,some of which led to a HD, some did not (“HD not found”). Withthese we were able to pinpoint that hw = ghw = hw < BIP <
3, etc.) and to contribute tothe benchmark by uploading hypergraphs, which are then analysedand incorporated into our HyperBench benchmark.
B FURTHER DETAILS FOR SECTION 3
Table 7 presents the exact numbers used in Figure 1.
Vertices −
10 11 −
20 21 −
30 31 −
40 41 − > CQ Application
199 126 67 51 29 63
CQ Random
28 59 58 62 60 233
CSP Application
CSP Random
10 224 269 210 0 150
CSP Other
Edges −
10 11 −
20 21 −
30 31 −
40 41 − > CQ Application
462 47 11 1 6 8
CQ Random
75 96 106 106 117 0
CSP Application
CSP Random
20 55 50 110 0 628
CSP Other
Arities − −
10 11 −
15 16 − > CQ Application
315 64 70 41 45
CQ Random
102 144 160 94 0
CSP Application
667 263 1 56 103
CSP Random
594 244 25 0 0
CSP Other
74 7 1 0 0
Table 7: Hypergraph Sizes FURTHER DETAILS FOR SECTION 4
In Table 2, statistics on several hypergraph invariants were provided,namely degree, intersection width, c -multi-intersection width for c ∈ { , } , and VC-dimension. In Table 8 and Figure 6, additionaldetails are provided by distinguishing the three classes of CSPinstances.In Figure 2, the results of our hw -analysis were presented. In Table 9and Figure 7, additional details are provided by distinguishing thethree classes of CSP instances. As in Figure 2, we also write theaverage runtimes in the bars of Figure 7. In the tabular presentationin Table 9, this information is given by putting the number ofseconds in parentheses. CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 12345>5
Degree
CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 012345 >5
BIP
CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 012345>5
CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 012345>5
CSP OtherCSP RandomCSP ApplicationCQ RandomCQ Application 0% 25% 50% 75% 100% 012345
VC−Dimension
Figure 6: Hypergraph Properties Q Application i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 0 118 173 101 2 421 348 302 3932 176 85 59 50 1323 137 7 5 5 04 87 5 5 5 05 35 17 0 0 06 98 0 0 0 0
CQ Random i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 1 16 49 01 1 17 77 125 202 15 53 90 120 1333 38 62 103 74 2404 31 63 62 42 1065 33 71 47 28 16 382 233 105 62 0
CSP Application i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 0 596 597 01 0 1030 459 486 02 596 59 34 7 10643 1 0 1 0 264 1 0 0 0 05 2 0 0 0 0 > CSP Random i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 0 0 0 01 0 200 200 238 02 0 224 312 407 2203 0 76 147 95 5154 12 181 161 97 575 8 99 14 1 71 > CSP Other i Deg BIP 3-BMIP 4-BMIP VC-dim0 0 0 1 6 01 0 7 36 39 02 1 36 23 16 513 5 29 20 21 264 19 10 2 0 05 4 0 0 0 0 > Table 8: Hypergraph properties
CQ Application k yes no timeout1 454 (0) 81 (0) 02 73 (0) 8 (0) 03 8 (0) 0 0 CQ Random k yes no timeout1 36 (0) 464 (0) 02 68 (0) 396 (0) 03 70 (0) 326 (32) 04 59 (0) 167 (544) 1005 54 (0) 55 (610) 15810 206 (5) 0 715 7 (0) 0 0 CSP Application k yes no timeout1 0 1090 (0) 02 29 (0) 1061 (0) 03 116 (0) 802 (736) 1434 283 (18) 62 (707) 6005 231 (13) 0 43110 261 (0) 0 17015 12 (0) 0 15825 118 (0) 0 4050 40 (0) 0 0 CSP Random k yes no timeout1 0 863 (0) 02 47 (0) 816 (1) 03 111 (0) 602 (1319) 1034 39 (42) 160 (1332) 5065 136 (59) 0 53010 530 (0) 0 0 CSP Other k yes no timeout1 0 82 (1) 02 19 (0) 55 (219) 83 5 (0) 11 (1257) 474 5 (0) 2 (943) 515 6 (0) 1 (0) 4610 24 (0) 0 2315 6 (1) 0 1725 7 (10) 0 1050 5 (0) 0 575 4 (0) 0 1 Table 9: HW of instances with average runtime in s s0s 0s0s 0s k N o . o f i n s t a n ces yes no timeout CQ Application k N o . o f i n s t a n ces yes no timeout CQ Random
0s 0s0s 736s0s 707s18s 13s 0s 0s 0s 0s k N o . o f i n s t a n ces yes no timeout CSP Application
0s 1s0s 1319s0s 1332s42s 59s 0s k N o . o f i n s t a n ces yes no timeout CSP Random
1s 219s0s 1257s0s 943s0s 0s0s 0s 1s 10s 0s 0s k N o . o f i n s t a n ces yes no timeout CSP Other
Figure 7: HW analysis (labels are average runtimes in s) FURTHER DETAILS FOR SECTION 5
In Table 4, we gave an overview of the improvements of the widthwhen switching from hw to ghw . In Table 10 and Figure 8, additionaldetails are provided by distinguishing the three classes of CSPinstances. The runtimes are given in parentheses in the tabularrepresentation and in the bars of the bar chart, respectively. Thepseudocode of the GHD-algorithm via balanced separators is givenin Figure 9. CQ Applicatoinhw → ghw yes no timeout3 → CQ Randomhw → ghw yes no timeout3 → → → → CSP Applicationhw → ghw yes no timeout3 → → → → CSP Randomhw → ghw yes no timeout3 → → → → CSP Otherhw → ghw yes no timeout3 → → → → Table 10: GHW of instances with average runtime in s k N o . o f i n s t a n ces yes no timeout CQ Application
29s 65s 41s 655s k N o . o f i n s t a n ces yes no timeout CQ Random
7s 66s 9s 458s41s k N o . o f i n s t a n ces yes no timeout CSP Application
0s 0s 4s 47s221s k N o . o f i n s t a n ces yes no timeout CSP Random
14s 120s 2s2s k N o . o f i n s t a n ces yes no timeout CSP Other
Figure 8: GHW analysis (labels are average runtime in s) LGORITHM
Find_GHD_via_balancedSeparators// high-level sketch
Input: hypergraph H ′ , integer k ≥ Output: a GHD ⟨ T , B u , λ u ⟩ of width ≤ k if exists,“Reject”, otherwise. Procedure
Find_GHD ( H : Hypergraph, Sp : Set of Special-Edges) begin // 1. Stop if there are at most two special edges left: If E ( H ) = ∅ and | Sp | ≤ then return a GHD having a node for each s ∈ Sp with B u : = s and λ u : = { s } ;// 2. Find a balanced separator λ u for H : Guess a balanced separator λ u ⊆ E ( H ′ ) with | λ u | ≤ k for root u of a GHD of H such that: • B ( λ u ) (cid:60) Sp • B ( λ u ) ⊆ V ( H ) If no such balanced separator exists then return Reject; B u : = B ( λ u ) ;// 3. Split H into connected components C , . . . , C ℓ w.r.t. λ u : V u : = V ( H ) \ B u ; E u : = { e ∩ V u | e ∈ ( E ( H ) ∪ Sp )} ; Compute the connected components of ( V u , E u ) Let the connected components be denoted by C , . . . , C ℓ ;// 4. Build the pairs (cid:10) H i , Sp i (cid:11) for each connected component C i : For every i ∈ { , . . . , ℓ } do E i : = { e | e ∈ E ( H ) and e ∩ C i (cid:44) ∅} ; V i : = V ( E i ) ∪ B u ; Let H i be the hypergraph ( V i , E i ) ; Sp i : = { s ∈ Sp | s ∩ C i (cid:44) ∅} ∪ { B u } ; od ;// 5. Call Find_GHD( H i , Sp i ) for each pair (cid:10) H i , Sp i (cid:11) : For each i ∈ { , . . . , ℓ } T i : = Find_GHD ( H i , Sp i ) If ∃ i s.t. recursive call returns Reject then return Reject;// 6. Create and return a new GHD for H having B u and λ u as root: Create a new root u with B u and λ u ; Reroot all T i at the node t i where λ t i = { B u } ; Attach all children of t i to u ; return the new GHD rooted at u ; endbegin (* Main *) return Find_GHD ( H ′ , ∅ ); end Figure 9: Recursive GHD-algorithm via balanced separators FURTHER DETAILS FOR SECTION 6
In Table 5, we presented the achieved improvements of the widthby switching from integral covers of HDs to fractional covers. In Ta-bles 11 and 12 and Figures 10 and 11, additional details are providedby distinguishing the three classes of CSP instances and by showingthe results of the two algorithms SimpleImproveHD (in Table 11and Figure 10) and FracImproveHD (in Table 12 and Figure 11)separately.
CQ Applicationhw ≥ ≥ . ≥ . CQ Randomhw ≥ ≥ . ≥ . CSP Applicationhw ≥ ≥ . ≥ . CSP Randomhw ≥ ≥ . ≥ . CSP Otherhw ≥ ≥ . ≥ . Table 11: SimpleImproveHD of instances k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no CQ Application k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no CQ Random k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no CSP Application k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no CSP Random k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no CSP Other
Figure 10: SimpleImproveHD analysis Q Applicationhw ≥ ≥ . ≥ . CQ Randomhw ≥ ≥ . ≥ . CSP Applicationhw ≥ ≥ . ≥ . CSP Randomhw ≥ ≥ . ≥ . CSP Otherhw ≥ ≥ . ≥ . Table 12: FracImproveHD of instances k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no timeout CQ Application k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no timeout CQ Random k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no timeout CSP Application k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no timeout CSP Random k N o . o f i n s t a n ces >=1 >=0.5 >=0.1no timeout CSP Other