Materializing Knowledge Bases via Trigger Graphs
Efthymia Tsamoura, David Carral, Enrico Malizia, Jacopo Urbani
MMaterializing Knowledge Bases via Trigger Graphs(Technical Report)
Efthymia Tsamoura ∗ , David Carral † , Enrico Malizia ‡ , Jacopo Urbani (cid:5)∗ Samsung AI Research, United Kingdom; † TU Dresden, Germany; ‡ University of Bologna, Italy; (cid:5)
Vrije Universiteit Amsterdam, The Netherlands
ABSTRACT
The chase is a well-established family of algorithms usedto materialize Knowledge Bases (KBs), like KnowledgeGraphs (KGs), to tackle important tasks like query an-swering under dependencies or data cleaning. A generalproblem of chase algorithms is that they might performredundant computations. To counter this problem, we in-troduce the notion of
Trigger Graphs (TGs), which guidethe execution of the rules avoiding redundant computa-tions. We present the results of an extensive theoreticaland empirical study that seeks to answer when and howTGs can be computed and what are the benefits of TGswhen applied over real-world KBs. Our results includeintroducing algorithms that compute (minimal) TGs.We implemented our approach in a new engine, and ourexperiments show that it can be significantly more effi-cient than the chase enabling us to materialize KBs with17B facts in less than 40 min on commodity machines.
PVLDB Reference Format: . Materializing Knowledge Bases via Trigger Graphs.
PVLDB ,12(xxx): xxxx-yyyy, 2020.DOI: https://doi.org/10.14778/xxxxxxx.xxxxxxx
1. INTRODUCTION
Motivation.
Knowledge Bases (KBs) are becomingincreasingly important with many industrial key playersinvesting on this technology. For example, KnowledgeGraphs (KGs) [32] have emerged as the main vehiclefor representing factual knowledge on the Web and en-joy a widespread adoption [48]. Moreover, several techgiants are building KGs to support their core business.For instance, the KG developed at Microsoft containsinformation about the world and supports question an-swering, while, at Google, KGs are used to help Google
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. To view a copyof this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. Forany use beyond those covered by this license, obtain permission by [email protected]. Copyright is held by the owner/author(s). Publication rightslicensed to the VLDB Endowment.
Proceedings of the VLDB Endowment,
Vol. 12, No. xxxISSN 2150-8097.DOI: https://doi.org/10.14778/xxxxxxx.xxxxxxx products respond more appropriately to user requestsby mapping them to concepts in the KG. The use ofKBs and KGs in such scenarios is not restricted only todatabase-like analytics or query answering: KBs playalso a central role in neural-symbolic systems for efficientlearning and explainable AI [23, 36].A KB can be viewed as a classical database B withfactual knowledge and a set of logical rules P , called program , allowing the derivation of additional knowledge.One class of rules that is of particular interest both toacademia and to industry is Datalog [2]. Datalog is arecursive language with declarative semantics that allowsusers to succinctly write recursive graph queries. Be-yond expressing graph queries, e.g., reachability, Datalogallows richer fixed-point graph analytics via aggregatefunctions. LogicBlox and LinkedIn used Datalog todevelop high-performance applications, or to computeanalytics over its KG [3,46]. Google developed their ownDatalog engine called Yedalog [21]. Other industrialusers include Facebook, BP [10] and Samsung [40]. Materializing a KB (
P, B ) is the process of derivingall the facts that logically follow when reasoning over thedatabase B using the rules in P . Materialization is a coreoperation in KB management. An obvious use is that ofcaching the derived knowledge. A second use is that of goal-driven query answering , i.e., deriving the knowledgespecific to a given query only , using database techniquessuch as magic sets and subsumptive tabling [8, 9, 13, 55].Beyond knowledge exploration, other applications ofmaterialization are data wrangling [35], entity resolu-tion [37], data exchange [26] and query answering overOWL [44] and RDFS [16] ontologies. Finally, material-ization has been also used in probabilistic KBs [56]. Problem.
The increasing sizes of modern KBs [48], andthe fact that materialization is not a one-off operationwhen used for goal-driven query answering, make im-proving the materialization performance critical. The chase , which was introduced in 1979 by Maier et al. [42],has been the most popular materialization technique andhas been adopted by several commercial and open sourceengines such as VLog [58], RDFox [47] and Vadalog [10].To improve the performance of materialization, differ-ent approaches have focused on different inefficiencyaspects. One approach is to reduce the number of1 a r X i v : . [ c s . D B ] F e b acts added in the KB. This is the take of some ofthe chase variants proposed by the database and AIcommunities [11, 24, 49]. A second approach is to paral-lelize the computation. For example, RDFox proposesa parallelization technique for Datalog rules [47], whileWebPIE [59] and Inferray [54] propose parallelizationtechniques for fixed RDFS rules. Orthogonal to those ap-proaches are those employing compression and columnarstorage layouts to reduce memory consumption [34, 58].In this paper, we focus on a different aspect: that ofavoiding redundant computations. Redundant computa-tions is a problem that concerns all chase variants andhas multiple causes. A first cause is the derivation offacts that either have been derived in previous rounds, orare logically redundant, i.e., they can be ignored withoutcompromising query answering. The above issue hasbeen partially addressed in Datalog with the well-knownsemina¨ıve evaluation (SNE) [2]. SNE restricts the exe-cution of the rules over at least one new fact. However,it cannot block the derivation of the same or logicallyredundant facts by different rules. A second cause ofredundant computations relates to the execution of therules: when executing a rule, the chase may considerfacts that cannot lead to any derivations. Our approach.
To reduce the amount of redundantcomputations, we introduce the notion of
Trigger Graphs(TGs) . A TG is an acyclic directed graph that capturesall the operations that should be performed to material-ize a KB (
P, B ). Each node in a TG is associated witha rule from P and with a set of facts, while the edgesspecify the facts over which we execute each rule.Intuitively, a TG can be viewed as a blueprint forreasoning over the KB. As such, we can use it to “guide”a reasoning procedure without resorting to an exhaustiveexecution of the rules, as it is done with the chase. Inparticular, our approach consists of traversing the TG,executing the rule r associated with a node v over theunion of the facts associated with the parent nodes of v and storing the derived facts “inside” v . After thetraversal is complete, then the materialization of the KBis simply the union of the facts in all the nodes.TG-guided materialization addresses at the same timeall causes of inefficiencies described above. In particu-lar, TGs block the derivation of the same or logicallyredundant facts that cannot be blocked by SNE. Thisis achieved by effectively partitioning into smaller sub-instances the facts currently in the KB. This partitioningalso enables us to reduce the cost of executing the rules.Furthermore, in specific cases, TGs allow us reasoningvia either completely avoiding certain steps involvedwhen executing rules, or performing them at the end andcollectively for all rules. Our experiments show that weget good runtime improvements with both alternatives. Contributions.
We propose techniques for computingboth instance-independent and instance-dependent TGs.The former TGs are computed exclusively based on therules of the KB and allow us to reason over any possibleinstance of the KB making them particularly useful when the database changes frequently. In contrast, instance-dependent TGs are computed based both on the rulesand the data of the KB and, thus, support reasoning overthe given KB only . We show that not every programadmits a finite instance-independent TG. We define aspecial class, called
FTG , including all programs thatadmit a finite instance-independent TG and explore itsrelationship with other known classes.As a second contribution, we propose algorithms tocompute and minimize (instance-independent) TGs forlinear programs: a class of programs relevant in practice.A program P not admitting a finite instance-independent TG may still admit a finite instance-dependent
TG.As a third contribution, we show that all programsthat admit a finite universal model also admit a finite instance-dependent
TG. We use this finding to propose aTG-guided materialization technique that supports any such program (not necessarily in
FTG ). The techniqueworks by interleaving the reasoning process with thecomputation of the TG, and it reduces the number ofredundant computations via query containment and viaa novel TG-based rule execution strategy.We implemented our approach in a new reasoner,called GLog, and compared its performance versus mul-tiple state-of-the-art chase and RDFS engines includingRDFox, VLog, WebPIE [59] and Inferray [54], usingwell-established benchmarks, e.g., ChaseBench [11]. Ourevaluation shows that GLog outperforms all its com-petitors in all benchmarks. Moreover, in our largestexperiment, GLog was able to materialize a KB with17B facts in 37 minutes on commodity hardware.
Summary.
We make the following contributions: • ( New idea ) We propose a new reasoning techniquebased on traversing acyclic graphs, called TGs, totackle multiple sources of inefficiency of the chase; • ( New theoretical contribution ) We study the classof programs admitting finite instance-independentTGs and its relationship with other known classes; • ( New algorithms ) We propose techniques for com-puting minimal instance-independent TGs for lin-ear programs, and techniques for computing mini-mal instance-dependent TGs for Datalog programs; • ( New system ) We introduce a new reasoner, GLog,which has competitive performance, often superiorto the state-of-the-art, and has good scalability.Supplementary material with all proofs, code and eval-uation data is in https://bitbucket.org/tsamoura/trigger-graphs/src/master/ .
2. MOTIVATING EXAMPLE
We start our discussion with a simple example todescribe how the chase works, its inefficiencies, and howthey can be overcome with TGs. For the moment, wegive only an intuitive description of some key concepts toaid the understanding of the main ideas. In the followingsections, we will provide a formal description.The chase works in rounds during which it executesthe rules over the facts that are currently in the KB.2 irst roundInput Second round Third round T ( c ,c ,c ) T ( c ,c ,n ) r ( c ,c ) R ( c ,c ) T ( c ,c ,c ) T ( c ,c ,n ) r ( c ,c ) R ( c ,c ) T ( c ,c ,n ) r ( c ,c ) R ( c ,c ) r ( c ,c ) r r r r r r r r r r r r T ( c ,c ,n ) R ( c ,c ) T ( c ,c ,c ) R ( c ,c ) T ( c ,c ,c ) (a)(b) (c) u (created by r ) u (created by r ) u (created by r ) u (created by r ) u (created by r ) Figure 1: (a) Chase execution for Example 1, (b) the TG G ,(c) the TG G . In (b) and (c), the facts shown inside thenodes are the results of reasoning over B using the TG. In most chase variants, the execution of a rule involvesthree steps: retrieving all the facts that instantiate thepremise of the rule, then, checking whether the facts tobe derived logically hold in the KB and finally, addingthem to the KB if they do.
Example Consider the KB B = { r ( c , c ) } witha single fact and the program P = { r , r , r , r } : r ( X, Y ) → R ( X, Y ) ( r ) R ( X, Y ) → T ( Y, X, Y ) ( r ) T ( Y, X, Y ) → R ( X, Y ) ( r ) r ( X, Y ) → ∃ Z.T ( Y, X, Z ) ( r ) Figure 1 (a) depicts the rounds of the chase with such aninput. In the first round, the only rules that can derivefacts are r and r . Rule r derives the fact R ( c , c ) .Since this fact is not in the KB, the chase adds it to theKB. Let us now focus on r . Notice that variable Z in r does not occur in the premise of r . The chase dealswith such variables by introducing fresh null (values) .Nulls can be seen as “placeholders” for objects that arenot known. In our case, r derives the fact T ( c , c , n ) ,where n is a null, and the chase adds it to the KB.The chase then continues to the second round whererules are executed over B (cid:48) = B ∪{ R ( c , c ) , T ( c , c , n ) } .The execution of r derives the fact T ( c , c , c ) , whichis added to the KB, yielding B (cid:48)(cid:48) = B (cid:48) ∪ { T ( c , c , c ) } .Finally, the chase proceeds to the third round where onlyrule r derives R ( c , c ) from B (cid:48)(cid:48) . However, since thisfact is already in B (cid:48)(cid:48) , the chase stops. The above steps expose two inefficiencies of the chase.The first is that of incurring in the cost of deriving thesame or logically redundant facts.
Example Let us return back to Example 1. Thechase pays the cost of executing r despite that r ’s exe-cution always derives facts derived in previous rounds.This is due to the cyclic dependency between rules r and r : r derives T -facts by flipping the arguments of the R -facts, while r derives R -facts by flipping the argumentsof the T -facts. Despite that the SNE effectively blocksthe execution of r and r in the third chase round, itcannot block the execution of r in the third chase round,since T ( c , c , c ) was derived in the second round. Now, consider the fact T ( c , c , n ) . This fact is logi-cally redundant, because it provides no extra informationover T ( c , c , c ) , derived by r . Despite being logicallyredundant, the chase pays the cost of deriving it. The second inefficiency that is exposed is that of subop-timally executing the rules themselves: when computingthe facts instantiating the premise of a rule, the chaseconsiders all facts currently in the KB even the ones thatcannot instantiate the premise of the rule.
Example Continuing with Example 1, considerthe execution of r in the second round of the chase.No fact derived by r can instantiate the premise of r ,since the premise of r requires the first and the thirdarguments of the T -facts to be the same. Hence, the costpaid for executing r over those facts is unnecessary. The root of these inefficiencies is that the chase, ineach round, considers the entire KB as a source forpotential derivations, with only SNE as means to avoidsome redundant derivations. If we were able to “guide”the execution of the rules in a more clever way, then wecan avoid the inefficiencies stated above.For instance, consider an alternative execution strat-egy where r is executed only over the derivations of r ,while r and r are not executed at all. This strategywould not face any of the inefficiencies highlighted above,and it can be defined with a graph like the one in Figure 1(c). Informally, a Trigger Graph (TG) is precisely sucha graph-based blueprint to compute the materialization.In the remaining, we will first provide a formal definitionof TGs and study their properties. Then, we will showthat in some cases we can build a TG that is optimal forany possible set of facts given as input. In other cases,we can still build TGs incrementally. Such TGs allow toavoid redundant computations that will occur with thechase but only with the given input.
3. PRELIMINARIES
Let
Consts , Nulls , Vars , and
Preds be mutually disjoint,(countably infinite) sets of constants , nulls , variables , and predicates , respectively. Each predicate p is associatedwith a non-negative integer arity ( p ) ≥
0, called the arity of p . Let EDP and
IDP be disjoint subsets of
Preds of intensional and extensional predicates , respectively. A term is a constant, a null, or a variable. A term is groundif it is either a constant or a null. An atom A has theform p ( t , . . . , t n ), where p is an n -ary predicate, and t , . . . , t n are terms. An atom A is extensional (resp.,intensional), if the predicate of A is in EDP (resp.,
IDP ).A fact is an atom of ground terms. A base fact is anatom of constants whose predicate is extensional. An instance I is a set of facts (possibly comprising nullterms). A base instance B is a set of base facts.A rule is a first-order formula of the form ∀ X ∀ Y (cid:94) ni =1 P i ( X i , Y i ) → ∃ Z P ( Y , Z ) , (1)where, P is an intensional predicate and for all 1 ≤ i ≤ n , X i ⊆ X and Y ⊆ Y ( X i and Y i might be empty). We3ssume w.l.o.g. that the body of a rule includes onlyextensional predicates or intensional predicates. Wewill denote extensional predicates with lowercase let-ters, while intensional predicates with uppercase letters.Quantifiers are commonly omitted. The left-hand andthe right-hand side of a rule r are its body and head ,respectively, and are denoted by body ( r ) and head ( r ). Arule is Datalog if it has no existentially quantified vari-ables, extensional if body ( r ) includes only extensionalatoms, and linear if it has a single atom in its body.A program is a set of rules. A knowledge base (KB) isa pair ( P, B ) with P a program and B a base instance.Symbol | = denotes logical entailment, where sets ofatoms and rules are viewed as first-order theories. Sym-bol ≡ denotes logical equivalence, i.e., logical entailmentin both directions.A term mapping σ is a (possibly partial) mapping ofterms to terms; we write σ = { t (cid:55)→ s , . . . , t n (cid:55)→ s n } todenote that σ ( t i ) = s i for 1 ≤ i ≤ n . Let α be a term,an atom, a conjunction of atoms, or a set of atoms. Then σ ( α ) is obtained by replacing each occurrence of a term t in α that also occurs in the domain of σ with σ ( t ) (i.e.,terms outside the domain of σ remain unchanged). A substitution is a term mapping whose domain containsonly variables and whose range contains only groundterms. For two sets, or conjunctions, of atoms A and A , a term mapping σ from the terms occurring in A tothe terms occurring in A is said to be a homomorphism from A to A if the following hold: (i) σ maps eachconstant in its domain to itself, (ii) σ maps each nullin its domain to Consts ∪ Nulls and (iii) for each atom A ∈ A , σ ( A ) ∈ A . We denote a homomorphism σ from A into A by σ : A → A .It is known that, for two sets of facts A and A , thereexists a homomorphism from A into A iff A | = A (and hence, there exists a homomorphism in both waysiff A ≡ A ). When A and A are null-free instances, A | = A iff A ⊆ A and A ≡ A iff A = A .For a set of two or more atoms A = { A , . . . , A n } a most general unifier (MGU) µ for A is a substitutionso that: (i) µ ( A ) = · · · = µ ( A n ); and (ii) for each othersubstitution σ for which σ ( A ) = · · · = σ ( A n ), there ex-ists a σ (cid:48) such that σ = σ (cid:48) ◦ µ [5].Consider a rule r of the form (1) and an instance I . A trigger for r in I is a homomorphism from thebody of r into I . We denote by h s the extension of atrigger h mapping each Z ∈ Z into a unique fresh null.A rule r holds or is satisfied in an instance I , if for eachtrigger h for r in I , there exists an extension h (cid:48) of h to ahomomorphism from the head of r into I . A model of aKB ( P, B ) is a set I ⊇ B , such that each r ∈ P holds in I . A KB may admit infinitely many different models. Amodel M is universal , if there exists a homomorphismfrom M into every other model of ( P, B ). A program P is Finite Expansion Set ( FES ), if for each base instance B , ( P, B ) admits a finite universal model.A conjunctive query (CQ) is a formula of the form Q ( X , . . . , X n ) ← (cid:86) mi =1 A i , where Q is a fresh predicatenot occurring in P , A i are null-free atoms and each X j occurs in some A i atom. We usually refer to a CQ byits head predicate. We refer to the left-hand and theright-hand side of the formula as the head and the body of the query, respectively. A CQ is atomic if its bodyconsists of a single atom. A Boolean CQ (BCQ) is a CQwhose head predicate has no arguments. A substitution σ is an answer to Q on an instance I if the domain of σ is precisely its head variables, and if σ can be extendedto a homomorphism from (cid:86) i A i into I . We often identify σ with the n -tuple ( σ ( X ) , . . . , σ ( X n )). The output of Q on I is the set Q ( I ) of all answers to Q on I . Theanswer to a BCQ Q on an instance I is true, denotedas I | = Q , if there exists a homomorphism from (cid:86) i =1 A i into I . The answer to a BCQ Q on a KB ( P, B ) is true,denoted as (
P, B ) | = Q , if M | = Q holds, for each model M of ( P, B ). Finally, a CQ Q is contained in a CQ Q ,denoted as Q ⊆ Q , if for each instance I , each answerto Q on I is in the answers to Q on I [20].The chase refers to a family of techniques for repairinga base instance B relative to a set of rules P so that theresult satisfies the rules in P and contains all base factsfrom B . In particular, the result is a universal model of( P, B ), which we can use for query answering [26]. By“chase” we refer both to the procedure and its output.The chase works in rounds during which it executesone or more rules from the KB. The result of each round i ≥ I i (with I = B ), which includesthe facts of all previous instances plus the newly derivedfacts. The execution of a rule in the i -th chase round,involves computing all triggers from the body of r into I i − , then (potentially) checking whether the facts tobe derived satisfy certain criteria in the KB and finally,adding to the KB or discarding the derived facts. Differ-ent chase variants employ different criteria for decidingwhether a fact should be added to the KB or whetherto stop or continue the reasoning process [11, 49]. Forexample, the restricted chase (adopted by VLog andRDFox) adds a fact if there exists no homomorphismfrom this fact into the KB and terminates when no newfact is added. The warded chase (adopted by Vadalog)replaces homomorphism checks by isomorphism ones [10]and terminates, again, when no new fact is added. Theequivalent chase omits any checks and terminates whenthere is a round i which produces an instance that is logi-cally equivalent to the instance produced in the ( i − Ch i ( K ) or Ch i ( P, B ) todenote the instance computed during the i -th chaseround and Ch ( P, B ) to denote the (possibly infinite)result of the chase. Furthermore, we define the chasegraph chaseGraph ( P, B ) for a KB (
P, B ) as the edge-labeled directed acyclic graph having as nodes the factsin Ch ( P, B ) and having an edge from a node f to f r ∈ P if f is obtained from f andpossibly from other facts by executing r .
4. TRIGGER GRAPHS
In this section, we formally define Trigger Graphs(TGs) and study the class of programs admitting finite instance-independent
TGs. First, we introduce the no-tion of
Execution Graphs (EGs). Intuitively, an EG fora program is a digraph stating a “plan” of rule execu-tion to reason via the program. In its general definition,an EG is not required to characterize a plan of reason-ing guaranteeing completeness. Particular EGs, definedlater, will also satisfy this property.
Definition An execution graph (EG) for a pro-gram P is an acyclic, node- and edge-labelled digraphG = ( V, E, rule , (cid:96) ) , where V and E are the graph nodesand edges sets, respectively, and rule and (cid:96) are the node-and edge-labelling functions. Each node v (i) is labelledwith some rule, denoted by rule ( v ) , from P ; and (ii) ifthe j -th predicate in the body of rule ( v ) equals the headpredicate of rule ( u ) for some node u , then there is anedge labelled j from node u to node v , denoted by u → j v . Figures 1(b) and 1(c) show two EGs for P from Ex-ample 1. Next to each node is the associated rule. Laterwe show that both EGs are also TGs for P .Since the nodes of an execution graph are associatedwith rules of a program, when, in the following, we referto the head and the body of a node v , we actually meanthe head and the body of rule ( v ). Observe that, bydefinition, nodes associated with extensional rules donot have entering edges, and nodes v associated withan intensional rule have at most one incoming edgeassociated with the j -th predicate of the body of v , i.e.,there is at most one node u such that u → j v . Thelatter might seem counterintuitive as, in a program, the j -th predicate in the body of a rule can appear in theheads of many different rules. It is precisely to takeinto account this possibility that, in an execution graph,more than one node can be associated with the same rule r of the program. In this way, different nodes v , . . . , v q associated with the same rule r can be linked with anedge labeled j to different nodes u , . . . , u q whose head’spredicate is the j -th predicate of the body of r . Thismodels that to evaluate a rule r we might need to matchthe j -th predicate in the body of r with facts generatedby the heads of different rules.We now define some notions on EGs that we will usethroughout the paper. For an EG G for a program P ,we denote by ν ( G ) and (cid:15) ( G ) the sets of nodes and edgesin G . The depth of a node v ∈ ν ( G ) is the length of thelongest path that ends in v . The depth d ( G ) of G is 0if G is the empty graph; otherwise, it is the maximumdepth of the nodes in ν ( V ).As said earlier, EGs can be used to guide the reasoningprocess. In the following definition, we formalise howthe reasoning over a program P is carried out by follow-ing the plan encoded in an EG for P . The definition assumes the following for each rule r in P : (i) r is ofthe form ∀ X ∀ Y (cid:86) ni =1 P i ( X i , Y i ) → ∃ Z P ( Y , Z ); and (ii)if r is intensional and is associated with a node v in anEG for P , then the EG includes an edge of the form u i → i v , for each 1 ≤ i ≤ n . Definition Let ( P, B ) be a KB, G be an EG for P and v be a node in G associated with rule r ∈ P . v ( B ) includes a fact h s ( head ( r )) , for each h that is either: • a homomorphism from the body of r to B , if r isextensional; or otherwise • a homomorphism from the body of r into (cid:83) ni =1 u i ( B ) so that the following holds: the restriction of h over X i ∪ Y i is a homomorphism from P i ( X i , Y i ) into u i ( B ) , for each ≤ i ≤ n .We pose G ( B ) = B ∪ (cid:83) v ∈ V v ( B ) . TGs are EGs guaranteeing the correct computationof conjunctive query answering.
Definition An EG G is a TG for ( P, B ) , if foreach BCQ Q , ( P, B ) | = Q iff G ( B ) | = Q . G is a TG for P , if for each base instance B , G is a TG for ( P, B ) . TGs that depend both on P and B are called instance-dependent , while TGs that depend only on P are called instance-independent . The EGs shown in Figure 1 areboth instance-independent TGs for P .We provide an analysis of the class of programs thatadmit a finite instance-independent TG denoted as FTG .Theorem 7 summarizes the relationship between
FTG and the classes of programs that are bounded (
BDD ,[24]), term-depth bounded (
TDB , [39]) and first-order-rewritable (
FOR , [18]).
Theorem The following hold: P is FTG iff it is
BDD ; and P is TDB ∩ FOR iff it is
BDD . This result is obtained by showing that if P is FTG ,then it is
BDD with bound the maximal depth of anyinstance-independent TG for P . If it is BDD with bound k , then the (finite) EG G k , which is described afterDefinition 9, is a TG for P .If a program is FOR , then all facts that contain termsof depth at most k are produced in a fixed number ofchase steps. Therefore, if it is also TDB , then all relevantfacts in the chase are also produced in a fixed numberof steps. Finally, the undecidability of
FTG followsfrom the fact that
FOR and
FTG coincide for Datalogprograms, which are always
TDB . See the appendix fora detailed explanation.We conclude our analysis by showing that any KBthat admits a finite model, also admits a finite instance-dependent TG, as stated in the following statement.
Theorem For each KB ( P, B ) that admits a finitemodel, there exists an instance-dependent TG. The key insight is that we can build a TG that mimicsthe chase. Below, we analyze the conditions under which5he same rule execution takes place both in the chaseand when reasoning over a TG. Based on this analysis wepresent a technique for computing instance-dependentTGs that mimic breadth-first chase variants.Consider a rule of the form (1) and assume that thechase over a KB (
P, B ) executes r in some round k byinstantiating its body using the facts R ( c i ). Considernow a TG G for ( P, B ). If k = 1, then this rule execution(notice that the rule has to be extensional) takes placein G if there is a node v associated with r . Otherwise,if k >
1, then this rule execution takes place in G if thefollowing holds: (i) there is a node v associated with r ,(ii) each R ( c i ) is stored in some node u i and (iii) there isan incoming edge u i → i v , for each 1 ≤ i ≤ n . We referto each combination of nodes of depth < k whose factsmay instantiate the body of a rule r when reasoning overan EG, as k -compatible nodes for r : Definition Let P be a program, r be an inten-sional rule in P and G be an EG for P . A combinationof n (not-necessarily distinct) nodes ( u , . . . , u n ) from G is k -compatible with r , where k ≥ is an integer, if: • the predicate in the head of u i is R i ; • the depth of each u i is less than k ; and • at least one node in ( u , . . . , u n ) is of depth k − . The above ideas are summarized in an iterative proce-dure, which builds at each step k a graph G k : • ( Base step ) if k = 1, then for each extensionalrule r add to G k a node v associated with r . • ( Inductive step ) otherwise, for each intensionalrule r and each combination of nodes ( u , . . . , u n )from G k − that is k -compatible with r , add to G k :(i) a fresh node v associated with r and (ii) an edge u i → i v , for each 1 ≤ i ≤ n .The inductive step ensures that G k encodes each ruleexecution that takes place in the k -th chase round.So far, we did not specify when the TG computa-tion process stops. When P is Datalog, we can stopwhen G k − ( B ) = G k ( B ). Otherwise, we can employthe termination criterion of the equivalent chase, e.g., G k − ( B ) | = G k ( B ), or of the restricted chase.
5. TGS FOR LINEAR PROGRAMS
In the previous section, we outlined a procedure tocompute instance-dependent TGs that mimics the chase.Now, we propose an algorithm for computing instance-independent
TGs for linear programs.Our technique is based on two ideas. The first oneis that, for each base instance B , the result of chasing B using a linear program P is logically equivalent tothe union of the instances computed when chasing eachsingle fact in B using P .The second idea is based on pattern-isomorphic facts:facts with the same predicate name and for which thereis a bijection between their constants. For example, R (1 , ,
3) is pattern-isomorphic to R (5 , ,
7) but notto R (9 , , Algorithm 1 tglinear ( P )1: Let G be an empty EG for each f ∈ H ( P ) do Γ is an empty EG ; µ is the empty mapping for each f → r f ∈ chaseGraph ( P, { f } ) do add a fresh node u to ν (Γ) with rule ( u ) ·· = r µ ( u ) ·· = f → r f for each v, u ∈ ν (Γ) do if µ ( v ) = f → r f and µ ( u ) = f → r (cid:48) f then add v → u to (cid:15) (Γ) G ·· = G ∪ Γ return G in the same order during chasing. We denote by H ( P )a set of facts formed over the extensional predicatesin a program P , where no fact f ∈ H ( P ) is patternisomorphic to some other fact f ∈ H ( P ).Algorithm 1 combines these two ideas: it runs thechase for each fact in H ( P ) then tracks the rule execu-tions and (iii) based on these rule executions it computesa TG. In particular, for each fact f that is derived afterexecuting a rule r over f , Algorithm 1 will create afresh node u and associate it with rule r , lines 4–6. Themapping µ associates nodes with rule executions. Then,the algorithm adds edges between the nodes based onthe sequences of rule executions that took place duringchasing, lines 7–9.Algorithm 1 is (implicitly) parameterized by the chasevariant. The results below are based on the equivalentchase, as it ensures termination for FES programs.
Theorem
For any linear program P that is FES , tglinear ( P ) is a TG for P . Algorithm 1 has a double-exponential overhead.
Theorem
The execution time of Algorithm 1 for
FES programs is double exponential in the input program P . If the arity of the predicates in P is bounded, theexecution time is (single) exponential. The TGs computed by Algorithm 1 may comprisenodes which can be deleted without compromising queryanswering. Let us return to Example 1 and to the TG G from Figure 1: we can safely ignore the facts associatedwith the node u from G and still preserve the answersto all queries over ( P , B ). In this section, we show atechnique for minimizing TGs for linear programs.Our minimization algorithm is based on the following.Consider a TG G for a linear program P , a base instance B of P and the query Q ( X ) ← R ( X, Y ) ∧ S ( Y, Z, Z ).Assume that there exists a homomorphism from thebody of the query into the facts f = R ( c , n ) and f = S ( n , n , n ) and that f ∈ v ( B ) and f ∈ u ( B ) with v, u being two nodes of G . Since n is shared among twodifferent facts associated with two different nodes, it issafe to remove u if there is another node u (cid:48) ∈ ν ( G ) whoseinstance u (cid:48) ( B ) includes a fact of the form S ( n , n (cid:48) , n (cid:48) ).6quivalently, it is safe to remove u if there exists a ho-momorphism from u ( B ) into u (cid:48) ( B ) that maps to itselfeach null occurring both in u ( B ) and u (cid:48) ( B ). Since a nullcan occur both in u ( B ) and in u (cid:48) ( B ) if u, u (cid:48) share a com-mon ancestor we can rephrase the previous statement asfollows: we can remove u ( B ) if there exists a homomor-phism from u ( B ) into u (cid:48) ( B ) preserving each null (from u ( B )) that also occurs in some w ( B ) with w being anancestor of u in G . We refer to such homomorphisms as preserving homomorphisms : Definition
Let G be a TG for a program P , u, v ∈ ν ( G ) and B be a base instance. A homomor-phism from u ( B ) into v ( B ) is preserving , if it maps toitself each null occurring in some u (cid:48) ( B ) with u (cid:48) being anancestor of u . It suffices to consider only the facts in H ( P ) to verifythe existence of preserving homomorphisms. Lemma
Let P be a linear program, G be an EGfor P and u, v ∈ ν ( G ) . Then, there exists a preservinghomomorphism from u ( B ) into v ( B ) for each base in-stance B , iff there exists a preserving homomorphismfrom u ( { f } ) into v ( { f } ) , for each fact f ∈ H ( P ) . From Definition 12 and from Lemma 13 it follows thata node v of a TG can be “ignored” for query answeringif there exists a node v (cid:48) and a preserving homomorphismfrom v ( { f } ) into v (cid:48) ( { f } ), for each f ∈ H ( P ). If theabove holds, then we say that v is dominated by v (cid:48) . Theabove implies a strategy to reduce the size of TGs. Definition
For a TG G for a linear program P ,the EG minLinear ( G ) is obtained by exhaustively applyingthe steps: (i) choose a pair of nodes v, v (cid:48) from G where v is dominated by v (cid:48) , (ii) remove v from ν ( G ) ; and (iii)add an edge v (cid:48) → u , for each edge v → u from (cid:15) ( G ) . The minimization procedure described in Definition 14is correct: given a TG for a linear program P , the outputof minLinear is still a TG for P . Theorem
For a TG G for a linear program P , minLinear ( G ) is a TG for P . We present an example demonstrating the TG com-putation and minimizes techniques described above.
Example
Recall Example 1. Since r is the onlyextensional predicate in P , H ( P ) will include two facts,say r ( c , c ) and r ( c , c ) , where c , c and c are con-stants. Algorithm 1 computes a TG by tracking therule executions that take place when chasing each factin H ( P ) . For example, when considering r ( c , c ) , thegraph Γ computed in lines 3–9 will be the TG G fromFigure 1(b), where nodes are denoted as u , u , and u .Let us now focus on the minimization algorithm. Tominimize G , we need to identify nodes that are domi-nated by others. Recall that a node u in G is dominated by a node v , if for each f in H ( P ) , there exists a pre-serving homomorphism from u ( { f } ) into v ( { f } ) . Basedon the above, we can see that u is dominated by u . Forexample, when B ∗ = { r ( c , c ) } , there exists a preserv-ing homomorphism from u ( B ∗ ) = { R ( c , c , n ) } into u ( B ∗ ) = { R ( c , c , c ) } mapping n to c . Since u isdominated by u , the minimization process eliminates u from G . The result is the TG G from Figure 1(c),since no other node in G is dominated.
6. OPTIMIZING TGS FOR DATALOG
There are cases where we cannot compute instance-independent TG, e.g., for Datalog programs that are notalso in
FTG class. In such cases, we can still create aninstance-dependent TG using the procedure outlined inSection 4. In this section, we present two optimizationsto this procedure which avoid redundant computations.These optimizations work with Datalog programs; thusalso with non-linear rules.
Our first technique is based on a simple observation.Consider a node v of a TG G . Assume that v is asso-ciated with the rule a ( X, Y, Z ) → A ( Y, X ) with a beingextensional. We can see that for each base instance B and each fact a ( σ ( X ) , σ ( Y ) , σ ( Z )) in B , where σ is avariable substitution, the fact A ( σ ( Y ) , σ ( X )) is in v ( B ).Equivalently, for each answer σ to Q ( Y, X ) ← a ( X, Y, Z ),a fact A ( σ ( Y ) , σ ( X )) is associated with v ( B ). The abovecan be generalized. Consider a node v of a TG G suchthat rule ( v ) is (cid:86) ni = i A i ( Y i ) → A ( X ). The facts in v ( B )can be obtained by (i) computing the rewriting of thequery Q ( X ) ← (cid:86) ni = i A i ( Y i ) w.r.t. the rules in the ances-tors of v up to the extensional predicates; (ii) evaluatingthe rewritten query over B ; and (iii) adding A ( t ) to v ( B ), for each answer t to the rewritten query over B –recall that we denote answers either as substitutions or astuples, cf. Section 3. We refer to Q ( X ) ← (cid:86) ni = i A i ( Y i )as the characteristic query of v .This observation suggests we can use query contain-ment tests to identify nodes that can be safely removedfrom TGs (and EGs). Intuitively, the na¨ıve algorithmabove can be modified so that, at each step i , right aftercomputing G i , and before computing G i ( B ), we elimi-nate each node u if the EG-guided rewriting over of thecharacteristic query of u is contained in the EG-guidedrewriting of the characteristic query of another node v .Below, we formalize the notion of EG-rewritings, thenwe show the correspondence between the answers to EG-rewritings and the facts associated with the nodes, andwe finish with an algorithm eliminating nodes from TGs. Definition
Let v be a node in an EG G for aDatalog program. Let rule ( v ) be (cid:86) ni =1 A i → R ( Y ) . TheEG-rewriting of v , denoted as rew ( v ) , is the CQ com-puted as follows (w.l.o.g. no pair of rules rule ( u ) and rule ( v ) with u, v ∈ ν ( G ) and u (cid:54) = v shares variables): • form Q ( Y ) ← R ( Y ) ; associate R ( Y ) with v ; repeat the following rewriting step until no inten-sional atom is left in body ( Q ) : (i) choose an inten-sional atom α ∈ body ( Q ) ; (ii) compute the MGU θ of { head ( u ) , α } , where u is the node associatedwith α ; (iii) replace α in body ( Q ) with body ( u ) andapply θ on the resulting Q ; (iv) associate θ ( B j ) in body ( Q ) with the node w j , where B j is the j -thatom in body ( u ) and w j → j u ∈ (cid:15) ( G ) . The rewriting algorithm described in Definition 17 is avariant of the rewriting algorithm in [29]. Our differencefrom [29] is that at each step of the rewriting process,we consider only the rule rule ( u ) with u being the nodewith which α is associated with.There is a correspondence between the answers to thenodes’ EG-rewritings with the facts stored in the nodes. Lemma
Let G be an EG for a Datalog program P and B be a base instance of P . Then for each v ∈ ν ( G ) we have: v ( B ) includes exactly a fact A ( t ) with A beingthe head predicate of rule ( v ) , for each answer t to theEG-rewriting of v on B . Our algorithm for removing nodes from EGs is below.
Definition
The EG minDatalog ( G ) is obtainedfrom an EG G for a program P by exhaustively applyingthese steps: for each pair of nodes u and v such that(i) the depth of v is equal or larger than that of u , (ii)the predicates of head ( rule ( v )) and of head ( rule ( u )) arethe same and (iii) the EG-rewriting of v is containedin the EG-rewriting of u : (a) remove the node v from ν ( G ) , and (b) add an edge u → j w , for each edge v → j w occurring in G . The minimization technique of Definition 19 can beproven sound and to produce a TG with fewest nodes.
Theorem If G is a TG for a Datalog program P ,then minDatalog ( G ) is a minimum size TG for P . Deciding whether a TG of a Datalog program is ofminimum size can be proven co-NP-complete. The prob-lem’s hardness lies is the necessity of performing querycontainment tests, carried out via homomorphism tests,which require exponential time on deterministic machines(unless P = NP ) [20]. This hardness result supports theoptimality of minDatalog in terms of complexity. Theorem
For a Datalog program P and a TG G for P , deciding whether G is a TG of minimum size for P is co-NP-complete. EG-rewritings can be further used to optimize theexecution of the rules, as shown in the example below.
Example
Consider the program P a ( X ) ∧ b ( X ) → A ( X ) ( r ) a (cid:48) ( X ) ∧ b (cid:48) ( X ) → A ( X ) ( r ) where a , a (cid:48) , b and b (cid:48) are extensional predicates. Wedenote by a , a (cid:48) , b and b (cid:48) the relations storing the tuples c c b b c c a a c c c c d ⋈ = ⋈ = ⟕ == ⋈ c c de e =a b A a' b' a' ⋈ b' a' ⊳ A (i) (ii) (iii)(v) (vi)c c d a' ⋈ b' c c d c c Ac c b b c c a a c c ⋈ =a b A (iv) c c da' c c A a' ⊳ A d c c de e b' dd d ⊳ Figure 2: Different strategies for executing the rules from P . of the corresponding predicates in the input instance. Thedata of each relation are shown in Figure 2.The upper part of Figure 2 shows the steps involvedwhen executing r and r using the chase: (i) showsthe joins involved when executing r ; (ii)–(iii) show thejoins involved when executing r : (ii) shows the join tocompute body ( r ) while (iii) shows the outer join involvedwhen checking whether the conclusions of r have beenpreviously derived. Assuming that the cost of executingeach join is the cost of scanning the smallest relation,the total cost of the chase is: 100 (step (i)) + 51 (step(ii)) + 50 (step (iii))=201.The lower part of Figure 2 shows a more efficientstrategy. The execution of r stays the same (step (iv)),while for r we first compute all tuples that are in a (cid:48) butnot in A (step (v)) and use a (cid:48) \ A to restrict the tuplesinstantiating the body of r (step (vi)). The intuition isthat the tuples of a (cid:48) that are already in A will be discarded,so it is not worth considering them when instantiatingthe body of r . The total cost of this strategy is: 100(step (iv)) + 51 (step (v)) + 1 (step (vi))=152. Example 22 suggests a way to optimize the executionof the rules, which reduces the cost of instantiating therule bodies. This is achieved by considering only the in-stantiations leading to the derivation of new conclusions.Our new rule execution strategy is described below.
Definition
Let v be a node of an EG G for a Dat-alog program P , B be a base instance and I ⊆ G ( B ) . Let A ( X ) be the head atom of rule ( v ) and let Q ( Y ) ← (cid:86) ni =1 f i be the EG-rewriting of v . The computation of v ( B ) under I , denoted as v ( B, I ) , is:1. pick m ≥ atoms f i , . . . , f i m from the body of Q whose variables include all variables in Y andform Q (cid:48) ( Y ) ← f i ∧ · · · ∧ f i m ;2. compute v ( B ) as in Definition 5, however restrictto homomorphisms h for which (i) h ( X ) is an an-swer to Q (cid:48) on B and (ii) A ( h ( X )) (cid:54)∈ I . To help us understand Definition 23, let us apply itto Example 22. We have Q (cid:48) ( X ) ← a (cid:48) ( X ). The antijoinbetween Q (cid:48) and A (step (v) of Figure 2) corresponds torestricting to homomorphisms that are answers to Q (cid:48) (step (2.i) of Definition 23), but are not in I (step (2.ii)of Definition 23). In our implementation, we pick one8 lgorithm 2 TGmat ( P, B )1: k ·· = 0; G is the empty graph; I ·· = ∅ do k ·· = k + 1; I k ·· = I k − Compute G k starting from G k − as in Section 4 G k ·· = minDatalog ( G k ) for each node v of depth k do add v ( B, I k − ) (cf. Definition 23) to I k while I k (cid:54) = I k − return I ∞ extensional atom ( m = 1) in step (1). To pick this atom,we consider each f i in the body of rew ( v ), then computethe join as in step (v) of Example 22 between a subsetof the f i -tuples and the A -tuples in I and finally, choosethe f i leading to the highest join output.We summarize TG-guided reasoning for Datalog pro-grams in Algorithm 2. Correctness is stated below. Theorem
For a Datalog program P and a baseinstance B , TGmat ( P, B ) = Ch ( P, B ) .
7. EVALUATION
We implemented Algorithm 1, TG-guided reasoningover a fixed TG (Def. 5) and Algorithm 2 in a new open-source reasoner called
GLog . GLog is a fork of VLog [60]that shares the same code for handling the extensionalrelations while the code for reasoning is entirely novel.We consider three performance measures: the absolutereasoning runtime, the peak RAM consumption observedat reasoning time, and the total number of triggers. Thelast measure is added because it reflects the ability ofTGs to reduce the number of redundant rule executionsand it is robust to most implementation choices.
Systems.
We compared against the following systems: • VLog, as, to our knowledge, is the most efficientsystem both time- and memory- wise [58, 60]; • the latest public release of RDFox from [1] asit outperforms all chase engines tested againstChaseBench [11]: ChaseFun, DEMo [50], LLu-natic [27], PDQ [12] and Pegasus [43]; • the commercial state of the art chase engine COM(name is anonymized due to licensing restrictions); • Inferray, an RDFS reasoner that uses a columnarlayout and that outperforms RDFox [54]; and • WebPIE, another high-performance RDFS rea-soner that runs over Hadoop [59].We ran VLog, RDFox and the commercial chase engineCOM using their most efficient chase implementations.For VLog, this is the restricted chase, while for RDFoxand COM this is the Skolem one [11]. All engines ranusing a single thread. We could not obtain access to theVadalog [10] binaries. However, we perform an indirectcomparison against Vadalog: we both compare againstRDFox using the ChaseBench scenarios from [11].
Benchmarks.
To asses the performance of GLog onlinear and Datalog scenarios, we considered benchmarks
IDP ’sScenario
EDP ’s LI L LE LI L LE
Linear and Datalog scenarios
LUBM var. 163 170 182 116% 120% 232%UOBM 2.1 337 561 NA 3.5 3.9 NADBpedia 29 4204 9396 NA 31.9 33.1 NAClaros 13.8 1749 2689 2749 65.8 8.9 548React. 5.6 259 NA NA 11.3 NA NA
ChaseBench scenarios
S-128 0.15 167 1.9O-256 1 529 5.6
RDFS ( ρ DF) scenarios
LUBM 16.7 160 18YAGO 18.2 498016 27
Table 1: The considered benchmarks.
EDP ’s and
IDP ’sabsolute numbers are stated in millions of facts. previously used to evaluate the performance of rea-soning engines including VLog and RDFox: LUBM[30] and UOBM [41] are synthetic benchmarks; DB-pedia [14] (v2014, available online ) is a KG extractedfrom Wikipedia; Claros [51] and Reactome [22] are real-world ontologies . With both VLog and GLog, the KBsare stored with the RDF engine Trident [57]. Linear scenarios.
Linear scenarios were created usingLUBM, UOBM, DBpedia, Claros and Reactome. For thefirst four KBs, we considered the linear rules returnedby translating the OWL ontologies in each KB using themethod described by [61], which was the technique usedfor evaluating our competitors [45, 58]. This methodconverts an OWL ontology O into a Datalog program P L such that O | = P L . For instance, the OWL axiom A (cid:118) B (concept inclusion) can be translated into therule A ( X ) → B ( X ). This technique is ideal for ourpurposes since this subset is what is mostly supportedby RDF reasoners [45]. Here, the subscript “L” standsfor “lower bound”. In fact, not every ontology can befully captured by Datalog (e.g., ontologies that are notin OWL 2 RL) and in such cases the translation capturesa subset of all possible derivations.For Reactome, we considered the subset of linear rulesfrom the program used in [60]. The programs for thefirst four KBs do not include any existential rules whilethe program for Reactome does. Linear scenarios aresuffixed by “LI”, e.g., LUBM-LI. Datalog scenarios.
Datalog scenarios were created us-ing LUBM, UOBM, DBpedia and Claros, as Reactomeincludes non-Datalog rules only. LUBM comes with agenerator, which allows controlling the size of the baseinstance by fixing the number of different universities X in the instance. One university roughly corresponds to132k facts. In our experiments, we set X to the followingvalues: 125, 1k, 2k, 4k, 8k, 32k, 64k, 128k. This meansthat our largest KB contains about 17B facts. As pro-grams, we used the entire Datalog programs (linear andnon-linear) obtained with [61] as described above. Theseprograms are suffixed by “L”. For Claros and LUBM, we Both datasets are available in our code repository.9
Log RDFox COM GLog TG SizesScenario
Run. Mem Run. Mem Run. Mem Comp Reason w/o cleaning w/ cleaning Mem . . .
007 0 . .
207 1 . . . . .
01 0 .
015 0 .
025 0 . . . . .
448 0 .
776 1 .
224 4 . . . . .
006 0 .
407 0 .
413 4 . . . . .
002 0 .
329 0 .
329 0 . Table 2: Linear scenarios. Time is in sec and memory in MB.
VLog RDFox COM GLog Runtime GLog Memory TG SizesScenario
Run. Mem Run. Mem Run. Mem No opt m m+r No opt m m+r . . . . . . . . . . . . . . . . . . . . . . . . Table 3: Datalog scenarios. Time is in sec and memory in MB. ∗ denotes timeout after 1h. used two additional programs, suffixed by “LE”, createdby [45] as harder benchmarks. These programs extendthe “L” ones with extra rules, such as the transitiveand symmetric rules for owl:sameAs . The relationshipbetween the various rulesets is LI ⊂ L ⊂ LE . ChaseBench scenarios.
ChaseBench was introducedfor evaluating the performance of chase engines [11].The benchmark comes with four different families ofscenarios. Out of these four families, we focused onthe iBench scenarios, namely STB-128 and ONT-256 [4]because they come with non-linear rules with existentialsthat involve many joins and that are highly recursive.Moreover, as we do compare against RDFox which wasthe top-performing chase engine in [11], we can usethese two scenarios to indirectly compare against all theengines considered in [11].
RDFS scenarios.
In the Semantic Web, it has beenshown that a large part of the inference that is possibleunder the RDF Schema (RDFS) [16] can be capturedinto a set of Datalog rules. A number of works havefocused on the execution of such rules. In particular,WebPIE and more recently Inferray returned state-of-the-art performance for ρDF – a subset of RDFS thatcaptures its essential semantics. It is interesting tocompare the performance of GLog, which is a genericengine not optimized for RDFS rules, against such ad-hoc systems. To this end, we considered YAGO [31] anda LUBM KB with 16.7M triples. As rules for GLog, wetranslated the ontologies under the ρDF semantics.Table 1 shows, for each scenario, the correspondingnumber of rules and
EDP -facts as well as the number of
IDP -facts in the model of the KB. With LUBM and thelinear and Datalog scenarios, the number of
IDP -factsis proportional to the input size, thus it is stated as %.For instance, with the “LI” rules, the output is 116%,which means that if the input contains 1M facts, thenreasoning returns 1.16M new facts.
Hardware.
All experiments except the ones on scala-bility (Section 7.5) ran on an Ubuntu 16.04 Linux PCwith Intel i7 64-bit CPU and 94.1 GiB RAM. For our ex-periments on scalability, we used a second machine withan Intel Xeon E5 and 256 GiB of RAM due to the largesizes of the KBs. The cost of both machines is < $ Table 2 summarizes the results of our empirical evalua-tion for the linear scenarios. Recall that when a programis linear and
FES it admits a finite TG which can be com-puted prior to reasoning using tglinear (Algorithm 1) andminimized using minLinear from Definition 14. Columnstwo to seven show the runtime and the peak memory con-sumption for VLog, RDFox and the commercial engineCOM. The remaining columns show results related toTG-guided reasoning. Column
Comp shows the time tocompute and minimize a TG using tglinear and minLinear .Column
Reason shows the time to reason over the com-puted TG given a base instance (i.e., apply Definition 5).Column w/o cleaning shows the total runtime if we donot filter out redundant facts at reasoning time, whilecolumn w/ cleaning shows the total runtime if we addi-tionally filter out redundancies at the end and collectivelyfor all the rules. Notice that in both cases the total run-time includes the time to compute and reason over theTG (columns
Comp and
Reason ). Column
Mem showsthe peak memory consumption. As we will explain later,in the case of linear rules, the memory consumption inGLog is the same both with and without filtering outredundant facts. Finally, the last three columns
C1: TGs outperform the chase in terms of run-time and memory.
The runtime improvements overthe chase vary from multiple orders of magnitude (w/ofiltering of redundancies) to almost two times (w/o fil-tering). When redundancies are discarded, the vastimprovements are attributed to structure sharing , a tech-nique which is also implemented in VLog.Structure sharing is about reusing the same columnsto store the data of different facts. For example, con-sider the rule R ( X, Y ) → S ( Y, X ). Instead of creatingdifferent S - and R -facts, we can simply add a pointerfrom the first column of R to the second column of S and a pointer from the second column of R to the firstcolumn of S . When a rule is linear, both VLog andGLog perform structure sharing and, hence, do not al-locate extra memory to store the derived facts. Apart10 Log RDFox COM GLog TG SizesS
Run. Mem Run. Mem Run. Mem Run. Mem . . . . Table 4: ChaseBench scenarios (S=STB-128,O=ONT-256). Runtime in sec,memory in MB.
Scenario VLog GLog no opt m m+rLUBM-L 38 32 29 25LUBM-LE 239 100 98 93UOBM-L 47 9 8 8DBpedia-L 79 63 61 47Claros-L 286 218 195 185Claros-LE 1099 1072 1049 1039
Table 5:
WebPIE Inferray GLog TG SizesS
Run. Mem Run. Mem Run. Mem . . Table 6: RDFS scenarios (L=LUBM,Y=YAGO). Runtime in sec,memory in MB.
IDP ’s 160M 320M 641M 1B 2B 5B 10B 20B
Table 7: Scalability results. Runtime in sec, memory in GB. from the obvious benefits memory-wise, structure shar-ing also provides benefits in runtime as it allows derivingnew facts without actually executing rules. The above,along with the fact that the facts (redundant or not) arenot explicitly materialized in memory makes GLog veryefficient time-wise.When redundancies are filtered out, GLog still out-performs the other engines: it is multiple orders of mag-nitude faster than RDFox and COM and almost twotimes faster than VLog (Reactome-LI). The performanceimprovements are attributed to a more efficient strategyfor filtering out redundancies: TGs allow filtering out re-dundancies after reasoning has terminated, in contrast tothe chase, which is forced to filter out redundancies rightafter the derivation of new facts. This strategy is moreefficient because we can use a single n-way join ratherthan multiple binary joins to remove redundancies.With regards to memory, GLog has similar memoryrequirements with VLog, while it is much more memoryefficient than RDFox and the commercial engine COM.
C2: The TG computation overhead is small.
Thetime to compute and minimize a TG in advance of rea-soning is only a small fraction of the total runtime, seeTable 2. We argue that even if this time was not neg-ligible, TG-guided reasoning would still be beneficial:first, once a TG is computed reasoning over it is multipletimes faster than the chase and, second, the same TGcan be used to reason over the same rules independentlyof any changes in the database.
Table 3 summarizes our results on generic (linear andnon-linear) Datalog rules. The last nine columns showresults for
TGmat (Algorithm 2). To assess the impactof minDatalog and ruleExec , the rule execution strategyfrom Definition 23, we ran
TGmat as follows: without minDatalog or ruleExec , column No opt ; with minDatalog ,but without ruleExec , column m ; with both minDatalog and ruleExec , column m+r . The total runtime in the lasttwo cases includes the runtime overhead of minDatalog and ruleExec . The last three columns report the numberof nodes, edges, and depth of the computed TGs whenboth minDatalog or ruleExec are employed. Table 4 showsresults for ChaseBench while Table 5 shows the numberof triggers for the Datalog scenarios for VLog and GLog(we could not extract this information for RDFox and COM).We summarize the main conclusions of our analysis. C3: TGs outperform the chase in terms of run-time and memory.
Even without any optimizations,GLog is faster than VLog, RDFox and COM in all butone case. With regards to VLog, GLog is up to ninetimes faster in the Datalog scenarios (LUBM-LE) andup to two times faster in ChaseBench (ONT-256). Withregards to RDFox, GLog is up to 20 times faster in theDatalog scenarios (Claros-L) and up to 67 times fasterin ChaseBench (ONT-256). When all optimizations areenabled GLog outperforms the competitors in all cases.We have observed that the bulk of the computationlies in the execution of the joins involved when executingfew expensive rules. In GLog, joins are executed moreefficiently than in the other engines (GLog uses onlymerge joins), since the considered instances are smaller–recall that in TGs, the execution of a rule associated witha node v considers only the instances of the parents of v .Due to the above, the optimizations do not decrease theruntime considerably. The only exception is LUBM-L,where the optimizations half the runtime.Continuing with the optimizations, their runtime over-head is very low: it is 9% of the total runtime (LUBM-L),while the overhead of minDatalog is less than 1% of thetotal runtime (detailed results are in the appendix). Weconsider this overhead to be acceptable, since, as weshall see later, the optimizations considerably decreasethe number of triggers, a performance measure which isrobust to hardware and most implementation choices.It is important to mention that GLog implements thetechnique in [33] for executing transitive and symmetricrules. The improvements brought by this technique aremost visible with LUBM-LE where the runtime increasesfrom 18s with this technique to 71s without it. Otherimprovements occur with UOBM-L and DBpedia-L (69%and 57% resp.). In any case, even without this technique,GLog remains faster than its competitors in all cases.Last, the ChaseBench experiments allow us to compareagainst Vadalog. According to [10], Vadalog is threetimes faster than RDFox on STB-128 and ONT-256. Ourempirical results show that GLog brings more substantialruntime improvements: GLog is from 49 times to morethan 67 times faster than RDFox in those scenarios.With regards to memory, the memory footprint ofGLog again is comparable to that of VLog and it is11ower than that of RDFox and of COM. C4: TGs outperform the chase in terms of thenumber of triggers.
Table 5 shows that the totalnumber of triggers and, hence, the amount of redundantcomputations, is considerably lower than the total num-ber of triggers in VLog even when the optimizations aredisabled. This is due to the different approaches em-ployed to filter out redundancies: VLog filters out redun-dancies right after the execution of each rule [58], whileGLog performs this filtering after each round. Whenthe optimizations are enabled, the number of triggersfurther decreases: in the best case (DBpedia-L), GLogcomputes 1.69 times fewer triggers (79M/47M).
Table 6 summarizes the results of the RDFS scenarioswhere GLog is configured with both optimizations en-abled. We can see that GLog is faster than both RDFSengines. With regards to Inferray, GLog is two orders ofmagnitude faster on LUBM and more than four timesfaster on YAGO. With regards to WebPIE, GLog is threeorders of magnitude faster on LUBM and more than 32times faster on YAGO. With regards to memory, GLogis more memory efficient in all but one cases.
We used the LUBM benchmark to create several KBswith 133M, 267M, 534M, 1B, 2B, 4B, 8B, and 17B factsrespectively. Table 7 summarizes the performance withthe Datalog program LUBM-L. Columns are labeledwith the size of the input database. Each column showsthe runtime, the peak RAM memory consumption, andthe number of derived facts for each input database.We can see that GLog can reason with up to 17B factsin less than 40 minutes without resorting to expensivehardware. We are not aware of any other centralizedreasoning engine that can scale up to such an extent.
8. RELATED WORK
One approach to improve the reasoning performance isto parallelize the execution of the rules. RDFox proposesa parallelization technique for Datalog materializationwith mostly lock-free data insertion. Parallelization hasbeen also been studied for reasoning over RDFS andOWL ontologies. For example, WebPIE encodes the ma-terialization process into a set of MapReduce programswhile Inferray executes each rule on a dedicated thread.Our experiments show that GLog outperforms all theseengines in a single-core scenario. This motivates furtherresearch on parallelizing TG-based materialization.A second approach is to reduce the number of logi-cally redundant facts by appropriately ordering the rules.In [59], the authors describe a rule ordering that is op-timal only for a fixed set of RDFS rules. In contrast,we focus on generic programs. ChaseFun [15] proposesa new rule ordering technique that focuses on equalitygenerating dependencies . Hence, it is orthogonal to ourapproach. In a similar spirit, the rewriting technique from [33] targets transitive and symmetric rules. GLogapplies this technique by default to improve the perfor-mance, but our experiments show it outperforms thestate of the art even without this optimization.To optimize the execution of the rules themselves,most chase engines rely on external DBMSs or employstate of the art query execution algorithms: LLunatic[27], PDQ and ChaseFun run on top of PostgreSQL;RDFox and VLog implement their own in-memory ruleexecution engine. However, none of these engines caneffectively reduce the instances over which rules areexecuted as TGs do. Other approaches involve exploringcolumnar memory layouts as in VLog and Inferray toreduce memory consumption and to guarantee sequentialaccess and efficient sort-merge join inference.Orthogonal to the above is the work in [10], whichintroduces a new chase variant for materializing KBs ofwarded Datalog programs. Warded Datalog is a classof programs not admiring a finite model for any baseinstance. The variant works as the restricted chase doesbut replaces homomorphism with isomorphism checks.As a result, the computed models become bigger. Animplementation of the warded chase is also introducedin [10] which focuses on decreasing the cost of isomor-phism checks. The warded chase implementation doesnot apply any techniques to detect redundancies in theoffline fashion as we do for linear rules, or to reduce theexecution cost of Datalog rules as we do in Section 6.We now turn our attention to the applications ofmaterialization in goal-driven query answering. Twowell-known database techniques that use materializa-tion as a tool for goal-driven query answering are magicsets and subsumptive tabling [8, 9, 53, 55]. The advan-tage of these techniques over the query rewriting ones,which are not based on materialization, e.g., [6, 19, 29],is the full support of Datalog. The query rewriting tech-niques can support Datalog of bounded recursion only.Beyond Datalog, materialization-based techniques havebeen recently proposed for goal-driven query answeringover KBs with equality [13], as well as for probabilisticKBs [56], leading in both cases to significant improve-ments in terms of runtime and memory consumption.The above automatically turns TGs to a very powerfultool to also support query-driven knowledge exploration.TGs are different from acyclic graphs of rule depen-dencies [7]: the former contain a single node per rulewhile TGs do not.
9. CONCLUSION
We introduced a novel approach for materializing KBsthat is based on traversing acyclic graphs of rules calledTGs. Our theoretical analysis and our empirical evalua-tion over well-known benchmarks show that TG-guidedreasoning is a more efficient alternative to the chase,since it effectively overcomes all of its limitations.Future research involves studying the problem of up-dating TGs in response to KB updates, as well as ex-tending TGs to materialize distributed KBs.12
0. REFERENCES [1] RDFox public release. https://github.com/dbunibas/chasebench/tree/master/tools/rdfox .Accessed: 2020-11-10.[2] S. Abiteboul, R. Hull, and V. Vianu.
Foundationsof Databases . Addison Wesley, 1995.[3] M. Aref, B. ten Cate, T. J. Green, B. Kimelfeld,D. Olteanu, E. Pasalic, T. L. Veldhuizen, andG. Washburn. Design and Implementation of theLogicBlox System. In
SIGMOD , pages 1371–1382,2015.[4] P. C. Arocena, B. Glavic, R. Ciucanu, and R. J.Miller. The iBench Integration MetadataGenerator. In
VLDB , page 108–119, 2015.[5] F. Baader and T. Nipkow.
Term Rewriting and AllThat . Cambridge University Press, USA, 1999.[6] J. Baget, M. Lecl`ere, M. Mugnier, S. Rocher, andC. Sipieter. Graal: A Toolkit for Query Answeringwith Existential Rules. In
RuleML , 2015.[7] J. Baget, M. Lecl`ere, M. Mugnier, and E. Salvat.On rules with existential variables: Walking thedecidability line.
Artificial Intelligence ,175(9-10):1620–1654, 2011.[8] F. Bancilhon, D. Maier, Y. Sagiv, and J. D.Ullman. Magic Sets and Other Strange Ways toImplement Logic Programs. In
PODS , pages 1–15,1986.[9] C. Beeri and R. Ramakrishnan. On the Power ofMagic.
Journal of Logic Programming ,10(3,4):255–299, 1991.[10] L. Bellomarini, E. Sallinger, and G. Gottlob. TheVadalog System: Datalog-based Reasoning forKnowledge Graphs.
PVLDB , 11(9):975–987, 2018.[11] M. Benedikt, G. Konstantinidis, G. Mecca,B. Motik, P. Papotti, D. Santoro, andE. Tsamoura. Benchmarking the chase. In
PODS ,pages 37–52, 2017.[12] M. Benedikt, J. Leblay, and E. Tsamoura. PDQ:Proof-driven query answering over web-based data.In
VLDB , page 1553–1556, 2014.[13] M. Benedikt, B. Motik, and E. Tsamoura.Goal-driven query answering for existential ruleswith equality. In
AAAI , pages 1761 – 1770, 2018.[14] C. Bizer, J. Lehmann, G. Kobilarov, S. Auer,C. Becker, R. Cyganiak, and S. Hellman. DBpedia- A crystallization point for the Web of Data.
Journal of Web Semantics , 7(3):154–165, 2009.[15] A. Bonifati, I. Ileana, and M. Linardi. FunctionalDependencies Unleashed for Scalable DataExchange. In
SSDBM , 2016.[16] D. Brickley, R. V. Guha, and B. McBride. Rdfschema 1.1.
W3C recommendation , 25:2004–2014,2014.[17] A. Cal`ı, G. Gottlob, and M. Kifer. Taming theinfinite chase: Query answering under expressiverelational constraints.
J. Artif. Int. Res. ,48(1):115–174, 2013. [18] A. Cal`ı, G. Gottlob, and T. Lukasiewicz. A generalDatalog-based framework for tractable queryanswering over ontologies.
Journal of WebSemantics , 14:57–83, 2012.[19] D. Calvanese, B. Cogrel, S. Komla-Ebri,R. Kontchakov, D. Lanti, M. Rezk,M. Rodriguez-Muro, and G. Xiao. Ontop:Answering SPARQL queries over relationaldatabases.
Semantic Web , 8(3):471–487, 2017.[20] A. K. Chandra and P. M. Merlin. Optimalimplementation of conjunctive queries in relationaldata bases. In
STOC , pages 77–90, 1977.[21] B. Chin, D. von Dincklage, V. Ercegovac,P. Hawkins, M. S. Miller, F. Och, C. Olston, andF. Pereira. Yedalog: Exploring knowledge at scale.In
SNAPL , pages 63–78, 2015.[22] D. Croft, A. F. Mundo, R. Haw, M. Milacic,J. Weiser, G. Wu, M. Caudy, P. Garapati,M. Gillespie, M. R. Kamdar, et al. The reactomepathway knowledgebase.
Nucleic acids research ,42(D1):D472–D477, 2013.[23] A. S. d’Avila Garcez, K. Broda, and D. M. Gabbay.
Neural-symbolic learning systems: foundations andapplications . Perspectives in neural computing.Springer, 2002.[24] S. Delivorias, M. Lecl`ere, M. Mugnier, andF. Ulliana. On the k-Boundedness for ExistentialRules. In
RuleML+RR , pages 48–64, 2018.[25] A. Deutsch, A. Nash, and J. B. Remmel. The chaserevisited. In
PODS , pages 149–158, 2008.[26] R. Fagin, P. G. Kolaitis, R. J. Miller, and L. Popa.Data exchange: semantics and query answering.
Theoretical Computer Science , 336(1):89–124, 2005.[27] F. Geerts, G. Mecca, P. Papotti, and D. Santoro.That’s All Folks! LLUNATIC Goes Open Source.In
VLDB , page 1565–1568, 2014.[28] G. Gottlob, G. Orsi, and A. Pieris. Ontologicalquery answering via rewriting. In
ADBIS , pages1–18, 2011.[29] G. Gottlob, G. Orsi, and A. Pieris. QueryRewriting and Optimization for OntologicalDatabases.
ACM TODS , 39(3):25:1–25:46, 2014.[30] Y. Guo, Z. Pan, and J. Heflin. LUBM: Abenchmark for OWL knowledge base systems.
Journal of Web Semantics , 3(2-3), 2011.[31] J. Hoffart, F. Suchanek, K. Berberich, andG. Weikum. Yago2: A spatially and temporallyenhanced knowledge base from wikipedia.
ArtificialIntelligence , 194:28–61, 2013.[32] A. Hogan, E. Blomqvist, M. Cochez, C. d’Amato,G. de Melo, C. Gutierrez, J. E. L. Gayo,S. Kirrane, S. Neumaier, A. Polleres, R. Navigli,A.-C. N. Ngomo, S. M. Rashid, A. Rula,L. Schmelzeisen, J. Sequeda, S. Staab, andA. Zimmermann. Knowledge Graphs. arXiv:2003.02320 [cs] , 2020. arXiv: 2003.02320.[33] P. Hu, B. Motik, and I. Horrocks. Modular13aterialisation of datalog programs. In
AAAI ,pages 2859–2866, 2019.[34] P. Hu, J. Urbani, B. Motik, and I. Horrocks.Datalog Reasoning over Compressed RDFKnowledge Bases. In
CIKM , pages 2065–2068,2019.[35] N. Konstantinou, M. Koehler, E. Abel, C. Civili,B. Neumayr, E. Sallinger, A. A. Fernandes,G. Gottlob, J. A. Keane, L. Libkin, and N. W.Paton. The VADA Architecture for Cost-EffectiveData Wrangling. In
SIGMOD , pages 1599–1602,2017.[36] B. Kruit, P. A. Boncz, and J. Urbani. Extractingnovel facts from tables for knowledge graphcompletion. In
ISWC , pages 364–381, 2019.[37] B. Kruit, H. He, and J. Urbani. Tab2know:Building a knowledge base from tables in scientificpapers. In
ISWC , pages 349–365. Springer, 2020.[38] M. Lecl`ere, M. Mugnier, M. Thomazo, andF. Ulliana. A Single Approach to Decide ChaseTermination on Linear Existential Rules. In
ICDT ,pages 18:1–18:19, 2019.[39] M. Lecl`ere, M. Mugnier, and F. Ulliana. Onbounded positive existential rules. In DL , 2016.[40] J. Lee, T. Hwang, J. Park, Y. Lee, B. Motik, andI. Horrocks. A Context-Aware RecommendationSystem for Mobile Devices. In ISWC , pages380–382, 2020.[41] L. Ma, Y. Yang, Z. Qiu, G. Xie, Y. Pan, andS. Liu. Towards a complete OWL OntologyBenchmark. In
ESWC , pages 125–139, 2006.[42] D. Maier, A. O. Mendelzon, and Y. Sagiv. Testingimplications of data dependencies.
ACMTransactions on Database Systems , 4(4):45–5469,1979.[43] M. Meier. The backchase revisited.
VLDB J. ,23(3):495–516, 2014.[44] B. Motik, B. C. Grau, I. Horrocks, Z. Wu,A. Fokoue, C. Lutz, et al. OWL 2 web ontologylanguage profiles.
W3C recommendation , 27:61,2009.[45] B. Motik, Y. Nenov, R. Piro, I. Horrocks, andD. Olteanu. Parallel Materialisation of DatalogPrograms in Centralised, Main-Memory RDFSystems. In
AAAI , pages 129–137, 2014.[46] W. E. Moustafa, V. Papavasileiou, K. Yocum, andA. Deutsch. Datalography: Scaling datalog graphanalytics on graph processing systems. In
IEEEInternational Conference on Big Data , pages 56–65,2016.[47] Y. Nenov, R. Piro, B. Motik, I. Horrocks, Z. Wu,and J. Banerjee. RDFox: A Highly-Scalable RDFStore. In
ISWC , pages 3–20, 2015.[48] N. Noy, Y. Gao, A. Jain, A. Narayanan,A. Patterson, and J. Taylor. Industry-scaleKnowledge Graphs: Lessons and Challenges.
Commun. ACM , 62(8):36–43, July 2019. [49] A. Onet. The chase procedure and its applicationsin data exchange. In
DEIS , pages 1–37, 2013.[50] R. Pichler and V. Savenkov. DEMo: DataExchange Modeling Tool. In
VLDB , pages1606–1609, 2009.[51] S. Rahtz, A. Dutton, D. Kurtz, G. Klyne,A. Zisserman, and R. Arandjelovic.CLAROS—Collaborating on Delivering the Futureof the Past. In DH , pages 355–357, 2011.[52] Y. Sagiv and M. Yannakakis. Equivalences amongrelational expressions with the union and differenceoperators. Journal of the ACM , 27(4):633–655,1980.[53] D. Sereni, P. Avgustinov, and O. de Moor. AddingMagic to an Optimising Datalog Compiler. In
SIGMOD , pages 553–566, 2008.[54] J. Subercaze, C. Gravier, J. Chevalier, andF. Laforest. Inferray: Fast in-Memory RDFInference.
Proceedings of the VLDB Endowment ,9(6):468–479, 2016.[55] K. T. Tekle and Y. A. Liu. More Efficient DatalogQueries: Subsumptive Tabling Beats Magic Sets.In
SIGMOD , pages 661–672, 2011.[56] E. Tsamoura, V. Guti´errez-Basulto, andA. Kimmig. Beyond the Grounding Bottleneck:Datalog Techniques for Inference in ProbabilisticLogic Programs. In
AAAI , pages 10284–10291,2020.[57] J. Urbani and C. Jacobs. Adaptive Low-levelStorage of Very Large Knowledge Graphs. In
WWW , pages 1761–1772, 2020.[58] J. Urbani, C. Jacobs, and M. Kr¨otzsch.Column-Oriented Datalog Materialization forLarge Knowledge Graphs. In
AAAI , pages 258–264,2016.[59] J. Urbani, S. Kotoulas, J. Maassen, F. vanHarmelen, and H. Bal. OWL Reasoning withWebPIE: Calculating the Closure of 100 BillionTriples. In
ESWC , pages 213–227, 2010.[60] J. Urbani, M. Kr¨otzsch, C. Jacobs, I. Dragoste, andD. Carral. Efficient Model Construction for HornLogic with VLog. In
IJCAR , pages 680–688, 2018.[61] Y. Zhou, B. Cuenca Grau, I. Horrocks, Z. Wu, andJ. Banerjee. Making the Most of your Triple Store:Query Answering in OWL 2 using an RL Reasoner.In
WWW , pages 1569–1580, 2013.14 cenario
VLog GLogLUBM-LI 34 346 35 093UOBM-LI 7625 6718DBpedia-LI 61 134 115 150Claros-LI 129 098 134 800Reactome-LI 17 120 23 218 (a)
Scenario m rLUBM-L 0 . . . . .
08 0 . .
05 0 . .
03 2 . . . (b) Cost of optimizations. Table 8: Additional Experimental Results
VLog RDFox COM GLog Runtime GLog MemoryScenario
Runtime Memory Runtime Memory Runtime Memory No opt m m+r No opt m m+rLUBM-L 1 . . . . . . . . . . . . . . . . . . . . . . . . . . Table 9: Datalog scenarios. GLog is ran without the optimization from [33]. Time is in sec and memory in MB.
VLog RDFox COM GLogScenario
Runtime Memory Runtime Memory Runtime Memory Runtime MemorySTB-128 0 . . . . Table 10: ChaseBench scenarios. GLog is ran without the optimization from [33]. Time is in sec and memory in MB.
VLog WebPIE Inferray GLog Runtime GLog MemoryScenario
Runtime Memory Runtime Memory Runtime Memory No opt m m+r No opt m m+rLUBM 0 . . . . . Table 11: RDFS scenarios. GLog is ran without the optimization from [33]. Time is in sec and memory in MB.
APPENDIXA. ADDTIONAL EXPERIMENTAL RESULTS
Number of triggers in the linear scenarios.
Table 8a summarizes the number of triggers for the linear scenarios.We can see that the number of triggers in GLog is often higher than in VLog. This is due to the fact that GLog doesnot eliminate redundancies right at their creation. However, these redundancies are harmless: due to structure sharingthese redundant facts are not explicitly materialized in memory and hence, they do not slow down the runtime.
Cost of optimizations.
Table 8b summarizes the cost of optimizations for the Datalog scenarios. Recall that theoptimizations in Section 6 are not applicable to ChaseBench as the rules have existential variables. Column m showsthe total runtime cost of minDatalog , while column r shows the total runtime cost of ruleExec . Impact on rewriting on GLog.
Tables 9, 10 and 11 summarize the performance of GLog when disabling theoptimization from [33]. To ease the presentation, we also copy the results of the competitor engines on the samebenchmarks from Tables 3, 4 and 6. We can see that the only scenario whose performance degrades considerably isLUBM-LE shown in Table 9. Even in this case though, the performance of GLog is still better than the performance ofits competitors: it is twice as fast as VLog, RDFox and COM in most scenarios and more than an order of magnitudefaster than RDFox and COM in Claros-L.
Running RDFox in multiple threads.
Tables 12, 13 and 14 show the runtime performance of RDFox whenincreasing the number of threads from 1 to 8 and 16. For completeness, we also copy the runtime of GLog using a single thread on the same scenarios from Tables 2, 3 and 4. We can see that the runtime of RDFox drops considerablywhen using 16 threads. However, it is still higher than the runtime of GLog in all cases except UOBM-L, whereRDFox’s runtime is 1.6s, while GLog’s runtime is 2.6s. In the other scenarios, GLog is up to 7.8 times faster thanRDFox (ONT-256). 15
DFox GLog RuntimeScenario
Runtime (1 thread) Runtime (8 threads) Runtime (16 threads) Runtime (32 threads) w/o cleaning w/ cleaningLUBM-LI 22 4 . . . .
010 1 . . . . . .
012 0 . . . . . .
980 4 . . . .
051 4 . . . . . .
131 0 . Table 12: Linear scenarios. RDFox is ran in one, eight, 16 threads and 32 threads. GLog is ran in a single thread. Time in sec.
RDFox GLog RuntimeScenario
Runtime (1 thread) Runtime (8 threads) Runtime (16 threads) Runtime (32 threads) No opt m m+rLUBM-L 23 4 . . . . . . . . . . . . . . . . . . . . . . . . . . Table 13: Datalog scenarios. RDFox is ran in one, eight, 16 threads and 32 threads. GLog is ran in a single thread. Time in sec.* denotes runtime exception.
RDFox GLog no opt
Scenario
Runtime (1 thread) Runtime (8 threads) Runtime (16 threads) Runtime (32 threads) RuntimeSTB-128 13 . . . . . . Table 14: ChaseBench scenarios. RDFox is ran in 1, 8, 16 threads and 32 threads. GLog is ran in a single thread. Time in sec.
B. ADDITIONAL DEFINITIONS
We provide some definitions that will be useful for the proofs in the next section.For a KB K where Ch ( K ) is defined, the depth d ( t ) of a term t in Ch ( K ) is defined as follows: if t ∈ Consts , then d ( t ) = 1; otherwise, if t is a null of the form n r,h,z , then d ( t ) = max ( d ( t ) , . . . , d ( t n )) + 1, where { t , . . . , t n } are allterms in the range of h .Next, we recapitulate the definitions of some known classes of programs. Definition
Consider a program P and some k ≥ . • P is Finite Expansion Set ( FES ), if for each base instance B , the KB ( P, B ) has a terminating chase. • P is k - Term Depth Bounded ( k - TDB ), if for each base instance B , each i ≥ and each term t in Ch i ( P, B ) , d ( t ) ≤ k . P is TDB , if it is k - TDB . • P is Finite Order Rewritable ( FOR ), if, for each BCQ Q , there is a union of BCQs (UBCQs) Q (cid:48) such that, foreach base instance B , we have that ( P, B ) | = Q iff B | = Q (cid:48) . C. PROOFS FOR RESULTS IN SECTION 4
For a program P , we refer to the graph computed by applying the base and the inductive steps from Section 4 asthe level- k full EG for P , and denote it as Φ kP . Below, we show that reasoning via a level- k full EG produces logicallyequivalent facts with the k -th round of the chase when the chase is applied in a breadth-first fashion: Theorem
For a program P , a base instance B and a k ≥ , Ch k ( P, B ) ≡ Φ kP ( B ) . Proof.
Let P = { r , . . . , r n } . Let Φ kP = G k = ( V k , E k ) and let Φ k +1 P = G k +1 = ( V k +1 , E k +1 ). Let I k = Ch k (( P, B ))and let J k = G k ( B ).( ⇐ ) To prove the claim, we show the following property, for each k ≥ • ψ . there exists a homomorphism g k : J k → I k .For k = 0, ψ holds, since I = G ( B ) = B . For k + 1 and assuming that ψ holds for k , the proof proceeds asfollows. Let v k +11 , . . . , v k +1 m be all nodes in V k +1 of depth k + 1. For each 1 ≤ ι ≤ m , let rule ( v k +1 ι ) = r ι and let v k +1 ι ( B ) = { F ι, , . . . , F ι,n ι } . Since each rule has a single atom in its head, it follows from Definition 5 that for each1 ≤ ι ≤ m and each 1 ≤ κ ≤ n ι , there exists a homomorphism h ι,κ , such that h ι,κ s ( head ( r ι )) = F ι,κ . We distinguishthe cases, for each rule r ι , for 1 ≤ ι ≤ m : • r ι is an extensional rule. Hence, for each each 1 ≤ κ ≤ n ι , h ι,κ is a homomorphism from body ( r ι ) into B . • r ι is not an extensional rule. WLOG, assume that r ι comprises only IDP -atoms in its body. Since G k +1 is a fullEG, it follows that for each 1 ≤ λ ≤ e ι , there exists an edge u ι,λ → ι,λ v k +1 ι in E k +1 . Due to the above, and dueto Definition 5, it follows that for each 1 ≤ κ ≤ n ι , h ι,κ is a homomorphism from body ( r ι ) into e ι (cid:83) λ u ι,λ ( B ).Let N k = m (cid:83) ι n ι (cid:83) κ h ι,κ s ( head ( r ι )). We further distinguish the cases: • for each 1 ≤ ι ≤ m and each 1 ≤ κ ≤ n ι , F ι,κ ∈ G k ( B ). Then ψ trivially holds.16 there exists 1 ≤ ι ≤ m and 1 ≤ κ ≤ n ι , such that F ι,κ (cid:54)∈ G k ( B ). Since ψ holds for k and due to g k , it followsthat for each 1 ≤ ι ≤ m and each 1 ≤ κ ≤ n ι , there exists a homomorphism χ ι,κ = h ι,κ ◦ g k from body ( r ι ) into I k . Due to the above, for each 1 ≤ ι ≤ m and each 1 ≤ κ ≤ n ι , there also exists a homomorphism ω ι,κ from h ι,κ s ( head ( r ι )) into χ ι,κ s ( head ( r ι )), mapping each h ι,κ s ( n z ) to χ ι,κ s ( n (cid:48) z ), where h ι,κ s ( z ) = n z and χ ι,κ s ( z ) = n (cid:48) z ,for each existentially quantified variable z occurring in r . Due to the above and since for each 1 ≤ ι ≤ m andeach 1 ≤ κ ≤ n ι , h ι,κ s ( c ) = g k ( c ), there also exists a homomorphism from N k into M k = m (cid:83) ι n ι (cid:83) κ χ ι,κ s ( head ( r ι ))and hence from I k ∪ N k into J k ∪ M k . The above shows that ψ holds for k + 1.( ⇒ ) To prove the claim, we show the following property, for each k ≥ • φ . there exists a homomorphism g k : I k → J k mapping each F ∈ I k to some fact F (cid:48) ∈ J k of the same depth.For k = 0, φ holds, since I = G ( B ) = B . For k + 1 and assuming that φ holds for k the proof proceeds as follows.For each 1 ≤ i ≤ n and each 1 ≤ j ≤ n i , let h i,j be the j -th homomorphism from the body of rule r i into I k , where h i,j ( body ( r i )) comprises at least one fact of depth k . Due to the inductive hypothesis, we know that for each 1 ≤ i ≤ n and 1 ≤ j ≤ n i , there exists a homomorphism χ i,j = h i,j ◦ g k from body ( r i ) into J k . We distinguish the cases, foreach rule r i , for 1 ≤ i ≤ n : • r ι is an extensional rule. Hence, I k = B . • r ι is not an extensional rule. WLOG, assume that r ι comprises only IDP -atoms in its body. Let u i,j , . . . , u | body ( r i ) | i,j be the nodes in V k , such that for each 1 ≤ j ≤ n i and each 1 ≤ l ≤ | body ( r i ) | , the l -th fact in χ i,j ( body ( r i ))belongs to u li,j ( B ). Since φ holds for k , it follows that for each 1 ≤ j ≤ n i , some fact in χ i,j ( body ( r i )) is of depth k . Hence some node in u i,j , . . . , u | body ( r i ) | i,j is of depth k . Since G k +1 is a full EG for P , it follows that for each1 ≤ j ≤ n i and each 1 ≤ l ≤ | body ( r i ) | , u li,j → j v ∈ E k +1 , where rule ( v ) = r i .Due to the above and due to Definition 5, it follows that for each 1 ≤ i ≤ n and each 1 ≤ j ≤ n i , we have χ i,j s ( head ( r i )) ∈ G k +1 ( B ). Since for each 1 ≤ i ≤ n and each 1 ≤ j ≤ n i , there exists a homomorphism from g k from h i,j ( body ( r i )) into χ i,j ( body ( r i )), it follows that for each 1 ≤ i ≤ n and each 1 ≤ j ≤ n i , there also exists ahomomorphism ω i,j from h i,j s ( head ( r i )) into χ i,j s ( head ( r i )), mapping each h i,j s ( n z ) to χ i,j s ( n (cid:48) z ), where h i,j s ( z ) = n z and χ i,j s ( z ) = n (cid:48) z , for each existentially quantified variable z occurring in r . Due to the above and since for each1 ≤ i ≤ n and each 1 ≤ j ≤ n i , h i,j s ( c ) = g k ( c ), there also exists a homomorphism from N k = n (cid:83) i n i (cid:83) j h i,j s ( head ( r i ))into M k = n (cid:83) i n i (cid:83) j χ i,j s ( head ( r i )) and hence from I k ∪ N k into J k ∪ M k . The above shows that the induction holds for k + 1. Theorem The following hold: P is FTG iff it is
BDD ; and P is TDB ∩ FOR iff it is
BDD . Proof.
The proof is based on the proofs of Theorem 27 and Theorem 28.
Theorem P is FTG iff it is
BDD . Proof. = ⇒ If P is FTG , then there is some finite TG G = ( V, E ) for this program. We proceed to show that P is k - BDD with k the depth of G . More precisely, we show by contradiction that Ch k ( P, B ) | = Ch k +1 ( P, B ) for anygiven base instance B .1. Suppose for a contradiction that Ch k ( K ) (cid:54)| = Ch k +1 ( K ) with K = ( P, B ).2. By the definition of the standard chase, Ch k ( K ) ⊆ Ch k +1 ( K ).3. By (1) and (2), there is some BCQ q such that Ch k ( K ) (cid:54)| = q and Ch k +1 ( K ) | = q .4. We can show via induction that G i ( B ) ⊆ Ch i ( K ) for all i ≥ G k ( B ) ⊆ Ch k ( K ).6. By (3) and (5), G ( B ) (cid:54)| = q .7. By (3), K | = q .8. By (6) and (7), the graph G is not a TG for P . ⇐ = Since P is BDD , we have that P is k - BDD for some k ≥
0. Therefore, the graph Φ kP is a TG for P byTheorem 26 and the program P is FTG . Theorem P is TDB and
FOR iff it is
BDD . Proof. = ⇒
17. Assume that P is (a) FOR and (b)
TDB .2. Let h be a homomorphism that maps every t ∈ Nulls ∪ Vars to a fresh c t ∈ Consts unique for t .3. For all facts ϕ that can be defined over some s ∈ Preds in P , we introduce the following notions.(a) Let ω ϕ be some (arbitrarily chosen) rewriting for the BCQ ϕ with respect to P . Note that, such a rewritingmust exist by (1.a).(b) Let k ϕ be the smallest number such that Ch k ϕ ( P, h ( β )) | = Ch k ϕ +1 ( P, h ( β )) for every disjunct β in therewriting ω ϕ . Note that, such a number must exist by (1.b).4. For all s ∈ Preds , let k s be the smallest number such that k s > k ϕ for all facts ϕ that can be defined over thepredicate s . Note that, the number k s is well-defined despite the fact that we can define infinitely many differentfacts over any given predicate. This is because k s ( t ,...,t n ) = k s ( u ,...,u n ) if we have that there is a bijectivefunction mapping t i to u i for all 1 ≤ i ≤ n .5. Let k P be the smallest number such that k P > k s + 1 for all s ∈ Preds in P .6. Consider some fact ϕ = s ( t , . . . , t n ), some base instance B , and some i ≥
0. We show that, if the terms t , . . . , t n are in Ch i ( P, B ) and ϕ ∈ Ch ( P, B ), then ϕ ∈ Ch i + k P − ( P, B ).a. h ( ϕ ) ∈ Ch (( P, B (cid:48) )) with B (cid:48) = h ( Ch i ( P, B )).b. By (a): B (cid:48) | = ω h ( ϕ ) where ω h ( ϕ ) is a UBCQ of the form ∃ x .β ∨ . . . ∨ ∃ x n .β n .c. By (b): B (cid:48) | = ∃ x k .β k for some 1 ≤ k ≤ n .d. By (c): there is a homomorphism such that h (cid:48) ( β k ) ⊆ B (cid:48) .e. By (d): h (cid:48) ( ϕ ) ∈ Ch (( P, h ( β k ))).f. By (5) and (e): Ch k P − ( P, h (cid:48) ( β k )) ⊇ Ch ( P, h (cid:48) ( β k )).g. By (e) and (f): h (cid:48) ( ϕ ) ∈ Ch k P − ( P, h (cid:48) ( β k )).h. By (d) and (g): h (cid:48) ( ϕ ) ∈ Ch k P − ( P, B (cid:48) ).i. By (h): ϕ ∈ Ch i + k P − ( P, B ).7. For a fact ϕ = s ( t , . . . , t n ), let d ( ϕ ) = max ≤ i ≤ n ( d ( t i )).8. By (6) and (7): We show via induction that, for all d ≥
1, the set Ch d · k P ( P, F ) contains all of the facts ϕ ∈ Ch ( P, F ) with d ( ϕ ) ≤ d . • Base case: The set Ch ( P, F ) contains all terms of depth 1 (i.e., all constants) that occur in Ch ( P, F ). Hence,by (6), the set Ch k P ( P, F ) contains every ϕ ∈ Ch ( P, F ) with d ( ϕ ) = 1. • Inductive step: Let i ≥
1. Then, Ch ( i − · k P ( P, F ) contains all ϕ ∈ Ch ( P, F ) with d ( ϕ ) = i −
1. Hence, the set Ch ( i − · k P +1 ( P, F ) contains all t ∈ Terms in Ch ( P, F ) with d ( ϕ ) = i . By (6), the set Ch i · k P ( P, F ) containsall ϕ ∈ Ch ( P, F ) with d ( ϕ ) = i .9. By (1.b): There is some k d ≥ P such that, for every term in t occurring in Ch (( P, F )),the depth t is equal or smaller than k d .10. By (8) and (9): P is ( k d · k P )- BDD . Note that, neither k d nor k P depend on the set of facts F . ⇐ = Since P ∈ BDD , P ∈ k - BDD for some k ≥
0. Via induction on i ∈ { , . . . , k } , we can show that all termsin Ch i ( P, B ) are of depth i or smaller for any instance B , and hence, P ∈ k - TDB . Note that
BDD ⊆ FOR has beenshown in [39].
Theorem
The language of all programs that admit a finite TG is undecidable.
Proof.
This follows from the fact that
FTG decidability implies decidability of
FOR for Datalog programs, whichis undecidable [28].
Theorem For each KB ( P, B ) that admits a finite model, there exists an instance-dependent TG. Proof.
Recall that we denote by Φ kP the level- k full EG for a program P . We can stop the expansion of thelevel- k full EG for a program P when Φ k − P ( B ) | = Φ kP ( B ) holds. The above, along with Theorem 26 and the factthat whenever Ch k − ( P, B ) | = Ch k ( P, B ) holds for some k ≥
1, then the KB (
P, B ) admits a finite model (see [24])conclude the proof of Theorem 8. 18 . PROOFS FOR RESULTS IN SECTION 5
Theorem
For any linear program P that is FES , tglinear ( P ) is a TG for P . Proof.
In order to show that the EG G = ( V, E ) computed by Algorithm 1 is a TG for P , it suffices to show that Ch ∞ ( P, B ) is logically equivalent to G ( B ). The proof makes use of Propositions 30 and 31, as well as of Lemma 32.Note that Propositions 30 is a known result and, hence, we skip its proof, while Proposition 31 directly follows fromDefinition 5. Proposition
For each linear program P and each base instance B , the following holds:Ch ∞ ( P, B ) ≡ (cid:91) F ∈ B Ch ∞ ( P, { F } ) (2) Proposition
For each linear TG G , each node v ∈ G and each base instance B , the following holds: v ( B ) = (cid:91) F ∈ B u ( { F } ) (3) Lemma
For each fact f ∈ H ( P ) , Γ( { f } ) ≡ Ch ∞ ( P, { f } ) , where Γ is the EG computed in lines 2–10 for fact f . Proof. ( ⇒ ) We want to show that for each fact f ∈ H ( P ), there exists a homomorphism from Γ( { f } ) into Ch ∞ ( P, { f } ). Due to Algorithm 1, we know that Γ is a graph of the form (cid:32) n (cid:91) j =1 u j , n − (cid:91) j =1 { u j → u j +1 } (cid:33) (4)where u j ∈ V , for each 0 ≤ i ≤ n ; u j → u j +1 ∈ E , for each 0 ≤ i < n ; and u n = v . To prove this direction, we willshow that the following property holds, for each 0 ≤ i ≤ n : • φ . there exists a homomorphism h i from Γ (cid:22) u i ( { f } ) into Ch ∞ ( P, { f } ).For i = 0, since Γ (cid:22) v i is the empty graph and hence Γ (cid:22) u i ( B ) = B by Definition 5, it follows that φ holds. For i + 1and assuming that φ holds for i the proof proceeds as follows. Since φ holds for i , we know that there exists ahomomorphism h i from Γ (cid:22) u i ( { f } ) into Ch ∞ ( P, { f } ). If u i +1 ( { f } ) = ∅ , then φ trivially holds for i + 1. Hence, wewill consider the case where v i +1 ( { f } ) (cid:54) = ∅ . Since v i +1 is a child of v i and since v i +1 is associated with some rule r i +1 ∈ P , it follows that there exists a homomorphism g from body ( r i +1 ) into v i ( { f } ) and v i +1 ( { f } ) = g s ( head ( r i +1 )).Due to g and due to h i , we know that there exists a homomorphism ψ = g ◦ h i from body ( r i +1 ) into Ch ∞ ( P, { f } )and, hence, a homomorphism ω from g s ( head ( r i +1 )) into ψ s ( head ( r i +1 )) mapping each value c occurring in dom ( g )into ( g ◦ h i )( c ) and n z into n (cid:48) z , for each existentially quantified variable z of r i +1 , where g ( z ) = n z and ψ ( z ) = n (cid:48) z .We distinguish the cases: • ψ s ( head ( r i +1 )) ∈ Ch ∞ ( P, { f } ). Due to h i and due to ω , it follows that h i +1 = h i ∪ ω is a homomorphism fromΓ (cid:22) v i +1 ( { f } ) into Ch ∞ ( P, { f } ). • ψ s ( head ( r i +1 )) (cid:54)∈ Ch ∞ ( P, { f } ). Then Ch ∞ ( P, { f } ) | = ψ s ( head ( r i +1 )) holds and hence, there exists a homo-morphism θ from ψ s ( head ( r i +1 )) into Ch ∞ ( P, { f } ). Due to h i , due to ω and due to θ , we can see that h i +1 = ( h i ∪ ω ) ◦ θ is a homomorphism from Γ (cid:22) v i +1 ( { f } ) into Ch ∞ ( P, { f } ).The above shows that φ holds for i + 1 concluding the proof of this direction.( ⇐ ) We want to show that for each fact f ∈ H ( P ), there exists a homomorphism from Ch ∞ ( P, { f } ) into Γ( { f } ).We use I i to denote Ch i ( P, { f } ) and chaseGraph i ( P, { f } ) to denote the chase graph corresponding to Ch i ( P, { f } ).The proof of this direction proceeds by showing that the following properties hold, for each i ≥ • φ . there exists a homomorphism h i from Ch i ( P, { f } ) into Γ( { f } ). • φ . for each f → r f → r · · · → r j f j +1 in chaseGraph i ( P, { f } ), a path of the form u → · · · → u j is in Γ,where for each 1 ≤ k ≤ j : u k is associated with r k and there exists a homomorphism from f k +1 into u k ( { f } ).For i = 0, since Ch ( P, { f } ) = { f } and since f ∈ Γ( { f } ) by definition, it follows that the inductive hypotheses φ and φ holds. For i + 1 and assuming that φ and φ hold for i the proof proceeds as follows. Let Σ r be the set of allactive triggers for each r ∈ P in I i . Let also∆ I = (cid:91) r ∈ P (cid:91) h ∈ Σ r h s ( head ( r )) (5)We distinguish the following cases: • I i | = I i ∪ ∆ I holds. Then the equivalent chase terminates and hence Ch i ( P, { f } ) = Ch ∞ ( P, { f } ). Since theinductive hypotheses φ and φ hold for i and due to the above, it follows that the inductive hypotheses will holdfor i + 1. 19 I i | = I i ∪ ∆ I does not hold. Then, for each rule r ∈ P for which Σ r (cid:54) = ∅ and for each h ∈ Σ r , the chase graph chaseGraph i +1 ( P, { f } ) will include an edge h ( body ( r )) → r h s ( head ( r )). Due to the steps in lines 4–6, we knowthat Γ includes a node v associated with r ( ∗ ). We have the following two subcases: – There is no edge of the form f (cid:48) → r (cid:48) h ( body ( r )) in chaseGraph i +1 ( P, { f } ), for some r (cid:48) ∈ P . Then, it followsthat h ( body ( r )) = f . Due to the above, due to ( ∗ ) and due to Definition 5, it follows that the inductivehypotheses φ and φ hold for i + 1. – Otherwise. Since the inductive hypothesis φ holds for i and due to the steps in lines 7–9, it follows that anedge of the form v (cid:48) → v will be in Γ, where node v (cid:48) is associated with rule r (cid:48) . Furthermore, due to φ , thereexists a homomorphism g i from h ( body ( r )) into v (cid:48) ( { f } ). Due to g i , due to the fact that the edge v (cid:48) → v isin Γ and due to Definition 5, there exists a homomorphism g i +1 from h s ( head ( r )) into v ( { f } ). Finally, dueto the above, and since φ holds for i , it follows that h i +1 = h i ◦ g i +1 is a homomorphism from Ch i ( P, { f } )into Γ( { f } ). Hence, φ and φ hold for i + 1 concluding the proof of this direction and, consequently ofLemma 32.We are now ready to return to the proof of Theorem 10. Let Γ f be the EG computed in lines 2–10 for each f ∈ H ( P ). Since Algorithm 1 only adds new nodes and new edges to set of nodes and the set of edges of an input EG,it follows that V = (cid:91) ∀ f ∈H ( P ) ν (Γ f ) (6) E = (cid:91) ∀ f ∈H ( P ) (cid:15) (Γ f ) (7)Since for each base instance B of P and each fact f (cid:48) ∈ B , there exists a fact f ∈ H ( P ) and a bijective function g over the constants in C , such that g ( f (cid:48) ) = f and from Lemma 32, it follows that Γ f ( { f (cid:48) } ) ≡ Ch ∞ ( P, f (cid:48) ). Due to theabove, due to (6) and (7) and since each node in V has up to one incoming edge, it follows that for each base instance B of P (cid:91) F ∈ B Ch ∞ ( P, { F } ) ≡ (cid:91) F ∈ B G ( { F } ) (8)Due to Definition 5 and since each node in V has up to one incoming edge, it follows that for each base instance B of P we have (cid:91) F ∈ B G ( { F } ) = G ( B ) (9)Finally, due to Proposition 30, and due to (8) and (9), we have Ch ∞ ( P, B ) ≡ G ( B ), for each base instance B of P .The above concludes the proof of the first part of Theorem 10. Theorem
The execution time of Algorithm 1 for
FES programs is double exponential in the input program P .If the arity of the predicates in P is bounded, the execution time is (single) exponential. Proof.
We first show the first part of the theorem with a step-by-step argument.1. Consider some program P .2. The set H ( P ) is exponential in P .(a) Let Ary be the maximal arity of an extensional predicate occurring in P .(b) For an extensional predicate p occurring in P , there are at most Ary
Ary facts in H ( P ) defined over p .(c) Since P is a linear rule set and extensional predicates may only appear in the body of a rule, we have thatnumber of extensional predicates in P is at most | P | .(d) By (b) and (c): there are at most | P | × Ary
Ary facts in H ( P ).3. For every fact f ∈ H ( P ), we may have to add at most | Ch ( P, { f } ) | nodes to the graph tglinear ( P ). (Note that Ch ( P, { f } ) is defined since P is FES .) Therefore, this graph contains at most | Ch ( P, { f } ) | edges.4. From results in [38], the size of Ch ( P, { f } ) is double exponential in P .(a) By Proposition 30 in [38]: if Ch ( P, { f } ) is finite, then there exists a finite entailment tree T such that theset of atoms associated with T is a complete core.(b) In Algorithm 1 in [38], the authors describe how to construct the finite entailment tree for P and { f } .20c) The complexity of this algorithm, as well as the size of the output tree, is double exponential in the input P and { f } . Note the discussion right after Algorithm 1 in [38].5. By (2–4): the algorithm tglinear ( P ) runs in double exponential in P .To show that the procedure tglinear ( P ) runs in single exponential time when ( ∗ ) the arity of the predicates in P is bounded, we can show that the size of Ch ( P, { f } ) is (single) exponential in P if ( ∗ ). In fact, this claim alsofollows from then results in [38]. Namely, if ( ∗ ), then entailment trees for P and f (as they are defined in [38]) are ofpolynomial depth because the number of “sharing types” for P is polynomial. Therefore, the size of these trees isexponential and so is the size of Ch ( P, { f } ). Lemma
Let P be a linear program, G be an EG for P and u, v ∈ ν ( G ) . Then, there exists a preservinghomomorphism from u ( B ) into v ( B ) for each base instance B , iff there exists a preserving homomorphism from u ( { f } ) into v ( { f } ) , for each fact f ∈ H ( P ) . Proof. ( ⇒ ) This direction trivially holds.( ⇐ ) We want to show that if there exists a preserving homomorphism from u ( { f } ) into v ( { f } ), for each fact f ∈ H ( P ), then there exists a preserving homomorphism from u ( B ) into v ( B ), for each base instance B . The proofworks by contradiction. Suppose that there exists a base instance B (cid:48) , such that there does not exist a preservinghomomorphism from u ( B (cid:48) ) into v ( B (cid:48) ). Since there does not exist a preserving homomorphism from u ( B (cid:48) ) into v ( B (cid:48) )and due to Proposition 31, it follows that there does not exist a preserving homomorphism from (cid:83) F (cid:48) ∈ B (cid:48) u ( { F (cid:48) } ) into (cid:83) F (cid:48) ∈ B (cid:48) v ( { F (cid:48) } ).Next we show that there exists some F (cid:48) ∈ B (cid:48) , so that there does not exist a preserving homomorphism h F (cid:48) from u ( { F (cid:48) } ) into v ( { F (cid:48) } ). The proof proceeds as follows. Suppose by contradiction that there exists a preservinghomomorphism h F (cid:48) from u ( { F (cid:48) } ) into v ( { F (cid:48) } ), for each F (cid:48) ∈ B (cid:48) , but there does not exist a preserving homomorphism h B (cid:48) from u ( { B (cid:48) } ) into v ( { B (cid:48) } ). The above assumption, will be referred to as Assumption ( A ). By definition,we know that a preserving homomorphism from u ( { B (cid:48) } ) into v ( { B (cid:48) } ) maps each value c either (i) to itself if c was a schema constant or a null occurring in ( G ( B (cid:48) ) \ G (cid:23) u ( B (cid:48) )) ∩ G (cid:23) u ( B (cid:48) ) or (ii) to a fresh null occurring ina single fact from v ( { F (cid:48) } ). Since for each F (cid:48) ∈ B (cid:48) , h F (cid:48) maps each schema constant and each null occurring in( G ( B (cid:48) ) \ G (cid:23) u ( B (cid:48) )) ∩ G (cid:23) u ( B (cid:48) ) to itself according to Assumption ( A ) and due to the above, it follows that there existtwo facts F (cid:48) , F (cid:48) ∈ B (cid:48) and a null n occurring in G (cid:23) u ( B (cid:48) ) \ G ( B (cid:48) ), such that h F (cid:48) ( n ) = m and h F (cid:48) ( n ) = m , where h F (cid:48) j is a preserving homomorphism from u ( { F (cid:48) j } ) into v ( { F (cid:48) j } ), for each 1 ≤ j ≤
2. However, the above leads to acontradiction, since in linear TGs each null from u ( { B (cid:48) } ) or v ( { B (cid:48) } ) occurs in only one fact. The above shows that ifthere does not exist a preserving homomorphism from (cid:83) F (cid:48) ∈ B (cid:48) u ( { F (cid:48) } ) into (cid:83) F (cid:48) ∈ B (cid:48) v ( { F (cid:48) } ), then there exists some F (cid:48) ∈ B (cid:48) , such that there does not exist a preserving homomorphism from u ( { F (cid:48) } ) into v ( { F (cid:48) } ). The proof proceedsas follows.Since for each F (cid:48) ∈ B (cid:48) , there exists a bijective function g over the constants in C and an instance f ∈ H ( P ), suchthat g ( F (cid:48) ) = f , it follows that there does not exist a preserving homomorphism from u ( { f } ) into v ( { f } ), for some f ∈ H ( P ). This leads to a contradiction. Hence, there exists a preserving homomorphism from u ( B ) into v ( B ), foreach base instance B and thus, Lemma 13 holds. Theorem
For a TG G for a linear program P , minLinear ( G ) is a TG for P . Proof.
Recall from Definition 14 that minLinear ( G ) results from G after applying the following step until reachinga fixpoint: (i) find a pair of nodes u and v with u being dominated by v ; (ii) remove v from ν ( G ); and (iii) add anedge v → j u (cid:48) , for each edge u → j u (cid:48) from (cid:15) ( G ). Let G i be the EG computed at the beginning of the i -th step of thisiterative process with G = G . In order to show that minLinear ( G ) is a TG for P , we need to show that the followingproperty holds for each BCQ Q entailed by ( P, B ): G i ( B ) | = Q ( ∗ )We can see that ( ∗ ) holds i = 0, since G = G . For i + 1 and assuming that ( ∗ ) holds for i ≥ q from Q into G i ( B ). Furthermore, let C and C be two conjuncts, suchthat Q = C ∧ C and q maps C into G (cid:23) u ( B ) and C into G ( B ) \ G (cid:23) u ( B ). Due to the above, it follows that q mapseach variable occurring both in C and C either to a constant or to a null occurring in ( G ( B ) \ G (cid:23) u ( B )) ∩ G (cid:23) u ( B ).From Definition 14 we know that (i) there exists a pair of nodes u, v ∈ ν ( G i ) with u being dominated by v and that(ii) the graph Γ i +1 that results from G i (cid:23) v after adding an edge v → u (cid:48) , for each edge u → u (cid:48) in (cid:15) ( G i ), is a subgraphof G i +1 . Since q maps each variable occurring both in C and C either to a constant or to a null occurring in( G ( B ) \ G (cid:23) u ( B )) ∩ G (cid:23) u ( B ) it follows that ∗ holds for i + 1 if the following holds: Lemma
There exists a homomorphism g from G i (cid:23) u ( B ) into Γ i +1 ( B ) so that g ( c ) = h ( c ) , for each c ∈ dom ( h ) . u ( B )into v ( B ), for each base instance B and (ii) all subgraphs rooted at each child of u are copied below v .From Lemma 33 and since Γ i +1 is a subgraph of G i +1 , we have that there exists a homomorphism ( q ◦ g ) from Q into G i +1 ( B ) concluding the proof of ( ∗ ) for i + 1 and hence the proof of Theorem 10. E. PROOFS FOR RESULTS IN SECTION 5.1
Below, we recapitulate the notion of the answers of non-Boolean CQs on a KB. The answers to a CQ Q on a KB ( P, B ), denoted as ans ( Q, P, B ), is the set of tuples that are answers to each model of (
P, B ), i.e., { t | t ∈ Q ( I ) , for each model I of ( P, B ) } . Lemma
Let G be an EG for a Datalog program P and B be a base instance of P . Then for each v ∈ ν ( G ) wehave: v ( B ) includes exactly a fact A ( t ) with A being the head predicate of rule ( v ) , for each answer t to the EG-rewritingof v on B . Proof.
Let R be the predicate in the body of the characteristic query of v . In order to prove Lemma 18, we haveto show that t is an answer to rew ( v ) on B iff R ( t ) ∈ v ( B ). The proof is based on (i) the correspondence between therewriting process of Definition 17 and the query rewriting algorithm from [29], called XRewrite and (ii) the correctnessof
XRewrite . In particular, the steps of the proof are as follows. First, we compute a new set of rules P ∗ by rewritingthe rules associated with the nodes in G (cid:22) v . This rewriting process is described in Definition 34. Then, we establishthe relationship between the rew v and the rewritings computed by XRewrite . In particular, Lemma 36 shows thatDefinition 17 and
XRewrite result in the same rewritings modulo bijective variable renaming, when the latter isprovided with P ∗ and a rewriting of the characteristic query of v denoted as Q ∗ . A direct consequence of Lemma 36is that XRewrite terminates when provided with P ∗ and Q ∗ . In order to show our goal, we make use of the aboveresults as well as of Lemma 37 .Below, we describe XRewrite . Given a CQ Q and a set of rules P , XRewrite computes a rewriting Q r of Q sothat for any null-free instance I , the answers to Q r on ( P, I ) coincide with the answers to Q on ( P, I ). We describehow
XRewrite ( P, Q ) works when P is Datalog. At each step i , XRewrite computes a tree T i , where each node κ isassociated with a CQ denoted as query ( κ ). When i = 0, T includes a single root node associated with Q . When i >
0, then T i is computed as follows: for each leaf node κ in ν ( T i − ), each atom β occurring in the body of query ( κ )and each rule r ∈ P whose head unifies with β , XRewrite :1. computes a new query Q (cid:48) by (i) computing the MGU θ of { head ( r ) , α } , (ii) replacing α in the body of query ( κ )with body ( r ) and (iii) applying θ on the resulting query;2. adds a new node κ (cid:48) in T i and associates it with Q (cid:48) ; and3. adds the edge κ ( α,r ) −−−→ κ (cid:48) to T i .Notice that XRewrite also includes a factorization step, however, this step is only applicable in the presence ofexistential rules.We now introduce some notation related to Definition 17. We denote by rew i ( v ) the CQ at the beginning of the i -th iteration of the rewriting step of Definition 17 with rew ( v ) being equal to the characteristic query of v . Weuse rew i ( v ) α i −→ rew i +1 ( v ) to denote that rew i +1 ( v ) results from rew i ( v ) after choosing the atom α i from the body of rew i ( v ) at the beginning of the i -th rewriting step.Below, we describe a process that computes a new ruleset by rewriting the rules associated with the nodes of an EG. Definition
Let G be an EG of a program P with a single leaf node. Let π be a mapping associating each edge ε ∈ (cid:15) ( G ) ∪ {(cid:5)} with (cid:5) denoting the empty edge, with a fresh predicate π ( (cid:15) ) . We denote by ρ ( G, π ) the rules obtainedfrom the rules associated with the nodes in G after rewriting them as follows: replace each A ( X ) that is either • i. the i -th intensional atom in the body of rule ( v ) or the head atom of rule ( u ) and ε ·· = u → i v in (cid:15) ( G ) ; or • ii. the head atom of rule ( κ ) with κ being the leaf node of G ,by A ∗ ( X ) , where A ∗ = π ( ε ) when (i) holds, or A ∗ = π ( (cid:5) ) when (ii) holds.For a node u ∈ ν ( G ) , we denote by r πu the rule from ρ ( G, π ) that results after rewriting rule ( u ) . From Definition 34 we can see that the following holds:
Corollary
For an EG G of a program P with a single leaf node, a mapping π associating each edge ε ∈ (cid:15) ( G ) ∪ {(cid:5)} with a fresh predicate and two rules (cid:37) and (cid:37) from ρ ( G, π ) , we have: the i -th body atom of (cid:37) has the same predicate with the head atom of (cid:37) only if (cid:37) j is of the form r πu j , for ≤ j ≤ and u → i u is in (cid:15) ( G ) . Let Γ = G (cid:22) v and n be the depth of Γ. Let π be a mapping from edges to predicate as defined above. Let P ∗ be the rules in ρ (Γ , π ). Let R be the predicate occurring in the head of rule ( v ) and let head ( rule ( v )) = R ( Y ). Let22 ∗ = π ( (cid:5) ). Let Q ( Y ) ← R ( Y ) and Q ∗ ( Y ) ← R ∗ ( Y ). Let T i be the tree computed at the end of the i -th iteration of XRewrite ( P ∗ , Q ∗ ).Below, we establish the relationship between the rew v and the rewritings computed by XRewrite . Lemma
For each branch κ (cid:15) −→ . . . (cid:15) n −→ κ n +1 with (cid:15) i = ( β i , (cid:37) i ) and κ i is a node of depth i in T n +1 , there existsa sequence rew ( v ) α −−→ . . . α n −−→ rew n +1 ( v ) (10) such that rew n +1 ( v ) equals query ( κ n +1 ) modulo bijective variable renaming. The proof of Lemma 36 follows from: (i) Q ( Y ) ← R ( Y ) and Q ∗ ( Y ) ← R ∗ ( Y ), where R ∗ = π ( (cid:5) ), (ii) the correspondencebetween the rewriting steps (1)–(3) of XRewrite and the rewriting process of Definition 17 and (iii) Corollary 35.From Lemma 36 and since rew n +1 ( v ) includes only EDP -atoms, we have that:
XRewrite ( P ∗ , Q ∗ ) terminates after n + 1 iterations. Furthermore, since XRewrite terminates and due to its correctness we have: R ∗ ( t ) ∈ Ch n ( P ∗ , B ) iff t is an answer to XRewrite ( P ∗ , Q ∗ ) on B .Due to Corollary 35, we also have: Lemma
For each base instance B , the following inductive property holds for each ≤ i ≤ n + 1 : • φ . S ( t ) ∈ u ( B ) , with u being a node of depth i in Γ iff S ∗ ( t ) ∈ Ch i ( P ∗ , B ) , where S ∗ is the predicate of the headatom of r πu . We now establish the correspondence between v ( B ) and the answers to rew ( v ) on B . From Lemma 37, we have: R ∗ ( t ) ∈ Ch n ( P ∗ , B ) iff R ( t ) ∈ v ( B ). Since R ∗ ( t ) ∈ Ch n ( P ∗ , B ) iff t is an answer to XRewrite ( P ∗ , Q ∗ ) on B , it followsthat R ( t ) ∈ v ( B ) iff t is an answer to XRewrite ( P ∗ , Q ∗ ) on B . Since t is an answer to XRewrite ( P ∗ , Q ∗ ) on B iff t isan answer to rew ( v ) on B and due to the above, it follows that: t is an answer to rew ( v ) on B iff R ( t ) ∈ v ( B ). Theabove completes the proof of Lemma 18. Lemma
For each EG G for a Datalog program P and each base instance B of P , G ( B ) = minDatalog ( G )( B ) . Proof.
Let G i be the EG at the beginning of the i -th iteration of minDatalog ( G ) with G = G . We show that thefollowing property holds for each i ≤ • φ . G ( B ) = minDatalog ( G i )( B ).For i = 0, φ trivially holds, since G = G . For i + 1 and assuming that φ holds for i ≤
0, we have. Let u , v be apair of nodes in ν ( G i ), such that (i) v ’s depth is not less than u ’s depth, (ii) the predicates of head ( rule ( v )) and of head ( rule ( u )) are the same and (iii) the EG-rewriting of v is contained in the EG-rewriting of u . In order to showthat the inductive property for φ , it suffices to show that for each node w ∈ ν ( G i ) for which v → j w ∈ (cid:15) ( G i ) holds forsome j , and each base instance B of P , w ( B ) is the same both in G i ( B ) and in G i +1 ( B ). However, this holds since(i) for each v → j w ∈ (cid:15) ( G i ), we have u → j w ∈ (cid:15) ( G i +1 ), (ii) R ( t ) ∈ v ( B ) implies R ( t ) ∈ u ( B ) and G i (cid:22) u = G i +1 (cid:22) u . Theabove shows that φ holds for i + 1 and concludes the proof of Lemma 38. Theorem If G is a TG for a Datalog program P , then minDatalog ( G ) is a minimum size TG for P . Proof.
Part I . The proof follows from Lemma 38.
Part II . First, we can see that the following holds due to Definition 19:
Corollary
For a TG G of a Datalog program P , there exists no two nodes u, v in minDatalog ( G ) satisfyingthe following: (i) u and v define the same predicate A and (ii) rew ( u ) ⊆ rew ( v ) . The proof works by contradiction. Let P be a Datalog program, G be a TG for P and Γ = minDatalog ( G ). Supposeby contradiction that there exists a TG Γ (cid:48) for P with ν (Γ) > ν (Γ (cid:48) ). From Lemma 18 we know that for each set ofnodes u , . . . , u m from ν (Γ) defining a predicate A , there exists a set of nodes u (cid:48) , . . . , u (cid:48) n from ν (Γ (cid:48) ) defining also A ,such that the following holds: rew ( u ) ∪ · · · ∪ rew ( u m ) ≡ rew ( u (cid:48) ) ∪ · · · ∪ rew ( u (cid:48) n ) (11)Since ν (Γ) > ν (Γ (cid:48) ), we know that there exist a set u , . . . , u m and a set u (cid:48) , . . . , u (cid:48) n so that m > n . Since (11) holds,we know from [52] that the following hold: • for each rew ( u i ) with 1 ≤ i ≤ m , there exists a rew ( u (cid:48) j ) with 1 ≤ j ≤ n , such that rew ( u i ) ⊆ rew ( u (cid:48) j ); • for each rew ( u (cid:48) j ) with 1 ≤ j ≤ n , there exists a rew ( u (cid:96) ) with 1 ≤ (cid:96) ≤ n , such that rew ( u (cid:48) j ) ⊆ rew ( u (cid:96) ).Since m > n , it follows that there exist i , i with 1 ≤ i , i ≤ m and an (cid:96) with 1 ≤ (cid:96) ≤ n , such that rew ( u i ) ⊆ rew ( u (cid:48) (cid:96) )and rew ( u i ) ⊆ rew ( u (cid:48) (cid:96) ) hold. Below, we show how we reach a contradiction. We consider the following cases: We say that a node u in a TG defines a predicate A if the predicate of head ( u ) is A .23 there exists an i with 1 ≤ i ≤ m and i (cid:54) = i , i , such that rew ( u (cid:48) (cid:96) ) ⊆ rew ( u i ). From the above, it follows that rew ( u i ) ⊆ rew ( u i ) and rew ( u i ) ⊆ rew ( u i ) leading to a contradiction due to Corollary 39. • rew ( u i ) ⊆ rew ( u (cid:48) (cid:96) ). From the above, it follows that rew ( u i ) ⊆ rew ( u i ) leading again to a contradiction due toCorollary 39.The above completes the proof of Theorem 20. Theorem
For a Datalog program P and a TG G for P , deciding whether G is a TG of minimum size for P isco-NP-complete. Proof.
Membership . We show that deciding wether G is a TG of P not of minimum size is in NP. By Definition 19and Theorem 20, G is a TG of P not of minimum size iff there exists a pair of vertices u and v in G satisfyingthe conditions in Definition 19 for which rew ( v ) ⊆ rew ( u ) (remember that the last condition holds iff there exists ahomomorphism from rew ( u ) to rew ( v )). Hence, to disprove that G is a TG of P of minimum size, it is sufficient toguess such nodes u and v , guess the homomorphism from rew ( u ) to rew ( v ) (observe that the size of rew ( u ) and rew ( v )is polynomial), then compute rew ( u ) and rew ( v ) (feasible in deterministic polynomial time), and then check that theguessed homomorphism is correct (feasible in deterministic polynomial time). This procedure is feasible in NP. Hardness . We show the co-NP-hardness of the problem by showing the NP-hardness of its complement. Thereduction is from the NP-complete problem of query containment in relational DBs: given two CQs Q ( X ) and Q ( X ) for a relational DB, decide whether Q ( X ) ⊆ Q ( X ). Let the queries be Q ( X ) ← a i ( X i ) , . . . , a i n ( X i n ) and Q ( X ) ← a j ( X j ) , . . . , a j m ( X j m ).We now describe the reduction. Consider the following program P and TG G .The rules of P are obtained as follows. Let D = { a k , . . . , a k (cid:96) } be the set of all the distinct predicates from { a i , . . . , a i n } . For each of the predicates a k t in D , there is a rule a k t ( X k t ) → A k t ( X k t ) in P . In P there are also therules A i ( X i ) , . . . , A i n ( X i n ) → Q ( X ) and a j ( X j ) , . . . , a j m ( X j m ) → Q ( X ).The TG G is as follows. There is a node v k t associated with each of the rules a k t ( X k t ) → A k t ( X k t ); there is anode v associated with the rule A i ( X i ) , . . . , A i n ( X i n ) → Q ( X ); and there is a node u associated with the rule a j ( X j ) , . . . , a j m ( X j m ) → Q ( X ). The edges of G are: for each 1 ≤ s ≤ n , there is an edge labelled s to node v fromthe node v k t such that the predicate of head ( rule ( v k t )) is A i s .We show the G is a TG of minimum size for P iff Q ( X ) ⊆ Q ( X ).First, observe that the predicates of the rules associated with nodes v k t are all distinct, and they differ from thepredicate of the heads of the rules associated with u and v . Hence none of the nodes v k t can be removed from G inthe minimization process. Nodes u and v are the only nodes in G associated with rules with the same head predicate.The depth of u is 0, while the depth of v is 1. Hence, v is the only node that can be removed in the minimizationprocess. Therefore, G is not of minimum size iff v can be removed. The node v can be removed iff rew ( v ) ⊆ rew ( u ),and hence, by the definition of P , iff Q ( X ) ⊆ Q ( X ). Theorem
For a Datalog program P and a base instance B , TGmat ( P, B ) = Ch ( P, B ) . Proof.
We first show that
Claim
For each node v ∈ ν ( G ) and each instance I , we have v ( B, I ) = v ( B ) \ I (12) Proof.
Let A ( X ) be the head atom of rule ( v ) and let Q ( Y ) ← (cid:86) ni =1 f i be the EG-rewriting of v .Recall from Lemma 18 that for each base instance of B of P we have: v ( B ) includes exactly a fact A ( t ) for eachanswer t to the EG-rewriting of v on B .Now consider any m ≥ f i , . . . , f i m from the body of Q whose variables include all variables in Y . Consideralso the query Q (cid:48) ( Y ) ← f i ∧ · · · ∧ f i m . From [17], it follows that Q is contained in Q (cid:48) , i.e., for each base instance B ,each answer t to Q on B is an answer to Q (cid:48) on B . From the above, we have: each t , for which A ( t ) ∈ v ( B ) holds, isalso an answer to Q (cid:48) ( Y ) ← f i ∧ · · · ∧ f i m on B . We refer to this conclusion as ( ∗ ).Since step (2) of Definition 23 considers each homomorphism h for which (i) h ( X ) is an answer to Q (cid:48) on B and (ii) A ( h ( X )) (cid:54)∈ I and due to ( ∗ ), it follows that Claim 40 holds.Let I k be the instance computed the beginning of the k -th iteration of the steps in lines 2–8 of Algorithm 2. Then,using Claim 40, Theorem 26 and Lemma 38, we can easily show that for each k ≥
0, the following property holds: • φ . I k = Ch k ( P, B )The above concludes the proof of Theorem 24.
F. ADDITIONAL EXAMPLES xample We show how reasoning over the TG G from Figure 1 proceeds for the base instance B = { r ( c , c ) } .Reasoning starts from the root nodes u and u , which are associated with the rules r and r , respectively. Sincethere exists a homomorphism h = { X (cid:55)→ c , Y (cid:55)→ c } from body ( r ) into B and from body ( r ) into B , we have u ( { f } ) = { R ( c , c ) } (13) u ( { f } ) = { T ( c , c , n ) } (14) where n is a null. Then, since there exists an edge from u to u and since u is associated with r , we compute allhomomorphisms from body ( r ) into u ( B ) . Since there exists a homomorphism h = { X (cid:55)→ c , Y (cid:55)→ c } from body ( r ) into u ( B ) , we have u ( { f } ) = { T ( c , c , c ) } (15) Since there is no other node, reasoning stops.
Example
We demonstrate the notion of preserving homomorphisms introduced in Definition 12.Consider the facts f = r ( c , c ) and f = r ( c , c ) from the set H ( P ) . By applying Definition 5 for the baseinstance { f } , we have u ( { f } ) , u ( { f } ) and u ( { f } ) as in (13) , (14) and (15) . Similarly, by applying Definition 5for the base instance { f } , we have u ( { f } ) = { R ( c , c ) } (16) u ( { f } ) = { T ( c , c , n ) } (17) u ( { f } ) = { T ( c , c , c ) } (18) Above, n is a null. We can see that there exists a preserving homomorphism from u ( { f } ) into u ( { f } ) mapping n to c , since the null n is not shared among the facts occurring in the instances associated with u and u . For thesame reason, there exists a preserving homomorphism from u ( { f } ) into u ( { f } ) mapping n to c . Hence accordingto Lemma 13, there exists a preserving homomorphism from u ( B ) into u ( B ) for each base instance B . Example
We demonstrate the computation of EG-rewritings introduced in Definition 17.Consider the rules r ( X , Y , Z ) → T ( X , X , Y ) ( r ) T ( X , Y , Z ) → R ( Y , Z ) ( r ) where r is the only extensional predicate. Consider now an EG having nodes u and u , where u i is associated with r i for each ≤ i ≤ , and the edge u → u .To compute the EG-rewriting rew ( u ) of u we first form the query Q ( Y , Z ) ← R ( Y , Z ) (19) and associate the atom R ( Y , Z ) with u . The following steps take place in the first iteration of the rewriting algorithm.First, since R ( Y , Z ) is the only intensional atom in the query we have α = R ( Y , Z ) . Then, according to step (ii)and since the node u is associated with R ( Y , Z ) , we compute the MGU θ of the set { head ( u ) , R ( Y , Z ) } . Wehave θ = { Y → Y , Z → Z } , since head ( u ) = R ( Y , Z ) . By applying the step (iii), the query in (19) becomes Q ( Y , Z ) ← T ( X , Y , Z ) (20) In step (iv) we associate the fact T ( X , Y , Z ) with node u due to the edge u → u .In the second iteration of the rewriting algorithm, we have α = T ( X , Y , Z ) . Since the fact T ( X , Y , Z ) isassociated with node u , in step (ii) we compute the MGU θ of the set { head ( u ) , T ( X , Y , Z ) } . We have θ = { X → Y , X → Y , Y → Z } . In step (iii) we replace α = T ( X , Y , Z ) in (20) with body ( u ) = r ( X , Y , Z ) andapply θ to the resulting query. The query in (20) becomes Q ( Y , Z ) ← r ( Y , Z , Z ) (21) Since there is no incoming edge to u , we associate no node to the fact r ( Y , Z , Z ) . The algorithm then stops, sincethere is no extensional fact in (21) . The EG-rewriting of u is the query shown in (21) . Example
We demonstrate the notion of compatible nodes introduced in Definition 9, as well as the procedurefor computing instance-dependent TGs from Section 4. \ r u \ r u \ r u \ r u \ r u \ r
12 1 112 22
Figure 3: Part of the graph from Example 44.
Consider the program P a ( X ) → A ( X ) ( r ) r ( X, Y ) → R ( X, Y ) ( r ) R ( X, Y ) ∧ A ( Y ) → A ( X ) ( r ) R ( X, Y ) ∧ R ( Y, Z ) → A ( X ) ( r ) where a and r are extensional predicates. Figure 3 shows part of the graph computed up to level 3. Next to each node,we show the rule associated with it. For example, node u is associated with rule r and node u is associated withrule r .When k = 1 , G includes two nodes, one associated with rule r ( u ) and one associated with rule r ( u ). When k = 2 , r has only one -compatible combination of nodes. That is ( u , u ) . Hence, the technique will add one freshnode u , associated with r and will add the edges u → u and u → u . The -compatible combination of nodes for r is ( u , u ) . Hence, the technique will add one fresh node u , associated with r and will add the edges u → u and u → u .When k = 3 , r has the following -compatible combinations of nodes: ( u , u ) and ( u , u ) . For each such -compatible combinations of nodes, the algorithm adds a fresh node and associates it with r . For k = 3 , the -compatiblecombinations of nodes for r are: ( u , u ) , ( u , u ) , ( u , u ) , ( u , u ) , ( u , u ) , ( u , u ) , ( u , u ) , ( u , u ) . Again, foreach such combination of nodes, the algorithm adds a fresh node and associates it with r ..