Formulog: Datalog for SMT-Based Static Analysis (Extended Version)
1141Formulog: Datalog for SMT-Based Static Analysis
Extended Version ∗ AARON BEMBENEK,
Harvard University, USA
MICHAEL GREENBERG † , Pomona College, USA
STEPHEN CHONG,
Harvard University, USASatisfiability modulo theories (SMT) solving has become a critical part of many static analyses, includingsymbolic execution, refinement type checking, and model checking. We propose Formulog, a domain-specificlanguage that makes it possible to write a range of SMT-based static analyses in a way that is both close totheir formal specifications and amenable to high-level optimizations and efficient evaluation.Formulog extends the logic programming language Datalog with a first-order functional language andmechanisms for representing and reasoning about SMT formulas; a novel type system supports the constructionof expressive formulas, while ensuring that neither normal evaluation nor SMT solving goes wrong. Ourcase studies demonstrate that a range of SMT-based analyses can naturally and concisely be encoded inFormulog, and that — thanks to this encoding — high-level Datalog-style optimizations can be automaticallyand advantageously applied to these analyses.CCS Concepts: •
Software and its engineering → Automated static analysis ; Domain specific lan-guages ; Constraint and logic languages .Additional Key Words and Phrases: Datalog, SMT solving
Satisfiability modulo theories (SMT) solving provides a way to reason logically about commonprogram constructs such as arrays and bit vectors, and as such has become a key component ofmany static analyses. For example, symbolic execution tools use SMT solving to prune infeasibleexecution paths [Cadar et al. 2008; Cadar and Sen 2013]; type checkers use it to prove subtypingrelations between refinement types [Bierman et al. 2012; Rondon et al. 2008]; and model checkersuse it to abstract program states [Cimatti and Griggio 2012; McMillan 2006]. This paper presentsFormulog, a domain-specific language for writing SMT-based static analyses. Formulog makes itpossible to concisely encode a range of SMT-based static analyses in a way that is close to theirformal specifications. Furthermore, Formulog is designed so that analyses implemented in it areamenable to efficient evaluation and powerful, high-level optimizations, including parallelizationand automatic transformation of exhaustive analyses into goal-directed ones.Formulog is based on Datalog, a logic programming language used to implement static analysesranging from points-to analyses [Bravenboer and Smaragdakis 2009; Whaley and Lam 2004] todecompilers [Flores-Montoya and Schulte 2020; Grech et al. 2019] to security analyses [Grech et al.2018; Guarnieri and Livshits 2009; Jordan et al. 2016; Livshits and Lam 2005; Tsankov et al. 2018]. Em-bodying the principle of separating the logic of a computation from the control necessary to performthat computation [Kowalski 1979], Datalog frees analysis designers from low-level implementationdetails and enables them to program at the level of specifications (such as formal inference rules).This leads to concise implementations [Whaley et al. 2005] that can be easier to reason about andimprove at the algorithmic level compared to analyses in more traditional languages [Smaragdakisand Bravenboer 2011]. Datalog-based analyses can be fast and scalable, even outperforming the ∗ This article extends one published in PACMPL [Bembenek et al. 2020] with technical appendices. † Work done while on sabbatical at Harvard University.Authors’ addresses: Aaron Bembenek, Harvard University, USA, [email protected]; Michael Greenberg, PomonaCollege, USA, [email protected]; Stephen Chong, Harvard University, USA, [email protected]. a r X i v : . [ c s . P L ] O c t non-Datalog state-of-the-art [Bravenboer and Smaragdakis 2009]. Indeed, Datalog’s high-levelnature makes it amenable to high-level optimizations, such as parallelization [Scholz et al. 2016]and synthesis of goal-directed analyses from exhaustive ones [Reps 1995].However, despite the appeal of Datalog for static analysis and the importance of SMT solvingin static analysis, until now there has not been a focused study of how to effectively extend thebenefits of Datalog to SMT-based analyses; our work bridges this gap.Formulog augments Datalog with an interface to an external SMT solver and a first-orderfragment of the functional language ML. It provides a library of constructors for building terms thatare interpreted as logical formulas when applied to special SMT operators; in the backend, theseoperators are implemented by calls to an external SMT solver. A Formulog program is essentially aset of ML-style function definitions and Datalog-style rules; both pieces can refer to each other andinvoke the SMT operators. As in Datalog, the goal of Formulog evaluation is to compute all possibleinferences with respect to the rules, which correspond to logical implications. Unlike Datalog, ruleevaluation might involve both ML evaluation and calls to an SMT solver.The way this design combines Datalog, ML, and SMT solving gives Formulog some desirableproperties. First, Formulog programs can use SMT solving the way it is used in SMT-based analyses.This results from the choice to represent SMT formulas as ML terms, and contrasts with theapproach of most prior work combining logic programming and constraint solving (where, e.g.,checking for formula validity is hard). Second, the combination of Datalog-style rules and ML-stylefunctions mirrors the combination of inference rules and helper functions commonly used inanalysis specifications, making it easier to translate formal analysis specifications into executablecode. This close correspondence between specification and implementation means that specification-level reasoning is still applicable to analysis implementations (and vice versa: unexpected behaviorin Formulog programs has revealed bugs in specifications). Third, because Formulog is based onDatalog, analyses written in it can be effectively optimized and evaluated via powerful Datalogalgorithms, making them competitive with analyses written in more mature languages.It takes care to fit Datalog, ML, and SMT solving together in a way that truly achieves theseproperties. Along these lines, part of our technical contribution is a novel bimodal type system thattreats terms appearing in SMT formulas more liberally than terms appearing outside of formulas,making it possible to construct expressive logical formulas, while still ensuring that neither concrete(i.e., Datalog/ML) evaluation nor SMT solving goes wrong.To test the practicality of Formulog, we implemented a fully featured, prototype Formulog runtimeand wrote three substantial SMT-based analyses in Formulog: a type checker for a refinement typesystem, a bottom-up points-to analysis for JVM bytecode, and a bounded symbolic evaluator fora subset of LLVM bitcode. Our implementations for the first two case studies are almost directtranslations of previously published formal specifications [Bierman et al. 2012; Feng et al. 2015];indeed, Formulog allowed us to program close enough to the specifications to uncover bugs in bothof them. Despite encoding complex analysis logic, each of our analyses is concise (no more than1.5K LOC). Furthermore, our Formulog-based implementations have acceptable performance, evenwhen compared against reference implementations running on more mature language platforms.In some cases, we actually achieve substantial speedups over the reference implementations.These performance results are possible only because Formulog’s design allows our runtime toautomatically and effectively apply high-level optimizations to Formulog programs. Our third casestudy makes this point emphatically. Due to automatic parallelization, our symbolic evaluatorachieves a speedup of 8 × over the symbolic execution tool KLEE [Cadar et al. 2008]. Moreover,this speedup increases to 12 × when we use the magic set transformation [Bancilhon et al. 1985;Beeri and Ramakrishnan 1991] to automatically transform our exhaustive symbolic evaluator into ormulog: Datalog for SMT-Based Static Analysis 141:3 Programs prog :: = 𝐻 ∗ Horn clauses 𝐻 :: = 𝑝 ( 𝑒 ∗ ) : − 𝑃 ∗ Premises 𝑃 :: = 𝐴 | ! 𝐴 Atoms 𝐴 :: = 𝑝 ( 𝑒 ∗ ) | 𝑒 = 𝑒 Expressions 𝑒 :: = 𝑋 | 𝑐 Variables 𝑋 ∈ Var
Constructors 𝑐 ∈ CtorVar
Predicates 𝑝 ∈ PredVar
Fig. 1. A Datalog program is a collection of Horn clauses that represent rules for making inferences. a goal-directed one that explores only paths potentially leading to assertion failures. That Datalogcan speed up analyses like points-to analysis is well established [Bravenboer and Smaragdakis2009; Whaley and Lam 2004]; that it can automatically scale symbolic evaluation is a novel result.In sum, this paper makes the following contributions: • the design of Formulog (Section 3), a domain-specific language for writing SMT-based staticanalyses that judiciously combines Datalog, a fragment of ML, and SMT solving; • a lightweight bimodal type system (Section 4) that mediates the interface between con-crete evaluation and SMT solving, enabling the construction of expressive formulas whilepreventing many kinds of runtime errors in both concrete evaluation and SMT solving; • a fully featured prototype and three substantial case studies (Section 5), showing that thedesign of Formulog can be the basis of a practical tool for writing SMT-based analyses; and • an evaluation of Formulog’s design in light of these case studies (Section 6), demonstratinghow careful design decisions make Formulog an effective medium for encoding a range ofSMT-based analyses in a way that is both close to their formal specifications and amenableto efficient evaluation and high-level optimizations. The starting point for Formulog is Datalog with stratified negation (Figure 1) [Apt et al. 1988;Gallaire and Minker 1978; Green et al. 2013; Przymusinski 1988; Van Gelder 1989]. A Datalogprogram is a collection of Horn clauses, where a clause 𝐻 consists of a head predicate 𝑝 ( 𝑒 ∗ ) and asequence of body premises 𝑃 . Each premise 𝑃 is either a positive atom 𝐴 or a negated atom ! 𝐴 . An atom 𝐴 has one of two forms: It is either a predicate symbol applied to a list of expressions, or thespecial equality predicate 𝑒 = 𝑒 . An expression 𝑒 is a variable 𝑋 or a nullary constructor 𝑐 , i.e., anuninterpreted constant. Each predicate symbol 𝑝 is associated with an extensional database (EDB)relation or an intensional database (IDB) relation. An EDB relation is tabulated explicitly through facts (clauses with empty bodies), whereas an
IDB relation is computed through rules (clauses withnon-empty bodies). A rule should be read as a universally quantified logical implication, with theconjunction of the body premises implying the predicate in the head. Datalog evaluation amountsto computing every possible inference with respect to these implications; the restriction to stratifiednegation (a relation cannot be defined, either directly or indirectly, by its complement) ensures thatthis can be done via a sequence of fixed point computations.Datalog has proven to be a natural and effective way to encode a range of static analyses [Braven-boer and Smaragdakis 2009; Flores-Montoya and Schulte 2020; Grech et al. 2019, 2018; Guarnieriand Livshits 2009; Jordan et al. 2016; Livshits and Lam 2005; Tsankov et al. 2018; Whaley and Lam2004]. EDB relations are used to represent the program under analysis; for example, EDB relationsmight encode a control flow graph (CFG) of the input program. The logic of the analysis is encodedusing rules that define IDB relations; these rules are fixed and do not depend on the program under analysis (which is already captured by the EDB relations). The Datalog program will compute thecontents of the IDB relations, which can be thought of as the analysis results.That being said, standard Datalog is a very restricted language and there are many other analysesthat cannot easily be encoded in it, if at all. Recent variants extend Datalog for analyses that operateover interesting lattices [Madsen et al. 2016; Szabó et al. 2018]. Following in this spirit, Formulogproposes a way to support analyses that need access to SMT solving.
The design of Formulog is driven by three main desiderata. First, it should be possible to implementSMT-based static analyses in a form close to their formal specifications. Second, it should be easyto use logical terms the way that they are commonly used in many analyses. For example, analysesoften need to create formulas about entities such as arrays and machine integers, test those formulasfor satisfiability or validity, and generate models of them. Third, Formulog programs should still beamenable to powerful Datalog optimizations and evaluable using scalable Datalog algorithms.Section 6 demonstrates how the design of Formulog largely meets these desiderata. Here, wegive a warm-up example of Formulog, provide an overview of its language features, discuss howthese features support logical formulas, and conclude with its operational semantics.
To give the flavor of Formulog-based analyses, this section presents a bounded symbolic evaluatorfor CFGs of a simple imperative language (Figures 2 and 3). A symbolic evaluator [King 1976]interprets a program in which some values are unknown. When the evaluator reaches a conditionthat depends on one of these symbolic values, it forks into two processes, one in which thecondition is assumed to be true and one in which it is assumed to be false. At this point, it can avoidexploring an impossible path by checking whether the condition along that branch is consistentwith the conditions encountered so far during execution (which are known collectively as the “pathcondition”). Our symbolic evaluator uses fuel to bound the depth of its execution.We use algebraic data types to represent the input language to the evaluator (lines 1-9). Valuesare formulas representing 32-bit vectors (i.e., terms of type i32 smt ), and operands are either valuesor variables. Our input language has binary operations, conditional jumps, and fail instructions(indicating that control flow has reached, e.g., an assertion failure). The program-to-analyze is givenby two EDB (i.e., input ) relations (lines 11-12). The relation node_has_inst maps a CFG node to thecorresponding instruction, and the relation node_has_succ relates it to its fall-through successor.The state of the symbolic evaluator (line 14) is a record with a store mapping variables to values,and a path condition; an initial state (line 16) consists of an empty map and the true path condition. ML-style functions are used to update and query the state. The function update_store (lines 18-19)updates a store binding, while the function update_path_cond (lines 21-24) adds another conjunctto the path condition, returning none in the case that the resulting path condition is unsatisfiable.The built-in operator is_sat queries an external SMT solver for the satisfiability of its argument(an SMT proposition). The function operand_value (lines 26-34) looks up the value of an operandin the state, returning a pair of the value and a (possibly) new state. In the case that the operand isa value or a mapped variable, the relevant value is returned with the input state. In the case thatthe operand is a variable that is not in the store, the function returns a fresh symbolic bit vector( ` ` ) with an updated state mapping the variable to that value.The symbolic evaluator itself is defined through two IDB (i.e., output ) relations (lines 36-37). Therelation reached consists of tuples ( 𝑛𝑜𝑑𝑒, 𝑠𝑡, 𝑓 𝑢𝑒𝑙 ) that indicate that the symbolic evaluator has We omit the definitions for maps; we use association lists with the standard operations empty_map , get , and put . ormulog: Datalog for SMT-Based Static Analysis 141:5 type val = i32 smt type var = string type operand = o_val ( val ) | o_var ( var ) type binop = b_add | b_mul | b_eq | b_lt type node = i32 type inst = | i_binop (var , binop , operand , operand ) | i_jnz ( operand , node ) (* jump if the operand is not zero *) | i_fail input node_has_inst ( node , inst ) input node_has_succ ( node , node ) type state = { store : (var , val ) map ; path_cond : bool smt ; } fun initial_state : state = { store = empty_map ; path_cond = ` true ` ; } fun update_store (x: var , v: val , st : state ) : state = { st with store = put (x , v , store ( st )) } fun update_path_cond (x: bool smt , st : state ) : state option = let y = path_cond ( st ) in let z = ` x /\ y ` in if is_sat (z) then some ({ st with path_cond =z }) else none fun operand_value (o: operand , st : state ) : val * state = match o with | o_val (v) => (v , st ) | o_var (x) => match get (x , store ( st )) with | some (v) => (v , st ) | none => let v = ` ` in (v , update_store (x , v , st )) end end Fig. 2. A combination of types and input relations represent the program under evaluation; ML-style functionsare defined for manipulating the complex types that represent evaluator state. reached a node 𝑛𝑜𝑑𝑒 with state 𝑠𝑡 and the amount of fuel 𝑓 𝑢𝑒𝑙 . The relation failed consists of pairs ( 𝑛𝑜𝑑𝑒, 𝑠𝑡 ) that indicate that the evaluator has reached a failure node 𝑛𝑜𝑑𝑒 with the state 𝑠𝑡 .Before defining these relations, we define some helper functions. The function decr (line 39)decrements an integer if it is greater than zero, and else returns none ; it is used to decrementthe amount of fuel. The function do_binop (lines 41-52) is used to perform a binary operationon two operands. It looks up the value of those operands in the given state, and then returns abit-vector-valued SMT formula representing the binary operation applied to those values. It alsoreturns a state, since the resolution of the operands might have resulted in an updated state (ifone of the operands is an unmapped variable). The locally scoped function b2i converts an SMT output reached ( node , state , i32 option ) output failed ( node , state ) fun decr (n: i32 ) : i32 option = if n > 0 then some (n - 1) else none fun do_binop (b: binop , op1 : operand , op2 : operand , st : state ) : val * state = let (v1 , st1 ) = operand_value (op1 , st ) in let (v2 , st2 ) = operand_value (op2 , st1 ) in let fun b2i (x: bool smt ) : i32 smt = ` ` in let v = match b with | b_add => ` bv_add (v1 , v2) ` | b_mul => ` bv_mul (v1 , v2) ` | b_eq => b2i ( ` v1 ` ) | b_lt => b2i ( ` bv_slt (v1 , v2 ) ` ) end in (v , st2 ) reached (0 , initial_state , some (10)). (* start with 10 units of fuel *) reached ( Next , St2 , decr (N )) :- reached ( Curr , St , some (N)) , node_has_inst ( Curr , i_binop (Def , B , Op1 , Op2 )) , node_has_succ ( Curr , Next ), St2 = let (v , st1 ) = do_binop (B , Op1 , Op2 , St ) in update_store (Def , v , st1 ). reached (Dst , St2 , decr (N )) :- reached ( Curr , St , some (N)) , node_has_inst ( Curr , i_jnz (Op , Dst )) , some ( St2 ) = let (v , st1 ) = operand_value (Op , St ) in update_path_cond ( ` ~( v ` , st1 ). reached ( Next , St2 , decr (N )) :- reached ( Curr , St , some (N)) , node_has_inst ( Curr , i_jnz (Op , _)) , node_has_succ ( Curr , Next ), some ( St2 ) = let (v , st1 ) = operand_value (Op , St ) in update_path_cond ( ` v ` , st1 ). failed ( Node , St ) :- reached ( Node , St , _), node_has_inst ( Node , i_fail ). Fig. 3. Horn clauses and ML-style helper functions define the logic of the symbolic evaluator. ormulog: Datalog for SMT-Based Static Analysis 141:7 proposition to a bit-vector-valued SMT formula by building an if-then-else SMT expression (viathe · then · else constructor) that is if the proposition is true and otherwise.Four rules define the reached relation. The first one (line 54) states the base case: node 0 (thestart of the CFG) is reachable with the initial state and 10 units of fuel. The remaining recursiverules match each possible step of execution and have a shared form: They check whether executionhas reached a particular type of instruction with a non-zero amount of fuel, do whatever operationis required for that instruction, and then, if successful, step to the appropriate successor instructionwith one less unit of fuel (computed via decr ). For example, the second rule (lines 56-62) handles abinary operation: the operation is performed symbolically (via the function do_binop ), the store isupdated with the resulting value, and evaluation steps to the fall-through successor node Next .The third and fourth rules define what happens when evaluation reaches a conditional jump.The first of these (lines 64-69) handles the case where the jump condition succeeds (i.e., when theoperand in the jump can be nonzero) in which case the evaluator steps to the jump destination
Dst with an updated path condition constraining the operand to be nonzero. The second of these(lines 71-77) handles the case where the jump condition fails (i.e, the operand can be zero). Notethat these two cases are not mutually exclusive; the fact that these two rules can “fire” at the sametime means that the symbolic evaluator can explore both branches in parallel.One final rule (lines 79-81) defines the failed relation, and states that evaluation has uncovereda failure if it has reached a node with a fail instruction.The symbolic evaluator can correctly determine that this program is safe: if (x < y) { x ++; assert (x <= y ); }
It can also determine that this program is not (because the bit vector y can wrap around): if (x < y) { x ++; y ++; assert (x <= y ); } While seemingly simple, this toy symbolic evaluator captures the essence of the more developedsymbolic evaluator we describe as a case study (Section 5.4).
Formulog extends Datalog with a fragment of first-order ML and a language of SMT formulas(Figure 4). Accordingly, a program consists of Horn clauses, type and function definitions, and SMTdeclarations. The Horn clause fragment is the same as in Datalog, except with a richer variety ofexpressions 𝑒 that can occur as arguments to predicates. Type definitions.
Formulog users can define ML-style algebraic data types, which can be polymor-phic and mutually recursive. An algebraic data type definition consists of a list of type variables 𝛼 , atype name 𝐷 , and a list of constructors 𝑐 with their argument types 𝜏 . Section 4 explains Formulog’stype system in more detail; we provide a brief sketch now. Algebraic data types 𝐷 𝜏 ∗ , base types 𝐵 , and type variables 𝛼 are treated as pre-types ; intuitively, a pre-type 𝑡 is the type of a concrete(non-formula) term. In addition to pre-types, there are types that represent SMT-relevant terms: a 𝑡 -valued SMT formula has type 𝑡 smt , a 𝑡 -valued SMT variable has type 𝑡 sym , and an SMT model— a finite map from formula variables to concrete terms — has type model . The Formulog typesystem distinguishes the first three types where it is computationally relevant (i.e., during concreteevaluation, where confusing a 𝑡 -valued formula for a concrete 𝑡 term might lead to a computationgetting stuck), and collapses them where it is not (i.e., during SMT evaluation, where there is nomeaningful distinction between a 𝑡 -valued formula and a concrete 𝑡 value). It also prevents SMTmodels, which are not representable as SMT expressions, from flowing into SMT formulas. Types
Types 𝜏 :: = 𝑡 | 𝑡 smt | 𝑡 sym | model Pre-types 𝑡 :: = 𝐵 | 𝐷 𝜏 ∗ | 𝛼 Base types 𝐵 :: = bool | string | bv [ k ] 𝑘 ∈ N + | . . . Terms
Programs prog :: = 𝐻 ∗ 𝑇 ∗ 𝐹 ∗ 𝑍 ∗ Horn clauses 𝐻 :: = 𝑝 ( 𝑒 ∗ ) : − 𝑃 ∗ Premises 𝑃 :: = 𝐴 | ! 𝐴 Atoms 𝐴 :: = 𝑝 ( 𝑒 ∗ ) | 𝑒 = 𝑒 Type definitions 𝑇 :: = type 𝛼 ∗ 𝐷 = [ 𝑐 ( 𝜏 ∗ )] ∗ Functions 𝐹 :: = fun 𝑓 ([ 𝑋 : 𝜏 ] ∗ ) : 𝜏 = 𝑒 SMT declarations 𝑍 :: = uninterpreted fun 𝑐 ([ 𝑡 smt ] ∗ ) : 𝑡 smt | uninterpreted sort 𝛼 ∗ 𝐷 Expressions 𝑒 :: = 𝑋 | 𝑐 ( 𝑒 ∗ ) | 𝑘 | 𝑓 ( 𝑒 ∗ ) | match 𝑒 with [ 𝑐 ( 𝑋 ∗ ) → 𝑒 ] ∗ | let 𝑋 = 𝑒 in 𝑒 | if 𝑒 then 𝑒 else 𝑒 | ⊗( 𝑒 ∗ ) | ` 𝜙 ` | 𝑝 ( 𝑤 ∗ ) Constants 𝑘 :: = true | false | | | . . . SMT formulas 𝜙 :: = , 𝑒 | 𝑐 SMTforall ( 𝜙, 𝜙 ) | 𝑐 SMTlet ( 𝜙, 𝜙, 𝜙 ) | 𝑐 SMTctor [ 𝑐 ] ( 𝜙 ∗ ) | . . . Wildcard 𝑤 :: = ?? | 𝑒 Values 𝑣 ∈ Val :: = 𝑘 | 𝑐 ( 𝑣 ∗ ) Namespaces
Data type names 𝐷 ∈ ADTVar
Type variables 𝛼 ∈ TVar
Constructors 𝑐 ∈ CtorVar
Variables 𝑋 ∈ Var
Predicates 𝑝 ∈ PredVar
Functions 𝑓 ∈ FunVar
Fig. 4. Formulog extends the abstract syntax of Datalog with type definitions, functions, SMT declarations,and a richer language of expressions.
Functions.
Formulog supports ML-style function definitions, although functions are limited tobeing first-order and are not first-class values. They can be polymorphic and mutually recursive.
SMT declarations.
Formulog users can declare uninterpreted functions and polymorphic unin-terpreted sorts. An uninterpreted function amounts to a special constructor for building a purelysymbolic term of type 𝑡 smt (for some pre-type 𝑡 ). An uninterpreted sort amounts to a specialsymbolic pre-type 𝑡 , where 𝑡 is not inhabited by any value, but 𝑡 sym and 𝑡 smt are. Expressions and formulas.
Expressions 𝑒 occur as function bodies and as predicate argumentsin Horn clauses. Although Datalog traditionally limits ground terms to nullary constructors, weadmit 𝑛 -ary constructors. While this comes with the cost of possibly-diverging programs — adding 𝑛 -ary constructors makes Datalog Turing-complete [Green et al. 2013] — many recent Datalogvariants allow complex terms, including Soufflé [Scholz et al. 2016], LogicBlox [Aref et al. 2015],and Flix [Madsen et al. 2016]. For us, complex terms provide a natural way to reify logical formulas,and they also can be used to create data structures that make it easier to encode certain analyses.Additional Formulog expressions include standard ML fare like constants (booleans, strings,machine integers, and floats), function calls, and match, let, and if-then-else expressions. Theexpression ⊗( 𝑒 ∗ ) represents the application of a primitive operator to a sequence of subexpressions. ormulog: Datalog for SMT-Based Static Analysis 141:9 These cover both basic arithmetic operations (e.g., addition) and SMT-specific operations (e.g.,checking for satisfiability, generating model s; see Section 3.3.2).The expression ` 𝜙 ` is a quasi-quoted SMT formula, where the language of formulas 𝜙 consistsof unquoted expressions , 𝑒 and formula constructors of the form 𝑐 SMTc ′ applied to SMT formulas.Some of these constructors directly reflect SMT formula constructs; for example, the constructor 𝑐 SMTforall builds a universally quantified formula, and the constructor 𝑐 SMTlet builds an SMT let formula.Formula constructors can appear only in formulas, and non-formula constructors cannot appeardirectly in formulas. We embed algebraic data type constructors in formulas using a family of formula constructors . Each formula constructor 𝑐 SMTctor [ 𝑐 ] lifts the user-defined algebraic data typeconstructor 𝑐 to SMT. Quotes are used to delineate formulas and trigger a different type checkingmode, in which the types 𝑡 , 𝑡 smt , and 𝑡 sym are conflated (with some restrictions, as explained inSection 4). The unquote operator , escapes from this type checking mode and makes it possible toinject a non-formula expression into a formula. Section 3.3 discusses formulas in more detail.We have already seen how the Datalog fragment of Formulog can include expressions from theML fragment; the final expression 𝑝 ( 𝑤 ∗ ) ties the loop by providing a way for the ML fragmentto reference the Datalog fragment. The expression 𝑝 ( 𝑤 ∗ ) acts like a function call that queriesthe contents of the relation 𝑝 . Its exact behavior depends on its arguments, which are eitherexpressions or the special wildcard term ?? . If its arguments contain no wildcards, then 𝑝 ( 𝑒 ∗ ) returns a boolean indicating whether the tuple identified by its arguments is in the 𝑝 relation. If ithas 𝑘 > wildcards, it returns a list of 𝑘 -tuples: For each tuple 𝑣 ∗ in the relation corresponding to 𝑝 , there is a corresponding 𝑘 -tuple in this list that is 𝑣 ∗ projected to the wildcard positions; if thereare 𝑛 matching tuples in 𝑝 , then the list is of length 𝑛 . In other words, given complete arguments,a predicate is really just a predicate; given partial arguments with wildcards, a predicate is themultiset consisting of matching tuples after they have been appropriately projected. Remarks.
Extending Datalog with our fragment of ML is not foundational, as it can relativelyeasily be translated to Datalog rules (this would not necessarily be the case for a higher-orderfragment of ML). However, despite the fact that the ML fragment could be treated as just syntacticsugar, it has a significant positive impact on the usability of Formulog, as we argue in Section 6.The concrete syntax of formulas in our prototype (and in the examples we give in this paper)differs from the abstract syntax given here. We differentiate between ML variables (initial lowercase)and Datalog variables (initial caps). Algebraic data type constructors are allowed to appear directlyin formulas, and are implicitly lifted to the appropriate formula constructor (so data type constructor 𝑐 is automatically lifted to 𝑐 SMTctor [ 𝑐 ] ). We do not support an explicit unquote operator; instead, weimplicitly unquote variables, constants, and invocations of nullary functions. We support additionalfeatures (records, locally scoped functions, etc.) that can be easily compiled to the abstract syntax. Formulog uses data types and operators to support constructing and reasoning about logicalformulas. Formulog provides a library of data types that define logical terms. Most of the timeduring evaluation, these terms are unremarkable and treated just like any other ground term.However, these terms are interpreted as logical formulas when they are used as arguments tobuilt-in operators that make calls to an external SMT solver. In our current prototype, it is possibleto create logical terms in first-order logic extended with (fragments of) the SMT-LIB theories ofuninterpreted functions, integers, bit vectors, floating point numbers, arrays, and algebraic data Multisets can arise if the “anonymous” variable _ is used to project out unwanted columns. For example, given that exactly p(1, 2) and p(3, 2) hold, the expression p(_, ??) would evaluate to a multiset represented by the list [2, 2] . Negation ~ : bool smt → bool smt Conjunction /\ : ( bool smt , bool smt ) → bool smt Implication ==> : ( bool smt , bool smt ) → bool smt Equality smt_eq[ 𝑡 ] : ( 𝑡 smt , 𝑡 smt ) → bool smt SMT variable smt_var[ 𝑡 ′ , 𝑡 ] : 𝑡 ′ → 𝑡 sym Bit vector constant bv_const[ 𝑘 ] : bv[32] → bv[ 𝑘 ] smt Bit vector addition bv_add : ( bv[ 𝑘 ] smt , bv[ 𝑘 ] smt ) → bv[ 𝑘 ] smt Fig. 5. Logical formulas are created in Formulog via built-in constructors, such as the ones shown here. types [Barrett et al. 2016], as well as the theory of strings shared by the SMT solvers Z3 [de Mouraand Bjørner 2008] and CVC4 [Barrett et al. 2011].
Users create logical terms through constants and formula con-structors. For example, to represent the formula
𝐹𝑎𝑙𝑠𝑒 = ⇒ 𝑇𝑟𝑢𝑒 , one would use the term ` false ==> true ` , where false and true are the standard boolean values and ==> is the infixconstructor for implication.Our current prototype offers around 70 constructors for creating logical terms ranging fromsymbolic string concatenation to logical quantifiers; others could be added in the future. Figure 5shows a sample of these constructors and their types. Some constructors require explicit indices,either to guarantee that type information is available at runtime when the formula is serializedto SMT-LIB, or to make sure that the type of the arguments can be determined by the type of theconstructed term (which makes type checking easier). For example, bv_const[ 𝑘 ] creates a symbolic 𝑘 -bit-vector value from a concrete 32-bit vector; at runtime, it is necessary to know the width 𝑘 so that we can serialize it correctly. The constructor smt_eq[ 𝑡 ] denotes the equality of two termsof type 𝑡 smt (alternatively stated using the infix notation ); here, the index makes sure that thetype checker knows what types the arguments should have. A programmer typically does not needto provide these indices explicitly, as they can often be inferred (our prototype does this).Formulog distinguishes between logic programming variables and formula variables. A formulavariable is a ground term that, when interpreted logically, represents a symbolic value. A term smt_var[ 𝑡 ′ , 𝑡 ]( 𝑣 ) — typically abbreviated as 𝑣 }[ 𝑡 ] — is a formula variable of type 𝑡 sym identifiedby a value 𝑣 of type 𝑡 ′ . Intuitively, 𝑣 is the “name” of the variable. The term 𝑣 }[ 𝑡 ] is guaranteednot to occur in 𝑣 , which means that the variable it represents is fresh with respect to the set offormula variables in 𝑣 ; this makes it easy to deterministically construct a new variable that is freshwith respect to an environment, a trick we use often in our case studies. For example, if X is boundto a list of boolean formula variables, the formula variable will not unify with any termin X . The shorthand 𝑖𝑑 [ 𝑡 ] is equivalent to 𝑖𝑑 "}[ 𝑡 ] , where 𝑖𝑑 is a syntactically valid identifier.Importantly, because formula variables are ground terms, we can derive facts containing formulavariables without violating Datalog’s range restriction, which requires that every derived fact isvariable-free. This restriction enables efficient evaluation by simplifying table lookups, one of thefundamental operations in Datalog evaluation. Built-in operators provide a way to reason about logical terms as formulas(Figure 6). When an operator in the SMT interface is invoked, its formula argument is serializedinto the SMT-LIB format and a call is made to an external SMT solver. These operators are assumedto act deterministically during a single Formulog run; an implementation can achieve this in thepresence of a non-deterministic SMT solver by memoizing operations. ormulog: Datalog for SMT-Based Static Analysis 141:11
Satisfiability is_sat : bool smt → boolis_sat_opt : ( bool smt list , bv[32] option ) → bool option Validity is_valid : bool smt → bool Model generation get_model : ( bool smt list , bv[32] option ) → model option Model inspection query_model : ( 'a sym , model ) → 'a option Fig. 6. Formulog provides built-in operators for reasoning about logical terms.
For example, to test the validity of the principle of explosion (any proposition follows fromfalse premises), one could make the call is_valid( ` false ==> ` ) . Like other operators,the SMT interface operators can be invoked from the bodies of rules, as here: ok :- ` ` ) = true ,is_sat ( ` ~( ` ) = true . This rule derives the fact ok : The term is not unifiable with the term , since theyare different formulas, representing different SMT variables. But these terms both may and maynot be equal when interpreted as formula variables via the operator is_sat . Within an invocationof is_sat , constraints are formed between and — in the first case they must beequal, and in the second case they must not be — but these constraints do not leak into the largercontext. This is an intentional design decision and differs from the approach taken by paradigmslike constraint logic programming (see Section 6).Formulog provides two sets of operators for testing the satisfiability and logical validity ofpropositions. In general, an SMT solver can return three possible answers to such a query: “yes,”“no,” and “unknown.” The operators is_sat and is_valid return booleans. In the case that thebackend SMT solver is not be able to determine whether a formula 𝜙 is satisfiable, these operatorsfail (as explained in Section 4). The operator is_sat_opt( 𝜙 ∗ , timeout) provides more fine-grainedcontrol: it takes a list of propositions (interpreted as conjuncts) and an optional timeout, and returnsan optional boolean, with none corresponding to “unknown.” While we suspect that the simplerversions will be sufficient for most applications, this more complex version does allow applicationsto explicitly handle the “unknown” case if need be (e.g., pruning paths in symbolic execution).The operator get_model takes a list of propositions and an optional timeout; it returns a modelfor the conjunction of the propositions if the SMT solver is able to find one in time, and none otherwise. The values of formula variables in this model can be inspected using query_model , whichreturns none if the variable does not occur free in the formula or if a concrete value for it is notrepresentable in Formulog (for example, Formulog does not have a type for a concrete 13-bit vector).The values of symbolic expressions can be indirectly extracted through formula variables: Beforefinding the model, add the equality ` 𝑥 𝑒 ` to the formula, where 𝑥 is a fresh formula variable and 𝑒 is an expression; in the extracted satisfying model, 𝑥 will be assigned the value of 𝑒 in that model. Formulog’s algebraic data types can be reflected in SMT formulasvia SMT-LIB’s support for algebraic data types. Thus, Formulog permits arbitrary term constructorsto be used within logical formulas. For example, we can define a type foo with a single nullaryconstructor bar and then write formulas involving foo -valued terms: type foo = | barok :- is_valid ( ` ` ) = true . This program would derive the fact ok : Since there is only one way to construct a foo — throughthe constructor bar — any symbolic value of type foo must be the term bar . Namespaces and constructs
World
W ∈
PredVar → P (
Val ∗ ) Substitution 𝜃 ∈ Var ⇀ Val
Error ⊥ ∈
Err 𝑢 -term 𝑢 :: = 𝑋 | 𝑘 | 𝑐 ( (cid:174) 𝑢 𝑖 ) Clause semantics (cid:174) 𝐹 ; W ⊢ 𝐻 → W ⊥ | (cid:174) 𝑃 𝑖 | = 𝑛 𝜃 = · ∀ 𝑖 ∈ [ , 𝑛 ) , 𝜃 𝑖 ⊢ 𝑃 𝑖 → 𝜃 𝑖 + (cid:174) 𝐹 ; W ⊢ 𝑝 ( (cid:174) 𝑋 𝑗 ) : − (cid:174) 𝑃 𝑖 → W [ 𝑝 ↦→ W ( 𝑝 ) ∪ { 𝜃 𝑛 ( (cid:174) 𝑋 𝑗 )}] Clause
Premise semantics (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝑃 → 𝜃 ⊥ (cid:174) 𝑣 ∈ W ( 𝑝 ) 𝜃 ⊢ (cid:174) 𝑋 ∼ (cid:174) 𝑣 : 𝜃 ′⊥ W ; 𝜃 ⊢ 𝑝 ( (cid:174) 𝑋 ) → 𝜃 ′⊥ PosAtom 𝜃 ⊢ 𝑌 ∼ 𝑐 ( (cid:174) 𝑋 ) : 𝜃 ′⊥ W ; 𝜃 ⊢ 𝑌 = 𝑐 ( (cid:174) 𝑋 ) → 𝜃 ′⊥ EqCtor
Expression semantics (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ (cid:174) 𝐹 ; W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 ⊥ W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 [[⊗]] ((cid:174) 𝑣 ) = 𝑣 W ; 𝜃 ⊢ ⊗((cid:174) 𝑒 ) ⇓ 𝑒 𝑣 ⇓ 𝑒 -Op W ; 𝜃 ⊢ 𝜙 ⇓ 𝜙 𝑣 ⊥ W ; 𝜃 ⊢ ` 𝜙 ` ⇓ 𝑒 𝑣 ⊥ ⇓ 𝑒 -Quote Formula semantics (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝜙 ⇓ 𝜙 𝑣 ⊥ (cid:174) 𝐹 ; W ; 𝜃 ⊢ (cid:174) 𝜙 ⇓ (cid:174) 𝜙 (cid:174) 𝑣 ⊥ W ; 𝜃 ⊢ (cid:174) 𝜙 ⇓ (cid:174) 𝜙 (cid:174) 𝑣 W ; 𝜃 ⊢ 𝑐 SMTc ′ ( (cid:174) 𝜙 ) ⇓ 𝜙 𝑐 SMTc ′ ((cid:174) 𝑣 ) ⇓ 𝜙 -Ctor W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 W ; 𝜃 ⊢ , 𝑒 ⇓ 𝜙 toSMT ( 𝑣 ) ⇓ 𝜙 -Unqote SMT conversion toSMT ( 𝑣 ) = 𝑣 toSMT ( 𝑐 SMTlet ( 𝑣 , 𝑣 , 𝑣 )) = 𝑐 SMTlet ( 𝑣 , 𝑣 , 𝑣 ) toSMT ( 𝑐 SMTforall ( 𝑣 , 𝑣 )) = 𝑐 SMTforall ( 𝑣 , 𝑣 ) toSMT ( 𝑐 ( (cid:174) 𝑣 𝑖 )) = 𝑐 SMTctor [ 𝑐 ] (−−−−−−−−−−→ toSMT ( 𝑣 𝑖 )) . . . Fig. 7. A fragment of Formulog’s operational semantics (see Appendix C for full formalization).
For each algebraic data type, we automatically generate two kinds of constructors that make iteasier to write formulas involving terms of that type. The first kind is a constructor tester. For eachconstructor 𝑐 of a type 𝑡 , Formulog provides a constructor 𝑐 of type 𝑡 smt → bool smt . Theproposition 𝑐 ( 𝑒 ) holds if the outermost constructor of 𝑒 is 𝑐 . The second kind is an argumentgetter. If 𝑐 is a constructor for type 𝑡 with 𝑛 arguments of types 𝑡 𝑖 for ≤ ≤ 𝑛 , Formulog generates 𝑛 argument getters of the form 𝑐 _ 𝑖 , where 𝑐 _ 𝑖 has the type 𝑡 smt → 𝑡 𝑖 smt . When interpreted as aformula, the term 𝑐 _ 𝑖 ( 𝑒 ) represents the value of the 𝑖 th argument of 𝑒 . For example, we can statethat a symbolic list of booleans is non-empty and its first argument is true : ` ` We could use the operator get_model to find a model of this satisfiable formula; in this model, might be assigned the concrete value cons(true, nil) . ormulog: Datalog for SMT-Based Static Analysis 141:13 This section presents Formulog’s operational semantics, making reference to a selection of the formalrules (Figures 7).
3, 4
Formulog imposes the standard stratification requirements upon programs:no recursive dependencies involving negation or aggregation between relations. As a stratifiableprogram can be evaluated one stratum at a time, we focus on the evaluation of a single stratum.A stratum is evaluated by repeatedly evaluating its Horn clauses until no new inferences can bemade. The semantics of a Horn clause 𝐻 is defined through the judgment (cid:174) 𝐹 ; W ⊢ 𝐻 → W ⊥ , wherea world W is a map from predicate symbols to sets of tuples (i.e., those that have been derivedso far). A Horn clause takes a world to either a new world or the error value ⊥ . Going wrong canresult for two reasons: either because a variable is unbound at a point where it needs to be bound,or because an operator is applied to a value outside of its domain. It is important to distinguishbetween a rule going wrong and a rule failing to complete because two terms fail to unify: The firstis an undesirable error (ruled out by our type system), whereas the second is expected behavior.A rule is evaluated by evaluating its premises one-by-one, using a left-to-right order (Clause).The judgment (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝑃 → 𝜃 ⊥ defines the semantics of a premise, which takes a world and asubstitution 𝜃 (a partial function from variables to values) and returns a new substitution or an error.The substitution produced by one premise is used as the input to the next one. A successful inferenceextends the input world with a (potentially novel) tuple 𝜃 𝑛 ( (cid:174) 𝑋 𝑗 ) , i.e., the result of element-wiseapplying the substitution produced by the rightmost premise to the variables in the head of therule. Clause evaluation goes wrong if the evaluation of one of the premises goes wrong.Without loss of generality, we assume that premises occur in a limited form: predicates are appliedto only variables, written 𝑝 ( (cid:174) 𝑋 𝑖 ) , and equality predicates bind variables, as in 𝑌 = 𝑒 . (Our prototypesimilarly desugars premises.) An atom 𝑝 ( (cid:174) 𝑋 ) is evaluated by non-deterministically choosing a tuple (cid:174) 𝑣 from the tuples in W ( 𝑝 ) , and then pairwise unifying its elements with the variables (cid:174) 𝑋 (PosAtom).The premise 𝑌 = 𝑐 ( (cid:174) 𝑋 ) unifies its two terms (EqCtor). The judgment 𝜃 ⊢ 𝑢 ∼ 𝑢 : 𝜃 ⊥ definesthe unification of terms 𝑢 and 𝑢 under the substitution 𝜃 ; it results in an error if 𝑢 and 𝑢 bothcontain unbound variables, and a new substitution if they are otherwise unifiable.Most expressions have standard semantics. An operator produces a value if its arguments areevaluated to values in its domain ( ⇓ 𝑒 -Op); it goes wrong if the argument values are outside itsdomain, e.g., if a string and number are added together. A quoted formula ` 𝜙 ` evaluates to whatever 𝜙 evaluates to ( ⇓ 𝑒 -Quote). Formula 𝑐 SMTc ′ ( (cid:174) 𝜙 ) evaluates to formula 𝑐 SMTc ′ ((cid:174) 𝑣 ) if arguments (cid:174) 𝜙 evaluateto values (cid:174) 𝑣 ( ⇓ 𝜙 -Ctor). If the expression 𝑒 evaluates to the value 𝑣 , then the formula , 𝑒 evaluates tothe term toSMT ( 𝑣 ) ( ⇓ 𝜙 -Unqote), where the function toSMT lifts a term to its formula version. Formulog’s type system is designed to meet three desiderata. The first desideratum is that concreteevaluation should never go wrong, which might happen if an operator is applied to an operandoutside its domain or a variable is unbound at a point when it needs to be evaluated. The seconddesideratum is that SMT solving should never go wrong, which might happen if a term that doesnot represent a well-sorted formula under the SMT-LIB standard reaches the external SMT solver(e.g., a formula representing the addition of a 16-bit vector and 32-bit vector). The third desideratum Formulog can also be given a model-theoretic semantics: because the ML features can be desugared into Datalog rules, themodel theory of Formulog is essentially that of stratified Datalog. Appendix F sketches this out further. In the boxed rule schemata, implicit parameters are in gray; we conserve space by stating the rules without threadingimplicit parameters through, which are unchanging. We write (cid:174) 𝑥 𝑖 for some metavariable 𝑥 to mean a possibly empty sequenceof 𝑥 s indexed by 𝑖 , and write 𝑆 ⊥ for some set 𝑆 to mean the set 𝑆 + Err . is that the type system should make it easy to construct expressive logical formulas, includingformulas that involve terms drawn from user-defined types.There is some tension between the first and third of these desiderata. The first one requires thatwe differentiate between, for example, a concrete bit-vector value and a symbolic bit-vector value(e.g., a bit-vector-valued formula) since an operator that is expecting a concrete bit vector mightget stuck if its argument is a symbolic bit vector. For instance, we want to rule out this program: Example 1 (A bad program we would want to reject) . type foo = | bar ( bv [32])fun f(x: foo ) : bv [32] = match x with bar (y) => y + y endnot_ok :- X = This program gets stuck evaluating f(bar(X)) , since y is bound to a symbolic value in f but the MLfragment’s addition operator needs concrete arguments. On the other hand, we are able to constructmore expressive formulas if we can occasionally conflate concrete and symbolic expressions: Example 2 (A good program we would want to accept) . ok :- X = ` bar (X) ` ) = true . This rule asks whether there exists a symbolic bit vector 𝑥 such that bar( 𝑥 ) equals bar(5) , where bar is the constructor defined above. This reasonable formula is not well-typed under a type systemthat uniformly distinguishes between concrete and symbolic values, since the constructor bar expects a concrete bit vector argument but instead receives the symbolic one 𝑥 .Formulog resolves the tension between these desiderata through a bimodal type system thatacts differently inside and outside formulas (which are demarcated by quotations). In essence, theFormulog type system differentiates between the pre-type 𝑡 , the SMT formula type 𝑡 smt , and theSMT variable type 𝑡 sym outside of formulas, but typically conflates them within formulas. Thisbimodal approach disallows Example 1 (since outside a formula, a term of type bv [ ] sym cannotbe used where a term of type bv [ ] is expected), while permitting Example 2 (since within aformula, a term of type bv [ ] sym can be used anywhere a term of type bv [ ] is expected).Intuitively, this bimodal approach is safe because it distinguishes between concrete and symbolicvalues during concrete evaluation — where conflating them might lead to going wrong — andconflates them only during SMT evaluation, where the distinction is not meaningful. We haveformalized the Formulog type system and proven it sound with respect to the operational semanticsof Formulog. We present only a small subset of it here (Figure 8); the full system is in Appendix B.The rule defining a well-typed Horn clause ( 𝐻 -Clause) depends on two notable judgments.The premise typing judgment Γ ⊢ 𝑃 ▷ Γ ′ takes a variable typing context Γ and a premise 𝑃 andproduces a new variable typing context Γ ′ . The variable binding and typing judgment Γ ⊢ 𝑥, 𝜏 ▷ Γ ′ holds if either 𝑋 is not in Γ , in which case Γ ′ extends Γ with 𝑋 mapped to 𝜏 ( 𝑋𝜏 -Bind), or 𝑋 ismapped to 𝜏 by Γ , in which case Γ = Γ ′ ( 𝑋𝜏 -Check). As can be seen from rule 𝐻 -clause, the typechecking of Horn clauses is flow-sensitive and proceeds left-to-right across the clause, with the“output” context of checking premise 𝑃 𝑖 used as the “input” context for checking premise 𝑃 𝑖 + .This left-to-right type checking mirrors the left-to-right evaluation strategy Formulog uses; this is It does not conflate them in binding positions where formula variables are required, such as in quantifiers. ormulog: Datalog for SMT-Based Static Analysis 141:15
Contexts
Data type declarations Δ :: = · | Δ , 𝐷 : ∀ (cid:174) 𝛼 𝑖 . {−−−−→ 𝑐 𝑗 : (cid:174) 𝜏 𝑘 } Program declarations Φ :: = · | Φ , 𝑓 : ∀ (cid:174) 𝛼, (cid:174) 𝜏 → 𝜏 | Φ , 𝑝 ⊆ (cid:174) 𝜏 Variable contexts Γ :: = · | Γ , 𝑥 : 𝜏 | Γ , 𝛼 Clause typing Δ ; Φ ⊢ 𝐻 · ⊢ 𝑃 ▷ Γ . . . Γ 𝑗 ⊢ 𝑃 𝑗 ▷ Γ 𝑗 + . . . Γ 𝑛 ⊢ 𝑃 𝑛 ▷ Γ ′ 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ′ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ ⊢ 𝑝 ( (cid:174) 𝑋 𝑖 ) : − (cid:174) 𝑃 𝑗 𝐻 -Clause Variable binding and typing Γ ⊢ 𝑥, 𝜏 ▷ Γ Γ ⊢ (cid:174) 𝑥, (cid:174) 𝜏 ▷ Γ 𝑋 ∉ dom ( Γ ) Γ ⊢ 𝑋, 𝜏 ▷ Γ , 𝑋 : 𝜏 𝑋𝜏 -Bind Γ ( 𝑋 ) = 𝜏 Γ ⊢ 𝑋, 𝜏 ▷ Γ 𝑋𝜏 -Check Premise typing Δ ; Φ ; Γ ⊢ 𝑃 ▷ Γ 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ Γ ⊢ 𝑝 ( (cid:174) 𝑋 𝑖 ) ▷ Γ ′ 𝑃 -PosAtom Γ ⊢ 𝑒 : 𝜏 Γ ⊢ 𝑌, 𝜏 ▷ Γ ′ Γ ⊢ 𝑌 = 𝑒 ▷ Γ ′ 𝑃 -Eq-FB Function and expression well formedness Δ ; Φ ⊢ 𝐹 Δ ; Φ ; Γ ⊢ 𝑒 : 𝜏 typeof (⊗) = (cid:174) 𝜏 𝑖 → 𝜏 Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 Γ ⊢ ⊗( (cid:174) 𝑒 𝑖 ) : 𝜏 𝑒 -Op Γ ⊢ 𝜙 : 𝜏 Γ ⊢ ` 𝜙 ` : 𝜏 𝑒 -Quote SMT constructors and formula well formedness Δ ; Φ ; Γ ⊢ 𝑐 SMT ... : (cid:174) 𝜏 𝑖 → 𝜏 Δ ; Φ ; Γ ⊢ 𝜙 : 𝜏 Γ ⊢ 𝑐 SMTc : (cid:174) 𝜏 𝑖 → 𝜏 Γ ⊢ 𝜙 𝑖 : 𝜏 𝑖 Γ ⊢ 𝑐 SMTc ( (cid:174) 𝜙 𝑖 ) : 𝜏 𝜙 -Ctor Γ ⊢ 𝑒 : 𝜏 Γ ⊢ , 𝑒 : toSMT ( 𝜏 ) 𝜙 -Unqote Γ ⊢ 𝜙 : 𝑡 sym Γ ⊢ 𝜙 : 𝑡 smt 𝜙 -Promote SMT representations erase ( 𝜏 ) = 𝑡 toSMT ( 𝜏 ) = 𝜏 erase ( 𝐵 ) = 𝐵 erase ( 𝐷 (cid:174) 𝜏 𝑖 ) = 𝐷 −−−−−−−→ erase ( 𝜏 𝑖 ) erase ( 𝑡 smt ) = erase ( 𝑡 ) erase ( 𝑡 sym ) = erase ( 𝑡 ) toSMT ( 𝑡 ) = erase ( 𝑡 ) smt toSMT ( 𝑡 smt ) = erase ( 𝑡 ) smt toSMT ( 𝑡 sym ) = erase ( 𝑡 ) sym Fig. 8. A fragment of Formulog’s type system (see Appendix B for full formalization). important for ensuring that variables are bound at the correct points. The second line of premises The fact that the operational semantics and type system assume a certain order of evaluation does not prohibit a Formulogruntime from reordering premises within rules (for example, when applying database-style query planning optimizations);it just needs to check that the new order is also well typed. This type of rewriting does not affect the result of running therule provided that all subexpressions terminate (an assumption we make). in rule 𝐻 -clause ensures that every variable in the head of the rule is bound at the type specifiedby the head relation’s signature.A positive atom is well typed if each of its variable arguments has the type given to that argumentby the relation’s signature ( 𝑃 -PosAtom). A premise of the form 𝑌 = 𝑒 is typed according to a fewdifferent rules depending on which side of the equation is ground with respect to the input context Γ . The key is that our type system only types premises of the form 𝑌 = 𝑒 when unification isguaranteed to not go wrong at runtime.The typing rules for most expressions are standard. An operation is well-typed if its argumentsmatch its type signature ( 𝜙 -Ctor). A quoted formula ` 𝜙 ` types at whatever 𝜙 types at ( 𝑒 -Quote).The formula constructor 𝑐 SMTc is well typed if the types of its arguments match its type signature( 𝜙 -Ctor); in the case of a constructor for an algebraic data type that has been lifted to a formulaconstructor, that signature will require the constructed term and all of its arguments to have typesof the form 𝑡 smt . If an expression types at 𝜏 , then the formula , 𝑒 types at toSMT ( 𝜏 ) ( 𝜙 -Unqote).The helper function toSMT lifts a type to a formula type; for example, it lifts bool to bool smt . Thetyping rules for formulas also include a rule promoting from 𝑡 sym to 𝑡 smt , reflecting the fact that,within a formula, a 𝑡 -valued formula variable can be used anywhere a 𝑡 -valued formula can be. Type soundness with respect to the semantics of Formulog comes from safety and preservation:Theorem 4.1 (Safety). If Δ ; Φ ⊢ (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 and Δ ; Φ | = W then for all 𝐻 ∈ (cid:174) 𝐻 𝑗 , ¬( (cid:174) 𝐹 𝑖 ; W ⊢ 𝐻 → ⊥) . Theorem 4.2 (Preservation). If Δ ; Φ ⊢ (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 and Δ ; Φ | = W and (cid:174) 𝐹 𝑖 ; W ⊢ 𝐻 → W ′ for some 𝐻 ∈ (cid:174) 𝐻 𝑗 then Δ ; Φ | = W ′ . Safety (Theorem 4.1) guarantees that a Horn clause from a well-typed program, evaluated on awell-typed world (i.e., one where all the tuples have the right types), cannot step to error. Thus,safety means that an operator is never applied to an operand outside its domain, and a variable isnever unbound when it needs to be bound. Preservation (Theorem 4.1) guarantees that if a Hornclause, from a well-typed program, is evaluated on a well-typed world and results in a new world,then that new world is also well-typed. Taken together, these theorems imply that a well-typedFormulog program does not go wrong during concrete evaluation (see Appendix E for proofs).The type system is sound with respect to the semantics of SMT-LIB because the types of theformula constructors provided by Formulog are consistent with the types given by the SMT-LIBstandard. The Formulog type system guarantees that, at runtime, terms (including formulas) arewell-typed, and the type system prevents terms that are not representable in SMT (such as those oftype model ) from flowing into SMT formulas. We distinguish between SMT-compatible types andnon-SMT types formally by indexing the type well formedness judgment with a mode , which iseither smt (for those types that can be sent to the solver) or exp (for those types that cannot). It isfair to think of these modes as kinds with a subkinding relationship: types of kind smt can safelybe treated as general types of kind exp , but not the other way round (Lemma D.1).
Assumptions.
An actual implementation of Formulog, such as our prototype, has to contend witha few sources of going wrong that are not captured in our formal model. First, our model assumesthat patterns in match clauses are exhaustive; this is just for simplicity, and could be staticallychecked using standard algorithms. Second, our model assumes that operators are total with respectto terms with the correct type. There are three places where this assumption might break: 1) divisionor remainder by zero; 2) the operators is_sat or is_valid may induce an “unknown” responsefrom the external SMT solver; and 3) the SMT solver may reject patterns used in trigger-basedquantifier instantiation that it considers to be ill formed (for example, if the pattern contains a The opposite is not true, since some formula constructors (i.e., quantifiers and let expressions) bind formula variables. ormulog: Datalog for SMT-Based Static Analysis 141:17 binding operation). The first case is standard for many languages; the second can be avoided if theprogrammer uses the option-returning SMT operator is_sat_opt . The last case would be hard tocheck statically; however, an implementation could dynamically check patterns before making acall to the SMT solver, dropping invalid patterns and issuing a warning to the user. Our prototypeuses “hard exceptions” by default, aborting the program. We also support a “soft exception” mode,which treats all these cases analogously to unification failures, halting execution on the currentpath but allowing execution on other paths to continue.
In this section we briefly describe our prototype implementation of Formulog, and then discussthree analyses we have built as case studies: refinement type checking, bottom-up points-to analysis,and bounded symbolic evaluation.
Our prototype runtime ( ∼ Unless otherwise noted, we ran experiments on an Ubuntu Server 16.04 LTS machine with a 3.1GHz Intel Xeon Platinum 8175 processor (24 physical CPUs, each hyperthreaded) and 192 GiB ofmemory. We configured our Formulog runtime to use up to 40 threads and up to 40 Z3 instances(v4.8.7); all comparison systems were set to use the same version of Z3 (with one exception, notedlater). For each result, we report the median of three trials. Times are given as minutes:seconds.
We have implemented a type checker in Formulog for Dminor, a first-order functional programminglanguage for data processing that combines refinement types with dynamic type tests [Biermanet al. 2012]. This type system can, e.g., prove that x in Int ? x : (x ? 1 : 0) type checks as
Int in a context in which x has the union type (Int|Bool) . Proving this entailsencoding types and expressions as logical formulas and invoking an SMT solver over these formulas.We built a type checker for Dminor by almost directly translating the formal inference rules used todescribe the bidirectional Dminor type system. In fact, we programmed so closely to the formalismthat debugging an infinite loop in our implementation helped us, along with the Dminor authors,uncover a subtle typo in the formal presentation! Our Dminor type checker is 1.2K lines of Formulog.The implementation of Bierman et al. is 3.2K lines of F ♯ and 400 lines of SMT-LIB; we estimate thatthe functionality we implemented accounts for over two thousand of these lines. The encoding of Dminor types and expressions is complex, requiring uninterpreted sorts, unin-terpreted functions, universally quantified axioms, and arrays (among other features). The fact thatwe were able to code this relatively concisely speaks to the expressiveness of Formulog’s formulalanguage. For example, Figure 9 shows an axiom describing the denotation of the base case of aDminor accumulate expression, which is essentially a fold over a multiset. Here, the type closure is Our prototype is available at https://github.com/HarvardPL/formulog. For each case study, we use a tool to translate the input programs into Formulog facts. We do not include these times,which are typically quite short. Extracting libraries can take a few minutes, but this needs to be done only once per library. The reference implementation is closed source; the authors have kindly provided us with line counts for each file. fun accum_nil_axiom : bool smt =let (f , i) = ( ` forall f , i : accum (f , v_zero , i ). accum (f , v_zero , i) ` Fig. 9. This axiom encodes the denotation of a Dminor accumulate expression over an empty multiset. Theterm in the formula between : and . is a quantifier pattern [Detlefs et al. 2005]. fun encode_type (t: typ , v: enc_val smt ) : bool smt * bool smt =match t with| t_any => ( ` true ` , ` true ` )| t_bool => ( ` ` , ` true ` )| t_coll (s) =>let x = ` x ` ) in( ` good_c (v) /\ forall x : mem (x , v ). mem (x , v) == > phi ` , ax ) Fig. 10. This function (fragment) constructs a formula capturing the logical denotation of a Dminor type. an uninterpreted sort, enc_val is an algebraic data type that represents an encoded Dminor value,and accum and v_zero are uninterpreted functions, where the latter represents an empty multiset.We defined a set of mutually-recursive functions that encode expressions, environments, andtypes. For example, the type encoding function (fragment, Figure 10) takes a type 𝜏 and an (encoded)Dminor value 𝑣 , and returns two propositions. The first is true when 𝑣 has type 𝜏 . The secondis a conjunction of axioms: new axioms are created to describe the denotation of the bodies ofaccumulate expressions as they are encountered when encoding expressions. The first case inthe figure encodes the fact that any value has type Any . The second one says that a value hastype
Bool if it is constructed using the constructor ev_bool ; the constructor is anautomatically-generated constructor tester. The third case handles multiset types. It creates a freshencoded value x , uses x to recursively create a proposition representing the encoding of the type s of items in the multiset, and then returns a proposition requiring the value to be a “good” collection(defined using the uninterpreted function good_c ) and every item in the multiset to have type s (where mem is another uninterpreted function).Although we use ML-style functions to define the logical denotation of expressions, environments,and types, we use logic programming rules to define the bidirectional type checker, which allowsus to write rules that are very similar to the inference rules given in the paper. Figure 11 gives theone rule defining the subtype relation: T is a subtype of T1 in environment Env if T1 is well formedand the denotation of T , given our axioms and the denotation of Env , implies the denotation of T1 .This rule is an almost exact translation of the inference rule given in the paper.Finally, the type checker needs to ensure that any expressions that occur in refinements arepure (i.e., terminate and are deterministic). We have written a termination checker based on thesize-change principle [Lee et al. 2001]. Our implementation is another good example of the synergybetween ML-style functions and Datalog rules, as we use the former to define the composition oftwo size-change graphs and use the latter to find the fixed point of composing size-change graphs.We tested our type checker on six of the sample programs included in the Dminor documentation(the other three examples make use of a feature — the ability to generate an instance of a type —that we did not implement, although it should be possible to do so; to the best of our knowledge, ormulog: Datalog for SMT-Based Static Analysis 141:19 subtype (Env , T , T1 ) :-type_wf (Env , T1 ),encode_env ( Env ) = Phi_env ,X = ` ` ,encode_type (T , X) = ( Phi_t , Axioms1 ),encode_type (T1 , X) = ( Phi_t1 , Axioms2 ),Premises = [ Phi_t , Phi_env , Axioms2 , Axioms1 , axiomatization ],is_sat_opt ( ` ~ Phi_t1 ` :: Premises , z3_timeout ) = some ( false ). Fig. 11. This rule defines Dminor’s semantic subtyping relation. It uses the operator is_sat_opt instead of is_valid because its SMT queries can sometimes result in “unknown.” these are the only publicly available Dminor programs). We combined these examples into a singleaggregate program of ∼
150 LOC. The reference implementation type checked this program in 1.5seconds using an optimization that tries syntactic subtyping before semantic subtyping; with thisoptimization disabled, it took 3.6 seconds. Our implementation completed in 4.7 seconds; it did notuse this optimization (which is not detailed in the paper), but did use a newer version of Z3. Thanksto parallelization, our implementation automatically scaled to larger programs: On a syntheticprogram consisting of ten copies of the original aggregate program, it completed in 19.8 seconds(2.0 seconds per program copy); on a synthetic program consisting of 100 copies, it completed in153.6 seconds (1.5 seconds per program copy). In contrast, the reference implementation did notscale: even with the syntactic-subtyping optimization enabled, it took 68 seconds on the ten-copyprogram and over 100 minutes on the 100-copy program.
We have implemented the bottom-up context-sensitive points-to analysis for Java proposed byFeng et al. [2015]. A points-to analysis computes a static approximation of the objects that stackvariables and heap locations can point to at runtime. A bottom-up points-to analysis does thisthrough constructing method summaries that describe the effect of a method on the heap; it isbottom-up in the sense that summaries are propagated up the call graph, from callees to callers.In Feng et al.’s algorithm, a method summary is an abstract heap that maps abstract locationsto heap objects, where an abstract location might be a stack variable, an explicitly allocated heapobject, or an argument-derived heap location. Edges in the abstract heap are labeled with logicalformulas that describe the conditions under which the edges hold; when a method summary isinstantiated at a call site, a constraint solver can be used to filter out edges with unsatisfiable labels.Feng et al.’s tool based on this algorithm, Scuba, is ∼
15K lines of Java, builds on the Chordanalysis framework [Naik 2011], and uses Z3 to discharge constraints. As for many realistic staticanalysis tools, there is a gap between what is implemented in Scuba and the formal specification ofthe analysis. This is partly because Scuba is written in Java: Object-oriented programming doesnot naturally capture inference rules, the form of the specification. In contrast, our Formulogimplementation, which is ∼ Here we used a machine with Microsoft Windows Server 2019 and the same hardware specs as our Ubuntu machine. instantiate_ptsto (C , O1 , Phi1 , O2 , widen (C , Phi_all )) :-instantiate_loc (C , heap ( O1 ), heap ( O2 ), Phi2 ),instantiate_constraint (C , Phi1 , Phi3 ),Phi_all = conjoin ( Phi2 , Phi3 ).
Fig. 12. This rule describes how a points-to edge to object O1 labeled with constraint Phi1 is instantiated at acall site C : if at C a heap location heap(O1) can be instantiated to a heap location heap(O2) under constraint Phi2 , and the original constraint on the edge
Phi1 can be instantiated to a constraint
Phi3 , then the points-to edge to O1 labeled with Phi1 instantiates to a points-to edge to O2 labeled with widen(C, Phi_all) ,where Phi_all is the conjunction of
Phi2 and
Phi3 and widen is a function that widens constraints inmutually-recursive functions (one of the heuristics we borrowed from Scuba).Table 1. In the median, our implementation of a bottom-up points-to analysis for Java was 6.7 × slower thanScuba, the reference implementation (times in mm:ss); however, the two tools use different heuristics and thuscompute very different things, as indicated by the discrepancy in the number of points-to edges computed inthe summary for main (which also captures the effect on the heap of methods invoked transitively from it). Scuba FormulogBenchmark Time main edges Time main edgesantlr 1:11 3,313 12:16 112,415avrora 1:05 714 7:40 127,535hedc 0:57 867 5:04 2,962hsqldb 0:51 780 4:53 7,039luindex 1:40 3,395 T/O -polyglot 0:55 117 4:52 4,245sunflow 3:48 7,456 T/O -toba-s 0:58 521 4:57 12,284weblech 1:10 1,262 17:58 6,785xalan 0:54 183 5:40 55,722specifications’ correctness: while implementing in Formulog one of the judgments specified by Fenget al., we discovered an inconsistency between the judgment’s definition and its type signature.Scuba employs a range of sophisticated heuristics that are essential to making the algorithmperform in practice, as they tune precision to achieve scalability. Some go far beyond the algorithmdescribed in the paper and are interesting in their own right. Our implementation uses someheuristics based on the ones in Scuba. The fact that we were able to implement useful heuristics — anecessity for a realistic static analysis tool — argues for the practicality of Formulog. Moreover, wewere able to do so such that our code still closely reflects the core algorithm specified in the paper.We ran both tools on the benchmarks used in the evaluation by Feng et al., which represent aselection from the pjbench suite plus the benchmark polyglot. These experiments include librarycode and use a context-sensitivity of two call sites; reflection is ignored, as are many native methods.Given an hour timeout, our implementation completed on eight of the ten benchmarks, with timesranging from five to 18 minutes (Table 1). In the median, we were 6.7 × slower than Scuba. However,a performance comparison between the tools should be taken with a grain of salt: Since they usedifferent heuristics, they compute very different things. The pjbench suite is available at https://bitbucket.org/psl-lab/pjbench/src/master/. ormulog: Datalog for SMT-Based Static Analysis 141:21
In sum, we were able to implement the algorithm in a way that is still very close to its specificationand achieve decent performance on many realistic benchmarks while implementing only a smallselection of heuristics. Other heuristics might have helped our version complete on the twobenchmarks it timed out on. Making the algorithm practical is a significant engineering challenge:Even with its sophisticated heuristics, Scuba does not complete on all benchmarks in pjbench. Moreover, our implementation could be used as a platform for exploring potential optimizationsto Scuba. First, because it is automatically parallelized (with a user-chosen number of workerthreads), it could be used to evaluate how well the underlying points-to algorithm parallelizesbefore going through all the trouble of parallelizing Scuba, which uses mutable state in a complexway. Second, thanks to the magic set transformation, we have automatically derived a goal-directedversion of the analysis that computes only the summaries necessary for constructing user-requestedsummaries. The points-to algorithm resulting from this transformation could be used as a road mapfor implementing a demand-driven version of Scuba, which Feng et al. describe as future work.
We have written a symbolic evaluator ( ∼
1K LOC) for a fragment of LLVM bitcode [Lattner and Adve2004] corresponding to a simple imperative language with integer arrays and symbolic integers(a symbolic integer represents a set of integer values that might occur at runtime). It implementsa form of bounded symbolic execution [King 1976], exploring all feasible program paths up to agiven length, evaluating concretely whenever possible, and aggressively pruning infeasible paths.Our implementation uses a different logic rule to define each of the possible cases duringevaluation, and uses ML functions to manipulate and reason about complex terms representingevaluator state. For example, one rule defines when an assertion fails (Figure 13). This rule saysthat the path
Path ends in a failure with evaluator state St if: (1) there is an assert instruction Instr with argument X , (2) following Path has led the evaluator to that instruction with state St , (3) X could have the (possibly symbolic) integer value V in state St , and (4) V may be zero. The function may_be_zero(V, St) returns true if and only if V may be zero given St . We represent symbolicvalues as SMT formulas, so when V is symbolic, this function invokes the SMT solver.We have evaluated our symbolic evaluator on ten benchmarks based on five template programs.The first template (shuffle- 𝑁 ) non-deterministically shuffles an array of size 𝑁 and asserts that theresulting array represents the same set as the input array. The second template (sort- 𝑁 ; Figure 14)splits into two branches, sorts an array using selection sort in both branches, and asserts that theresulting array is sorted in the second branch. The third template completes a partially filled-in 4 × 𝑁 ) tests the equivalence of two implementations of a priority queue(one based on a heap, the other on an unsorted array) by pushing the same 𝑁 symbolic integerson them and verifying that they have the same behavior during a sequence of operations. Thefifth template (interp- 𝑁 ) runs an interpreter for a simple bytecode language for 𝑁 steps; the inputbytecode is represented by an array of symbolic integers that can be interpreted as commands forbinary operations, register loads and stores, and conditional jumps.We compared our times on these benchmarks against the symbolic execution tool KLEE (v2.1)[Cadar et al. 2008] and the bounded model checker CBMC (v5.11) [Clarke et al. 2004] (Table 2);we used a timeout of 30 minutes. These should not be taken as apples-to-apples comparisons:KLEE operates over all of LLVM bitcode and CBMC operates over C source code, whereas we For example, we found it timed out on batik, chart, fop, lusearch, and pmd (as did our implementation). failed_assert ( Path , St ) :-assert_instruction ( Instr , X),stepped ( Instr , St , _ , Path ),has_value (X , St , v_int (V)) ,may_be_zero (V , St ) = true .
Fig. 13. This rule states that the symbolic eval-uator has reached a failing assertion when theargument of the assert instruction may be zero. a := array of 𝑁 symbolic ints ;b := symbolic int ;if (b) { sort a; }else { sort a; assert a sorted ; } Fig. 14. This pseudocode sketches a C programthat creates an array, branches, sorts it in eachbranch, but asserts that the result is sorted onlyin one branch.Table 2. We report absolute times (mm:ss) for KLEE, CBMC, and a Formulog-based symbolic evaluation toolon ten benchmark programs; for the latter, we also report speedups ( ↑ ) and slowdowns ( ↓ ) relative to KLEE. Benchmark ↑ . × )shuffle-5 1,296 1:56 0:01 0:07 ( ↑ . × )sort-6 2,718 2:29 0:24 0:18 ( ↑ . × )sort-7 22,070 27:13 2:46 3:16 ( ↑ . × )numbrix-sat 1 0:15 0:01 1:10 ( ↓ . × )numbrix-unsat 1 0:15 0:01 0:59 ( ↓ . × )prioqueue-5 1,132 0:43 6:45 0:16 ( ↑ . × )prioqueue-6 4,409 3:24 T/O 1:10 ( ↑ . × )interp-5 994 0:55 0:05 0:39 ( ↑ . × )interp-6 3,433 3:19 0:12 T/O ( ↓ ∞× )handle just a fragment of LLVM bitcode; CBMC implements bounded model checking and notsymbolic execution, with the result that it generates many fewer (but presumably more complex)SMT queries; and all three tools might translate program constructs into SMT formulas in differentways, leading to different external solver performance. Nonetheless, these comparisons providesome context for our evaluation numbers.In general, our tool achieved speedups over KLEE, but did not quite match the performance ofCBMC. It performed relatively poorly for benchmarks with a single path (numbrix-sat and numbrix-unsat), but on most other programs we were able to achieve substantial speedups ( . × - . × )over KLEE and perform within striking distance of CBMC. This was at least partly due to the factthat our analysis is automatically parallelized, whereas KLEE and CBMC are single threaded. Theinterp- 𝑁 benchmarks caused trouble for our tool: Our trials for interp-5 had an unusually highdegree of variance (with two trials taking less than 40 seconds, and one trial taking ∼
18 minutes),and our tool timed out on interp-6. We suspect that this might be because, on this benchmark, ourtool generates SMT queries involving the theory of arrays, and our particular naive encoding mightbe leading to slowdowns with the external SMT solver. Additionally, our tool can be run in a goal-directed mode: If we only want to check that noassertion fails, we can add the query failed_assert(_Path, _St) , triggering the Formulog runtimeto rewrite our evaluator to explore only paths that could potentially lead to a failed assertion.For the sorting benchmarks (Figure 14), this means the symbolic evaluator can ignore the firstbranch of the program. This leads to significant performance gains, as we completed sort-6 in The shuffle- 𝑁 benchmarks are the only other ones during which our tool generates SMT queries with array constructs. ormulog: Datalog for SMT-Based Static Analysis 141:23 Table 3. Formulog analyses can be concise and close to the formal specifications. This table gives the numberof rules, non-nullary functions, and line counts for our case studies; in parentheses, we give the number ofrules and functions that correspond to the formal specifications (the rest handle other parts of the analyses,e.g., the termination checker in Dminor, and the context-insensitive points-to analysis used by the bottom-up points-to analysis). For comparison, we provide the number of rules and functions used in the formalspecifications, as well as the line count of the reference implementations. Our symbolic evaluator is not basedon a particular specification; we omit a LOC comparison with the reference implementations, KLEE andCBMC, as they handle much larger input languages and it would be difficult to isolate the parts of theircodebases that correspond to the language our analysis supports.
Formulog impl. Specification Reference impl.Analysis ∼
2K lines F ♯ & SMT-LIBBottom-up points-to 203 (47) 49 (28) 1.5K 19 8 15K lines JavaSymbolic evaluator 51 37 1K13 seconds and sort-7 in 2:18, representing increased speedups of 11.5 × and 11.8 × , respectively,over KLEE. We ran CBMC in a similar directed mode (it can use program slicing [Weiser 1984]to ignore parts of the program irrelevant to assertions); it was slightly slower than our Formulogimplementation, completing sort-6 in 28 seconds and sort-7 in 2:25. This suggests the potential ofFormulog’s automatic optimizations, which help make it competitive with hand-optimized systems. In this section, we evaluate the design of Formulog with respect to our case studies. We argue thatFormulog is an effective and usable tool for writing SMT-based analyses.
Formulog makes it possible to write SMT-based analyses in a way that is close to their mathematicalspecification, leading to concise encodings (Table 3) . Our implementations of the Dminor typechecker (Section 5.2) and the bottom-up points-to analysis (Section 5.3) directly mirror theirpublished formal specifications; our third case study (Section 5.4), which was not based on anyparticular formalization, would itself be the basis of a reasonable specification of symbolic evaluation.Formulog provides language features that are a good match for the way that SMT-based analysesare specified: algebraic data types naturally encode BNF grammars (a common feature in analysisspecifications); Horn clauses match judgments; ML functions fit helper functions; and the reificationof formulas as terms captures the way that formulas are treated in analysis specifications.As a corollary, analyses written in Formulog can be concise. Despite encoding quite complexlogic, each of our case studies is less than 1.5K lines of code. In the case of the points-to analysis, thisis 10 × smaller than the reference implementation (which also uses functionality defined externallyin Chord). This is partly because Scuba implements heuristics that we do not and Java is a verboselanguage; however, we suggest that much of the difference is because Formulog is a better fit forencoding the logic of the analysis than an imperative, object-oriented language like Java. Therelative concision of Formulog matches the results reported by previous work on Datalog-basedstatic analysis, which found that Datalog-based analyses can be orders of magnitude more concisethan counterparts written in more traditional languages [Whaley et al. 2005]. The ML fragment ofFormulog also helps it be concise, since ML expressions — through supporting sequenced, nested,and scoped computation — can encode logic that would be more verbose to write in Datalog.We have shown that three diverse case studies can be naturally encoded in Formulog, suggestingthat its design is a good match for a range of SMT-based analyses. However, not all analysis logic can be easily encoded in Formulog. There is currently no way to join facts, a useful operationfor abstract interpretation-based analyses [Cousot and Cousot 1977]. The restriction to stratifiednegation is sometimes too severe: For example, one Dminor rule for the type synthesis relation synth is not directly expressible in Formulog, because it is defined in terms of the negation of thetype well formedness relation, which is in turn defined using the relation synth . Finally, given itslack of mutable state, Formulog is probably not a good fit for analyses that can most naturally bespecified in an imperative manner, such as lazy abstraction model checking [Henzinger et al. 2002].Because Formulog is designed to be compatible with Datalog, we can expand the type of analysislogic it supports by taking advantage of research on Datalog extensions. For instance, lattice-basedrecursive aggregation [Madsen et al. 2016; Szabó et al. 2018] would make it possible to join facts,and local stratification [Przymusinski 1988] would support the Dminor logic we previously cited.
Formulog provides a rich and flexible language of formulas that supports the type of logic-basedreasoning found in SMT-based analyses.
The formula fragment of Formulog makes it possible touse formulas the way they need to be used by static analyses. A good example of this is thedecision to reify logical formulas as terms, a departure from the approach taken by constraintlogic programming [Jaffar and Lassez 1987; Jaffar and Maher 1994] and constrained Horn clause(CHC) solving [Bjørner et al. 2015; Grebenshchikov et al. 2012; Gurfinkel et al. 2015; Hoder andBjørner 2012], the two major previous paradigms for combining logic programming and constraintsolving. In these systems, constraints are represented as predicates, not terms, and an inferenceis made if the constraints in the body of a rule are satisfiable. This approach makes sense in thecontext of programming with constraints ; however, it seems overly restrictive in the context ofprogramming with formulas , which do not necessarily have to be used directly as constraints. Forexample, analyses like our Dminor type checker need to check the validity of a formula, which is theunsatisfiability of its negation. Checking validity does not easily fit in constraint-based paradigms,since constraint programming is built around satisfiability. Similarly, we might want to write ananalysis that uses Craig interpolants [Craig 1957]. One could imagine extending Formulog’s SMTinterface to include an operator interpolate that takes two formulas and returns a third (optional)formula, the interpolant; it is not clear how to do this in one of the constraint-based paradigms.Our treatment of formula variables through the constructor { 𝑒 }[ 𝑡 ] provides further evidence.This mechanism makes it easy to identify a formula variable with an object-level construct (e.g., avariable in the input program) by choosing for 𝑒 the expression representing that construct. It alsomakes it easy to create a variable that is guaranteed to be fresh relative to a set of constructs (e.g.,fresh with respect to an environment), an extremely useful operation. This is done by choosing for 𝑒 a tuple of the constructs that the variable needs to be fresh with respect to. We use this trick inboth the Dminor type checker and the symbolic evaluator. Crucially, this freshness mechanism isdeterministic, which means that we can safely rewrite Formulog programs and parallelize them.The logic programming language Calypso [Aiken et al. 2007; Hackett 2010] provides a similarmechanism, except that it requires that all the variables in a formula are identified by terms withthe same type; this severely limits its usability and is too restrictive for our case studies.Our case studies exercise a range of the SMT-LIB standard and demonstrate the richness of ourformula language. The case studies variously use algebraic data types and uninterpreted functions(the Dminor type checker and the bottom-up points-to analysis); bit vectors and arrays (the Dminortype checker and the symbolic evaluator); and integers, uninterpreted sorts, and quantifiers (theDminor type checker). It is easy to extend Formulog with additional theories (by adding newconstructors) and different types of logical reasoning (by adding new operators, like interpolate ).As Formulog so loosely couples Datalog evaluation and constraint solving, it is easy to swap in To get around this, our implementation uses a less precise rule that drops the negated premise. ormulog: Datalog for SMT-Based Static Analysis 141:25 new solver backends without major changes to the Formulog runtime; our prototype currentlysupports Z3 [de Moura and Bjørner 2008], CVC4 [Barrett et al. 2011], and Yices [Dutertre 2014].
The design of Formulog makes it possible to advantageously apply Datalog-style optimizations toSMT-based analyses, with the result that Formulog programs can compete with analyses written in moremature languages.
All of our case study implementations benefit from automatic parallelization:this scales our Dminor type checker and symbolic evaluator over the reference implementations,and helps our bottom-up points-to analysis be reasonably performant. The points-to analysis andsymbolic evaluator also demonstrate the potential of the magic set transformation, as we have usedit to derive demand-driven versions of these SMT-based analyses. While these types of optimizationscould be added by hand to the reference implementations we compare against, the point is that thedesign of Formulog means that Formulog-based analyses get these optimizations for free, withoutthe explicit effort of the analysis designer. Moreover, because of Formulog’s close affinity to Datalog,a Formulog runtime can be augmented with additional Datalog-style optimizations. For instance,a Formulog runtime could use an incremental Datalog evaluation algorithm, which efficientlyevaluates Datalog programs while facts are added or retracted from EDB relations [Gupta et al.1993; Szabó et al. 2018]. This would be helpful for using SMT-based analyses in situations wherethe code under analysis changes, such as in IDEs or rapidly evolving codebases.It speaks to the design of Formulog that the high-level optimizations it enables can, in many cases,make up for the naivety of our prototype runtime. Nonetheless, we are optimistic that significantlybetter performance can be achieved with a sophisticated backend. As we have designed Formulog tobe close to Datalog, we can take advantage of many of the optimizations that have helped Datalogsystems scale. For example, since we maintain the range restriction (which entails that every derivedfact is ground), we can use concurrent data structures specialized for Datalog evaluation [Jordanet al. 2019]; since Formulog can be evaluated using standard semi-naive evaluation, we can compileFormulog programs to C++ following Soufflé’s strategy [Jordan et al. 2016].
The ML fragment is an integral part of Formulog and has a substantial impact on its usability.
As discussed in Section 3.2, the first-order fragment of ML we use can be translated in a prettystraightforward way to Datalog rules, and hence can be thought of as syntactic sugar. Despite this,the ML fragment is an integral part of the Formulog programming experience. First, it improvesthe ergonomics of Formulog, by making it more natural to manipulate complex terms. In particular,pattern matching and let expressions provide a structured way to reflect on complex terms andsequence computation on them; this same effect is not always as easy to achieve in Datalog rules.Second, it helps Formulog achieve its design goal of allowing SMT-based analyses to be implementedin a style close to their specification, since formal specifications often involve functions in addition toinference rules. Third, it improves the performance of Formulog, as there is more overhead involvedwith evaluating Datalog rules than evaluating an ML expression. A substantial amount of our casestudy code is in the ML fragment: The ratio of functions to rules is 1:4 for the bottom-up points-toanalysis and 3:4 for the two other case studies (Table 3). Typically, the case studies use Horn clausesto define the overall structure of the analysis, and ML functions for structuring lower-level controlflow, mirroring the use of judgments and helper functions in analysis specifications.The limitation to first-order ML has several advantages. From a theoretical perspective, it meansthat there is an easy translation from it to Datalog rules, which allows us to give the standardHerbrand model-based semantics to Formulog programs. From a practical perspective, it ensures thatwe never have to unify functions, which would require higher-order unification. The specificationsof our case studies did not make heavy use of higher-order functions, so they were not much missed.However, a future version of Formulog could allow a limited use of higher-order functions (forexample, those programs that can be compiled to the first-order fragment).
Datalog-based frameworks and domain-specific languages for static analysis.
A variety of staticanalysis frameworks have been developed based on more-or-less standard Datalog, such as bddb-ddb [Whaley et al. 2005], Chord [Naik 2011], Doop [Bravenboer and Smaragdakis 2009], QL [Av-gustinov et al. 2016], and Soufflé [Scholz et al. 2016]. Recent work has explored synthesizingDatalog-based analyses [Albarghouthi et al. 2017; Raghothaman et al. 2019]. Flix [Madsen et al.2016] and IncA [Szabó et al. 2018] extend Datalog for analyses that operate over lattices besidesthe powerset lattice. IncA supports incremental evaluation, while Flix (like Formulog) includesalgebraic data types and a pure functional language. Dataflow analysis is used as a case study forDatafun, a language combining Datalog and higher-order functional programming [Arntzeniusand Krishnaswami 2016]. It might be possible to encode something like Formulog in Datafun;however, although it has recently been shown that Datafun can be evaluated using semi-naiveevaluation [Arntzenius and Krishnaswami 2020], it is not clear to what extent other Datalogoptimizations can be applied to Datafun programs. By combining Datalog with functional program-ming, Formulog, Flix, and Datafun are related to functional logic programming [Antoy and Hanus2010]. The functional fragment of Formulog is less expressive than what is typically found in suchlanguages, as Formulog functions are not first-class values and not higher-order.
Logic programming with constraints and formulas.
The two dominant prior paradigms for combin-ing logic programming and constraint solving are constraint logic programming (CLP) [Jaffar andLassez 1987; Jaffar and Maher 1994] and constrained Horn clause (CHC) solving [Bjørner et al. 2015;Grebenshchikov et al. 2012; Gurfinkel et al. 2015; Hoder and Bjørner 2012]. As discussed in Section 6,these systems typically encode constraints as predicates, not terms, and thus support programmingwith constraints as things to be satisfied, rather than programming with formulas, which can bemanipulated in more interesting ways (e.g., validity checking). In the context of static analysis,these systems have been used primarily for model checking, where a model of the input systemis encoded using Horn clauses [Bjørner et al. 2015; Delzanno and Podelski 1999; Flanagan 2004;Fribourg and Richardson 1996; Grebenshchikov et al. 2012]. The rules depend on the program beinganalyzed, and the solutions to these rules reveal properties of the model; e.g., SeaHorn [Gurfinkelet al. 2015] checks programs by solving a CHC representation of their verification conditions. Thisdiffers than the approach taken in this paper, where the rules encode an analysis independent ofthe input program. The Datalog mode of 𝜇 Z [Hoder et al. 2011] can be thought of as a bottom-upCLP system with special support for abstract interpretation.A few existing logic programming systems support programming with formulas (vs constraints);we would argue that none do so with the same richness and flexibility as Formulog. Codish et al.[2008] extend Prolog with an interface to a SAT solver. SICStus Prolog [Carlsson and Mildner2012], with its CLP extensions, has been used to write model checkers [Delzanno and Podelski1999; Fribourg and Richardson 1996; Grebenshchikov et al. 2012; Podelski and Rybalchenko 2007];these implementations typically rely on Prolog’s non-logical features, like assert , making it harderto apply high-level optimizations like parallelization. Calypso [Aiken et al. 2007; Hackett 2010] isa Datalog variant that interfaces with external constraint solvers and has specialized support forbottom-up analyses. Calypso has been used with SAT and integer constraint solvers; in theory, itcould be connected to an SMT solver. However, Formulog offers several advantages over Calypsofor SMT-based analyses. First, Formulog’s approach to constructing formulas (via complex terms)and manipulating them (via its ML fragment) scales to the complex and heterogeneous formulasthat arise in the SMT context, whereas Calypso’s approach to formulas (opaque terms, constructedvia predicates) would be cumbersome in this setting. Second, Formulog’s type system supportsthe construction of expressive (and safe) formulas involving user-defined terms such as algebraic ormulog: Datalog for SMT-Based Static Analysis 141:27 data types and uninterpreted functions. Third, the ML fragment of Formulog goes a long waytowards making it practical for SMT-based analyses, by closing the gap between specification andimplementation, and improving ergonomics and performance.The logic programming language 𝜆 Prolog provides a natural way to represent logical formulasusing a form of higher-order abstract syntax based on 𝜆 -terms and higher-order unification [Millerand Nadathur 1987; Pfenning and Elliott 1988]. Although this representation simplifies some aspectsof using formulas, moving to a higher-order setting would complicate Formulog, widen the gapbetween Formulog and other Datalog variants, and potentially be an impediment to building aperformant and scalable Formulog implementation. Answer set programming (ASP) uses specializedsolvers to find a stable model (if it exists) of a set of Horn clauses [Brewka et al. 2011; Gelfond andLifschitz 1988]. Common extensions support constraints on the shape of the stable model that willbe found. ASP enables concise encoding of classic NP-complete constraint problems such as graph 𝑘 -coloring, but it is not as obviously applicable to static analysis problems. Type system engineering.
PLT Redex [Felleisen et al. 2009] and Spoofax [Kats and Visser 2010]support exploratory type system engineering. PLT Redex supports a notion of judgment modeled ex-plicitly on inference rules. Spoofax’s type engineering framework, Statix, uses a logic programmingsyntax to specify type systems, with a custom solver for resolving the binding information in scopegraphs simultaneously with solving typing constraints [van Antwerpen et al. 2018]. Both of thesesystems use custom approaches to finding typing derivations; neither supports SMT queries, butStatix’s custom solver can resolve constraint systems that might not always terminate in Formulog.
Solver-aided languages.
Scala Z3 [Köksal et al. 2011] supports mixed computations combining nor-mal Scala evaluation and Z3 solving; we avoid this level of integration. Smten [Uhler and Dave 2013]is a solver-aided language that supports both concrete and symbolic evaluation; Rosette [Torlakand Bodik 2013] is a framework for creating solver-aided languages that have this property. Formulog is a domain-specific language for writing SMT-based static analyses that judiciouslycombines Datalog, ML, and SMT solving (via an external SMT solver). As demonstrated by our casestudies, it makes it possible to concisely implement a range of SMT-based analyses — refinementtype checking, bottom-up points-to analysis, and symbolic evaluation — in a way close to theirformal specifications, while also making it possible to automatically and advantageously applyhigh-level optimizations to these analyses like parallelization and goal-directed rewriting.
ACKNOWLEDGMENTS
This material is based upon work supported by the Defense Advanced Research Projects Agency(DARPA) under Contract No. FA8750-19-C-0004. Any opinions, findings and conclusions or recom-mendations expressed in this material are those of the author(s) and do not necessarily reflect theviews of the Defense Advanced Research Projects Agency (DARPA). We thank Arlen Cox, ScottMoore, the Harvard PL group, and anonymous reviewers for thoughtful feedback on earlier drafts.
REFERENCES
Alex Aiken, Suhabe Bugrara, Isil Dillig, Thomas Dillig, Brian Hackett, and Peter Hawkins. 2007. An Overview of theSaturn Project. In
Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools andEngineering . 43–48. https://doi.org/10.1145/1251535.1251543Aws Albarghouthi, Paraschos Koutris, Mayur Naik, and Calvin Smith. 2017. Constraint-Based Synthesis of DatalogPrograms. In
Proceedings of the 23rd International Conference on Principles and Practice of Constraint Programming .689–706. https://doi.org/10.1007/978-3-319-66158-2_44
Sergio Antoy and Michael Hanus. 2010. Functional Logic Programming.
Commun. ACM
53, 4 (2010), 74–85. https://doi.org/10.1145/1721654.1721675Krzysztof R Apt, Howard A Blair, and Adrian Walker. 1988. Towards a Theory of Declarative Knowledge. In
Foundations ofDeductive Databases and Logic Programming . Elsevier, 89–148. https://doi.org/10.1016/B978-0-934613-40-8.50006-3Molham Aref, Balder ten Cate, Todd J Green, Benny Kimelfeld, Dan Olteanu, Emir Pasalic, Todd L Veldhuizen, and GeoffreyWashburn. 2015. Design and Implementation of the LogicBlox System. In
Proceedings of the 2015 ACM SIGMODInternational Conference on Management of Data . 1371–1382. https://doi.org/10.1145/2723372.2742796Michael Arntzenius and Neel Krishnaswami. 2020. Seminäive Evaluation for a Higher-Order Functional Language.
Proceed-ings of the ACM on Programming Languages
4, POPL (2020), 22:1–22:28. https://doi.org/10.1145/3371090Michael Arntzenius and Neelakantan R. Krishnaswami. 2016. Datafun: A Functional Datalog. In
Proceedings of the 21st ACMSIGPLAN International Conference on Functional Programming . 214–227. https://doi.org/10.1145/2951913.2951948Pavel Avgustinov, Oege De Moor, Michael Peyton Jones, and Max Schäfer. 2016. QL: Object-Oriented Queries on RelationalData. In
Proceedings of the 30th European Conference on Object-Oriented Programming . 2:1–2:25. https://doi.org/10.4230/LIPIcs.ECOOP.2016.2Isaac Balbin, Graeme S. Port, Kotagiri Ramamohanarao, and Krishnamurthy Meenakshi. 1991. Efficient Bottom-upComputation of Queries on Stratified Databases.
The Journal Of Logic Programming
11, 3&4 (1991), 295–344.https://doi.org/10.1016/0743-1066(91)90030-SFrancois Bancilhon. 1986. Naive Evaluation of Recursively Defined Relations. In
On Knowledge Base Management Systems .Springer, 165–178. https://doi.org/10.1007/978-1-4612-4980-1_17Francois Bancilhon, David Maier, Yehoshua Sagiv, and Jeffrey D Ullman. 1985. Magic Sets and Other Strange Ways toImplement Logic Programs. In
Proceedings of the Fifth ACM SIGACT-SIGMOD Symposium on Principles of DatabaseSystems . 1–15. https://doi.org/10.1145/6012.15399Clark Barrett, Christopher L. Conway, Morgan Deters, Liana Hadarean, Dejan Jovanović, Tim King, Andrew Reynolds, andCesare Tinelli. 2011. CVC4. In
Proceedings of the 23rd International Conference on Computer Aided Verification
The Journal of Logic Programming
10, 3-4 (1991),255–299. https://doi.org/10.1016/0743-1066(91)90038-QAaron Bembenek, Michael Greenberg, and Stephen Chong. 2020. Formulog: Datalog for SMT-Based Static Analysis.
Proceedings of the ACM on Programming Languages
4, OOPSLA (2020), 141:1–141:31. https://doi.org/10.1145/3428209Gavin M. Bierman, Andrew D. Gordon, Cătălin Hriţcu, and David Langworthy. 2012. Semantic Subtyping with an SMTSolver.
Journal of Functional Programming
22, 1 (2012), 31–105. https://doi.org/10.1145/1863543.1863560Nikolaj Bjørner, Arie Gurfinkel, Ken McMillan, and Andrey Rybalchenko. 2015. Horn Clause Solvers for Program Verification.In
Fields of Logic and Computation II . Springer, 24–51. https://doi.org/10.1007/978-3-319-23534-9_2Martin Bravenboer and Yannis Smaragdakis. 2009. Strictly Declarative Specification of Sophisticated Points-to Analyses. In
Proceedings of the 24th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications .243–262. https://doi.org/10.1145/1640089.1640108Gerhard Brewka, Thomas Eiter, and Mirosław Truszczyński. 2011. Answer Set Programming at a Glance.
Commun. ACM
54, 12 (2011), 92–103. https://doi.org/10.1145/2043174.2043195Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-CoverageTests for Complex Systems Programs. In
Proceedings of the 8th USENIX Conference on Operating Systems Design andImplementation . 209–224.Cristian Cadar and Koushik Sen. 2013. Symbolic Execution for Software Testing: Three Decades Later.
Commun. ACM
56, 2(Feb. 2013), 82–90. https://doi.org/10.1145/2408776.2408795Mats Carlsson and Per Mildner. 2012. SICStus Prolog–The First 25 years.
Theory and Practice of Logic Programming
12, 1-2(2012), 35–66. https://doi.org/10.1017/S1471068411000482Alessandro Cimatti and Alberto Griggio. 2012. Software Model Checking via IC3. In
Proceedings of the 24th InternationalConference on Computer Aided Verification . 277–293. https://doi.org/10.1007/978-3-642-31424-7_23Edmund Clarke, Daniel Kroening, and Flavio Lerda. 2004. A Tool for Checking ANSI-C Programs. In
Proceedings of the10th International Conference on Tools and Algorithms for the Construction and Analysis of Systems . 168–176. https://doi.org/10.1007/978-3-540-24730-2_15Michael Codish, Vitaly Lagoon, and Peter J Stuckey. 2008. Logic Programming with Satisfiability.
Theory and Practice ofLogic Programming
8, 1 (2008), 121–128. https://doi.org/10.1017/S1471068407003146Patrick Cousot and Radhia Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programsby Construction or Approximation of Fixpoints. In
Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principlesof Programming Languages . 238–252. https://doi.org/10.1145/512950.512973 ormulog: Datalog for SMT-Based Static Analysis 141:29
William Craig. 1957. Three Uses of the Herbrand-Gentzen Theorem in Relating Model Theory and Proof Theory.
TheJournal of Symbolic Logic
22, 3 (1957), 269–285. https://doi.org/10.2307/2963594Luis Damas and Robin Milner. 1982. Principal Type-Schemes for Functional Programs. In
Proceedings of the 9th ACMSIGPLAN-SIGACT Symposium on Principles of Programming Languages . 207–212. https://doi.org/10.1145/582153.582176Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In
Proceedings of the 14th International Conferenceon Tools and Algorithms for the Construction and Analysis of Systems . 337–340. https://doi.org/10.1007/978-3-540-78800-3_24Giorgio Delzanno and Andreas Podelski. 1999. Model Checking in CLP. In
Proceedings of the 5th International Conference onTools and Algorithms for the Construction and Analysis of Systems . 223–239. https://doi.org/10.1007/3-540-49059-0_16David Detlefs, Greg Nelson, and James B. Saxe. 2005. Simplify: A Theorem Prover for Program Checking.
J. ACM
52, 3(2005), 365–473. https://doi.org/10.1145/1066100.1066102Bruno Dutertre. 2014. Yices 2.2. In
Proceedings of the 26th International Conference on Computer Aided Verification . 737–744.https://doi.org/10.1007/978-3-319-08867-9_49Matthias Felleisen, Robert Bruce Findler, and Matthew Flatt. 2009.
Semantics Engineering with PLT Redex (1st ed.). The MITPress.Yu Feng, Xinyu Wang, Isil Dillig, and Thomas Dillig. 2015. Bottom-up Context-Sensitive Pointer Analysis for Java. In
Proceedings of the 13th Asian Symposium on Programming Languages and Systems . 465–484. https://doi.org/10.1007/978-3-319-26529-2_25Cormac Flanagan. 2004. Automatic Software Model Checking via Constraint Logic.
Science of Computer Programming . 1075–1092.Laurent Fribourg and Julian Richardson. 1996. Symbolic Verification with Gap-Order Constraints. In
Proceedings of the 6thInternational Workshop on Logic Programming Synthesis and Transformation . 20–37. https://doi.org/10.1007/3-540-62718-9_2Hervé Gallaire and Jack Minker (Eds.). 1978.
Logic and Data Bases . Plenum Press.Michael Gelfond and Vladimir Lifschitz. 1988. The Stable Model Semantics for Logic Programming. In
Proceedings of the 5thInternational Conference and Symposium on Logic Programming . 1070–1080.Sergey Grebenshchikov, Nuno Lopes, Corneliu Popeea, and Andrey Rybalchenko. 2012. Synthesizing Software Verifiers fromProof Rules. In
Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation .405–416. https://doi.org/10.1145/2254064.2254112Neville Grech, Lexi Brent, Bernhard Scholz, and Yannis Smaragdakis. 2019. Gigahorse: Thorough, Declarative Decompilationof Smart Contracts. In
Proceedings of the 41st International Conference on Software Engineering . 1176–1186. https://doi.org/10.1109/ICSE.2019.00120Neville Grech, Michael Kong, Anton Jurisevic, Lexi Brent, Bernhard Scholz, and Yannis Smaragdakis. 2018. Madmax:Surviving Out-of-Gas Conditions in Ethereum Smart Contracts.
Proceedings of the ACM on Programming Languages
Foundations and Trends in Databases
5, 2 (2013), 105–195. https://doi.org/10.1561/1900000017Salvatore Guarnieri and V Benjamin Livshits. 2009. GATEKEEPER: Mostly Static Enforcement of Security and ReliabilityPolicies for JavaScript Code. In
Proceedings of the 18th USENIX Security Symposium . 78–85.Ashish Gupta, Inderpal Singh Mumick, and Venkatramanan Siva Subrahmanian. 1993. Maintaining Views Incrementally.
ACM SIGMOD Record
22, 2 (1993), 157–166. https://doi.org/10.1145/170035.170066Arie Gurfinkel, Temesghen Kahsai, Anvesh Komuravelli, and Jorge A Navas. 2015. The SeaHorn Verification Framework. In
Proceedings of the 27th International Conference on Computer Aided Verification . 343–361. https://doi.org/10.1007/978-3-319-21690-4_20Brian Hackett. 2010.
Type Safety in the Linux Kernel . Ph.D. Dissertation. Stanford University.Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2002. Lazy Abstraction. In
Proceedings of the 29thACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages . 58–70. https://doi.org/10.1145/503272.503279Roger Hindley. 1969. The Principal Type-Scheme of an Object in Combinatory Logic.
Trans. Amer. Math. Soc.
146 (1969),29–60. https://doi.org/10.2307/1995158Kryštof Hoder and Nikolaj Bjørner. 2012. Generalized Property Directed Reachability. In
Proceedings of the 15th InternationalConference on Theory and Applications of Satisfiability Testing . Springer, 157–171. https://doi.org/10.1007/978-3-642-31612-8_13Kryštof Hoder, Nikolaj Bjørner, and Leonardo De Moura. 2011. 𝜇 Z–An Efficient Engine for Fixed Points with Constraints.In
Proceedings of the 23rd International Conference on Computer Aided Verification . 457–462. https://doi.org/10.1007/978-3-642-22110-1_36
Joxan Jaffar and Jean-Louis Lassez. 1987. Constraint Logic Programming. In
Proceedings of the 14th ACM SIGACT-SIGPLANSymposium on Principles of Programming Languages . 111–119. https://doi.org/10.1145/41625.41635Joxan Jaffar and Michael J. Maher. 1994. Constraint Logic Programming: A Survey.
The Journal of Logic Programming
Proceedings of the28th International Conference on Computer Aided Verification . 422–430. https://doi.org/10.1007/978-3-319-41540-6_23Herbert Jordan, Pavle Subotic, David Zhao, and Bernhard Scholz. 2019. A Specialized B-tree for Concurrent DatalogEvaluation. In
Proceedings of the 24th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming .327–339. https://doi.org/10.1145/3293883.3295719Lennart C.L. Kats and Eelco Visser. 2010. The Spoofax Language Workbench: Rules for Declarative Specification of Languagesand IDEs. In
Proceedings of the 25th ACM International Conference on Object-Oriented Programming, Systems, Languages,and Applications . 444–463. https://doi.org/10.1145/1869459.1869497James C. King. 1976. Symbolic Execution and Program Testing.
Commun. ACM
19, 7 (1976), 385–394. https://doi.org/10.1145/360248.360252Ali Sinan Köksal, Viktor Kuncak, and Philippe Suter. 2011. Scala to the Power of Z3: Integrating SMT and Programming. In
Proceedings of the 23rd International Conference on Automated Deduction . 400–406. https://doi.org/10.1007/978-3-642-22438-6_30Robert Kowalski. 1979. Algorithm = Logic + Control.
Commun. ACM
22, 7 (1979), 424–436. https://doi.org/10.1145/359131.359136Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation.In
Proceedings of the 2nd IEEE/ACM International Symposium on Code Generation and Optimization . 75–88. https://doi.org/10.1109/CGO.2004.1281665Chin Soon Lee, Neil D. Jones, and Amir M. Ben-Amram. 2001. The Size-Change Principle for Program Termination. In
Proceedings of the 28th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages . 81–92. https://doi.org/10.1145/360204.360210V. Benjamin Livshits and Monica S. Lam. 2005. Finding Security Vulnerabilities in Java Applications with Static Analysis. In
Proceedings of the 14th USENIX Security Symposium . 271–286.Magnus Madsen, Ming-Ho Yee, and Ondřej Lhoták. 2016. From Datalog to Flix: a Declarative Language for Fixed Pointson Lattices. In
Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation .194–208. https://doi.org/10.1145/2908080.2908096Kenneth L McMillan. 2006. Lazy Abstraction with Interpolants. In
Proceedings of the 18th International Conference onComputer Aided Verification . Springer, 123–136. https://doi.org/10.1007/11817963_14Michael Meskes and Jörg Noack. 1993. The Generalized Supplementary Magic-Sets Transformation for Stratified Datalog.
Inform. Process. Lett.
47, 1 (1993), 31–41. https://doi.org/10.1016/0020-0190(93)90154-2Dale Miller and Gopalan Nadathur. 1987. A Logic Programming Approach to Manipulating Formulas and Programs. In
Proceedings of the 1987 Symposium on Logic Programming . 379–388.Inderpal Singh Mumick, Hamid Pirahesh, and Raghu Ramakrishnan. 1990. The Magic of Duplicates and Aggregates. In
Proceedings of the 16th International Conference on Very Large Data Bases
Proceedings of the ACM SIGPLAN 1988 Conferenceon Programming Language Design and Implementation . 199–208. https://doi.org/10.1145/53990.54010Andreas Podelski and Andrey Rybalchenko. 2007. ARMC: The Logical Choice for Software Model Checking with AbstractionRefinement. In
Proceedings of the 9th International Symposium on Practical Aspects of Declarative Languages . 245–259.https://doi.org/10.1007/978-3-540-69611-7_16Teodor C Przymusinski. 1988. On the Declarative Semantics of Deductive Databases and Logic Programs. In
Foundations ofDeductive Databases and Logic Programming . Elsevier, 193–216. https://doi.org/10.1016/b978-0-934613-40-8.50009-9Mukund Raghothaman, Jonathan Mendelson, David Zhao, Mayur Naik, and Bernhard Scholz. 2019. Provenance-GuidedSynthesis of Datalog Programs.
Proceedings of the ACM on Programming Languages
4, POPL (2019), 1–27. https://doi.org/10.1145/3371130Thomas W. Reps. 1995. Demand Interprocedural Program Analysis Using Logic Databases. In
Proceedings of the 3rd ACMSIGSOFT Symposium on Foundations of Software Engineering . 163–196.Patrick M. Rondon, Ming Kawaguci, and Ranjit Jhala. 2008. Liquid Types. In
Proceedings of the 29th ACM SIGPLAN Conferenceon Programming Language Design and Implementation . 159–169. https://doi.org/10.1145/1375581.1375602Bernhard Scholz, Herbert Jordan, Pavle Subotić, and Till Westmann. 2016. On Fast Large-Scale Program Analysis in Datalog.In
Proceedings of the 25th International Conference on Compiler Construction . 196–206. https://doi.org/10.1145/2892208.2892226 ormulog: Datalog for SMT-Based Static Analysis 141:31
Yannis Smaragdakis and Martin Bravenboer. 2011. Using Datalog for Fast and Easy Program Analysis. In
Datalog Reloaded .Springer, 245–251. https://doi.org/10.1007/978-3-642-24206-9_14Tamás Szabó, Gábor Bergmann, Sebastian Erdweg, and Markus Voelter. 2018. Incrementalizing Lattice-Based ProgramAnalyses in Datalog.
Proceedings of the ACM on Programming Languages
2, OOPSLA (2018), 139:1–139:29. https://doi.org/10.1145/3276509Emina Torlak and Rastislav Bodik. 2013. Growing Solver-Aided Languages with Rosette. In
Proceedings of the 2013ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software . 135–152.https://doi.org/10.1145/2509578.2509586Petar Tsankov, Andrei Dan, Dana Drachsler-Cohen, Arthur Gervais, Florian Buenzli, and Martin Vechev. 2018. Securify:Practical Security Analysis of Smart Contracts. In
Proceedings of the 2018 ACM SIGSAC Conference on Computer andCommunications Security . 67–82. https://doi.org/10.1145/3243734.3243780Richard Uhler and Nirav Dave. 2013. Smten: Automatic Translation of High-Level Symbolic Computations into SMT Queries.In
Proceedings of the 25th International Conference on Computer Aided Verification . 678–683. https://doi.org/10.1007/978-3-642-39799-8_45Hendrik van Antwerpen, Casper Bach Poulsen, Arjen Rouvoet, and Eelco Visser. 2018. Scopes as Types.
Proceedings of theACM on Programming Languages
2, OOPSLA (2018), 114:1–114:30. https://doi.org/10.1145/3276484Allen Van Gelder. 1989. Negation as Failure Using Tight Derivations for General Logic Programs.
The Journal of LogicProgramming
6, 1-2 (1989), 109–133. https://doi.org/10.1016/0743-1066(89)90032-0Mark Weiser. 1984. Program Slicing.
IEEE Transactions on Software Engineering
Proceedings of the Third Asian Symposium on Programming Languages and Systems . 97–118.https://doi.org/10.1007/11575467_8John Whaley and Monica S. Lam. 2004. Cloning-Based Context-Sensitive Pointer Alias Analysis Using Binary DecisionDiagrams. In
Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation .131–144. https://doi.org/10.1145/996841.996859
Types
Types 𝜏 :: = 𝑡 | 𝑡 smt | 𝑡 sym | model Pre-types 𝑡 :: = 𝐵 | 𝐷 (cid:174) 𝜏 | 𝛼 Base types 𝐵 :: = bool | bv [ k ] 𝑘 ∈ N + | . . . Contexts
Data type declarations Δ :: = · | Δ , 𝐷 : ∀ (cid:174) 𝛼 𝑖 . {−−−−→ 𝑐 𝑗 : (cid:174) 𝜏 𝑘 } Program declarations Φ :: = · | Φ , 𝑓 : ∀ (cid:174) 𝛼, (cid:174) 𝜏 → 𝜏 | Φ , uf : (cid:174) 𝑡 → 𝑡 | Φ , 𝑝 ⊆ (cid:174) 𝜏 Variable contexts Γ :: = · | Γ , 𝑥 : 𝜏 | Γ , 𝛼 Terms
Programs prog :: = (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 Functions 𝐹 :: = fun 𝑓 ( (cid:174) 𝑋 𝑖 : (cid:174) 𝜏 𝑖 ) : 𝜏 = 𝑒 Horn clauses 𝐻 :: = 𝑝 ( (cid:174) 𝑋 𝑖 ) : − (cid:174) 𝑃 𝑗 Premises 𝑃 :: = 𝐴 | ! 𝐴 Atoms 𝐴 :: = 𝑝 ( (cid:174) 𝑒 𝑖 ) | 𝑋 = 𝑒 Expressions 𝑒 :: = 𝑘 | 𝑋 | 𝑐 ( (cid:174) 𝑒 𝑖 ) | 𝑓 ( (cid:174) 𝑒 𝑖 ) | 𝑝 ( (cid:174) 𝑒 𝑖 ) | ⊗( (cid:174) 𝑒 𝑖 ) | ` 𝜙 ` let 𝑋 = 𝑒 in 𝑒 | if 𝑒 then 𝑒 else 𝑒 | match 𝑒 with −−−−−−−−−−→ 𝑐 𝑖 ( (cid:174) 𝑋 𝑗 ) → 𝑒 𝑖 SMT formulas 𝜙 :: = 𝑐 SMTvar [ 𝑥, 𝑡 ] () | 𝑐 SMTconst [ 𝑘 ] () | 𝑐 SMTlet ( 𝜙 , 𝜙 , 𝜙 ) | 𝑐 SMTctor [ 𝑐 ] ( (cid:174) 𝜙 𝑖 ) | 𝑐 SMTforall ( 𝜙 , 𝜙 ) | 𝑐 SMTuf [ uf ] ( (cid:174) 𝜙 𝑖 ) | , 𝑒 Constants 𝑘 :: = true | false | | | . . . Namespaces
Type modes 𝑚 :: = exp | smt Data type names 𝐷 ∈ ADTVar
Type variables 𝛼 ∈ TVar
Constructors 𝑐 ∈ CtorVar
Formulog variables 𝑋 ∈ Var
SMT variables 𝑥 ∈ SMTVar
Predicates 𝑝 ∈ PredVar
Functions 𝑓 ∈ FunVar
Uninterpreted functions uf ∈ UninterpFunVar
Fig. 15. Syntax of Formulog’s formal model
A FORMULOG’S FORMAL MODEL
We define a ‘middleweight’ formal model of Formulog, designing a type system (Section B) and anoperational semantics (Section C), relating the two in a proof of type safety (Section D).Our model characterizes Formulog as a two-level system (Figure 15), comprising Datalog-esqueHorn clauses 𝐻 and first-order functions 𝐹 ; Horn clause “rules” are made up of premises 𝑃 , whereeach premise is a series of (possibly negated) atoms 𝐴 . Each atom 𝐴 either references a Datalogpredicate or binds a variable to an expression 𝑒 . Expressions themselves have two mutually recursive ormulog: Datalog for SMT-Based Static Analysis 141:33 modes: ordinary functional computation 𝑒 and quoted SMT terms ` 𝜙 ` , which can include unquotedexpressions , 𝑒 .The Datalog fragment of Formulog is fairly standard syntactically, up to the addition of theatomic form 𝑋 = 𝑒 . We constrain premises to a sort of administrative normal form: predicatereferences apply only to variables, written 𝑝 ( (cid:174) 𝑋 𝑖 ) , and expression constraints bind variables, as in 𝑌 = 𝑒 . Our implementation can handle compound premises like 𝑝 ( 𝑒 , 𝑒 ) ; our formal model wouldrequire rewriting such a premise to three premises: 𝑝 ( 𝑋, 𝑌 ) , 𝑋 = 𝑒 , and 𝑌 = 𝑒 (for some fresh 𝑋 and 𝑌 ).The functional programming fragment fully annotates the types on its functions 𝐹 ; variablenames in both fragments are written in capital letters. (Our implementation merely demands thatthe first letter be capitalized.) SMT variables are written using lowercase letters and annotated withtheir type, as in 𝑐 SMTvar [ 𝑥, 𝑡 ] () . As described in Section 3.3, our implementation allows any value tobe used as the name of an SMT variable; here, without loss of generality, we treat SMT variables asbeing drawn from a distinct universe. Code in the functional fragment can treat Datalog relationsas predicates, i.e., 𝑝 ( (cid:174) 𝑣 𝑖 ) returns true when (cid:174) 𝑣 𝑖 ∈ 𝑝 . In our implementation, some elements of (cid:174) 𝑣 𝑖 canbe the wildcard ?? , turning a Datalog predicate into a list. For example, if 𝑝 ⊆ bool × bv [ ] , then: 𝑝 ( true , ) yields a bool ; 𝑝 ( ?? , ) returns a list of bool s 𝑏 such that 𝑝 ( 𝑏, ) holds; 𝑝 ( true , ?? ) returns a list of bv [ ] s 𝑛 such that 𝑝 ( true , 𝑛 ) holds; 𝑝 ( ?? , ?? ) returns a list of bool × bv [ ] , i.e.,the relation 𝑝 . We don’t include this behavior in our formal model.The set of available base types 𝐵 must include bool at a minimum; any other SMT-embeddablebase types are acceptable, e.g., 𝑘 -width bit vectors for a statically known 𝑘 .As a matter of notation, we write (cid:174) 𝑒 𝑖 for a metavariable 𝑒 to mean a possibly empty sequence of 𝑒 s,indexed by 𝑖 . When more than one variable shares the same index, we mean that those sequencesmust be of the same length (e.g., in the match syntax, each branch of a match is a triple of aconstructor 𝑐 , a vector of variable names for 𝑐 ’s arguments, and a corresponding single expression). B FORMULOG’S TYPE SYSTEM
We begin by presenting type checking rules for Formulog (Figures 16, 17, 18, and 19). Our imple-mentation of Formulog not only performs type checking, but can also perform type inference, e.g.,automatically finding type variable substitutions.Our types are broken into two levels: types 𝜏 and pre-types 𝑡 . Every pre-type 𝑡 can be directlyconsidered as a type, but there are two additional types: 𝑡 smt , the type of SMT formulas yielding 𝑡 , and 𝑡 sym , the type of SMT variables of type 𝑡 . We factor the syntax in this way to preventanomalies like bool smt smt , which would mean SMT formulas that yield SMT formulas that yieldbooleans. It is not the case, however, that every pre-type 𝑡 is necessarily representable as an SMTtype, because data types may contain SMT formulas as arguments; we discuss how we categorizeSMT-representable types shortly.Before we begin, some further notational clarification. Rules are named by their primary subjectsfollowed by a hyphen and a descriptive name. Whenever we use indices in rules, we will alwaysmap (stating a single premise in terms of the index, e.g., prog -WF) or fold (stating first, indexed, andlast, e.g., (cid:174) 𝑋 (cid:174) 𝜏 -All) over the sequence. We omit the indices when selecting an element of a sequenceor set (as in, e.g., 𝑒 -Match in Figure 18).All of our typing rules are in terms of a fixed set of data type declarations Δ and programdeclarations Φ (Figure 15). Data type declarations Δ map data type names 𝐷 to some number oftype arguments (cid:174) 𝛼 𝑖 and a set of constructors (cid:174) 𝑐 𝑗 , each of which takes some number of argumentsof type (cid:174) 𝜏 𝑘 ; each 𝑐 𝑗 can have a different number of arguments. Program declarations Φ collect the Type and typing context well formedness Δ ⊢ Γ Δ ; Γ ⊢ 𝑚 𝜏 ⊢ · Γ -Empty ⊢ Γ Γ ⊢ exp 𝜏 ⊢ Γ , 𝑥 : 𝜏 Γ -Var ⊢ Γ ⊢ Γ , 𝛼 Γ -TVar Γ ⊢ 𝑚 𝐵 𝑡 -Base 𝛼 ∈ ΓΓ ⊢ exp 𝛼 𝑡 -TVar Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑖 , { . . . } Γ ⊢ 𝑚 𝜏 𝑖 Γ ⊢ 𝑚 𝐷 (cid:174) 𝜏 𝑖 𝑡 -ADT Γ ⊢ smt 𝑡 Γ ⊢ 𝑚 𝑡 smt 𝜏 -SMT Γ ⊢ smt 𝑡 Γ ⊢ 𝑚 𝑡 sym 𝜏 -Sym Γ ⊢ exp model 𝜏 -Model Data type and program signature well formedness ⊢ Δ ⊢ Φ ⊢ Δ ⇔ ∀ 𝐷 : ∀ (cid:174) 𝛼 . { 𝑐 : (cid:174) 𝜏 , . . . , 𝑐 𝑛 : (cid:174) 𝜏 𝑛 } ∈ Δ ∀ 𝑖, (1) ∀ 𝐷 ′ ∈ dom ( Δ ) , 𝑐 𝑖 ∈ Δ ( 𝐷 ′ ) ⇒ 𝐷 = 𝐷 ′ (2) (cid:174) 𝛼 ⊢ exp 𝜏 𝑖 (3) ∀ 𝛽 ∈ (cid:174) 𝛼, 𝛽 ∈ (cid:174) 𝜏 𝑖 ⊢ · Φ -empty ⊢ Φ ∀ 𝛽 ∈ (cid:174) 𝛼 𝑖 , 𝛽 ∈ (cid:174) 𝜏 𝑗 , 𝜏 (cid:174) 𝛼 𝑖 ⊢ exp 𝜏 𝑗 (cid:174) 𝛼 𝑖 ⊢ exp 𝜏 ⊢ Φ , 𝑓 : ∀ (cid:174) 𝛼 𝑖 , (cid:174) 𝜏 𝑗 → 𝜏 Φ -Fun ⊢ Φ · ⊢ exp 𝜏 𝑖 ⊢ Φ , 𝑝 ⊆ (cid:174) 𝜏 𝑖 Φ -Rel ⊢ Φ · ⊢ smt 𝑡 𝑖 · ⊢ smt 𝑡 ⊢ Φ , uf : (cid:174) 𝑡 𝑖 → 𝑡 Φ -UFun Program and function typing Δ ; Φ ⊢ prog ⊢ Δ ⊢ Φ Δ ; Φ ⊢ 𝐹 𝑖 Δ ; Φ ⊢ 𝐻 𝑗 Δ ; Φ ⊢ (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 prog -WF Fig. 16. Type, context, and definition well formedness; top-level program typing signatures of first-order polymorphic functions 𝑓 : ∀ (cid:174) 𝛼, (cid:174) 𝜏 → 𝜏 , uninterpreted functions for use inthe SMT solver uf : (cid:174) 𝑡 → 𝑡 , and relations 𝑝 ⊆ (cid:174) 𝜏 𝑖 .The highest level typing rule is prog -WF (Figure 16), which ensures that the declarations arewell formed and each part of the program is well formed.The context and type well formedness rules (Figure 16) are mostly straightforward, type wellformedness being the most interesting. Each type can be found to be well formed in either SMTmode smt —i.e., it can be exported to the SMT solver–or in expression mode exp , meaning it cannotbe. There is a sub-moding relationship: well formed types at smt are also well formed at exp , butnot necessarily vice-versa: for example, there is no way to export an SMT formula or variable as the object of another SMT formula, only as a constituent. We assume that all Formulog constants areSMT representable, i.e., · ⊢ smt typeof ( 𝑘 ) for all constants 𝑘 . Data type declarations are polymorphic,but we disallow phantom type variables. Data types can freely mutually recurse. Uninterpretedfunctions must be in terms of pre-types, and those pre-types must be closed and SMT representable( smt ); functions and relations can use any well formed types ( exp ). Functions can be polymorphicbut we disallow phantom type variables; relations have monomorphic types. Disallowing phantomtypes in constructors and functions and keeping relations monomorphic ensure that these formsare “reverse determinate”, i.e., the types of their arguments uniquely determine their types. ormulog: Datalog for SMT-Based Static Analysis 141:35 Variable binding and typing Γ ⊢ 𝑥, 𝜏 ▷ Γ Γ ⊢ (cid:174) 𝑥, (cid:174) 𝜏 ▷ Γ 𝑋 ∉ dom ( Γ ) Γ ⊢ 𝑋, 𝜏 ▷ Γ , 𝑋 : 𝜏 𝑋𝜏 -Bind Γ ( 𝑋 ) = 𝜏 Γ ⊢ 𝑋, 𝜏 ▷ Γ 𝑋𝜏 -Check Γ ⊢ 𝑋 , 𝜏 ▷ Γ . . . Γ 𝑖 ⊢ 𝑋 𝑖 , 𝜏 𝑖 ▷ Γ 𝑖 + . . . Γ 𝑛 ⊢ 𝑋 𝑛 , 𝜏 𝑛 ▷ Γ ′ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ (cid:174) 𝑋 (cid:174) 𝜏 -All Premise typing Δ ; Φ ; Γ ⊢ 𝑃 ▷ Γ 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ Γ ⊢ 𝑝 ( (cid:174) 𝑋 𝑖 ) ▷ Γ ′ 𝑃 -PosAtom 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ ΓΓ ⊢ ! 𝑝 ( (cid:174) 𝑋 𝑖 ) ▷ Γ 𝑃 -NegAtom (cid:174) 𝑋 𝑖 ⊈ Γ Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 : (cid:174) 𝜏 𝑖 , . . . } Γ ⊢ 𝑌, 𝐷 (cid:174) 𝜏 ′ 𝑗 ▷ Γ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 [ (cid:174) 𝜏 ′ 𝑗 / (cid:174) 𝛼 𝑗 ] ▷ Γ ′ Γ ⊢ 𝑌 = 𝑐 ( (cid:174) 𝑋 𝑖 ) ▷ Γ ′ 𝑃 -EqCtor-BF (cid:174) 𝑋 𝑖 ⊈ Γ Γ ⊢ 𝑐 SMTc ′ : (cid:174) 𝜏 𝑖 → 𝜏 Γ ⊢ 𝑌, 𝜏 ▷ Γ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ Γ ⊢ 𝑌 = ` 𝑐 SMTc ′ ( (cid:174) , 𝑋 𝑖 ) ` ▷ Γ ′ 𝑃 -EqSMT-BF Γ ⊢ 𝑒 : 𝜏 Γ ⊢ 𝑌, 𝜏 ▷ Γ ′ Γ ⊢ 𝑌 = 𝑒 ▷ Γ ′ 𝑃 -Eq-FB Γ ⊢ 𝑒 : 𝜏 Γ ⊢ 𝑌, 𝜏 ▷ ΓΓ ⊢ ! ( 𝑌 = 𝑒 ) ▷ Γ 𝑃 -NegEq Clause typing Δ ; Φ ⊢ 𝐻 · ⊢ 𝑃 ▷ Γ . . . Γ 𝑗 ⊢ 𝑃 𝑗 ▷ Γ 𝑗 + . . . Γ 𝑛 ⊢ 𝑃 𝑛 ▷ Γ ′ 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ′ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ ⊢ 𝑝 ( (cid:174) 𝑋 𝑖 ) : − (cid:174) 𝑃 𝑗 𝐻 -Clause Fig. 17. Typing rules: Horn clauses (rules)
Since the declaration environments Δ and Φ are statically determined for an entire program, wetypically leave them implicit . Implicit parameters are in gray in the boxed rule schemata in thefigures. In proofs we will treat these parameters explicitly, but we conserve space by stating therules without threading implicit parameters through. For example, the data type declarations Δ arenecessary to ensure that 𝑡 -ADT only allows us to name data types that have actually been defined.Rather than threading Δ through every rule for context and type well formedness, we write Δ inthe rule schemata.The type checking of the Datalog fragment of Formulog (Figure 17) must encode two Dataloginvariants in addition to conventional typing constraints: the range restriction, i.e., every variablein the head of a rule appears somewhere in a premise; and appropriate binding, i.e., it is possible tointerpret a Horn clause in such a way that all of the variables will be bound at the end. Our formalrules ensure that the program has correct binding structure for a left-to-right evaluation of each Horn clause. An implementation could determine whether or not an ordering would work andcould reorder programs into an appropriate order automatically. Our formal model does not enforcethat the dependencies between relations are appropriately stratified, though doing so would beeasy: the relation-and-function call graph should not have any “negative” edge in a cycle, wherea negative edge is created whenever there is a negated predicate in a rule body or a predicate isinvoked as a function.Concretely, 𝐻 -Clause ensures that (a) a left-to-right binding order produces some appropriatefinal context Γ ′ (via the premise typing judgment), (b) the range restriction is satisfied, ( (cid:174) 𝑋 𝑖 ⊆ Γ ′ ) bymaking sure that (c) every variable is well typed and bound ( Γ ′ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ —having the same Γ ′ means no new bindings were introduce when checking the head variables).Premise typing Γ ⊢ 𝑃 ▷ Γ and variable binding and typing Γ ⊢ 𝑥, 𝜏 ▷ Γ work together to generateappropriate types for each premise. Positive references to relations are well formed in binding Γ ′ according to 𝑃 -PosAtom when (a) the use is well typed ( 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ ) and (b) the variables usedin the premise yield the binding Γ ′ . Negative references to relations ! 𝑝 ( (cid:174) 𝑋 𝑖 ) additionally requirethat all of the 𝑋 𝑖 be already bound, i.e., the resulting Γ is the same as the starting one. We splitexpression equality constraints 𝑌 = 𝑒 into three main cases:(1) 𝑌 is bound and 𝑒 is a constructor 𝑐 ( (cid:174) 𝑋 𝑖 ) where all of (cid:174) 𝑋 𝑖 are unbound ( 𝑃 -EqCtor-BF).(2) 𝑌 is bound and 𝑒 is a quoted SMT constructor ` 𝑐 SMTc ′ ( (cid:174) , 𝑋 𝑖 ) ` where all of (cid:174) 𝑋 𝑖 are unbound( 𝑃 -EqSMT-BF).(3) 𝑌 is possibly unbound and 𝑒 has no unbound variables ( 𝑃 -Eq-FB).It is critical that we avoid the case where both 𝑌 and some of the 𝑋 𝑖 are unbound, in which casewe would need to perform true unification (or even higher-order unification, depending on ourtreatment of functional programs). In the case where 𝑌 is bound and the expression has no unboundvariables, only the 𝑃 -Eq-FB case could apply. There is a fourth, irrelevant case: 𝑃 -NegEq. No bindingcan possibly occur there, so the constraint is imply checked by running 𝑒 and making sure it isn’tequal to 𝑌 .The binding rules come in three forms: 𝑋𝜏 -Bind for adding a new binding, 𝑋𝜏 -Check forensuring that an already bound variable is matched at appropriate type, and a vectorized form (cid:174) 𝑋 (cid:174) 𝜏 -All for folding over a sequence of such bindings. Note that the resulting bindings are the same,i.e., Γ ⊢ 𝑋, 𝜏 ▷ Γ , if and only if 𝑋 ∈ dom ( Γ ) ; the same holds for vectors of variables and types, aswell (Lemma E.5).We split the rules for expressions 𝑒 and formulas 𝜙 in two parts (Figures 18 and 19, respectively).Expression typing is conventional for functional languages. We adopt a declarative style for typesubstitutions ( 𝑒 -Ctor, 𝑒 -Fun, 𝑒 -Op, 𝑒 -Match). Our actual implementation uses Hindley–Damas–Milner type inference [Damas and Milner 1982; Hindley 1969] to find the correct types to use. Asto Formulog-specific features, we ensure type well formedness is in exp -mode; the ` 𝜙 ` expressionswitches from expression mode to formula mode. Relations 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ are treated as if they arefunctions of type (cid:174) 𝜏 𝑖 → bool . Since Datalog predicates can occur in functional terms, we must usethe control-flow graph of the program when analyzing for stratification. Consider the followingprogram: fun f(X : bv [32]) : bv [32] = if p(X) then ... else Xp(Y) :- q(Y , Y ).q(A , B) :- r(A), B = f(A ).r (42). Here the relation q calls the function f , which in turn relies on the negation of the relation p (sincethe behavior of f is conditioned on the contents of p ). As p is defined in terms of q , this leads to a ormulog: Datalog for SMT-Based Static Analysis 141:37 Function and expression well formedness Δ ; Φ ⊢ 𝐹 Δ ; Φ ; Γ ⊢ 𝑒 : 𝜏𝑓 : ∀ (cid:174) 𝛼 𝑗 , (cid:174) 𝜏 𝑖 → 𝜏 ∈ Φ (cid:174) 𝛼 𝑗 , −−−−→ 𝑋 𝑖 : 𝜏 𝑖 ⊢ 𝑒 : 𝜏 ⊢ fun 𝑓 ( (cid:174) 𝑋 𝑖 : (cid:174) 𝜏 𝑖 ) : 𝜏 = 𝑒 𝐹 -WF ⊢ Γ Γ ( 𝑋 ) = 𝜏 Γ ⊢ 𝑋 : 𝜏 𝑒 -Var Γ ⊢ 𝑘 : typeof ( 𝑘 ) 𝑒 -Const Γ ⊢ 𝑒 : 𝜏 Γ , 𝑋 : 𝜏 ⊢ 𝑒 : 𝜏 Γ ⊢ let 𝑋 = 𝑒 in 𝑒 : 𝜏 𝑒 -Let Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 : (cid:174) 𝜏 𝑖 , . . . } Γ ⊢ exp 𝜏 ′ 𝑗 Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] Γ ⊢ 𝑐 ( (cid:174) 𝑒 𝑖 ) : 𝐷 (cid:174) 𝜏 ′ 𝑗 𝑒 -Ctor Γ ⊢ 𝜙 : 𝜏 Γ ⊢ ` 𝜙 ` : 𝜏 𝑒 -Quote 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 Γ ⊢ 𝑝 ( (cid:174) 𝑒 𝑖 ) : bool 𝑒 -Rel 𝑓 : ∀ (cid:174) 𝛼 𝑗 , (cid:174) 𝜏 𝑖 → 𝜏 ∈ Φ Γ ⊢ exp 𝜏 ′ 𝑗 Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] Γ ⊢ 𝑓 ( (cid:174) 𝑒 𝑖 ) : 𝜏 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] 𝑒 -Fun typeof (⊗) = ∀ (cid:174) 𝛼 𝑗 , (cid:174) 𝜏 𝑖 → 𝜏 Γ ⊢ exp 𝜏 ′ 𝑗 Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] Γ ⊢ ⊗( (cid:174) 𝑒 𝑖 ) : 𝜏 𝑒 -Op Γ ⊢ 𝑒 : bool Γ ⊢ 𝑒 : 𝜏 Γ ⊢ 𝑒 : 𝜏 Γ ⊢ if 𝑒 then 𝑒 else 𝑒 : 𝜏 𝑒 -If Γ ⊢ 𝑒 : 𝐷 (cid:174) 𝜏 𝑗 Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 𝑖 : (cid:174) 𝜏 𝑘 , . . . } Γ , −−−−−−−−−−−−→ 𝑋 𝑘 : 𝜏 𝑘 [ 𝜏 𝑗 / 𝛼 𝑗 ] ⊢ 𝑒 𝑖 : 𝜏 Γ ⊢ match 𝑒 with −−−−−−−−−−→ 𝑐 𝑖 ( (cid:174) 𝑋 𝑘 ) → 𝑒 𝑖 : 𝜏 𝑒 -Match Fig. 18. Typing rules: expressions; implicit parameters are in gray circularity we want to avoid: It is possible to derive q(42, 42) , but this derivation nonsensicallyrelies on q(42, 42) being false (since it requires that p(42) is false).While Formulog’s expressions compute values, the Formulog’s formulas construct
ASTs, to beshipped off to an SMT solver. Our formal account here uniformly uses SMT constructors to modelthe SMT syntax, but our implementation offers special-purpose syntax. For example, we write 𝑐 SMTvar [ 𝑥, bv [ ]] in our formalism to name a 32-bit integer variable 𝑥 , while in our implementationone might write .Every 𝜙 - . . . typing rule generates a value with an SMT type, i.e., either 𝑡 sym or 𝑡 smt for SMTtypes 𝑡 , i.e., Γ ⊢ smt 𝑡 (Lemma D.9). The 𝑐 -SMT-* rules yield 𝑡 sym and 𝑡 smt . SMT variables 𝑐 SMTvar [ 𝑥, 𝑡 ] are written in lowercase to emphasize their distinction from expression variables 𝑋 ; these SMTvariables will be used as names in the formulas sent to the SMT solver. We keep track of whichterms are SMT variables 𝑐 SMTvar [ 𝑥, 𝑡 ] of type 𝑡 sym (generated by 𝑐 -SMT-Var) and which are plainSMT formulas of type 𝑡 smt (all other rules). We treat 𝑡 sym as a subtype of 𝑡 smt ( 𝜙 -Promote).The ` 𝜙 ` operator is an expression term that introduces quoted SMT formulas represented asspecial, SMT constructors of the form 𝑐 SMT ... (described below); the , 𝑒 operator is the corresponding‘unquote’ operator that introduces an expression ( 𝜙 -Unqote).While unquoting generally suffices for embedding the results of expressions in formulas, wetreat constructors specially so that we can mix concrete and symbolic (i.e., SMT) arguments in asingle data type constructor ( 𝑐 -SMT-Ctor): we assign them types that are fully SMT-ized via the SMT constructors and formula well formedness Δ ; Φ ; Γ ⊢ 𝑐 SMT ... : (cid:174) 𝜏 𝑖 → 𝜏 Δ ; Φ ; Γ ⊢ 𝜙 : 𝜏 Γ ⊢ smt 𝑡 Γ ⊢ 𝑐 SMTvar [ 𝑥, 𝑡 ] : · → 𝑡 sym 𝑐 -SMT-Var Γ ⊢ 𝑐 SMTconst [ 𝑘 ] : · → typeof ( 𝑘 ) smt 𝑐 -SMT-Const Γ ⊢ smt 𝑡 Γ ⊢ smt 𝑡 Γ ⊢ 𝑐 SMTlet : 𝑡 sym × 𝑡 smt × 𝑡 smt → 𝑡 smt 𝑐 -SMT-Let Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 : (cid:174) 𝜏 𝑖 , . . . } Γ ⊢ smt 𝜏 𝑖 [ 𝑡 ′ 𝑗 / 𝛼 𝑗 ] Γ ⊢ smt 𝑡 ′ 𝑗 Γ ⊢ 𝑐 SMTctor [ 𝑐 ] : −−−−−−−−−−−−−−−−→ toSMT ( 𝜏 𝑖 [ 𝑡 ′ 𝑗 / 𝛼 𝑗 ]) → ( 𝐷 −→ 𝑡 ′ 𝑗 ) smt 𝑐 -SMT-Ctor Γ ⊢ smt 𝑡 Γ ⊢ 𝑐 SMTforall : 𝑡 sym × bool smt → bool smt 𝑐 -SMT-Forall uf : (cid:174) 𝑡 𝑖 → 𝑡 ∈ ΦΓ ⊢ 𝑐 SMTuf [ uf ] : −−−−→ 𝑡 𝑖 smt → 𝑡 smt 𝑐 -SMT-UFun Γ ⊢ 𝜙 : 𝑡 sym Γ ⊢ 𝜙 : 𝑡 smt 𝜙 -Promote Γ ⊢ 𝑒 : 𝜏 Γ ⊢ smt 𝜏 Γ ⊢ , 𝑒 : toSMT ( 𝜏 ) 𝜙 -Unqote Γ ⊢ 𝑐 SMTc : (cid:174) 𝜏 𝑖 → 𝜏 Γ ⊢ 𝜙 𝑖 : 𝜏 ′ 𝑖 Γ ⊢ 𝑐 SMTc ( (cid:174) 𝜙 𝑖 ) : 𝜏 𝜙 -Ctor Conversion to SMT types erase ( 𝜏 ) = 𝑡 toSMT ( 𝜏 ) = 𝜏 erase ( 𝐵 ) = 𝐵 erase ( 𝐷 (cid:174) 𝜏 𝑖 ) = 𝐷 −−−−−−−→ erase ( 𝜏 𝑖 ) erase ( 𝑡 smt ) = erase ( 𝑡 ) erase ( 𝑡 sym ) = erase ( 𝑡 ) toSMT ( 𝑡 ) = erase ( 𝑡 ) smt toSMT ( 𝑡 smt ) = erase ( 𝑡 ) smt toSMT ( 𝑡 sym ) = erase ( 𝑡 ) sym Fig. 19. Typing rules: SMT constructors and formulas; conversion to SMT types toSMT function, but unquoting allows for easy mixing of values of SMT-types 𝑡 as if they wereof type 𝑡 smt . The toSMT metafunction alters the type of 𝑒 to make sure it is SMT representable; toSMT relies on an erase function to avoid nesting . . . smt and . . . sym type constructors. Onecan only run these functions on SMT types. For example, we can write terms like the following inconcrete syntax: let 𝐻 = in ` cons ( 𝐻, 𝑙 [ bv [ ] list ]) ` which desugars to the SMT constructors: let 𝐻 = in ` 𝑐 SMTctor [ cons ] ( , 𝐻, 𝑐
SMTvar [ 𝑙, bv [ ] list ] ()) ` Note that 𝐻 is an expression variable and 𝑙 is an SMT variable; the type conversion in 𝜙 -Unqotelets us mix them in the same list of 32-bit numbers. The 𝑡 sym type is used in 𝜙 -Let and 𝜙 -Forall,which construct SMT formuale that use binders. The only way to get a value of type 𝑡 sym is eitherwith 𝑐 -SMT-Var/ 𝜙 -Ctor or with 𝜙 -Unqote, as in let 𝑋 = ` 𝑐 SMTvar [ 𝑥, bv [ ]] ` in ` , 𝑋 ` . ormulog: Datalog for SMT-Based Static Analysis 141:39 Uninterpreted functions must be applied to appropriate SMT types ( 𝜙 -UFun); recall that Φ -UFunensures that each uninterpreted function’s types are SMT representable.Finally, there are a suite of SMT constructors of the form 𝑐 SMT ... . Each of these special 𝑐 SMT ... constructors is treated as an ordinary constructor by the operational semantics, even though theconstructors don’t appear in Δ . Rather than making SMT terms opaque, we model them withconstructors to allow for matching on generated formulae in 𝑃 -EqSMT-BF and 𝑃 -Eq-FB. The typesof the SMT constructors 𝑐 SMT ... are given in Figure 19. It is a crucial invariant that all of these typesbe SMT types: we would not want to treat an SMT variable 𝑐 SMTvar [ 𝑥, bool ] as though it were an actual bool ! Reusing the 𝑐 -SMT- . . . rules is convenient—we need only state the types of theseconstructors once and we get precise types in our premises. Several SMT constructors take specialarguments in square brackets: 𝑐 SMTvar [ 𝑥, 𝑡 ] is a 0-ary SMT constructor, while 𝑐 SMTvar itself is a familyof SMT constructors for given variable names 𝑥 and pre-types 𝑡 ; similarly, 𝑐 SMTconst [ 𝑘 ] is a 0-ary SMTconstructor, while 𝑐 SMTconst itself is a family of SMT constructors for given constants 𝑘 . The embeddingof data type constructors 𝑐 SMTctor [ 𝑐 ] is similarly parameterized on a constructor name 𝑐 , and theembedding of uninterpreted functions 𝑐 SMTuf [ uf ] takes an uninterpereted function as a parameter.Separating these parameters from the interesting, 𝜙 -shaped subparts of each SMT constructor letsus reuse the 𝑐 -SMT- . . . rules when typing premises that might bind to subparts of an SMT formula( 𝑃 -EqSMT- . . . , Figure 17).Our formal model elides some of the detail of our SMT encoding, such as constructors for SMToperations like bit vector manipulation or equality. These operations are all encoded as more SMT-specific constructors, i.e., 𝑐 SMTctor [ bv _ add ] ( 𝑐 SMTvar [ 𝑥, bv [ ]] , 𝑐 SMTconst [ ])) represents result of addingthe 32-bit vector 𝑥 and . There are some subtle issues around polymorphism and determinacy. Wegive a monomorphic interpretation of SMT here, though our SMT constructors can work with ourpolymorphic data types. Our implementation treats equality and other polymorphic SMT operationsspecially, where each use of a polymorphic operator must be fully instantiated. In practice ourimplementation can usually infer the instantiation; users must annotate in those places we cannotinfer. C OPERATIONAL SEMANTICS
Formulog’s operational semantics operates over worlds W and substitutions 𝜃 (Figure 20); thesemantics is a mix of small-step rules modeling a single application of a Datalog rule (Figure 21),which depend on a small-step rules explaining how premises unify (Figures 22 and 23); the premisesemantics in turn depends on a semantics of expressions (Figures 24 and 25) and formulas (Figure 26).Our worlds W are (subsets of) Herbrand models. Our small-step semantics iteratively builds upa world that is in fact a Herbrand model of the original relations in the program. We could havemodeled our semi-naive evaluation model for Formulog in more detail, showing that all programsgenerate a world W that is a well typed Herbrand model of the user’s program (possibly takinginfinite time to do so). Doing so wouldn’t add anything materially interesting to our formulation.Throughout, the type system’s goal is prevent a program yielding ⊥ , the bottom “wrong” value.Such a value denotes a serious, unrecoverable error, such as using a relation with the wrong arity orconditioning on a non-boolean. It is important to distinguish bad, ⊥ -yielding programs from thosethat simply fail to step. The goal of Datalog evaluation is to reach a fixed point, i.e., to be unable tostep! Finally, as is common, we assume that built-in operations do not yield ⊥ , i.e., they are total.While we could in principle design a type system for Formulog that avoids, say, division by zero,we are more interested in making the hard parts easy (generating well typed SMT formulas) ratherthan making the easy parts foolproof (statically protecting partial functions). Namespaces
World
W ∈
PredVar → P (
Val × · · · ×
Val ) World or error W ⊥ ∈ World + Error
Substitution 𝜃 ∈ Var ⇀ Val
Substitution or error 𝜃 ⊥ ∈ Substitution + Error
Values
Results 𝑣 ⊥ :: = 𝑣 | ⊥ Values 𝑣 ∈ Val :: = 𝑘 | 𝑐 ( (cid:174) 𝑣 𝑖 ) Unifiable term 𝑢 ∈ UTerm :: = 𝑋 | 𝑘 | 𝑐 ( (cid:174) 𝑢 𝑖 ) Substitution and world well formedness Δ ; Φ ; Γ | = 𝜃 Δ ; Φ | = W Γ | = 𝜃 ⇔∀ 𝑋 ∈ dom ( Γ ) (cid:26) (1) 𝑋 ∈ dom ( 𝜃 ) (2) · ⊢ 𝜃 ( 𝑋 ) : Γ ( 𝑋 ) Δ ; Φ | = W⇔∀ 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ (1) 𝑝 ∈ dom (W) (2) (cid:174) 𝑣 𝑗 ∈ W ( 𝑝 ) ⇒ 𝑖 = 𝑗 (3) ∀ (cid:174) 𝑣 𝑖 ∈ W ( 𝑝 ) , Δ ; Φ ; · ⊢ 𝑣 𝑖 : 𝜏 𝑖 Fig. 20. Definitions for semantics
Clause semantics (cid:174) 𝐹 ; W ⊢ 𝐻 → W ⊥ · ⊢ 𝑃 → 𝜃 . . . 𝜃 𝑖 ⊢ 𝑃 𝑖 → 𝜃 𝑖 + . . . 𝜃 𝑛 ⊢ 𝑃 𝑛 → 𝜃 (cid:174) 𝐹 ; W ⊢ 𝑝 ( (cid:174) 𝑋 𝑗 ) : − (cid:174) 𝑃 𝑖 → W [ 𝑝 ↦→ W ( 𝑝 ) ∪ { 𝜃 ( (cid:174) 𝑋 𝑗 )}] Clause · ⊢ 𝑃 → 𝜃 . . . 𝜃 𝑖 ⊢ 𝑃 𝑗 → ⊥(cid:174) 𝐹 ; W ⊢ 𝑝 ( (cid:174) 𝑋 𝑗 ) : − (cid:174) 𝑃 𝑖 → ⊥ Clause-E1 · ⊢ 𝑃 → 𝜃 . . . 𝜃 𝑖 ⊢ 𝑃 𝑖 → 𝜃 𝑖 + . . . 𝜃 𝑛 ⊢ 𝑃 𝑛 → 𝜃 (cid:174) 𝑋 𝑗 ⊈ dom ( 𝜃 )(cid:174) 𝐹 ; W ⊢ 𝑝 ( (cid:174) 𝑋 𝑗 ) : − (cid:174) 𝑃 𝑖 → ⊥ Clause-E2
Fig. 21. Clause semantics
Rules of the form . . . -E 𝑛 denote ⊥ -yielding rules. Each such rule characterizes a form of wrong-ness avoided by our static type system. We write 𝑣 ⊥ to denote the disjoint sum of values 𝑣 and thewrong value ⊥ .During correct execution, Clause takes a Horn clause 𝑝 ( (cid:174) 𝑋 𝑗 ) : − (cid:174) 𝑃 𝑖 , executes each premise 𝑃 𝑖 from left to right, yielding a final substitution for the variables (cid:174) 𝑋 𝑗 in the head of the rule. There aretwo possible failing rules. Clause-E1 simply propagates the first error from a premise; Clause-E2fails because not every 𝑋 𝑗 in the head of the rule is bound by the end. Since Formulog enforcesthe range restriction ( 𝐻 -Clause), Clause-E2 can never apply in a well typed program. Finally, theClause* operational rules and the 𝐻 -Clause typing rule both use the fixed, given order of premises ormulog: Datalog for SMT-Based Static Analysis 141:41 Premise semantics (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝑃 → 𝜃 ⊥ (cid:174) 𝑣 ∈ W ( 𝑝 ) 𝜃 ⊢ (cid:174) 𝑋 ∼ (cid:174) 𝑣 : 𝜃 ′⊥ W ; 𝜃 ⊢ 𝑝 ( (cid:174) 𝑋 ) → 𝜃 ′⊥ PosAtom 𝜃 ( (cid:174) 𝑋 ) = (cid:174) 𝑣 (cid:174) 𝑣 ∉ W ( 𝑝 )W ; 𝜃 ⊢ ! 𝑝 ( (cid:174) 𝑋 ) → 𝜃 NegAtom 𝜃 ⊢ 𝑌 ∼ 𝑐 ( (cid:174) 𝑋 ) : 𝜃 ′⊥ W ; 𝜃 ⊢ 𝑌 = 𝑐 ( (cid:174) 𝑋 ) → 𝜃 ′⊥ EqCtor 𝜃 ⊢ 𝑌 ∼ 𝑐 SMTc ′ ( (cid:174) 𝑋 ) : 𝜃 ′⊥ W ; 𝜃 ⊢ 𝑌 = ` 𝑐 SMTc ′ ( (cid:174) , 𝑋 ) ` → 𝜃 ′⊥ EqSMT 𝑒 is not a constructor W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 𝜃 ⊢ 𝑌 ∼ 𝑣 : 𝜃 ′⊥ W ; 𝜃 ⊢ 𝑌 = 𝑒 → 𝜃 ′⊥ EqExpr (cid:174) 𝑋 ⊈ dom ( 𝜃 )W ; 𝜃 ⊢ ! 𝑝 ( (cid:174) 𝑋 ) → ⊥ NegAtom-E 𝑒 is not a constructor W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 ⊥W ; 𝜃 ⊢ 𝑌 = 𝑒 → ⊥ EqExpr-E W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 𝜃 ( 𝑌 ) ≠ 𝑣 W ; 𝜃 ⊢ ! ( 𝑌 = 𝑒 ) → 𝜃 NegExpr W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 ⊥W ; 𝜃 ⊢ ! ( 𝑌 = 𝑒 ) → ⊥ NegExpr-E1 𝑌 ∉ dom ( 𝜃 )W ; 𝜃 ⊢ ! ( 𝑌 = 𝑒 ) → ⊥ NegExpr-E2
Fig. 22. Premise semantics for checking. Different orderings induce different binding orders, some of which may succeed andsome of which may not.The premise semantics (Figure 22) uses unification (Figure 23) to match and bind variables. Posi-tive atoms 𝑝 ( (cid:174) 𝑋 𝑖 ) try to unify their arguments with a tuple for 𝑝 drawn from the world W (PosAtom).Negative atoms 𝑝 ( (cid:174) 𝑋 𝑖 ) require that all of their arguments 𝑋 𝑖 are already bound (NegAtom); failingto find such bound terms yields an error (NegAtom-E). Rules for equations also use unification,whether for a constructor over variables (EqCtor) or an expression (EqExpr). The latter can fail ifevaluation fails (EqExpr-E). Before discussing term evaluation, we give rules for unification.Unification is split into two levels. Unification proper takes a pair of unifiable terms—valueswith variables in them—and tries to yield a substitution.
Value unification takes a unifiable termand a value and tries to yield a substitution. The unification rules are of the form 𝑢𝑢 - . . . . Theserules analyze the two unifiable terms to find which side is completely bound—i.e., applying 𝜃 cancompletely fill in the variables—and so can be passed to value unification as a value. The B and Fin these rules stand for Bound and Free. The only error in unification is in 𝑢𝑢 -FF, when neitherunifiable term is bound to a value. We write a case for when both unifiable terms are bound ( 𝑢𝑢 -BB)and require that they are directly equal—but it would also work to drop this rule and rely on valueunification to identify the equality.Value unification rules are of the form 𝑢𝑣 - . . . . The rules here lookup variables in the unifiableterm and either check that the binding conforms to the given value ( 𝑢𝑣 -Eq-Var, cf. 𝑋𝜏 -Check)or binds the value ( 𝑢𝑣 -Bind-Var, cf. 𝑋𝜏 -Bind). The remaining value unification rules match thestructure of the unifiable term to the structure of the value ( 𝑢𝑣 -Constant, 𝑢𝑣 -Ctor) or fold valueunification along a vector ( (cid:174) 𝑢 (cid:174) 𝑣 -All). Value unification never produces ⊥ . It isn’t an error when twovalues fail to unify, since one might have to search through many tuples for a relation in W to finda one that matches, say, a given constructor. Value unification 𝜃 ⊢ 𝑢 ∼ 𝑣 ▷ 𝜃 𝜃 ⊢ (cid:174) 𝑢 ∼ (cid:174) 𝑣 ▷ 𝜃𝜃 ( 𝑋 ) = 𝑣𝜃 ⊢ 𝑋 ∼ 𝑣 ▷ 𝜃 𝑢𝑣 -Eq-Var 𝑋 ∉ dom ( 𝜃 ) 𝜃 ⊢ 𝑋 ∼ 𝑣 ▷ 𝜃 [ 𝑋 ↦→ 𝑣 ] 𝑢𝑣 -Bind-Var 𝜃 ⊢ 𝑘 ∼ 𝑘 ▷ 𝜃 𝑢𝑣 -Constant 𝜃 ⊢ (cid:174) 𝑢 𝑖 ∼ (cid:174) 𝑣 𝑖 ▷ 𝜃 ′ 𝜃 ⊢ 𝑐 ( (cid:174) 𝑢 𝑖 ) ∼ 𝑐 ( (cid:174) 𝑣 𝑖 ) ▷ 𝜃 ′ 𝑢𝑣 -Ctor 𝜃 ⊢ 𝑢 ∼ 𝑣 ▷ 𝜃 . . . 𝜃 ⊢ 𝑢 𝑖 ∼ 𝑣 𝑖 ▷ 𝜃 𝑖 . . . 𝜃 𝑛 ⊢ 𝑢 𝑛 ∼ 𝑣 𝑛 ▷ 𝜃 ′ 𝜃 ⊢ (cid:174) 𝑢 𝑖 ∼ (cid:174) 𝑣 𝑖 ▷ 𝜃 (cid:174) 𝑢 (cid:174) 𝑣 -All Unification 𝜃 ⊢ 𝑢 ∼ 𝑢 : 𝜃 ⊥ 𝜃 ⊢ (cid:174) 𝑢 ∼ (cid:174) 𝑢 : 𝜃 ⊥ 𝜃 ( 𝑢 ) = 𝑣 𝜃 ( 𝑢 ) = 𝑣 𝑣 = 𝑣 𝜃 ⊢ 𝑢 ∼ 𝑢 : 𝜃 𝑢𝑢 -BB (cid:154) 𝑣 , 𝜃 ( 𝑢 ) = 𝑣 𝜃 ( 𝑢 ) = 𝑣 𝜃 ⊢ 𝑢 ∼ 𝑣 ▷ 𝜃 ′ 𝜃 ⊢ 𝑢 ∼ 𝑢 : 𝜃 ′ 𝑢𝑢 -FB 𝜃 ( 𝑢 ) = 𝑣 (cid:154) 𝑣 , 𝜃 ( 𝑢 ) = 𝑣 𝜃 ⊢ 𝑢 ∼ 𝑣 ▷ 𝜃 ′ 𝜃 ⊢ 𝑢 ∼ 𝑢 : 𝜃 ′ 𝑢𝑢 -BF (cid:154) 𝑣 , 𝜃 ( 𝑢 ) = 𝑣 (cid:154) 𝑣 , 𝜃 ( 𝑢 ) = 𝑣 𝜃 ⊢ 𝑢 ∼ 𝑢 : ⊥ 𝑢𝑢 -FF 𝜃 ⊢ 𝑢 𝑖 ∼ 𝑢 ′ 𝑖 ▷ 𝜃 𝑖 𝜃 ⊢ (cid:174) 𝑢 𝑖 ∼ (cid:174) 𝑢 ′ 𝑖 ▷ 𝜃 (cid:174) 𝜃 𝑖 (cid:174) 𝑢 (cid:174) 𝑢 -All . . . 𝜃 ⊢ 𝑢 ∼ 𝑢 ′ ▷ ⊥ . . .𝜃 ⊢ (cid:174) 𝑢 𝑖 ∼ (cid:174) 𝑢 ′ 𝑖 ▷ ⊥ (cid:174) 𝑢 (cid:174) 𝑢 -All-E 𝜃 ( 𝑐 ( (cid:174) 𝑢 𝑖 )) = 𝑐 (−−−→ 𝜃 ( 𝑢 𝑖 )) Fig. 23. Unification
The expression semantics is an entirely conventional big-step semantics using explicit substi-tutions. The operational rules implicitly take the function definitions (cid:174) 𝐹 for use in applications( ⇓ 𝑒 -Fun).There are a variety of wrong behaviors prevented by our type system, mostly concerningmismatches between values and elimination forms: unbound variables ( ⇓ 𝑒 -Var-E); mistyped argu-ments to built-in operations ( ⇓ 𝑒 -Op-E2); function, relation and constructor arity errors ( ⇓ 𝑒 -Fun-E2, ⇓ 𝑒 -Rel-E2, ⇓ 𝑒 -Match-E4); non-existent functions and relations ( ⇓ 𝑒 -Fun-E3, ⇓ 𝑒 -Rel-E3); condi-tionals on inappropriate values ( ⇓ 𝑒 -Match-E2, ⇓ 𝑒 -Ite-E2); and ill formed constructor names( ⇓ 𝑒 -Match-E3); The remaining rules propagate errors ( ⇓ 𝑒 -Let-E, ⇓ 𝑒 -Op-E1, ⇓ 𝑒 -Fun-E1, ⇓ 𝑒 -Rel-E1, ⇓ 𝑒 -Match-E1, ⇓ 𝑒 -Ite-E1). As mentioned in the early discussion of our semantics in this section, ⇓ 𝑒 -Op-E2 is not about division by zero (a form of going wrong our type system doesn’t prevent),but about mis-application of built-in functions, e.g., taking the boolean negation of a number.The operational semantics on formulas is simple: the rules generate ASTs for the SMT solverusing the 𝑐 SMT ... constructors: constants ( 𝑐 SMTconst ), SMT variables ( 𝑐 SMTvar ), SMT data types ( 𝑐 SMTctor ), letbindings ( 𝑐 SMTlet ), quantification ( 𝑐 SMTforall ), and uninterpreted function application ( 𝑐 SMTuf ).When unquoting values resulting from evaluating expressions, we use the toSMT function totranslate expression values into the SMT’s AST. The toSMT function is an identity on SMT ASTs,but it explicitly tags the constants and Formulog-defined constructors using 𝑐 SMTconst and 𝑐 SMTctor . ormulog: Datalog for SMT-Based Static Analysis 141:43 Expression semantics (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ (cid:174) 𝐹 ; W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 ⊥ W ; 𝜃 ⊢ · ⇓ (cid:174) 𝑒 · ⇓ (cid:174) 𝑒 -Empty W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 W ; 𝜃 ⊢ 𝑒, (cid:174) 𝑒 ⇓ (cid:174) 𝑒 𝑣, (cid:174) 𝑣 ⇓ (cid:174) 𝑒 -All W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 𝑖 W ; 𝜃 ⊢ 𝑐 ( (cid:174) 𝑒 𝑖 ) ⇓ (cid:174) 𝑒 𝑐 ( (cid:174) 𝑣 𝑖 ) ⇓ 𝑒 -Ctor W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 ⊥W ; 𝜃 ⊢ 𝑐 ( (cid:174) 𝑒 𝑖 ) ⇓ 𝑒 ⊥ ⇓ 𝑒 -Ctor-E W ; 𝜃 ⊢ 𝑘 ⇓ 𝑒 𝑘 ⇓ 𝑒 -Const 𝜃 ( 𝑋 ) = 𝑣 W ; 𝜃 ⊢ 𝑋 ⇓ 𝑒 𝑣 ⇓ 𝑒 -Var W ; 𝜃 ⊢ 𝜙 ⇓ 𝜙 𝑣 ⊥ W ; 𝜃 ⊢ ` 𝜙 ` ⇓ 𝑒 𝑣 ⊥ ⇓ 𝑒 -Quote W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 [[⊗]] ((cid:174) 𝑣 ) = 𝑣 W ; 𝜃 ⊢ ⊗((cid:174) 𝑒 ) ⇓ 𝑒 𝑣 ⇓ 𝑒 -Op fun 𝑓 ( (cid:174) 𝑋 𝑖 : (cid:174) 𝜏 𝑖 ) : 𝜏 = 𝑒 ∈ (cid:174) 𝐹 W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 𝑖 W ; 𝜃 [ (cid:174) 𝑋 𝑖 ↦→ (cid:174) 𝑣 𝑖 ] ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ W ; 𝜃 ⊢ 𝑓 ( (cid:174) 𝑒 𝑖 ) ⇓ 𝑒 𝑣 ⊥ ⇓ 𝑒 -Fun W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 𝑖 (cid:174) 𝑣 𝑖 ∈ W ( 𝑝 )W ; 𝜃 ⊢ 𝑝 ( (cid:174) 𝑒 𝑖 ) ⇓ 𝑒 true ⇓ 𝑒 -Rel-T W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 𝑖 (cid:174) 𝑣 𝑖 ∉ W ( 𝑝 )W ; 𝜃 ⊢ 𝑝 ( (cid:174) 𝑒 𝑖 ) ⇓ 𝑒 false ⇓ 𝑒 -Rel-F W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 W ; 𝜃 [ 𝑋 ↦→ 𝑣 ] ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ W ; 𝜃 ⊢ let 𝑋 = 𝑒 in 𝑒 ⇓ 𝑒 𝑣 ⊥ ⇓ 𝑒 -Let W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑐 ( (cid:174) 𝑣 𝑖 ) W ; 𝜃 [−−−−−−→ 𝑋 𝑖 ↦→ 𝑣 𝑖 ] ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ W ; 𝜃 ⊢ match 𝑒 with . . . 𝑐 ( (cid:174) 𝑋 𝑖 ) → 𝑒 . . . ⇓ 𝑒 𝑣 ⊥ ⇓ 𝑒 -Match W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 true W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ W ; 𝜃 ⊢ if 𝑒 then 𝑒 else 𝑒 ⇓ 𝑒 𝑣 ⊥ ⇓ 𝑒 -IteT W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 false W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ W ; 𝜃 ⊢ if 𝑒 then 𝑒 else 𝑒 ⇓ 𝑒 𝑣 ⊥ ⇓ 𝑒 -IteF Fig. 24. Expression semantics
D METATHEORY
We break the metatheory into two parts: lemmas characterizing the SMT conversion (Section D.1)and lemmas showing type safety (Section E). The SMT lemmas culminate in two proofs: first,regularity (Lemma D.9) guarantees that (a) every type or context generated by the operationalsemantics is well formed and (b) that formula evaluation generates well typed SMT ASTs; second,we show that SMT conversion of values agrees with SMT conversion of types (Lemma D.10).Type safety culminates in theorems showing that premises don’t yield ⊥ and generate well typedsubstitutions (Lemma E.4) and so Horn clauses (a) never yield ⊥ (Theorem E.6) and (b) take welltyped worlds to well typed worlds (Theorem E.7). D.1 SMT Conversion
We show a variety of properties of the erasure and SMT conversion functions: smt is a sub-kind of exp (Lemma D.1); erasures and SMT conversion yield well formed types from SMT types
Expression semantics (continued) (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ (cid:174) 𝐹 ; W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 ⊥ W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 𝑖 W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 ⊥W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 , 𝑒, (cid:174) 𝑒 𝑗 ⇓ (cid:174) 𝑒 ⊥ ⇓ (cid:174) 𝑒 -All-E 𝑋 ∉ dom ( 𝜃 )W ; 𝜃 ⊢ 𝑋 ⇓ 𝑒 ⊥ ⇓ 𝑒 -Var-E W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 ⊥W ; 𝜃 ⊢ let 𝑋 = 𝑒 in 𝑒 ⇓ 𝑒 ⊥ ⇓ 𝑒 -Let-E W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 ⊥W ; 𝜃 ⊢ ⊗((cid:174) 𝑒 ) ⇓ 𝑒 ⊥ ⇓ 𝑒 -Op-E1 W ; 𝜃 ⊢ (cid:174) 𝑒 ⇓ (cid:174) 𝑒 (cid:174) 𝑣 (cid:174) 𝑣 ∉ dom ([[⊗]])W ; 𝜃 ⊢ ⊗((cid:174) 𝑒 ) ⇓ 𝑒 ⊥ ⇓ 𝑒 -Op-E2 W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 ⊥W ; 𝜃 ⊢ 𝑓 ( (cid:174) 𝑒 𝑖 ) ⇓ 𝑒 ⊥⇓ 𝑒 -Fun-E1 fun 𝑓 ( (cid:174) 𝑋 𝑖 : (cid:174) 𝜏 𝑖 ) : 𝜏 = 𝑒 ∈ (cid:174) 𝐹 𝑖 ≠ 𝑗 W ; 𝜃 ⊢ 𝑓 ( (cid:174) 𝑒 𝑗 ) ⇓ 𝑒 ⊥⇓ 𝑒 -Fun-E2 𝑓 ∉ (cid:174) 𝐹 W ; 𝜃 ⊢ 𝑓 ( (cid:174) 𝑒 𝑗 ) ⇓ 𝑒 ⊥⇓ 𝑒 -Fun-E3 W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ (cid:174) 𝑒 ⊥W ; 𝜃 ⊢ 𝑝 ( (cid:174) 𝑒 𝑖 ) ⇓ 𝑒 ⊥⇓ 𝑒 -Rel-E1 W ( 𝑝 ) ⊆ P (−−−→ Val 𝑖 ) 𝑖 ≠ 𝑗 W ; 𝜃 ⊢ 𝑝 ( (cid:174) 𝑒 𝑗 ) ⇓ 𝑒 ⊥⇓ 𝑒 -Rel-E2 𝑝 ∉ dom (W)W ; 𝜃 ⊢ 𝑝 ( (cid:174) 𝑒 𝑗 ) ⇓ 𝑒 ⊥⇓ 𝑒 -Rel-E3 W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 ⊥W ; 𝜃 ⊢ match 𝑒 with −−−−−−−−−−→ 𝑐 𝑖 ( (cid:174) 𝑋 𝑗 ) → 𝑒 𝑖 ⇓ 𝑒 ⊥⇓ 𝑒 -Match-E1 W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 𝑣 ≠ 𝑐 ( (cid:174) 𝑣 ′ )W ; 𝜃 ⊢ match 𝑒 with −−−−−−−−−−→ 𝑐 𝑖 ( (cid:174) 𝑋 𝑗 ) → 𝑒 𝑖 ⇓ 𝑒 ⊥⇓ 𝑒 -Match-E2 W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑐 ( (cid:174) 𝑣 𝑘 ) 𝑐 ∉ { (cid:174) 𝑐 𝑖 }W ; 𝜃 ⊢ match 𝑒 with −−−−−−−−−−→ 𝑐 𝑖 ( (cid:174) 𝑋 𝑗 ) → 𝑒 𝑖 ⇓ 𝑒 ⊥⇓ 𝑒 -Match-E3 W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑐 ( (cid:174) 𝑣 𝑘 ) 𝑗 ≠ 𝑘 W ; 𝜃 ⊢ match 𝑒 with −−−−−−−−−−−−−−−−→ . . . 𝑐 ( (cid:174) 𝑋 𝑗 . . . ) → 𝑒 𝑖 ⇓ 𝑒 ⊥⇓ 𝑒 -Match-E4 W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 ⊥W ; 𝜃 ⊢ if 𝑒 then 𝑒 else 𝑒 ⇓ 𝑒 ⊥ ⇓ 𝑒 -Ite-E1 W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 𝑣 ∉ { true , false }W ; 𝜃 ⊢ if 𝑒 then 𝑒 else 𝑒 ⇓ 𝑒 ⊥ ⇓ 𝑒 -Ite-E2 Fig. 25. Expression semantics (error rules) (Lemmas D.2, D.3, D.4, and D.5); weakening and strengthening of typing contexts (Lemmas D.6and D.7); type variable substitution (Lemma D.8)—we have no need of a value substitution lemmabecause our semantics uses environments; regularity (Lemma D.9); and, finally, that SMT conversionof values agrees with SMT conversion of types (Lemma D.10).Lemma D.1 ( smt is a subkind of exp ). If Γ ⊢ smt 𝜏 then Γ ⊢ exp 𝜏 . Proof. By induction on 𝜏 .( 𝜏 = 𝐵 ) Immediate: 𝜏 -Base allows any 𝑚 .( 𝜏 = 𝛼 ) Contradictory—type variables are only well formed at exp .( 𝜏 = 𝐷 (cid:174) 𝜏 𝑖 ) By the IH on each 𝜏 𝑖 and 𝑡 -ADT. ormulog: Datalog for SMT-Based Static Analysis 141:45 Formula semantics (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝜙 ⇓ 𝜙 𝑣 ⊥ (cid:174) 𝐹 ; W ; 𝜃 ⊢ (cid:174) 𝜙 ⇓ (cid:174) 𝜙 (cid:174) 𝑣 ⊥ W ; 𝜃 ⊢ · ⇓ (cid:174) 𝜙 · ⇓ (cid:174) 𝜙 -Empty W ; 𝜃 ⊢ 𝜙 ⇓ 𝜙 𝑣 W ; 𝜃 ⊢ (cid:174) 𝜙 𝑖 ⇓ (cid:174) 𝜙 (cid:174) 𝑣 𝑖 W ; 𝜃 ⊢ 𝜙, (cid:174) 𝜙 𝑖 ⇓ (cid:174) 𝜙 𝑣, (cid:174) 𝑣 𝑖 ⇓ (cid:174) 𝜙 -All W ; 𝜃 ⊢ (cid:174) 𝜙 𝑖 ⇓ (cid:174) 𝜙 (cid:174) 𝑣 𝑖 W ; 𝜃 ⊢ 𝜙 ⇓ 𝜙 ⊥W ; 𝜃 ⊢ (cid:174) 𝜙 𝑖 , 𝜙, (cid:174) 𝜙 𝑗 ⇓ (cid:174) 𝜙 ⊥ ⇓ (cid:174) 𝜙 -All-E W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 W ; 𝜃 ⊢ , 𝑒 ⇓ 𝜙 toSMT ( 𝑣 ) ⇓ 𝜙 -Unqote W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 ⊥W ; 𝜃 ⊢ , 𝑒 ⇓ 𝜙 ⊥ ⇓ 𝜙 -Unqote-E W ; 𝜃 ⊢ (cid:174) 𝜙 𝑖 ⇓ (cid:174) 𝜙 (cid:174) 𝑣 𝑖 W ; 𝜃 ⊢ 𝑐 SMTc ( (cid:174) 𝜙 ) ⇓ 𝜙 𝑐 SMTc ((cid:174) 𝑣 ) ⇓ 𝜙 -Ctor W ; 𝜃 ⊢ (cid:174) 𝜙 𝑖 ⇓ (cid:174) 𝜙 ⊥W ; 𝜃 ⊢ 𝑐 SMTc ( (cid:174) 𝜙 𝑖 ) ⇓ 𝜙 ⊥ ⇓ 𝜙 -Ctor-E W ; 𝜃 ⊢ 𝑐 SMTc ( (cid:174) 𝑣 𝑖 ) ⇓ 𝜙 𝑐 SMTc ( (cid:174) 𝑣 𝑖 ) ⇓ 𝜙 -SMT-Value SMT conversion toSMT ( 𝑣 ) = 𝑣 toSMT ( 𝑘 ) = 𝑐 SMTconst [ 𝑘 ] () toSMT ( 𝑐 ( (cid:174) 𝑣 𝑖 )) = 𝑐 SMTctor [ 𝑐 ] (−−−−−−−−−−→ toSMT ( 𝑣 𝑖 )) toSMT ( 𝑐 SMTconst [ 𝑘 ] ()) = 𝑐 SMTconst [ 𝑘 ] () toSMT ( 𝑐 SMTvar [ 𝑥, 𝑡 ] ()) = 𝑐 SMTvar [ 𝑥, 𝑡 ] () toSMT ( 𝑐 SMTctor [ 𝑐 ] ( (cid:174) 𝑣 𝑖 )) = 𝑐 SMTctor [ 𝑐 ] ( (cid:174) 𝑣 𝑖 ) toSMT ( 𝑐 SMTlet ( 𝑣 , 𝑣 , 𝑣 )) = 𝑐 SMTlet ( 𝑣 , 𝑣 , 𝑣 ) toSMT ( 𝑐 SMTforall ( 𝑣 , 𝑣 )) = 𝑐 SMTforall ( 𝑣 , 𝑣 ) toSMT ( 𝑐 SMTuf [ uf ] ( (cid:174) 𝑣 𝑖 )) = 𝑐 SMTuf [ uf ] ( (cid:174) 𝑣 𝑖 ) Fig. 26. Formula semantics ( 𝜏 = 𝑡 smt ) Immediate: 𝜏 -SMT allows any 𝑚 .( 𝜏 = 𝑡 sym ) Immediate: 𝜏 -SMT allows any 𝑚 .( 𝜏 = model ) Contradictory— model is only well formed at exp . □ Lemma D.2 (Erasure is well formed). If Γ ⊢ smt 𝜏 then Γ ⊢ smt erase ( 𝜏 ) . Proof. By induction on the well formedness derivation.( 𝑡 -Base) Immediate, since erase ( 𝐵 ) = 𝐵 .( 𝑡 -TVar) Contradictory—type variables aren’t well typed at smt .( 𝑡 -ADT) By the IH on each constituent of 𝐷 (cid:174) 𝜏 𝑖 , and then by 𝑡 -ADT.( 𝜏 -SMT) Since erase ( 𝑡 smt ) = erase ( 𝑡 ) , by the IH on Γ ⊢ smt 𝑡 .( 𝜏 -Sym) Since erase ( 𝑡 sym ) = erase ( 𝑡 ) , by the IH on Γ ⊢ smt 𝑡 .( 𝜏 -Model) Contradictory— model isn’t well typed at smt . □ Lemma D.3 (SMT types have only SMT parts). If Γ ⊢ smt 𝜏 , then all of 𝜏 ’s subparts are also wellformed at smt . Proof. By induction on 𝜏 . ( 𝜏 = 𝐵 ) Immediate.( 𝜏 = 𝛼 ) Contradictory—type variables are only well formed at exp .( 𝜏 = 𝐷 (cid:174) 𝜏 𝑖 ) By the IH on each 𝜏 𝑖 .( 𝜏 = 𝑡 smt ) By the IH on 𝑡 .( 𝜏 = 𝑡 sym ) By the IH on 𝑡 .( 𝜏 = model ) Contradictory— model is only well formed at exp . □ Lemma D.4 (SMT conversion is well formed). If Γ ⊢ smt 𝜏 then toSMT ( 𝜏 ) = 𝑡 sym or 𝑡 smt such that Γ ⊢ smt 𝑡 (and so Γ ⊢ smt toSMT ( 𝜏 ) ). Proof. By induction on the well formedness derivation.( 𝑡 -Base) toSMT ( 𝐵 ) = 𝐵 smt , which is well formed by 𝜏 -SMT and 𝑡 -B.( 𝑡 -TVar) Contradictory—type variables are only well formed at exp .( 𝑡 -ADT) We know that erase ( 𝐷 (cid:174) 𝜏 𝑖 ) is still well formed by Lemma D.2; then by 𝜏 -SMT.( 𝜏 -SMT) Since it must be that Γ ⊢ smt 𝑡 , then erase ( 𝑡 ) is also well formed by Lemma D.2; thenby 𝜏 -SMT.( 𝜏 -Sym) Since it must be that Γ ⊢ smt 𝑡 , then erase ( 𝑡 ) is also well formed by Lemma D.2; thenby 𝜏 -Sym.( 𝜏 -Model) Contradictory— model is only well formed at smt □ Lemma D.5 (SMT conversion is only for SMT types). toSMT ( 𝜏 ) is defined iff Γ ⊢ smt 𝜏 . Proof. The right-to-left direction is proved by Lemma D.4. For left-to-right, we go by inductionon 𝜏 . ( 𝜏 = 𝐵 ) By 𝑡 -Base.( 𝜏 = 𝛼 ) Contradictory—type variables are undefined for erase .( 𝜏 = 𝐷 (cid:174) 𝜏 𝑖 ) By the IH on each 𝜏 𝑖 and 𝑡 -ADT.( 𝜏 = 𝑡 smt ) By the IH on 𝑡 and 𝜏 -SMT.( 𝜏 = 𝑡 sym ) By the IH on 𝑡 and 𝜏 -Sym.( 𝜏 = model ) Contradictory— model is undefined for erase . □ We say a type 𝑡 is an “SMT type” when Γ ⊢ smt 𝑡 ; a type 𝜏 is an SMT type when it is equal to anSMT type 𝑡 or when it is of the form 𝑡 smt or 𝑡 sym . Note that toSMT always produces an SMTtype, but does not work on types that contain type variables or the unrepresentable model type.Lemma D.6 (Weakening). If ⊢ Γ and ⊢ Γ ′ and dom ( Γ ) ∩ dom ( Γ ′ ) = ∅ then:(1) ⊢ Γ , Γ ′ (2) If Γ ⊢ 𝑚 𝜏 then Γ , Γ ′ ⊢ 𝑚 𝜏 ;(3) If Γ ⊢ 𝑒 : 𝜏 then Γ , Γ ′ ⊢ 𝑒 : 𝜏 ; and(4) If Γ ⊢ 𝜙 : 𝜏 then Γ , Γ ′ ⊢ 𝜙 : 𝜏 . Proof. By mutual induction on the derivations.
Contexts. ( Γ -Empty) We have Γ ′ = · ; immediate by assumption.( Γ -Var) We have Γ ′ = Γ ′′ , 𝑋 : 𝜏 . By the IH on Γ ′′ and Γ -Var, finding Γ , Γ ′′ ⊢ 𝑚 𝜏 by part (2)of the IH.( Γ -TVar) We have Γ ′ = Γ ′′ , 𝛼 . By the IH on Γ ′′ and Γ -TVar. ormulog: Datalog for SMT-Based Static Analysis 141:47 Type well formedness. ( 𝑡 -Base) Immediate, by 𝑡 -Base.( 𝑡 -TVar) Since Γ and Γ ′ have disjoint domains, we know 𝛼 ∈ Γ —by 𝑡 -TVar.( 𝑡 -ADT) By the IH on each constituent of 𝐷 (cid:174) 𝜏 𝑖 , followed by 𝑡 -ADT.( 𝜏 -SMT) By the IH on Γ ⊢ smt 𝑡 and then 𝜏 -SMT.( 𝜏 -Sym) By the IH on Γ ⊢ smt 𝑡 and then 𝜏 -Sym.( 𝜏 -Model) Immediate, by 𝜏 -Model. □ Expressions. ( 𝑒 -Var) Since the domains are disjoint, ( Γ , Γ ′ )( 𝑋 ) = 𝜏 and we can still find 𝑒 -Var.( 𝑒 -Const) Immediate, by 𝑒 -Const.( 𝑒 -Let) By the 𝑒 -Let and the IH on 𝑒 and 𝑒 , 𝛼 -renaming 𝑋 appropriately.( 𝑒 -Ctor) By 𝑒 -Ctor and the IH, using part (2) on 𝜏 ′ 𝑗 and part (3) on 𝑒 𝑖 .( 𝑒 -Quote) By the part (4) of the IH.( 𝑒 -Rel) By 𝑒 -Rel and the IH on each 𝑒 𝑖 .( 𝑒 -Fun) By 𝑒 -Fun and the the IH, using part (2) on 𝜏 ′ 𝑗 and part (3) on 𝑒 𝑖 .( 𝑒 -Op) By 𝑒 -Op and the the IH, using part (2) on 𝜏 ′ 𝑗 and part (3) on 𝑒 𝑖 .( 𝑒 -If) By 𝑒 -If and the IH on each of the 𝑒 𝑖 .( 𝑒 -Match) By 𝑒 -Match and the IH on 𝑒 and each of the 𝑒 𝑖 , 𝛼 -renaming each 𝑋 𝑘 appropriately. Formulas. ( 𝜙 -Var) By 𝜙 -Var and part (2) of the IH.( 𝜙 -Promote) By 𝜙 -Promote and the IH.( 𝜙 -Unqote) By 𝜙 -Unqote and part (3) of the IH.( 𝜙 -Ctor) By 𝜙 -Ctor and the IH, using part (2) on the 𝜏 𝑖 and part (4) on 𝜙 𝑖 , observing thatthe actual types are unchanged, and so the toSMT conversions are the same.Lemma D.7 (Type well formedness strengthening). If Γ , 𝑋 : 𝜏, Γ ′ ⊢ 𝑚 𝜏 ′ then Γ , Γ ′ ⊢ 𝜏 ′ . Proof. ( 𝑡 -Base) Immediate.( 𝑡 -TVar) Immediate: removing the variable binding can’t affect 𝛼 .( 𝑡 -ADT) By the IH on each constituent of 𝐷 (cid:174) 𝜏 𝑖 .( 𝜏 -SMT) By the IH on Γ ⊢ smt 𝑡 .( 𝜏 -Sym) By the IH on Γ ⊢ smt 𝑡 .( 𝜏 -Model) Immediate. □ Lemma D.8 (Type variable substitution). If ⊢ Γ , 𝛼, Γ ′ and Γ ⊢ 𝑚 𝜏 ′ , then:(1) ⊢ Γ , Γ ′ [ 𝜏 / 𝛼 ] ;(2) If Γ , 𝛼, Γ ′ ⊢ 𝑚 𝜏 then Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ 𝑚 𝜏 ′ [ 𝜏 / 𝛼 ] ;(3) If Γ , 𝛼, Γ ′ ⊢ 𝑋, 𝜏 ′ ▷ Γ ′′ then Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ 𝑋, 𝜏 ′ [ 𝜏 / 𝛼 ] ▷ Γ ′′ [ 𝜏 / 𝛼 ] ; and(4) If Γ , 𝛼, Γ ′ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 ′ 𝑖 ▷ Γ ′′ then Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 ′ 𝑖 [ 𝜏 / 𝛼 ] ▷ Γ ′′ [ 𝜏 / 𝛼 ] . Proof. For parts (1) and (2), by mutual induction on the derivations. Note that if 𝛼 actuallyoccurs in the type, we could only have found well formedness at exp . ( Γ -Empty) Contradictory: · ≠ Γ , 𝛼, Γ ′ .( Γ -Var) We have Γ ′ = Γ ′′ , 𝑋 : 𝜏 ′ where Γ , Γ ′′ ⊢ exp 𝜏 ′ . By the IH on Γ ′′ , we know that ⊢ Γ , Γ ′′ [ 𝜏 / 𝛼 ] ; by part (2), we have Γ , Γ ′′ [ 𝜏 / 𝛼 ] ⊢ exp 𝜏 ′ [ 𝜏 / 𝛼 ] ; and so we have ⊢ Γ , Γ ′ [ 𝜏 / 𝛼 ] by Γ -Var.( Γ -TVar) We have Γ ′ = Γ ′′ , 𝛽 ; by the IH on Γ ′′ , we have ⊢ Γ , Γ ′′ [ 𝜏 / 𝛼 ] ; since 𝛽 [ 𝜏 / 𝛼 ] = 𝛽 , wecan apply Γ -TVar to find ⊢ Γ , Γ ′ [ 𝜏 / 𝛼 ] as desired.( 𝑡 -B) Immediate by 𝑡 -B, since 𝐵 [ 𝜏 / 𝛼 ] = 𝐵 .( 𝑡 -TVar) We have 𝜏 ′ = 𝛽 . If 𝛼 = 𝛽 , then we have Γ ⊢ 𝑚 𝛼 [ 𝜏 / 𝛼 ] by assumption. If 𝛼 ≠ 𝛽 , thenit must be that 𝛽 ∈ Γ or Γ ′ —either way, 𝛽 is unaffected by the substitution and wehave 𝛽 ∈ Γ , Γ ′ [ 𝜏 / 𝛼 ] and so Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ 𝑚 𝛽 by 𝑡 -TVar.( 𝑡 -ADT) By the IH on each premise, followed by 𝑡 -ADT.( 𝜏 -SMT) By the IH on Γ , 𝛼, Γ ′ ⊢ smt 𝑡 and then by 𝜏 -SMT.( 𝜏 -Sym) By the IH on Γ , 𝛼, Γ ′ ⊢ smt 𝑡 and then by 𝜏 -Sym.( 𝜏 -Model) Immediate by 𝜏 -Model.For parts (3) and (4), by mutual induction on the derivations.( 𝑋𝜏 -Bind) We have 𝑋 ∉ dom ( Γ , 𝛼, Γ ′ ) , so it must also be the case that 𝑋 ∉ dom ( Γ , Γ ′ [ 𝜏 / 𝛼 ]) .We therefore find Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ 𝑋, 𝜏 ′ [ 𝜏 / 𝛼 ] ▷ Γ , Γ ′ [ 𝜏 / 𝛼 ] , 𝜏 ′ [ 𝜏 / 𝛼 ] by 𝑋𝜏 -Bind.( 𝑋𝜏 -Check) We have ( Γ , 𝛼, Γ ′ )( 𝑋 ) = 𝜏 ′ . Is 𝑋 : 𝜏 ′ in Γ or Γ ′ ? Either way we will find Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ 𝑋, 𝜏 ′ [ 𝜏 / 𝛼 ] ▷ Γ , Γ ′ [ 𝜏 / 𝛼 ] by 𝑋𝜏 -Check.If 𝜏 ′ ∈ dom ( Γ ) , then Γ ⊢ 𝜏 ′ and so 𝜏 ′ [ 𝜏 / 𝛼 ] = 𝜏 ′ ( Γ , Γ ′ [ 𝜏 / 𝛼 ])( 𝑋 ) = 𝜏 ′ and we have Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ 𝑋, 𝜏 ′ ▷ Γ , Γ ′ [ 𝜏 / 𝛼 ] .If, on the other hand, 𝜏 ′ ∈ dom ( Γ ′ ) , then ( Γ , Γ ′ [ 𝜏 / 𝛼 ])( 𝑋 ) = 𝜏 ′ [ 𝜏 / 𝛼 ] . We thereforehave Γ , Γ ′ [ 𝜏 / 𝛼 ] ⊢ 𝑋, 𝜏 ′ [ 𝜏 / 𝛼 ] ▷ Γ , Γ ′ [ 𝜏 / 𝛼 ] .( (cid:174) 𝑋 (cid:174) 𝜏 -All) By part (3) of the IH on each premise. □ Lemma D.9 (Regularity; formulas have SMT types). (1) If ⊢ Γ and Γ ( 𝑋 ) = 𝜏 then Γ ⊢ exp 𝜏 .(2) If ⊢ Φ then (a) if 𝑓 : ∀ (cid:174) 𝛼 𝑗 , (cid:174) 𝜏 𝑖 → 𝜏 ∈ Φ then (cid:174) 𝛼 𝑗 ⊢ exp 𝜏 , and (b) if uf : (cid:174) 𝑡 ′ 𝑖 → 𝑡 ∈ Φ then · ⊢ smt 𝑡 .(3) If Δ ; Φ ; Γ ⊢ 𝑒 : 𝜏 then Γ ⊢ exp 𝜏 .(4) If Δ ; Φ ; Γ ⊢ 𝑐 SMTc ( (cid:174) 𝜙 𝑖 ) : (cid:174) 𝜏 𝑖 → 𝜏 then 𝜏 𝑖 = 𝑡 𝑖 smt or 𝜏 𝑖 = 𝑡 𝑖 sym and 𝜏 = 𝑡 smt or 𝜏 = 𝑡 sym and Γ ⊢ smt 𝜏 𝑖 and Γ ⊢ smt 𝜏 .(5) If Δ ; Φ ; Γ ⊢ 𝜙 : 𝜏 then 𝜏 = 𝑡 smt or 𝜏 = 𝑡 sym and Γ ⊢ smt 𝜏 .(6) If Δ ; Φ ; Γ ⊢ (cid:174) 𝑒 𝑖 : (cid:174) 𝜏 𝑖 then Γ ⊢ exp 𝜏 𝑖 .(7) If Δ ; Φ ; Γ ⊢ (cid:174) 𝜙 𝑖 : (cid:174) 𝜏 𝑖 then 𝜏 𝑖 = 𝑡 𝑖 smt or 𝜏 = 𝑡 𝑖 sym and Γ ⊢ smt 𝑡 𝑖 (and so Γ ⊢ exp 𝜏 𝑖 ). Proof. By induction on the typing derivation.
Contexts. ( Γ -Empty) Contradictory—there’s no way · has a binding for 𝑋 .( Γ -Var) Γ = Γ ′ , 𝑌 : 𝜏 . If 𝑋 = 𝑌 , then we know Γ ⊢ exp 𝜏 by assumption; otherwise, by the IHon Γ ′ .( Γ -TVar) Γ = Γ ′ , 𝛼 . By the IH on Γ . Program signatures. ( Φ -Empty) Contradictory—there are no function definitions in · . ormulog: Datalog for SMT-Based Static Analysis 141:49 ( Φ -Fun) Φ = Φ ′ , 𝑔 : . . . . For case (a) when 𝑓 = 𝑔 , then by assumption. Otherwise, by the IHon Φ ′ .( Φ -Rel) Φ = Φ ′ , 𝑝 ⊆ (cid:174) 𝜏 𝑖 . By the IH on Φ .( Φ -UFun) Φ = Φ ′ , uf ′ : (cid:174) 𝑡 ′ 𝑖 → 𝑡 . For case (b) when uf = uf ′ , then by assumption. Otherwise,by the IH on Φ ′ . Expressions. ( 𝑒 -Var) By the part (1) on ⊢ Γ .( 𝑒 -Const) By assumption, we know that Γ ⊢ smt typeof ( 𝑘 ) ; by Lemma D.1 we can find Γ ⊢ exp typeof ( 𝑘 ) .( 𝑒 -Let) By the IH on Γ , 𝑋 : 𝜏 ⊢ 𝑒 : 𝜏 , using strengthening (Lemma D.7) to find that if Γ , 𝑋 : 𝜏 ⊢ exp 𝜏 then Γ ⊢ exp 𝜏 .( 𝑒 -Ctor) Since Γ ⊢ exp 𝜏 ′ 𝑗 , we know by 𝑡 -ADT that Γ ⊢ exp 𝐷 (cid:174) 𝜏 ′ 𝑗 .( 𝑒 -Quote) By the IH on part (5), we know that Γ ⊢ smt 𝜏 (and, less relevantly, that 𝜏 = 𝑡 smt or 𝑡 sym ). We can find the same well formedness at exp by Lemma D.1.( 𝑒 -Rel) Immediate by 𝑡 -B.( 𝑒 -Fun) Since 𝑓 : ∀ (cid:174) 𝛼 𝑗 , (cid:174) 𝜏 𝑖 → 𝜏 ∈ Φ and ⊢ Φ , we know by part (2) of the IH know that (cid:174) 𝛼 𝑗 ⊢ exp 𝜏 . By weakening (Lemma D.6) we can lift that well formedness judgment to Γ . Since each Γ ⊢ exp 𝜏 ′ 𝑗 , we can find that Γ ⊢ 𝜏 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] by substitution (Lemma D.8).( 𝑒 -Fun) We have by assumption that typeof (⊗) yields a well-formed type, i.e., (cid:174) 𝛼 𝑗 ⊢ exp 𝜏 (and also for each 𝜏 𝑖 ). By weakening (Lemma D.6) we can lift that well formednessjudgment to Γ . Since each Γ ⊢ exp 𝜏 ′ 𝑗 , we can find that Γ ⊢ 𝜏 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] by substitution(Lemma D.8).( 𝑒 -If) By the IH on Γ ⊢ 𝑒 : 𝑡 .( 𝑒 -Match) By the IH on Γ , −−−−−−−−−−−−→ 𝑋 : 𝜏 [ 𝜏 𝑗 / 𝛼 𝑗 ] ⊢ 𝑒 : 𝜏 we have Γ , −−−−−−−−−−−−→ 𝑋 : 𝜏 [ 𝜏 𝑗 / 𝛼 𝑗 ] ⊢ 𝜏 ; we can usestrengthening (Lemma D.7) to find Γ ⊢ exp 𝜏 . SMT constructors. ( 𝑐 -SMT-Var) Immediate, with Γ ⊢ smt 𝑡 coming from the rule itself.( 𝑐 -SMT-Const) Immediate, since we have by assumption that · ⊢ smt typeof ( 𝑘 ) .( 𝑐 -SMT-Let) Immediate, with the necessary well formedness assumptions coming from the ruleitself.( 𝑐 -SMT-Ctor) We know that 𝐷 (cid:174) 𝑡 ′ 𝑗 is well formed by 𝑡 -ADT. We can find the translation of theargument types well formed by Lemma D.4 on each of the Γ ⊢ smt 𝜏 𝑖 [ 𝑡 ′ 𝑗 / 𝛼 𝑗 ] deriva-tions.( 𝑐 -SMT-Forall) Immediate, with the necessary Γ ⊢ smt 𝑡 coming from the rule itself.( 𝑐 -SMT-UFun) Since ⊢ Φ and uf : (cid:174) 𝑡 𝑖 → 𝑡 ∈ Φ , we know that · ⊢ smt 𝑡 by part (2) of the IH and so · ⊢ smt 𝑡 smt , which we can lift to Γ by weakening (Lemma D.6). Formulas. ( 𝜙 -Promote) Since Δ ; Φ ; Γ ⊢ 𝜙 : 𝑡 sym , we know that Γ ⊢ smt 𝑡 and so we are correct in yielding 𝜏 = 𝑡 smt .( 𝜙 -Unqote) We have Δ ; Φ ; Γ ⊢ 𝑒 : 𝜏 such that Γ ⊢ smt 𝜏 . By Lemma D.4 we know that toSMT ( 𝜏 ) is a well formed SMT type.( 𝜙 -Ctor) By the IH part (4) on Γ ⊢ 𝑐 SMTc : (cid:174) 𝜏 𝑖 → 𝜏 . Vectored expressions and formulas.
By the IH for parts (3) and (5), respectively. □ Lemma D.10 (SMT value conversion is type correct). If Δ ; Φ ; Γ ⊢ 𝑣 : 𝜏 and Γ ⊢ smt 𝜏 then Δ ; Φ ; Γ ⊢ toSMT ( 𝑣 ) : toSMT ( 𝜏 ) . Proof. First, observe that 𝜏 and all of its parts must be well formed at smt , by Lemmas D.5and D.3. By induction on the typing derivation. In expression mode, the applicable rules are 𝑒 -Constand 𝑒 -Ctor; only a few typing rules could even have applied to a value in formula mode: the 𝜙 -Ctorand 𝜙 -Promote.( 𝑒 -Const) We have Γ ⊢ 𝑘 : typeof ( 𝑘 ) ; since toSMT ( 𝑘 ) = 𝑐 SMTconst [ 𝑘 ] () and toSMT ( typeof ( 𝑘 )) = 𝑘 smt (since Γ ⊢ smt typeof ( 𝑘 ) by assumption), we must show that Γ ⊢ 𝑐 SMTconst [ 𝑘 ] () :typeof ( 𝑘 ) smt , which we have by 𝜙 -SMT-Const.( 𝑒 -Ctor) We have 𝑣 = 𝑐 ( (cid:174) 𝑣 𝑖 ) and: Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 : (cid:174) 𝜏 𝑖 , . . . } Γ ⊢ exp 𝜏 ′ 𝑗 Γ ⊢ 𝑣 𝑖 : 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] Further, we know that Γ ⊢ smt 𝐷 (cid:174) 𝜏 ′ 𝑗 (and so each of the subderivations must also be smt ) and that toSMT ( 𝑐 ( (cid:174) 𝑣 𝑖 )) = 𝑐 SMTctor [ 𝑐 ] (−−−−−−−−−−→ toSMT ( 𝑣 𝑖 )) . By the IH on each of these 𝑣 𝑖 , we know that we find appropriate values at appropriately converted types, i.e., Γ ⊢ toSMT ( 𝑣 𝑖 ) : toSMT ( 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ]) . By Lemma D.4, we know toSMT ( 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ]) is some well formed SMT type. We are almost able to apply 𝜙 -SMT-Ctor, but wemust pick appropriate 𝑡 ′ 𝑗 . We know that toSMT ( 𝜏 ′ 𝑗 ) is a well formed SMT type ofthe form 𝑡 ′ 𝑗 smt or 𝑡 ′ 𝑗 sym (Lemma D.4). Whether it’s symbolic or not, let the inner 𝑡 ′ 𝑗 there be our 𝑡 ′ 𝑗 . We can now apply 𝜙 -SMT-Ctor to find that Γ ⊢ toSMT ( 𝑐 ( (cid:174) 𝑣 𝑖 )) :toSMT ( 𝐷 (cid:174) 𝜏 ′ 𝑗 ) .( 𝜙 -Promote) By the IH on Γ ⊢ 𝑣 : 𝑡 sym , we know that Γ ⊢ toSMT ( 𝑣 ) : toSMT ( 𝑡 sym ) , i.e., Γ ⊢ toSMT ( 𝑣 ) : 𝑡 sym . By reapplying 𝜙 -Promote we can find that Γ ⊢ toSMT ( 𝑣 ) : 𝑣 smt .( 𝜙 -Ctor) Immediate: toSMT does nothing to the 𝑐 SMT ... constructed value nor to the SMT-type 𝜏 assigned to it (which is 𝑡 smt in all cases except for 𝑐 SMTvar ). □ E TYPE SAFETY
To prove type safety, we prove two properties for every mode of evaluation: first, it is safe , i.e, neveryields ⊥ ; and second, it is type preserving , i.e., well typed inputs yield well typed outputs.The proofs are fairly conventional. For all but the last step, we prove safety and type preservationsimultaneously. We start with expressions and formulas (Lemma E.2), which requires a modestnotion of canonical forms (Lemma E.1). Next, we prove that value unification (Lemma E.3) is typepreserving, reasoning about unification in general within the lemma showing safety and typepreservation for premises (Lemma E.4). After a brief lemma about bindings (Lemma E.5), we canprove that program evaluation is safe (Theorem E.6) and type preserving (Theorem E.7).Lemma E.1 (Canonical forms for 𝑡 sym ). If Δ ; Φ ; Γ ⊢ 𝑣 : 𝑡 sym then 𝑣 = 𝑐 SMTvar [ 𝑥, 𝑡 ] () . Proof. The only typing rule that could have applied is 𝜙 -SMT-Var. □ Lemma E.2 (Term and formula type safety). If Δ ; Φ | = W and Δ ; Φ ⊢ (cid:174) 𝐹 and Γ | = 𝜃 , wheneither: ormulog: Datalog for SMT-Based Static Analysis 141:51 (1) Δ ; Φ ; Γ ⊢ 𝑒 : 𝜏 and W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ ; or(2) Δ ; Φ ; Γ ⊢ 𝜙 : 𝜏 and W ; 𝜃 ⊢ 𝜙 ⇓ 𝜙 𝑣 ⊥ (3) Δ ; Φ ⊢ fun 𝑓 ( (cid:174) 𝑋 𝑖 : (cid:174) 𝜏 𝑖 ) : 𝜏 = 𝑒 and (cid:174) 𝛼 𝑗 , −−−−→ 𝑋 𝑖 : 𝜏 𝑖 | = 𝜃 ′ and W ; 𝜃 ′ ⊢ 𝑒 ⇓ 𝑒 𝑣 ⊥ then 𝑣 ⊥ = 𝑣 (i.e., 𝑣 ⊥ ≠ ⊥ ) and Δ ; Φ ; Γ ⊢ 𝑣 : 𝜏 .Similarly, when either:(1) if Δ ; Φ ; Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 and W ; 𝜃 ⊢ (cid:174) 𝑒 𝑖 ⇓ 𝑒 (cid:174) 𝑣 𝑖 then Δ ; Φ ; Γ ⊢ (cid:174) 𝑣 𝑖 ⊥ : (cid:174) 𝜏 𝑖 ; and(2) if Δ ; Φ ; Γ ⊢ 𝜙 𝑖 : 𝜏 𝑖 and W ; 𝜃 ⊢ (cid:174) 𝜙 𝑖 ⇓ (cid:174) 𝜙 (cid:174) 𝑣 𝑖 then Δ ; Φ ; Γ ⊢ (cid:174) 𝑣 𝑖 ⊥ : 𝜏 then (cid:174) 𝑣 𝑖 ⊥ = (cid:174) 𝑣 𝑖 (i.e., it is not ⊥ ) and Δ ; Φ ; Γ ⊢ 𝑣 𝑖 : 𝜏 𝑖 . Proof. By mutual induction on derivations and the length of the vectored expressions/formulas,leaving 𝜃 general (for, e.g., 𝑒 -Let and 𝑒 -Match). Expressions. ( 𝑒 -Var) We have Γ ( 𝑋 ) = 𝜏 ; since Γ | = 𝜃 , we have 𝜃 ( 𝑋 ) = 𝑣 (and so ⇓ 𝑒 -Var-E didn’t apply).So it must be the case that ⇓ 𝑒 -Var applied. We can see further that Δ ; Φ ; · ⊢ 𝑣 : 𝜏 ,and we are done by weakening (Lemma D.6).( 𝑒 -Const) It must be that ⇓ 𝑒 -Const applied, and we immediately see that 𝑣 ⊥ ≠ ⊥ and 𝑘 iswell typed in any well formed context by assumption and 𝑒 -Const.( 𝑒 -Let) We know that Γ ⊢ 𝑒 : 𝜏 and Γ , 𝑋 : 𝜏 ⊢ 𝑒 : 𝜏 . By the IH on 𝑒 , we know that 𝜃 ; W ⊢ 𝑒 ⇓ 𝑒 𝑣 , so it can’t be the case that ⇓ 𝑒 -Let-E applied—it must hae been ⇓ 𝑒 -Let. By the IH on 𝑒 , we know that the final result is also not ⊥ and is welltyped.( 𝑒 -Ctor) We have Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 : (cid:174) 𝜏 𝑖 , . . . } and Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] . By the IH, we knowthat each of the (cid:174) 𝑒 𝑖 must have reduced to non- ⊥ values, and so ⇓ 𝑒 -Ctor-E could nothave applied. We can therefore see that each 𝑒 𝑖 reduces to an appropriately typed 𝑣 𝑖 , and our resulting value is well typed by 𝑒 -Ctor.( 𝑒 -Quote) Only ⇓ 𝑒 -Quote could have applied. By the IH, we know that 𝜙 reduces to a non- ⊥ value 𝑣 well typed at 𝜏 .( 𝑒 -Rel) We know that 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ and Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 . The IH on (cid:174) 𝑒 𝑖 rules out 𝑆𝑡𝑒𝑝𝑠𝑡𝑜𝐸 -Rel-E1;the typing rule rules out the arity mismatch in ⇓ 𝑒 -Rel-E2 and the missing relationin ⇓ 𝑒 -Rel-E3. So it must be the case that ⇓ 𝑒 -Rel-True or ⇓ 𝑒 -Rel-False applied;either way, we yield a bool , which is appropriately typed by 𝑒 -Const.( 𝑒 -Fun) We know that 𝑓 : ∀ (cid:174) 𝛼 𝑗 , (cid:174) 𝜏 𝑖 → 𝜏 ∈ Φ and Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] . The IH on (cid:174) 𝑒 𝑖 rules out 𝑆𝑡𝑒𝑝𝑠𝑡𝑜𝐸 -Fun-E1; the typing rule rules out the arity mismatch in ⇓ 𝑒 -Fun-E2 andthe missing function in ⇓ 𝑒 -Fun-E3. So it must be the case that ⇓ 𝑒 -Fun applied. Since Δ ; Φ ⊢ 𝐹 , we know by the IH on part (3) that the resulting value is non- ⊥ and welltyped at 𝜏 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] .( 𝑒 -Op) We know that typeof (⊗) = ∀ (cid:174) 𝛼 𝑗 , (cid:174) 𝜏 𝑖 → 𝜏 and Γ ⊢ 𝑒 𝑖 : 𝜏 𝑖 [ 𝜏 ′ 𝑗 / 𝛼 𝑗 ] . The IH on (cid:174) 𝑒 𝑖 rules out 𝑆𝑡𝑒𝑝𝑠𝑡𝑜𝐸 -Op-E1; the typing rule rules out the arity/domain mismatch in ⇓ 𝑒 -Op-E2. So it must be the case that ⇓ 𝑒 -Op applied. We know that the result iswell typed by our assumption that typeof (⊗) and [[⊗]] agree.( 𝑒 -If) We have Γ ⊢ 𝑒 : bool and Γ ⊢ 𝑒 : 𝜏 and Γ ⊢ 𝑒 : 𝜏 . By the IH on 𝑒 , we knowthat 𝑒 reduces to true or false (since those are the only values of type bool ). So wecan rule out ⇓ 𝑒 -Ite-E1 and ⇓ 𝑒 -Ite-E2—we must have stepped by either ⇓ 𝑒 -Ite-Tor ⇓ 𝑒 -Ite-F. The IH on 𝑒 or 𝑒 (respectively) guarantees we step to a non- ⊥ , welltyped value. ( 𝑒 -Match) We have Γ ⊢ 𝑒 : 𝐷 (cid:174) 𝜏 𝑗 and Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 𝑖 : (cid:174) 𝜏 𝑘 , . . . } and Γ , −−−−−−−−−−−−→ 𝑋 𝑘 : 𝜏 𝑘 [ 𝜏 𝑗 / 𝛼 𝑗 ] ⊢ 𝑒 𝑖 : 𝜏 .The IH on 𝑒 guarantees that we get a non- ⊥ value at type 𝐷 (cid:174) 𝜏 𝑗 , which rules outthe error case ⇓ 𝑒 -Match-E1, the non-constructor value of ⇓ 𝑒 -Match-E2, the mis-named constructor of ⇓ 𝑒 -Match-E3, and the arity error of ⇓ 𝑒 -Match-E4. So it mustbe the case that we applied ⇓ 𝑒 -Match; by the IH, the matching pattern reduces toa well typed non- ⊥ value. Formulas. ( 𝜙 -Promote) We have Δ ; Φ ; Γ ⊢ 𝜙 : 𝑡 sym ; by the IH, we know that 𝜙 steps to a non- ⊥ value 𝑣 well typed at 𝑡 sym ; by 𝜙 -Promote we can see that 𝑣 is also well typed at 𝑡 smt .( 𝜙 -Unqote) We have , 𝑒 ; since Δ ; Φ ; Γ ⊢ 𝑒 : 𝜏 , we know by the IH that 𝑒 reduces to a non- ⊥ value 𝑣 that is also well typed at 𝜏 . We can therefore rule out ⇓ 𝜙 -Unqote-E, so we musthave stepped by ⇓ 𝜙 -Unqote.Since Δ ; Φ ; Γ ⊢ 𝑣 : 𝜏 and Γ ⊢ smt 𝜏 , we have Δ ; Φ ; Γ ⊢ toSMT ( 𝑣 ) : toSMT ( 𝜏 ) byLemma D.10, as desired.( 𝜙 -Ctor) We have 𝑐 SMTc ( (cid:174) 𝜙 𝑖 ) such that Γ ⊢ 𝑐 SMTc : (cid:174) 𝜏 𝑖 → 𝜏 and Γ ⊢ 𝜙 𝑖 : 𝜏 𝑖 . We know by the IHthat each 𝜙 𝑖 is well typed at 𝜏 𝑖 and so none of them step to ⊥ , and so ⇓ 𝜙 -Ctor-Ecannot apply.Therefore either ⇓ 𝜙 -Ctor or ⇓ 𝜙 -Value applied; the resulting value is well typedby the IH or remains well typed, respectively. Functions.
By part (1) on (cid:174) 𝛼 𝑗 , −−−−→ 𝑋 𝑖 : 𝜏 𝑖 ⊢ 𝑒 : 𝜏 , using weakening (Lemma D.6) to recover typing in Γ . Vectored expressions and formulas.
By induction on the vector length, using parts (1) and (2) ineach case. □ Lemma E.3 (Value unification preservation). If Γ | = 𝜃 when either:(1) Γ ⊢ (cid:174) 𝑋, (cid:174) 𝜏 ▷ Γ ′ and Γ ⊢ (cid:174) 𝑣 : (cid:174) 𝜏 and 𝜃 ⊢ (cid:174) 𝑋 ∼ (cid:174) 𝑣 : 𝜃 ′ ; or(2) Γ ⊢ 𝑋, 𝜏 ▷ Γ ′ and Γ ⊢ 𝑣 : 𝜏 and 𝜃 ⊢ 𝑋 ∼ 𝑣 ▷ 𝜃 ′ ;then Γ ′ | = 𝜃 ′ . Proof. By induction on the derivation of well typing.( 𝑋𝜏 -Bind) Only 𝑢𝑣 -Bind-Var could have applied, so we have Γ ⊢ 𝑣 : 𝜏 and Γ | = 𝜃 and mustshow that Γ , 𝑋 : 𝜏 | = 𝜃 [ 𝑋 ↦→ 𝑣 ] , which we have immediately.( 𝑋𝜏 -Check) Here 𝑋 ∈ Γ , so it must be that 𝜃 ( 𝑋 ) is defined. One of three rules could haveapplied:( 𝑢𝑣 -Eq-Var) We have Γ ′ = Γ and 𝜃 ′ = 𝜃 , so Γ ′ | = 𝜃 ′ by assumption.( 𝑢𝑣 -Ctor) By the IH, we know that Γ ′ 𝑚𝑜𝑑𝑒𝑙𝑠𝜃 ′ .( 𝑢𝑣 -Constant) As for 𝑢𝑣 -Eq-Var, we have Γ ′ = Γ and 𝜃 ′ = 𝜃 , so Γ ′ | = 𝜃 ′ byassumption.( 𝑋𝜏 -All) It must be that (cid:174) 𝑢 (cid:174) 𝑣 -All applied; by the IH on each sub-derivation, we can find that Γ 𝑖 | = 𝜃 𝑖 , and so Γ ′ | = 𝜃 ′ in particular. □ User code will never directly trigger a use of 𝑢𝑣 -Eq-Var directly, because the unification ruleswon’t call value unification with a defined LHS (we’d just use 𝑢𝑢 -BB instead). But a use of (cid:174) 𝑢 (cid:174) 𝑣 -Allcould lead to a variable being unified early on and then used again in the same unification process. ormulog: Datalog for SMT-Based Static Analysis 141:53 Lemma E.4 (Premise preservation and safety). If Δ ; Φ ; Γ ⊢ 𝑃 ▷ Γ ′ and Δ ; Φ | = (cid:174) 𝐹 and Δ ; Φ | = W and Γ | = 𝜃 then if (cid:174) 𝐹 ; W ; 𝜃 ⊢ 𝑃 → 𝜃 ′⊥ then:(1) 𝜃 ′⊥ = 𝜃 ′ (i.e., it is not ⊥ ); and(2) Γ ′ | = 𝜃 ′ . Proof. By induction on the premise typing derivation, followed by cases on the step taken.( 𝑃 -PosAtom) We have: 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ The only rule that could have applied is PosAtom, i.e., (cid:174) 𝑣 ∈ W ( 𝑝 ) and 𝜃 ⊢ (cid:174) 𝑋 𝑖 ∼ (cid:174) 𝑣 𝑖 : 𝜃 ′⊥ . We must show that 𝜃 ′⊥ = 𝜃 ′ and Γ ′ | = 𝜃 ′ .Since Δ ; Φ | = W , we know that · ⊢ (cid:174) 𝑣 𝑖 : (cid:174) 𝜏 𝑖 ; by weakening we have Γ ⊢ (cid:174) 𝑣 𝑖 : (cid:174) 𝜏 𝑖 (Lemma D.6).Syntactically, we know that (cid:174) 𝑋 𝑖 are all variables and that (cid:174) 𝑣 𝑖 are all values. For eachone, therefore only two unification rules could possibly apply: 𝑢𝑢 -BB ( 𝑋 𝑖 is bound)and 𝑢𝑢 -FB ( 𝑋 𝑖 is free). In particular, 𝑢𝑢 -FF cannot apply, and so we cannot produce ⊥ , so 𝜃 ′⊥ = 𝜃 ′ = 𝜃 (cid:174) 𝜃 𝑖 . By Lemma E.3, we know that Γ ′ | = 𝜃𝜃 𝑖 for each 𝑖 , and so Γ ′ | = 𝜃 (cid:174) 𝜃 𝑖 .( 𝑃 -NegAtom) We have: 𝑝 ⊆ (cid:174) 𝜏 𝑖 ∈ Φ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ Two rules are possible: NegAtom and NegAtom-E. We must show that the lattercannot apply and that the former preserves typing.Since Γ | = 𝜃 , it must be that case that each (cid:174) 𝑋 𝑖 ∈ dom ( 𝜃 ) , and so NegAtom-E cannothave applied. It remains to be seen that Γ | = 𝜃 ′ —but in NegAtom we have 𝜃 = 𝜃 ′ ,and so we are done.( 𝑃 -EqCtor-BF) We have: Δ ( 𝐷 ) = ∀ (cid:174) 𝛼 𝑗 , { . . . , 𝑐 : (cid:174) 𝜏 𝑖 , . . . } Γ ⊢ 𝑌, 𝜏 [ (cid:174) 𝜏 ′ 𝑗 / (cid:174) 𝛼 𝑗 ] ▷ Γ (cid:174) 𝑋 𝑖 ⊈ Γ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 [ (cid:174) 𝜏 ′ 𝑗 / (cid:174) 𝛼 𝑗 ] ▷ Γ ′ The only rule that could have applied is EqCtor, where 𝜃 ⊢ 𝑌 𝑐 ( (cid:174) 𝑋 𝑖 ) : 𝜃 ′⊥ . We mustshow that 𝜃 ′⊥ = 𝜃 ′ (i.e., it is not ⊥ ) and that Γ ′ | = 𝜃 ′ .Since Γ ⊢ 𝑌, 𝜏 [ (cid:174) 𝜏 ′ 𝑗 / (cid:174) 𝛼 𝑗 ] ▷ Γ , it must be the case that 𝑌 ∈ dom ( Γ ) and so 𝜃 ( 𝑌 ) = 𝑣 (andso · ⊢ 𝑣 : 𝜏 [ (cid:174) 𝜏 ′ 𝑗 / (cid:174) 𝛼 𝑗 ] , which also holds under Γ thanks to weakening (Lemma D.6)).Only two rules could have applied to show 𝜃 ⊢ 𝑌 𝑐 ( (cid:174) 𝑋 𝑖 ) : 𝜃 ′⊥ : 𝑢𝑢 -BF (when someof (cid:174) 𝑋 𝑖 are unbound) or 𝑢𝑢 -BB (when all of the (cid:174) 𝑋 𝑖 are bound). In either case, 𝑢𝑢 -FFcan’t have a applied, and so 𝜃 ′⊥ = 𝜃 ′ .One of two rules could have applied: 𝑢𝑣 -Eq-Var or 𝑢𝑣 -Ctor.In the former case, we applied 𝑢𝑢 -BB, because 𝜃 ( 𝑐 ( (cid:174) 𝑋 𝑖 )) = 𝑐 ( (cid:174) 𝑣 𝑖 ) . We have 𝜃 ′ = 𝜃 [ 𝑋 ↦→ 𝑐 ( (cid:174) 𝑣 𝑖 ) and Γ ′ | = 𝜃 ′ by substitution on Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 [ (cid:174) 𝜏 ′ 𝑗 / (cid:174) 𝛼 𝑗 ] ▷ Γ ′ (Lemma D.8).In the latter case, we can find that Γ ′ | = 𝜃 ′ by Lemma E.3 on the assumption that Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 [ (cid:174) 𝜏 ′ 𝑗 / (cid:174) 𝛼 𝑗 ] ▷ Γ ′ , and the fact 𝜃 ( 𝑌 ) = 𝑣 is well typed in Γ .( 𝑃 -EqSMT-BF) We have: Γ ⊢ 𝑐 SMTc ′ : (cid:174) 𝜏 𝑖 → 𝜏 Γ ⊢ 𝑌, 𝜏 ▷ Γ (cid:174) 𝑋 𝑖 ⊈ Γ Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ The only rule that could have applied is EqSMT, where 𝜃 ⊢ 𝑌 ∼ 𝑐 SMTc ′ ( (cid:174) 𝑋 𝑖 ) : 𝜃 ′⊥ . Wemust show that 𝜃 ′⊥ = 𝜃 ′ (i.e., it is not ⊥ and that Γ | = 𝜃 ′ .Since Γ ⊢ 𝑌, 𝜏 ▷ Γ , it must be the case that 𝑌 ∈ dom ( Γ ) and so 𝜃 ( 𝑌 ) = 𝑣 . We canconclude that · ⊢ 𝑣 : 𝜏 and so Γ ⊢ 𝑣 : 𝜏 (Lemma D.6).Only two rules could have applied to show 𝜃 ⊢ 𝑌 𝑐
SMTc ′ ( (cid:174) 𝑋 𝑖 ) : 𝜃 ′⊥ , noting the removalof the unquote, since unification doesn’t care: 𝑢𝑢 -BF (when some of (cid:174) 𝑋 𝑖 are unbound)or 𝑢𝑢 -BB (when all of the (cid:174) 𝑋 𝑖 are bound). In either case, 𝑢𝑢 -FF can’t have a applied,and so 𝜃 ′⊥ = 𝜃 ′ .It remains to show that Γ | = 𝜃 ′ . If the outer unification rule was 𝑢𝑢 -BB, we have 𝜃 = 𝜃 ′ and so Γ | = 𝜃 ′ by assumption. If outer unification rule was 𝑢𝑢 -BF, one of tworules could have applied to find value unification: either 𝑢𝑣 -Eq-Var or 𝑢𝑣 -Ctor.In the former case, we apply 𝑢𝑢 -BB inside, because 𝜃 ( 𝑌 ) = 𝑐 SMTc ′ ( (cid:174) 𝑣 𝑖 ) . We have 𝜃 ′ = 𝜃 [ 𝑋 ↦→ 𝑐 SMTc ′ ( (cid:174) 𝑣 𝑖 ) and Γ ′ | = 𝜃 ′ by substitution on Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ (Lemma D.8).In the latter case, we can find that Γ ′ | = 𝜃 ′ by Lemma E.3 on the assumption that Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ ′ , and the fact 𝜃 ( 𝑌 ) = 𝑣 is well typed in Γ .( 𝑃 -Eq-FB) We have: Γ ⊢ 𝑒 : 𝜏 Γ ⊢ 𝑌, 𝜏 ▷ Γ ′ The two possible rules are EqExpr and EqExpr-E. We must show that the lattercould not have applied (and so 𝜃 ′⊥ = 𝜃 ′ ) and that Γ ′ | = 𝜃 ′ . By Lemma E.2, we knowthat EqExpr-E cannot apply and that W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 (and so Γ ⊢ 𝑣 : 𝜏 ).Since 𝑣 is a value, either 𝑢𝑢 -FB or 𝑢𝑢 -BB applied, depending on whether or not 𝑌 is bound. Either way, 𝑢𝑢 -FF couldn’t have applied, and so 𝜃 ′⊥ = 𝜃 ′ .We can find that Γ ′ | = 𝜃 ′ by Lemma E.3 on Γ ⊢ 𝑌, 𝜏 ▷ Γ ′ (along with the well typingof 𝑣 ).( 𝑃 -NegEq) We have Γ ⊢ 𝑒 : 𝜏 and Γ ⊢ 𝑌, 𝜏 ▷ Γ . By Lemma E.2, we know that NegEqExpr-E1cannot apply and that W ; 𝜃 ⊢ 𝑒 ⇓ 𝑒 𝑣 (and so Γ ⊢ 𝑣 : 𝜏 ). Since Γ | = 𝜃 , it must be thatcase that 𝑌 ∈ dom ( 𝜃 ) , and so NegAtom-E2 cannot have applied. It remains to beseen that Γ | = 𝜃 ′ —but in NegExpr we have 𝜃 = 𝜃 ′ , and so we are done. □ Lemma E.5 (Identical bindings implies containment). If Γ ⊢ 𝑋, 𝜏 ▷ Γ , then 𝑋 ∈ dom ( Γ ) .Similarly, if Γ ⊢ (cid:174) 𝑋 𝑖 , (cid:174) 𝜏 𝑖 ▷ Γ , then (cid:174) 𝑋 𝑖 ⊆ dom ( Γ ) . Proof. By induction on the derivation.( 𝑋𝜏 -Bind) Contradictory: this rule could not have applied, since Γ ≠ Γ , 𝑋 : 𝜏 .( 𝑋𝜏 -Check) We have 𝑋 ∈ dom ( Γ ) by assumption.( (cid:174) 𝑋 (cid:174) 𝜏 -All) By the IH on each of our premises. □ Theorem E.6 (Program safety). If Δ ; Φ ⊢ (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 and Δ ; Φ | = W then for all 𝐻 ∈ (cid:174) 𝐻 𝑗 , ¬( (cid:174) 𝐹 𝑖 ; W ⊢ 𝐻 → ⊥) . Proof. The program prog = (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 must have been well typed according to prog -WF, and so wehave ⊢ Δ and ⊢ Φ along with derivations for each 𝐹 and 𝐻 : Δ ; Φ ⊢ 𝐹 . . . Δ ; Φ ⊢ 𝐹 𝑖 . . . Δ ; Φ ⊢ 𝐹 𝑛 Δ ; Φ ⊢ 𝐻 . . . Δ ; Φ ⊢ 𝐻 𝑗 . . . Δ ; Φ ⊢ 𝐻 𝑚 ormulog: Datalog for SMT-Based Static Analysis 141:55 Let an 𝐻 = 𝑝 ( 𝑋 𝑘 ) : − (cid:174) 𝑃 ℓ ∈ (cid:174) 𝐻 𝑗 be given. We know that Δ ; Φ ⊢ 𝐻 by 𝐻 -Clause, i.e.: · ⊢ 𝑃 ▷ Γ . . . Γ ℓ ⊢ 𝑃 ℓ ▷ Γ ℓ + . . . Γ 𝑝 ⊢ 𝑃 𝑝 ▷ Γ ′ 𝑝 ⊆ (cid:174) 𝜏 𝑘 ∈ Φ Γ ′ ⊢ (cid:174) 𝑋 𝑘 , (cid:174) 𝜏 𝑘 ▷ Γ ′ Let W be given such that Δ ; Φ | = W . We must show that it is not the case that (cid:174) 𝐹 𝑖 ; W ⊢ 𝐻 → ⊥ ,i.e., Clause-E1 and Clause-E2 cannot apply. We can rule out Clause-E1 by Lemma E.4(1: it is notthe case that a typesafe premise steps to ⊥ . To rule out Clause-E2, we need to know that if we canbuild a final substitution, i.e.: · ⊢ 𝑃 → 𝜃 . . . 𝜃 ℓ ⊢ 𝑃 ℓ → 𝜃 ℓ + . . . 𝜃 𝑝 ⊢ 𝑃 𝑝 → 𝜃 then (cid:174) 𝑋 𝑘 ∈ dom ( 𝜃 ) . We know that (cid:174) 𝑋 𝑘 ⊆ dom ( Γ ′ ) by Lemma E.5 on Γ ′ ⊢ (cid:174) 𝑋 𝑘 , (cid:174) 𝜏 𝑘 ▷ Γ ′ ; since Γ 𝑝 ⊢ 𝑃 𝑝 ▷ Γ ′ ,we know by Lemma E.4(2) that Γ ′ | = 𝜃 . We can therefore conclude that ∀ 𝑋 ∈ dom ( Γ ′ ) , 𝑋 ∈ dom ( 𝜃 ) ,and so (cid:174) 𝑋 𝑘 ∈ dom ( 𝜃 ) ... and Clause-E2 cannot apply. □ Theorem E.7 (Program preservation). If Δ ; Φ ⊢ (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 and Δ ; Φ | = W and (cid:174) 𝐹 𝑖 ; W ⊢ 𝐻 → W ′ for some 𝐻 ∈ (cid:174) 𝐻 𝑗 then Δ ; Φ | = W ′ . Proof. The program prog = (cid:174) 𝐹 𝑖 (cid:174) 𝐻 𝑗 must have been well typed according to prog -WF, and so wehave ⊢ Δ and ⊢ Φ along with derivations for each 𝐹 and 𝐻 : Δ ; Φ ⊢ 𝐹 . . . Δ ; Φ ⊢ 𝐹 𝑖 . . . Δ ; Φ ⊢ 𝐹 𝑛 Δ ; Φ ⊢ 𝐻 . . . Δ ; Φ ⊢ 𝐻 𝑗 . . . Δ ; Φ ⊢ 𝐻 𝑚 Let an 𝐻 = 𝑝 ( 𝑋 𝑘 ) : − (cid:174) 𝑃 ℓ ∈ (cid:174) 𝐻 𝑗 be given. We know that Δ ; Φ ⊢ 𝐻 by 𝐻 -Clause, i.e.: · ⊢ 𝑃 ▷ Γ . . . Γ ℓ ⊢ 𝑃 ℓ ▷ Γ ℓ + . . . Γ 𝑝 ⊢ 𝑃 𝑝 ▷ Γ ′ 𝑝 ⊆ (cid:174) 𝜏 𝑘 ∈ Φ Γ ′ ⊢ (cid:174) 𝑋 𝑘 , (cid:174) 𝜏 𝑘 ▷ Γ ′ Let W be given such that Δ ; Φ | = W . It must have been the case that we stepped by Clause, andso: · ⊢ 𝑃 → 𝜃 . . . 𝜃 𝑖 ⊢ 𝑃 𝑖 → 𝜃 𝑖 + . . . 𝜃 𝑛 ⊢ 𝑃 𝑛 → 𝜃 W ′ = W [ 𝑝 ↦→ W ( 𝑝 ) ∪ 𝜃 ( (cid:174) 𝑋 𝑗 )] By Lemma E.4(2), we know that Γ 𝑖 | = 𝜃 𝑖 and Γ ′ | = 𝜃 . We have (cid:174) 𝑋 𝑘 ⊆ dom ( Γ ′ ) by Lemma E.5 on Γ ′ ⊢ (cid:174) 𝑋 𝑘 , (cid:174) 𝜏 𝑘 ▷ Γ ′ , we can conclude that (cid:174) 𝑋 𝑘 ⊆ dom ( 𝜃 ) and that Δ ; Φ ; · ⊢ 𝜃 ( 𝑋 𝑘 ) : 𝜏 𝑘 by Lemma E.3 on Γ ′ ⊢ (cid:174) 𝑋 𝑘 , (cid:174) 𝜏 𝑘 ▷ Γ ′ .To see that Δ ; Φ | = W , we need to see that adding 𝜃 ( (cid:174) 𝑋 𝑘 ) to W ( 𝑝 ) is safe. We already knew that 𝑝 ⊆ (cid:174) 𝜏 𝑘 ∈ Φ and 𝑝 ∈ dom (W) ; we have 𝑘 = 𝑘 immediately, and we have seen that each 𝜃 ( 𝑋 𝑘 ) iswell typed at 𝜏 𝑘 . □ F MODEL-THEORETIC SEMANTICS
We have focused on the operational semantics of Formulog, as it helps us to reason about typesafety. However, since we have kept Formulog close to Datalog, it is also possible to give a model-theoretic semantics to a Formulog program. First, all ML functions and expressions are desugaredinto Datalog rules; this translation is relatively straightforward, with the trickiest part being thetranslation of non-mutually exclusive patterns occurring in match expressions. For each primitiveoperator, we introduce a (possibly infinite) EDB relation that defines that operator; for example,the addition operator + is represented through a ternary relation add( 𝑥 , 𝑦 , 𝑧 ) , which states that 𝑧 is the sum of 𝑥 and 𝑦 . Terms of the form 𝑝 ( 𝑤 ∗ ) (i.e., invocations of predicates as functions) aretranslated to aggregate predicates. Formulog requires the use of these terms, as well as negation,to be stratified; thus, the program resulting from the translation can be given a perfect model semantics in line with stratified negation [Apt et al. 1988; Przymusinski 1988; Van Gelder 1989]and stratified aggregation [Mumick et al. 1990].For a small example, consider this Formulog program: fun length ( Xs : 'a list ) : bv [32] =match Xs with| [] => 0| _ :: T => 1 + length (T)endok :- length ([1 , 2, 3]) = 3. This would be translated into a program like this: length ([] , 0).length (_ :: T , Z) :-length (T , L),add (1 , L , Z ).ok :- length ([1 , 2, 3] , 3).
Note that the rules defining the length predicate violate the range restriction, and in fact definean infinite relation. This does not pose a fundamental problem to the model theory. To make thisprogram evaluable, we could rewrite the lengthlength