Bounded Model Checking for Hyperproperties
BBounded Model Checking for Hyperproperties
Tzu-Han Hsu , César Sánchez , and Borzoo Bonakdarpour Department of Computer Science and EngineeringMichigan State University, USA {tzuhan,borzoo}@msu.edu IMDEA Software Institute, Spain [email protected]
Abstract.
Hyperproperties are properties of systems that relate multi-ple computation traces, including security properties and properties inconcurrency. This paper introduces a bounded model checking (BMC)algorithm for hyperproperties expressed in HyperLTL, which—to thebest of our knowledge—is the first such algorithm. Just as the classicBMC technique for LTL primarily aims at finding bugs, our approachalso targets identifying counterexamples. LTL describes a property viainspecting individual traces, so BMC for LTL is reduced to SAT solving.HyperLTL allows explicit and simultaneous quantification over tracesand describes properties that involves multiple traces and, hence, ourBMC approach naturally reduces to QBF solving. We report on success-ful and efficient model checking, implemented in a tool called
HyperQube ,of a rich set of experiments on a variety of case studies, including secu-rity/privacy, concurrent data structures, and path planning in roboticsapplications.
Hyperproperties [10] have been shown to be a powerful framework for specifyingand reasoning about important classes of requirements that were not possiblewith trace-based languages such as the classic temporal logics. Examples in-clude information-flow security, consistency models in concurrent computing [5],and robustness models in cyber-physical systems [6, 33]. The temporal logic Hy-perLTL [9] extends LTL by allowing explicit and simultaneous quantificationover execution traces, describing the property of multiple traces. For example,the security policy observational determinism can be specified by the followingHyperLTL formula: ∀ π A . ∀ π B . ( o π A ↔ o π B ) W ¬ ( i π A ↔ i π B ) which stipulates that every pair of traces π A and π B have to agree on the valueof the (public) output o as long as they agree on the value of the (secret) input i , where ‘ W ’ denotes the weak until operator.There has been a recent surge of model checking techniques for HyperLTLspecifications [9, 12, 22, 24]. These approaches employ various techniques (e.g., a r X i v : . [ c s . F L ] O c t alternating automata, model counting, strategy synthesis, etc) to verify hyper-properties. However, they generally fall short in proposing an effective methodto deal with identifying bugs with respect to alternating HyperLTL formulas.Indeed, quantifier alternation has been shown to generally elevate the com-plexity class of model checking HyperLTL specifications in different shapes ofKripke structures (KS) [2, 9]. For example, consider the simple Kripke struc-ture K in Fig. 1 and HyperLTL formulas ϕ = ∀ π A . ∀ π B . ( p π A ↔ p π B ) and ϕ = ∀ π A . ∃ π B . ( p π A (cid:54) ↔ p π B ) . Proving that K (cid:54)| = ϕ (where traces for π A and π B are taken from K ) can be reduced to building the self-composition of K and applying standard LTL model checking, resulting in worst-case complexity | K | in the size of the system. On the contrary, proving that K | = ϕ is not asstraightforward. In the worst case, this requires a subset generation to encodethe existential quantifier within the Kripke structure, resulting in | K | · | K | blowup. In addition, the quantification is over traces rather than states, adding tothe complexity of reasoning. { p } s { p } s { p } s { p, halt } s { q, halt } s Fig. 1: A Kripke structure.Following the great success of bounded model checking (BMC) for LTL spec-ifications [8], in this paper, we propose the first BMC algorithm for HyperLTL.To the best of our knowledge this is the first such algorithm. Just as BMC forLTL is reduced to SAT solving to search for a counterexample trace whose lengthis bounded by some integer k , we reduce BMC for HyperLTL to QBF solving tobe able to deal with quantified counterexample traces in the input model. Moreformally, given a HyperLTL formula (for example, of the form) ϕ = ∀ π A . ∃ π B .ψ and a family of Kripke structures K = ( K A , K B ) (one per trace variable), thereduction involves three main components. First, the transition relation of K π (for every π ) is represented by a Boolean encoding (cid:74) K π (cid:75) . Secondly, the innerLTL subformula ψ is translated to a Boolean fixpoint representation (cid:74) ψ (cid:75) in asimilar fashion to the standard BMC technique for LTL. This way, the QBFencoding for a bound k ≥ roughly appears as: (cid:74) K , ¬ ϕ (cid:75) k = ∃ x A . ∀ x B . (cid:74) K A (cid:75) k ∧ (cid:0) (cid:74) K B (cid:75) k ! (cid:74) ¬ ψ (cid:75) k (cid:1) (1)where the vector of Boolean variables x A (respectively, x B ) are used to representthe state and propositions of the kripke structures K A (resp. K B ) for stepsfrom to k . Formulas (cid:74) K A (cid:75) k and (cid:74) K B (cid:75) k are the unrollings K A (using x A ) and K B (using x B ), and (cid:74) ¬ ψ (cid:75) (that uses both x A and x B ) is the fixpoint Booleanencoding of ¬ ψ . We note that the proposed technique in this paper does notincorporate a loop condition, as implementing such a condition for multipletraces is not straightforward at all. This, of course, comes at the cost of lack ofa completeness result.While our QBF encoding is a natural generalization of BMC for HyperLTL,the first contribution of this paper is a more refined view of how to interpretthe behavior of the formula beyond the unrolling depth k . Consider LTL formula ∀ π. p π . BMC for LTL attempts to find a counterexample by unrolling the modeland check for satifiability of ∃ π. ¬ p π . In this case satifiability means existenceof a counterexample within the first k steps. Now consider LTL formula ∀ π. p π whose negation is of the form ∃ π. ¬ p π . In the classic BMC, due to its pessimistic handling of the unstatisfiability of the formula can not be established in thefinite unrolling (handling these formulas requires to appeal to either loopingconditions or to reach the diameter of the the system). This is because ¬ p π is not sometimes finitely satisfiable (SFS), in the terminology introduced byHavelund and Peled [27], meaning that not all satisfying traces of p π have afinite prefix that witness the satisfiability.We propose a method that allows to interpret a wide range of outcomes ofthe QBF solver and relate these to the original model checking decision problem.To this end, we propose the following semantics for BMC for HyperLTL: – Pessimistic semantics (which is the common for LTL BMC) under whichpending eventualities are considered to be unfulfilled. This semantics workfor sometime finitely satisfiable temporal formulas and paves the way for bughunting. – Optimistic semantics considers the dual case, where pending eventualitiesare assumed to be fulfilled at the end of the trace. This semantics work for sometimes finitely refutable formulas, and allows us to interpret unsatisfia-bility of QBF as proof of verification even with bounded traces. – Halting variants of the optimistic and pessimistic semantics, which allowssound and complete decision on a verdict for terminating models.We have fully implemented our technique in the tool
HyperQube . Our exper-imental evaluation includes a rich set of case studies, such as information-flowsecurity/privacy, concurrent data structures (in particular, linearizability), andpath planning in robotic applications. Our evaluation shows that our technique iseffective and efficient in identifying bugs in several prominent examples. We alsoshow that our QBF-based approach is certainly more efficient than an brute-forceSAT-based approach, where universal and existential quantifiers are eliminatedby combinatorial expansion to conjunctions and disjunctions. We also show thatin some cases our approach can also be used as as tool for synthesis. Indeed, awitness to an existential quantifier in a HyperLTL formula is an execution paththat satisfies the formula. For example, our experiments on path planning forrobots showcases this feature of
HyperQube .In summary, the contributions of this paper are as follows: – We propose a QBF-based BMC approach for verification and falsification ofHyperLTL specifications. – We introduce complementary semantics that allow proving and disprovingformulas, given a finite set of finite traces. – We rigorously analyze the performance of our technique by case studies fromdifferent areas of computing.The rest of the paper is structured as follows. Section 2 contains the pre-liminaries. Section 3 introduces the different bounded semantics for HyperLTL.Section 4 presents the encoding into QBF of the different formulas and boundedsemantics of choice, and what can be inferred in each case about the HyperLTLmodel checking problem in each case. Sections 5 and 6 present an empirical eval-uation of our tool
HyperQube . Section 7 presents the related work and Section 8concludes.
Let AP be a finite set of atomic propositions and Σ = 2 AP be the alphabet . A letter is an element of Σ . A trace t ∈ Σ ω over alphabet Σ is an infinite sequenceof letters: t = t (0) t (1) t (2) · · · Definition 1. A Kripke structure is a tuple K = (cid:104) S, S init , δ, L (cid:105) , where – S is a finite set of states ; – S init ⊆ S is the set of initial states ; – δ ⊆ S × S is a transition relation , and – L : S ! Σ is a labeling function on the states of K .We require that for each s ∈ S , there exists s (cid:48) ∈ S , such that ( s, s (cid:48) ) ∈ δ . Fig. 1 shows a Kripke structure, where S init = { s } , L ( s ) = { p } , L ( s ) = { q, halt } , etc.The size of the Kripke structure is the number of its states.A loop in K is a finite sequence s (0) s (1) · · · s ( n ) , such that ( s ( i ) , s ( i + 1)) ∈ δ ,for all ≤ i < n , and ( s ( n ) , s (0)) ∈ δ . We call a Kripke frame acyclic , if theonly loops are self-loops on otherwise terminal states, i.e., on states that haveno other outgoing transition. Since Definition 1 does not allow terminal states,we only consider acyclic Kripke structures with such added self-loops. We alsolabel such states by atomic proposition halt .A path of a Kripke structure is an infinite sequence of states s (0) s (1) · · · ∈ S ω ,such that: – s (0) ∈ S init , and – ( s ( i ) , s ( i + 1)) ∈ δ , for all i ≥ .A trace of a Kripke structure is a trace t (0) t (1) t (2) · · · ∈ Σ ω , such that thereexists a path s (0) s (1) · · · ∈ S ω with t ( i ) = L ( s ( i )) for all i ≥ . We denote by Traces ( K, s ) the set of all traces of K with paths that start in state s ∈ S , anduse Traces ( K ) as a short for (cid:83) s ∈ S init Traces ( K, s ) . HyperLTL [9] is an extension of the linear-time temporal logic (LTL) for hy-perproperties. The syntax of HyperLTL formulas is defined inductively by thefollowing grammar: ϕ ::= ∃ π.ϕ | ∀ π.ϕ | φφ ::= true | a π | ¬ φ | φ ∨ φ | φ ∧ φ | φ U φ | φ R φ | φ where a ∈ AP is an atomic proposition and π is a trace variable from an infi-nite supply of variables V . The Boolean connectives ¬ , ∨ and ∧ have the usualmeaning, U is the temporal until operator, R is the temporal release operator,and is the temporal next operator. We also consider other derived Booleanconnectives, such as ! , and ↔ , and the derived temporal operators eventually ϕ ≡ true U ϕ and globally ϕ ≡ ¬ ¬ ϕ . Even though the set of operatorspresented is not minimal, we have introduced this set to uniform the treatmentwith the variants in Section 3. The quantified formulas ∃ π and ∀ π are read as“along some trace π ” and “along all traces π ”, respectively. A formula is closed (i.e., a sentence ) if all trace variables used in the formula are quantified. Weassumed, without lost of generality that no variable is quantified twice. We use Vars ( ϕ ) for the set of path variables used in formula ϕ . Semantics.
An interpretation T = (cid:104) T π (cid:105) π ∈ Vars ( ϕ ) of a formula ϕ consists of a setof traces, one set T π per trace variable π in Vars ( ϕ ) . We use T π for the set oftraces assigned to π . The idea here is to allow quantifiers to range over differentmodels. We will use this feature in the verification of hyperproperties such aslinearizabiliity, where different quantifiers are associated with different sets ofexecutions (in this case one for the concurrent implementation and one for thesequential implementation). That is, each set of traces comes from a Kripkestructure and we use K = (cid:104) K π (cid:105) π ∈ Vars ( ϕ ) to denote a family of Kripke structure,so T π = Traces ( K π ) is the traces that π can range over, which comes from K π .Abusing notation, we write T = Traces ( K ) .Note that all trace sets being the same set of traces for a single Kripkestructure K (i.e. K π = K for all π ) is a particular case, which leads to theoriginal semantics of HyperLTL [9]. The semantics of HyperLTL are defined withrespect to a trace assignment, which is a partial map Π : Vars ( ϕ ) (cid:42) Σ ω . Theassignment with empty domain is denoted by Π ∅ . Given a trace assignment Π ,a trace variable π , and a concrete trace t ∈ Σ ω , we denote by Π [ π ! t ] theassignment that coincides with Π everywhere but at π , which is mapped totrace t .The satisfaction of a HyperLTL formula ϕ is a binary relation | = that asso-ciates a formula to the models ( T , Π, i ) where i ∈ Z ≥ is a pointer that indicates the current position of all traces in T . The semantics is defined as follows: ( T , Π, | = ∃ π. ψ iff there is a t ∈ T π such that ( T , Π [ π ! t ] , | = ψ, ( T , Π, | = ∀ π. ψ iff for all t ∈ T π such that ( T , Π [ π ! t ] , | = ψ, ( T , Π, i ) | = true ( T , Π, i ) | = a π iff a ∈ Π ( π )( i ) , ( T , Π, i ) | = ¬ ψ iff ( T , Π, i ) (cid:54)| = ψ, ( T , Π, i ) | = ψ ∨ ψ iff ( T , Π, i ) | = ψ or ( T , Π, i ) | = ψ , ( T , Π, i ) | = ψ ∧ ψ iff ( T , Π, i ) | = ψ and ( T , Π, i ) | = ψ , ( T , Π, i ) | = ψ iff ( T , Π, i + 1) | = ψ, ( T , Π, i ) | = ψ U ψ iff there is a j ≥ i for which ( T , Π, j ) | = ψ andfor all k ∈ [ i, j ) , ( T , Π, k ) | = ψ , ( T , Π, i ) | = ψ R ψ iff either for all j ≥ i, ( T , Π, j ) | = ψ , or,for some j ≥ i, ( T , Π, j ) | = ψ andfor all k ∈ [ i, j ] : ( T , Π, k ) | = ψ . We say that an interpretation T satisfies a sentence ϕ , denoted by T | = ϕ , if ( T , Π ∅ , | = ϕ . We say that a family of Kripke structures K satisfies asentence ϕ , denoted by K | = ϕ , if (cid:104) Traces ( K π ) (cid:105) π ∈ Vars ( ϕ ) | = ϕ . When the samekripke structure K is used for all path variables we write K | = ϕ .For example, the Kripke structure in Fig. 1 satisfies HyperLTL formula ϕ = ∀ π A . ∃ π B . ( p π A ↔ q π B ) .These semantics are slightly different from the definition in [9], but equiv-alent. First, we use the pointer i instead of chopping the trace with the headelements when traversing the tuple of traces forward, but this is clearly equiv-alent and more convenient later in the paper when we define finite unrollings.Second, we use a multi-model semantics allowing different trace variables tochoose traces from different trace sets. In terms of the model checking problem,multi-model and (the conventional) single-model semantics [9] are equivalent.One can instantiate all models with the same Kripke structure (so multi-modelcan simulate single-model). For the other direction, one can merge all Kripkestructures into a single Kripke structure and add fresh predicates to distinguisheach Kripke structure, and then require each path to belong to the desired orig-inal Kripke structure. The quantified Boolean formula (QBF) satisfiability problem [25] is the following:
Given is a set of Boolean variables, { x , x , . . . , x n } , and a quantifiedBoolean formula F = Q x . Q x . . . Q n − x n − . Q n x n .ψ , where each Q i ∈{∀ , ∃} ( i ∈ [1 , n ] ) and ψ is an arbitrary Boolean formula over variables { x , . . . , x n } . Is F true? Solving the satisfiability problem for QBF is known to be PSPACE-complete.Figure 2 shows a satisfying model for the following formula: F = ∃ x . ∀ x . ∃ x . ∃ x . ∀ x . ( x ∨ ¬ x ∨ x ) ∧ ( ¬ x ∨ x ∨ ¬ x ) ∧ ( ¬ x ∨ x ∨ ¬ x ) ∧ ( x ∨ x ∨ x ) . T T T TT TTF FFF FT T T x x x x x x x x Fig. 2: Model for the QBF formula.
In this section, we introduce the bounded semantics of HyperLTL, which will belater used in Section 4 to generate queries to a QBF solver to aid solving themodel checking problem.
We assume the formula is closed and of the form: Q A π A . Q B π B . . . Q Z π Z .ψ where Q ∈ {∀ , ∃} and it has been converted into negation-normal form (NNF)so that the negation symbol only appears in front of atomic propositions, e.g., ¬ a π A . Without loss of generality and for the sake of clarity from other numericalindices, we use roman alphabet as indices of trace variables. Thus, we assumethat Vars ( ϕ ) ⊆ { π A , π B , . . . , π Z } . The main idea of bounded model checking isto perform incremental exploration of the state space of the systems by unrollingthe systems and the formula up-to a bound. Let k ≥ be the unrolling bound and let T = (cid:104) T A . . . T Z (cid:105) be a tuple of finite sets of finite traces, one per tracevariable. We start by defining a satisfaction relation between HyperLTL formulasfor a bounded exploration k and models ( T , Π, i ) , where T is the tuple of set oftraces, Π is a trace assignment mapping (as defined in Section 2), and i ∈ Z ≥ that points to the position of traces. We will define different finite satisfactionrelations for general models (for ∗ = pes , opt , hpes , hopt ): – | = ∗ k , the common satisfaction relation among all semantics, – | = pes k , called pessimistic semantics, – | = opt k , called optimistic semantics, and – | = hpes k and | = hopt k , variants of | = pes k and | = opt k , respectively, for Kripke struc-tures that encode termination of traces (modeled as self-loops to provideinfinite traces).All these semantics coincide in the interpretation of quantifiers, Boolean con-nectives, and in the interpretation of the temporal operators up-to instant k − ,but differ in their assumptions about unseen future events after the bound ofobservation k . Quantifiers.
The satisfaction relation for the quantifiers is the following: ( T , Π, | = ∗ k ∃ π. ψ iff there is a t ∈ T π : ( T , Π [ π ! t ] , | = k ψ, (1)( T , Π, | = ∗ k ∀ π. ψ iff for all t ∈ T π : ( T , Π [ π ! t ] , | = k ψ. (2) Boolean operators.
For every i ≤ k , we have: ( T , Π, i ) | = ∗ k true (3)( T , Π, i ) | = ∗ k a π iff a ∈ Π ( π )( i ) , (4)( T , Π, i ) | = ∗ k ¬ a π iff a (cid:54)∈ Π ( π )( i ) , (5)( T , Π, i ) | = ∗ k ψ ∨ ψ iff ( T , Π, i ) | = k ψ or ( T , Π, i ) | = k ψ , (6)( T , Π, i ) | = ∗ k ψ ∧ ψ iff ( T , Π, i ) | = k ψ and ( T , Π, i ) | = k ψ (7) Temporal connectives.
The case where ( i < k ) is common between the opti-mistic and pessimistic semantics: ( T , Π, i ) | = ∗ k ψ iff ( T , Π, i + 1) | = k ψ (8)( T , Π, i ) | = ∗ k ψ U ψ iff ( T , Π, i ) | = k ψ , or ( T , Π, i ) | = k ψ and ( T , Π, i + 1) | = k ψ U ψ (9)( T , Π, i ) | = ∗ k ψ R ψ iff ( T , Π, i ) | = k ψ , and ( T , Π, i ) | = k ψ or ( T , Π, i + 1) | = k ψ R ψ (10) For ( i = k ) , in the pessimistic semantics the eventualities (including ) areassumed to never be fulfilled in the future, so the current instant k is the lastchance: ( T , Π, i ) | = pes k ψ iff never happens ( P )( T , Π, i ) | = pes k ψ U ψ iff ( T , Π, i ) | = pes k ψ ( P )( T , Π, i ) | = pes k ψ R ψ iff ( T , Π, i ) | = pes k ψ ∧ ψ ( P ) On the other hand, in the optimistic semantics the eventualities are assumed tobe fulfilled in the future: ( T , Π, i ) | = opt k ψ iff always happens ( O )( T , Π, i ) | = opt k ψ U ψ iff ( T , Π, i ) | = opt k ψ ∨ ψ ( O )( T , Π, i ) | = opt k ψ R ψ iff ( T , Π, i ) | = opt k ψ ( O ) In order to capture the halting semantics, we assume that the Kripke struc-ture is equipped with a predicate halt that is true if the state corresponds to ahalting state, and define the auxiliary predicate halted def = (cid:86) π Vars ( ϕ ) halt π thatholds whenever all traces have halted (and their final state will be repeated adinfinitum), where halt is an atomic proposition denoting the termination of atrace. Then, the halted semantics of the temporal case for i = k in the pes-simistic case consider the halting case to infer the actual value of the temporaloperators on the (now fully known) trace: ( T , Π, i ) | = hpes k ψ iff ( T , Π, i ) | = ∗ k halted and ( T , Π, i ) | = hpes k ψ ( HP )( T , Π, i ) | = hpes k ψ U ψ iff ( T , Π, i ) | = hpes k ψ ( HP )( T , Π, i ) | = hpes k ψ R ψ iff ( T , Π, i ) | = hpes k ψ ∧ ψ , or ( T , Π, i ) | = ∗ k halted and ( T , Π, i ) | = hpes k ψ ( HP ) Dually, in the halting optimistic case: ( T , Π, i ) | = hopt k ψ iff ( T , Π, i ) (cid:54)| = ∗ k halted or ( T , Π, i ) | = hopt k ψ ( HO )( T , Π, i ) | = hopt k ψ U ψ iff ( T , Π, i ) | = hopt k ψ , or ( T , Π, i ) (cid:54)| = ∗ k halted and ( T , Π, i ) | = hopt k ψ ( HO )( T , Π, i ) | = hopt k ψ R ψ iff ( T , Π, i ) | = hpes k ψ ( HO ) Complete semantics.
We are now ready to define the four semantics: − The pessimistic semantics | = pes k is comprised of rules (1) - (10) and ( P ) - ( P ) . − The optimistic semantics | = opt k consists of rules (1) - (10) and ( O ) - ( O ) . − The halting pessimistic semantics | = hpes k use rules (1) - (10) and ( HP ) - ( HP ) . − The halting optimistic semantics | = hopt k use rules (1) - (10) and ( HO ) - ( HO ) . Observe that the pessimistic semantics is the semantics in the traditional BMCfor LTL, where pending eventualities are considered to be unfulfilled. In thepessimistic semantics a formula is declared false unless it is witnessed to betrue within the bound explored. In other words, formulas can only get “truer”with more information obtained by a longer unrolling. Dually, the optimisticsemantics considers a formula true unless there is evidence within the boundedexploration on the contrary. Therefore, formulas only get “falser” with furtherunrolling. For example, formula p always evaluates to false in the pessimisticsemantics. In the optimistic semantics, it evaluates to true upto bound k if p holds in all states of the trace upto and including k . However, if the formulaevaluates to false at some point before k , then it evaluates to false for all j ≥ k .The following lemma formalizes this intuition in HyperLTL. Lemma 1.
Let k ≤ j . Then,1. If ( T , Π, | = pes k ϕ , then ( T , Π, | = pes j ϕ .2. If ( T , Π, (cid:54)| = opt k ϕ , then ( T , Π, (cid:54)| = opt j ϕ .3. If ( T , Π, | = hpes k ϕ , then ( T , Π, | = hpes j ϕ .4. If ( T , Π, (cid:54)| = hopt k ϕ , then ( T , Π, (cid:54)| = hopt j ϕ . In turn, the verdict obtained from the exploration up-to k can (in some cases)be used to infer the verdict of the model checking problem. As in classical BMC,if the pessimistic semantics find a model, then it is indeed a model. Similarly, ifthe optimistic semantics fail to find a model, then there is no model. The nextlemma formally captures this intuition. Lemma 2 (Infinite inference).
The following hold for every k ,1. If ( T , Π, | = pes k ϕ , then ( T , Π, | = ϕ .2. If ( T , Π, (cid:54)| = opt k ϕ , then ( T , Π, (cid:54)| = ϕ .3. If ( T , Π, | = hpes k ϕ , then ( T , Π, | = ϕ .4. If ( T , Π, (cid:54)| = hopt k ϕ , then ( T , Π, (cid:54)| = ϕ . Consider the Kripke structure in Fig. 1, bound k = 3 , and formula ϕ = ∀ π A . ∃ π B . (cid:0) ( p π A (cid:54) ↔ p π B ) R ¬ q π A (cid:1) It is easy to see that instantiating π A with trace s s s s is a trace π A of thenegation, ¬ ϕ as follows, in the pessimistic semantics. ¬ ϕ = ∃ π A . ∀ π B . (cid:0) ( a π A ↔ p π B ) U q π A (cid:1) By Lemma 2, this counterexample shows that the kripke structure is a model of ¬ ϕ in the infinite semantics as well. That is, K | = pes ¬ ϕ and, hence, K | = ¬ ϕ ,so K (cid:54)| = ϕ .Consider again the same Kripke structure, bound k = 3 , and formula ϕ = ∀ π A . ∃ π B . ( p π A ↔ q π B ) To disprove ϕ , we need to find a trace π A such that for all other π B , proposition q in π B always disagrees with p in π A , as the following formula, ¬ ϕ = ∃ π A . ∀ π B . ( p π A (cid:54) ↔ q π B ) It is straightforward to observe that such a trace π A does not exist. By Lemma 2,proving the formula is not satisfiable upto bound 3 in the optimistic semanticsimplies that K is not a model of ¬ ϕ in the infinite semantics. That is, K (cid:54)| = opt ¬ ϕ implies K (cid:54)| = ¬ ϕ . Hence, we conclude K | = ϕ . Consider again the same Kripke structure which has two terminating states, s and s , labeled by atomic proposition halt with only a self-loop. Let k = 3 ,and formula, ϕ = ∀ π A . ∃ π B . ( ¬ q π B U ¬ p π A ) To disprove, we want to find a trace π A that fulfills the negation, ¬ ϕ = ∃ π A . ∀ π B . ( q π B R p π A ) Take the halting state in to consideration, s s s is a trace of the form { p } ω . Itsatisfies the halting optimistic semantic of R in s because of the halting condi-tion. By Lemma 2, the fulfillment of formula implies that in infinite semantics itwill be fulfilled as well. That is, K | = hpes ¬ ϕ implies K | = ¬ ϕ . Hence, K (cid:54)| = ϕ .Consider again the same Kripke structure with halting states and formula, ϕ = ∀ π A . ∃ π B . ( p π A (cid:54) ↔ p π B ) A counterexample is an instantiation of π A such that for all π B , both traces willalways eventually agree on p as follows, ¬ ϕ = ∃ π A . ∀ π B . ( p π A ↔ p π B ) Trace s s s s is of the form { p }{ p }{ p }{ r, halt } ω with k = 3 . This trace neveragrees with a trace that ends in state s (which is of the form { p } ω ) and viceversa. By Lemma 2, the absence of counterexample upto bound 3 in the haltingoptimistic semantics implies that K is not a model of ¬ ϕ in the infinite seman-tics. That is, K (cid:54)| = hopt ¬ ϕ implies K (cid:54)| = ¬ ϕ . Hence, we conclude K | = ϕ . We describe in this section (1) how to generate a QBF query from an instanceof the model checking problem, and (2) what can be inferred in each case fromthe outcome of the QBF solver about the model checking problem.
Given a family of Kripke structures K , a HyperLTL formula ϕ , and bound k ≥ ,our goal is to construct a quantified Boolean formula (cid:74) K , ϕ (cid:75) k whose satisfiabilitycan be used to infer whether or not K | = ϕ . We first describe how to encode themodel and the formula, and then how to combine the two to generate the QBFquery. Encoding the models.
The unrolling of the transition relation of a Kripke struc-ture K A = (cid:104) S, S init , δ, L (cid:105) up to bound k is analogous to the BMC encoding forLTL [8]. First, note that the state space S can be encoded with a (logarithmic)number of bits in | S | . We introduce additional variables n , n , . . . to encode the state of the Kripke structure and use AP ∗ = AP ∪ { n , n , . . . } for the ex-tended alphabet that includes the encoding S . In this manner, the set of initialstates of a Kripke structure is a Boolean formula over AP ∗ . For example, for theKripke structure K A in Fig. 1 the set of initial states (in this case S init = { s } )corresponds to the following Boolean formula: I A := ( ¬ n ∧ ¬ n ∧ ¬ n ) ∧ p ∧ ¬ q ∧ ¬ halt assuming that ( ¬ n ∧ ¬ n ∧ ¬ n ) encodes state s (we need three bits to encodefive states.) Similarly R A is a binary relation that encodes the transition relation δ of K A (encoding the relation between a state and its successor). The encodinginto QBF works by introducing fresh Boolean variables (a new copy of AP ∗ foreach Kripke structure K A and position), and then producing a Boolean formulathat encodes the unrolling up-to k . We use x iA for the set of fresh copies ofthe variables AP ∗ of K A corresponding to position i ∈ [0 , k ] . Therefore, thereare k | x A | = k | AP ∗ A | Boolean variables to represent the unrolling of K A . We use I A ( x ) for the Boolean formula (using variables from x ) that encodes the initialstates, and R A ( x, x (cid:48) ) (for two copies of the variables x and x (cid:48) ) for the Booleanformula whether x (cid:48) encodes a successor states of x .For example, for bound k = 3 , we unroll the transition relation up-to asfollows, (cid:74) K A (cid:75) = I A ( x A ) ∧ R A ( x A , x A ) ∧ R ( x A , x A ) ∧ R ( x A , x A ) which is the Boolean formula representing valid traces of length , using fourcopies of the variables AP ∗ A that represent the Kripke structure K A . Encoding the inner LTL formula.
The idea of the construction of the innerLTL formula is analogous to the standard BMC as well, except for the choiceof different semantics described in Section 3. In particular, we introduce thefollowing inductive construction and define four different unrollings for a given k : (cid:74) · (cid:75) pes i,k , (cid:74) · (cid:75) opt i,k , (cid:74) · (cid:75) hpes i,k , and (cid:74) · (cid:75) hopt i,k . – Inductive Case : Since the semantics only differ on the temporal opera-tors at the end of the unrolling, the inductive case is common to all un-rollings and we use (cid:74) · (cid:75) ∗ i,k to mean any of the choices of semantic (for ∗ = pes , opt , hpes , hopt ). For all i ≤ k : (cid:74) p π (cid:75) ∗ k,i := p iπ (cid:74) ¬ p π (cid:75) ∗ k,i := ¬ p iπ (cid:74) ψ ∨ ψ (cid:75) ∗ k,i := (cid:74) ψ (cid:75) ∗ k,i ∨ (cid:74) ψ (cid:75) ∗ k,i (cid:74) ψ ∧ ψ (cid:75) ∗ k,i := (cid:74) ψ (cid:75) ∗ k,i ∧ (cid:74) ψ (cid:75) ∗ k,i (cid:74) ψ U ψ (cid:75) ∗ k,i := (cid:74) ψ (cid:75) ∗ k,i ∨ (cid:16) (cid:74) ψ (cid:75) ∗ k,i ∧ (cid:74) ψ U ψ (cid:75) ∗ k,i +1 (cid:17) (cid:74) ψ R ψ (cid:75) ∗ k,i := (cid:74) ψ (cid:75) ∗ k,i ∧ (cid:16) (cid:74) ψ (cid:75) ∗ k,i ∨ (cid:74) ψ R ψ (cid:75) ∗ k,i +1 (cid:17) (cid:74) ψ (cid:75) ∗ k,i := (cid:74) ψ (cid:75) ∗ k,i +1 Note that, for a given path variable π A , the atom p iπ A that results from (cid:74) p π (cid:75) ∗ k,i is one of the Boolean variables in x iA . – For the base case , the formula generate is different depending on the in-tended semantics: (cid:74) ψ (cid:75) pes k,k +1 := false (cid:74) ψ (cid:75) opt k,k +1 := true (cid:74) ψ (cid:75) hpes k,k +1 := (cid:74) halted (cid:75) hpes k,k ∧ (cid:74) ψ (cid:75) hpes k,k (cid:74) ψ (cid:75) hopt k,k +1 := (cid:74) halted (cid:75) hopt k,k ! (cid:74) ψ (cid:75) hopt k,k Note that the base case defines the value to be assumed for the formula afterthe end k of the unrolling, which is spawned in the temporal operators inthe inductive case at k . The pessimistic semantics assume the formula tobe false, and the optimistic semantics assume the formula to be true. Thehalting cases consider the case at which the traces have halted (using in thiscase the evaluation at k ) and using the unhalting choice otherwise. Combining the encodings.
Now, let ϕ be a HyperLTL formula of the form ϕ = Q A π A . Q B π B . . . . . Q Z π Z .ψ and K = (cid:104) K A , K B , . . . , K Z (cid:105) . Combining all thecomponents, the encoding of the HyperLTL BMC problem in QBF is the follow-ing (for ∗ = pes , opt , hpes , hopt ): (cid:74) K , ϕ (cid:75) ∗ k = Q A x A . Q B x B · · · . Q Z x Z (cid:16) (cid:74) K A (cid:75) k ◦ A (cid:74) K B (cid:75) k ◦ B · · · (cid:74) K Z (cid:75) k ◦ Z (cid:74) ψ (cid:75) ∗ ,k (cid:17) where (cid:74) ψ (cid:75) ∗ ,k is the choice of semantics and, ◦ j = ∧ if Q j = ∃ and ◦ j = ! if Q j = ∀ , for j ∈ Vars ( ϕ ) . Example.
Consider formula ϕ in Section 3.3, whose negation is the following: ¬ ϕ := ∃ π A . ∀ π B . (cid:0) ( p π A ↔ p π B ) U q π A (cid:1)(cid:124) (cid:123)(cid:122) (cid:125) ¬ ψ The unrolling of ¬ ψ using the pessimistic semantics is (cid:74) ¬ ψ (cid:75) pes , = (cid:74) (cid:0) ( p π A ↔ p π B ) U q π A (cid:1) (cid:75) pes , == q π A ∨ (cid:16) ( p π A ↔ p π B ) ∧ (cid:16) q π A ∨ (cid:16) ( p π A ↔ p π B ) ∧ (cid:16) q π A ∨ (cid:16) ( p π A ↔ p π B ) ∧ (cid:16) q π A (cid:17)(cid:17)(cid:17)(cid:17)(cid:17)(cid:17) Note that in the final encoding, for example the collection x A , contains allvariables of AP ∗ of K A (for example, p π A ) connecting to the corresponding valu-ation for p π A in the trace of K A at step in the unrolling of K A . In other words,the formula (cid:74) ¬ ψ (cid:75) pes , uses variables from x A , x A , x A , x A and x B , x B , x B , x B (thatis, from x A and x B ). To combine the model description with the encoding of theHyperLTL formula, we use two identical copies of the given Kripke structure torepresent different paths π A and π B on the model, denoted as K A and K B . Theresulting formula is: (cid:74) K , ¬ ϕ (cid:75) := ∃ x A . ∀ x B . (cid:0) (cid:74) K A (cid:75) ∧ ( (cid:74) K B (cid:75) ! (cid:74) ¬ ϕ (cid:75) pes , ) (cid:1) The sequence of assignment { ( ¬ n , ¬ n , ¬ n , p, ¬ q ) , ( ¬ n , ¬ n , n , p, ¬ q ) , ( ¬ n , n , ¬ n , p, ¬ q ) , ( n , ¬ n , ¬ n , ¬ p, q ) } on K A , corresponding to the trace s s s s , satisfies (cid:74) ¬ ϕ (cid:75) pes , for all traces on K B . The satisfaction results showsthat (cid:74) K , ¬ ϕ (cid:75) pes is true, indicating that a witness of violation is found. Theo-rem 1, by a successful detection of a counterexample witness, and the use of thepessimistic semantics, allows to conclude that K (cid:54)| = ϕ . (cid:117)(cid:116) Let ϕ be a closed HyperLTL formula and T = Traces ( K ) be aninterpretation. For ∗ = pes , opt , hpes , hopt, it holds that (cid:74) K , ϕ (cid:75) ∗ k is satisfiable if and only if ( T , Π ∅ , | = ∗ k ϕ. Proof (sketch).
The proof proceeds in two steps. First, let ψ be the largestquantifier-free sub-formula of ϕ . Then, every tuple of traces of length k (onefor each π ) is in one to one correspondence with the collection of variables p iπ ,that satisfies that the tuple is a model of ψ (in the choice semantics) if andonly if the corresponding assigment makes (cid:74) ψ (cid:75) ∗ . Then, the second part showsinductively in the stack of quantifiers that each subformula obtained by addinga quantifier is satisfiable if and only the semantics hold. (cid:117)(cid:116) Lemma 3, together with Lemma 2, allows to infer the outcome of the modelchecking problem from satisfying (or unsatisfying) instances of QBF queries,summarized in the following theorem.
Theorem 1.
Let ϕ be a HyperLTL formula. Then,1. For ∗ = pes , hpes , if (cid:74) K , ¬ ϕ (cid:75) ∗ k is satisfiable then K (cid:54)| = ϕ .2. For ∗ = opt , hopt , if (cid:74) K , ¬ ϕ (cid:75) ∗ k is unsatisfiable then K | = ϕ .Example. Finally, we make the connection between satisfiability of QBF andthe infinite semantics of the examples in Section 3.3 using Theorem 1. Table 1illustrates what the different semantics allows to soundly conclude.
In this section, we introduce a rich set of case studies to verify and falsify hyper-properties for different systems. These include proving symmetry of the BakeryAlgorithm mutual exclusion protocol, linearizability of the SNARK algorithm,non-interference in multi-threaded programs and fairness in non-repudiation pro-tocols. [12, 14, 30, 31] We also show to strategies can be synthesized for roboticplanning and mutation testing using our QBF encoding [15, 34]. SemanticsFormula Bound pessimistic optimistic halting ϕ k = 2 UNSAT (inconclusive) SAT (inconclusive) UNSAT (inconclusive) k = 3 SAT ( counterexample ) SAT (inconclusive) UNSAT (inconclusive) ϕ k = 2 UNSAT (inconclusive) SAT (inconclusive) UNSAT (inconclusive) k = 3 UNSAT (inconclusive) UNSAT ( proved ) UNSAT (inconclusive) ϕ k = 2 UNSAT (inconclusive) UNSAT (inconclusive) non-halted (inconclusive) k = 3 UNSAT (inconclusive) UNSAT (inconclusive) halted ( counterexample ) ϕ k = 2 UNSAT (inconclusive) UNSAT (inconclusive) non-halted (inconclusive) k = 3 UNSAT (inconclusive) UNSAT (inconclusive) halted ( proved ) Table 1: Comparison of Properties with Different Semantics
We first investigate the symmetry property in Lamport’s Bakery algorithm forenforcing mutual exclusion in a concurrent program. [12] The Bakery algorithmworks as follows. When a process p intends to enter the critical section, p drawsa “ticket” modeled by a number. When more than one process attempt to enterthe critical section, the process with the smallest ticket number enters first, whileother processes wait. In a concurrent program, it is also possible that two or moreprocesses hold tickets with same number if they drew tickets simultaneously.To solve this tie, when processes with the same ticket try to access the criticalsection, the process with smaller process ID enters first while the other processeswait. The Bakery algorithm is shown in Algorithm 1. Algorithm 1:
Bakery init(MAX/ P .ticket ... P n .ticket / P .status ... P n .status ):= 0/ 0...0/noncrit...noncrit ; while true do foreach i in 0...n do if select( P i ) then P i .ticket = MAX + 1; P i .status = waiting; else if P i .status = wait then if P i .ticket = min( P .ticket ... P n .ticket ) then P i .status = crit ; else P i .status = waiting ; end end end We are interested in studying the symmetry property, which informally statesthat no specific process has special privileges in terms of a faster access to thecritical section. We use the atomic proposition select to represent the processselected to proceed in the next state, and pause to indicate if the processes areboth not moving. Each process P n has a program counter denoted by pc ( P n ) .The symmetry property for the Bakery algorithm is formally express as follows.For all traces π A , there exists a trace π B , such that if both traces at every stepselect the next process to execute symmetrically, then the program counter ofeach process would be completely symmetric as well. For example, consider twoprocesses P and P and let trace π A select P iff trace π B selects P , and π A select P iff π B selects P . Such a dual choice of selection is presented as sym ( select π A , select π B ) . We are ready to describe the symmetry property as thefollowing HyperLTL formula: Symmetry ϕ sym = ∀ π A . ∃ π B . (cid:16) sym ( select π A , select π B ) ∧ ( pause π A = pause π B ) ∧ (cid:0) pc ( P ) π A = pc ( P ) π B (cid:1) ∧ (cid:0) pc ( P ) π A = pc ( P ) π B (cid:1)(cid:17) Next, we investigate whether the SNARK algorithm [14] satisfies the lineariz-ability property.Linearizability is a correctness property of concurrent libraries or datatypes [29].The history of the execution of a concurrent datatype, is the sequence of method invocations by the different threads and the response observed. A history is lin-earizable , if there exists a sequential order of invocations and responses, suchthat the same responses could be produced with atomic executions of the meth-ods invoked. A concurrent datatype is linearizable if all possible histories arelinearizable. In [5], the authors show that linearizability is a hyperproperty ofthe form ∀∃ , where the domain of the universal quantifier ranges over all possibleexecutions of the concurrent data structure and the domain of the existentialquantifier ranges over all possible executions of a sequential implementation ofthe data structure (or over the sequential reference implementation or declara-tive specification of the datatype). Thus, reasoning about linearizability requiresour multi-model semantics introduced in Section 2.The SNARK algorithm [14] is a concurrent implementation of a double-endedqueue data structure (the pseudo-code is shown in Algorithm 2). It uses double-compare-and-swap (DCAS) with doubly linked-list that stores values in nodeswhile each node is connected to its two neighbors, L and R . When a modificationof data happens, e.g., by invoking pushRight() or popLeft() , SNARK performs aDCAS by comparing two memory locations to decide if such modification isappropriate.We define linearizability as a hyperproperty using two different models. Let π A denote the trace variable over the traces of the concurrent program (inthis case SNARK). This program is created by allows multiple to execute eachmethod with interleavings. Let π B represents the trace variable over traces of Algorithm 2:
SNARK popRight() while true do rh = RightHat ; lh = LeftHat ; if rh ! R = rh then return "empty"; end if rh = lh then if DCAS( & RightHat, & LeftHat, rh, lh, Dummy, Dummy) then return rh ! V ; end else rhL = rh ! L; if DCAS( & RightHat, & rh ! L, rh, rhL, rhL, rh) then result = rh ! V ; rh ! R = Dummy ; return result ; end end pushRight() nd = new Node(); if nd = null then return "full"; end nd ! R = Dummy ; nd ! V = v ; while true do rh = RightHat s; rhR = rh ! R ; if rhR = rh then nd ! L = Dummy ; lh = LeftHat ; if DCAS( & RightHat, & LeftHat, rh, lh, nd, Dummy) then return success; end else nd ! L = rh ; if DCAS( & RightHat, & lh ! R, rh, rhR, nd, nd) then return success; end end the sequential implementation of a double-ended queue (i.e., the specification),where only atomic invocations are allowed. The HyperLTL formula that specifieslinearizability is: ϕ lin = ∀ π A . ∃ π B . ( history π A ↔ history π B ) We also investigate non-interference in a multi-threaded program with type sys-tem. Non-interference is a security policy that states that low-security variablesare independent from the high-security variables, thus, preserving secure infor-mation flow. Each variable is labeled as a high-variable (high security) or low-variable (low security). Non-interference requires that all information about ahigh-variable cannot be inferred by observing any the values of a low-variable.In this case study, we look at a concurrent system example from [31], whichcontains three threads α , β , and γ . The variables are assigned with differentsecurity level as follows: PIN , trigger0 , and trigger1 are as high-variables, and maintrigger , mask , and result are low-variables.Assuming that thread scheduling is fair, the program satisfies non-interference,if for all executions, there exists another execution that starts from a differenthigh-inputs (i.e., the values of PIN are not equal) and at termination point, theyare in low-equivalent states (i.e., the values of
Result are equal). Furthermore,in order to search for a witness of non-interference violation in bounded time,we also consider halting as introduced in Section 3. In this particular program,the execution terminates when the low-variable
MASK contains value zero. Thecorresponding HyperLTL formula is: NI ϕ NI = ∀ π A . ∃ π B . (cid:0) PIN π A (cid:54) = PIN π B (cid:1) ∧ (cid:16) ( ¬ halt π A ∨ ¬ halt π B ) U (cid:0) ( halt π A ∧ halt π B ) ∧ ( Result π A = Result π B ) (cid:1)(cid:17) where atomic proposition halt denotes the halting state ( MASK contains a zerobit) and by abuse of notation
PIN π (respectively, Result π ) denotes the value of PIN (respectively,
Result ) in trace π . A non-repudiation protocol consists of three parties: a message sender ( P ), a mes-sage receiver ( Q ), and a trusted third party T . In a message exchange event, themessage receiver should obtain a receipt from the sender, named non-repudiationof origin ( NRO ), and the message sender should end up having an evidencenamed non-repudiation of receipt ( NRR ). The three participants can take thefollowing actions:
Act P = { P ! Q : m, P ! T : m, P ! Q : NRO , P ! T : NRO , P : skip } Act Q = { Q ! P : NRR , Q ! T : NRR , Q : skip } Act T = { T ! P : NRR , T ! Q : NRO , T : skip } Algorithm 3:
Typed Multi-threaded Program Thread α : while mask != 0 do while trigger0 = 0 do no-op; end result = result (cid:107) mask ; // bitwise ’or’ trigger0 = 0 ; maintrigger = matintrigger + 1 ; if maintrigger = 1 then trigger1 = ; end end Thread β : while mask != 0 do while trigger1 = 0 do no-op; end result = result & ! mask ; // bitwise ’and’ trigger1 = 0 ; maintrigger = matintrigger + 1 ; if maintrigger = 1 then trigger0 = 1 ; end end Thread γ : while mask != 0 do maintrigger = 0 ; if PIN & mask = 0 then trigger0 = 1 ; else trigger1 = 1 ; end while maintrigger != 2 do no-op; end mask = mask/2 ; end trigger0 = 1 ; trigger1 = 1 ; In this case study, we evaluate two different models of trusted third partyfrom [30]. First, we pick an incorrect implementation from [30], named T incorrect ,which Q can choose not to send out NRR after receiving
NRO . We also considera correct implementation of the protocol. Both versions are show in Alg. 4. Algorithm 4:
Non-repudiation Protocol T correct : (1) skip until P ! T: m ; (2) skip until P ! T: NRO ; (3) T ! Q: m ; (4) skip until Q ! T: NRR ; (5) T ! Q: NRO ; (6) T ! P: NRR ; T incorrect : (1) skip until P ! T: m ; (2) skip until P ! T: NRO ; (3) T ! Q:m ; (4) T ! Q:NRO ; (5) skip until Q ! T: NRR s; (6) T ! P:NRR ; A fair non-repudiation protocol guarantees that two parties can exchangemessages fairly without any party being able to deny sending out evidence whilehaving received an evidence. Furthermore, we say that a trace is effective if message , NRR , and
NRO are all received. Assuming that each party will taketurns and take different actions, the fairness of non-repudiation protocol can bedefined as a hyperproperty as follows. There exists an effective trace π A , suchthat for all other traces π B , if P in both traces always take the same action while Q behave arbitrarily, or both Q take the same action and P behave arbitrarily,then for π B , eventually NRR gets received by P if and only if NRO gets receivedby Q .The complete specification for non-repudiation is the following: Fairness ϕ fair = ∃ π A . ∀ π B . ( m π A ) ∧ ( NRR π A ) ∧ ( NRO π A ) ∧ (cid:16) ( (cid:86) act ∈ Act P act π A ↔ act π B ) ! (cid:0) ( NRR π B ) ↔ ( NRO π B ) (cid:1)(cid:17) ∧ (cid:16) ( (cid:86) act ∈ Act Q act π A ↔ act π B ) ! (cid:0) ( NRR π B ) ↔ ( NRO π B ) (cid:1)(cid:17) Observe that trace π A expresses effectiveness (i.e., an honest behavior of allparties), while trace π B is a trace that behaves similarly to trace π A as far as theactions of P or Q are concerned while ensuring fair receipt of NRR and
NRO . In addition to model checking problems, inspired by the work in [34], we exploreother applications of our QBF encoding that also involve hyperproperties withquantifier alternation. One such application is searching the optimal solution forrobotic planning. For example, given a 2-D grid with an initial state and a goalstate, a shortest path from initial state to goal state is a trace π A , such that π A reaches the goal state and for all other traces π B , π B has not reached the goalstate before π A has. In other words, the shortest path is a path on the grid thatreaches the goal state before all other paths. We express this specification as thefollowing hyperproperty: Shortest Path ϕ sp = ∃ π A . ∀ π B . ( ¬ goal π B U goal π A ) where the atomic proposition goal denotes that the path has reached the goalstate.To further analyze the result, we also consider that traces halt. An optimalpath searching should terminate when the shortest path is found because whena shortest path has been discovered on the map, any further exploration will notaffect the outcome .Besides optimal solution searching, HyperLTL also allows us to specify the robustness of paths that are derived by uncertainty in robotic planning. Forexample, instead of one single initial state, we now consider a map with a set ofinitial states. We are interested in a strategy that can help all traces to reachthe goal state regardless of which initial state the path start from. The robuststrategy searching problem can be presented as follows. There exists a robustpath π A , such that for all paths π B starting from as arbitrary state from the setof initial states, π B is able to reach the goal state using the same strategy as π A . We use the proposition strategy to represent the sequence of movements thepath takes. We write the formula as follows: Robustness ϕ rb = ∃ π A . ∀ π B . ( strategy π B ↔ strategy π A ) U ( goal π A ∧ goal π B ) Another application of hyperproperty with quantifier alternation is the efficientgeneration of test suites for mutation testing. We look at the beverage machinemodel from [15]. The beverage machine has three possible inputs: request , fill ,or none . Based on the input, the machine may output coffee , tea , or none . Wealso use an atomic proposition mut to mark mutated traces, and ¬ mut for non-mutated traces. In this non-deterministic model, a potentially killable mutantis a trace (mutated) trace π A such that, for all other (non-mutated) π B , if theyhave same inputs as π A , then the outputs eventually diverge. Mutant inNon-det Model ∃ π A ∀ π B ( mut π A ∧ ¬ mut π B ) ∧ (cid:0) ( in π A ↔ in π B ) U ( out π A (cid:54) ↔ out π B ) (cid:1) We have implemented the technique described in Section 4 in a tool called
Hy-perQube . In this section, we describe this implementation and the empirical eval-uation of the case studies described in Section 5. The tool
HyperQube works asfollows. Given a transition relation, we automatically unfold it up to a givenbound k ≥ by a procedure genqbf using a home-grown tool written in Ocaml .Given the choice of the semantics (pessimistic, optimistic, h-pessimistic or h-optimistic) the unfolded transition relation is combined with the QBF encodingof the input HyperLTL formula to form a complete QBF instance which is thenbe fed to the state-of-the-art QBF solver
Quabs [28]. All experiments in thissection are run on an iMac desktop with Intel i7 CPU @3.4 GHz and 32 GB ofRAM. The off-the-self Bakery algorithm described does not satisfy the symmetry prop-erty, because when two or more process are intending to enter the critical sectionwith the same tickets number, the algorithm always gives priority to the processwith the smaller process ID. We encode the Bakery program as Boolean formulasthat encode the initial states and the transition relation. Then, we encoded thenegation of the symmetry formula: ¬ Symmetry ¬ ϕ sym = ∃ π A . ∀ π B . (cid:16) ¬ sym ( select π A , select π B ) ∨ ( pause π A (cid:54) = pause π B ) ∨ ( pc ( P ) π A (cid:54) = pc ( P ) π B ) ∨ ( pc ( P ) π A (cid:54) = pc ( P ) π B ) (cid:17) HyperQube returns SAT using the pessimistic semantics, which indicates thatthere exists a trace that satisfy ¬ ϕ sym . The returned trace represents a witnesstrace of Bakery that violates symmetry and thus falsifies the original formula ϕ sym .An observable witness within finite bound is sufficient with the pessimistic semantics to infer that all future observations are consistently indicating thegiven model does not satisfy original property. The SNARK algorithm is not linearizable, which means that there is a witnesstrace that has no sequential equivalent trace. The violation of linearizability canbe expressed as xthe negation of the original property, as follows: ¬ Linearizability ¬ ϕ lin = ∃ π A . ∀ π B . ( history π A (cid:54) ↔ history π B ) In this case,
HyperQube returns SAT using the pessimistic semantics, indicat-ing that a witness of linearizability violation has been found. Again, with theuse of pessimistic semantics, a witness of linearizability violation of length k isenough to infer that the given system does not satisfy the linearizability prop-erty. The bug we identified by using HyperQube is the same as the bug tracereported in [14] with an ad-hoc technique.
To verify non-interference, we use
HyperQube to search for a counterexampleexists. We encode the following formula: ¬ NI ¬ ϕ NI = ∃ π A . ∀ π B . (cid:0) PIN π A (cid:54) = PIN π B (cid:1) ! (cid:16) ( terminate π A ∧ terminate π B ) R (cid:0) ( ¬ terminate π A ∨ ¬ terminate π B ) ∨ ( Result π A (cid:54) = Result π B ) (cid:1)(cid:17) In this case we use halting − pessimistic to further exploit the terminatingnature of the system and, HyperQube returns SAT, indicating that there is atrace in which we can detect the difference of high-variable by observing lowvariable, that is, violating non-interference.
In order to handle fairness in non-repudiation protocols we study the negatedformula, which is in ∀∃ form against the T incorrect implementation. ¬ Fairness ¬ ϕ fair = ∀ π A . ∃ π B . ¬ (cid:0) ( m π A ) ∧ ( NRR π A ) ∧ ( NRO π A ) (cid:1) ∨ (cid:16) ( (cid:86) act ∈ Act P act π A ↔ act π B ) ∧ ¬ (cid:0) ( NRR π B ) ↔ ( NRO π B ) (cid:1)(cid:17) ∨ (cid:16) ( (cid:86) act ∈ Act Q act π A ↔ act π B ) ∧ ¬ (cid:0) ( NRR π B ) ↔ ( NRO π B ) (cid:1)(cid:17) We obtain a SAT result from
HyperQube , but since the formula passed tothe solver is ∀∃ the solver does not return an witness. Alternatively, one couldverify the protocol with respect to formula ∃ π A . ( m π A ∧ NRR π A ∧ NRO π A ) .This step was successful, meaning that an effective trace exists, meaning thatthe original SAT result implies that the protocol includes an unfair trace.We then studied the implementation named T correct in [30], where T al-ways guarantees the message exchange event is fair between the two parties.In this case, HyperQube returns UNSAT, which indicates that all traces in thecorrect system satisfies fairness in non-repudiation. In this case study, bothSAT and UNSAT results from
HyperQube can be meaningful because of theuse of halting semantics ( halting − pessimistic for falsification of T incorrect and halting − optimistic for verification of T correct ). The use of HyperQube for robotic path planning is slightly different from theabove-mentioned cases. In this case, we focus on synthesizing a qualified strat-egy that satisfies the properties described above. Thus, we enforce the originalformulas including shortest path and robustness properties directly with the mapmodel. – Shortest path.
By encoding the map grid together with ϕ sp , HyperQube returns SAT. The returned path as shown in fig. 3 represents a path thatcan reach the goal from the initial state with the least steps compared to allother paths. – Robustness Path.
Encoding the map with , ϕ rb , HyperQube again returnsSAT. This corresponds to a robust strategy, in the sense that all other robotsstarting from an arbitrary initial state will eventually reach to the goal stateby following exactly the same strategy. The result can be visualized in 4 Fig. 3: Shortest Path Fig. 4: Robust StrategyWe investigate the scalability and performance of our technique for thisparticular study, in comparison with the technique introduced in [34]. In [34],the paths for robotic planning are synthesized by unfolding the transition rela-tions and properties using python scripts, and solve satisfiability using Z3 SMTsolver [13]. The results shown in Table 2 suggest that the QBF-based approachof
HyperQube outperforms the solution in [34]—which is based on more matureSMT technology—, on several challenging robotic planning problems. As QBFsolvers improve we anticipate that
HyperQube will automatically benefit fromtheir improvements.
We also evaluated HyperQube to synthesize valid mutants for mutation testingas in [15]. We again directly apply the original formula that describes a goodmutant together with the model. In this case,
HyperQube returns SAT, indicatingthat we have successfully found a good qualified mutant. Our experiment showsthat
HyperQube is able to output a mutant with the given formula in a very shortamount of time, which provides an efficient solution for test suite generation ofmutation testing.
Table 3 shows the running times of
HyperQube in the different case studies.In Table 4, we separately address how we use Theorem 1 to infer from theoutput of
HyperQube the result of the corresponding model-checking problem.The results shown in Table 4 (
HyperQube is ca-pable of solving a variety of model checking problems for alternating HyperLTLproperties. These instances are very challenging for techniques that attempt re-duce to model-checking of LTL because due to the complexity of eliminatingthe alternation of quantifiers. Additionally, QBF solvers allow to more efficientexplore the search space than a brute-force SAT-based approach, where uni-versal and existential quantifiers are eliminated by combinatorial expansion toconjunctions and disjunctions.
HyperQube ( map size : 10 )
20 8.31 0.33
Shortest path ( map size : 20 )
40 124.66 6.41
Shortest path ( map size : 40 )
80 1093.12 72.99
Shortest path ( map size : 60 )
120 4360.75 532.11
Initial state robustness ( map size : 10 )
20 11.14 0.45
Initial state robustness ( map size : 20 )
40 49.59 2.67
Initial state robustness ( map size : 40 )
80 216.16 19.81
Table 2: Case studies results of hyperproperties for robotic planning on largermaps using
HyperQube , in comparison with the experimental results in [12]In cases
HyperQube to solvechallenging synthesis problems by leveraging the existential quantifier in a
Hy-perLTL formula as the synthesized result that satisfies the specification.
There has been a lot of recent progress in automatically verifying [12, 22–24]and monitoring [1, 5, 7, 20, 21, 26, 32] HyperLTL specifications. HyperLTL is alsosupported by a growing set of tools, including the model checker MCHyper [12,24], the satisfiability checkers EAHyper [19] and MGHyper [17], and the runtimemonitoring tool RVHyper [20].The complexity of the model checking for HyperLTL for tree-shaped, acyclic,and general graphs was rigorously investigated in [2]. The first algorithms formodel checking HyperLTL and HyperCTL ∗ using alternating automata were in-troduced in [24]. These techniques, however, were not able to deal in practicewith alternating HyperLTL formulas in a fully automated fashion. We also notethat previous approaches that reduce model checking HyperLTL—typically offormulas without quantifier alternations—to model checking LTL can use BMCin the LTL model checking phase. However, this is a completely different ap-proach than the one presented here, as these approaches simply instruct the ϕ ∀∀ ( sym pes ∀∀ ( sym
12 SAT pes ∀∀ ( sym
20 UNSAT opt ϕ sym
10 SAT pes ϕ sym
10 SAT pes ϕ sym
10 SAT pes ϕ sym
10 SAT pes ϕ lin
26 SAT pes ϕ lin
40 SAT pes incorrect ) ϕ NI
57 SAT h-pes correct) ϕ NI
57 UNSAT h-opt T incorrect ) ϕ fair
15 SAT h-pes T correct ) ϕ fair
15 UNSAT h-opt ϕ sp
20 SAT h-pes ϕ rb
20 SAT h-pes ϕ mut
20 SAT h-pes
Table 3: Performance of
HyperQube in the case studies. Column case identifiesthe artifact, and the rest of the columns represent the models, properties, numberof unrolling in BMC, semantic used for infinite inference, and the running timefor generating the query and for solving it.model checker to use a BMC after the problem has beenfully reduced to an LTLmodel checking problem while we avoid this translation.These algorithms were then extended to deal with hyperliveness and alter-nating formulas in [12] by finding a winning strategy in ∀∃ games. In this paper,we take an alternative approach by reducing the model checking problem toQBF solving, which is arguably more effective for finding bugs (in case a finitewitness exists). pessimistic ϕ sym SAT
K | = pes k ¬ ϕ thus K | = ¬ ϕ K (cid:54)| = ϕ sym ϕ lin SAT
K (cid:54)| = ϕ lin optimsitic ϕ sym UNSAT
K | = opt k ¬ ϕ thus K (cid:54)| = ¬ ϕ K | = ϕ sym h - pessimistic ϕ NI SAT
K | = hpes k ¬ ϕ thus K | = ¬ ϕ K (cid:54)| = ϕ NI ϕ fair SAT
K (cid:54)| = ϕ fair h - optimsitic ϕ NI UNSAT
K (cid:54)| = hopt k ¬ ϕ thus K (cid:54)| = ¬ ϕ K | = ϕ NI ϕ fair UNSAT
K | = ϕ fair Synthesis( pessimistic ) 5.1 ϕ sp SAT
K | = pes k ϕ thus K | = ϕ shortest path exists5.2 ϕ rb SAT robust path exists6.1 ϕ mut SAT mutant synthesized
Table 4: Mappings of cases studies and model checking problem conclusions,with different semantics used for infinite inference from Theorem 1.The satisfiability problem for HyperLTL is shown to be undecidable in generalbut decidable for the ∃ ∗ ∀ ∗ fragment and for any fragment that includes a ∀∃ quantifier alternation [16]. The hierarchy of hyperlogics beyond HyperLTL werestudied in [11]. The synthesis problem for HyperLTL has been studied in problemin [3] in the form of program repair , in [4] in the form of controller synthesis , andin [18] for the general case. In this paper, we introduced the first bounded model checking (BMC) techniquefor verification of hyperproperties expressed in HyperLTL. To this end, we pro-posed four different semantics that ensure the soundness of inferring the outcomeof the model-checking problem. To handle trace quantification in HyperLTL,we reduced the BMC problem to checking satisfiability of quantified Booleanformulas (QBF). This is analogous to the reduction of BMC for LTL to thesimple propositional satisfiability problem. We have introduced different classesof semantics, beyond the pessimistic semantics common in LTL model checking,namely optimistic semantics that allow to infer full verification by observing only a finite prefix and halting variations of these semantics that additionally exploitthe termination of the execution, when available.Through a rich set of case studies, we demonstrated the effectiveness andefficiency of our approach in verification of information-flow properties, lineariz-ability in concurrent data structures, path planning in robotics, and fairness innon-repudiation protocols.As for future work, our first step is to solve the loop condition problem.This is necessary to establish completeness conditions for BMC and can helpcover even more examples efficiently. The application of QBF-based techniquesin the framework of abstraction/refinement is another unexplored area. Successof BMC for hyperproperties inherently depends on effectiveness of QBF solvers.Even though QBF solving is not as mature as SAT/SMT solving techniques,recent breakthroughs on QBF have enabled the construction of HyperQube , andmore progress in QBF solving will improve its efficiency.
References
1. S. Agrawal and B. Bonakdarpour. Runtime verification of k -safety hyperpropertiesin HyperLTL. In Proceedings of the IEEE 29th Computer Security Foundations(CSF) , pages 239–252, 2016.2. B. Bonakdarpour and B. Finkbeiner. The complexity of monitoring hyperproper-ties. In
Proceedings of the 31st IEEE Computer Security Foundations SymposiumCSF , pages 162–174, 2018.3. B. Bonakdarpour and B. Finkbeiner. Program repair for hyperproperties. In
Proceedings of the 17th Symposium on Automated Technology for Verification andAnalysis (ATVA) , pages 423–441, 2019.4. B. Bonakdarpour and B. Finkbeiner. Controller synthesis for hyperproperties. In
Proceedings of the 33rd IEEE Computer Security Foundations Symposium (CSF) ,pages 366–379, 2020.5. B. Bonakdarpour, C. Sánchez, and G. Schneider. Monitoring hyperproperties bycombining static analysis and runtime verification. In
Proceedings of the 8th Lever-aging Applications of Formal Methods, Verification and Validation (ISoLA) , pages8–27, 2018.6. Borzoo Bonakdarpour, Pavithra Prabhakar, and César Sánchez. Model checkingtimed hyperproperties in discrete-time systems. In
Proc. of NFM’20 , volume 12229of
LNCS , pages 311–328. Springer, 2020.7. N. Brett, U. Siddique, and B. Bonakdarpour. Rewriting-based runtime verificationfor alternation-free HyperLTL. In
Proceedings of the 23rd International Conferenceon Tools and Algorithms for the Construction and Analysis of Systems (TACAS) ,pages 77–93, 2017.8. E. M. Clarke, A. Biere, R. Raimi, and Y. Zhu. Bounded model checking usingsatisfiability solving.
Formal Methods in System Design , 19(1):7–34, 2001.9. M. R. Clarkson, B. Finkbeiner, M. Koleini, K. K. Micinski, M. N. Rabe, andC. Sánchez. Temporal logics for hyperproperties. In
Proceedings of the 3rd Con-ference on Principles of Security and Trust POST , pages 265–284, 2014.10. M. R. Clarkson and F. B. Schneider. Hyperproperties.
Journal of ComputerSecurity , 18(6):1157–1210, 2010.911. N. Coenen, B. Finkbeiner, C. Hahn, and J. Hofmann. The hierarchy of hyperlogics.In
Proceedings 34th Annual ACM/IEEE Symposium on Logic in Computer Science(LICS) , pages 1–13, 2019.12. N. Coenen, B. Finkbeiner, C. Sánchez, and L. Tentrup. Verifying hyperliveness. In
Proceedings of the 31st International Conference on Computer Aided Verification(CAV) , pages 121–139, 2019.13. L. de Moura and N. Bjorner. Z3 – a tutorial. Technical report, Microsoft, 2012.14. S. Doherty, D. Detlefs, L. Groves, C. H. Flood, V. Luchangco, P. A. Martin,M. Moir, N. Shavit, and G. L. Steele Jr. DCAS is not a silver bullet for non-blocking algorithm design. In
Proceedings of the 16th Annual ACM Symposium onParallelism in Algorithms and Architectures (SPAA) , pages 216–224, 2004.15. A. Fellner, M. Tabaei Befrouei, and G. Weissenbacher. Mutation testing withhyperproperties. In
Proceedings of the 17th International Conference on SoftwareEngineering and Formal Methods (SEFM) , pages 203–221. Springer, 2019.16. B. Finkbeiner and C. Hahn. Deciding hyperproperties. In
Proceedings of the 27thInternational Conference on Concurrency Theory (CONCUR) , pages 13:1–13:14,2016.17. B. Finkbeiner, C. Hahn, and T. Hans. MGHyper: Checking satisfiability of Hyper-LTL formulas beyond the \exists ˆ*\forall ˆ* ∃ ∗ ∀ ∗ fragment. In Proceedings ofthe 16th International Symposium on Automated Technology for Verification andAnalysis (ATVA) , pages 521–527, 2018.18. B. Finkbeiner, C. Hahn, P. Lukert, M. Stenger, and L. Tentrup. Synthesis fromhyperproperties.
Acta Informatica , 57(1-2):137–163, 2020.19. B. Finkbeiner, C. Hahn, and M. Stenger. Eahyper: Satisfiability, implication, andequivalence checking of hyperproperties. In
Proceedings of the 29th InternationalConference on Computer Aided Verification (CAV) , pages 564–570, 2017.20. B. Finkbeiner, C. Hahn, M. Stenger, and L. Tentrup. RVHyper: A runtime verifi-cation tool for temporal hyperproperties. In
Proceedings of the 24th InternationalConference on Tools and Algorithms for the Construction and Analysis of Systems(TACAS) , pages 194–200, 2018.21. B. Finkbeiner, C. Hahn, M. Stenger, and L. Tentrup. Monitoring hyperproperties.
Formal Methods in System Design (FMSD) , 54(3):336–363, 2019.22. B. Finkbeiner, C. Hahn, and H. Torfah. Model checking quantitative hyperprop-erties. In
Proceedings of the 30th International Conference on Computer AidedVerification , pages 144–163, 2018.23. B. Finkbeiner, Ch. Müller, H. Seidl, and E. Zalinescu. Verifying Security Policiesin Multi-agent Workflows with Loops. In
Proceedings of the 15th ACM Conferenceon Computer and Communications Security (CCS) , 2017.24. B. Finkbeiner, M. N. Rabe, and C. Sánchez. Algorithms for model checking Hy-perLTL and HyperCTL*. In
Proceedings of the 27th International Conference onComputer Aided Verification (CAV) , pages 30–48, 2015.25. M.R. Garey and D.S. Johnson.
Computers and Intractability: A Guide to theTheory of NP-Completeness . W. H. Freeman, New York, 1979.26. C. Hahn, M. Stenger, and L. Tentrup. Constraint-based monitoring of hyperprop-erties. In
Proceedings of the 25th International Conference on Tools and Algorithmsfor the Construction and Analysis of Systems (TACAS) , pages 115–131, 2019.27. K. Havelund and D. Peled. Runtime verification: From propositional to first-ordertemporal logic. In
Proceedings of the 18th International Conference on RuntimeVerification (RV) , pages 90–112, 2018.028. J. Hecking-Harbusch and L. Tentrup. Solving QBF by abstraction. In
Proceedingsof the 9th International Symposium on Games, Automata, Logics, and FormalVerification (GandALF) , volume 277 of
EPTCS , pages 88–102, 2018.29. M. Herlihy and J. M. Wing. Linearizability: A correctness condition for concurrentobjects.
ACM Transactions on Programming Languages and Systems , 12(3):463–492, 1990.30. W. Jamroga, S. Mauw, and M. Melissen. Fairness in non-repudiation protocols. In
Proceedings of the 7th International Workshop on Security and Trust Management(STM) , volume 7170, pages 122–139. Springer, 2011.31. G. Smith and D. M. Volpano. Secure information flow in a multi-threaded im-perative language. In
Proceedings of the 25th ACM Symposium on Principles ofProgramming Languages (POPL) , pages 355–364, 1998.32. S. Stucki, C. Sánchez, G. Schneider, and B. Bonakdarpour. Graybox monitoringof hyperproperties. In
Proceedings of the 23rd International Symposium on FormalMethods (FM) , pages 406–424, 2019.33. Y. Wang, M. Zarei, B. Bonakdarpour, and M. Pajic. Statistical verification of hy-perproperties for cyber-physical systems.
ACM Transactions on Embedded Com-puting systems (TECS) , 18(5s):92:1–92:23, 2019.34. S. Nalluri Y. Wang and M. Pajic. Hyperproperties for robotics: Planning viaHyperLTL. In