Program Repair for Hyperproperties
aa r X i v : . [ c s . L O ] J a n Program Repair for Hyperproperties
Borzoo Bonakdarpour and Bernd Finkbeiner Iowa State University, USA, [email protected] Saarland University, Germany, [email protected]
Abstract.
We study the repair problem for hyperproperties specifiedin the temporal logic HyperLTL. Hyperproperties are system propertiesthat relate multiple computation traces. This class of properties includesinformation flow policies like noninterference and observational deter-minism. The repair problem is to find, for a given Kripke structure, asubstructure that satisfies a given specification. We show that the repairproblem is decidable for HyperLTL specifications and finite-state Kripkestructures. We provide a detailed complexity analysis for different frag-ments of HyperLTL and different system types: tree-shaped, acyclic, andgeneral Kripke structures.
Information-flow security is concerned with the detection of unwanted flows ofinformation from a set of variables deemed as secrets to another set of vari-ables that are publicly observable. Information-flow security is foundational forsome of the pillars of cybersecurity such as confidentiality, secrecy, and privacy.Information-flow properties belong to the class of hyperproperties [12], whichgeneralize trace properties to sets of sets of traces. Trace properties are usuallyinsufficient, because information-flow properties relate multiple executions. Thisalso means that classic trace-based specification languages such as linear-timetemporal logic (LTL) cannot be used directly to specify information-flow proper-ties. HyperLTL [11] is an extension of LTL with trace variables and quantifiers.HyperLTL can express information-flow properties by simultaneously referringto multiple traces. For example, noninterference [28] between a secret input h and a public output o can be specified in HyperLTL by stating that, for all pairsof traces π and π ′ , if the input is the same for all input variables I except h ,then the output o must be the same at all times: ∀ π. ∀ π ′ . (cid:0) ^ i ∈ I \{ h } i π = i π ′ (cid:1) ⇒ ( o π = o π ′ )Another prominent example is generalized noninterference (GNI) [36], which canbe expressed as the following HyperLTL formula: ∀ π. ∀ π ′ . ∃ π ′′ . ( h π = h π ′′ ) ∧ ( o π ′ = o π ′′ )The existential quantifier is needed to allow for nondeterminism. Generalizednoninterference permits nondeterminism in the low-observable behavior, buttipulates that low-security outputs may not be altered by the injection of high-security inputs.There has been a lot of recent progress in automatically verifying [14, 23–25]and monitoring [2, 8, 9, 21, 22, 29, 39] HyperLTL specifications. The automatic construction of systems that satisfy a given set of information-flow properties isstill, however, in its infancy. So far, the only known approach is bounded syn-thesis [14, 19], which searches for an implementation up to a given bound onthe number of states. While there has been some success in applying boundedsynthesis to systems like the dining cryptographers [10], this approach does notyet scale to larger systems. The general synthesis problem (without the boundon the number of states) becomes undecidable as soon as the HyperLTL formulacontains two universal quantifiers [19]. A less complex type of synthesis is pro-gram repair , where, given a model K and a property ϕ , the goal is to construct amodel K ′ , such that (1) any execution of K ′ is also an execution of K , and (2) K ′ satisfies ϕ . A useful application of program repair is program sketching , wherethe developer provides a program with “holes” that are filled in by the synthesisalgorithm [38]. Filling a hole in a program sketch is a repair step that eliminatesnondeterminism. While such a repair is guaranteed to preserve trace properties,it is well known that this is not the case in the context of information-flow secu-rity policies [30]. In fact, this problem has not yet been studied in the contextof hyperproperties.In this paper, we study the problem of automated program repair of finite-state systems with respect to HyperLTL specifications. We provide a detailedanalysis of the complexity of the repair problem for different shapes of the struc-ture: we are interested in general , acyclic , and tree-shaped Kripke structures.The need for investigating the repair problem for tree-shaped and acyclic graphsstems from two reasons. First, many trace logs that can be used as a basis forexample-based synthesis [4] and repair are in the form of a simple linear collec-tion of the traces seen so far. Or, for space efficiency, the traces are organizedby common prefixes and assembled into a tree-shaped Kripke structure, or bycommon prefixes as well as suffixes assembled into an acyclic Kripke structure.The second reason is that tree-shaped and acyclic Kripke structures often occuras the natural representation of the state space of a protocol. For example, cer-tain security protocols, such as authentication and session-based protocols (e.g.,TLS, SSL, SIP) go through a finite sequence of phases , resulting in an acyclicKripke structure.Table 1 summarizes the contributions of this paper. It shows our results onthe complexity of automated program repair with respect to different fragmentsof HyperLTL. The complexities are in the size of the Kripke structure. This system complexity is the most relevant complexity in practice, because the systemtends to be much larger than the specification. Our results show that the shape ofthe Kripke structure plays a crucial role in the complexity of the repair problem: – Trees.
For trees, the complexity in the size of the Kripke structure doesnot go beyond NP . The problem for the alternation-free fragment and thefragment with one quantifier alternation where the leading quantifier is exis-2 yperLTLfragment Tree Acyclic General E ∗ L-complete (Theorem 1)
NL-complete (Theorems 5and 6)
NL-complete (Theorem 8) A ∗ NP-complete (Theorem 9) EA ∗ PSPACE ( T h e o r e m ) E ∗ A ∗ Σ p ( T h e o r e m ) AE ∗ P-complete (Theorem 2) Σ p -complete PSPACE-complete A ∗ E ∗ NP-complete (Corollary 1) (E ∗ A ∗ ) k Σ pk -complete ( k − -EXPSPACE-complete (A ∗ E ∗ ) k Σ pk +1 -complete (A ∗ E ∗ ) ∗ PSPACE (Corollary 2)
NONELEMENTARY (Corollary 3)
Table 1: Complexity of the HyperLTL repair problem in the size of the Kripkestructure, where k is the number of quantifier alternations in the formula.tential is L-complete . The problem for the fragment with one quantifier alter-nation where the leading quantifier is universal is
P-complete . The problemis
NP-complete for full HyperLTL. – Acyclic graphs.
For acyclic Kripke structures, the complexity is
NL-complete for the alternation-free fragment and the fragment with one quan-tifier alternation where the leading quantifier is existential. The complexityis in the level of the polynomial hierarchy that corresponds to the numberof quantifier alternations. – General graphs.
For general Kripke structures, the complexity is
NL-complete for the existential fragment and
NP-complete for the universal frag-ment. The complexity is
PSPACE-complete for the fragment with one quanti-fier alternation and ( k − EXPSPACE-complete in the number k of quantifieralternations.We believe that the results of this paper provide the fundamental under-standing of the repair problem for secure information flow and pave the way forfurther research on developing efficient and scalable techniques. Organization
The remainder of this paper is organized as follows. In Section 2,we review Kripke structures and HyperLTL. We present a detailed motivatingexample in Section 3. The formal statement of our repair problem is in Section 4.Section 5 presents our results on the complexity of repair for HyperLTL in thesize of tree-shaped Kripke structures. Sections 6 and 7 present the results on3he complexity of repair in acyclic and general graphs, respectively. We discussrelated work in Section 8. We conclude with a discussion of future work inSection 9. Detailed proofs are available in the full version of this paper.
Let AP be a finite set of atomic propositions and Σ = 2 AP be the alphabet . A letter is an element of Σ. A trace t ∈ Σ ω over alphabet Σ is an infinite sequenceof letters: t = t (0) t (1) t (2) . . . Definition 1. A Kripke structure is a tuple K = h S, s init , δ, L i , where – S is a finite set of states ; – s init ∈ S is the initial state ; – δ ⊆ S × S is a transition relation , and – L : S → Σ is a labeling function on the states of K .We require that for each s ∈ S , there exists s ′ ∈ S , such that ( s, s ′ ) ∈ δ . Figure 1 shows an example Kripke structure where L ( s init ) = { a } , L ( s ) = { b } , etc. The size of the Kripke structure is the number of its states. The directedgraph F = h S, δ i is called the Kripke frame of the Kripke structure K . A loop in F is a finite sequence s s · · · s n , such that ( s i , s i +1 ) ∈ δ , for all 0 ≤ i < n , and( s n , s ) ∈ δ . We call a Kripke frame acyclic , if the only loops are self-loops onotherwise terminal states, i.e., on states that have no other outgoing transition.See Fig. 1 for an example. Since Definition 1 does not allow terminal states, weonly consider acyclic Kripke structures with such added self-loops. { a } s init { a } s { b } s { b } s Fig. 1: An acyclic Kripke structure.We call a Kripke frame tree-shaped ,or, in short, a tree , if every state s has aunique state s ′ with ( s ′ , s ) ∈ δ , exceptfor the root node, which has no prede-cessor, and the leaf nodes, which, againbecause of Definition 1, additionallyhave a self-loop but no other outgoingtransitions.A path of a Kripke structureis an infinite sequence of states s (0) s (1) · · · ∈ S ω , such that: – s (0) = s init , and – ( s ( i ) , s ( i + 1)) ∈ δ , for all i ≥ t (0) t (1) t (2) · · · ∈ Σ ω , such that thereexists a path s (0) s (1) · · · ∈ S ω with t ( i ) = L ( s ( i )) for all i ≥
0. We denote by
Traces ( K , s ) the set of all traces of K with paths that start in state s ∈ S .4n some cases, the system at hand is given as a tree-shaped or acyclic Kripkestructure. Examples include session-based security protocols and space-efficientexecution logs, because trees allow us to organize the traces according to commonprefixes and acyclic graphs according to both common prefixes and commonsuffixes. HyperLTL [11] is an extension of linear-time temporal logic (LTL) for hyperprop-erties. The syntax of HyperLTL formulas is defined inductively by the followinggrammar: ϕ ::= ∃ π.ϕ | ∀ π.ϕ | φφ ::= true | a π | ¬ φ | φ ∨ φ | φ U φ | φ where a ∈ AP is an atomic proposition and π is a trace variable from an infinitesupply of variables V . The Boolean connectives ¬ and ∨ have the usual meaning, U is the temporal until operator and is the temporal next operator. Wealso consider the usual derived Boolean connectives, such as ∧ , ⇒ , and ⇔ , andthe derived temporal operators eventually ϕ ≡ true U ϕ and globally ϕ ≡¬ ¬ ϕ . The quantified formulas ∃ π and ∀ π are read as ‘along some trace π ’ and‘along all traces π ’, respectively.The semantics of HyperLTL is defined with respect to a trace assignment, apartial mapping Π : V → Σ ω . The assignment with empty domain is denoted by Π ∅ . Given a trace assignment Π , a trace variable π , and a concrete trace t ∈ Σ ω ,we denote by Π [ π → t ] the assignment that coincides with Π everywhere butat π , which is mapped to trace t . Furthermore, Π [ j, ∞ ] denotes the assignmentmapping each trace π in Π ’s domain to Π ( π )( j ) Π ( π )( j + 1) Π ( π )( j + 2) · · · . Thesatisfaction of a HyperLTL formula ϕ over a trace assignment Π and a set oftraces T ⊆ Σ ω , denoted by T, Π | = ϕ , is defined as follows: T, Π | = a π iff a ∈ Π ( π )(0) ,T, Π | = ¬ ψ iff T, Π = ψ,T, Π | = ψ ∨ ψ iff T, Π | = ψ or T, Π | = ψ ,T, Π | = ψ iff T, Π [1 , ∞ ] | = ψ,T, Π | = ψ U ψ iff ∃ i ≥ T, Π [ i, ∞ ] | = ψ ∧ ∀ j ∈ [0 , i ) : T, Π [ j, ∞ ] | = ψ ,T, Π | = ∃ π. ψ iff ∃ t ∈ T : T, Π [ π → t ] | = ψ,T, Π | = ∀ π. ψ iff ∀ t ∈ T : T, Π [ π → t ] | = ψ. We say that a set T of traces satisfies a sentence ϕ , denoted by T | = φ , if T, Π ∅ | = ϕ . If the set T is generated by a Kripke structure K , we write K | = ϕ . A real-life example that demonstrates the importance of the problem under in-vestigation in this paper is the information leak in the EDAS Conference Man-agement System , first reported in [2]. The system manages the review process void Output ( ) { bool n t f = G e t N o t i f i c a t i o n S t a t u s ( ) ; bool dec = GetDecision ( ) ; bool s e s = g e t S e s s i o n ( ) ; s t r i n g s t a t u s = i f ( n t f ) then i f ( dec ) then ” Accept ” e l s e ” Re j e c t” e l s e ” Pending” ; s t r i n g s e s s i o n = i f ( ? ) then ”Yes” e l s e ”No” P rin t ( statu s , s e s s i o n ) ; } Fig. 2: Program sketch for a conference management system.for papers submitted to conferences. Throughout this process, authors can checkon the status of their papers, but should not learn whether or not the paper hasbeen accepted until official notifications are sent out. The system is correctlyprogrammed to show status “Pending” before notification time and “Accept”or “Reject” afterwards. The leak (which has since then been fixed) occurredthrough another status display, which indicates whether or not the paper hasbeen scheduled for presentation in a session of the conference. Since only ac-cepted papers get scheduled to sessions, this allowed the authors to infer thestatus of their paper.The problem is shown in Table 2. The first two rows show the output in theweb interface for the authors regarding two papers submitted to a conferenceafter their notification, where the first paper is accepted while the second isrejected. The last two rows show two other papers where the status is pending.The internal decisions on notification ( ntf ), acceptance ( dec ), and session ( ses ),shown in the table with a gray background, are not part of the observable outputand are added for the reader’s convenience. However, by comparing the rows forthe two pending papers, the authors can observe that the Session column valuesare not the same. Thus, they can still deduce that the first paper is rejected andthe second paper is accepted.
Internal Decisions
OutputPaper ntf dec ses
Status Session foo1 true true true
Accept Yes bar1 true false false
Reject No foo2 false false false
Pending No bar2 false true true
Pending Yes
Table 2: Output with leak.
Internal Decisions
OutputPaper ntf dec ses
Status Session foo1 true true true
Accept Yes bar1 true false false
Reject No foo2 false false false
Pending No bar2 false true true
Pending No
Table 3: Output without leak.6he information leak in the EDAS system has previously been addressed byadding a monitor that detects such leaks [2, 6]. Here, we instead eliminate theleak constructively. We use program sketching [38] to automatically generate thecode of our conference manager system. A program sketch expresses the high-level structure of an implementation, but leaves “holes” in place of the low-leveldetails. In our approach, the holes in a sketch are interpreted as nondeterministicchoices. The repair eliminates nondeterministic choices in such a way that thespecification becomes satisfied.Figure 2 shows a simple sketch for the EDAS example. The hole in thesketch (line 14) is indicated by the question mark in the if statement. Thereplacement for the hole determines how the the value of the session outputin the the web interface for the authors is computed. We wish to repair thesketch so that whenever two computations both result in status = "Pending" ,the value of session is also the same. This requirement is expressed by thefollowing HyperLTL formula: ϕ = ∀ π. ∀ π ′ . (cid:16)(cid:0) ( status = "Pending" ) π ∧ ( status = "Pending" ) π ′ (cid:1) ⇒ ( session π ⇔ session π ′ ) (cid:17) In this example, an incorrect repair would be to replace the hole in line 14with ses , which would result in the output of Table 2. A correct repair wouldbe to replace the hole with the Boolean condition ntf ∧ ses , which would resultin the output of Table 3.In the rest of the paper, we formally define the repair problem and study itscomplexity for different fragments of HyperLTL. The repair problem is the following decision problem. Let K = h S, s init , δ, L i bea Kripke structure and ϕ be a closed HyperLTL formula. Does there exist aKripke structure K ′ = h S ′ , s ′ init , δ ′ , L ′ i such that: – S ′ = S , – s ′ init = s init , – δ ′ ⊆ δ , – L ′ = L , and – K ′ | = ϕ ?In other words, the goal of the repair problem is to identify a Kripke structure K ′ ,whose set of traces is a subset of the traces of K that satisfies ϕ . Note that sincethe witness to the decision problem is a Kripke structure, following Definition 1,it is implicitly implied that in K ′ , for every state s ∈ S ′ , there exists a state s ′ such that ( s, s ′ ) ∈ δ ′ . In other words, the repair does not create a deadlock state.We use the following notation to distinguish the different variations of theproblem: 7 R[Fragment, Frame Type] ,where – PR is the program repair decision problem as described above; – Fragment is one of the following for ϕ : • We use regular expressions to denote the order and pattern of repetitionof quantifiers. For example, E ∗ A ∗ -HyperLTL denotes the fragment, wherean arbitrary (possibly zero) number of existential quantifiers is followedby an arbitrary (possibly zero) number of universal quantifiers. Also, AE + -HyperLTL means a lead universal quantifier followed by one or moreexistential quantifiers. E ≤ A ∗ -HyperLTL denotes the fragment, where zeroor one existential quantifier is followed by an arbitrary number of uni-versal quantifiers. • (EA) k -HyperLTL , for k ≥
0, denotes the fragment with k alternationsand a lead existential quantifier, where k = 0 means an alternation-freeformula with only existential quantifiers; • (AE) k -HyperLTL , for k ≥
0, denotes the fragment with k alternationsand a lead universal quantifier, where k = 0 means an alternation-freeformula with only universal quantifiers, • HyperLTL is the full logic HyperLTL, and – Frame Type is either tree , acyclic , or general . In this section, we analyze the complexity of the program repair problem fortrees. This section is organized based on the rows in Table 1. We consider thefollowing three HyperLTL fragments: (1) E ∗ A ∗ , (2) AE ∗ , and (3) the full logic. E ∗ A ∗ Fragment
Our first result is that the repair problem for tree-shaped Kripke structurescan be solved in logarithmic time in the size of the Kripke structure for thefragment with only one quantifier alternation where the leading quantifier isexistential. This fragment is the least expensive to deal with in tree-shapedKripke structures and, interestingly, the complexity is the same as for the modelchecking problem [6].
Theorem 1.
PR[E ∗ A ∗ -HyperLTL, tree] is L-complete in the size of the Kripke struc-ture.Proof.
We note that the number of traces in a tree is bounded by the number ofstates, i.e., the size of the Kripke structure. The repair algorithm enumerates allpossible assignments for the existential trace quantifiers, using, for each existen-tial trace variable, a counter up to the number of traces, which requires only a8ogarithmic number of bits in size of the Kripke structure. For each such assign-ment to the existential quantifiers, the algorithm steps through the assignmentsto the universal quantifiers, which again requires only a logarithmic number ofbits in size of the Kripke structure. We consider only assignments with tracesthat have also been assigned to a existential quantifier. For each assignmentof the trace variables, we verify the formula, which can be done in logarithmicspace [6]. If the verification is affirmative for all assignments to the universalvariables, then the repair consisting of the the traces assigned to the existentialvariables satisfies the formula.In order to show completeness, we prove that the repair problem for the exis-tential fragment is
L-hard . The L -hardness for PR[E ∗ -HyperLTL, tree] and PR[A ∗ -HyperLTL, tree] follows from the L-hardness of ORD [16]. ORD is the graph-reachability problem for directed line graphs. Graph reachabilityfrom s to t can be checked with with the repair problems for ∃ π. ( s π ∧ t π )or ∀ π. ( s π ∧ t π ). ⊓⊔ AE ∗ Fragment
We now consider formulas with one quantifier alternation where the leadingquantifier is universal. The type of leading quantifier has a significant impact onthe complexity of the repair problem: the complexity jumps from
L-completeness to P-completeness , although the model checking complexity for this fragment re-mains
L-complete [6].
Theorem 2.
PR[AE ∗ -HyperLTL, tree] is P-complete in the size of the Kripke struc-ture.Proof sketch.
Membership to P can be shown by the following algorithm. For ϕ = ∀ π . ∃ π . ψ , we begin by marking all the leaves. Then, in several rounds,we go through all marked leaves v and instantiate π with the trace leading to v . We then again go through all marked leaves v and instantiate π with thetrace leading to v , and check ψ on the pair of traces. If the check is successfulfor some instantiation of π , we leave v marked, otherwise we remove the mark.When no more marks can be removed, we eliminate all branches of the tree thatare not marked. For additional existential quantifiers, the number of rounds willincrease linearly.For the lower bound, we reduce the Horn satisfiability problem, which is
P-hard , to the repair problem for AE ∗ formulas. We first transform the given Hornformula to one that every clause consists of two negative and one positive literals.We map this Horn formula to a tree-shaped Kripke structure and a constant-sizeHyperLTL formula. For example, formula ( ¬ x ∨ ¬ x ∨ f ) ∧ ( ¬ x ∨ ¬ f ∨ x ) ∧ ( ¬ x ∨ ¬ x ∨ x ) ∧ ( ¬ x ∨ ¬ x ∨ ⊥ ) is mapped to the Kripke structure in Fig. 3.The Kripke structure includes one branch for each clause of the given Hornformula, where the length of each branch is in logarithmic order of the numberof variables in the Horn formula. We use atomic propositions neg and neg to indicate negative literals and pos for the positive literal. We also include9 neg , pos }{ neg }{ pos } { neg , neg ,c }{ neg }{ neg , pos } { neg , neg ,h }{ pos } { neg , neg }{ h } b b b b b b b b b b b b b b b b Fig. 3: The Kripke structure of the Horn formula.propositions c and h to mark each clause with a bitsequence. That is, for eachclause {¬ x n ∨ ¬ x n ∨ x p } , we label states of its branch by atomic proposition neg according to the bitsequence of x n , atomic proposition neg according tothe bitsequence of x n , and atomic proposition pos according to the bitsequenceof x p . We reserve values 0 and | X | − ⊥ and ⊥ , respectively, where X is theset of variables of the Horn formula. Finally, we use the atomic proposition c to assign to each clause a number (represented as the bitsequence of valuationsof c , starting with the lowest-valued bit; the position after the highest-level bitis marked by the occurrence of atomic proposition h , which does not appearanywhere else).The HyperLTL formula enforces that (1) ⊤ is assigned to true, (2) ⊥ isassigned to false, (3) all clauses are satisfied, and (4) if a positive literal l appearson some clause in the repaired Kripke structure, then all clauses with l must bepreserved by the repair. ⊓⊔ We now turn to full HyperLTL. We first show that the repair problem is in NP . Theorem 3.
PR[HyperLTL, tree] is in NP in the size of the Kripke structure.Proof. We nondeterministically guess a solution K ′ to the repair problem. Sincedetermining whether or not K ′ | = ϕ can be solved in logarithmic space [6], therepair problem is in NP . ⊓⊔ For the lower bound, the intuition is that an additional leading universalquantifier allows us to encode full Boolean satisfiability, instead of just Hornsatisfiability as in the previous section. Interestingly, the model checking problemremains
L-complete for this fragment [6].10 heorem 4.
PR[AAE-HyperLTL, tree] is NP-hard in the size of the Kripke struc-ture.Proof sketch.
We map an instance of the 3SAT problem to a Kripke structureand a HyperLTL formula. Figure 4 shows an example, where each clause in3SAT is mapped to a distinct branch and each literal in the clause is mappedto a distinct sub-branch. We label positive and negative literals by pos and neg , respectively. Also, propositions c and h are used to mark the clauses withbitsequences in the same fashion as in the construction of proof of Theorem 2.The HyperLTL formula ϕ map ensures that (1) at least one literal in each clause istrue, (2) a literal is not assigned to two values, and (3) all clauses are preservedduring repair: ϕ map = ∀ π . ∀ π . ∃ π . " (cid:0) ¬ pos π ∨ ¬ neg π (cid:1) ∧ "(cid:16)(cid:0) c π ∧¬ c π (cid:1) U (cid:0) ¬ c π ∧ c π ∧ (( c π ↔ c π ) U h π ) (cid:1)(cid:17) ∨ ( c π ∧¬ c π ) U h π The answer to the 3SAT problem is affirmative if and only if a repair exists forthe mapped Kripke structure with respect to formula ϕ map . ⊓⊔ Corollary 1.
The following are
NP-complete in the size of the Kripke struc-ture:
PR[A ∗ E ∗ HyperLTL, tree] , PR[(EA) k -HyperLTL, tree] , PR[(AE) k -HyperLTL, tree] ,and PR[HyperLTL, tree] . We now turn to acyclic graphs. Acyclic Kripke structures are of practical interest,because certain security protocols, in particular authentication algorithms, oftenconsist of sequences of phases with no repetitions or loops. We develop ourresults first for the alternation-free fragment, then for formulas with quantifieralternation.
We start with the existential fragment. The complexity of the repair problemfor this fragment is interestingly the same as the model checking problem.
Theorem 5.
PR[E ∗ -HyperLTL, acyclic] is NL -complete in the size of the Kripkestructure.Proof. For existential formulas, the repair problem is equivalent to the modelchecking problem. A given Kripke structure satisfies the formula iff it has a re-pair. If the formula is satisfied, the repair is simply the original Kripke structure.Since the model checking problem for existential formulas over acyclic graphs is NL -complete [6, Theorem 2], the same holds for the repair problem. ⊓⊔ r { c } r { h } r { neg } v v v v v ′ { neg } v ′ v ′ v ′ v ′′ v ′′ { pos } v ′′ v ′′ { pos } v v v v v ′ { pos } v ′ v ′ v ′ v ′′ v ′′ v ′′ { neg } v ′′ Fig. 4: The Kripke structure for the 3SAT formula ( ¬ x ∨ ¬ x ∨ x ) ∧ ( x ∨ x ∨¬ x ). The truth assignment x = true , x = false , x = false , x = false rendersthe tree with white branches, i.e., the grey branches are removed during repair.We now switch to the universal fragment. Theorem 6.
PR[A ∗ -HyperLTL, acyclic] and PR[EA ∗ -HyperLTL, acyclic] is NL -complete in the size of the Kripke structure.Proof. To solve the repair problem of a HyperLTL formula ϕ = ∃ π. ∀ π . ∀ π . . . ∀ π m . ψ ( π, π , π , . . . , π m ) with at most one existential quan-tifier, which appears as the first quantifier, it suffices to find a single trace π thatsatisfies ψ ( π, π, . . . , π ): suppose there exists a repair that satisfies ϕ and that hasmore than one path, then any repair that only preserves one of these paths alsosatisfies the universal formula ϕ . For the upper bound, we nondeterministicallyguess a path for the trace π and remove all other paths. Since the length of thepath is bounded by the size of the acyclic Kripke structure, we can guess thepath using logarithmically many bits for a counter measuring the length of thepath. NL -hardness of PR[A ∗ -HyperLTL, acyclic] follows from the NL-hardness of thegraph-reachability problem for ordered graphs [34]. Ordered graphs are acyclicgraphs with a vertex numbering that is a topological sorting of the vertices. Weexpress graph reachability from vertex s to vertex t as the repair problem of theuniversal formula ∀ π. ( s π ∧ t π ). ⊓⊔ .2 Formulas with Quantifier Alternation Next, we consider formulas where the number of quantifier alternations is boundedby a constant k . We show that changing the frame structure from trees to acyclicgraphs results in a significant increase in complexity (see Table 1). The complex-ity of the repair problem is similar to the model checking problem, with the repairproblem being one level higher in the polynomial hierarchy (cf. [6]). Theorem 7.
For k ≥ , PR[(EA) k -HyperLTL, acyclic] is Σ pk -complete in the size ofthe Kripke structure. For k ≥ , PR[(AE) k -HyperLTL, acyclic] is Σ pk + -complete inthe size of the Kripke structure.Proof sketch. For the upper bound, suppose that the first quantifier is existential.Since the Kripke structure is acyclic, the length of the traces is bounded by thenumber of states. We can thus nondeterministically guess the repair and theexistentially quantified traces in polynomial time, and then verify the correctnessof the guess by model checking the remaining formula, which has k − Π pk − [6, Theorem 3]. Hence, the repair problem is in Σ pk . Analogously, if thefirst quantifier is universal, the model checking problem in Π pk and the repairproblem in Σ pk + .We establish the lower bound via a reduction from the quantified Booleanformula (QBF) satisfiability problem [27]. The Kripke structure (see Fig. 5) con-tains a path for each clause, and a separate structure that consists of a sequenceof diamond-shaped graphs, one for each variable. A path through the diamondsselects a truth value for each variable, by going right or left, respectively, at thebranching point.In our reduction, the quantifiers in the QBF instance are translated to tracequantifiers (one per alternation depth), resulting in a HyperLTL formula with k quantifier alternations and a leading existential quantifier. Note that, the out-ermost existential quantifiers are not translated to a quantifier, but instead re-solved by the repair. For this reason, it suffices to build a HyperLTL formulawith one less quantifier alternation than the original QBF instance. Also, in ourmapping, we must make sure that the clauses and the diamonds for all variablesexcept the outermost existential variables are not removed during the repair.Similar to the proof of Theorem 4, we add a counter to the clauses and add aconstraint to the HyperLTL formula that ensures that all counter values are stillpresent in the repair; for the diamonds of the variables, the valuations themselvesform such a counter, and we add a constraint that ensures that all valuationsfor the variables (except for the outermost existential variables) are still presentin the repair. ⊓⊔ Finally, Theorem 7 implies that the repair problem for acyclic Kripke struc-tures and HyperLTL formulas with an arbitrary number of quantifiers is in
PSPACE . Corollary 2.
PR[HyperLTL, acyclic] is in
PSPACE in the size of the Kripke struc-ture. d traces π ′ traces { q , p } { q , ¯ p }{ q , p } { q , ¯ p }{ q , p } { q , ¯ p } { c }{ q , p }{ q , ¯ p }{ q , p } { c }{ q , ¯ p }{ q , p }{ q , ¯ p } Fig. 5: Kripke structure for the formula y = ∃ x . ∀ x . ∃ x . ( x ∨ ¬ x ∨ x ) ∧ ( ¬ x ∨ x ∨ ¬ x ). In this section, we investigate the complexity of the repair problem for generalgraphs. We again begin with the alternation-free fragment and then continuewith formulas with quantifier alternation.
We start with the existential fragment. Similar to the case of acyclic graphs, therepair problem can be solved with a model checking algorithm.
Theorem 8.
PR[E ∗ -HyperLTL, general] is NL-complete in the size of the Kripkestructure.Proof.
Analogously to the proof of Theorem 5, we note that, for existentialformulas, the repair problem is equivalent to the model checking problem. Agiven Kripke structure satisfies the formula if and only if it has a repair. Ifthe formula is satisfied, the repair is simply the original Kripke structure. Sincethe model checking problem for existential formulas for general graphs is NL -complete [25], the same holds for the repair problem. ⊓⊔ Unlike the case of acyclic graphs, the repair problem for the universal frag-ment is more expensive, although the model checking problem is
NL-complete [6].
Theorem 9.
PR[A + -HyperLTL, general] is NP-complete in the size of the Kripkestructure. roof. For membership in NP , we nondeterministically guess a solution to therepair problem, and verify the correctness of the universally quantified Hyper-LTL formula against the solution in polynomial time in the size of the Kripkestructure. NP -hardness follows from the NP-hardness of the repair problem forLTL [5].
Next, we consider formulas where the number of quantifier alternations is boundedby a constant k . We show that changing the frame structure from acyclic to gen-eral graphs again results in a significant increase in complexity (see Table 1). Theorem 10.
PR[E ∗ A ∗ -HyperLTL, general] is in PSPACE in the size of the Kripkestructure.
PR[A ∗ E ∗ -HyperLTL, general] is PSPACE -complete in the size of the Kripkestructure. For k ≥ , PR[(EA) k -HyperLTL, general] and PR[(AE) k -HyperLTL, general] are ( k − -EXPSPACE -complete in the size of the Kripke structure.Proof idea. The claimed complexities are those of the model checking prob-lem [37]. We prove that the repair problem has the same complexity as the modelchecking problem. To show the upper bound of
PR[A ∗ E ∗ -HyperLTL, general] , weenumerate, in PSPACE , all possible repairs, and then verify against the HyperLTLformula.For the lower bounds, we modify the Kripke structure and the HyperLTLformula such that the only possible repair is the unchanged Kripke structure.After the modification, the repair problem thus has the same result as the modelchecking problem. The idea of the modification is to assign numbers to the suc-cessors of each state. We add extra states such that the traces that originate fromthese states correspond to all possible number sequences. Finally, the HyperLTLformula states that for each such number sequence there exists a correspondingtrace in the original Kripke structure. ⊓⊔ Finally, Theorem 10 implies that the repair problem for general Kripke struc-tures and HyperLTL formulas with an arbitrary number of quantifiers is in
NONELEMENTARY . Corollary 3.
PR[HyperLTL, general] is NONELEMENTARY in the size of the Kripkestructure.
There has been a lot of recent progress in automatically verifying [14, 23–25]and monitoring [2, 8, 9, 21, 22, 29, 39] HyperLTL specifications. HyperLTL is alsosupported by a growing set of tools, including the model checker MCHyper [14,25], the satisfiability checkers EAHyper [20] and MGHyper [18], and the runtimemonitoring tool RVHyper [21].Directly related to the repair problem studied in this paper are the satis-fiability and synthesis problems. The satisfiability problem for HyperLTL was15hown to be decidable for the ∃ ∗ ∀ ∗ fragment and for any fragment that includesa ∀∃ quantifier alternation [17]. The hierarchy of hyperlogics beyond HyperLTLhas been studied in [13].The synthesis problem was shown to be undecidable in general, and decid-able for the ∃ ∗ and ∃ ∗ ∀ fragments. While the synthesis problem becomes, ingeneral, undecidable as soon as there are two universal quantifiers, there is aspecial class of universal specifications, called the linear ∀ ∗ -fragment, which isstill decidable [19]. The linear ∀ ∗ -fragment corresponds to the decidable dis-tributed synthesis problems [26]. The bounded synthesis problem considers onlysystems up to a given bound on the number of states. Bounded synthesis fromhyperproperties is studied in [14, 19]. Bounded synthesis has been successfullyapplied to small examples such as the dining cryptographers [10].The problem of model checking hyperproperties for tree-shaped and acyclicgraphs was studied in [6]. Earlier, a similar study of the impact of structuralrestrictions on the complexity of the model checking problem has also beencarried out for LTL [33].For LTL, the complexity of the repair problem was studied independentlyin [5,15,32] and subsequently in [7] for distributed programs. The repair problemis also related to supervisory control , where, for a given plant, a supervisor isconstructed that selects an appropriate subset of the plant’s controllable actionsto ensure that the resulting behavior is safe [31, 35, 40]. In this paper, we have developed a detailed classification of the complexity ofthe repair problem for hyperproperties expressed in HyperLTL. We consideredgeneral, acyclic, and tree-shaped Kripke structures. We showed that for trees,the complexity of the repair problem in the size of the Kripke structure does notgo beyond NP . The problem is complete for L , P , and NP for fragments with onlyone quantifier alternation, depending upon the outermost quantifiers. For acyclicKripke structures, the complexity is in PSPACE (in the level of the polynomial hi-erarchy that corresponds to the number of quantifier alternations). The problemis
NL-complete for the alternation-free fragment. For general graphs, the prob-lem is
NONELEMENTARY for an arbitrary number of quantifier alternations. Fora bounded number k of alternations, the problem is ( k − EXPSPACE-complete .These results highlight a crucial insight to the repair problem compared to thecorresponding model checking problem [6]. With the notable exception of trees,where the complexity of repair is
NP-complete , compared to the
L-completeness of model checking, the complexities of repair and model checking are largelyaligned. This is mainly due to the fact that computing a repair can be done byidentifying a candidate substructure, which is comparatively inexpensive, andthen verifying its correctness.The work in this paper opens many new avenues for further research. Animmediate question left unanswered in this paper is the lower bound complexityfor the ∃ ∗ ∀ ∗ fragment in acyclic and general graphs. It would be interesting to see16f the differences we observed for HyperLTL carry over to other hyperlogics (cf.[1, 11, 13, 24]). One could extend the results of this paper to the reactive setting,where the program interacts with the environment. And, finally, the ideas of thispaper might help to extend popular synthesis techniques for general (infinite-state) programs, such as program sketching [38] and syntax-guided synthesis [3],to hyperproperties. Acknowledgments
We would like to thank Sandeep Kulkrani, Reza Hajisheykhi (Michigan StateUniversity), and Shreya Agrawal (Google). The repair problem was originallyinspired through our interaction with them. This work is sponsored in part byNSF SaTC Award 1813388. It was also supported by the German ResearchFoundation (DFG) as part of the Collaborative Research Center “Methods andTools for Understanding and Controlling Privacy” (CRC 1223) and the Collab-orative Research Center “Foundations of Perspicuous Software Systems” (TRR248, 389792660), and by the European Research Council (ERC) Grant OSARES(No. 683300).
References
1. E. ´Abrah´am and B. Bonakdarpour. HyperPCTL: A temporal logic for proba-bilistic hyperproperties. In
Proceedings of the 15th International Conference onQuantitative Evaluation of Systems QEST , pages 20–35, 2018.2. S. Agrawal and B. Bonakdarpour. Runtime verification of k -safety hyperpropertiesin HyperLTL. In Proceedings of the IEEE 29th Computer Security Foundations(CSF) , pages 239–252, 2016.3. R. Alur, R. Bod´ık, G. Juniwal, M. M. K. Martin, M. Raghothaman, S. A. Seshia,R. Singh, A. Solar-Lezama, E. Torlak, and A. Udupa. Syntax-guided synthesis. In
Formal Methods in Computer-Aided Design (FMCAD) , pages 1–8, 2013.4. R. Alur and S. Tripakis. Automatic synthesis of distributed protocols.
SIGACTNews , 48(1):55–90, 2017.5. B. Bonakdarpour, A. Ebnenasir, and S. S. Kulkarni. Complexity results in revis-ing UNITY programs.
ACM Transactions on Autonomous and Adaptive Systems(TAAS) , 4(1):1–28, January 2009.6. B. Bonakdarpour and B. Finkbeiner. The complexity of monitoring hyperprop-erties. In
Proceedings of the IEEE 31th Computer Security Foundations (CSF) ,pages 162–174, 2018.7. B. Bonakdarpour and S. S. Kulkarni. Masking faults while providing bounded-timephased recovery. In
International Symposium on Formal Methods (FM) , pages 374–389, 2008.8. B. Bonakdarpour, C. S´anchez, and G. Schneider. Monitoring hyperproperties bycombining static analysis and runtime verification. In
Proceedings of the 8th Lever-aging Applications of Formal Methods, Verification and Validation (ISoLA) , pages8–27, 2018. . N. Brett, U. Siddique, and B. Bonakdarpour. Rewriting-based runtime verificationfor alternation-free HyperLTL. In Proceedings of the 23rd International Conferenceon Tools and Algorithms for the Construction and Analysis of Systems (TACAS) ,pages 77–93, 2017.10. D. Chaum. Security without identification: Transaction systems to make bigbrother obsolete.
Commun. ACM , 28(10):1030–1044, 1985.11. M. R. Clarkson, B. Finkbeiner, M. Koleini, K. K. Micinski, M. N. Rabe, andC. S´anchez. Temporal logics for hyperproperties. In
Proceedings of the 3rd Con-ference on Principles of Security and Trust (POST) , pages 265–284, 2014.12. M. R. Clarkson and F. B. Schneider. Hyperproperties.
Journal of ComputerSecurity , 18(6):1157–1210, 2010.13. N. Coenen, B. Finkbeiner, C. Hahn, and J. Hofmann. The hierarchy of hyperlogics.In ,2019.14. N. Coenen, B. Finkbeiner, C. S´anchez, and L. Tentrup. Verifying hyperliveness. InIsil Dillig and Serdar Tasiran, editors,
Computer Aided Verification , pages 121–139,Cham, 2019. Springer International Publishing.15. A. Ebnenasir, S. S. Kulkarni, and B. Bonakdarpour. Revising UNITY programs:Possibilities and limitations. In
On Principles of Distributed Systems (OPODIS) ,pages 275–290, 2005.16. K Etessami. Counting quantifiers, successor relations, and logarithmic space.
Jour-nal of Computer and System Sciences , 54(3):400–411, 1997.17. B. Finkbeiner and C. Hahn. Deciding hyperproperties. In
Proceedings of the 27thInternational Conference on Concurrency Theory (CONCUR) , pages 13:1–13:14,2016.18. B. Finkbeiner, C. Hahn, and T. Hans. MGHyper: Checking satisfiability of Hyper-LTL formulas beyond the ∃ ∗ ∀ ∗ fragment. In Shuvendu K. Lahiri and Chao Wang,editors, Automated Technology for Verification and Analysis , pages 521–527, Cham,2018. Springer International Publishing.19. B. Finkbeiner, C. Hahn, P. Lukert, M. Stenger, and L. Tentrup. Synthesizingreactive systems from hyperproperties. In
Proceedings of the 30th InternationalConference on Computer Aided Verification (CAV) , pages 289–306, 2018.20. B. Finkbeiner, C. Hahn, and M. Stenger. EAHyper: Satisfiability, implication, andequivalence checking of hyperproperties. In
Proceedings of the 29th InternationalConference on Computer Aided Verification (CAV) , pages 564–570, 2017.21. B. Finkbeiner, C. Hahn, M. Stenger, and L. Tentrup. RVHyper: A runtime verifi-cation tool for temporal hyperproperties. In
Proceedings of the 24th InternationalConference on Tools and Algorithms for the Construction and Analysis of Systems(TACAS) , pages 194–200, 2018.22. B. Finkbeiner, C. Hahn, M. Stenger, and L. Tentrup. Monitoring hyperproperties.
Formal Methods in System Design , Jun 2019.23. B. Finkbeiner, C. Hahn, and H. Torfah. Model checking quantitative hyperprop-erties. In Hana Chockler and Georg Weissenbacher, editors,
Computer Aided Ver-ification , pages 144–163, Cham, 2018. Springer International Publishing.24. B. Finkbeiner, Ch. M¨uller, H. Seidl, and E. Zalinescu. Verifying Security Policiesin Multi-agent Workflows with Loops. In
Proceedings of the 15th ACM Conferenceon Computer and Communications Security (CCS) , pages 633–645, 2017.25. B. Finkbeiner, M. N. Rabe, and C. S´anchez. Algorithms for model checking Hy-perLTL and HyperCTL*. In
Proceedings of the 27th International Conference onComputer Aided Verification (CAV) , pages 30–48, 2015.
6. B. Finkbeiner and S. Schewe. Uniform distributed synthesis. In
Proceedings ofLICS , pages 321–330. IEEE Computer Society, 2005.27. M.R. Garey and D.S. Johnson.
Computers and Intractability: A Guide to theTheory of NP-Completeness . W. H. Freeman, New York, 1979.28. J. A. Goguen and J. Meseguer. Security policies and security models. In
Proceedingsof the IEEE Symposium on Security and Privacy (S & P) , pages 11–20, 1982.29. C. Hahn, M. Stenger, and L. Tentrup. Constraint-based monitoring of hyper-properties. In Tom´aˇs Vojnar and Lijun Zhang, editors,
Tools and Algorithms forthe Construction and Analysis of Systems , pages 115–131, Cham, 2019. SpringerInternational Publishing.30. J. Jacob. On the derivation of secure components. In
Proceedings of the IEEESymposium on Security and Privacy (S & P) , page 242–247, 1989.31. S. Jiang and R. Kumar. Supervisory control of discrete event systems with ctl* tem-poral logic specifications.
SIAM Journal on Control and Optimization , 44(6):2079–2103, 2006.32. B. Jobstmann, A. Griesmayer, and R. Bloem. Program repair as a game. In
Computer Aided Verification (CAV) , pages 226–238, 2005.33. L. Kuhtz and B. Finkbeiner. Weak Kripke structures and LTL. In
Proceedingsof the 22nd International Conference on Concurrency Theory (CONCUR) , pages419–433, 2011.34. T. Lengauer and K.W. Wagner. The correlation between the complexities of thenonhierarchical and hierarchical versions of graph problems.
Journal of Computerand System Sciences , 44(1):63 – 93, 1992.35. F. Lin. Analysis and synthesis of discrete event systems using temporal logic.In
Proceedings of the 1991 IEEE International Symposium on Intelligent Control ,pages 140–145, Aug 1991.36. D. McCullough. Noninterference and the composability of security properties. In
Proc. IEEE Symposium on Security and Privacy , pages 177–186, April 1988.37. M. N. Rabe.
A Temporal Logic Approach to Information-flow Control . PhD thesis,Saarland University, 2016.38. A. Solar-Lezama. Program sketching.
Springer Journal on Software Tool for Tech-nology Transfer STTT , 15(5-6):475–495, 2013.39. S. Stucki, C. S´anchez, G. Schneider, and B. Bonakdarpour. Graybox monitoringof hyperproperties. In
Proceedings of the 23rd International Symposium on FormalMethods (FM) , 2019. To appear.40. J. G. Thistle and W. M. Wonham. Control problems in a temporal logic framework.
International Journal of Control , 44(4):943–976, 1986., 44(4):943–976, 1986.