Automata for Hyperlanguages
AAutomata for Hyperlanguages Borzoo Bonakdarpour Department of Computer Science, Iowa State University, U.S.A. [email protected] Sarai Sheinvald Department of Software Engineering, ORT Braude College, Israel [email protected] Abstract
Hyperproperties lift conventional trace properties from a set of execution traces to a set of sets of executiontraces. Hyperproperties have been shown to be a powerful formalism for expressing and reasoning aboutinformation-flow security policies and important properties of cyber-physical systems such as sensitivity androbustness, as well as consistency conditions in distributed computing such as linearizability. Although thereis an extensive body of work on automata-based representation of trace properties, we currently lack suchcharacterization for hyperproperties.We introduce hyperautomata for hyperlanguages , which are languages over sets of words. Essentially,hyperautomata allow running multiple quantified words over an automaton. We propose a specific type ofhyperautomata called nondeterministic finite hyperautomata (NFH), which accept regular hyperlanguages .We demonstrate the ability of regular hyperlanguages to express hyperproperties for finite traces. We thenexplore the fundamental properties of NFH and show their closure under the Boolean operations. We showthat while nonemptiness is undecidable in general, it is decidable for several fragments of NFH. We furthershow the decidability of the membership problem for finite sets and regular languages for NFH, as well asthe containment problem for several fragments of NFH. Finally, we introduce learning algorithms based onAngluin’s L ∗ algorithm for the fragments NFH in which the quantification is either strictly universal or strictlyexistential. Author: Please fill in 1 or more \ccsdesc macro
Keywords and phrases
Author: Please fill in \keywords macro
Digital Object Identifier
Funding
Borzoo Bonakdarpour : Optional footnote, e.g. to mark corresponding author © B. Bonakdarpour and S. Sheinvald;licensed under Creative Commons License CC-BYLeibniz International Proceedings in InformaticsSchloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany a r X i v : . [ c s . F L ] F e b X:2 Automata for Hyperlanguages
Hyperproperties [7] generalize the traditional trace properties [1] to system properties , i.e., a set ofsets of traces. Put it another way, a hyperproperty prescribes how the system should behave in itsentirety and not just based on its individual executions. Hyperproperties have been shown to be apowerful tool for expressing and reasoning about information-flow security policies [7] and importantproperties of cyber-physical systems [16] such as sensitivity and robustness, as well as consistencyconditions in distributed computing such as linearizability [4].Automata theory has been in the forefront of developing techniques for specification and verifica-tion of computing systems. For instance, in the automata-theoretic approach to verification [14, 15],the model-checking problem is reduced to checking the nonemptiness of the product automaton ofthe model and the complement of the specification. In the industry and other disciplines (e.g., controltheory), automata are an appealing choice for modeling the behavior of a system. Unfortunately,we currently lack a deep understanding about the relation between hyperproperties and automatatheory. To our knowledge, work in this area is limited to [8], in which the authors develop anautomata representation for the class of regular k -safety hyperproperties. These are hyperpropertieswhere execution traces are only universally quantified and their behaviors are non-refutable. Theyintroduce the notion of a k bad-prefix automaton – a finite-word automaton that recognizes sets of k bad prefixes as finite words. Based on this representation, they present a learning algorithm for k -safety hyperproperties. In [10], the authors offer a model-checking algorithm for hyperCTL ∗ [6],which constructs an alternating Büchi automaton that has both the formula and the Kripke structure“built-in”. These approaches translate a hyperproperty-related problem to word automata.We generalize the idea in [8] to a broader view of an automata-based representation of hyperprop-erties, and introduce hyperautomata for hyperlanguages , which are languages whose elements aresets of finite words, which we call hyperwords . In this paper, we propose nondeterministic finite-wordhyperautomata (NFH). An NFH runs on hyperwords that contain finite words, by using quantified word variables that range over the words in a hyperword, and a nondeterministic finite-word auto-maton (NFA) that runs on the set of words that are assigned to the variables. We demonstrate the ideawith two examples. (cid:73) Example 1.
Consider the NFH A in Figure 1 (left), whose alphabet is Σ = { a, b } , over twoword variables x and x . The NFH A contains an underlying standard NFA, whose alphabetcomprises pairs over Σ , i.e., elements of Σ , in which the first letter represents the letters of the wordassigned to x , and dually for the second letter and x . The underlying NFA of A requires that(1) these two words agree on their a (and, consequently, on their b ) positions, and (2) once one ofthe words has ended (denoted by ), the other must only contain b letters. Since the quantificationcondition of A is ∀ x ∀ x , in a hyperword S that is accepted by A , every two words agree on their a positions. As a result, all the words in S must agree on their a positions. The hyperlanguage of A is then all hyperwords in which all words agree on their a positions. (cid:73) Example 2.
Next, consider the NFH A in Figure 1 (right), over the alphabet Σ = { a } , andtwo word variables x and x . The underlying NFA of A accepts the two words assigned to x and x iff the word assigned to x is longer than the word assigned to x . Since the quantificationcondition of A is ∀ x ∃ x , we have that A requires that for every word in a hyperword S acceptedby A , there exists a longer word in S . This holds iff S contains infinitely many words. Therefore,the hyperlanguage of A is the set of all infinite hyperwords over { a } .We call the hyperlanguages accepted by NFH regular hyperlanguages . A regular hyperlanguage L can also be expressed by the regular expression for the language of the underlying NFA of an NFH A for L , augmented with the quantification condition of A . We call such an expression a hyperregular . Bonakdarpour and S. Sheinvald XX:3 𝑎, 𝑎 ( ∀𝑥 ∃𝑥 𝑎, 𝑎 ,(𝑏, 𝑏) ( ∀𝑥 ∀𝑥 Figure 1
The NFH A (left) and A (right). expression (HRE). We demonstrate the ability of HREs to express important information-flow securitypolicies such as different variations of noninteference [11] and observational determinism [17].We proceed to conduct a comprehensive study of properties of NFH (see Table 1). In particular,we show that NFH are closed under union, intersection, and complementation. We also prove thatthe nonemptiness problem is in general undecidable for NFH. However, for the alternation-freefragments (which only allow one type of quantifier), as well as for the ∃∀ fragment (in whichthe quantification condition is limited to a sequence of ∃ quantifiers followed by a sequence of ∀ quantifiers), nonemptiness is decidable. These results are in line with the results on satisfiability ofHyperLTL [9]. We also study the membership and inclusion problems. These results are aligned withthe complexity of HyperLTL model checking for tree-shaped and general Kripke structures [3]. Thisshows that, surprisingly, the complexity results in [9, 3] mainly stem from the nature of quantificationover finite words and depend on neither the full power of the temporal operators nor the infinite natureof HyperLTL semantics.Finally, we introduce learning algorithms for the alternation-free fragments of NFH. Our al-gorithms are based on Angluin’s L ∗ algorithm [2] for regular languages, and are inspired by [8],where the authors describe a learning algorithm that is tailored to learn a k -bad prefix NFA fora k -safety formula. In fact, the algorithm there can be viewed of as a special case of learning ahyperlanguage in the ∃ -fragment of NFH.In a learning algorithm, a learner aims to construct an automaton for an unknown target language L , by means of querying a teacher , who knows L . The learner asks two types of queries: membershipqueries (“is the word w in L ?”) and equivalence queries (“is A an automaton for L ?”). In case of afailed equivalence query, the teacher returns a counterexample word on which A and L differ. Thelearning algorithm describes how the learner uses the answers it gets from the teacher to construct itscandidate automaton.In the case of NFH, the membership queries, as well as the counterexamples, are hyperwords.The number of variables is unknown in advance, and is also part of the learning goal. We first definecanonical forms for the alternation-free fragments of NFH, which is essential for this type of learningalgorithm. Then, we proceed to describe the learning algorithms for both fragments. Organization.
The rest of the paper is organized as follows. Preliminary concepts are presented inSection 2. We introduce the notion of NFH and HRE in Sections 3 and 4, while their properties arestudied in Section 5. We propose our learning algorithm in Section 6. Finally, we make concludingremarks and discuss future work in Section 7. Detailed proofs appear in the appendix. An alphabet is a nonempty finite set Σ of letters . A word over Σ is a finite sequence of letters from Σ .The empty word is denoted by (cid:15) , and the set of all finite words is denoted by Σ ∗ . A language is asubset of Σ ∗ . (cid:73) Definition 3. A nondeterministic finite-word automaton (NFA) is a tuple A = h Σ , Q, Q , δ, F i , where Σ is an alphabet, Q is a nonempty finite set of states , Q ⊆ Q is X:4 Automata for Hyperlanguages
Property Result
Closure Complementation, Union, Intersection (Theorem 8)Nonemptiness ∀∃∃
Undecidable (Theorem 9) ∃ ∗ / ∀ ∗ NL-complete (Theorem 10) ∃ ∗ ∀ ∗ PSPACE-complete (Theorem 11)Finite membership NFH
PSPACE (Theorem 12) O (log( k )) ∀ NP-complete (Theorem 12)Regular membership Decidable (Theorem 13)Containment ∃ ∗ / ∀ ∗ / ∃ ∗ ∀ ∗ ⊆ ∃ ∗ / ∀ ∗ PSPACE-complete (Theorem 14)
Table 1
Summary of results on properties of NHF. a set of initial states , F ⊆ Q is a set of accepting states , and δ ⊆ Q × Σ × Q is a transition relation . Given a word w = σ σ · · · σ n over Σ , a run of A on w is a sequence of states ( q , q , . . . q n ) ,such that q ∈ Q , and for every < i ≤ n , it holds that ( q i − , σ i , q i ) ∈ δ . The run is accepting if q n ∈ F . We say that A accepts w if there exists an accepting run of A on w . The language of A ,denoted by L ( A ) , is the set of all finite words that A accepts. A language L is called regular if thereexists an NFA such that L ( A ) = L .An NFA A is called deterministic (DFA), if for every q ∈ Q and σ ∈ Σ , there exists exactly one q for which ( q, σ, q ) ∈ δ , i.e., δ is a transition function . It is well-known that every NFA has anequivalent DFA. Before defining hyperautomata, we explain the idea behind them. We first define hyperwords andhyperlanguages. (cid:73)
Definition 4. A hyperword over Σ is a set of words over Σ and a hyperlanguage is a set ofhyperwords. A hyperautomaton A uses a set of word variables X = { x , x , . . . , x k } . When running on ahyperword S , these variables are assigned words from S . We represent an assignment v : X → S as the k -tuple ( v ( x ) , v ( x ) , . . . , v ( x k )) . Notice that the variables themselves do not appear in thisrepresentation of v , and are manifested in the order of the words in the k -tuple: the i ’th word is theone assigned to x i . This allows a cleaner representation with less notations.The hyperautomaton A consists of a quantification condition α over X , and an underlying wordautomaton ˆ A , which runs on words that represent assignments to X (we explain how we representassignments as words later on). The condition α defines the assignments that ˆ A should accept. Forexample, α = ∃ x ∀ x requires that there exists a word w ∈ S (assigned to x ), such that for everyword w ∈ S (assigned to x ), the word that represents ( w , w ) is accepted by ˆ A . The hyperword S is accepted by A iff S meets these conditions.We now elaborate on how we represent an assignment v : X → S as a word. We encode thetuple ( v ( x ) , v ( x ) , . . . v ( x k )) by a word w whose letters are k -tuples in Σ k , where the i ’th letterof w represents the k i ’th letters of the words v ( x ) , . . . , v ( x k ) (in case that the words are not ofequal length, we “pad” the end of the word with signs). For example, the assignment v ( x ) = aa, v ( x ) = abb , represented by the tuple ( aa, abb ) , is encoded by the word ( a, a )( a, b )( , b ) . Welater refer to w as the zipping of v . Once again, notice that due to the indexing of the word variables,the variables do not explicitly appear in w . . Bonakdarpour and S. Sheinvald XX:5 We now turn to formally define hyperautomata.
We begin with some terms and notations.Let s = ( w , w , . . . , w k ) be a tuple of finite words over Σ . We denote the length of the longestword in s by d s e . We represent s by a word over (Σ ∪ { } ) k of length d s e , which is formed by afunction zip ( s ) that “zips” the words in s together: the i ’th letter in zip ( s ) represents the i ’th lettersin w , w , . . . , w k , and is used to pad the words that have ended. For example, zip ( aab, bc, abdd ) = ( a, b, a )( a, c, b )( b, , d )( , , d ) . Formally, we have zip ( s ) = s s · · · s d s e , where s i [ j ] = w j i if j ≤ | w | , and s i [ j ] = , otherwise.Given a zipped word s , we denote the word formed by the letters in the i ’th positions in s by s [ i ] .That is, s [ i ] is the word σ σ · · · σ m formed by defining σ j = s j [ i ] , for s j [ i ] ∈ Σ . Notice that zip ( s ) is reversible, and we can define an unzip function as unzip ( s ) = ( s [1] , s [2] , . . . , s [ k ]) . We sometimesabuse the notation, and use unzip ( s ) to denote { s [1] , s [2] , . . . , s [ k ] } , and zip ( S ) to denote the zippingof the words in a finite hyperword S in some arbitrary order. (cid:73) Definition 5. A nondeterministic finite-word hyperautomaton (NFH) is a tuple A = h Σ , X, Q, Q , F, δ, α i , where Σ , Q , Q , and F are as in Definition 3, X = { x , . . . , x k } is a finite set of word variables , δ ⊆ Q × (Σ ∪ { } ) k × Q is a transition relation, and α = Q x Q x . . . Q n x k is a quantification condition , where Q i ∈ {∀ , ∃} for every ≤ i ≤ k . In Definition 5, the tuple h (Σ ∪ { } ) k , Q, Q , δ, F i forms an underlying NFA of A , which we denoteby ˆ A . We denote the alphabet of ˆ A by ˆΣ .Let S be a hyperword and let v : X → S be an assignment of the word variables of A to words in S . We denote by v [ x → w ] the assignment obtained from v by assigning the word w ∈ S to x ∈ X .We represent v by the word zip ( v ) = zip ( v ( x ) , . . . v ( x k )) . We now define the acceptance conditionof a hyperword S by an NFH A . We first define the satisfaction relation | = for S , A , a quantificationcondition α , and an assignment v : X → S , as follows.For α = (cid:15) , we denote S | = v ( α, A ) if ˆ A accepts zip ( v ) .For α = ∃ x i α , we denote S | = v ( α, A ) if there exists w ∈ S , such that S | = v [ x i → w ] ( α , A ) .For α = ∀ x i α , we denote S | = v ( α, A ) if for every w ∈ S , it holds that S | = v [ x i → w ] ( α , A ) . Since the quantification condition of A includes all of X , the satisfaction is independent of theassignment v , and we denote S | = A , in which case, we say that A accepts S . (cid:73) Definition 6.
Let A be an NFH. The hyperlanguage of A , denoted L ( A ) , is the set of allhyperwords that A accepts. We call a hyperlanguage L a regular hyperlanguage if there exists an NFH A such that L ( A ) = L . (cid:73) Example 7.
Consider the NFH A in Figure 2, over the alphabet Σ = { a, b } and two wordvariables x and x . From the initial state, two words lead to the left component in ˆ A iff in everyposition, if the word assigned to x has an a , the word assigned to x has an a . In the right component,the situation is dual – in every position, if the word assigned to x has an a , the word assigned to x has an a . Since the quantification condition of A is ∀ x ∀ x , in a hyperword S accepted by A , in every two words in S , the set of a positions of one is a subset of the a positions of the other.Therefore, L ( A ) includes all hyperwords in which there is a full ordering on the a positions. In case that α begins with ∀ , satisfaction holds vacuously with an empty hyperword. We restrict the discussion tononempty hyperwords. X:6 Automata for Hyperlanguages 𝑏, 𝑏 ,(𝑎, 𝑎) (𝑎, 𝑏) 𝑎, 𝑏 ,𝑏, 𝑏 , (𝑎, 𝑎)( (𝑏, 𝑎) (𝑏, ∀𝑥 ∀𝑥 (𝑏, ( ( Figure 2
The NFH A . We consider several fragments of NFH, which limit the structure of the quantification condition α . NFH ∀ is the fragment in which α contains only ∀ quantifiers, and similarly, in NFH ∃ , α containsonly ∃ quantifiers. In the fragment NFH ∃∀ , α is of the form ∃ x · · · ∃ x i ∀ x i +1 · · · ∀ x k . We present several more terms and notations which we use throughout the following sections. We saythat a word w over (Σ ∪ k is legal if w = zip ( u , . . . u k ) for some u , u , . . . u k ∈ Σ ∗ . Note that w is legal iff there is no w [ i ] in which there is an occurrence of followed by some letter σ ∈ Σ .Consider two letter tuples σ = ( t , . . . t k ) and σ = ( s , . . . s k ) . We denote by σ + σ thetuple ( t , . . . t k , s , . . . s k ) . We extend the notion to zipped words. Let w = zip ( u , . . . u k ) and w = zip ( v , . . . v k ) . We denote by w + w the word zip ( u , . . . u k , v , . . . v k ) .Consider a tuple t = ( t , t , . . . t k ) of items. A sequence of t is a tuple ( t , t , . . . t k ) , where t i ∈ { t , . . . t k } for every ≤ i ≤ k . A permutation of t is a reordering of the elements of t . We extend these notions to zipped words, to assignments, and to hyperwords, as follows. Let ζ = ( i , i , . . . i k ) be a sequence (permutation) of (1 , , . . . , k ) .Let w = zip ( w , . . . w k ) be a word over k -tuples. The word w ζ , defined as zip ( w i , w i , . . . w i k ) is a sequence (permutation) of w .Let v be an assignment from a set of variables { x , x , . . . x k } to a hyperword S . The assignment v ζ , defined as v ζ ( x j ) = v ( x i j ) for every ≤ i, j ≤ k , is a sequence (permutation) of v .Let S be a hyperword. The tuple w = ( w , . . . w k ) , where w i ∈ S , is a sequence of S . if { w , . . . w k } = S , then w is a permutation of S . Given an NFH A , the language of its underlying NFA ˆ A can be expressed as a regular expression r .Augmenting r with the quantification condition α of A constitutes a hyperregular expression (HRE) αr . For example, consider the NFH A in Figure 1. The HRE of A is: ∀ x ∀ x (cid:16) ( a, a ) | ( b, b ) (cid:17) ∗ (cid:16) ( , b ) ∗ | ( b, ∗ (cid:17) We now show the application of HREs in specifying well-known information-flow security policies.
Noninteference [11] requires that commands issued by users holding high clearances be removablewithout affecting observations of users holding low clearances: ϕ ni = ∀ x ∃ x ( l, lλ ) ∗ where l denotes a low state and lλ denotes a low state where all high commands are replaced by adummy value λ . . Bonakdarpour and S. Sheinvald XX:7 Observational determinism [17] requires that if two executions of a system start with low-security-equivalent events, then these executions should remain low equivalent: ϕ od = ∀ x ∀ x ( l, l ) + | (¯ l, ¯ l )($ , $) ∗ | ( l, ¯ l )($ , $) ∗ | (¯ l, l )($ , $) ∗ where l denotes a low event, ¯ l ∈ Σ \ { l } , and $ ∈ Σ . We note that similar policies such as Boudoland Castellani’s noninterference [5] can be formulated in the same fashion. Generalized noninterference (GNI) [12] allows nondeterminism in the low-observable behavior,but requires that low-security outputs may not be altered by the injection of high-security inputs: ϕ gni = ∀ x ∀ x ∃ x (cid:18) ( h, l, hl ) | (¯ h, l, ¯ hl ) | ( h, ¯ l, h ¯ l ) | (¯ h, ¯ l, ¯ h ¯ l ) (cid:19) ∗ where h denotes the high-security input, l denotes the low-security output, ¯ l ∈ Σ \{ l } , and ¯ h ∈ Σ \{ h } . Declassification [13] relaxes noninterference by allowing leaking information when necessary.Some programs need to reveal secret information to fulfill functional requirements. For example, apassword checker must reveal whether the entered password is correct or not: ϕ dc = ∀ x ∀ x ( li, li )( pw, pw )( lo, lo ) + where li denotes low-input state, pw denotes that the password is correct, and lo denotes low-outputstates. We note that for brevity, in the above formula, we do not include behaviors where the first twoevents are not low or in the second event, the password is not valid. Termination-sensitive noninterference requires that for two executions that start from low-observable states, information leaks are not permitted by the termination behavior of the program: ϕ tsni = ∀ x ∀ x ( l, l )($ , $) ∗ ( l, l ) | (¯ l, ¯ l )($ , $) ∗ | ( l, ¯ l )($ , $) ∗ | (¯ l, l )($ , $) ∗ where l denotes a low state and $ ∈ Σ . In this section, we consider the basic operations and decision problems for the various fragments ofNFH. We mostly provide proof sketches, and the complete details appear in the appendix. Throughoutthis section, A is an NFH h Σ , X, Q, Q , δ, F, α i , where X = { x , . . . x k } .We first show that NFH are closed under all the Boolean operations. (cid:73) Theorem 8.
NFH are closed under union, intersection, and complementation.
Proof Sketch.
Complementing A amounts to dualizing its quantification condition (replacing every ∃ with ∀ and vice versa), and complementing ˆ A via the standard construction for NFA.Now, let A and A be two NFH. The NFH A ∩ for L ( A ) ∩ L ( A ) is based on the productconstruction of ˆ A and ˆ A . The quantification condition of A ∩ is α · α . The underlying automaton ˆ A ∩ advances simultaneously on both A and A : when ˆ A and ˆ A run on zipped hyperwords w and w , respectively, ˆ A ∩ runs on w + w , and accepts only if both ˆ A and ˆ A accept.Similarly, the NFH A ∪ for L ( A ) ∪ L ( A ) is based on the union construction of ˆ A and ˆ A . Thequantification condition of A ∪ is again α · α . The underlying automaton ˆ A ∪ advances either on A or A . For every word w read by ˆ A , the NFH ˆ A ∪ reads w + w , for every w ∈ ˆΣ ∗ , and dually, forevery word w read by ˆ A , the NFH ˆ A ∪ reads w + w , for every w ∈ ˆΣ ∗ . (cid:74) This policy states that every two executions that start from bisimilar states (in terms of memory low-observability),should remain bisimilarly low-observable.
X:8 Automata for Hyperlanguages
We now turn to study various decision problems for NFH. We begin with the nonemptinessproblem: given an NFH A , is L ( A ) = ∅ ? We show that while the problem is in general undecidablefor NFH, it is decidable for the fragments that we consider. (cid:73) Theorem 9.
The nonemptiness problem for NHF is undecidable.
The proof of Theorem 9 mimics the ideas in [9], which uses a reduction from the
Post corres-pondence problem (PCP) to prove the undecidability of HyperLTL satisfiability.For the alternation-free fragments, we can show that a simple reachability test on their underlyingautomata suffices to verify nonemptiness. Hence, we have the following. (cid:73)
Theorem 10.
The nonemptiness problem for NFH ∃ and NFH ∀ is NL-complete . The nonemptiness of NFH ∃∀ is harder, and reachability does not suffice. However, we show thatthe problem is decidable. (cid:73) Theorem 11.
The nonemptiness problem for NFH ∃∀ is PSPACE-complete . Proof Sketch.
We can show that an NFH ∃∀ A is nonempty iff it accepts a hyperword S of sizethat is bounded by the number m of ∃ quantifiers in α . We can then construct an NFA A whoselanguage is nonempty iff it accepts zip ( S ) for such a hyperword S . The size of A is O ( | δ | ) m k − m ) .Unless A only accepts hyperwords of size , which can be easily checked, | δ | must be exponentialin the number k − m of ∀ quantifiers, to account for all the assignments to the variables under ∀ ,and so overall | A | is of size O ( |A| k ) . The problem can then be decided in PSPACE by traversing A on-the-fly. We show that a similar result holds for the case that k − m is fixed.We use a reduction from the unary version of the tiling problem to prove PSPACE lower boundsboth for the general case and for the case of a fixed number of ∀ quantifiers. (cid:74) We turn to study the membership problem for NFH: given an NFH A and a hyperword S , is S ∈ L ( A ) ? When S is finite, the set of possible assignments from X to S is finite, and so the problemis decidable. We call this case the finite membership problem . (cid:73) Theorem 12.
The finite membership problem for NFH is in
PSPACE .The finite membership problem for NFH with O (log( k )) ∀ quantifiers is NP-complete . Proof Sketch.
We can decide the membership of a hyperword S in L ( A ) by iterating over allrelevant assignments from X to S , and for every such assignment v , checking on-the-fly whether zip ( v ) is accepted by ˆ A . This algorithm uses space of size that is polynomial in k and logarithmic in |A| and in | S | .When the number of ∀ quantifiers in A is | O (log( k )) | , we can iterate over all assignments to the ∀ variables in polynomial time, while guessing assignments to the variables under ∃ . Thus, membershipin this case is in NP .We use a reduction from the Hamiltonian cycle problem to prove NP-hardness for this case.Given a graph G = h{ v , . . . v n } , E i , we construct a hyperword S with n different words of length n over { , } , each of which contains a single . We also construct an NFH ∃ A over { , } with n variables, a graph construction similar to that of G , and a single accepting and initial state v . Fromvertex v i there are transitions to all its neighbors, labeled by the letter (0) i − + (1) + (0) n − i . Thus, A accepts S iff there exists an assignment f : X → S such that zip ( f ) ∈ L ( ˆ A ) . Such an assignment f describes a cycle in G , where f ( x i ) = w j matches traversing v i in the j ’th step. The words in S ensure a single visit in every state, and their length ensures a cycle of length n . Note: for every hyperword of size at least , the number of transitions in δ must be exponential in thenumber k of ∀ quantifiers, to account for all the different assignments to these variables. Thus, if k = O ( k ) , an algorithm that uses a space of size k is in fact logarithmic in the size of A . (cid:74) . Bonakdarpour and S. Sheinvald XX:9 When S is infinite, it may still be finitely represented. We now address the problem of decidingwhether a regular language L (given as an NFA) is accepted by an NFH. We call this the regularmembership problem for NFH . We show that this problem is decidable for the entire class of NFH. (cid:73) Theorem 13.
The regular membership problem for NFH is decidable.
Proof Sketch.
Let A be an NFA, and let A be an NFH, both over Σ . We describe a recursiveprocedure for deciding whether L ( A ) ∈ L ( A ) .For the base case of k = 1 , if α = ∃ x , then L ( A ) ∈ L ( A ) iff L ( A ) ∩ L ( ˆ A ) = ∅ . Otherwise, if α = ∀ x , then L ( A ) ∈ L ( A ) iff L ( A ) / ∈ L ( A ) , where A is the NFH for L ( A ) . The quantificationcondition for A is ∃ x , which conforms to the previous case.For k > , we construct a sequence of NFH A , A , . . . , A k . If α starts with ∃ , then we set A = A . Otherwise, we set A = A . Given A i with a quantification condition α i , we construct A i +1 as follows. If α i starts with ∃ , then the set of variables of A i +1 is { x i +1 , . . . x k } , and thequantification condition α i +1 is Q i +1 x i +1 · · · Q k x k , where α i = Q i x i Q i +1 · · · Q k x k . The NFH A i +1 is roughly constructed as the intersection between A and ˆ A i , based on the first position inevery ( k − i ) -tuple letter in ˆΣ i . Then, ˆ A i +1 accepts a word zip ( u , . . . u k − i ) iff there exists a word u ∈ L ( A ) , such that ˆ A i accepts zip ( u, u , . . . u k − i ) . Notice that this exactly conforms to the ∃ condition. Therefore, if Q i = ∃ , then L ( A ) ∈ L ( A i ) iff L ( A ) ∈ L ( A i +1 ) .If Q i = ∀ , then L ( A ) ∈ L ( A i ) iff L ( A ) / ∈ L ( A i ) . The quantification condition of A i beginswith ∃ x i . We then construct A i +1 w.r.t. A i as described above, and check for non-membership.Every ∀ quantifier requires complementation, which is exponential in | Q | . Therefore, in the worstcase, the complexity of this algorithm is O (2 ... | Q || A | ) , where the tower is of height k . If the numberof ∀ quantifiers is fixed, then the complexity is O ( | Q || A | k ) . (cid:74) Since nonemptiness of NFH is undecidable, so are its universality and containment problems.However, we show that containment is decidable for the fragments that we consider. (cid:73)
Theorem 14.
The containment problems of NFH ∃ and NFH ∀ in NFH ∃ and NFH ∀ and of NFH ∃∀ in NFH ∃ and NFH ∀ are PSPACE-complete . Proof Sketch.
The lower bound follows from the
PSPACE-hardness of the containment problemfor NFA. For the upper bound, for two NFH A and A , we have that L ( A ) ⊆ L ( A ) iff L ( A ) ∩ L ( A ) = ∅ . We can use the constructions in the proof of Theorem 8 to compute a matching NFH A = A ∩ A , and check its nonemptiness. Complementing A is exponential in its number of states,and the intersection construction is polynomial.If A ∈ NFH ∃ and A ∈ NFH ∀ or vice versa, then A is an NFH ∃ or NFH ∀ , respectively, whosenonemptiness can be decided in space that is logarithmic in |A| .It follows from the construction in the proof of Theorem 8, that the quantification condition of A may be any interleaving of the quantification conditions of the two intersected NFH. Therefore, forthe rest of the fragments, we can construct the intersection such that A is an NFH ∃∀ .The PSPACE upper bound of Theorem 11 is derived from the number of variables and not fromthe state-space of the NFH. Therefore, while | ¯ A | is exponential in the number of states of A ,checking the nonemptiness of A is in PSPACE . (cid:74) In this section, we introduce L ∗ -based learning algorithms for the fragments NFH ∀ and NFH ∃ . Wefirst survey the L ∗ algorithm [2], and then describe the relevant adjustments for our case. X:10 Automata for Hyperlanguages ∗ Algorithm L ∗ consists of two entities: a learner , who wishes to learn a DFA A for an unknown (regular) language L , and a teacher , who knows L . During the learning process, the learner asks the teacher two types ofqueries: membership queries (“is the word w in L ?”) and equivalence queries (“is A a DFA for L ?”).The learner maintains A in the form of an observation table T of truth values, whose rows D, D · Σ and columns E are sets of words over Σ , where D is prefix-closed, and E is suffix-closed.Initially, D = E = { (cid:15) } . For a row d and a column e , the entry for T ( d, e ) is tt iff d · e ∈ L . Theentries are filled via membership queries. The vector of truth values for row d is denoted row ( d ) .Intuitively, the rows in D determine the states of A , and the rows in D · Σ determine the transitionsof A : the state row ( d · σ ) is reached from row ( d ) upon reading σ .The learner updates T until it is closed , which, intuitively, ensures a full transition relation and consistent , which, intuitively, ensures a deterministic transition relation. If T is not closed or notconsistent then more rows or more columns are added to T , respectively.When T is closed and consistent, the learner constructs A : The states are the rows of D , the initialstate is row ( (cid:15) ) , the accepting states are these in which T ( d, (cid:15) ) = tt , and the transition relation is asdescribed above. The learner then submits an equivalence query. If the teacher confirms, the algorithmterminates. Otherwise, the teacher returns a counterexample w ∈ L ( A ) but w / ∈ L (which we call a positive counterexample ), or w / ∈ L ( A ) but w ∈ L (which we call a negative counterexample ). Thelearner then adds w and all its suffixes to E , and proceeds to construct the next candidate DFA A .It is shown in [2] that as long as A is not a DFA for L , it has less states than a minimal DFA for L .Further, every change in the table adds at least one state to A . Therefore, the procedure is guaranteedto terminate successfully with a minimal DFA A for L .The correctness of the L ∗ algorithm follows from the fact that regular languages have a canonicalform , which guarantees a single minimal DFA for a regular language L . To enable an L ∗ -basedalgorithm for NFH ∀ and NFH ∃ , we first define canonical forms for these fragments. We begin with the basic terms on which our canonical forms are based. (cid:73)
Definition 15. 1.
An NFH ∀ A ∀ is sequence complete if for every word w , it holds that ˆ A ∀ accepts w iff it accepts every sequence of w . An NFH ∃ A ∃ is permutation complete if for every word w , it holds that ˆ A ∃ accepts w iff it acceptsevery permutation of w . An NFH ∀ A ∀ accepts a hyperword S iff ˆ A ∀ accepts every sequence of size k of S . If somesequence is missing from L ( ˆ A ) , then removing the rest of the sequences of S from L ( ˆ A ∀ ) does notaffect the non-acceptance of S . Therefore, the underlying automata of sequence-complete NFH ∀ onlyaccept necessary sequences. Similarly, an NFH ∃ A ∃ accepts a hyperword S iff ˆ A ∃ accepts somepermutation p of size k of words in S . Adding the rest of the permutations of p to L ( ˆ A ∃ ) does notaffect the acceptance of S . Therefore, the underlying automata of permutation-complete NFH ∃ onlyreject the necessary permutations of every hyperword. As a conclusion, we have the following. (cid:73) Lemma 16. 1.
Let A ∀ be an NFH ∀ , and let A be a sequence-complete NFH ∀ over Σ and X such that for every word w , the underlying NFA ˆ A accepts w iff ˆ A ∀ accepts every sequence of w .Then L ( A ∀ ) = L ( A ) . Let A ∃ be an NFH ∃ , and let A be a permutation-complete NFH ∃ over Σ and X such that forevery word w , the underlying NFA ˆ A ∃ accepts w iff ˆ A accepts all permutations of w . Then L ( A ∃ ) = L ( A ) . . Bonakdarpour and S. Sheinvald XX:11 Next, we show that we can construct a sequence- or permutation-complete NFH for a given NFH ∀ or NFH ∃ , respectively. Intuitively, given A , for every sequence (permutation) ζ of (1 , . . . k ) , weconstruct an NFA that runs on w ζ in the same way that ˆ A runs on w , for every w . The underlying NFAwe construct for the NFH ∀ and NFH ∃ are the intersection and union, respectively, of all these NFA. (cid:73) Lemma 17.
Every NFH ∀ (NFH ∃ ) A has an equivalent sequence-complete (permutation-complete)NFH ∀ (NFH ∃ ) A over the same set of variables. Finally, as the following theorem shows, sequence- and permutation- complete NFH offer aunified model for the alternation-free fragments. (cid:73)
Theorem 18.
Let A and A be two sequence-complete (permutation-complete) NFH ∀ (NFH ∃ )over the same set of variables. Then L ( A ) = L ( A ) iff L ( ˆ A ) = L ( ˆ A ) . Regular languages have a canonical form, which are minimal DFA. We use this property to definecanonical forms for NFH ∀ and NFH ∃ as sequence-complete (permutation-complete) NFH ∀ (NFH ∃ )with a minimal number of variables and a minimal underlying DFA. NFH ∀ and NFH ∃ We now describe our L ∗ -based learning algorithms for NFH ∃ and NFH ∀ . These algorithms aim tolearn an NFH with the canonical form defined in Section 6.2 for a target hyperlanguage L . Figure 4presents the overall flow of the learning algorithms for both fragments.In the case of hyperautomata, the membership queries and the counterexamples provided bythe teacher consist of hyperwords. Similarly to [8], we assume a teacher that returns a minimalcounterexample in terms of size of the hyperword.During the procedure, the learner maintains an NFH A via an observation table for ˆ A , overthe alphabet ˆΣ = (Σ ∪ { } ) k , where k is initially set to . When the number of variables isincreased to k > k , the alphabet of ˆ A is extended accordingly to (Σ ∪ { } ) k . To this end, wedefine a function ↑ k k : (Σ ∪ { } ) k → (Σ ∪ { } ) k , which replaces every letter ( σ , . . . σ k ) , with ( σ , . . . σ k ) + ( σ k ) k − k . That is, the last letter is duplicated to create a k -tuple. We extend ↑ k k to words: ↑ k k ( w ) is obtained by replacing every letter σ in w with ↑ k k ( σ ) . Notice that, for bothfragments, if unzip ( d · e ) ∈ L ( A ) , then unzip ( ↑ k k ( d · e )) ∈ L ( A ) . Accordingly, when the numberof variables is increased, every word w in the rows and columns of T is replaced with ↑ k k ( w ) , anaction which we denote by ↑ k k ( T ) . NFH ∀ In the case of NFH ∀ , when the teacher returns a counterexample S , it holds that if | S | > k , then S must be positive. Indeed, assume by way of contradiction that S is negative. Then, for every k words w , . . . , w k in S , it holds that zip ( w , . . . , w k ) ∈ L ( ˆ A ) , but S / ∈ L . Therefore, in an NFH ∀ A for L , there exists some word of the form w = zip ( w , . . . w k ) such that w i ∈ S for ≤ i ≤ k ,and w / ∈ L ( ˆ A ) . As a result, { w , . . . , w k } / ∈ L . Since zip ( w , . . . , w k ) and all its sequences are in L ( ˆ A ) , then a smaller counterexample is { w , . . . , w k } , a contradiction to the minimality of S .In fact, if | S | > k , then it must be that | S | = k + 1 . Indeed, since S is a positive counterexample,and A accepts all representations of subsets of size k of S (otherwise the teacher would return acounterexample of size k ), then there exists a subset S ⊆ S of size k + 1 that should be represented,but is not. Therefore, S is a counterexample of size k + 1 .When a counterexample S of size k + 1 is returned, the learner updates k ← k + 1 , updates T to ↑ k +1 k ( T ) , arbitrarily selects a permutation p of the words in S , and adds zip ( p ) and all its suffixes to E . In addition, it updates D · ˆΣ in accordance with the new updated ˆΣ , and fills in the missing entries. X:12 Automata for Hyperlanguages
When | S | ≤ k , then the counterexample is either positive or negative. If S is positive, then thereexists some permutation p of the words in S such that A does not accept zip ( p ) (a permutation and nota proper sequence, or there would be a smaller counterexample). The learner finds such a permutation p , and adds zip ( p ) and all its suffixes to E . Notice that zip ( p ) does not already appear in T , since amembership query would have returned “yes”, and so ˆ A would have accepted zip ( p ) .if S is negative, then A accepts all sequences of length k of words in S , though it should not.Then there exists a permutation p of the words in S that does not appear in T , and which A accepts.The learner then finds such a permutation p and adds zip ( p ) and all its suffixes to E .If p is a permutation of the words in S , and S is a negative counterexample, then zip ( p ) shouldnot be in L ( ˆ A ) due to any other hyperword, and if S is a positive counterexample, then it should bein L ( ˆ A ) for every S such that S ⊆ S . Therefore, the above actions by the learner are valid.When an equivalence query succeeds, then A is indeed an NFH ∀ for L . However, A is notnecessarily sequence-complete, as ˆ A may accept a word w = zip ( w , . . . , w k ) but not all of itssequences. This check can be performed by the learner directly on ˆ A . Notice that w does notoccur in T , since a membership query on w would return “no”. Once it is verified that A is notsequence-complete, the counterexample w (and all its suffixes) are added to E , and the procedurereturns to the learning loop.As we have explained above, variables are added only when necessary, and so the output A isindeed an NFH for L with minimally many variables. The correctness of L ∗ and the minimality ofthe counterexamples returned by the teacher guarantee that for each k ≤ k , the run learns a minimaldeterministic ˆ A for hyperwords in L that are represented by k variables. Therefore, a smaller ˆ A for L does not exist, as restricting ˆ A to the first k letters in each k -tuple would produce a smallerunderlying automaton for k variables, a contradiction. 𝜖𝑇
1𝜖 1(𝑎) 1(𝑏) 1( (𝑎, 𝑏)𝜖𝑇
11𝜖 11(𝑎, 𝑎) 11(𝑏, 𝑏) 11(𝑎, 𝑏) 01(𝑎, (𝑐𝑜𝑛𝑡. ) 01(𝑏, 𝑎) 01(𝑏,
11𝜖 11(𝑎, 𝑎)
Figure 3
The first stages of learning L ( A ) of Figure 1. (cid:73) Example 19.
Figure 3 displays the first two stages of learning L ( A ) of Figure 2. T displaysthe initial table, with D = E = { (cid:15) } , and ˆΣ = { a, b, } . since { a } , { b } , and { (cid:15) } are all in L ( A ) ,the initial candidate NFH A includes a single variable, and, following the answers to the membershipqueries, a single accepting state.Since L ( A ) includes all hyperwords of size , which are now accepted by A , the smallestcounterexample the teacher returns is of size , which, in the example, is { a, b } . Table T isthen obtained from T by applying ↑ , updating the alphabet ˆΣ to { a, b, } , and updating D · ˆΣ accordingly. T is filled by submitting membership queries. For example, for ( b, a ) ∈ D · ˆΣ and ( a, b ) ∈ E , the learner submits a membership query for { ba, ab } , to which the teacher answers “no”. NFH ∃ The learning process for NFH ∃ is similar to the one for NFH ∀ . We briefly describe the differences. . Bonakdarpour and S. Sheinvald XX:13 T closedand consistent?no: membership queries construct A permutation/sequencecomplete? n o : a d d c e x w equivalent? no : ce x S | S | > k ? y e s y e s A no : a dd ce x S y e s y e s : A a dd S T ←↑ k k ( T ) Figure 4
The learning process flow for NFH ∀ and NFH ∃ . As in NFH ∀ , relying on the minimality of the counterexamples returned by the teacher guaranteesthat when a counterexample S such that | S | > k is returned, it is a positive counterexample. Indeed,assume by way of contradiction that S is a negative counterexample of size k . Since ˆ A accepts S ,there exists a word zip ( w , . . . , w k ) in L ( ˆ A ) such that { w , . . . , w k } ∈ S . According to the semanticsof ∃ , if zip ( w , w , . . . , w k ) ∈ L ( ˆ A ) then S ∈ L ( A ) . Since S / ∈ L , we have that { w , . . . , w k } is asmaller counterexample, a contradiction.Therefore, when the teacher returns a counterexample S of size k > k , the alphabet ˆΣ is extendedto (Σ ∪ { } ) k , and the table T is updated by ↑ k k , as is done for NFH ∀ .If | S | ≤ k , then S may be either positive or negative. If S is negative, then there exists somepermutation of S that is accepted by ˆ A . However, no such permutation is in T , as a membershipquery would have returned “no”. Similarly, if S is positive, then there exists no permutation of S that ˆ A accepts. In both cases, the learner chooses a permutation of S and adds it, and all its suffixes, to E .As in the case of NFH ∀ , the success of an equivalence query does not necessarily imply that A is permutation-complete. If A is not permutation-complete, the learner finds a word w that is apermutation of w such that w ∈ L ( ˆ A ) but w / ∈ L ( ˆ A ) , and adds w as a counterexample to E . Theprocedure then returns to the learning loop. We have introduced and studied hyperautomata and hyperlanguages , focusing on the basic modelof regular hyperlanguages, in which the underlying automaton is a standard NFA. We have shownthat regular hyperlanguages are closed under set operations (complementation, intersection, andunion) and are capable of expressing important hyperproperties for information-flow security policiesover finite traces. We have also investigated fundamental decision procedures such as checkingnonemptiness and membership. We have shown that their regular properties allow the learnability ofthe alternation-free fragments. Fragments that combine the two types of quantifiers prove to be morechallenging, and we leave their learnability to future work.The notion of hyperlanguages, as well as the model of hyperautomata, can be lifted to handlehyperwords that consist of infinite words: instead of an underlying finite automaton, we can use anymodel that accepts infinite words. In fact, we believe using an underlying alternating Büchi automaton,such hyperautomata can express the entire logic of HyperLTL [6], using the standard Vardi-Wolperconstruction for LTL [15] as basis. Our complexity results for the various decision procedures forNFH, combined with the complexity results shown in [9], suggest that using hyperautomata would beoptimal, complexity-wise, for handling HyperLTL.Further future directions include studying non-regular hyperlanguages (e.g., context-free), andobject hyperlanguages (e.g., trees). Other open problems include a full investigation of the complexityof decision procedures for alternating fragments of NFH.
X:14 Automata for Hyperlanguages
References B. Alpern and F. B. Schneider. Defining liveness.
Information Processing Letters , 21:181–185, 1985. Dana Angluin. Learning regular sets from queries and counterexamples.
Infornation and Computation ,75(2):87–106, 1987. B. Bonakdarpour and B. Finkbeiner. The complexity of monitoring hyperproperties. In
Proceedings ofthe 31st IEEE Computer Security Foundations Symposium CSF , pages 162–174, 2018. B. Bonakdarpour, C. Sánchez, and G. Schneider. Monitoring hyperproperties by combining staticanalysis and runtime verification. In
Proceedings of the 8th Leveraging Applications of Formal Methods,Verification and Validation (ISoLA) , pages 8–27, 2018. G. Boudol and I. Castellani. Noninterference for concurrent programs and thread systems.
TheoreticalComputer Science (TCS) , 281(1-2):109–130, 2002. M. R. Clarkson, B. Finkbeiner, M. Koleini, K. K. Micinski, M. N. Rabe, and C. Sánchez. Temporal logicsfor hyperproperties. In
Proceedings of the 3rd Conference on Principles of Security and Trust POST ,pages 265–284, 2014. M. R. Clarkson and F. B. Schneider. Hyperproperties.
Journal of Computer Security , 18(6):1157–1210,2010. B. Finkbeiner, L. Haas, and H. Torfah. Canonical representations of k -safety hyperproperties. In Proceedings of the 32nd IEEE Computer Security Foundations Symposium (CSF) , pages 17–31, 2019. B. Finkbeiner and C. Hahn. Deciding hyperproperties. In
Proceedings of the 27th International Conferenceon Concurrency Theory (CONCUR) , pages 13:1–13:14, 2016. B. Finkbeiner, M. N. Rabe, and C. Sánchez. Algorithms for model checking HyperLTL and HyperCTL*.In
Proceedings of the 27th International Conference on Computer Aided Verification (CAV) , pages 30–48,2015. J. A. Goguen and J. Meseguer. Security policies and security models. In
IEEE Symp. on Security andPrivacy , pages 11–20, 1982. D. McCullough. Noninterference and the composability of security properties. In
Proceedings of the1988 IEEE Symposium on Security and Privacy , pages 177–186, 1988. A. Sabelfeld and D. Sands. Probabilistic noninterference for multi-threaded programs. In
Proceedings ofthe 13th IEEE Computer Security Foundations Workshop (CSFW) , pages 200–214, 2000. M.Y. Vardi and P. Wolper. Automata theoretic techniques for modal logic of programs.
Journal ofComputer and System Sciences , 32:183–221, 1986. M.Y. Vardi and P. Wolper. Reasoning about infinite computations.
Information and Computation ,115(1):1–37, 1994. Y. Wang, M. Zarei, B. Bonakdarpour, and M. Pajic. Statistical verification of hyperproperties for cyber-physical systems.
ACM Transactions on Embedded Computing systems (TECS) , 18(5s):92:1–92:23,2019. S. Zdancewic and A. C. Myers. Observational determinism for concurrent program security. In
Proceed-ings of the 16th IEEE Computer Security Foundations Workshop (CSFW) , page 29, 2003. . Bonakdarpour and S. Sheinvald XX:15
Appendix
A ProofsTheorem 8
Proof.
Complementation.
Let A be an NFH. The NFA ˆ A can be complemented with respect to itslanguage over ˆΣ to an NFA ˆ A . Then for every assignment v : X → S , it holds that ˆ A accepts zip ( v ) iff ˆ A does not accept zip ( v ) . Let α be the quantification condition obtained from α by replacing every ∃ with ∀ and vice versa. We can prove by induction on α that A , the NFH whose underlying NFA is ˆ A , and whose quantification condition is α , accepts L ( A ) . The size of A is exponential in | Q | , due tothe complementation construction for ˆ A .Now, let A = h Σ , X, Q, Q , δ , F , α i and A = h Σ , Y, P, P , δ , F , α i be two NFH with | X | = k and | Y | = k variables, respectively. Union.
We construct an NFH A ∪ = h Σ , X ∪ Y, Q ∪ P ∪ { p , p } , Q ∪ P , δ, F ∪ F ∪ { p , p } , α i ,where α = α α (that is, we concatenate the two quantification conditions), and where δ is definedas follows.For every ( q σ ,...,σ k ) −−−−−−→ q ) ∈ δ we set ( q σ ,...,σ k )+ t −−−−−−−−→ q ) ∈ δ for every t ∈ (Σ ∪ { } ) k .For every ( q σ ,...,σ k ) −−−−−−−→ q ) ∈ δ we set ( q t +( σ ,...,σ k ) −−−−−−−−→ q ) ∈ δ for every t ∈ (Σ ∪ { } ) k .For every q ∈ F , we set ( q ( k + t −−−−−→ p ) ∈ δ and ( p k + t −−−−−→ p ) ∈ δ for every t ∈ (Σ ∪ { } ) k .For every q ∈ F , we set ( q t +( k −−−−−→ p ) ∈ δ and ( p t +( k −−−−−→ p ) ∈ δ for every t ∈ (Σ ∪ { } ) k . X:16 Automata for Hyperlanguages
Let S be a hyperword. For every v : ( X ∪ Y ) → S , it holds that if zip ( v | X ) ∈ L ( ˆ A ) , then zip ( v ) ∈ L ( ˆ A ∪ ) . Indeed, according to our construction, every word assigned to the Y variables isaccepted in the A component of the construction, and so it satisfies both types of quantifiers. Asimilar argument holds for v | Y and A .Also, according to our construction, for every v : ( X ∪ Y ) → S , if zip ( v ) ∈ L ( ˆ A ∪ ) , then either zip ( v | X ) ∈ L ( ˆ A ) , or zip ( v | Y ) ∈ L ( ˆ A ) . As a conclusion, we have that L ( A ∪ ) = L ( A ) ∪ L ( A ) .The state space of A ∪ is linear in the state spaces of A , A . However, the size of the alphabetof A ∪ may be exponentially larger than that of A and A , since we augment each letter with allsequences of size k (in A ) and k (in A ). Intersection.
The proof follows the closure under union and complementation. However, we alsooffer a direct translation, which avoids the need to complement. We construct an NFH A ∩ = h Σ , X ∪ Y, ( Q ∪ { q } × P ∪ { p } ) , ( Q × P ) , δ, ( F ∪ { q } ) × ( F ∪ { p } ) , α α i , where δ is definedas follows.For every ( q σ ,...,σ k ) −−−−−−→ q ) ∈ δ and every ( p σ ,...,σ k ) −−−−−−−→ p ) ∈ δ , we have (cid:16) ( q , p ) ( σ ...,σ k ,σ ,...σ k ) −−−−−−−−−−−−→ ( q , p ) (cid:17) ∈ δ For every q ∈ F , ( p σ ,...,σ k ) −−−−−−−→ p ) ∈ δ we have (cid:16) ( q , p ) ( k +( σ ,...σ k ) −−−−−−−−−−−→ ( q, p ) (cid:17) , (cid:16) ( q, p ) ( k +( σ ,...,σ k ) −−−−−−−−−−−→ ( q, p ) (cid:17) ∈ δ For every ( q σ ,...,σ k ) −−−−−−→ q ) ∈ δ and p ∈ F , we have (cid:16) ( q , p ) ( σ ,...,σ k )+( k −−−−−−−−−−−→ ( q , p ) (cid:17) , (cid:16) ( q , p ) ( σ ,...,σ k )+( k −−−−−−−−−−−→ ( q , p ) (cid:17) ∈ δ Intuitively, the role of q, p is to keep reading ( k and ( k after the word read by ˆ A or ˆ A ,respectively, has ended.The NFH ˆ A ∩ simultaneously reads two words zip ( w , w , . . . , w k ) and zip ( w , w , . . . w k ) thatare read along ˆ A and ˆ A , respectively, and accepts iff both words are accepted. The correctnessfollows from the fact that for v : ( X ∪ Y ) → S , we have that zip ( v ) is accepted by ˆ A iff zip ( v | X ) and zip ( v | Y ) are accepted by ˆ A and ˆ A , respectively.This construction is polynomial in the sizes of A and A . (cid:74) Theorem 9
Proof.
We mimic the proof idea in [9], which uses a reduction from the
Post correspondence problem(PCP) . A PCP instance is a collection C of dominoes of the form: (h u v i , h u v i , . . . , h u k v k i) where for all i ∈ [1 , k ] , we have v i , u i ∈ { a, b } ∗ . The problem is to decide whether there exists afinite sequence of the dominoes of the form h u i v i ih u i v i i · · · h u i m v i m i where each index i j ∈ [1 , k ] , such that the upper and lower finite strings of the dominoes are equal,i.e., u i u i · · · u i m = v i v i · · · v i m . Bonakdarpour and S. Sheinvald XX:17 For example, if the set of dominoes is C exmp = (h abb i , h baa i , h aaba i) Then, a possible solution is the following sequence of dominoes from C exmp : sol = h aaba ih baa ih abb i . Given an instance C of PCP, we encode a solution as a word w sol over the following alphabet: Σ = n σσ | σ, σ ∈ { a, b, ˙ a, ˙ b, $ } o . Intuitively, ˙ σ marks the beginning of a new domino, and $ marks the end of a sequence of the upperor lower parts of the dominoes sequence.We note that w sol encodes a legal solution iff the following conditions are met: For every σσ that occurs in w sol , it holds that σ, σ represent the same domino letter (both a orboth b , either dotted or undotted). The number of dotted letters in the upper part of w sol is equal to the number of dotted letters inthe lower part of w sol . w sol starts with two dotted letters, and the word u i between the i ’th and i + 1 ’th dotted letters inthe upper part of w sol , and the word v i between the corresponding dotted letters in the lower partof w sol are such that [ u i v i ] ∈ C , for every i .We call a word that represents the removal of the first k dominoes from w sol a partial solution ,denoted by w sol,k . Note that the upper and lower parts of w sol,k are not necessarily of equal lengths(in terms of a and b sequences), since the upper and lower parts of a domino may be of differentlengths, and so we use letter $ to pad the end of the encoding in the shorter of the two parts.We construct an NFH A , which, intuitively, expresses the following ideas: (1) There exists anencoding w sol of a solution to C , and (2) For every w sol,k = (cid:15) in a hyperword S accepted by A , theword w sol,k +1 is also in S . L ( A ) is then the set of all hyperwords that contain an encoded solution w sol , as well as all itssuffixes obtained by removing a prefix of dominoes from w sol . This ensures that w sol indeed encodesa legal solution. For example, a matching hyperword S (for solution sol discussed earlier) that isaccepted by A is: S = { w sol = ˙ a ˙ a ˙ bb aa ˙ a ˙ a b ˙ b , w sol, = ˙ b ˙ a a ˙ b ˙ a $ b $ , w sol, = ˙ a ˙ b b $ , w sol, = (cid:15) } Thus, the acceptance condition of A is α = ∀ x ∃ x ∃ x , where x is to be assigned a potentialpartial solution w sol,k , and x is to be assigned w sol,k +1 , and x is to be assigned w sol .During a run on a hyperword S and an assignment v : { x , x , x } → S , the NFH A checks thatthe upper and lower letters of w sol all match. In addition, A checks that the first domino of v ( x ) isindeed in C , and that v ( x ) is obtained from v ( x ) by removing the first tile. A performs the lattertask by checking that the upper and lower parts of v ( x ) are the upper and lower parts of v ( x ) thathave been “shifted” back appropriately. That is, if the first tile in v ( x ) is the encoding of [ w i v i ] , then A uses states to remember, at each point, the last | w i | letters of the upper part of v ( x ) and the last | v i | letters of the lower part of v ( x ) , and verifies, at each point, that the next letter in v ( x ) matchesthe matching letter remembered by the state. (cid:74) X:18 Automata for Hyperlanguages
Theorem 10
Proof.
The lower bound for both fragments follows from the
NL-hardness of the nonemptinessproblem for NFA.We turn to the upper bound, and begin with NFH ∃ . Let A ∃ be an NFH ∃ . We claim that A ∃ isnonempty iff ˆ A ∃ accepts some legal word w . The first direction is trivial. For the second direction, let w ∈ L ( ˆ A ∃ ) , and let S = unzip ( w ) . By assigning v ( x i ) = w [ i ] for every x i ∈ X , we get zip ( v ) = w ,and according to the semantics of ∃ , we have that A ∃ accepts S . To check whether ˆ A ∃ accepts alegal word, we can run a reachability check on-the-fly, while advancing from a letter σ to the nextletter σ only if σ contains in all the positions in which σ contains . While each transition T = q ( σ ,...σ n ) −−−−−−→ p in ˆ A is of size k , we can encode T as a set of size k of encodings of transitions oftype q σ i −→ p with a binary encoding of p, q, σ i , as well as i, t , where t marks the index of T withinthe set of transitions of ˆ A . Therefore, the reachability test can be performed within space that islogarithmic in the size of A ∃ .Now, let A ∀ be an NFH ∀ over X . We claim that A ∀ is nonempty iff A ∀ accepts a hyperword ofsize . For the first direction, let S ∈ L ( A ∀ ) . Then, by the semantics of ∀ , we have that for everyassignment v : X → S , it holds that zip ( v ) ∈ L ( ˆ A ∀ ) . Let u ∈ S , and let v u ( x i ) = u for every x i ∈ X . Then, in particular, zip ( v u ) ∈ L ( ˆ A ∀ ) . Then for every assignment v : X → { u } (whichconsists of the single assignment v u ), it holds that ˆ A ∀ accepts zip ( v ) , and therefore A ∀ accepts { u } .The second direction is trivial.To check whether A ∀ accepts a hyperword of size , we restrict the reachability test on ˆ A ∀ to k -tuples of the form ( σ, σ, . . . σ ) for σ ∈ Σ . (cid:74) Theorem 11
Proof.
We begin with the upper bound. Let S ∈ L ( A ) . Then, according to the semantics ofthe quantifiers, there exist w , . . . w m ∈ S , such that for every assignment v : X → S in which v ( x i ) = w i for every ≤ i ≤ m , it holds that ˆ A accepts zip ( v ) . Let v : X → S be such anassignment. Then, ˆ A accepts zip ( v ζ ) for every sequence ζ of the form (1 , , . . . m, i , i , . . . i k − m ) .In particular, it holds for such sequences in which ≤ i j ≤ m for every ≤ j ≤ k − m , thatis, sequences in which the last k − m variables are assigned words that are assigned to the first m variables. Therefore, again by the semantics of the quantifiers, we have that { v ( x ) , . . . v ( x m ) } is in L ( A ) . The second direction is trivial.We call zip ( v ζ ) as described above a witness to the nonemptiness of A , i.e., zip ( v ζ ) is an instan-tiation of the existential quantifiers. We construct an NFA A based on ˆ A that is nonempty iff ˆ A accepts a witness to the nonemptiness of A . Let Γ be the set of all sequences of the above form. Forevery sequence ζ = ( i , i , . . . i k ) in Γ , we construct an NFA A ζ = h ˆΣ , Q, Q , δ ζ , F i , where forevery q ( σ i ,σ i ,...σ ik ) −−−−−−−−−−→ q in δ , we have q ( σ ,σ ,...σ k ) −−−−−−−−→ q in δ ζ . Intuitively, A ζ runs on every word w the same way that ˆ A runs on w ζ . Therefore, ˆ A accepts a witness w to the nonemptiness of A iff w ∈ L ( A ζ ) for every ζ ∈ Γ .We define A = T ζ ∈ Γ A ζ . Then ˆ A accepts a witness to the nonemptiness of A iff A is nonempty.Since | Γ | = m k − m , the state space of A is of size O ( n m k − m ) , where n = | Q | , and its alphabet is ofsize | ˆΣ | . Notice that for A to be nonempty, δ must be of size at least | (Σ ∪ | ( k − m ) , to account forall the permutations of letters in the words assigned to the variables under ∀ quantifiers (otherwise,we can immediately return “empty”). Therefore, | ˆ A| is O ( n · | Σ | k ) . We then have that the size of A is O ( | ˆ A| k ) . If the number k − m of ∀ quantifiers is fixed, then m k − m is polynomial in k . However,now | ˆ A| may be polynomial in n, k , and | Σ | , and so in this case as well, the size of A is O ( | ˆ A | k ) .Since the nonemptiness problem for NFA is NL-complete , the problem for NFH ∃∀ can be decided . Bonakdarpour and S. Sheinvald XX:19 in space of size that is polynomial in | ˆ A| . PSPACE hardness
For the lower bound, we show a reduction from a polynomial version of the corridor tiling problem , defined as follows. We are given a finite set T of tiles, two relations V ⊆ T × T and H ⊆ T × T , an initial tile t , a final tile t f , and a bound n > . We have to decidewhether there is some m > and a tiling of a n × m -grid such that (1) The tile t is in the bottom leftcorner and the tile t f is in the top right corner, (2) A horizontal condition: every pair of horizontalneighbors is in H , and (3) A vertical condition: every pair of vertical neighbors is in V . When n isgiven in unary notation, the problem is known to be PSPACE-complete .Given an instance C of the tiling problem, we construct an NFH ∃∀ A that is nonempty iff C has asolution. We encode a solution to C as a word w sol = w · w · w m $ over Σ = T ∪ { , , . . . n, $ } ,where the word w i , of the form · t ,i · · t ,i , . . . n · t n,i , describes the contents of row i .To check that w sol indeed encodes a solution, we need to make sure that: w begins with t and w m ends with t f $ . w i is of the correct form. Within every w i , it holds that ( t j,i , t j +1 ,i ) ∈ H . For w i , w i +1 , it holds that ( t j,i , t j,i +1 ) ∈ V for every ≤ j ≤ n .Verifying items − is easy via an NFA of size O ( n | H | ) . The main obstacle is item .We describe an NFH ∃∀ A = h T ∪ { , , , . . . n, $ } , { y , y , y , x , . . . x log( n ) } , Q, { q } , δ, F, α i that is nonempty iff there exists a word that satisfies items − . The quantification condition α is ∃ y ∃ y ∃ y ∀ x . . . ∀ x log( n ) . The NFH A only proceeds on letters whose first three positions are ofthe type ( r, , , where r ∈ T ∪ { , . . . n, $ } . Notice that this means that A requires the existenceof the words | w sol | and | w sol | (the word and word, henceforth). A makes sure that the wordassigned to y matches a correct solution w.r.t. items − described above. We proceed to describehow to handle the requirement for V . We need to make sure that for every position j in a row, the tilein position j in the next row matches the current one w.r.t. V . We can use a state q j to rememberthe tile in position j , and compare it to the tile in the next occurrence of j . The problem is avoidinghaving to check all positions simultaneously, which would require exponentially many states. To thisend, we use log( n ) copies of the and words to form a binary encoding of the position j that is tobe remembered. The log( n ) ∀ conditions make sure that every position within − n is checked.We limit the checks to words in which x , . . . x log( n ) are the or words, by having ˆ A acceptevery word in which there is a letter that is not over , in positions , . . . log( n ) + 3 . This takes careof accepting all cases in which the word assigned to y is also assigned to one of the x variables.To check that x , . . . x log( n ) are the or words, ˆ A checks that the values in positions to log( n ) + 3 remain constant throughout the run. In these cases, upon reading the first letter, ˆ A remembers the value j that is encoded by the constant assignments to x , . . . x log( n ) in a state, andmakes sure that throughout the run, the tile that occurs in the assignment y in position j in the currentrow matches the tile in position j in the next row.We construct a similar reduction for the case that the number of ∀ quantifiers is fixed: insteadof encoding the position by log( n ) bits, we can directly specify the position by a word of theform j ∗ , for every ≤ j ≤ n . Accordingly, we construct an NFH ∃∀ over { x, y , . . . y n , z } , witha quantification condition α = ∃ x ∃ y . . . ∃ y n ∀ z . The NFA ˆ A advances only on letters whoseassignments to y , . . . y n are always , , . . . n , respectively, and checks only words assigned to z that are some constant ≤ j ≤ n . Notice that the fixed assignments to the y variables leads to δ of polynomial size. In a hyperword accepted by A , the word assigned to x is w sol , and the wordassigned to z specifies which index should be checked for conforming to V . (cid:74) X:20 Automata for Hyperlanguages
Theorem 12
Proof.
We can decide the membership of S in L ( A ) by iterating over all relevant assignments from X to S , and for every such assignment v , checking on-the-fly whether zip ( v ) is accepted by ˆ A . Thisalgorithm uses space of size that is polynomial in k and logarithmic in |A| and in | S | .In the case that k = O (log k ) , an NP upper bound is met by iterating over all assignments to thevariables under ∀ , while guessing assignments to the variables under ∃ . For each such assignment v ,checking whether zip ( v ) ∈ L ( ˆ A ) can be done on-the-fly.We show NP-hardness for this case by a reduction from the Hamiltonian cycle problem. Given agraph G = h V, E i where V = { v , v , . . . , v n } and | E | = m , we construct an NFH ∃ A over { , } with n states, n variables, δ of size m , and a hyperword S of size n , as follows. S = { w , . . . , w n } ,where w i is the word over { , } in which all letters are except for the i ’th. The structure of ˆ A isidentical to that of G , and we set Q = F = { v } . For the transition relation, for every ( v i , v j ) ∈ E ,we have ( v i , σ i , v j ) ∈ δ , where σ i is the letter over { , } n in which all positions are except forposition i . Intuitively, the i ’th letter in an accepting run of ˆ A marks traversing v i . Assigning w j to x i means that the j ’th step of the run traverses v i . Since the words in w make sure that every v ∈ V istraversed exactly once, and that the run on them is of length n , we have that A accepts S iff thereexists some permutation p of the words in S such that p matches a Hamiltonian cycle in G . remark To account for all the assignments to the ∀ variables, δ – and therefore, ˆ A – must be of sizeat least k (otherwise, we can return “no”). We then have that if k = O ( k ) , then space of size k is logarithmic in | ˆ A| , and so the problem in this case can be solved within logarithmic space. Amatching NL lower bound follows from the membership problem for NFA. (cid:74) Theorem 13
Proof.
Let A = h Σ , P, P , ρ, F i be an NFA, and let A = h Σ , { x , . . . , x k } , Q, Q , δ, F , α i be anNFH.First, we construct an NFA A = h Σ ∪ { } , P , P , ρ , F i by extending the alphabet of A to Σ ∪ { } , adding a new and accepting state p f to P with a self-loop labeled by , and transitionslabeled by from every q ∈ F to p f . The language of A is then L ( A ) · ∗ . We describe a recursiveprocedure (iterating over α ) for deciding whether L ( A ) ∈ L ( A ) .For the case that k = 1 , it is easy to see that if α = ∃ x , then L ( A ) ∈ L ( A ) iff L ( A ) ∩ L ( ˆ A ) = ∅ .Otherwise, if α = ∀ x , then L ( A ) ∈ L ( A ) iff L ( A ) / ∈ L ( A ) , where A is the NFH for L ( A ) described in Theorem 8. Notice that the quantification condition for A is ∃ x , and so this conformsto the base case.For k > , we construct a sequence of NFH A , A , . . . , A k . If Q = ∃ then we set A = A ,and otherwise we set A = A . Let A i = h Σ , { x i , . . . , x k } , Q i , Q i , δ i , F i , α i i . If α i starts with ∃ ,then we construct A i +1 as follows.The set of variables of A i +1 is { x i +1 , . . . , x k } , and the quantification condition α i +1 is Q i +1 x i +1 · · · Q k x k , for α i = Q i x i Q i +1 · · · Q k x k . The set of states of A i +1 is Q i × P , and the setof initial states is Q i × P . The set of accepting states is F i × F . For every ( q ( σ i ,...,σ k ) −−−−−−→ q ) ∈ δ i and every ( p σ i −→ p ) ∈ ρ , we have (( q, p ) ( σ i +1 ,...,σ k ) −−−−−−−−→ ( q , p )) ∈ δ i +1 . Then, ˆ A i +1 accepts a word zip ( u , u , . . . , u k − i ) iff there exists a word u ∈ L ( A ) , such that ˆ A i accepts zip ( u, u , u , . . . , u k − i ) .Let v : { x i , . . . , x k } → L ( A ) . Then L ( A ) | = v ( α i , A i ) iff there exists w ∈ L ( A ) suchthat L ( A ) | = v [ x i → w ] ( α i +1 , A i ) . For an assignment v : { x i +1 , . . . , x k } → L ( A ) , it holds that zip ( v ) is accepted by ˆ A i +1 iff there exists a word w ∈ L ( A ) such that zip ( v ) ∈ L ( ˆ A i ) , where v is obtained from v by setting v ( x i ) = w . Therefore, we have that L ( A ) | = v [ x i → w ] ( α i , A i ) iff L ( A ) | = v ( α i +1 , A i +1 ) , that is, L ( A ) ∈ L ( A i ) iff L ( A ) ∈ L ( A i +1 ) . . Bonakdarpour and S. Sheinvald XX:21 If α i starts with ∀ , then we have that L ( A ) ∈ L ( A i ) iff L ( A ) / ∈ L ( A i ) . We construct A i for L ( A i ) as described in Theorem 8. The quantification condition of A i then begins with ∃ x i , and weapply the previous case, and construct A i +1 w.r.t. A i , to check for non-membership.Every ∀ quantifier requires complementation, which is exponential in n , the number of states in A . Therefore, in the worst case, the complexity of this algorithm is O (2 ... | Q || A | ) , where the tower isof height k . If the number of ∀ quantifiers is fixed, then the complexity is O ( | Q || A | k ) . (cid:74) Theorem 14
Proof.
For the lower bound, we show a reduction from the containment problem for NFA, which isknown to be
PSPACE-hard . Let A , A be NFA. We “convert” them to NFH A , A by adding toboth a single variable x , and a quantification condition ∀ x . By the semantics of the ∀ quantifier, wehave that L ( A ) = { S | S ⊆ L ( A ) } , and similarly for A . Therefore, we have that L ( A ) ⊆ L ( A ) iff L ( A ) ⊆ L ( A ) .For the upper bound, first notice that complementing an NFH ∀ yields an NFH ∃ , and vice versa.Consider two NFH A and A . Then L ( A ) ⊆ L ( A ) iff L ( A ) ∩ L ( A ) = ∅ . We can use theconstructions in the proof of Theorem 8 to compute a matching NFH A = A ∩ A , and check itsnonemptiness. The complementation construction is exponential in n , the number of states of A ,and the intersection construction is polynomial in |A | , |A | .If A ∈ NFH ∃ and A ∈ NFH ∀ or vice versa, then A is an NFH ∃ or NFH ∀ , respectively, whosenonemptiness can be decided in space that is logarithmic in |A| .Now, consider the case where A and A are both NFH ∃ or both NFH ∀ . It follows from the proofof Theorem 8, that for two NFH A , A , the quantification condition of A ∩ A may be any interleavingof the quantification conditions of A and A . Therefore, if A , A ∈ NFH ∃ or A , A ∈ NFH ∀ ,we can construct A to be an NFH ∃∀ . This is also the case when A ∈ NFH ∃∀ and A ∈ NFH ∃ or A ∈ NFH ∀ .Either A or A is an NFH ∀ , whose underlying NFA has a transition relation of size that isexponential in k (otherwise the NFH ∀ is empty). The same holds for A ∈ NFH ∃∀ . The PSPACEupper bound of Theorem 11 is derived from the number of variables and not from the state-space of theNFH. Therefore, while | ¯ A | is exponential in the number of states of A , checking the nonemptinessof A is in PSPACE . (cid:74) Lemma 16
Proof.
We begin with NFH ∀ . For the first direction, since L ( ˆ A ) ⊆ L ( ˆ A ∀ ) , we have L ( A ) ⊆ L ( A ∀ ) . For the second direction, let S ∈ L ( A ∀ ) . Then for every v : S → X , it holds that zip ( v ) ∈ L ( ˆ A ∀ ) . Also, zip ( v ) ∈ L ( ˆ A ∀ ) for every sequence v of v . Then zip ( v ) and all itssequences are in L ( ˆ A ) . Since this holds for every v : X → S , we have that S ∈ L ( A ) .We proceed to NFH ∃ . For the first direction, since L ( ˆ A ∃ ) ⊆ L ( ˆ A ) , we have L ( A ) ⊆ L ( A ) .For the second direction, let S ∈ L ( A ) . Then there exists v : S → X such that zip ( v ) ∈ L ( ˆ A ) .Then zip ( v ) is a permutation of some word zip ( v ) ∈ L ( ˆ A ∃ ) . According to the semantics of the ∃ quantifier, we have that S ∈ L ( A ∃ ) . (cid:74) Lemma 17
Proof.
We begin with NFH ∀ . To construct A given A ∀ , we use a similar construction to the onepresented in the proof of Theorem 11. Essentially, for every sequence ζ of (1 , , . . . , k ) , we constructan NFA A ζ , in which every run on a word w matches a run of ˆ A ∀ on w ζ . The NFH ∀ A is then X:22 Automata for Hyperlanguages obtained from A ∀ by replacing the underlying NFA with T ζ ∈ Γ A ζ , where Γ is the set of sequencesof (1 , , . . . , k ) .For NFH ∃ , similarly to the case of NFH ∀ , we construct A given A ∃ by constructing A ζ forevery permutation ζ of (1 , , . . . , k ) . In this case, the NFH ∃ A is obtained from A ∃ by replacing theunderlying NFA with S ζ ∈ Γ A ζ , where Γ is the set of permutations of (1 , , . . . , k ) . (cid:74) Theorem 18
Proof.
We begin with NFH ∀ . For the first direction, let w ∈ L ( ˆ A ∀ ) . Since A is sequence-complete,then w ∈ L ( ˆ A ) for every sequence w of w . Then, by the semantics of the ∀ quantifier, we have that unzip ( w ) ∈ L ( A ) . Therefore, unzip ( w ) ∈ L ( A ) , and so w (and all its sequences) are in L ( ˆ A ) . Asimilar argument can be made to show that for every w ∈ L ( ˆ A ) , it holds that w ∈ L ( ˆ A ) . Therefore, L ( ˆ A ) = L ( ˆ A ) . The second direction is trivial.We continue to NFH ∃ . For the first direction, let w ∈ L ( ˆ A ) . Then unzip ( w ) ∈ L ( A ) . Then, bythe semantics of the ∃ quantifier, there exists some permutation w of w such that w ∈ L ( ˆ A ) . Since A is permutation-complete, we have that w ∈ L ( ˆ A ) . A similar argument can be made to show thatfor every w ∈ L ( ˆ A ) , it holds that w ∈ L ( ˆ A ) . Therefore, L ( ˆ A ) = L ( ˆ A ) . The second direction istrivial.. The second direction istrivial.