Parametric LTL on Markov Chains
aa r X i v : . [ c s . L O ] J un Parametric LTL on Markov Chains
Souymodip Chakraborty and Joost-Pieter Katoen ⋆ RWTH Aachen University, Ahornstraße 55, D-52074 Aachen, Germany
Abstract.
This paper is concerned with the verification of finite Markovchains against parametrized LTL (pLTL) formulas. In pLTL, the until-modality is equipped with a bound that contains variables; e.g., ♦ x ϕ asserts that ϕ holds within x time steps, where x is a variable on naturalnumbers. The central problem studied in this paper is to determine theset of parameter valuations V ≺ p ( ϕ ) for which the probability to satisfypLTL-formula ϕ in a Markov chain meets a given threshold ≺ p , where ≺ is a comparison on reals and p a probability. As for pLTL determin-ing the emptiness of V > ( ϕ ) is undecidable, we consider several logicfragments. We consider parametric reachability properties, a sub-logicof pLTL restricted to next and ♦ x , parametric B¨uchi properties andfinally, a maximal subclass of pLTL for which emptiness of V > ( ϕ ) isdecidable. Verifying a finite Markov chain (MC, for short) M against an LTL-formula ϕ amounts to determining the probability that M satisfies ϕ , i.e., the likelihoodof the set of infinite paths of M satisfying ϕ . Vardi [1] considered the qualita-tive version of this problem, that is, does M almost surely satisfy ϕ , or withpositive probability. Together with Wolper, he showed that the qualitative LTLmodel-checking problem for MCs is PSPACE-complete. The quantitative veri-fication problem – what is the probability of satisfying ϕ ? – has been treatedby Courcoubetis and Yannakakis [2]. An alternative algorithm that has a timecomplexity which is polynomial in the size of the MC and exponential in | ϕ | isby Couvreur et al. [3]. Recently, practical improvements have been obtained byChatterjee et al. for verifying the LTL(F,G)-fragment on MCs using generalizeddeterministic Rabin automata [4].This paper considers the verification of MCs against parametric LTL formu-las. In parametric LTL [5] (pLTL, for short), temporal operators can be sub-scripted by a variable ranging over the natural numbers. The formula ♦ x a means that in at most x steps a occurs, and (cid:3)♦ y a means that at every index a occurs within y steps. Note that x and y are variables whose value is not fixedin advance. The central question is now to determine the values of x and y suchthat the probability of a given MC satisfying the pLTL-formula ϕ meets a cer-tain threshold p . This is referred to as the valuation set V ≺ p ( ϕ ) for comparison ⋆ Currently on sabbatical leave at the University of Oxford, United Kingdom. operator ≺ . This problem has both a qualitative (threshold > < p < V > ( ϕ ) in gen-eral is undecidable. We therefore resort to fragments of pLTL. We show thatdetermining V > p ( ♦ x a ) can be done by searching in a range defined by the precision of the input, whereas polynomial time graph algorithms suffice for itsqualitative variant. The same applies to formulas of the form (cid:3)♦ x a . We pro-vide necessary and sufficient criteria for checking the emptiness of V > ( ϕ ) (and V =1 ( ϕ )) for the fragments pLTL(F,X) and pLTL ♦ , and prove that checking thesecriteria are NP-complete and PSPACE-complete, respectively. We also define arepresentation of these sets and provide algorithms to construct them. Related work.
The verification of parametric probabilistic models in which cer-tain transition probabilities are given as parameters (or functions thereof) hasrecently received considerable attention. Most of these works are focused on pa-rameter synthesis: for which parameter instances does a given (LTL or PCTL)formula hold? To mention a few, Han et al. [6] considered this problem for timedreachability in continuous-time MCs, Hahn et al. [7] and Pugelli et al. [8] forMarkov decision processes (MDPs), and Benedikt et al. [9] for ω -regular prop-erties of interval MCs. Hahn et al. [10] provide an algorithm for computing therational function expressing the probability of reaching a given set of states ina parametric (reward) MDP based on exploiting regular expressions as initiallyproposed by Daws [11]. Other related work includes the synthesis of loop invari-ants for parametric probabilistic programs [12]. To the best of our knowledge,verifying parametric properties on MCs has not been considered so far. Theclosest related works are on combining two-variable FO with LTL for MDPs byBenedikt et al. [13] and the computation of quantiles by Ummels and Baier [14]. Organization of the paper.
Section 2 presents pLTL and MCs and a first un-decidability result. Section 3 considers parametric reachability. Section 4 treatsthe fragment pLTL(F,X) and Section 5 parametric B¨uchi properties. Section 6treats the bounded always-free fragment of pLTL. Section 7 concludes the paper.
Parametric LTL.
Parametric LTL extends propositional LTL with bounded tem-poral modalities, for which the bound is either a constant or a variable. Let
Var be a finite set of variables ranged over by x, y , and AP be a finite set of propo-sitions ranged over by a and b . Let c ∈ IN. Parametric LTL formulas adhere tothe following syntax: ϕ ::= a | ¬ ϕ | ϕ ∧ ϕ | (cid:13) ϕ | ϕ U ϕ | ♦ ≺ x ϕ | ♦ ≺ c ϕ where ≺ ∈ { = , , <, >, > } . A pLTL structure is a triple ( w, i, v ) where w ∈ Σ ω with Σ = 2 AP is an infinite word over sets of propositions, i ∈ IN is an index, and v : Var → IN is a variable valuation. Analogously, we consider a valuation v asa vector in N d , where d for pLTL formula ϕ is the number of variables occurringin ϕ . E.g. for d = 1, the valuation is just a number v . We compare valuations v and v ′ as v v ′ iff v ( x ) v ′ ( x ) for all x . Let w [ i ] denote the i -th element of w .The satisfaction relation | = is defined by structural induction over ϕ as follows:( w, i, v ) | = a iff a ∈ w [ i ]( w, i, v ) | = ¬ ϕ iff ( w, i, v ) = ϕ ( w, i, v ) | = ϕ ∧ ϕ iff ( w, i, v ) | = ϕ and ( w, i, v ) | = ϕ ( w, i, v ) | = ♦ ≺ x ϕ iff ( w, j, v ) | = ϕ for some j ≺ v ( x )+ i. For the sake of brevity, we have omitted the semantics of the standard LTLmodalities. As usual, ϕ R ϕ ≡ ¬ ( ¬ ϕ U ¬ ϕ ), ♦ ϕ ≡ true U ϕ and (cid:3) ϕ ≡ ¬ ♦ ¬ ϕ .The language of ϕ is defined by L( ϕ ) = { ( w, v ) | ( w, , v ) | = ϕ } . Alur et al. [5]have shown that other modalities such as U x , ♦ >x , (cid:3) >x , U >x , R x and R >x ,can all be encoded in our syntax. For instance, the following equivalences hold: ♦ >x ϕ ≡ (cid:3) x ♦ (cid:13) ϕ, (cid:3) >x ϕ ≡ ♦ x (cid:3) (cid:13) ϕ,ϕ U x ψ ≡ ( ϕ U ψ ) ∧ ♦ x ψ, ϕ U >x ψ ≡ (cid:3) x ( ϕ ∧ (cid:13) ( ϕ U ψ )) (1)In the remainder of this paper, we focus on bounded always and eventualitieswhere all bounds are upper bounds. We abbreviate ♦ x by ♦ x and do similarfor the other modalities. For valuation v and pLTL-formula ϕ , let v ( ϕ ) denotethe LTL formula obtained from ϕ by replacing variable x by its valuation v ( x );e.g., v ( ♦ x ϕ ) equals ♦ v ( x ) v ( ϕ ). Markov chains.
A discrete-time Markov chain M is a quadruple ( S, P , s , L )where S is a finite set of states with m = | S | , P : S × S → [0 ,
1] is a stochasticmatrix, s ∈ S an initial state, and L : S → AP a state-labeling function. P ( u, v )denotes the one-step probability of moving from state u to v . A trajectory (orpath) of a Markov chain (MC, for short) M is a sequence { s i } i > such that P ( s i , s i +1 ) > i >
0. A trajectory π = s s s . . . induces the trace trace ( π ) = L ( s ) L ( s ) L ( s ) . . . . Let Paths ( M ) denote the set of paths of MC M .A path π satisfies the pLTL-formula ϕ under the valuation v , denoted π | = v ( ϕ ),whenever ( trace ( π ) , , v ) | = ϕ (or equivalently, ( trace ( π ) , v ) ∈ L ( ϕ )). A finitepath (or path fragment) satisfies a formula under a valuation if any infiniteextension of it also satisfies the formula. Let Pr be the probability measure onsets of paths, defined by a standard cylinder construction [1]. The probability ofsatisfying ϕ by M under valuation v is given by Pr { π ∈ Paths ( M ) | π | = v ( ϕ ) } ,generally abbreviated as Pr ( M | = v ( ϕ )). Valuation set.
The central problem addressed in this paper is to determine thevaluation set of a pLTL formula ϕ . Let M be an MC, p ∈ [0 ,
1] a probabilitybound, and ≺ ∈ { = , , <, >, > } . Then we are interested in determining: V ≺ p ( ϕ ) = { v | Pr ( M | = v ( ϕ )) ≺ p } , i.e., the set of valuations under which the probability of satisfying ϕ meets thebound ≺ p . In particular, we will focus on the decidability and complexity of theemptiness problem for V ≺ p ( ϕ ), i.e., the decision problem whether V ≺ p ( ϕ ) = ∅ or not, on algorithms (if any) determining the set V ≺ p ( ϕ ), and on the size of theminimal representation of V ≺ p ( ϕ ). In the qualitative setting, the bound ≺ p iseither >
0, or = 1.
Proposition 1.
For ϕ ∈ pLTL, the problem if V > ( ϕ ) = ∅ is undecidable.Proof. The proof is based on [5, Th. 4.1], see the appendix. (cid:4)
It follows that deciding whether V =1 ( ϕ ) = ∅ is undecidable, as V > ( ϕ ) = ∅ iff V =1 ( ¬ ϕ ) = ∅ . As a combination of ♦ x and (cid:3) x modalities can encode U = x ,e.g., ¬ a ∧ (cid:13) ( ¬ a U = x a ) ≡ (cid:13) ( ¬ a U x a ) ∧ ( ¬ a U >x a ) , we will restrict ourselves to fragments of pLTL where each formula is in negativenormal form and the only parametrized operator is ♦ x ϕ . We refer to thisfragment as pLTL ♦ : ϕ ::= a | ¬ a | ϕ ∧ ϕ | ϕ ∨ ϕ | (cid:13) ϕ | ϕ U ϕ | ϕ R ϕ | (cid:3) ϕ | ♦ x ϕ | ♦ c ϕ | (cid:3) c ϕ. (2)We show it is a sub-logic of pLTL for which the emptiness problem for V > ( ϕ )is decidable. The logic has a favourable monotonicity property, i.e., Remark 1.
For every pLTL ♦ -formula ϕ , infinite word w and valuations v , v ′ , v v ′ implies ( w, v ) | = ϕ = ⇒ ( w, v ′ ) | = ϕ .Here ( w, v ) | = ϕ is s shorthand for ( w, , v ) | = ϕ . We start off with briefly consid-ering (only) parametric eventualities and then consider the sub-logic pLTL(F,X)restricted to next and ♦ x . Later on, we also consider parametric B¨uchi formulas,and finally, pLTL ♦ . Most of the proofs are moved to the appendix. In this section, we consider pLTL-formulas of the form ♦ x a for proposition a ,or equivalently, ♦ x T for the set of target states T = { s ∈ S | a ∈ L ( s ) } . Weconsider bounds of the form > p with 0 < p <
1. The valuation set of interestis thus V > p ( ♦ x a ). Let µ i be the probability of reaching T within i steps; thesequence { µ i } is ascending. There can be two cases: (a) the sequence reaches aconstant value in m steps ( m being the size of Markov chain) or (b) the sequencemonotonically increases and converges to µ ∞ . This makes the emptiness problemfor V > p ( ♦ x a ) decidable. In the first case, we check µ m > p . In the second case,emptiness is decidable in time polynomial in m , by determining µ ∞ = Pr ( ♦ a )which can be done by solving a system of linear equations with at most m variables. Then, V > p ( ♦ x a ) = ∅ iff p < µ ∞ .Assume in the sequel that T is non-empty. Let min V > p ( ♦ x a ) = n . Thevaluation set can thus be represented by n (this gives a minimal representation of the set). Membership queries, i.e., does n ∈ V > p ( ♦ x a ), then simply boil downto checking whether n n , which can be done in constant time (modulo thesize of n ). The only catch is that n can be very large if p is close to µ ∞ . Asimple example elucidates this fact. Example 1.
Consider the MC M with S = { s , t } , L ( t ) = { a } , L ( s ) = ∅ , P ( s , s ) = = P ( s , t ) and P ( t, t ) = 1. Then Pr ( M | = ♦ n a ) = 1 − (cid:0) (cid:1) n . Itfollows that min V > p ( ♦ x a ) goes to infinity when p approaches one.The following bound on n can nonetheless be provided. This bound allows forobtaining the minimum value n by a binary search. Proposition 2.
For MC M , min V > p ( ♦ x a ) log γ (1 − (1 − γ ) pb ) , where < γ < and b > .Proof. Collapse all a -states into a single state t and make it absorbing (i.e.,replace all outgoing transitions by a self-loop with probability one). Let t be theonly bottom strongly connected component (BSCC) of M (other BSCCs can besafely ignored). Let { , . . . , m } be the states of the modified MC M , with theinitial state s and the target state t represented by 1 and m , respectively. Let Q be the ( m − × ( m −
1) transition matrix of the modified MC without thestate t . That is, Q ( i, j ) = P ( i, j ) iff j = m where P is the transition probabilitymatrix of M . We have the following observation:1. Let the coefficient of ergodicity τ ( Q ) of Q defined as τ ( Q ) = 1 − min i,j X k min { Q ( i, k ) , Q ( j, k ) } ! . As Q is sub-stochastic and no row of Q is zero, it follows 0 < τ ( Q ) < r T = ( r , . . . , r m − ) with r i = P ( i, m ), r max be the maximumelement in r and i T be (1 , , . . . , m from the state 1 in at most n +1 steps is the probability of being in somestate i < m within n steps and taking the next transition to m : µ n +1 = n +1 X j =0 i T Q j r n +1 X j =0 τ ( Q ) j r max . Let τ ( Q ) = γ and r max = b . The integer n is the smallest integer such that µ n > p , which implies that b · − γ n − γ > p . This yields n log γ (1 − (1 − γ ) pb ). (cid:4) As in the non-parametric setting, it follows that (for finite MCs) the valuationsets V > ( ♦ x a ) and V =1 ( ♦ x a ) can be determined by a graph analysis, i.e. noinspection of the transition probabilities is necessary for qualitative parametricreachability properties. Proposition 3.
The problem V > ( ♦ x a ) = ∅ is NL -complete. Proof.
The problem is the same as reachability in directed graphs. (cid:4)
Proposition 4.
The sets V > ( ♦ x a ) and V =1 ( ♦ x a ) can be determined in poly-nomial time by a graph analysis of MC M .Proof. Collapse all the a -states into a target state t and make t absorbing. If V > ( ♦ x a ) is non-empty, it suffices to determine min V > ( ♦ x a ) which equals thelength of a shortest path from s to t . To determine whether V =1 ( ♦ x a ) is emptyor not, we proceed as follows. If a cycle without t is reachable from s , then nofinite n exists for which the probability of reaching t within n steps equals one.Thus, V =1 ( ♦ x a ) = ∅ . If this is not the case, then the graph of M is a DAG(apart from the self-loop at t ), and min V =1 ( ♦ x a ) equals the length of a longestpath from s to t . (cid:4) This section considers the fragment pLTL(F,X) which is defined by: ϕ ::= a | ¬ a | ϕ ∧ ϕ | ϕ ∨ ϕ | (cid:13) ϕ | ♦ ϕ | ♦ x ϕ | ♦ c ϕ Our first result is a necessary and sufficient condition for the emptiness of V > ( ϕ ). Theorem 1.
For ϕ ∈ pLTL(F,X) and MC M with m states, V > ( ϕ ) = ∅ iff ¯ v ∈ V > ( ϕ ) with ¯ v ( x ) = m ·| ϕ | .Proof. Let ϕ be a pLTL(F,X)-formula and assume V > ( ϕ ) = ∅ . By monotonicity,it suffices to prove that v ∈ V > ( ϕ ) with v ¯ v implies ¯ v ∈ V > ( ϕ ). The proofproceeds in a number of steps. (1) We show that it suffices to consider formulaswithout disjunction. (2) We show that if path fragment π [0 ..l ] | = ¯ ϕ , (whereLTL(F,X)-formula ¯ ϕ is obtained from ϕ by omitting all parameters from ϕ )then π [0 ..l ] | = v l ( ϕ ) with v l ( x ) = l for every x . (3) We construct a deterministicB¨uchi automaton (DBA) A ¯ ϕ for ¯ ϕ such that its initial and final state are atmost | ¯ ϕ | transitions apart. (4) We show that reachability of a final state in theproduct of MC M and DBA A ¯ ϕ implies the existence of a finite path in M oflength at most m ·| ϕ | satisfying ¯ ϕ . See the appendix for details. (cid:4) The above Theorem 1 leads to the following proposition.
Proposition 5.
For ϕ ∈ pLTL(F,X), deciding if V > ( ϕ ) = ∅ is NP-complete.Proof. See the appendix. (cid:4)
For almost sure properties, a similar approach as for V > ( ϕ ) suffices. Theorem 2.
For ϕ ∈ pLTL(F,X) and MC M with m states, V =1 ( ϕ ) = ∅ iff ¯ v ∈ V =1 ( ¯ ϕ ) with ¯ v ( x ) = m ·| ϕ | . Proof.
Consider the direction from left to right. The argument goes along similarlines as the proof of Theorem 1. We build the DBA A ¯ ϕ for ¯ ϕ and take the crossproduct with Markov chain M . There are m ·| ϕ | state in the cross product. If Pr ( M | = ¯ v ( ϕ )) < V =1 ( ϕ ) is empty. (cid:4) Theorem 1 suggests that min V > ( ϕ ) lies in the hyper-cube H = { , . . . , N } d ,where N = m ·| ϕ | . A possible way to find min V > ( ϕ ) is to apply the bisectionmethod in d -dimensions. We recursively choose a middle point of the cube, say v ∈ H —in the first iteration v ( x ) = N/
2— and divide H in 2 d equally sizedhypercubes. If v ∈ V > ( ϕ ), then the hypercube whose points exceed v is dis-carded, else the cube whose points are below v is discarded. The asymptotictime-complexity of this procedure is given by the recurrence relation: T ( k ) = (2 d − · T ( k · − d ) + F (3)where k is the number of points in the hypercube and F is the complexity ofchecking v ∈ V > ( ϕ ) where | v | N . Section 6 presents an algorithm workingin O ( m · N d · | ϕ | ) for a somewhat more expressive logic. From (3), this yields acomplexity of O ( m · N d · | ϕ | · log N ). The size of a set of minimal points can beexponential in the number of variables, as shown below. Proposition 6. | min V > ( ϕ ) | ( N · d ) d − .Proof. See the appendix. (cid:4) r br br br b gx x x x x x Fig. 1.
MC and min V > ( ϕ ) for pLTL(F,X)-formula ϕ = ♦ x r ∧ ♦ x b ∧ ♦ x g Example 2.
There exist MCs for which | min V > ( ϕ ) | grows exponentially in d ,the number of parameters in ϕ , whereas the number m of states in the MC growslinearly in d . For instance, consider the MC M in Fig. 1 and ϕ = ♦ x r ∧ ♦ x b ∧ ♦ x g , i.e., d =3. We have | min V > ( ϕ ) | = 4 as indicated in the table.We conclude this section by briefly considering the membership query: does v ∈ V > ( ϕ ) for pLTL(F,X)-formula ϕ with d parameters? Checking membershipof a valuation v ∈ V > ( ϕ ) boils down to deciding whether there exists a v ′ ∈ min V > ( ϕ ) such that v > v ′ . A representation of min V > ( ϕ ) facilitating anefficient membership test can be obtained by putting all elements in this set inlexicographical order. This involves sorting over all d coordinates. A membershipquery then amounts to a recursive binary search over d dimensions. This yields: Proposition 7.
For pLTL(F,X)-formula ϕ , v ∈ V > ( ϕ )? takes O ( d · log N · d ) time, provided a representation of min V > ( ϕ ) is given. In this section, we consider pLTL-formulas of the form ϕ = (cid:3)♦ x a , for proposi-tion a . We are interested in V > ( ϕ ), i.e., does the set of infinite paths visiting a -states that are maximally x apart infinitely often, have a positive measure? LetMC M = ( S, P , s , L ). A bottom strongly-connected component (BSCC) B ⊆ S of M is a set of mutually reachable states with no edge leaving B . For BSCC B ,let n a,B = max { | π | | ∀ i | π | , π [ i ] ∈ B ∧ a / ∈ L ( π [ i ]) } . Proposition 8.
Let B be a BSCC and s ∈ B . Then, ∀ n ∈ N , n > n a,B ⇔ Pr ( s | = (cid:3)♦ n a ) = 1 and n n a,B ⇔ Pr ( s | = (cid:3)♦ n a ) = 0 .Proof. If n > n a,B , then each path π from any state s ∈ B will have at least one a -state in finite path fragment π [ i, . . . , i + n ] for all i . Hence, Pr ( s | = (cid:3)♦ n a ) = 1.If n n a,B , then there exists a finite path fragment ρ of B , such that, forall i n , a / ∈ L ( ρ [ i ]). Consider an infinite path π starting from any arbitrary s ∈ B . As s ∈ B , π will almost surely infinitely often visit the initial state of ρ .Therefore, by [15, Th.10.25], π will almost surely visit every finite path fragmentstarting in that state, in particular ρ . Path π thus almost surely refutes (cid:3)♦ n a ,i.e. Pr ( s | = (cid:3)♦ n a ) = 0. (cid:4) For any BSCC B and (cid:3)♦ x a , n a,B < ∞ iff every cycle in B has at least one a -state. Hence, n a,B can be obtained by analysing the digraph of B (in O ( m ),the number of edges). BSCC B is called accepting for (cid:3)♦ x a if n a,B < ∞ and B is reachable from the initial state s . Note that this may differ from being anaccepting BSCC for (cid:3)♦ a . Evidently, V > ( (cid:3)(cid:3)(cid:3)♦ x a ) = ∅ iff n a,B < ∞ . This resultcan be extended to generalized B¨uchi formula ϕ = (cid:3)♦ x a ∧ . . . ∧ (cid:3)♦ x d a d , bychecking n a i ,B < ∞ for each a i .As a next problem, we determine min V > ( (cid:3)♦ x a ). For the sake of simplicity,let MS M have a single accepting BSCC B . For states s and t in MC M , let d ( s, t ) be the distance from s to t in the graph of M . (Recall, the distance betweenstate s and t is the length of the shortest path from s to t .) For BSCC B , let d a,B ( s ) = min t ∈ B,a ∈ L ( t ) d ( s, t ), i.e., the minimal distance from s to an a -state in B . Let the proposition a B hold in state s iff s ∈ B and a ∈ L ( s ). Let G a = ( V, E )be the digraph defined as follows: V contains all a -states of M and the initialstate s and ( s, s ′ ) ∈ E iff there is path from s to s ′ in M . Let c be a cost functiondefined on a finite path s . . . s n in graph G a as: c ( s . . . s n ) = max i d ( s i , s i +1 ),( d is defined on the graph of M ). Using these auxiliary notions we obtain thefollowing characterization for min V > ( (cid:3)♦ x a ): Theorem 3. min V > ( (cid:3)♦ x a ) = n where n = max (cid:18) n a,B , min π = s ...s n ,s n | = a B c ( π ) (cid:19) if n a,B < d a,B ( s ) and n = n a,B otherwise.Proof. See the appendix. (cid:4)
If MC M has more than one accepting BSCC, say { B , . . . , B k } with k > n = min i n ,B i , where n ,B i for 0 < i k is obtained as in Theorem 3. Proposition 9.
The sets V > ( (cid:3)♦ x a ) and V =1 ( (cid:3)♦ x a ) can be determined inpolynomial time by a graph analysis of MC M .Proof. See the appendix. (cid:4)
Determining min V > p ( (cid:3)♦ x a ) for arbitrary p reduces to reachability of accepting BSCCs. In a similar way as for parametric reachability (cf. Section 3), this canbe done searching. For generalized B¨uchi formula ϕ = (cid:3)♦ x i a i ∧ . . . ∧ (cid:3)♦ x d a d and BSCC B , n a i B is at most m . Thus, min V > ( ϕ ) ∈ { , . . . , m · d } d and can befound by the bisection method, similar to the procedure described in Section 4. ♦ This section is concerned with the logical fragment pLTL ♦ , as defined in (2): ϕ ::= a | ¬ a | ϕ ∧ ϕ | ϕ ∨ ϕ | (cid:13) ϕ | ϕ U ϕ | ϕ R ϕ | (cid:3) ϕ | ♦ x ϕ. We will focus on the emptiness problem: is V > ( ϕ ) = ∅ . The decision problemwhether V =1 ( ϕ ) is very similar. Similar as for pLTL(F,X), we obtain necessaryand sufficient criteria for both cases. The proofs for these criteria depend on analgorithm that checks whether v ∈ V > ( ϕ ). This algorithm is presented first. Automata constructions.
Let ϕ be a pLTL ♦ -formula, and v a variable valuation.W.l.o.g. we assume that each variable occurs once in ϕ . We will extend theclassical automaton-based approach for LTL by constructing a nondeterministicB¨uchi automaton for ϕ that is amenable to treat the variables occurring in ϕ .To that end, inspired by [16], we proceed in a number of steps:1. Construct an automaton G ϕ for ϕ , independent from the valuation v , withtwo types of acceptance sets, one for treating until and release-modalities(as standard for LTL [17]), and one for treating the parameter constraints.2. Establish how for a given valuation v , a B¨uchi automaton B ϕ ( v ) can beobtained from G ϕ such that for infinite word w , ( w, v ) ∈ L( ϕ ) iff w is anaccepting run of B ϕ ( v ).3. Exploit the technique advocated by Couvreur et al. [3] to verify MC M versus B ϕ ( v ). The modalities ♦ c and (cid:3) c can be removed with only quadratic blow up.0 We start with constructing G ϕ . Like for the LTL-approach, the first step is toconsider consistent sets of sub-formulas of ϕ . Let cl ( ϕ ) be the set of all sub-formulas of ϕ . Set H ⊆ cl ( ϕ ) is consistent , when: – a ∈ H iff ¬ a H , – ϕ ∧ ϕ ∈ H iff ϕ ∈ H and ϕ ∈ H , – ϕ ∨ ϕ ∈ H iff ϕ ∈ H or ϕ ∈ H , – ϕ ∈ H implies ϕ U ϕ ∈ H , – ϕ , ϕ ∈ H implies ϕ R ϕ ∈ H , – ϕ ∈ H implies ♦ x ϕ ∈ H .We are now in a position to define G ϕ , an automaton with two acceptance sets.For ϕ ∈ pLTL ♦ , let G ϕ = ( Q, AP , Q , δ, Acc B , Acc P ) where – Q is the set of all consistent sub-sets of cl ( ϕ ) and Q = { H ∈ Q | ϕ ∈ H } . – ( H, a, H ′ ) ∈ δ , where a ∈ AP whenever: • H ∩ AP = { a } , • (cid:13) ϕ ∈ H ⇐⇒ ϕ ∈ H ′ , • ϕ U ϕ ∈ H ⇐⇒ ϕ ∈ H or ( ϕ ∈ H and ϕ U ϕ ∈ H ′ ), • ϕ R ϕ ∈ H ⇐⇒ ϕ ∈ H and ( ϕ ∈ H or ϕ R ϕ ∈ H ′ ), • ♦ x ϕ ∈ H ⇐⇒ ϕ ∈ H or ♦ x ϕ ∈ H ′ , – (generalized) B¨uchi acceptance Acc B and parametric acceptance Acc P : • Acc B = { F ϕ ′ | ϕ ′ ∈ cl ( ϕ ) ∧ ( ϕ ′ = ϕ U ϕ ∨ ϕ ′ = ϕ R ϕ ) } where ∗ F ϕ ′ = { H | ϕ ′ ∈ H ⇒ ϕ ∈ H } if ϕ ′ = ϕ U ϕ , and ∗ F ϕ ′ = { H | ϕ ∈ H ⇒ ϕ ′ ∈ H } if ϕ ′ = ϕ R ϕ , • Acc P = { F x i | ♦ x i ϕ i ∈ cl ( ϕ ) } with F x i = { H | ♦ x i ϕ i ∈ H ⇒ ϕ i ∈ H } .A run ρ ∈ Q ω of G ϕ is accepting under valuation v if it visits each set in Acc B infinitely often and each F x i ∈ Acc P in every infix of length v ( x i ). L( G ϕ )contains all pairs ( w, v ) such that there is an accepting run of w under thevaluation v . G ϕ is unambiguous if q a −→ q ′ and q a −→ q ′′ implies L( q ′ ) ∩ L( q ′′ ) = ∅ ,where L( q ) is the language starting from the state q . Proposition 10 ([16]).
For ϕ ∈ pLTL ♦ , the automaton G ϕ is unambiguousand L ( G ϕ ) = L ( ϕ ) . The automaton G ϕ can be constructed in O (2 | ϕ | ). Apart from the parametricacceptance condition, G ϕ behaves as a generalized B¨uchi automaton (GNBA)with accepting set Acc B = { F , . . . , F k } . In order to obtain a non-deterministicautomaton, we first apply a similar transformation as for GNBA to NBA [15].We convert G ϕ to U ϕ = ( Q ′ , AP , Q ′ , δ ′ , Acc ′ B , Acc ′ P ) where Q ′ = Q ×{ , . . . , k } , Q ′ = Q × { } . If ( q, a, q ′ ) ∈ δ , then (( q, i ) , a, ( q ′ , i ′ )) ∈ δ ′ with i = i ′ if q F i else i ′ = ( i mod k )+1. Acc B = F × { } and Acc ′ P = { F ′ x i | F x i ∈ Acc P } , where F ′ x i = F x i × { , . . . , k } . Note that the construction preserves unambiguity andthe size of U ϕ is in O ( | ϕ |· | ϕ | ).For a given valuation v , U ϕ can be converted into an NBA B ϕ ( v ). This isdone as follows. Let U ϕ = ( Q ′ , AP , Q ′ , δ ′ , Acc ′ B , Acc ′ P ) and v a valuation of ϕ with d parameters. Then B ϕ ( v ) = ( Q ′′ , AP , Q ′′ , δ ′′ , Acc ) with: – Q ′′ ⊆ Q ′ × { , . . . , v ( x ) } × . . . × { , . . . , v ( x d ) } , – (( q, n ) , a, ( q ′ , n ′ )) ∈ δ ′′ if ( q, a, q ′ ) ∈ δ ′ and for all x i : • if q ′ ∈ F ′ x i and n ( x i ) < v ( x i ) then n ′ ( x i ) = 0, • if q ′ / ∈ F ′ x i and n ( x i ) < v ( x i ) then n ′ ( x i ) = n ( x i ) + 1. – Q ′′ = Q ′ × d and Acc = Acc ′ B × { , . . . , v ( x ) } × . . . × { , . . . , v ( x d ) } .It follows that B ϕ ( v ) is unambiguous for any valuation v . Furthermore, everyrun of B ϕ ( v ) is either finite or satisfies the parametric acceptance condition forvaluation v . Thus we have: Proposition 11.
An infinite word w ∈ L ( B ϕ ( v )) if and only if ( w, v ) ∈ L ( ϕ ) . The size of B ϕ ( v ) is in O ( c v ·| ϕ |· | ϕ | ) where c v = Q x i ( v ( x i ) + 1).As a next step, we exploit the fact that B ϕ ( v ) is unambiguous, and applythe technique by Couvreur et al. [3] for verifying MC M against B ϕ ( v ). Let M ⊗ B ϕ ( v ) be the synchronous product of M and B ϕ ( v ) [15], Π the projectionto M and Π the projection to B ϕ ( v ). Let L( s, q ) = { π ∈ Paths ( s ) | trace ( π ) ∈ L( q ) } and Pr ( s, q ) = Pr ( L( s, q )). Let Pr ( M ⊗ B ϕ ( v )) = P q ∈ Q Pr ( s , q ). As B ϕ ( v ) is unambiguous, we have for any ( s, q ): Pr ( s, q ) = X ( t,q ′ ) ∈ δ ( s,q ) P ( s, t ) · Pr ( t, q ′ ) , where δ is the transition relation of M ⊗ B ϕ ( v ) and P ( s, t ) is the one-steptransition probability from s to t in MC M . A (maximal) strongly connectedcomponent (SCC, for short) C ⊆ S is complete if for any s ∈ Π ( C ) : Paths ( s ) = [ ( s,q ) ∈ C L C ( s, q )where L C ( s, q ) restricts runs to C (runs only visits states from C ). The SCC C is accepting if Acc ∩ Π ( C ) = ∅ (where Acc is the set of accepting states in B ϕ ( v )). Proposition 12 ([3]).
Let C be a complete and accepting SCC in M ⊗ B ϕ ( v ) .Then for all s ∈ Π ( C ) : Pr (cid:18) [ ( s,q ) ∈ C L C ( s, q ) (cid:19) = 1 . Moreover, since B ϕ ( v ) is unambiguous, Pr ( M ⊗ B ϕ ( v )) > implies there existsa reachable, complete and accepting SCC. Finding complete and accepting SCC in M ⊗ B ϕ ( v ) is done by standardgraph analysis. Altogether, v ∈ V > ( ϕ ) is decided in O ( m · c v ·| ϕ |· | ϕ | ). The spacecomplexity is polynomial in the size of the input (including the valuation), as M ⊗ B ϕ ( v ) can be stored in O (log m + | ϕ | + log c v ) bits. In the sequel, we exploitthese results to obtain a necessary and sufficient criterion for the emptiness of V > ( ϕ ) for ϕ in pLTL ♦ . Theorem 4.
For ϕ ∈ pLTL ♦ , V > ( ϕ ) = ∅ iff ¯ v ∈ V > ( ϕ ) s.t. ¯ v ( x ) = m ·| ϕ |· | ϕ | . Proof.
Consider the direction from left to right. The only non-trivial case iswhen there exists a valuation v ¯ v such that v ∈ V > ( ϕ ) implies ¯ v ∈ V > ( ϕ ).In the model checking algorithm described above, we first construct G ϕ , andthen U ϕ with a single B¨uchi accepting set Acc ′ B and d parametric acceptingsets F ′ x i , one for each variable x i in ϕ . For the sake of clarity, assume d = 1,i.e., we consider valuation v . The explanation extends to the general case in astraightforward manner. For valuation v , consider M ⊗ B ϕ ( v ). We show that,for r < v , Pr ( M ⊗ B ϕ ( v )) > Pr ( M ⊗ B ϕ ( r )) >
0, where r = m ·| U ϕ | ,which is in O ( m ·| ϕ |· | ϕ | ).Note that every cycle in M ⊗ B ϕ ( r ) contains a state ( s, q, i ) with i = 0.Moreover, the graph of M ⊗ B ϕ ( r ) is a sub-graph of M ⊗ B ϕ ( v ). We now provethat, if a (maximal) SCC C of M ⊗ B ϕ ( r ) is not complete (or accepting) thenany SCC C ′ of M ⊗ B ϕ ( v ) containing C is also not complete (or accepting,respectively).(a) Suppose C is not complete. Then there exists a finite path σ = s s . . . s k of M , such that from any q , with ( s, q, ∈ C , the run ρ = ( s, q, s , q , . . . ( s j ,q j , j ) leads to a deadlock state. This can have two causes: either ( s j , q j , j ) hasno successor for any j . Then, C ′ is not complete. Or, the path ρ terminatesin ( s j , q j , j ) where j = r . This means, for all ( s ′ , q ′ , j +1) ∈ δ ( s j , q j , j ) in C ′ , q ′ F x . As the length of ρ exceeds r , there are states in the run whose firstand second component appear multiple times. Thus, we can find another path σ ′ (possibly longer than σ ) for C ′ which goes through states where the first andthe second component of some of its states are repeated sufficiently many timesto have a run ( s, q, s , q , . . . ( s j , q j , v ) which is a deadlock state. Thus, C ′ is not complete.(b) Suppose C ′ is accepting. Then there exists ( s ′ , q ′ , i ′ ) with q ′ ∈ Acc . Since C ′ is an SCC and C ⊆ C ′ , there is a path from ( s, q, ∈ C to ( s ′ , q ′ , i ′ ). If thelength of the path is less than r , then we are done. If i ′ > r , then some ( s ′′ , q ′′ )pair in the path must be repeated. Thus, we can find another path of length lessthan r to a state ( s ′ , q ′ , i ), where i r . Therefore, C is accepting. The rest ofthe proof follows from Proposition 12. (cid:4) For almost sure properties, a similar approach as for V > ( ϕ ) suffices. Theorem 5.
For ϕ ∈ pLTL ♦ , V =1 ( ϕ ) = ∅ iff ¯ v ∈ V =1 ( ¯ ϕ ) with ¯ v ( x ) = m ·| ϕ |· | ϕ | . Let N ϕM = m ·| ϕ |· | ϕ | . Note that c ¯ v equals ( N ϕM ) d . Thus, we have: Proposition 13.
For ϕ ∈ pLTL ♦ , deciding if V > ( ϕ ) = ∅ is PSPACE-complete.Proof. Theorem 4 gives an algorithm in PSPACE, as M ⊗ B ϕ (¯ v ) can be storedin O (log m + | ϕ | + d log N ϕM ) bits. PSPACE hardness follows trivially, as forLTL formula ϕ and MC M , deciding Pr ( M | = ϕ ) > V > ( ϕ ). (cid:4) Just as for pLTL(F,X), we can use the bisection method to find min V > ( ϕ ).The search procedure invokes the model checking algorithm multiple times. Wecan reuse the space each time we check Pr ( M | = v ( ϕ )) >
0. Hence, min V > ( ϕ ) can be found in polynomial space. The time complexity of finding min V > ( ϕ ) is O ( m · ( N ϕM ) d · | ϕ | · log N ϕM ). Membership can also be similarly solved. Proposition 14.
For pLTL ♦ -formula ϕ , v ∈ V > ( ϕ )? takes O ( d · log N ϕM d ) time,provided a representation of V > ( ϕ ) is given. This paper considered the verification of finite MCs against parametric LTL.We obtained several results on the emptiness problem for qualitative verificationproblems, including necessary and sufficient conditions as well as some complex-ity results. Future work consists of devising more efficient algorithms for thequantitative verification problems, and lifting the results to extended temporallogics [18] and stochastic games, possibly exploiting [16].
Acknowledgement.
This work was partially supported by the EU FP7 projectsMoVeS and Sensation, the EU Marie Curie project MEALS and the Excellenceinitiative of the German federal government.
References
1. Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state pro-grams. In: FOCS, IEEE Computer Society (1985) 327–3382. Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. J.ACM (4) (1995) 857–9073. Couvreur, J.M., Saheb, N., Sutre, G.: An optimal automata approach to LTL modelchecking of probabilistic systems. In: LPAR. Volume 2850 of LNCS., Springer(2003) 361–3754. Chatterjee, K., Gaiser, A., Kret´ınsk´y, J.: Automata with generalized Rabin pairsfor probabilistic model checking and LTL synthesis. In: CAV. Volume 8044 ofLNCS., Springer (2013) 559–5755. Alur, R., Etessami, K., La Torre, S., Peled, D.: Parametric temporal logic for”model measuring”. ACM Trans. Comput. Log. (3) (2001) 388–4076. Han, T., Katoen, J.P., Mereacre, A.: Approximate parameter synthesis for prob-abilistic time-bounded reachability. In: IEEE Real-Time Systems Symposium(RTSS), IEEE Computer Society (2008) 173–1827. Hahn, E.M., Han, T., Zhang, L.: Synthesis for PCTL in parametric Markov decisionprocesses. In: NFM. Volume 6617 of LNCS., Springer (2011) 146–1618. Puggelli, A., Li, W., Sangiovanni-Vincentelli, A., Seshia, S.: Polynomial-time veri-fication of PCTL properties of MDPs with convex uncertainties. In: CAV. Volume8044 of LNCS., Springer (2013) 527–5429. Benedikt, M., Lenhardt, R., Worrell, J.: LTL model checking of interval Markovchains. In: TACAS. Volume 7795 of LNCS., Springer (2013) 32–4610. Hahn, E.M., Hermanns, H., Zhang, L.: Probabilistic reachability for parametricMarkov models. STTT (1) (2011) 3–1911. Daws, C.: Symbolic and parametric model checking of discrete-time Markov chains.In: ICTAC. Volume 3407 of LNCS., Springer (2005) 280–294412. Katoen, J.P., McIver, A., Meinicke, L., Morgan, C.C.: Linear-invariant generationfor probabilistic programs. In: Static Analysis Symposium (SAS). Volume 6337 ofLNCS., Springer (2010) 390–40613. Benedikt, M., Lenhardt, R., Worrell, J.: Two variable vs. linear temporal logic inmodel checking and games. Logical Methods in Computer Science (2) (2013)14. Ummels, M., Baier, C.: Computing quantiles in Markov reward models. In: FoS-SaCS. Volume 7794 of LNCS., Springer (2013) 353–36815. Baier, C., Katoen, J.P.: Principles of Model Checking. MIT Press (2008)16. Zimmermann, M.: Optimal bounds in parametric LTL games. Theor. Comput.Sci. (2013) 30–4517. Vardi, M.Y.: An automata-theoretic approach to linear temporal logic. In: Logicsfor Concurrency: Structure versus Automata. Volume 1043 of LNCS., Springer(1996) 238–26618. Vardi, M.Y., Wolper, P.: Reasoning about infinite computations. Information andComputation (1994) 1–3719. Alur, R., La Torre, S.: Deterministic generators and games for LTL fragments.ACM Trans. Comput. Log. (1) (2004) 1–2520. Sistla, A.P., Clarke, E.M.: The complexity of propositional linear temporal logics.J. ACM (3) (1985) 733–74921. Katerinochkina, N.: Sets containing a maximal number of pairwise incomparablen-dimensional k-ary sets. Mathematical notes of the Academy of Sciences of theUSSR (3) (1978) 696–7005 A Proposition 1.
The problem V > ( ϕ ) = ∅ is undecidable for ϕ ∈ pLTL. Proof.
The proof is based on [5, Th. 4.1], where the problem of deciding theexistence of a halting computation of a two-counter machine is reduced to thesatisfiability of ϕ .Let T be a counter machine with two counters { c , c } and k + 1 states { s , s , . . . , s k } , s being the initial state and s k the halting state. We constructa formula ϕ T such that any satisfiable structure ( w, v ), represent a sequence ofconfigurations of T that constitute a halting computation. In other words, thesequence of letters in the word w , will encode the an halting computation of T for the valuation v .Crucial argument is that, a parameter can be used to guess the maximumvalue of each counter in any halting computation of T . Thus using a parametersay x , each configuration of a halting computation can be stored in length x . Weuse propositions { p , . . . , p k } for each state. Let b be 0 (or 1) and ¯ b be 1 (or 0,respectively).Word w constains alternating sequence { q , q } denoting the start and endof a configuration. The distance between q , q is exactly x . This is imposed bythe formula: ϕ := ^ b (cid:18) q b → (cid:13) ( ¬ q ¯ b U = x q ¯ b ) ∧ (cid:13) ( q b U q ¯ b ) (cid:19) . The propositions { q − , q , q +01 } (or { q − , q , q +11 } ) will be used to keep track ofcounter c in the configuration starting with q (or q , respectively). Similarly, { q − , q , q +02 } , { q − , q , q +12 } do the same for counter c . We impose the con-dition that all these propositions occur exactly once between q and q , and { q − bi , q bi , q + bi } ( i = 1 ,
2) are always occur consecutively. ϕ bi := (cid:18) ( q b → ¬ q ¯ bi U q bi ) ∧ ( q bi → (cid:13) ( ¬ q bi U q b )) ∧ ( q + bi → (cid:13) ( ¬ q + bi U q b )) ∧ ( q − bi → (cid:13) ( ¬ q − bi U q b )) ∧ ( q bi → (cid:13) q + bi ) ∧ ( q − bi → (cid:13) q bi ) (cid:19) . Let ϕ := V b,i =1 , ϕ bi . Consider a configuration ( s i , c , c ) of T . If this configu-ration occur in a halting computation, then it is encoded in w as a sub-sequenceof propositions (of length x + 2) between q and q . Exactly one of the stateproposition, p i in this case is true at the start of the configuration q b . This isimposed by, ϕ := ^ b (cid:18) q b → P ∧ (cid:13) ( P ′ U q b ) (cid:19) where P := ( p ∧¬ p . . . ¬ p k ) ∨ . . . ∨ ( p k ∧¬ p . . . ¬ p k − ) and P ′ := ( ¬ p ∧ . . . ∧¬ p k ).The distance of q b from q b will be used to keep track of the value of c . To beprecise, at a distance c from q b the sequence q − b , q b , q + b occurs. Consider a transition e : s i c := c +1 −−−−−−→ s j . So if we are in a configuration wherethe distance of q − b from q b is c then in the next configuration, the distance of q − ¯ b from q ¯ b is c + 1 or the distance of q b to q − ¯ b is x . This can be encoded as: ϕ e := ^ b (cid:18) ( q b ∧ p i ) → ( ¬ q ¯ b U p j ∧ ¬ q ¯ b U ( q b → (cid:13) ( ¬ q b U = x q − ¯ b ))) (cid:19) A similar formula can be defined for transitions where the counter is decre-mented. For a transition where a counter value is compared to 0, e : s i c =0 −−−→ s j is encoded as: ϕ e := ^ b (cid:18) ( q b ∧ p i ) → ( ¬ q ¯ b U p j ∧ (cid:13) q − b ) (cid:19) . Thus, the entire transition relation of T can be encoded as ϕ := W e ϕ e . ϕ T := q ∧ ( ^ i =1 ϕ i ) U p k . As a satisfiable structure of ϕ T encodes a halting computation of T (vice-versa),satisfiability of ϕ T becomes undecidable. Furthermore, if ( w, v ) satisfies ϕ T then p k is true at some finite length of w . We can easily construct a Markov chain M such that the set of finite traces of M is Σ ∗ ( Σ ∗ is the set of sets of propositionused). We know that probability measure of any finite trace of M is greaterthan 0. Thus, we can decide whether ϕ T is satisfiable iff we can decide Pr ( M | = v ( ϕ T )) > v . Hence, we conclude that the emptiness problemof V > ( ϕ ) is undecidable. (cid:4) B Theorem 1.
For ϕ ∈ pLTL(F,X), V > ( ϕ ) = ∅ iff ¯ v ∈ V > ( ϕ ) with ¯ v ( x )= m ·| ϕ | . Proof.
The direction from right to left is trivial. Consider the other direction.Let ϕ be a pLTL(F,X)-formula and assume V > ( ϕ ) = ∅ . By monotonicity, itsuffices to prove that v ∈ V > ( ϕ ) with v ¯ v implies ¯ v ∈ V > ( ϕ ). The proofproceeds in a number of steps. (1) We show that it suffices to consider formulaswithout disjunction. (2) We show that if path fragment π [0 ..l ] | = ¯ ϕ , (whereLTL(F,X)-formula ¯ ϕ is obtained from ϕ by omitting all parameters from ϕ )then π [0 ..l ] | = v l ( ϕ ) with v l ( x ) = l for every x . (3) We construct a deterministicB¨uchi automaton (DBA) A ¯ ϕ for ¯ ϕ such that its initial and final state are atmost | ¯ ϕ | transitions apart. (4) We show that reachability of a final state in theproduct of MC M and DBA A ¯ ϕ implies the existence of a finite path in M oflength at most m ·| ϕ | satisfying ¯ ϕ .1. As disjunction distributes over ∧ , (cid:13)(cid:13)(cid:13) , ♦ , and ♦ x , each formula can be writtenin disjunctive normal form. Let ϕ ≡ ϕ ∨ . . . ∨ ϕ k , where each ϕ i is disjunction-free. Evidently, | ϕ i | | ϕ | . Assume v ∈ V > ( ϕ ). Then, v ∈ V > ( ϕ i ) for some < i k . Assuming the theorem holds for ϕ i (this will be proven below),¯ v i ∈ V > ( ϕ i ) with ¯ v i ( x ) = | ϕ i |· m . Since ¯ v > ¯ v i , it follows by monotonicitythat ¯ v ∈ V > ( ϕ i ), and hence, ¯ v ∈ V > ( ϕ ). It thus suffices in the remainderof the proof to consider disjunction-free formulas.2. For pLTL(F,X)-formula ϕ , let ¯ ϕ be the LTL(F,X)-formula obtained from ϕ by replacing all occurrences of ♦ x by ♦ , e.g., for ϕ = ♦ x ( a ∧ ♦ y b ), ¯ ϕ = ♦ ( a ∧ ♦ b ). We claim that π [0 ...l ] | = ¯ ϕ implies π [0 ...l ] | = v l ( ϕ ) with v l ( x ) = l for all x . This is proven by induction on the structure of ϕ . The base cases a and ¬ a are obvious. For the induction step, conjunctions, (cid:13) ϕ and ♦ ϕ arestraightforward. It remains to consider ♦ x ϕ . Assume π [0 ...l ] | = ♦ ¯ ϕ . Thus,for some i l , π [ i...l ] | = ¯ ϕ . By induction hypothesis, π [ i... ] | = v il ( ϕ ) with v il ( y ) = l − i for each variable y in ϕ . Thus, π [0 ..l ] | = v l ( ♦ x ϕ ) with v l ( x ) = l and for all y in ϕ , v l ( y ) = l .3. We provide a DBA A ¯ ϕ = h Q, Σ, δ, q , F i with Σ = 2 AP for each LTL(F,X)-formula ¯ ϕ using the construction from [19]. We first treat ¯ ϕ = a and ¯ ϕ = ♦ a .As every LTL(F,X)-formula can be obtained from ♦ ( a ∧ ϕ ), ϕ ∧ ϕ and (cid:13) ϕ ,we then treat these inductive cases. (Negations are treated similarly.) For¯ ϕ = a , A a = h{ q , q } , Σ, δ, q , { q }i with δ ( q , a ) = q and δ ( q , true) = q .For ¯ ϕ = ♦ a , the DBA A ♦ a = h{ q , q } , Σ, δ, q , { q }i , where δ ( q , a ) = q , δ ( q , ¬ a ) = q and δ ( q , true) = q . This completes the base cases. For thethree inductive cases, the DBA is constructed as follows.(a) Let A ¯ ϕ = h Q, Σ, δ, q , F i . A ♦ ( a ∧ ¯ ϕ ) = h Q ∪ { q ′ } , Σ, δ ′ , q ′ , F i where q ′ isfresh, δ ′ ( q, · ) = δ ( q, · ) if q ∈ Q , δ ′ ( q ′ , a ) = δ ( q , a ), and δ ′ ( q ′ , ¬ a ) = q ′ .(b) For ¯ ϕ ∧ ¯ ϕ , the DBA is a standard synchronous product of the DBAfor ¯ ϕ and ¯ ϕ .(c) Let A ¯ ϕ = h Q, Σ, δ, q , F i . A (cid:13) ¯ ϕ = h Q ∪ { q ′ } , Σ, δ ′ , q ′ , F i where q ′ isfresh, δ ′ ( q ′ , a ) = q for all a ∈ Σ and δ ′ ( q, a ) = δ ( q, a ) for every q ∈ Q .A few remarks are in order. The resulting DBA have a single final state.In addition, the DBA enjoy the property that the reflexive and transitiveclosure of the transition relation is a partial order [19]. Formally, q (cid:22) q ′ iff q ′ ∈ δ ∗ ( q, w ) for some w ∈ Σ ω . The diameter of A ¯ ϕ is the length of a longestsimple path from the initial to the final state. This implies that the diameterof A ♦ ( a ∧ ¯ ϕ ) and A (cid:13) ¯ ϕ is n +1 where n is this diameter of A ¯ ϕ , and the diameterof A ¯ ϕ ∧ ¯ ϕ is n + n where n i is the diameter of A ¯ ϕ i , i ∈ { , } .4. Let ϕ ≡ ϕ ∨ . . . ∨ ϕ k , where each ϕ i is disjunction-free, with DBA A ¯ ϕ i .Evidently, V > ( ϕ ) = ∅ iff V > ( ϕ i ) = ∅ for some disjunct ϕ i . Consider theproduct of MC M and DBA A ¯ ϕ i , denoted M ⊗ A ¯ ϕ i ; see, e.g., [15, Def.10.50]. By construction, M ⊗ A ¯ ϕ i is partially ordered and has diameter atmost m ·| ϕ i | . We have that Pr ( M | = ¯ ϕ i ) > M ⊗ A ¯ ϕ i is reachable. Thus, there exists a finite path π [0 ..m ·| ϕ i | ] in M with π [0 ..m ·| ϕ i ] | = ¯ ϕ , or, π [0 ..m ·| ϕ | ] | = ¯ v ( ϕ ). This concludes the proof. M ⊗ A ¯ ϕ i can also be used to show that, if we have a valuation v such that v ( x ) > m ·| ϕ | and for all other variables y = x , v ( x ) m ·| ϕ | and v ∈ V > ( ϕ )then v ′ ∈ V > ( ϕ ), where v ′ ( x ) = m ·| ϕ | and for y = x , v ′ ( y ) = v ( y ). Theargument proceed as induction on ¯ ϕ i . (cid:4) C Proposition 5.
The problem V > ( ϕ ) = ∅ is NP-complete for ϕ ∈ pLTL(F,X) Proof.
Similar to the NP-hardness proof of satisfiability of LTL(F,X) formu-las [20, Th. 3.7], we give a polynomial reduction from the 3-SAT problem. For3-CNF formula φ with boolean variables { t , . . . , t n } , we define MC M andpLTL(F,X) formula ϕ such that φ is satisfiable iff V > ( ϕ ) is not empty. Let 3-CNF formula φ = C ∧ . . . ∧ C k with C i = d i ∨ d i ∨ d i , where literal d il iseither t k or ¬ t k . Let MC M = ( S, P , s , L ) with AP = { C i | < i k } be: – S = { s i | i n } ∪ { t i | < i n } ∪ { ¬ t i | < i n } – P ( s i , t i +1 ) > P ( s i , ¬ t i +1 ) > i < n , P ( t i , s i ) > P ( ¬ t i , s i ) > < i n , and P ( s n , s n ) = 1 (the actual probabilities are not relevant), – C i ∈ L ( t j ) iff d il = t j for some 0 < l
3, and C i ∈ L ( ¬ t j ) iff d il = ¬ t j forsome 0 < l
3, and L ( s j ) = ∅ for all 0 j n .Let pLTL(F,X)-formula ϕ = ♦ y C ∧ . . . ∧ ♦ y k C k . Then φ is satisfiable iff V > ( ϕ )is not empty. Evidently, M and ϕ are obtained in polynomial time.It remains to show membership in NP. By the proof of Theorem 1, V > ( ϕ ) = ∅ iff there is a finite path of M of length m ·| ϕ | satisfying ¯ ϕ . Thus, we non-deterministically select a path of M of length m ·| ϕ | and check (using standardalgorithms) in polynomial time whether it satisfies ¯ ϕ . (cid:4) D Proposition 6. | min V > ( ϕ ) | ( N · d ) d − . Proof.
Let H = { , . . . , N } d . ( H, ) is a partially ordered set where iselement-wise comparison. A subset S ( k ) of H has rank k if the summation ofthe coordinates of every element of S is k . By [21], the largest set of incompa-rable elements (anti-chain) is given by Z ( k ) where k is N · d/ k is( N · d − /
2. Then | Z | = (cid:0) ⌊ N · d/ ⌋ + d − d − (cid:1) . (cid:4) E Theorem 3. min V > ( (cid:3)♦ x a ) = n where n = max (cid:18) n a,B , min π = s ...s n ,s n | = a B c ( π ) (cid:19) if n a,B < d a,B ( s ) and n = n a,B otherwise. Proof.
We show for n > n , Pr ( (cid:3)♦ n a ) >
0, and for n < n , Pr ( (cid:3)♦ n a ) = 0.Distinguish:1. n a,B > d a,B ( s ). Then, from s an a -state in B can be reached within n a,B steps, i.e., Pr ( s | = ♦ n a,B a B ) >
0. For this a B -state, s , say, by Proposition 8it follows Pr ( s | = (cid:3)♦ n a,B a ) = 1. Together this yields Pr ( s | = (cid:3)♦ n a ) > n > n a,B = n . For n < n = n a,B , it follows by Proposition 8 that Pr ( s | = (cid:3)♦ n a ) = 0 for every a B -state s . Thus, Pr ( s | = (cid:3)♦ n a ) = 0. n a,B < d a,B ( s ). As B is accepting, d a,B ( s ) = ∞ . Consider a simple path π from s to an a -state in B . Let c ( π ) be the maximal distance between twoconsecutive a -states along this path. Then it follows Pr ( s | = (cid:3)♦ k a ) > k = max( c ( π ) , n a,B ). By taking the minimum c min over all simplepaths between s and B , it follows Pr ( s | = (cid:3)♦ n a ) > n > n = max( n a,B , c min ) with c min = min π ∈ Paths ( s , ♦ a B ) c ( π ). For n < n ,distinguish between n = n a,B and n = c min . In the former case, it follows(as in the first case) by Proposition 8 that Pr ( s | = (cid:3)♦ n a ) = 0 for all n > n . Consider now n = c min > n a,B . Let n < n . By contra-position.Assume Pr ( s | = (cid:3)♦ n a ) >
0. Let π = s . . . s ,a . . . s ,a . . . . . . s k,a be a finitepath fragment in M where s i,a | = a and s k,a is the first a -state along π which belongs to B . Then, by definition of the digraph G a , the sequence π = s s ,a s ,a . . . s k,a is a path in G a satisfying c ( s i,a , s i +1 ,a ) n for all0 k < n . But then c min n . Contradiction. (cid:4) F Proposition 9.
The sets V > ( (cid:3)♦ x a ) and V =1 ( (cid:3)♦ x a ) can be determined inpolynomial time by a graph analysis of MC M . Proof.
We argue that min V > ( (cid:3)♦ x a ) can be determined in polynomial time.The proof for V =1 ( (cid:3)♦ x a ) goes along similar lines and is omitted here. We candetermine both n a,B and d a,B ( s ) in linear time. It remains to obtain c min =min π = s ...s n ,s n | = a B c ( π ) in case n a,B < d a,B ( s ). This can be done as follows.The distances d ( s, s ′ ), required for the function c in the digraph G a = ( V, E ),can be obtained by applying Floyd-Warshall’s all-pairs shortest path algorithmon the graph of M . This takes O ( m ). To obtain c min , we use a cost function F : V → IN which is initially set to 0 for initial state s and ∞ otherwise. Let pQ be a min priority queue, initially containing all vertices of G a , prioritized by thevalue of F . Algorithm 1 finds c min in O ( m · log m ). Its correctness follows from Algorithm 1
Input: MC M Output: c min
1: Initialize F , found := false and pQ .2: while ( ¬ found and pQ = ∅ ) do u := pop ( pQ ); found := ( a B ∈ L ( u ));4: for v ∈ pQ do F ( v ) := min ( F ( v ) , max( F ( u ) , c ( u, v )))5: end for end while the invariant F ( v ) max( F ( u ) , c ( u, v )). Using this we can find the minimum n for which we can reach an accepting BSCC via a finite path satisfying (cid:3)♦ n a . (cid:4) G Theorem 5.
For ϕ ∈ pLTL ♦ , V =1 ( ϕ ) = ∅ iff ¯ v ∈ V =1 ( ¯ ϕ ) with ¯ v ( x ) = m ·| ϕ |· | ϕ | . Proof.
Consider the direction left to right. If there exists a reachable maximalSCC C in the cross product which is not complete then Pr ( M | = ϕ ) <
1. Ifevery reachable maximal SCC is complete then the model checking task boilsdown to reachability of such SCC. Thus the existence of a cycle before reaching acomplete SCC implies that the probability measure of the set of paths satisfying ϕ is strictly less than 1 for any value of the parameters. The largest cycle in theproduct can have at most m ·| ϕ |· | ϕ | states. Thus, if Pr ( M | = ¯ v ( ϕ )) is less than1 then V =1 ( ϕ ) is empty.) is empty.