aa r X i v : . [ m a t h . P R ] J un Submitted to the Brazilian Journal of Probability and Statistics arXiv: arXiv:1410.1976
Yaglom limit via Holley inequality
Pablo A. Ferrari and Leonardo T. Rolla
Universidad de Buenos Aires
Abstract.
Let S be a countable set provided with a partial order anda minimal element. Consider a Markov chain on S ∪ { } absorbed at with a quasi-stationary distribution. We use Holley inequality to obtainsufficient conditions under which the following hold. The trajectory ofthe chain starting from the minimal state is stochastically dominated bythe trajectory of the chain starting from any probability on S , when bothare conditioned to nonabsorption until a certain time. Moreover, theYaglom limit corresponding to this deterministic initial condition is theunique minimal quasi-stationary distribution in the sense of stochasticorder. As an application, we provide new proofs to classical results inthe field. This preprint has the same numbering of sections, equations and theoremsas the the published article “
Braz. J. Probab. Stat. 29 (2015), 413–426. ” Let S be a countable set, be an element outside S and consider a Markovchain X n ∈ S ∪ { } with transition matrix Q ( x, y ) , x, y ∈ S ∪ { } . Weassume that the matrix Q yields a chain which is absorbed at , meaningthat Q (0 ,
0) = 1 . We assume also Q ( x, < for all x ∈ S .For a probability measure ν on S , we define νT n as the conditional distribu-tion at time n of the chain started with law ν given that it is not absorbeduntil time n . More precisely, νT n ( y ) := νQ n ( y )1 − νQ n (0) , y ∈ S. (1.1) Keywords and phrases. quasi stationary distributions, Yaglom limit, quasi limitingdistributions, Holley inequality P.A. Ferrari and L.T. Rolla
A probability measure ν is a quasi-stationary distribution (or simply qsd ) if ν = νT (and thus νT n = ν for all n > ). Rewriting (1.1), a measure ν is aqsd if and only if ν ( y ) = X x ∈ S ν ( x ) (cid:2) Q ( x, y ) + Q ( x, ν ( y ) (cid:3) , y ∈ S. (1.2)The Yaglom limit of ν is lim n νT n , if the limit exists and is a probabilitymeasure. In this case the limit is called a quasi-limiting distribution .Assume that S has a partial order denoted and a minimal state called .Let ν ν ′ denote the stochastic order of measures on S induced by .We say that a probability measure µ on S n is irreducible if the set { ( x , . . . , x n ) ∈ S n : µ ( x , . . . , x n ) > } is connected in the sense that anyelement of S n with positive µ -probability can be reached from any othervia successive coordinate changes without passing through elements withzero µ -probability. We say that a Markov chain on S ∪ { } with transitionmatrix Q and initial distribution ν on S has irreducible trajectories in S if for each n > the measure µ on S n +1 defined by µ ( x , . . . , x n ) := ν ( x ) Q ( x , x ) . . . Q ( x n − , x n ) is irreducible.Let δ x be the probability distribution on S concentrated on the state x ∈ S . Theorem 1.
Let S be a partially ordered countable space with a minimalelement called 1. Let Q be the transition matrix of a Markov chain on S ∪{ } absorbed at . Assume that the chain with initial distribution δ hasirreducible trajectories in S . If, for all x, x ′ , z, z ′ ∈ S with x x ′ , z z ′ ,whenever the denominators are positive, Q ( x, · ) Q ( · , z ) Q ( x, z ) Q ( x ′ , · ) Q ( · , z ′ ) Q ( x ′ , z ′ ) , (1.3) Q ( x, · )1 − Q ( x, Q ( x ′ , · )1 − Q ( x ′ , , (1.4) as probability measures on S , then the following hold:i. The sequence ( δ T n ) n > is monotone: δ T n δ T n +1 , for all n > .ii. For any probability ν on S , δ T n νT n .iii. In particular, if ν is a qsd, then δ T n ν , for all n > .iv. If there is a qsd for Q , then the Yaglom limit of δ converges. The limitdistribution ν := lim n δ T n is a qsd and satisfies ν ν for any other qsd ν . aglom limit via Holley inequality The proof of Theorem 1 is an application of Holley inequality in the space offinite-length trajectories of the chain. Roughly speaking, Holley inequalitysays that the local dominations (1.3) and (1.4) imply that the conditionallaw of a length- n trajectory of the chain starting with δ given nonabsorptionby time n is stochastically dominated by the conditional law of a length- n trajectory of the chain starting from any other measure ν .In Section 2, we state and prove Holley inequality, and use it to proveTheorem 1. We then give a sufficient condition for ν to be the minimalqsd in the sense of absorption time rather than stochastic domination.Convergence of the Yaglom limit has been studied for birth-and-death chainsand one-dimensional random walks in both continuous and discrete time.Theorem 1 gives an alternative proof to many of these classical results.References to previous works and details of our approach to the aperiodiccases are discussed in detail in Section 3. Periodic chains are discussed inSection 4, after condition (1.3) is relaxed so as to include this case.There is a large literature on qsd’s compiled and periodically updated byPollett [Pol14]. We quote the recent book of Collet, Martínez and SanMartín [CMSM13] and the work of Kesten [Kes95] on Yaglom limits ofdiscrete-time Markov chains. For integers n < m let X mn := { ( x n , . . . , x m ) : x k ∈ S for n k m } bethe set of possible trajectories of the chain with transition matrix Q in thetime interval [ n, m ] which are not absorbed by 0 in that time interval.Let ν be a probability measure on S and define the measure µ mn ( ν, Q ) on X mn by µ mn ( ν, Q )( x mn ) := ν ( x n ) Q ( x n , x n +1 ) . . . Q ( x m − , x m )1 − νQ n (0) (2.1)where x mn = ( x n , . . . , x m ) . The measure µ mn ( ν, Q ) is the conditional distri-bution of the chain X mn = ( X n , . . . , X m ) with initial distribution ν at time n P.A. Ferrari and L.T. Rolla and transition probabilities Q , given that the chain is not absorbed duringthe time interval [ n, m ] .Due to the conditioning, the first marginal of µ mn ( ν, Q ) is not ν in general,but its last marginal is νT m − n . Indeed, by (1.1) and (2.1), νT m − n ( y ) = X ( x n ,...,x m − ) µ mn ( ν, Q )( x n , . . . , x m − , y ) . (2.2) Let Ω be a set endowed with a partial order . Let µ, µ ′ be probabilitymeasures on Ω . The stochastic domination µ µ ′ is equivalent to theexistence of a measure ˜ µ on Ω × Ω with marginals µ and µ ′ such that ˜ µ (( ω, ω ′ ) : ω ω ′ ) = 1 , see for instance [Lin99]. In this case we say that ˜ µ isa monotone coupling of µ and µ ′ .Let us endow X mn with the partial order given by the coordinate-wise orderof trajectories: x mn y mn if x k y k for all k ∈ [ n, m ] .A local domination condition for global domination of measures is providedby Holley Inequality [Hol74]. Here is a version suited to our context. Proposition 2 (Holley inequality) . Let S be a partially ordered countablespace. Let ν ν ′ be probabilities on S and let Q , Q ′ be transition matriceson S ∪ { } absorbed at . Denote the conditional laws of trajectories by µ = µ mn ( ν, Q ) and µ ′ = µ mn ( ν ′ , Q ′ ) , respectively. Assume that µ is an irreducibleprobability on the space of trajectories X mn . If, for all x, x ′ , z, z ′ ∈ S with x x ′ , z z ′ , whenever the denominators are positive, ν ( · ) Q ( · , z ) νQ ( z ) ν ′ ( · ) Q ′ ( · , z ′ ) ν ′ Q ′ ( z ′ ) (a) Q ( x, · ) Q ( · , z ) Q ( x, z ) Q ′ ( x ′ , · ) Q ′ ( · , z ′ ) Q ′ ( x ′ , z ′ ) (b) Q ( x, · )1 − Q ( x, Q ′ ( x ′ , · )1 − Q ′ ( x ′ , (c) as measures on S , then µ µ ′ . Holley inequality was proved in [GHM01] for finite state space S . We use theMarkovian structure of µ and condition (c) to get around this assumption. aglom limit via Holley inequality Proof.
Let ( η t : t > be the Gibbs sampler for µ , a Markov jump processon X mn with the following evolution: the value at each site k ∈ [ n, m ] isupdated at rate to a new value using the conditional distribution of µ giventhe configuration at the sites [ n, m ] \ { k } . Different sites are never updatedsimultaneously since they use independent Poisson clocks. This amounts touse the measures in the left hand side of (a), (b) and (c) to update sites n , [ n +1 , m − and m , respectively. The measure µ is reversible for η t . Analogously,let ( η ′ t : t > be the Gibbs sampler for µ ′ for which the updating is donewith the measures in the right hand side of (a), (b) and (c), respectively.The measure µ ′ is reversible for η ′ t .We will use the stochastic inequalities (a,b,c) to construct a monotonecoupling (( η t , η ′ t ) : t > of both Gibbs sampler processes. In this coupling,at rate 1 the value at each site in [ n, m ] is simultaneously updated for bothmarginal trajectories using a monotone coupling of the measures in (a), (b)and (c) to update sites n , [ n +1 , m − and m , respectively. If the trajectoriesare ordered at time 0, then they will remain ordered at future times, that is,if η η ′ then η t η ′ t for all t > . We thus need to find η η ′ .We claim that, given any trajectory z mn with positive µ ′ -probability, thereexists a trajectory x mn z mn with positive µ -probability. We prove this byconstructing x mn as follows. Since ν ′ ( z n ) > and ν ν ′ , it is possible tochoose x n z n such that ν ( x n ) > . Suppose that x n z n , . . . , x k z k have been chosen. Since Q ′ ( z k , z k +1 ) > , from condition (c) it is possible tochoose x k +1 z k +1 such that Q ( x k , x k +1 ) > . This proves the claim.The coupled process starts from ( η , η ′ ) , where η ′ is distributed with thereversible measure µ ′ and, given η ′ , a trajectory η η ′ with positive µ -probability is chosen according to the previous claim. Since η η ′ ,by the coupling we have η t η ′ t for all t > , and thus the law of η t is stochastically dominated by that of η ′ t . Irreducibility implies that η t converges in distribution to its unique invariant measure µ . On the otherhand, the distribution of η ′ t is µ ′ for all t . Letting t → ∞ , we get µ µ ′ . (cid:3) The proof of Holley inequality works also for the nonhomogeneous case.Consider a family o transition matrices Q = ( Q k , k ∈ Z ) and let ( X k ) be a (nonhomogeneous) Markov chain satisfying P ( X k +1 = y | X k = x ) = P.A. Ferrari and L.T. Rolla Q k ( x, y ) , that is, the transition matrix Q k is used at time k . Then we havethe following corollary of the proof of Proposition 2. Corollary 3.
Proposition 2 holds for nonhomogeneous families of transitionmatrices Q and Q ′ such that Q n and Q ′ n satisfy (a) , Q m − and Q ′ m − satisfy (c) , and that, for k = n + 1 , . . . , m − , Q k − ( x, · ) Q k ( · , z ) Q k − Q k ( x, z ) Q ′ k − ( x, · ) Q ′ k ( · , z ) Q ′ k − Q ′ k ( x, z ) . (b’) We first prove (ii) using Holley inequality with Q ′ = Q and ν = δ . The probability measure on the left-hand side of condition (a)is δ , and, since is minimal in S , this condition is satisfied for any ν ′ on S .Conditions (b) and (c) are being assumed in (1.3) and (1.4). Irreducibility of µ n ( δ , Q ) has also been explicitly assumed. By Holley inequality, µ n ( δ , Q ) µ n ( ν ′ , Q ) , which by (2.2) implies δ T n ν ′ T n , concluding the proof of (ii).If ν ′ is a qsd, then ν ′ = ν ′ T n . Together with (ii), this implies (iii).To prove (i), we introduce a nonhomogeneous chain Q forced to make thefirst jump into state 1 while the rest of the jumps are governed by Q . Let Q = ( Q k , − k n ) be given by Q − ( x,
1) = 1 for all x ∈ S and Q k = Q for k = 0 , . . . , n . By definition of Q − , the projection of µ n − ( δ , Q ) onto X n is µ n ( δ , Q ) . Hence, the time- n marginal of µ n − ( δ , Q ) is δ T n , thesame as the time- n marginal of µ − n ( δ , Q ) . Let Q ′ = ( Q ′ k , − k n ) begiven by Q ′ k = Q for k = − , . . . , n (that is, the homogeneous chain). Thetime- n marginal of µ n − ( δ , Q ′ ) is δ T n +1 , the same as the time- n marginalof µ n − ( δ , Q ) . Again, condition (c) has been assumed in (1.4). Writing ν = ν ′ = δ , condition (a) holds trivially. Condition (b’) is trivial for k = 0 , andfor k = 1 , , . . . , n − it is assumed in (1.3). Also, irreducibility of µ n ( δ , Q ) ,and thus of µ n − ( δ , Q ) , has been explicitly assumed. Using Corollary 3 weget µ n − ( δ , Q ) µ n − ( δ , Q ′ ) , and thus δ T n δ T n +1 , proving (i).To show (iv), let ν ′ be a qsd. By (i), δ T n is an increasing sequence ofmeasures and by (iii) all elements of the sequence are dominated by ν ′ .Hence there is a limit ν := lim n δ T n ν ′ . To check that ν is a qsd, use aglom limit via Holley inequality that T n is a semigroup and that T is continuous to get ν = lim n δ T n +1 = lim n δ T n T = νT . (cid:3) For a measure ν on S , denote a ( ν ) := 1 − νQ (0) , (2.3)the mass staying at S after one step for the chain starting with ν . If ν is aqsd, then ν is a left eigenvector for Q | S with eigenvalue a ( ν ) : νQ | S = a ( ν ) ν. Let a ∗ := inf { a ( ν ) : ν is a qsd } . If there exists a qsd ν with a ( ν ) = a ∗ , thenit is called minimal and denoted ν min .The following lemma gives sufficient conditions in terms of Q so that themeasure ν given by Theorem 1 coincides with ν min . Lemma 4. If Q is such that Q ( x, > Q ( x ′ , , for all x x ′ ∈ S, and ν is a qsd such that ν ν for any other qsd ν , then ν = ν min . Proof.
The function f : S → R + given by f ( y ) = Q ( y, is nonincreasing,whence νf > νf for any qsd ν . Thus, a ( ν ) a ( ν ) , and taking the infimumover ν we get a ( ν ) = a ∗ , which proves the lemma. (cid:3) In this section we consider S = N with the usual order and birth-and-deathprocesses. The transition matrix Q is defined by: p x , r x , q x > , q x + r x + p x = 1 , for all x > Q ( x, x −
1) = q x , Q ( x, x ) = r x , Q ( x, x + 1) = p x , (3.1) Q ( x, y ) = 0 if | x − y | > and Q (0 ,
0) = 1 . P.A. Ferrari and L.T. Rolla
In this case there exist a qsd if the absorption time of the chain startingfrom a fixed state has an exponential moment; see for instance van Doornand Schrijner [DS95a, Corollary 4.1], Ferrari, Martínez and Picco [FMP92,Theorem 6.1] and Ferrari, Kesten, Martínez and Picco [FKMP95]. Underthese conditions, Cavender [Cav78] shows that there is a critical value γ > such that there is a one-parameter family of qsd’s { ν : ν (1) ∈ (0 , γ ] } indexedby ν (1) . Cavender fixes ν (1) γ and computes explicitly the other valuesusing the equation (1.2) and the nearest-neighbor structure (this proceduredoes not yield a probability if ν (1) > γ ). Cavender also shows that anypair of qsd’s ν , ν ′ satisfy a monotone likelihood ratio: ν ′ (1) > ν (1) implies ν (1) ν ′ (1) < ν (2) ν ′ (2) ν (3) ν ′ (3) . . . , which in turn implies the domination ν ′ ν .Van Doorn and Schrijner [DS95b] use the Karlin and McGregor polynomialrepresentation of the chain to give a sufficient condition for the Yaglom limitto converge to an explicit limit. Ferrari, Martínez and Picco [FMP91] describethe domain of attraction of qsd’s and show in particular that the Yaglomlimit of δ x converges to the minimal qsd, for any initial state x . Daley [Dal69]and Iglehart [Igl74] showed the Yaglom limit for random walks with negativedrift and and finite variance, respectively for discrete and continuous space.In the sequel we develop Theorem 1’s conditions and make them explicit forthe case of birth-and-death chains. Corollary 7 is about space-homogeneousdiscrete-time random walks with delay.Item (iv) of Corollary 8 gives the Yaglom limit for continuous-time walks. Itwas originally proven by Seneta [Sen66] using direct computation. Our proofuses monotonicity of the trajectories instead.Corollary 10, presented in the next section, gives the Yaglom limit for thediscrete-time periodic chain. It provides an alternative proof to that of Senetaand Vere-Jones [SVJ66]. The conditions of Theorem 1
Since the state space S = N is totallyordered and the transitions are only to nearest neighbors, we can obtainconditions (1.3) and (1.4) in explicit terms of p k , r k and q k . Take Q as aglom limit via Holley inequality defined in (3.1). Define for positive integers x, z, y : b (( x, z ) , y ) := X w > y Q ( x, w ) Q ( w, z ) Q ( x, z ) ; (3.2) c ( x, y ) := X w > y Q ( x, w )1 − Q ( x, . (3.3)Conditions (1.3) and (1.4) are equivalent to b (( x, z ) , y ) b (( x ′ , z ′ ) , y ) , for z z ′ , x x ′ ; (3.4) c ( x, y ) c ( x ′ , y ) , for x x ′ , (3.5)for all y > , whenever the denominators of both sides are positive.Inequalities (3.4) hold trivially when y = 1 or { x, x ′ , z, z ′ } 6⊂ { y − , y } .The remaining cases are the following. For y > the conditions (3.4) areequivalent to the following conditions: b (( y − , y − , y ) b (( y, y − , y ) b (( y, y ) , y ) ,b (( y − , y − , y ) b (( y − , y ) , y ) b (( y, y ) , y ) . (3.6)Using the convention p = 0 , conditions (3.6) for y > read p y − q y r y − + p y − q y + q y − p y − r y q y q y r y − + r y q y r y + p y q y +1 r y + p y q y +1 + q y p y − ,p y − q y r y − + p y − q y + q y − p y − p y − r y p y − r y + r y − p y − r y + p y q y +1 r y + p y q y +1 + q y p y − . (3.7)Analogously, conditions (3.5) on c ( x, y ) hold trivially when y = 1 or ( x, x ′ ) =( y − , y ) . Hence, (3.5) is equivalent to c ( y − , y ) c ( y, y ) , y > , which in the case y = 2 and y > read, respectively, p p + r r + p , p y − p y + r y , for y > . (3.8)We summarize these computations as a lemma. P.A. Ferrari and L.T. Rolla
Lemma 5.
Let Q be the transition matrix for the birth-and-death chaindefined in (3.1) . Then conditions (1.3) and (1.4) are equivalent to (3.7) and (3.8) . We are ready to state the result in this case.
Corollary 6.
Assume that the birth-and-death chain absorbed at zero de-fined in (3.1) has at least one qsd and satisfies conditions (3.7) and (3.8) .Then (i,ii,iii,iv) of Theorem hold. The Yaglom limit of δ coincideswith ν min , the minimal qsd in the sense of absorption time. Furthermore,for any x ∈ N , the Yaglom limit of δ x also converges to ν min . Proof.
Since for x > the probability of transitions from x to x and tonearest neighbors of x are positive, the birth-and-death chain starting with δ has irreducible trajectories in N . By Lemma 5, the conditions of Theorem 1are equivalent to the present conditions, hence (i,ii,iii,iv) of Theorem 1 holdand the Yaglom limit of δ converges to ν . Since Q ( x,
0) = 0 for all x > ,Lemma 4 applies and ν = ν min . By [FMP91, Theorem 3.1], if the Yaglomlimit of δ exists, then it coincides with the Yaglom limit of δ x for any x ,concluding the proof. (cid:3) The absorbed delayed random walk is a particular case of birth-and-deathchain on N ∪ { } defined in (3.1) with constant transition probabilitiesalong N : p x ≡ p, q x ≡ q, r x ≡ r,p, q, r > , p + q + r = 1 , p < q. (3.9)This walk has a drift towards 0 and it is absorbed at 0. A probability ν on N is a qsd if and only if it satisfies the equations (1.2), which in this case are ν ( x + 1) q + ν ( x − p + ( qν (1) − ( p + q )) ν ( x ) = 0 , x > , (3.10)with the convention ν (0) = 0 . Cavender [Cav78] proved that the set of qsd’sis a family indexed by ν (1) with ν (1) ∈ (0 , (1 − √ λ ) ] , where λ = p/q .Since the absorption probability of a qsd ν is νQ (0) = qν (1) , the qsd with aglom limit via Holley inequality maximal ν (1) is the minimal qsd ν min , a negative binomial with parameters and √ λ : ν min ( x ) = (cid:0) − √ λ (cid:1) x (cid:0) √ λ (cid:1) x − , x > . (3.11)The remaining qsd are given in function of ν (1) ∈ (cid:0) , (cid:0) − √ λ (cid:1) (cid:1) by ν ( x ) = ν (1) c h(cid:16) λ + 1 − ν (1) + c (cid:17) x − (cid:16) λ + 1 − ν (1) − c (cid:17) x i , (3.12)where c = [( ν (1) − λ − − λ ] / , [Cav78, p. 585]. Corollary 7.
Consider the random walk with delay absorbed at zero definedin (3.1) with constant rates (3.9) . If pq r , then the conclusions (i,ii,iii,iv) of Theorem 1 hold with ν = ν min given by (3.11) . Furthermore, for any x ∈ N the Yaglom limit of δ x converges to ν . Proof.
In the present context the worst case of (3.7) is when y = 2 , whichreduces to: pqpq + r r + pqr + 2 pq . (3.13)On the other hand (3.8) reads pp + r r + p, p p + r. (3.14)Condition pq r implies both (3.13) and (3.14). The result thus followsfrom Corollary 6. (cid:3) If r < √ pq , then trajectory domination is not true. Although the Yaglomlimit of δ is known to hold in this case [Dal69], it does not seem to follow fromthe arguments presented here, except for the periodic case r = 0 discussedin Section 4. Take positive p < q with p + q = 1 and consider a family of random walkswith delay ( X rn ) , indexed by r ∈ [0 , , with transition probabilities Q r ( x, x −
1) = q (1 − r ) , Q r ( x, x ) = r, Q r ( x, x + 1) = p (1 − r ) ,Q r ( x, y ) = 0 , otherwise, x > Q r (0 ,
0) = 1 . P.A. Ferrari and L.T. Rolla
Define the rescaled process Y rt := X r [ t/ (1 − r )] . As r → , the process ( Y rt ) converges in finite time-intervals to the process ( ˆ Y t ) , a continuous-time random walk with rates p, q to jump one unit forwardor backwards, respectively, and absorbed at . Call ˆ U t the correspondingsemigroup: ˆ U t ( x, y ) := P ( ˆ Y t = y | ˆ Y = x ) . Define νT rt as the probability given by νT rt ( y ) := νQ [ t/ (1 − r )] r ( y )1 − νQ [ t/ (1 − r )] r (0) , that is, νT rt is the distribution at time t of the walk Y rt starting with ν , conditioned to nonabsorption. This distribution converges as r → tothe distribution at time t of the continuous-time walk ˆ Y t under the samecondition: lim r → νT rt ( y ) = ν ˆ T t ( y ) := ν ˆ U t ( y )1 − ν ˆ U t (0) . (3.15)The resulting operator ˆ T t is a semigroup. For any r ∈ [0 , , the qsd’s for Y rt satisfy equations (3.10) because the factors (1 − r ) cancel out. Moreover,the qsd’s for the continuous-time walk ˆ Y t also satisfy the same equations.Indeed, ν = ν ˆ T t if and only if ν ( ˆ U t − I ) + ν ˆ U t (0) · ν = 0; dividing by t andletting t → yields (3.10). As a consequence, the minimal qsd for both Y rt and ˆ Y t is given by (3.11) while the remaining qsd are given by (3.12). In thecontinuous-time case, p and q may be any positive real numbers satisfying p < q ; the definitions (3.11) and (3.12) depend on p and q only through theratio λ = p/q . Corollary 8.
The continuous-time random walk with rates p, q absorbed atzero satisfies:i. The sequence ( δ ˆ T t , t > is monotone: δ ˆ T s δ ˆ T t for < s t < ∞ .ii. If ν is a probability measure on N , then δ ˆ T t ν ˆ T t for all t > .iii. In particular, if ν is a qsd, then δ ˆ T t ν .iv. The Yaglom limit of δ converges to ν min given by (3.11) . aglom limit via Holley inequality Proof.
Take r sufficiently close to one so that pq (1 − r ) r , to be underthe conditions of Corollary 7.To show (i) we use Corollary 7(i) to get δ T rt δ T rt + s , for all t, s > , andthen use (3.15) to conclude. To prove (ii,iii), we use Corollary 7(ii,iii) to get δ T rt νT rt (which equals ν if it is a qsd), and again use (3.15) to conclude.Let us show (iv). As discussed above, ν min given by (3.11) is a qsd, the otherqsd’s are given by (3.12), and in particular ν min is minimal also in the senseof stochastic ordering. By (i,iii), there is ν = lim t δ ˆ T t . As in the proof ofTheorem 1, using the semigroup property of ˆ T t , the limit ν is a qsd. It followsfrom (iii) that ν ν min , and therefore ν = ν min . (cid:3) Assume that the matrix Q is irreducible in S and that Q restricted to S has period d > . Let S , . . . , S d ⊂ S be the cyclic subclasses, that is, theequivalence classes induced by the equivalence relation ∼ defined by x ∼ y ifand only if Q dℓ ( x, y ) > for some ℓ > . Assume that the classes are labeledso that x ∈ S j , Q ( x, y ) > implies y ∈ S j +1 ∪ { } (with the convention S d +1 = S ). Theorem 9.
Let S be a partially ordered countable set with a minimalelement called and let X n be a Markov chain on S ∪ { } , absorbed at ,irreducible in S and with period d when restricted to S . Let S , . . . , S d ⊂ S denote the cyclic subclasses of the chain restricted to S , choosing S ∋ .Assume that the chain with initial state has irreducible trajectories.If, for all x, x ′ , z, z ′ ∈ S with x x ′ , z z ′ , x and x ′ in the sameclass, the stochastic inequalities (1.3) and (1.4) are satisfied whenever thedenominators are positive, then the following hold.i. Monotonicity: δ T n δ T n + d for any n > .ii. For any probability ν on S , one has δ T n νT n .iii. If ν is a qsd, then δ T dk + j − ν ( · | S j ) for any k > .iv. If the chain has a qsd, then there is a qsd ν ⋆ such that the Yaglom limitof δ along d -periodic subsequences is given by lim k δ T dk + j − = ν ⋆ ( · | S j ) . P.A. Ferrari and L.T. Rolla
Moreover, for any other qsd ν , one has ν ⋆ ( · | S j ) ν ( · | S j ) for all j .v. If moreover Q is such that ν ν ′ implies a ( ν ) a ( ν ′ ) , then ν ⋆ = ν min . Before giving the proof, we discuss the particular case of the p - q randomwalk. As an application of the above theorem, we prove convergence of theYaglom limit to the minimal qsd based on monotonicity of trajectories.The p - q discrete-time random walk is defined as follows. Consider the peri-odic random walk with transition probabilities Q (0 ,
0) = 1 , Q ( x, x −
1) = q, Q ( x, x + 1) = p, for x > ,Q ( x, y ) = 0 , otherwise ; p + q = 1 , p < q. (4.1)The chain has period and, starting from δ , the walk visits odd sites ateven times and vice-versa. The qsd’s for this random walk satisfy (3.10) asbefore. The minimal qsd ν min is given by (3.11), and the remaining qsd’s aregiven by (3.12). The cyclic subclasses are S = 2 N − and S = 2 N . Corollary 10.
Let X n be the discrete-time p - q random walk with transitionprobabilities (4.1) . The Yaglom limit of δ converges along even and odd timesto projections of ν min given by (3.11) . That is, for both j = 1 , , lim n δ T n + j − = ν min ( · | S j ) . Moreover, ν min ( · | S j ) ν ( · | S j ) for any other qsd ν . Proof.
By Theorem 9(iv) there is ν ⋆ with the above properties, and theYaglom limit converges to projections of ν ⋆ along even or odd subsequences.By Theorem 9(v) we have ν ⋆ = ν min , concluding the proof. (cid:3) In order to prove Theorem 9, we start with some basic properties of qsd’sfor periodic chains. For a probability ν on S , write ν = P j m j ν j , where ν j := ν ( · | S j ) and P j m j = 1 . For shortness, let S j , m j and ν j be indexedby j ∈ Z d , so that S d +1 = S , etc. Recall that a ( ν ) is defined in (2.3). Lemma 11.
Let Q be the transition matrix for a d -periodic chain in S absorbed at 0. If ν is a qsd, then for each class j , m j a ( ν j ) = a ( ν ) m j +1 , and ν j T n = ν j + n , for all n > . In particular, ( a ( ν )) d = a ( ν ) . . . a ( ν d ) . aglom limit via Holley inequality Proof.
For any measure ν on S , ν j Q is supported on S j +1 ∪ { } , and thus ν j Q | S = a ( ν j ) ν j T . Hence, νQ | S = P j m j ν j Q | S = P j m j a ( ν j ) ν j T . Onthe other hand, if ν is a qsd, νQ | S = a ( ν ) ν = P j a ( ν ) m j ν j , and thus P j m j a ( ν j ) ν j T = P j a ( ν ) m j +1 ν j +1 . Now notice that, for each class j , themeasures ν j T and ν j +1 are probabilities supported on S j +1 , and these setsare disjoint. Therefore, ν j T = ν j +1 , and m j a ( ν j ) = a ( ν ) m j +1 . Iterating theformer identity, we get ν j T n = ν j T n = ν j + n , and taking the product over j of both sides of the latter, we get ( a ( ν )) d = a ( ν ) . . . a ( ν d ) . (cid:3) Proof of Theorem 9.
Under the present assumptions on the transitionmatrix, Holley inequality holds for any pair of measures supported on thesubspace of trajectories that start in a given cyclic subclass. Therefore,parts (i) and (ii) can be proved just as in the proof of Theorem 1.To prove (iii), let ν be a qsd. By (ii) and Lemma 11, δ T n ν T n = ν n . Claim (iii) follows by taking n = dk + j − .Proof of (iv). As in the proof of Theorem 1(iv), by (i,iii) the limits ν j := lim k δ T dk + j − exist and satisfy ν j T = ν j +1 , and moreover ν j ν j for any qsd ν . It remains to find the right constants m j and show that ν ⋆ given by ν ⋆ = P j m j ν j is a qsd, that is, that there existsan α ∈ (0 , such that ν ⋆ Q | S = αν ⋆ . Since ν j Q | S = a ( ν j ) ν j +1 , the problemis equivalent to find α, m , . . . , m d solving the system of d linear equationsgiven by m j a ( ν j ) = αm j +1 . The system has a nonzero solution if and only if α d = a ( ν ) · · · a ( ν d ) . Choosing the positive α that satisfies this identity, thespace of solutions is one-dimensional and its elements have coordinates whichagree in sign. Choosing m to be the unique solution to satisfy P j m j = 1 ,we have that ν ⋆ is a probability and moreover it is a qsd with a ( ν ⋆ ) = α ,concluding the proof of (iv).Proof of (v). It suffices to prove that a ( ν ⋆ ) a ( ν ) for all qsd ν . By Lemma 11, ( a ( ν ⋆ )) d = a ( ν ) . . . a ( ν d ) a ( ν ) . . . a ( ν d ) = ( a ( ν )) d , where the inequality comes from (iv) and the hypothesis of (v). (cid:3) P.A. Ferrari and L.T. Rolla
Acknowledgements
We thank Pablo Groisman for preliminary discussions and for calling ourattention to [Igl74]. We thank an anonymous referee for several suggestionsand comments which helped us to improve the presentation of this paper.This work is partially supported by Consejo Nacional de InvestigacionesCientíficas y Técnicas, Agencia Nacional de Promoción Científica y Tecnológ-ica and the project “Mathematics, computation, language and the brain”,FAPESP project “NeuroMat” (grant 2011/51350-6).
References [Cav78]
J. A. Cavender , Quasi-stationary distributions of birth-and-death processes ,Adv. Appl. Probab., 10 (1978), pp. 570–586.[CMSM13]
P. Collet, S. Martínez, and J. San Martín , Quasi-stationary distri-butions , Probability and its Applications (New York), Springer, Heidelberg,2013. Markov chains, diffusions and dynamical systems.[Dal69]
D. J. Daley , Quasi-stationary behaviour of a left-continuous random walk ,Ann. Math. Statist., 40 (1969), pp. 532–539.[DS95a]
E. A. van Doorn and P. Schrijner , Geometric ergodicity and quasi-stationarity in discrete-time birth-death processes , J. Austral. Math. Soc. Ser.B, 37 (1995), pp. 121–144.[DS95b] ,
Ratio limits and limiting conditional distributions for discrete-timebirth-death processes , J. Math. Anal. Appl., 190 (1995), pp. 263–284.[FKMP95]
P. A. Ferrari, H. Kesten, S. Martinez, and P. Picco , Existence ofquasi-stationary distributions. A renewal dynamical approach , Ann. Probab.,23 (1995), pp. 501–521.[FMP91]
P. A. Ferrari, S. Martinez, and P. Picco , Some properties of quasi-stationary distributions in the birth and death chains: a dynamical approach ,in Instabilities and nonequilibrium structures, III (Valparaíso, 1989), vol. 64of Math. Appl., Kluwer Acad. Publ., Dordrecht, 1991, pp. 177–187.[FMP92]
P. A. Ferrari, S. Martínez, and P. Picco , Existence of nontrivial quasi-stationary distributions in the birth-death chain , Adv. in Appl. Probab., 24(1992), pp. 795–813.[GHM01]
H.-O. Georgii, O. Häggström, and C. Maes , The random geometryof equilibrium phases , in Phase transitions and critical phenomena, Vol. 18,vol. 18 of Phase Transit. Crit. Phenom., Academic Press, San Diego, CA, 2001,pp. 1–142.[Hol74]
R. Holley , Remarks on the
FKG inequalities , Comm. Math. Phys., 36 (1974),pp. 227–231.aglom limit via Holley inequality [Igl74] D. L. Iglehart , Random walks with negative drift conditioned to stay posi-tive , J. Appl. Probability, 11 (1974), pp. 742–751.[Kes95]
H. Kesten , A ratio limit theorem for (sub) Markov chains on { , , · · · } withbounded jumps , Adv. in Appl. Probab., 27 (1995), pp. 652–691.[Lin99] T. Lindvall , On Strassen’s theorem on stochastic domination , Electron.Comm. Probab., 4 (1999), pp. 51–59 (electronic).[Pol14]
P. K. Pollett , Quasi-stationary distributions: A bibliography . , 2014.[Sen66] E. Seneta , Quasi-stationary behaviour in the random walk with continuoustime , Austral. J. Statist., 8 (1966), pp. 92–98.[SVJ66]
E. Seneta and D. Vere-Jones , On quasi-stationary distributions indiscrete-time Markov chains with a denumerable infinity of states , J. Appl.Probability, 3 (1966), pp. 403–434.
Departamento de MatemáticaFacultad de Ciencias Exactas y NaturalesUniversidad de Buenos AiresPabellón 1, Ciudad Universitaria1428 Ciudad de Buenos Aires, ArgentinaURL: http://mate.dm.uba.ar/ pferrari/ ;;