Approximation of the Exit Probability of a Stable Markov Modulated Constrained Random Walk
aa r X i v : . [ m a t h . P R ] S e p Approximation of the Exit Probability of a Stable MarkovModulated Constrained Random Walk
Fatma Ba¸so˘glu Kabran ∗ , Ali Devin Sezer † September 17, 2019
Abstract
Let X be the constrained random walk on Z ` having increments p , q , p´ , q , p , ´ q with jump probabilities λ p M k q , µ p M k q , and µ p M k q where M is an irreducible aperiodicfinite state Markov chain. The process X represents the lengths of two tandem queueswith arrival rate λ p M k q , and service rates µ p M k q , and µ p M k q . We assume that theaverage arrival rate with respect to the stationary measure of M is less than the averageservice rates, i.e., X is assumed stable. Let τ n be the first time when the sum of thecomponents of X equals n for the first time. Let Y be the random walk on Z ˆ Z ` havingincrements p´ , q , p , q , p , ´ q with probabilities λ p M k q , µ p M k q , and µ p M k q . Let τ be the first time the components of Y are equal. For x P R ` , x p q ` x p q ă x p q ą x n “ t nx u , we show that P p n ´ x n p q ,x n p qq ,m q p τ ă 8q approximates P p x n ,m q p τ n ă τ q with exponentially vanishing relative error as n Ñ 8 . For the analysis we define acharacteristic matrix in terms of the jump probabilities of p X, M q . The 0-level set ofthe characteristic polynomial of this matrix defines the characteristic surface; conjugatepoints on this surface and the associated eigenvectors of the characteristic matrix areused to define (sub/super) harmonic functions which play a fundamental role both in ouranalysis and the computation / approximation of P p y,m q p τ ă 8q . Keywords:
Markov modulation, regime switch, multidimensional constrained randomwalks, exit probabilities, rare events, queueing systems, characteristic surface, superhar-monic functions, affine transformation2010 Mathematics Subject Classification: Primary 60G50 Secondary 60G40;60F10;60J45
A stochastic processes X is said to be Markov modulated if its dynamics depend on thestate of a secondary Markov process M modeling the environment within which X operates[7]. Markov modulation/regime switch is one of the most popular methods of building richermodels for a wide range of applications from finance to computer networks to queueingtheory. This paper studies the approximation of the probability of a large excursion in thebusy cycle of a constrained random walk X whose dynamics are modulated by a Markovprocess M . We assume M to be external, i.e, the transition probabilities of M do not dependon X . Constrained random walks arise naturally when there are barriers that keep a process ∗ Middle East Technical University, Institute of Applied Mathematics, Ankara, Turkey and ˙Izmir KavramVocational School, Department of Finance, Banking and Insurance, ˙Izmir, Turkey † Middle East Technical University, Institute of Applied Mathematics, Ankara, Turkey X represents a queueingsystem, a large excursion corresponds to a buffer overflow event; the analysis, simulationand approximation of probabilities of such events for ordinary (non-modulated) constrainedrandom walks have received considerable attention at least since [6, 4]; for further referencesand a literature review we refer the reader to [11]. To the best of our knowledge, there ishardly any study on the same probability for modulated constrained random walks: we areaware of only [9] treating the development of asymptotically optimal importance samplingalgorithms for the approximation of the buffer overflow event. For this reason, this work willfocus on one of the simplest multidimensional constrained random walks, the tandem walk,arising from the modeling of two servers working in tandem. Next we describe the dynamicsof this process and give a precise definition of the buffer overflow probability of interest.Our main process is a random walk X with increments t I , I , I , ... u , constrained toremain in Z ` : X “ x P Z ` , X k ` . “ X k ` π p X k , I k q , k “ , , , ...π p x, v q . “ v, if x ` v P Z ` , , otherwise.The map π ensures that when X is on the constraining boundaries B i . “ t x P Z : x p i q “ u , i “ , , it cannot jump out of Z ` . We assume the distribution of the increments I k to be modulatedby a Markov Chain M with state space M (with finite size | M | ) and with transition matrix P P R | M |ˆ| M |` . To ease analysis and notation we will assume P to be irreducible andaperiodic, which implies that it has a unique stationary measure π on M , i.e., π “ πP . Let F k . “ σ pt M j , j ď k ` u , t X j , j ď k uq , i.e., the σ -algebra generated by M and X . Theincrements I form an independent sequence given M and the increment I k has the followingdistribution given F k ´ : I k P tp , q , p , q , p´ , q , p , ´ qu , P p I k “ p , q| F k ´ q “ t M k ‰ M k ´ u P p I k “ p , q| F k ´ q “ λ p M k q t M k “ M k ´ u P p I k “ p´ , q| F k ´ q “ µ p M k q t M k “ M k ´ u P p I k “ p , ´ q| F k ´ q “ µ p M k q t M k “ M k ´ u . The dynamics of X are shown in Figure 1. The process p X, M q is the embedded randomwalk of a continuous time queueing system consisting of two tandem queues whose arrivaland service rates are determined by a finite state Markov process M .We assume p X, M q to be stable: ÿ m P M p λ p m q ´ µ i p m qq π p m q P p m, m q ă , i “ , . (1)In addition to (1), we need two further technical assumptions for our analysis see (27) and(28). Stability means that the queueing system represented by p X, M q serves customers2 p m q M “ | M | M “ | M | ´ M “ M “ µ p m q B A n λ p m q Figure 1: Markov modulated constrained random walk p X, M q ; the left figure shows dy-namics in a given layer, the right figure shows jumps between layers representing regimeswitchesfaster, on average, than the customer arrival rate; this keeps the lengths of both queues closeto 0 at all times with high probability; but p X, M q being a random process, components of X can grow arbitrarily large if one waits long enough. For a stable constrained random walksuch as p X, M q it is natural to measure time in cycles that restart each time X hits 0. Ifthe system represented by this walk has a shared buffer where all customers wait (or wherepackets are stored, if, e.g., p X, M q represents a network of two computers / processes) then anatural question is the following: what is the probability that the shared buffer overflows ina given cycle? To express this problem mathematically we introduce the following notation:the region A n “ x P Z ` : x p q ` x p q ď n ( (2)and the exit boundary B A n “ x P Z ` : x p q ` x p q “ n ( . (3) A on denotes the interior A n ´ B Y B . Similarly, Z ,o ` denotes Z ` ´ B Y B . Let τ n be the firsttime X hits B A n : τ n . “ inf t k ě X k P B A n u , n “ , , , , .. (4)Then the buffer overflow probability described above is p n p x, m q . “ P p x,m q p τ n ă τ q . (5)The Markov property of p X, M q implies that p n is p X, M q -harmonic i.e., it satisfies p n p x, m q “ E p x,m q r p n p x ` π p x, I q , M qs , x P A n ´ B A n , p n p x, m q “ , x P B A n . This is a system of equations satisfied by p n , where the number of unknowns is in the order of | M | n . More generally, for a d dimensional system the number of unknowns grows like | M | n d ,making the computation of p n via a direct solution of the linear system resource intensiveeven for moderate values of n . This justifies the development of approximations of p n andthe main goal of the present work is to find easily computable and accurate approximationsof p n . Stability and the bounded increments of X suggest that when x is away from the exitboundary B A n , p n decays exponentially in n , making the buffer overflow event rare. Theapproximation of p n , even when there is no modulation turns out to be a nontrivial problem.3here are two sources of difficulty: multidimensionality, and the discontinuous dynamics ofthe problem on the constraining boundaries. Asymptotically optimal importance samplingalgorithms for the non-modulated setup were constructed in [2], which proposed a dynamicimportance sampling algorithm based on subsolutions of a related Hamilton Jacobi Bellman(HJB) and its boundary conditions. The approach of [2] is tightly connected to the largedeviations analysis of p n , which identifies the exponential decay rate of p n . Large deviationsanalysis is based on transforming p n to V n “ ´p { n q log p n , scaling space by 1 { n and takinglimits; the limit V of V n satisfies the HJB equation mentioned above. The works [10, 11]obtained sharp estimates of p n for the non-modulated two dimensional tandem walk using anaffine transformation of the process X ; see Figure 2 and the summary below. Another goalof the present work is to show that this affine transformation approach can be extended tothe analysis of p n of the Markov modulated constrained random walk. As the present articleshows, this extension turns out to be possible but Markov modulation complicates almostevery aspect of the problem: the underlying functions, the geometry of the characteristicsurfaces, the limit analysis, etc. A detailed comparison with the non-modulated case is givenin Section 10.To the best our knowledge, there is very limited research on the analysis of the overflowprobability p n for Markov modulated constrained random walks; we are only aware of thearticle [9] which develops asymptotically optimal importance sampling algorithms for theapproximation of p n for the p X, M q process studied in the present work. In doing this, anecessary step is also to compute the large deviation decay rate of p n ; this was also done for x “ The starting point of our analysis is transforming X to another process Y n by an affinetransformation moving the origin to the point p n, q on the exit boundary; as n goes toinfinity, Y n converges to the limit process Y constrained only on B ; Figure 2 shows thesetransformations. p Y, M q ne n ne n p X, M q T n n Ñ 8B B n B B B A n p Y n , M q Figure 2: The transformation of p X, M q The formal definition of the limit process Y is as follows: define I . “ ˆ ´ ˙ . π p y, v q “ v, if y ` v P Z ˆ Z ` , , otherwise.Then the limit process Y is the M -modulated constrained random walk on Z ˆ Z ` withincrements J k . “ I I k : (6) Y k ` “ Y k ` π p Y k , J k q . Define the region
B . “ t y P Z ˆ Z ` : y p q ě y p qu and the exit boundary B B . “ t y P Z ˆ Z ` : y p q “ y p qu . Let τ be the hitting time τ . “ inf t k ě Y k P B B u .Y is a process constrained to Z ˆ Z ` with the constraining boundary B ; we will denote theinterior Z ˆ Z ` ´ B of this set by Z ˆ Z o ` . Define the affine transformations T n “ ne ` I where p e , e q is the standard basis for R . Our main approximation result is the following:
Theorem (Theorem 6.1) . For any x P R ` , x p q ` x p q ă , x p q ą , and m P M thereexist constants c ą , ρ P p , q and N ą such that | P p x n ,m q p τ n ă τ q ´ P p T n p x n q ,m q p τ ă 8q| P p x n ,m q p τ n ă τ q ă ρ cn , (7) for n ą N , where x n “ t nx u . Theorem 6.1 states that, as n increases, P p T n p x n q ,m q p τ ă 8q gives a very good approxi-mation of P p x n ,m q p τ n ă τ q . Parallel to the non-modulated case treated in [11], the proof ofTheorem 6.1 consists of the following steps 1) the difference between the events t τ n ă τ u and t τ ă 8u can be characterized by the event “ X first hits B then B and then B A n ” 2)the probability of this detailed event is very small compared to the probabilities of the events t τ n ă τ u and t τ ă 8u . The challenges arise from the implementation of these steps in theMarkov modulated framework.To bound the probabilities appearing in (7) we will use p Y, M q -(super)harmonic functionsconstructed from single and conjugate points on a characteristic surface H (see (14)) associ-ated with p Y, M q . The characteristic surface is the 0-level set of the characteristic polynomialof the characteristic matrix A (see (12)) defined in terms of the transition matrix P and thejump probabilities λ p¨q , µ p¨q and µ p¨q . The characteristic polynomial is of degree 3 | M | andtherefore the characteristic curve doesn’t have a simple algebraic parametrization; for thisreason, in the modulated case, the identification of points on the characteristic surface relieson the decomposition of the the surface into | M | components, by an eigenvalue analysis of A and the implicit function theorem. The decomposition is given in subsection 2.1 and thepoints relevant for our analysis are identified in Propositions 2.8, 2.11 and 2.12. These pointsall lie on the innermost component corresponding to the largest eigenvalue of A .
5n the presence of a modulating Markov chain, harmonic functions are constructed ingeneral from | M | ` P p y,m q p τ ă 8q usingthese functions is given in Section 3. Section 4 constructs an upper bound for the detailedevent described above characterizing the difference of the events t τ n ă τ u and t τ ă 8u .A lower bound for P p x,m q p τ n ă τ q based on subharmonic functions constructed from thefunctions of Section 2 is given in Section 5. These elements are combined in Section 6 toprove our main approximation theorem, Theorem 6.1.With Theorem 6.1 we know that P p x,m q p τ n ă τ q can be approximated very well with P p T n p x q ,m q p τ ă 8q . In the non-modulated case, a linear combination of two Y -harmonicfunctions constructed from points on the characteristic surface gives an exact formula for P y p τ ă 8q . This is no longer possible when there is modulation; Sections 7 and 8 devel-ops increasingly accurate approximate formulas for P p y,m q p τ ă 8q using p Y, M q -harmonicfunctions constructed from further points on the characteristic surface under further linearindependence assumptions (see (77), (89)), see Propositions 7.1 and Propositions 8.1 for the p Y, M q -harmonic functions constructed in these sections. As opposed to the limit analy-sis which uses points only on the innermost component of the characteristic surface, theconstruction of harmonic functions uses points on all components of the characteristic sur-face. Propositions 7.2 and 8.2 find bounds on the relative error of the approximations of P p y,m q p τ ă 8q provided by these functions based on the values they take on B B. Section 9gives a numerical example showing the effectiveness of the resulting approximations. Section10 compares the analysis of the current work with the non-modulated tandem walk treated in[10, 11] and the non-modulated parallel walk treated in [12]. Section 11 comments on futurework. p Y, M q A function h on Z ˆ Z ` ˆ M is said to be p Y, M q -harmonic if E p y,m q r h p Y , M qs “ h p y, m q , p y, m q P Z ˆ Z ` ˆ M ; (8)if we replace “ with ě [ ď ], h is said to be p Y, M q -subharmonic [superharmonic].For the case | M | “ Y -harmonic functions which arelinear combinations of exponential functions y ÞÑ rp β, α q , y s “ β y p q´ y p q α y p q , p β, α q P C , (9)and p β, α q lies on a characteristic surface associated with the process. Markov modulationintroduces an additional state variable m , which leads to the following generalization of (9) p y, m q ÞÑ β y p q´ y p q α y p q d p m q , (10)where d : M ÞÑ C is an arbitrary function on M . Let rp β, α, d q , ¨s denote the function given(10). We would like to choose p β, α, d q so that rp β, α, d q , ¨s is p Y, M q -harmonic at least over theinterior Z ˆ Z o ` . To this end, introduce the local characteristic polynomial for the modulatingstate m P M : p p β, α, m q . “ λ p m q β ` µ p m q α ` µ p m q βα ; (11)6o define the global characteristic polynomial introduce the | M | ˆ | M | matrix A : A p β, α q m ,m . “ P p m , m q , m ‰ m , P p m , m q p p β, α, m q , m “ m , p m , m q P M . Let I denote the | M | ˆ | M | identity matrix. Attempting to find functionsof the form rp β, α, d q , ¨s that satisfy (8) leads to the following characteristic equation: A p β, α q d “ d , (12)i.e, p p β, α q . “ det p I ´ A p β, α qq “ , (13)and d is an eigenvector of A p β, α q for the eigenvalue 1. The p p¨ , ¨q of (13) is the globalcharacteristic polynomial for the modulated process p Y, M q . Define the characteristic surfacefor the interior: H . “ ! p β, α, d q P C `| M | : A p β, α q d “ d , d ‰ ) . (14)Points on H give us p Y, M q -harmonic functions on Z ˆ Z o ` . Proposition 2.1. If p β, α, d q P H then rp β, α, d q , ¨s satisfies (8) for y P Z ˆ Z o ` . Proof.
By definition E p y,m q rp β, α, d q , p Y , M qs“ ÿ n P M ,n ‰ m P p m, n qrp β, α, d q , p y, n qs` P p m, m qp λ p m qrp β, α, d q , p y ` p´ , q , m qs ` µ p m qrp β, α, d q , p y ` p , q , m qs` µ p m qrp β, α, d q , p y ` p , ´ q , m qsq Expand rp β, α, d q , pp y ` v q , m qs terms: “ ÿ n P M ,n ‰ m P p m, n qrp β, α, d q , p y, n qs` P p m, m qp λ p m q β y p q´ y p q´ α y p q d p m q ` µ p m q β y p q´ y p q α y p q` d p m q` µ p m q β y p q´ y p q` α y p q´ d p m qq Factor out rp β, α, d q , p y, m qs from the last three terms: “ ÿ n P M ,n ‰ m P p m, n qrp β, α, d q , p y, n qs ` P p m, m qrp β, α, d q , p y, m qs p p β, α, m q“ β y p q´ y p q α y p q ˜ ÿ n P M ,n ‰ m P p m, n q d p n q ` P p m, m q d p m q p p β, α, m q ¸ . The expression in parenthesis equals the m th term of the vector A p β, α q d , which equals d p m q because p β, α, d q P H means A p β, α q d “ d . Therefore, “ β y p q´ y p q α y p q d p m q “ rp β, α, d q , p y, m qs . This proves the claim of the proposition.The previous proposition gives us p Y, M q -harmonic functions on Z ˆ Z o ` . we next studythe geometry of H , this will be useful in defining p Y, M q -(super/sub) harmonic functions overall of Z ˆ Z ` . .1 Geometry of the characteristic surface Define H βα , the projection of H onto its first two dimensions: H βα . “ tp β, α q P C : p p β, α q “ u ;we will to refer to H βα as the characteristic surface for the interior as well, which is justifiedby the next lemma; its proof follows from basic linear algebra. Lemma 2.1.
For each p β, α q P H βα there is at least one parameter family of points tp β, α, c d q , c P C ´ t uu Ă H , for some d P C | M | ´ t u . Conversely, for each p β, α, d q P H ,we have p β, α q P H βα . Furthermore, all points on H can be obtained from those on H βα .β | M | α | M | p is a polynomial of degree 3 | M | in p β, α q , which makes, in general, the analysisof the geometry of H βα nontrivial. A natural approach to the study of the geometry ofthis curve is through the eigenvalues of A p β, α q . The next two propositions show that thecurve H βα decomposes into | M | distinct pieces over any region D where A p β, α q has simpleeigenvalues. Proposition 2.2.
Let D Ă C or D Ă R be open and simply connected and suppose A p β, α q has simple eigenvalues for all p β, α q P D. Then the eigenvalues of A can be written as | M | distinct smooth functions Λ j p β, α q on D .Proof. The argument is the same for both real and complex variables. That the eigenvaluesΛ j can be defined smoothly in a neighborhood of any p β, α q P D follows from [8, Theorem5.3] and the assumption that they are distinct. Once defined locally, one extends them to allof D through continuous extension, which is possible because D is simply connected.Most of our analysis will be based on p β, α q P D “ R ,o ` . “ R ` ´ B Y B . For p β, α q P R ,o ` , A p β, α q is an irreducible matrix with positive entries. Perron-Frobenius Theorem implies that A p β, α q has a simple positive eigenvalue dominating all of the other eigenvalues in absolutevalue with an eigenvector with strictly positive entries. Λ p β, α q will always denote thislargest eigenvalue. Furthermore, if A p β, α q has distinct real eigenvalues for p β, α q P R ,o ` , wewill label them so that Λ j p β, α q ą Λ i p β, α q , for j ă i, i.e., the eigenvalues are assumed to be sorted in descending order.For D and Λ j as in Proposition 2.2 define L Dj . “ tp β, α q P D : Λ j p β, α q “ u . The last proposition implies
Proposition 2.3.
Let D and Λ j , j “ , , , ..., | M | , be as in Proposition 2.2. Then D X H βα “ \ | M | j “ L Dj (15) where \ denotes disjoint union. The proof follows from the definitions involved. For D “ R ,o ` we will omit the superscript D and write L j instead of L R ,o ` j . If A p β, α q has simple real eigenvalues for p β, α q P R ,o ` we can define R j . “ tp β, α q P R ,o ` : Λ j p β, α q ď u . The continuity of Λ j implies L j “ B R j . roposition 2.4. Suppose A p β, α q has simple real eigenvalues for p β, α q P R ,o ` . Then thecurve L j is strictly contained inside the curve L j ` for j “ , , ..., | M | ´ . Proof.
All diagonal entries of A p β, α q tend to when p β, α q Ñ B R ,o ` . This and Gershgorin’sTheorem [5, Appendix 7], imply Λ j p β, α q Ñ 8 for p β, α q Ñ B R ,o ` . This implies in particularthat R j is a compact subset of R ,o ` . Secondly, Λ j ` ă Λ j implies R j Ă R j ` ; the compactnessof these sets, the strictness of the inequality Λ j ` p β, α q ă Λ j p β, α q imply that B R j “ L j liesstrictly within R j ` with strictly positive distance from the boundary L j ` of R j ` ; thisproves the claim of the proposition.In Sections 7 and 8 we will employ the following assumption and the above decompositionof H βα to identify points on H βα to be used in the construction of p Y, M q -harmonic functions: Assumption 1. A p β, α q has real distinct eigenvalues for p β, α q P R ,o ` . To show that Assumption 1 is not vacuous, we now give a class of matrices A that satisfiesit. The following definitions are from [1, page 57]: a matrix is said to be totally nonnegative(totally positive) if all of its minors of any degree are nonnegative (positive). A totallynonnegative matrix is said to be oscillatory if some positive integer power of the matrix istotally positive. If A is oscillatory, Assumption 1 holds: Proposition 2.5.
Suppose A p β, α q is an oscillatory matrix for all p β, α q P R ,o ` , then A p β, α q has | M | distinct eigenvalues over R ,o ` . This proposition is a basic fact on oscillatory matrices [1, (6.28)]. [1, (6.26)] identifies aparticularly simple class of oscillatory matrices:
Proposition 2.6.
Suppose G p , q , G p , q , G p| M | ´ , | M |q , G p| M | , | M |q and G p j, j ´ q , G p j, j q , G p j, j ` q , j “ , , ..., | M | ´ are all strictly positive and the rest of the componentsof G are all zero, i.e., G is tridiagonal with strictly positive entries. Then G is an oscillatorymatrix. We will call any tridiagonal matrix with strictly positive entries on the three diagonals“strictly tridiagonal.” By the above proposition any strictly tridiagonal matrix is oscillatory.In particular, if the transition matrix P is strictly tridiagonal, A p β, α q will also be of thesame form for all p β, α q P R ` ; therefore, for such P Assumption 1 holds.The decomposition of H βα X R ,o ` into L j is shown in Figure 3 for the transition matrix P “ ¨˝ . . . . .
50 0 . . ˛‚ . (16)The matrix P of (16) is strictly tridiagonal; therefore, Proposition 2.6 applies and A p β, α q has distinct real eigenvalues for all p β, α q P R ,o ` and we have the decomposition (15) of H βα X R ,o ` given by Propositions 2.3 and 2.4; Figure 3 shows H βα and its components L j ;the jump probabilities for this example are ¨˝ . . . .
12 0 .
41 0 . .
09 0 .
39 0 . ˛‚ (17)where the i th row equals p λ p i q , µ p i q , µ p i qq . 9 α β Λ “ “ “ α β Λ Λ Λ Figure 3: Real section of the characteristic surface H βα “ Y j “ L j for the parameter valuesgiven in (16) and (17). On the right: detailed graph around the originBy Proposition 2.1 and Lemma 2.1 each point on any of the curves depicted in Figure3 gives a Y -harmonic function on Z ˆ Z o ` . Most of our analysis will be based on points onthe innermost curve L , the 1-level curve of the largest eigenvalue Λ ; before identifying therelevant points, let us look at two different methods of constructing p Y, M q -(super)harmonicfunctions from points on H βα . p Y, M q -harmonic and superharmonic functions We can proceed in two ways to get functions that satisfy E p y,m q r h p Y , M qs “ h p y, m q or E p y,m q r h p Y , M qs ď h p y, m q for y P B as well as the interior.The first is by defining the characteristic polynomial p , the boundary matrix A , and theboundary surface H associated with B and using points on H X H : p p β, α, m q . “ λ p m q β ` µ p m q α ` µ p m q , m P M , A p β, α q m ,m . “ P p m , m q , m ‰ m P p m , m q p p β, α, m q , m “ m , , p m , m q P M , H . “ ! p β, α, d q P C `| M | : A p β, α qq d “ d , d ‰ ) . (18)10efine Λ , p β, α q to be the largest eigenvalue of A p β, α q . Parallel to the interior case, define H βα . “ tp β, α q P C : p p β, α q “ u , L , . “ tp β, α q P R ` : Λ , p β, α q “ u . Proposition 2.7. rp β, α, d q , ¨s is p Y, M q -harmonic if p β, α, d q P H X H . Proof.
Proposition 2.1 says that for p β, α, d q P H , rp β, α, d q , ¨s satisfies the harmonicity con-dition when y P Z ˆ Z ` ´ B . Similar to the proof of Proposition 2.1, we would like to showthat rp β, α, d q , ¨s is p Y, M q -harmonic on B when p β, α, d q P H . By definition E p y,m q rp β, α, d q , p Y , M qs“ ÿ n P M ,n ‰ m P p m, n qrp β, α, d q , p y, n qs` P p m, m qp λ p m qrp β, α, d q , p y ` p´ , q , m qs ` µ p m qrp β, α, d q , p y ` p , q , m s` µ p m qrp β, α, d q , p y, m qsq“ ÿ n P M ,n ‰ m P p m, n qrp β, α, d q , p y, n qs` P p m, m qp λ p m q β y p q´ d p m q ` µ p m q β y p q α d p m q ` µ p m q β y p q d p m qq“ ÿ n P M ,n ‰ m P p m, n qrp β, α, d q , p y, n qs ` P p m, m qrp β, α, d q , p y, m qs p p β, α, m q“ β y p q ˜ ÿ n P M ,n ‰ m P p m, n q d p n q ` P p m, m q d p m q p p β, α, m q ¸ . The expression in parenthesis equals the m th term of the vector A p β, α qq d , which equals d p m q because p β, α, d q P H means A p β, α qq d “ d . Therefore, “ β y p q d p m q “ rp β, α, d q , p y, m qs . This argument and Proposition 2.1 prove the claim of the proposition.The real sections of H βα and H βα are 1 dimensional curves and their intersection will ingeneral consist of finitely many points. In the analysis of the tandem walk with no modulation,these points can easily be identified explicitly. There turn out to be three of them, of whichonly one is nontrivial (i.e., different from 0 and 1). In the present case, there will in generalbe 3 | M | ´ H βα X H βα ; one of these which lies on L X L , can beidentified using the implicit function theorem and the stability assumption (1); this pointand the p Y, M q -harmonic function it defines are given in Proposition 2.8 and 2.9 below. Forthe argument we need two auxiliary linear algebra results, Lemmas A.1 and A.2 given in theappendix. Proposition 2.8.
Under the stability assumption (1) there exists unique ă ρ ă suchthat p ρ , ρ q P L X L , Ă H βα X H βα , i.e., is the largest eigenvalue of A p ρ , ρ q and A p ρ , ρ q . roof. For q P R define H p q q . “ ´ log Λ ´ e q p q , e q p q ¯ . (19)By [9, Lemma 4.2, 4.3], H is convex in q . Proceeding parallel to [9, Proof of Lemma 4.4,page 515] define f p Λ , r q . “ det p Λ I ´ A p e r , e r qq . We know that f p Λ p e r , e r q , r q “ r P R . To prove our proposition, we will apply the implicit function theorem to f at p , q to provethat r ÞÑ Λ p e r , e r q is strictly increasing at r “ . Differentiating f at p , q with respect to r gives B f B r ˇˇˇˇ p , q “ ÿ m P M p λ p m q ´ µ p m qq P p m, m q det p I ´ P q m,m , which equals, by Lemma A.2, for some constant c ą “ c ÿ m P M p λ p m q ´ µ p m qq P p m, m q π p m qă f at p , q with respect to Λ gives: B f B Λ ˇˇˇˇ p , q “ . This implies that the implicit function theorem is applicable to f ; the last two display give: ddr Λ p e r , e r q| p , q ą . On the other hand, Gershgorin’s Theorem implies Λ p e r , e r q Ñ 8 as r Ñ ´8 (because ofthe λ p m q{ β term appearing in the diagonal terms of A , tending to `8 with β “ e r ). Tosum up: we have that Λ p e r , e r q is strictly monotone at r “ r decreases)and it tends to infinity as r Ñ ´8 . Then, by the continuity of Λ , there must exist at leastone point in p´8 , q where Λ p e r , e r q takes the value 1; the convexity of H implies that sucha point is unique, i.e., there is a unique point r ˚ ă p e r ˚ , e r ˚ q “ . Setting ρ “ e r ˚ proves the proposition.Let d be an eigenvector of A p ρ , ρ q corresponding to the eigenvalue 1; because 1 is thelargest eigenvalue of A p ρ , ρ q and because A p ρ , ρ q is irreducible and aperiodic, we canchoose d so that all of its components are strictly positive. The point p ρ , ρ , d q P H X H and Proposition 2.7 give us our first p Y, M q -harmonic function: Proposition 2.9. h ρ . “ rp ρ , ρ , d q , ¨s (20) is p Y, M q -harmonic. The second way of obtaining p Y, M q -harmonic functions is through conjugate points on H βα . The function α | M | p is a polynomial of degree 2 | M | in α. By the fundamental of theoremof algebra, α | M | p has 2 | M | roots, α p β q , ..., α p β q ,..., α | M | p β q , in C for each fixed β P C ;points p β, α i q P H βα , i “ , , ..., | M | are said to be conjugate points. In the non-modulatedcase, i.e., when | M | “ α p is only of second order, therefore, the conjugate points come12n pairs, and given one of the points in the pair, the other can be computed easily; in themodulated case, there are obviously no simple formulas to obtain all of the conjugate pointsgiven one among them, because computation of conjugate points involves finding the rootsof a polynomial of degree 2 | M | .For p β, α, d q P H define c p β, α, d q P C M , c p β, α, d qp m q . “ P p m, m q µ p m q d p m q ˆ ´ βα ˙ . (21)One can take linear combinations of functions defined by conjugate points to define p Y, M q -harmonic functions. This is based on the following lemma Lemma 2.2.
Suppose p β, α, d q P H . Then, for p y, m q P B ˆ M , E p y,m q rp β, α, d q , p Y , M qs ´ rp β, α, d q , p y, m qs “ β y p q c p β, α, d qp m q , (22) where c is defined as in (21) .Proof. The computation in the proof of Proposition 2.1 gives E p y,m q rp β, α, d q , p Y , M qs (23) “ β y p q ˜ ÿ n P M ,n ‰ m P p m, n q d p n q ` P p m, m q d p m q p p β, α, m q ¸ . On the other hand, p β, α, d q P H means rp β, α, d q , p y, m qs (24) “ β y p q d p m q “ β y p q ˜ ÿ n P M ,n ‰ m P p m, n q d p n q ` P p m, m q d p m q p p β, α, m q ¸ . Subtracting the last display from (23) gives E p y,m q r h p Y , M qs ´ h p y, m q“ β y p q P p m, m q µ p m q d p m q ˆ ´ βα ˙ “ β y p q c p β, α, d qp m q , which proves (22).We now identify a family of p Y, M q -harmonic functions constructed from conjugate pointson H : Proposition 2.10.
For β P C let p β, α i , d i q i “ , , ..., l ď | M | be distinct conjugatepoints on H . Take any subcollection t i , i , ..., i k u , k ď l such that c p β, α i j , d i j q are linearlydependent, i.e., there exists b P C k , b ‰ , such that k ÿ j “ b p j q c p β, α i j , d i j q “ . (25) Then h p y, m q “ k ÿ j “ b p j qrp β, α i j , d i j q , ¨s (26) is p Y, M q -harmonic. roof. We already know from Proposition 2.1, harmonic functions of the form rp β, α i , d i q , ¨s are p Y, M q -harmonic in the interior Z ˆ Z ` ´ B . So, their linear combinationsare also p Y, M q -harmonic in the interior and we need to check the harmonicity for y P B . ByLemma 2.2 E p y, ¨q rp β, α i , d i q , p Y , M qs ´ rp β, α i , d i q , p y, ¨qs “ β y p q c p β, α i , d i q . Taking linear combinations of these with weight vector b gives: E p y, ¨q r h p Y , M qs ´ h p y, ¨q “ β y p q ˜ k ÿ j “ b p j q c p β, α i j , d i j q ¸ which equals 0 P R | M | by (25). This proves that h is p Y, M q -harmonic on B . For any β P C such that p p β, α q “ α , α ,..., α | M | , all different from β , we have, by definition, c p β, α j , d j q ‰ j “ , , ..., | M | . Therefore, for such β ,and for any subcollection α j , α j , ..., α j k , with k ě | M | `
1, we can find a nonzero vector b satisfying (25).We will call a p Y, M q -harmonic function B B -determined if it of the form, p y, m q ÞÑ E p y,m q r f p Y τ , M τ q t τ ă8u s for some function f . The function p y, m q ÞÑ P p y,m q p τ ă 8q is the unique B B -determined p Y, M q -harmonic function taking the value 1 on B B . Among the functions of the form rp β, α, d q , ¨s , the closest we get to this type of behavior is when α “
1: for α “ rp β, , d q , p y, m qs depends only on m for y P B B . Therefore, α “ P p y,m q p τ ă 8q . The next proposition identifies a point on L of the form p ρ , α “ q with0 ă ρ ă Proposition 2.11.
Under assumption (1) there exists ă ρ ă such that p ρ , q P L Ă H βα ; i.e., is the largest eigenvalue of A p ρ , q .Proof. The proof is parallel to that of Proposition 2.8. We now define f p Λ , r q “ det p Λ I ´ A p e r , qq and observe, by assumption (1) and Lemma A.2, B f B r ˇˇˇˇ p , q “ ÿ m P M p λ p m q ´ µ p m qq P p m, m q det p I ´ P q m,m “ c ÿ m P M p λ p m q ´ µ p m qq P p m, m q π p m q ă c ą . The rest of the proof proceeds as in the proof of Proposition 2.8.Recall that p ρ , q P L , i.e., 1 is the largest eigenvalue of A p ρ , q ; the irreducibility of A implies that the eigenvectors corresponding to 1 have strictly negative or positive components;let d denote a right eigenvector of A p ρ , q corresponding to the eigenvalue 1 with strictlypositive components. Proposition 2.1 and the previous proposition imply that rp ρ , , d q , ¨s is p Y, M q -harmonic on Z ˆ Z ` ´ B . All of the prior works ([10, 11, 12]), use a conjugate pointof p ρ , q to construct a Y -harmonic function. In the present case, in general, p ρ , q willhave 2 | M | ´ L ; we will use p ρ , q along with this conjugate to define a p Y, M q - super harmonic14unction. This will be in two steps. Proposition 2.12 identifies the relevant conjugate point;Proposition 2.14 constructs the superharmonic function. We will use the superharmonicfunction in Sections 3 and 4 below in our analysis of the relative error (7).The identification of the conjugate point requires the following assumption: ÿ m P M p ρ µ p m q ´ µ p m qq P p m, m q det p I ´ A p ρ , qq m,m ă . (27)Remark 2.1 comments on this assumption and Proposition 2.13 gives simple conditionsunder which (27) holds. Proposition 2.12.
Let p ρ , q , ρ P p , q , be the point on L identified in Proposition 2.11.Then there exists a unique point p ρ , α ˚ q P L , α ˚ P p , q if (27) holds.Proof. Set r “ log p ρ q . Proof is parallel to those of Propositions 2.8 and 2.11 and is based onthe analysis of the function H of (19) at the point p r , q via the implicit function theorem.Define f p Λ , r q “ det p Λ I ´ A p ρ , e r qq and observe B f B r ˇˇˇˇ p r , q “ ÿ m P M p ρ µ p m q ´ µ p m qq P p m, m q det p I ´ A p ρ , qq m,m , which, by assumption (27), is strictly less than 0. The rest of the proof goes as that ofProposition 2.8. Remark 2.1.
Assumption (27) ensures that p ρ , q has a conjugate point on the principalcharacteristic surface L with α component less than . There is no corresponding assumptionin the non-modulated tandem case, because, in that setup, the conjugate of p ρ , q is p ρ , ρ q whose α component ρ is always less than by the stability assumption. In the simple con-strained random walk case (treated in [12]) the corresponding assumption is r ă ρ ρ (see[12, Display (14)]).The condition α ˚ ă is needed for the superharmonic function constructed in Proposition2.14 to be bounded on B B , see Proposition 3.2. Proposition 2.13.
Each of the following conditions is sufficient for (27) to hold:1. λ p m q{ µ p m q ă , λ p m q ă µ p m q for all m P M and the ratio λ p m q{ µ p m q does notdepend on m ,2. µ p m q ă µ p m q for all m P M .Proof. If λ p m q{ µ p m q ă m we can denote the common ratio by ρ ă p β, α q “ p ρ , q we see that A p ρ , q “ P . This implies that the root ρ identifiedin Proposition 2.11 must equal ρ . Setting ρ “ ρ on the left side of (27) gives ÿ m P M p ρ µ p m q ´ µ p m qq P p m, m q det p I ´ A p ρ , qq m,m “ ÿ m P M p ρ µ p m q ´ µ p m qq P p m, m q det p I ´ A p ρ , qq m,m “ ÿ m P M p λ p m q ´ µ p m qq P p m, m q det p I ´ A p ρ , qq m,m p I ´ A p ρ , qq m,m ą λ p m q ă µ p m q by assumption; these and thelast line imply (27): ă . That the condition µ p m q ă µ p m q for all m P M implies (27) follows from a similar argu-ment. Remark 2.2.
The argument used in the proof above can be used to prove that the conjugatepoint p ρ , α ˚ q satisfies α ˚ ą if one replaces ă with ą in (27) . For the rest of our analysis we will need a further assumption: ρ ‰ ρ , (28)where ρ is the first (or the second) component of the point on L X L , identified in Propo-sition 2.8 and ρ is the β component of the point on L identified in Proposition 2.11.Assumption (28) generalizes the assumption µ ‰ µ in [10, 11, 12].The following lemmaidentifies sufficient conditions for (28) to hold. Lemma 2.3. If µ p m q ą µ p m q for all m P M , or µ p m q ă µ p m q for all m P M , then (28) holds.Proof. The matrix D “ A p ρ , ρ q ´ A p ρ , q is a diagonal matrix whose m th entry equals p ´ ρ qp µ p m q ´ µ p m qq . Suppose µ p m q ą µ p m q for all m P M ; then ρ P p , q impliesthat D has strictly positive entries.We have then: A p ρ , ρ q d “ A p ρ , q d ` Dd “ d ` Dd ą p ` ǫ q d (29)for some ǫ ą
0; here we have used 1) d is an eigenvector of A p ρ , q corresponding to theeigenvalue 1 and 2) D has strictly positive entries. We know by [5, Proof of Theorem 1,Chapter 16] that Λ p A p ρ , ρ qq “ sup t c : D x P R | M |` , A p ρ , ρ q x ě cx u . (30)This and (29) imply that the largest eigenvalue of A p ρ , ρ q is strictly greater than 1. Thisimplies ρ ă ρ . That µ p m q ą µ p m q for all m P M implies ρ ą ρ follows from the sameargument applied to A p ρ , q d , . α β Λ α β Λ Figure 4: ρ ´ ρ and α ˚ ´ ρ have the same sign (Lemma 2.4); the points marked with ’x’are p ρ , ρ q , p ρ , α ˚ q and p , ρ q ; the point marked with ’o’ is p ρ , ρ q Lemma 2.4.
Let p ρ , α ˚ q be the conjugate point of p ρ , q on L identified in Proposition2.12. Then ρ ą ρ implies α ˚ ą ρ and ρ ă ρ implies α ˚ ă ρ . Figure 4 illustrates this lemma.
Proof.
By definition ρ is the unique positive number strictly less than 1 satisfying Λ p ρ , ρ q “ ρ ă ρ implies Λ p ρ , ρ q ą . But α ˚ satisfies Λ p ρ , α ˚ q “ p ρ , ρ q ď ρ P p α ˚ , ρ s . It follows that ρ ă α ˚ . The argument for the opposite implication is simi-lar.
Remark 2.3.
By the previous lemma the assumption (28) is equivalent to α ˚ ‰ ρ . (31) Remark 2.4. ρ is the unique solution of Λ p β, β q “ on p , q ; similarly ρ is the uniquesolution of Λ p β, q “ on p , q . That Λ is the largest eigenvalue of A p β, α q and the abovefacts imply that ρ [ ρ ] is the largest root of p p β, β q [ p p β, q ] on p , q . Therefore, one canstate the assumption (28) also as follows: “the largest roots of p p β, β q and p p β, q on p , q differ.” By definition, 1 is the largest eigenvalue of A p ρ , α ˚ q ; let d , denote a right eigenvector ofthis matrix with strictly positive entries. Next proposition constructs a p Y, M q -superharmonicfunction that we will use to find upper bounds on approximation errors; this is one of thekey steps of our argument. 17 roposition 2.14. Under assumption (28) one can choose a constant c P R ( c ą for α ˚ ă ρ and c ă for α ˚ ą ρ ) so that h ρ . “ rp ρ , , d q , ¨s ` c rp ρ , α ˚ , d , , ¨s , (32) is a p Y, M q -superharmonic function.Proof. By their construction, the conjugate points p ρ , q and p ρ , α ˚ q lie on L . Thisand Proposition 2.1 imply that the functions rp ρ , , d q , ¨s and rp ρ , α ˚ , d , q , ¨s are p Y, M q -harmonic on Z ˆ Z ` ´ B . This implies the same for their linear combination h ρ . Therefore,to prove that h ρ is p Y, M q -superharmonic, it suffices to check this on B . By definition h ρ is superharmonic on B if E p y,m q r h ρ p Y , M qs ď h ρ p y, m q for y “ p k, q and m P M . By Lemma 2.2, E p y,m q rp ρ , , d q , p Y , M qs ´ rp ρ , , d q , p y, m qs “ ρ k c p ρ , , d qp m q , E p y,m q rp ρ , α ˚ , d , q , p Y , M qs ´ rp ρ , α ˚ , d , q , p y, m qs “ ρ k c p ρ , α ˚ , d , qp m q , where c p¨ , ¨ , ¨q is defined as in (21). The last two lines give E p y,m q r h ρ p Y , M qs ´ h ρ p y, m q “ ρ k p c p ρ , , d qp m q ` c c p ρ , α ˚ , d , qp m qq . (33)For h ρ to be superharmonic, the right side of the last display must be negative. The sign ofthis expression is determined by c p ρ , , d qp m q ` c c p ρ , α ˚ , d , qp m q . (34)The definition (21) of c and ρ ă d p m q ą m P M imply that the first termis strictly positive for all m P M . Define d max . “ max m P M c p ρ , , d qp m q ą . The sign of the second term in (34) depends on whether α ˚ ă ρ or α ˚ ą ρ . For α ˚ ă ρ ,the definition (21) of c and d , p m q ą m P M imply that the c term in (34) is strictlynegative for all m . Define d ˚ max . “ max m P M c p ρ , α ˚ , d , qp m q ă . (35)If we choose c ą d max ` c d ˚ max ă , (36)(34) will be strictly less than 0 for all m . This and (33) imply that h ρ is superharmonic forany c satisfying (36).For α ˚ ą ρ the argument remains the same except that we replace the max in (35) withmin and c ă h ρ to find bounds on the approximation error (7).18 Upper bound for P p y,m q p τ ă 8q As we saw in Proposition 2.14 above, p Y, M q -superharmonic functions can be constructedfrom just two conjugate points on L Ă H βα . We will need an upper bound on P p y,m q p τ ă 8q in our analysis of the relative error (7);in the non-modulated tandem walk treated in [10, 11], this probability can be representedexactly using the harmonic functions constructed from points on the characteristic surface,which also obviously serves as an upper bound. In the present case, we will construct an upperbound for P p y,m q p τ ă 8q from p Y, M q -harmonic and superharmonic functions constructed inPropositions 2.10 and 2.14. The next proposition constructs the necessary function the onefollowing it derives the upper bound. Proposition 3.1.
Let h ρ “ rp ρ , ρ , d q , ¨s be as in (20) and h ρ be as in (32) . One canchoose c ě so that c . “ min y PB B,m P M h ρ p y, m q ` c h ρ p y, m q ą
0; (37) for α ˚ ă ρ one can choose c “ . Proof.
By its definition, h ρ p y, m q “ d p m q ` c p α ˚ q y p q d , p m q , (38)for y P B B. We know by Proposition 2.14 that c ą α ˚ ă ρ . This, α ˚ ą d , p m q ą y PB B h ρ p y, m q ě min m P M d p m q ą , which implies (37) with c “ . For α ˚ ą ρ , c ă y p q . But 0 ă α ˚ ă k ą h ρ p y, m q ě min m P M d p m q{ ą , y P B
B, y p q ě k . (39)On the other hand, d p m q ą m P M and ρ ą h ρ p y, m q ą y P B B , m P M . Then one can choose c ą c d p m q ρ y p q ` d p m q ` c p α ˚ q y p q d , p m q ą min m P M d p m q{ , y P B
B, y p q ď k , (40)since this inequality concerns only finitely many y P B B . c chosen thus, (39) and (40) imply(37). Proposition 3.2.
Let c ě , c ą be as in Proposition 3.1 P p y,m q p τ ă 8q ď c p h ρ p y, m q ` c h ρ p y, m qq . (41) Proof.
For ease of notation set f “ h ρ ` c h ρ ; ρ , ρ , α ˚ P p , q implies sup y P B,m P M | f p y, m q| ă 8 . f is p Y, M q -superharmonic. These imply that k ÞÑ f p Y k ^ τ , M k ^ τ q is a bounded supermartingale. Then by the optional sampling theorem([3, Theorem 5.7.6]) E p y,m q r f p Y τ , M τ q t τ ă8u s ď f p y, m q ;this, Y τ P B B when τ ă 8 and (37) imply c P p y,m q p τ ă 8q ď f p y, m q , which gives (41). P p x,m q p σ ă σ , ă τ n ă τ q Define σ i . “ inf t k ě X k P B i u ; i “ , , (42)and σ , . “ inf t k ě X k P B , k ě σ u ; (43) σ i is the first time X hits B i and σ , is the first time X hits B after hitting B . In the nextsection we find an upper bound on the probability P p x,m q p σ ă σ , ă τ n ă τ q , we will usethis bound in the analysis of the approximation error in the proof of Theorem 6.1. Define ρ . “ ρ _ ρ . (44)The goal of the section is to prove Proposition 4.1.
For any ǫ ą there exists n ą such that P p x,m q p σ ă σ , ă τ n ă τ q ď ρ n p ´ ǫ q (45) for n ě n and p x, m q P A n . We split the proof into cases ρ ą ρ and ρ ą ρ . The first subsection below treats thefirst case ρ ą ρ , the next gives the changes needed for the latter.Let A denote the characteristic matrix for B : p p β, α, m q . “ λ p m q β ` µ p m q ` µ p m q βα , m P M , A p β, α q m ,m . “ P p m , m q , m ‰ m P p m , m q p p β, α, m q , m “ m , , p m , m q P M . We will use the following fact several times in our analysis.
Lemma 4.1.
The function p x, m q ÞÑ rp ρ , , d q , p T n p x q , m qs “ ρ n ´p x p q` x p qq d p m q (46) is p X, M q -harmonic on Z ` ´ B . Proof.
We know by Proposition 2.1 and p ρ , , d q P H that rp ρ , , d q , ¨s is p Y, M q -harmonicon Z ˆ Z o ` , which implies that (46) is p X, M q -harmonic on Z ,o ` ; this and A p ρ , q “ A p ρ , q imply the p X, M q -harmonicity of (46) on B . 20 .1 ρ ą ρ To prove (45) we will construct a corresponding supermartingale; applying the optional sam-pling theorem to the supermartingale will give our desired bound. The event t σ ă σ , ă τ n ă τ u consists of three stages: X first hits B then B and finally B A n without ever hitting0. If h is an p X, M q -superharmonic function, it follows from the definitions that h p X, M q is asupermartingale. We will construct our supermartingale by applying three functions (one foreach of the above stages) to p X, M q : the function for the first stage is the constant ρ n , whichis trivially superharmonic. The function for the second stage will be a constant multiple of p x, m q ÞÑ h ρ p T n p x q , m q . By Proposition 2.8, p x, m q ÞÑ h ρ p T n p x q , m q is p X, M q -harmonic on Z ` ´ B . One can check directly that it is in fact subharmonic on B . The definition of thesupermartingale S will involve terms to compensate for this. The function for the third stageis h : p x, m q ÞÑ h ρ p T n p x q , m q ` c h ρ p T n p x q , m q , x P A n , m P M , “ h ρ pp n ´ x p q , x p qq , m q ` c h ρ pp n ´ x p q , x p qq , m q , “ ρ n ´p x p q` x p qq ´ d p m q ` c α ˚ x p q d , p m q ¯ ` c ρ n ´ x p q d p m q , (47)where c ě c is as in Proposition 2.14. The next twopropositions imply that h is p X, M q -superharmonic on Z ` ´ B . Proposition 4.2.
For ρ ą ρ , h ρ p T n p¨q , ¨q is superharmonic on all of Z ` . Proof.
That h ρ p T n p¨q , ¨q is p X, M q -superharmonic on Z ` ´ B follows from Proposition 2.14(i.e., from the fact that h ρ p¨ , ¨q is p Y, M q -harmonic). Therefore, it suffices to prove that h ρ p T n p¨q , ¨q is superharmonic on B . h ρ p T n p¨q , ¨q is a sum of two functions: h ρ p T n p¨q , ¨q “ rp ρ , , d q , p T n p¨q , ¨qs ` c rp ρ , α ˚ , d , q , p T n p¨q , ¨qs . (48)Let us show that each of these summands is p X, M q - superharmonic on B . The first summandis p X, M q -harmonic (and therefore, superharmonic) on B by Lemma 4.1. To treat the secondterm in (48) recall the following: ρ ă ρ implies ρ ă α ˚ (Lemma 2.4); then, by Proposition2.14, c ă
0. Therefore, if we can show that rp ρ , α ˚ , d , q , p T n p¨q , ¨qs is p X, M q -subharmonicon B we will be done. Let us now see that this is indeed the case.For ease of notation set h p x, m q “ rp ρ , α ˚ , d , q , p T n p x q , m qs “ ρ n ´p x p q` x p qq α ˚ x p q d , p m q . A calculation parallel to the proof of Proposition 2.1 shows E p x,m q r h p X , M qs ´ h p x, m q “ d , p m q µ p m qp ´ α ˚ q ρ n ´ x p q ą , (49)for x P B , i.e., h is p X, M q -subharmonic on B . This completes the proof of this proposition. Proposition 4.3. h ρ p T n p¨q , ¨q is harmonic (and therefore superharmonic) on Z ` ´ B . It issubharmonic on B where it satisfies E p x,m q r h ρ p T n p X q , M qs ´ h ρ p T n p x q , m q “ d p m q µ p m qp ´ ρ q ρ n ą . (50)21he proof is parallel to the computation given in the proof of Proposition 2.1 and isomitted. We can now define the supermartingale that we will use to prove (45): S k . “ $’&’% h , k ď σ ,h p X k , M k q , σ ă k ď σ , ,h p X k , M k q , k ą σ , ,S k . “ S k ´ c kρ n , where c . “ max m P M d p m q ` c max m P M d p m q min m P M d p m q ą , (51) h . “ c ρ n , c . “ c max m P M d p m q ą ,h . “ c h ρ p T n p¨q , ¨q “ c rp ρ , ρ , d q , p T n p¨q , ¨qs “ c ρ n ´ x p q d p¨q , (52) c . “ c p ´ ρ q max m P M d p m q µ p m q . (53)Two comments: h is a constant function, independent of x and m , and h ě h on B . Proposition 4.4. S is a supermartingale.Proof. The claim follows mostly from the fact that the functions involved in the definitionof S are p X, M q -superharmonic away from B . The term that breaks superharmonicity on B is rp ρ , ρ , d q , p T n p X k q , M k qs ; the ´ c kρ n term in the definition of S is introduced tocompensate for this. The details are as follows.The p X, M q -harmonicity of h , h and h implies E p x,m q r S k ` | F k s “ S k for X k P Z ` ´ B Y B ; i.e., S k satisfies the martingale equality condition for X k P Z ` ´ B Y B ;this implies that S k satisfies the supermartingale inequality condition over the same event. h and h are p X, M q -superharmonic on B by Propositions 4.2 and 4.3 ( h is trivially sobecause it is constant); this implies E p x,m q r S k ` | F k s ď S k for X k P B and k ‰ σ , . For k “ σ , we have S k ` “ h p X k ` , M k ` q . This, the p X, M q -superharmonicity of h on B implies E p x,m q r S k ` | F k s “ E p x,m q r h p X k ` , M k ` q| F k sď h p X k , M k q (54)for k “ σ , . On the other hand, S k “ h p X k , M k q for k “ σ , . (55)The definitions of c , h and h in (51), (52) and (47), ρ ă ρ and c ă h p x, m q ď h p x, m q x P B . This and (55) imply h p X k , M k q ď h p X k , M k q “ S k for k “ σ , . The last display and (54) imply E p x,m q r S k ` | F k s ď S k , i.e., S and S are p X, M q -supermartingales for k “ σ , as well.It remains to prove E p x,m q r S k ` | F k s ď S k , when X k P B . (56)The cases to be treated here are: k “ σ , σ ă k ă σ , and k ą σ , . For k “ σ , we have S k “ h p X k , M k q “ c ρ n d p M k q and S k ` “ h p X k ` , M k ` q ; theseand h ě h on B imply E p x,m q r S k ` | F k s ´ S k (57) “ E p x,m q r c h ρ p T n p X k ` q , M k ` q| F k s ´ c ρ n d p M k q ´ c ρ n , By (50) and σ “ k , this equals ď c d p M k q µ p M k qp ´ ρ q ρ n ´ c ρ n . By the definition of c : “ ρ n c p ´ ρ qp d p M k q µ p M k q ´ max m P M d p m q µ p m qq ď , which proves (56) for k “ σ . For σ ă k ă σ , , S k “ h p X k , M k q “ c h ρ p T n p X k q , M k q ; therefore the above argumentapplies to this case as well (except for the last step which is not needed here because S k and S k ` are defined by applying the same function h to p X k ` , M k ` q and p X k , M k q ).Finally, to treat the case X k P B and k ą σ , we start with E p x,m q r S k ` | F k s ´ S k “ E p x,m q r S k ` | F k s ´ S k ´ c ρ n ,S k “ h p X k , M k q for k ą σ , . Then by the definition of h : “ E p x,m q r h ρ p T n p X k ` q , M k ` q ` c h ρ p T n p X k ` q , M k ` q| F k s´ h ρ p T n p X k q , M k q ´ c h ρ p T n p X k q , M k q ´ c ρ n , “ ` E p x,m q r h ρ p T n p X k ` q , M k ` q| F k s ´ h ρ p T n p X k q , M k q ˘ ` E r c h ρ p T n p X k ` q , M k ` q| F k s ´ c h ρ p T n p X k q , M k q ´ c ρ n . The p X, M q -superharmonicity of h ρ p T n p¨q , ¨q implies that the difference inside the parenthesisis negative, therefore: ď E r c h ρ p T n p X k ` q , M k ` q| F k s ´ c h ρ p T n p X k q , M k q ´ c ρ n . “ c d p M k q µ p M k qp ´ ρ q ρ n ´ c ρ n . By its definition (53), c ą c d p m q µ p m qp ´ ρ q for all m P M , which implies: ď . This proves (56) for k ą σ , and completes the proof of this proposition.We are now ready to give a proof of Proposition 4.1 for ρ ą ρ : Proof of Proposition 4.1; case ρ ą ρ . By its definition (44), ρ of (45) equals ρ for ρ ą ρ .We begin by truncating time: [9, Theorem A.2] implies that there exists c ą N ą P p x,m q p τ n ^ τ ą c n q ď ρ n , for n ą N . Then: P p x,m q p σ ă σ , ă τ n ă τ q (58) “ P p x,m q p σ ă σ , ă τ n ă τ , τ n ^ τ ď c n q` P p x,m q p σ ă σ , ă τ n ă τ , τ n ^ τ ą c n qď P p x,m q p σ ă σ , ă τ n ă τ , τ n ^ τ ď c n q ` ρ n for n ą N . Therefore, to prove (45) it suffices to bound the first term on the right side ofthe last inequality. Now apply the optional sampling theorem to the supermartingale S atthe bounded stopping time τ “ τ ^ τ n ^ c n : E p x,m q r S τ ^ τ n ^ c n s ď S “ c ρ n . By definition, S k “ S k ´ c kρ n ; substituting this in the last display gives: ´ c c nρ n ` E p x,m q r S τ s ď c ρ n E p x,m q r S τ s ď p c ` nc c q ρ n . By its definition, S k ą
0, therefore restricting it to an event makes the last expectationsmaller: E p x,m q r S τ t σ ă σ , ă τ n ă τ ď c n u s ď p c ` nc c q ρ n . On the set t σ ă σ , ă τ n ă τ ď c n u , we have τ “ τ n and S τ n “ h p X τ n , M τ n q ;by definition X τ n P B A n . By definition of h and by Proposition 3.1 h p x, m q ě c ą x P B A n . These and the last display imply c P p x,m q p σ ă σ , ă τ n ă τ ď c n q ď p c ` nc c q ρ n . Substitute this in (58) to get P p x,m q p σ ă σ , ă τ n ă τ q ď ρ n p ´ ǫ n q where ǫ n “ n log { ρ ˆ c ` nc c c ˙ ;setting n ě N so that ǫ n ă ǫ for n ě n gives (45).24 .2 ρ ă ρ The previous subsection gave a proof of Proposition 4.1 for ρ ă ρ . The only changes neededin this proof for ρ ă ρ concern the functions used in the definition of the supermartingale S ; the needed changes are:1. Modify the function h for the second stage,2. The function h is no longer superharmonic on B ; quantify how much it deviates fromsuperharmonicity on B ,3. Modify the constants used in the definition of S in accordance with these changes.The next two propositions deal with the first two items above; the definition of thesupermartingale (taking also care of the third item) is given after them.The convexity of q ÞÑ ´ log p Λ p e q , e q qq and Λ p ρ , ρ q “ p ρ , ρ q ă ρ ą ρ . Let d ` be a right eigenvector of A p ρ , ρ q with strictly positive entries. Proposition 4.5.
The function f : p x, m q ÞÑ rp ρ , ρ , d ` q , p T n p x q , m qs is superharmonic on Z ` ´ B . On B it satisfies E p x,m q r f p X , M qs ´ f p x, m q ď d ` p m q µ p m qp ´ ρ q ρ n . (59)The proof is parallel to that of Proposition 4.3 and follows from Λ p ρ , ρ q ă A p ρ , ρ q “ A p ρ , ρ q and the definitions involved. Proposition 4.6.
Let h be as in (47) ; h is p X, M q -superharmonic on Z ` ´ B ; on B itsatisfies E p x,m q r h p X , M qs ´ h p x, m q “ c d , p m q µ p m qp ´ α ˚ q ρ n ´ x p q ą . (60) Proof.
Lemma 2.4 and ρ ą ρ imply α ˚ ă ρ ; this and Proposition 3.1 imply that c in thedefinition of h is 0; i.e., h p x, m q “ h ρ p T n p x q , m q “ ρ n ´p x p q` x p qq ´ d p m q ` c p α ˚ q x p q d , p m q ¯ ;That h is p X, M q -superharmonic on Z ` ´ B follows from the same property of h ρ (seeProposition 2.14). On the other hand, again by Proposition 2.14, α ˚ ă ρ implies that c in the definition of h ρ satisfies c ą
0. By Lemma 4.1 p x, m q ÞÑ rp ρ , , d q , p T n p x q , m qs is p X, M q -harmonic on B ; (60) follows from these and (49). ρ ą ρ implies ρ ą α ˚ (Lemma 2.4); this and Proposition 3.1 imply c “ ρ ą α ˚ andProposition 2.14 imply c ą
0. That c ą c “ S : S k . “ $’&’% h , k ď σ ,h p X k , M k q , σ ă k ď σ , ,h p X k , M k q , k ą σ , ,S k . “ S k ´ c kρ n , c . “ max m P M p d p m q ` c d , p m qq min m P M d ` p m q ,h . “ c ρ n , c . “ c max m P M d ` p m q ,h . “ c rp ρ , ρ , d ` q , p T n p¨q , ¨qs “ c ρ n ´ x p q d ` p¨q ,c . “ c p ´ ρ q max m P M d ` p m q µ p m q ` c p ´ α ˚ q max m P M d , p m q µ p m q . The modification in c ensures h ě h on B ; c ą h is no longer superhar-monic on B ; the second term in c compensates for this. Proposition 4.7. S as defined above is a supermartingale for ρ ą ρ . Proof.
With the modifications made as above, the proof proceeds exactly as in the case ρ ą ρ (Proposition 4.4) and follow from the following facts: h ě h on B , h ě h on B (these are guaranteed by the choices of the constants c , c ); p X, M q -superharmonicity of h and h on Z ` ´ B (guaranteed by Propositions 4.5 and 4.6), the ´ c kρ n term compensatingfor the lack of p X, M q -superharmonicity of h and h on B (guaranteed by (59) and (60) andthe choice of the constant c ). Proof of Proposition 4.1; case ρ ą ρ . With S defined as above, the proof given for the case ρ ą ρ works without change. P p x,m q p τ n ă τ q To get an upper bound on the relative error (7), we need a lower bound on the probability P p x,m q p τ n ă τ q . We will get the desired bound by applying the optional sampling theorem,this time to an p X, M q -submartingale. This we will do, following [12], by constructing asuitable p X, M q -subharmonic function. As opposed to superharmonic functions, subharmonicfunctions are simpler to construct. Proposition 5.1. p x, m q ÞÑ rp ρ , , d q , p T n p x q , m qs _ rp ρ , ρ , d q , p T n p x q , m qs (61) “ ρ n ´p x p q` x p qq d p m q _ ρ n ´ x p q d p m q is p X, M q -subharmonic on Z ` . Proof.
We know by Lemma 2.2 that E p x,m q rp ρ , , d q , p T n p X q , M qs ´ rp ρ , , d q , p x, m qs“ ρ n ´ x p q P p m, m q µ p m q d p m qp ´ ρ q ą , i.e, p x, m q ÞÑ rp ρ , , d q , p x, m qs is p X, M q -subharmonic on B . That p x, m q ÞÑ rp ρ , , d q , p T n p x q , m qs is p X, M q -subharmonic on Z ` ´B follows from Lemma4.1. Then, p x, m q ÞÑ rp ρ , , d q , p x, m qs is p X, M q -subharmonic on all of Z ` . Similarly, Proposition 4.3 and (50) imply that p x, m q ÞÑ rp ρ , ρ , d q , p x, m qs is p X, M q -subharmonic on all of Z ` .The maximum of two subharmonic functions is again subharmonic. This and the abovefacts imply the p X, M q -subharmonicity of (61).26 roposition 5.2. P p x,m q p τ n ă τ qě ˆ max m P M p d p m q _ d p m qq ˙ ´ (62) ˆ ˆ ρ n ´p x p q` x p qq d p m q _ ρ n ´ x p q d p m q ´ ρ n max m P M d p m q _ ρ n max m P M d p m q ˙ . Proof.
Set g p x, m q “ ρ n ´p x p q` x p qq d p m q _ ρ n ´ x p q d p m q ;by the previous proposition g is p X, M q -subharmonic. By its definition, g is positive andbounded from above for x P Z ` . It follows that s k “ g p X τ n ^ τ ^ k , M τ n ^ τ ^ k q is a bounded positive submartingale. By definition E r g p X τ n ^ τ , M τ n ^ τ qs “ E r g p X τ n , M τ n q t τ n ă τ u s ` E r g p X τ , M τ q t τ ď τ n u s . (63)That X τ n P B A n implies g p X τ n , M τ n q “ g p k, n ´ k q for some k ă n ; then g p X τ n , M τ n q ď max m P M p d p m q _ d p m qq . This, (63) and the optional sampling theorem applied to s at time τ n ^ τ give P p x,m q p τ n ă τ q ˆ max m P M p d p m q _ d p m qq ˙ ` g p , m q P p x,m q p τ ď τ n q ě g p x, m q . P p x,m q p τ ď τ n q ď ˆ max m P M p d p m q _ d p m qq ˙ P p x,m q p τ n ă τ q ě g p x, m q ´ max m P M r g p , m qs ;this and max m P M r g p , m qs “ ρ n max m P M d p m q _ ρ n max m P M d p m q give (62). This section puts together the results of the last two sections to derive an exponentiallydecaying upper bound on the relative error (7). As in previous works [10, 11, 12], this taskis simplified if we express the Y process in the x coordinates thus:¯ X k . “ T n p Y k q ;¯ X has the same dynamics as X , except that it is not constrained on B . In this section wewill set the initial condition using the scaled coordinate x P R ` , x p q ` x p q ă
1, the initialcondition for the X and ¯ X will be X “ ¯ X “ t nx u . As in the non-modulated case, the following relation between ¯ X and X will be very useful:27 emma 6.1. Let σ , be as in (43) . Then X k p q ` X k p q “ ¯ X k p q ` ¯ X k p q for k ď σ , . This lemma is the analog of [10, Proposition 7.2], which expresses the same fact for thenon-modulated two dimensional tandem walk; the proof is unchanged because it does notdepend on the modulating process. Example sample paths of X and ¯ X up to time σ , demonstrating Lemma 6.1 are shown in Figure 5. p , q p , qp , q p , q Figure 5: A sample path of X k (left) and ¯ X k (right)Define ¯ τ n . “ inf t k ą X k P B A n u , ¯ σ , . “ inf t k ą X k p q ` ¯ X k p q “ , k ě σ u .X and ¯ X have identical dynamics upto time σ ; ¯ σ , is the first time after ( σ , i.e., thefirst time X and ¯ X hit B ) that the sum of the components of ¯ X equals 0. By the definitionsof ¯ X and Y , ¯ τ n “ τ. What follows is an upper bound similar to (45) for the ¯ X process. This is a generalizationof [10, Proposition 7.5] to the present setup: Proposition 6.1.
For any ǫ ą there exists n ą such that P p x,m q p σ ă σ , ă ¯ τ n ă 8q ď ρ n p ´ ǫ q (64) for n ą n and p x, m q P A n .Proof. As in [10, Proposition 7.5] we partition the event t σ ă σ , ă ¯ τ n ă 8u into whether¯ X hits B A n before or after it hits t x P Z ˆ Z ` : x p q ` x p q “ u : P p x,m q p σ ă σ , ă ¯ τ n ă 8q (65) “ P p x,m q p σ ă σ , ă ¯ τ n ă ¯ σ , ă 8q ` P p x,m q p σ ă σ , ă ¯ σ , ă ¯ τ n ă 8q Lemma 6.1 implies ¯ X σ , p q ` ¯ X σ , p q “ X σ , p q ` X σ , p q i.e., at time σ , , X and ¯ X will be on the same line t x P Z ˆ Z ` : x p q ` x p q “ k u for some k P t , , ..., n ´ u . Then for ω P t σ ă σ , ă ¯ τ n u the fully constrained sample path X p ω q X p ω q hits t x P Z ˆ Z ` : x p q ` x p q “ u and it cannot hit B A n after ¯ X hits t x P Z ˆ Z ` : x p q ` x p q “ n u (intuitively: more constraints on X push it fasterto B A n and slower to 0 than less constraints do the process ¯ X ): these give t σ ă σ , ă ¯ τ n ă ¯ σ , ă 8u Ă t σ ă σ , ă τ n ă τ u ;the bound (45) on the probability of the last event and (65) imply that there exists n ą P p x,m q p σ ă σ , ă ¯ τ n ă 8q ď ρ n p ´ ǫ { q ` P p x,m q p σ ă σ , ă ¯ σ , ă ¯ τ n ă 8q (66)for n ą n . To bound the last probability we observe that ¯ X ¯ σ , lies on t x P Z ˆ Z ` : x p q ` x p q “ u ;by Proposition 3.2, starting from this line, the probability of ¯ X ever hitting t x P Z ˆ Z ` : x p q ` x p q “ n u is bounded from above by1 c p h ρ pp n ´ x p q , x p qq , m q ` c h ρ pp n ´ x p q , x p qq , m qqď c p ρ n d p m q ` c ρ n d p m qq ;this and the strong Markov property of ¯ X give: P p x,m q p σ ă σ , ă ¯ σ , ă ¯ τ n ă 8q ď c ρ n where c is a constant depending on d , d , c and c . Substituting this in (66) gives P p x,m q p σ ă σ , ă ¯ τ n ă 8q ď ρ n p ´ ǫ { q ` c ρ n , for n ą n . This implies the statement of the proposition.Finally, we state and prove our main theorem:
Theorem 6.1.
For any x P R ` , x p q ` x p q ă , and m P M (if ρ ą ρ and x p q ă ´ log p ρ q{ log p ρ q we also require x p q ą ) there exists c ą and N ą such that | P p x n ,m q p τ n ă τ q ´ P p T n p x n q ,m q p τ ă 8q| P p x n ,m q p τ n ă τ q ă ρ c n (67) for n ą N , where x n “ t nx u .Proof. Proposition 5.2, the choice of x (i.e., x p q ` x p q ă x p q ą x p q ă ´ log p ρ q{ log p ρ q ) when ρ ą ρ ) imply the lower bound P p x,m q p τ n ă τ q ě ρ n p ´ c q (68)for some constant 1 { ą c ą x .By definition ¯ X hits B A n exactly when Y hits B B , i.e., ¯ τ n “ τ ; therefore, P p x n ,m q p ¯ τ n ă8q “ P p T n p x n q ,m q p τ ă 8q and | P p x n ,m q p τ n ă τ q ´ P p T n p x n q ,m q p τ ă 8q| P p x n ,m q p τ n ă τ q (69) “ | P p x n ,m q p τ n ă τ q ´ P p x n ,m q p ¯ τ n ă 8q| P p x n ,m q p τ n ă τ q
29e partition the probabilities of events t τ n ă τ u and t τ ă 8u as follows P p x n ,m q p τ n ă τ q “ P p x n ,m q p τ n ă σ ă τ q ` P p x n ,m q p σ ă τ n ď σ , ^ τ q` P p x n ,m q p σ ă σ , ă τ n ă τ q (70) P p T n p x n q ,m q p τ ă 8q “ P p T n p x n q ,m q p τ ă σ q ` P p T n p x n q ,m q p σ ă τ ď σ , q` P p T n p x n q ,m q p σ ă σ , ă τ ă 8q . (71)Lemma 6.1 says the processes X and ¯ X move together until they hit B , so P p x n ,m q p τ n ă σ ă τ q “ P p T n p x n q ,m q p τ ă σ q . After hitting B , the sum of the components of X and ¯ X are still equal until one of theprocesses hits B . Lemma 6.1 now gives P p x n ,m q p σ ă τ n ď σ , ^ τ q “ P p T n p x n q ,m q p σ ă τ ď σ , q . The last two equalities, Propositions 4.1, 6.1, and partitions (70), (71) imply that there exists n ą | P p x n ,m q p σ ă σ , ă τ n ă τ q ´ P p T n p x n q ,m q p σ ă σ , ă τ ă 8q |ď ρ n p ´ c q (72)for n ą n . Substituting the last bound and (68) in (69) gives (67). P p τ ă 8q Theorem 6.1 tells us that P p T n p x n q ,m q p τ ă 8q approximates P p x n ,m q p τ n ă τ q very well. In thissection we develop approximate formulas for P p y,m q p τ ă 8q . Recall that a p Y, M q -harmonicfunction is said to be B B -determined if it of the form p y, m q ÞÑ E p y,m q r f p Y τ , M τ q t τ ă8u s for some function f . The function p y, m q ÞÑ P p y,m q p τ ă 8q (73)is p Y, M q -harmonic with f “ B B . Furthermore, by definition it is B B -determined, (for(73), f is the function taking the constant value 1 on B B ). Our approach to the approximationof P p y,m q p τ ă 8q is based on the classical superposition principle: take linear combinationsof the p Y, M q -harmonic functions identified in Propositions 2.7 and 2.10 to approximate thevalue 1 on B B as closely as possible. We need our p Y, M q -harmonic functions to be B B -determined; the next lemma identifies a simple condition for functions of the form (74) to be B B -determined. Lemma 7.1.
Suppose p β, α j , d j q are points on H and suppose h p y, m q “ k ÿ j “ b p j qrp β, α j , d j q , ¨s , (74) k ě , is p Y, M q -harmonic. If | β | ă and | α j | ď then h is B B -determined. Proof.
Define the region U “ t y P Z ˆ Z ` : 0 ď y p q ´ y p q ď n u and the boundaries of U B U “ t y P Z ˆ Z ` : y p q ´ y p q “ n u and B U “ B B . Define υ n . “ inf t k : Y k P B U u . Wemake the following claim: starting from a point y P U , p Y, M q hits B U Y B U in finite time,i.e., υ n ^ τ ă 8 almost surely. Let us first prove this claim. For each modulating state m ,the sample path of p Y, M q consisting only of increments p , ´ q hits B U in at most n stepsand the probability of this path is p λ p m q P p m, m qq n . Then if we set ε “ min m P M p λ p m q P p m, m qq n we have P p y,m q p τ ^ υ n ě n q ď p ´ ε q . An iteration of this inequality and the Markov property of p Y, M q give P p y,m q p τ ^ υ n ě kn q ď p ´ ε q k . Letting k Ñ 8 gives P p y,m q p τ ^ υ n “ 8q “ . (75)Definition (74) and | α j | ď | β | ă h is bounded on B . This and that h is p Y, M q -harmonic imply that S k “ h p Y τ ^ υ n ^ k , M τ ^ υ n ^ k q is a bounded martingale. The optional sampling theorem applied to this martingale and (75)imply h p y, m q “ E p y,m q r h p Y τ ^ υ n , M τ ^ υ n qs (76) “ E p y,m q r h p Y τ , M τ q t τ ă υ n u s ` E p y,m q r h p Y υ n , M υ n q t υ n ď τ u s . That | α j | ď | h p Y υ n , M υ n q| ď cβ n for some constant c ą
0. Therefore,lim n Ñ8 E p y,m q r h p Y υ n , M υ n q t υ n ď τ u s ď c lim n Ñ8 β n “ . The last expression, that lim n Ñ8 υ n “ 8 and letting n Ñ 8 in (76) imply h p y, m q “ E p y,m q r h p Y τ , M τ q t τ ă8u s , i.e, h p y, m q is B B -determined.The last lemma and 0 ă ρ ă Lemma 7.2. h ρ is B B -determined. Recall that we have constructed a p Y, M q -superharmonic function, h ρ from the roots p ρ , q , p ρ , α ˚ q P H βα . We would like to strengthen this to a p Y, M q -harmonic function. Thisrequires the use of further conjugate points of p ρ , q (in addition to p ρ , α ˚ q ). The nextlemma shows that under Assumptions 1 and (27) we have sufficient number of conjugatepoints of p ρ , q to work with: 31 emma 7.3. Let p ρ , α ˚ q be the point conjugate to p ρ , q identified in Proposition 2.12.Under Assumptions 1 and (27) , there exists | M | ´ additional conjugate points p ρ , α ˚ j q , j “ , , ..., | M | , of p ρ , q with ă α ˚ j ă α ˚ . Proof.
We know that Λ p ρ , α ˚ q “
1; then Λ j p ρ , α ˚ q ă j “ , , ..., | M | . On the otherhand, Gershgorin’s Theorem implies lim α Ñ Λ j p ρ , α q “ 8 . These and the continuity of Λ j imply the existence of α ˚ j P p , α ˚ q such that Λ j p ρ , α ˚ j q “ . To construct our p Y, M q -harmonic functions from the points identified in the previouslemma we need the following assumption: c p ρ , , d q P Span ` c p ρ , α ˚ j , d ,j q , j “ , , ..., | M | ˘ . (77) Remark 7.1.
By definition, c p ρ , α j , d ,j q “ if α j “ ρ . Therefore, only those j satisfying α ˚ j ‰ ρ have a role in determining Span ´ c p ρ , α ˚ j , d ,j q , j “ , , ..., | M | ¯ . In this sense,assumption (77) can be seen as an extension of (31) (or, equivalently, of (28) ). Remark 7.2.
The linear independence of c p ρ , α ˚ j , d ,j q , j “ , , ..., | M | , is sufficient for (77) to hold. That c p β, β, d q “ implies that ρ ‰ α ˚ j for all j “ , , .., | M | is a necessarycondition for this independence. Now on to the p Y, M q -harmonic function: Proposition 7.1.
Let p ρ , α ˚ j q be the conjugate points of p ρ , q identified in Proposition 2.12and Lemma 7.3. Under the additional assumption (77) , one can find a vector b , P R m suchthat h ρ . “ rp ρ , , d q , ¨s ` | M | ÿ j “ b , p j qrp ρ , α ˚ j , d ,j q , ¨s (78) is p Y, M q -harmonic and B B -determined.Proof. Assumption (77) implies that the collection of vectors c p ρ , , d q , c p ρ , α ˚ j , d ,j q , j “ , , ..., | M | are linearly dependent. Therefore, by Proposition2.10, there exists a vector b P R | M |` such that b p qrp ρ , , d q , ¨s ` n ÿ k “ b p j qrp ρ , α ˚ j k , d ,j k q , ¨s is p Y, M q -harmonic. Assumption (77) implies that one can choose b so that b p q ‰ . Renormalizing the last display by b p q gives (78). That h ρ is B B -determined follows from0 ă α ˚ j ď ρ ă P p y,m q p τ ă 8q with bounded relativeerror from functions h ρ and h ρ . Proposition 7.2.
There exist constants c , c and c such that P p y,m q p τ ă 8q ă h a, p y, m q ă c P p y,m q p τ ă 8q (79) where h a, . “ c p h ρ ` c h ρ q . (80)32 roof. The proof is similar to that of Proposition 3.1. That 0 ă α ˚ j ă j “ , , ..., | M | imply that rp ρ , α ˚ j , d ,j q , p k, k, m qs “ p α ˚ j q k d ,j p m q Ñ k Ñ 8 . We further have rp ρ , , d q , p k, k, m qs “ d p m q ą , (82)for all k ě
0. This and (81) imply that there exists k ą h ρ p k, k, m q ą min m P M d p m q{ k ą k . On the other hand, h ρ p k, k, m q “ rp ρ , ρ , d q , p k, k, m qs “ d p m q ρ k ą , (84)for all k . Then we can choose c ą h ρ p k, k, m q ` c h ρ p k, k, m q ě min m P M d p m q{ k ď k . The last display, (83) and the positivity of c h ρ imply that the last displayholds for all k and m P M . Set c . “ ˆ min m P M d p m q{ ˙ ´ , and h a, be as in (80). That (85) holds for k ě m P M implies h a, | B B ě . By Lemma 7.2 and Proposition 7.1 h a, is p Y, M q -harmonic and B B -determined. This andthe last display imply, h a, p y, m q “ E p y,m q r h a, p Y τ , M τ q t τ ă8u s ě P p y,m q p τ ă 8q . (86)This proves the first inequality in (79). To choose c so that the second inequality in (79)holds we note the following: (81), (82) and (84) imply c . “ max k ě ,m P M h a, p k, k, m q ă 8 . Now the same argument giving (86) implies the second inequality in (79).
Proposition 7.3.
Fix m P M and x P R ` , such that ă x p q ` x p q ă ; furthermoreassume x p q ą if ρ ą ρ and x p q ď ´ log p ρ q{ log p ρ q ; set x n “ t nx u . Then h a, p T n p x n qq approximates P p x n ,m q p τ n ă τ q with relative error whose lim sup in n is bounded by | c ´ | .Proof. We know by the previous proposition that h a, approximates P p y,m q p τ ă 8q withrelative error bounded by | c ´ | ; we also know by Theorem 6.1 that P p T n p x n q ,m q p τ ă 8q approximates P p x n ,m q p τ n ă τ q with vanishing relative error. These twoimply the statement of the proposition. 33 Improving the approximation
Proposition 7.3 tells us that h a, of (80) approximates P p y,m q p τ ă 8q and therefore P p x,m q p τ n ă τ q with bounded relative error. The works [10, 11, 12] covering the non-modulated case areable to construct progressively better approximations (i.e., reduction of the relative error)by using more harmonic functions constructed from conjugate points (in the tandem casewith no modulation, one is able to construct an exact representation of P y p τ ă 8q so noreduction in relative error is necessary). This is possible because the function in [10, 11, 12]corresponding to h ρ , takes the value 1 on B B away from the origin. Thus, by and large, thatsingle function provides an excellent approximation of P y p τ ă 8q for points away from B .Rest of the harmonic functions are added to the approximation to improve the approximationalong B . When a modulating chain is present, the situation is different. Note that (81), (82) implythat the value of h ρ on B B , away from the origin, is determined by the eigenvector d andin general, the components of d will change with m . We need to improve h ρ itself so thatwe have a p Y, M q -harmonic function that is close to 1 on B B away from the origin.How is this to be done? Remember that the construction of h ρ began with fixing α “ β | M | p p β, q “
0; (87) ρ is the largest root of this equation in the interval p , q . Then we fixed β “ ρ in α | M | p p ρ , α q “ α to find the conjugate points p ρ , α ˚ j q of p ρ , q ; fromthese points we constructed h ρ . Now to get our p Y, M q -harmonic function that almost takesthe value 1 on B B away from the origin we will use the rest of the roots of (87) in p , q . Thenext lemma shows that under the stability assumption and the simpleness of all eigenvalues, | M |´ β roots exist that lies in the interval p , ρ q . The proposition after that constructsthe desired p Y, M q -harmonic function from these roots. Lemma 8.1.
Under the stability assumption (1), and Assumption 1 ( all eigenvalues of A p β, α q are real and simple for p β, α q P R o ` ) there exist ρ ,j , j “ , , ..., | M | , such that ρ ą ρ , ą ρ , ą ¨ ¨ ¨ ą ρ , | M | ą and t e ‰ , e ‰ ,..., e | M | ‰ u Ă R M such that A p ρ ,j , q e j “ e j , j “ , , ..., | M | , holds. The proof is parallel to that of Lemma 7.3 and is based on Gershgorin’s Theorem and thefact that Λ j p ρ , q ă j “ , , ..., | M | . Each of the points p ρ ,j , q will in general have 2 | M | ´ B B -determined p Y, M q -harmonic functions from these we need the analog of (77) for each p ρ ,j , q : Assumption 2.
For each j “ , , ..., | M | there exists m j ď | M | conjugate points p ρ ,j , α ˚ j,l q , l “ , , ..., m j , of p ρ ,j , q and eigenvectors ‰ e j,l P R M such that | α ˚ j,l | ă , l “ , , ..., m j , A p ρ ,j , α ˚ j,l q e j,l “ e j,l c p ρ ,j , , e j q P Span p c p ρ ,j , α ˚ j,l , e j,l q , l “ , , .., m j q . (88)34 emark 8.1. Similar to the comments made in Remark 7.2, a set of sufficient conditionsfor (88) is 1) m j “ | M | and 2) c p ρ ,j , α ˚ j,l , e j,l q , l “ , , ..., | M | are linearly independent.By c p¨ , ¨ , ¨q ’s definition, linear independence of these vectors require α ˚ j,l ‰ ρ ,j , which is, yetanother generalization of the assumption ρ ‰ ρ . Remark 8.2.
One can introduce assumptions similar to (27) which imply, with an argumentsimilar to the proof of Lemma 7.3, that p ρ ,j , q has | M | ´ j conjugate points in the interval p , q . But in general, this number of conjugate points will not suffice for (88) to hold andwhen constructing p Y, M q -harmonic functions with β “ ρ ,j , j “ , , ..., | M | , we will useconjugate points with complex or negative α components. Instead of introducing even moreassumptions similar to (27) , we directly incorporate (88) as an assumption. To get our p Y, M q -harmonic function converging to 1 on the tail of B B (see (92) belowfor the precise statement) we need one more condition: P Span p d , e , ..., e | M | q . (89)A sufficient condition for (89) is that the vectors listed on the right of this display are linearlyindependent. Proposition 8.1.
Let e j , j “ , , ..., | M | be as in Lemma 8.1. and let d be as in Proposition2.11. Under Assumptions 2 and (89) there exist vectors b ,j P R m j , j “ , , .., | M | and b P R | M | such that h ρ ,j p y, m q . “ rp ρ ,j , , e j q , p y, m qs (90) ` m j ÿ l “ b ,j p l qrp ρ ,j , α ˚ j,l , e j,l q , p y, m qs , j “ , , ..., | M | , and h . “ b p q h ρ ` | M | ÿ j “ b p j q h ρ ,j (91) are all p Y, M q -harmonic and B B -determined; furthermore lim k Ñ8 h p k, k, m q Ñ for all m P M . Proof.
The existence of the vector b ,j , j “ , , ..., | M | , so that h ρ ,j defined in (90) is p Y, M q -harmonic follows from (2) and the argument given in the construction of h ρ (see the proofof Proposition 7.1). By (89) there is a vector b such that b p q d p m q ` | M | ÿ j “ b p j q e j p m q “ m P M . Then h as defined in (91) satisfies h p k, k, m q “ ` b p q | M | ÿ j “ b , p j qp α ˚ j q k d ,j p m q ` | M | ÿ j “ b p j q m j ÿ l “ b ,j p i qp α ˚ j,l q k e j,l p m q ; | α ˚ j | ă | α ˚ j,l | ă k . This gives (92).35n Lemma 8.1 we found points on t α “ uX H βα in addition to p ρ , q identified in, we usedthese points above in the construction of h . Similarly, one can go along the line β “ α to findpoints on H βα other than p ρ , ρ q defining further simple B B -determined p Y, M q -harmonicfunctions: Lemma 8.2.
Under the stability assumption (1), and Assumption 1 ( A p β, α q has real distincteigenvalues for p β, α q P R ,o ` q there exist ρ ,k , k “ , , .., | M | , such that ρ ą ρ , ą ρ , ą¨ ¨ ¨ ą ρ , | M | ą and t f ‰ , f ‰ ,..., f | M | ‰ u Ă R | M | such that A p ρ ,j , ρ ,j q f j “ f j , j “ , , ..., | M | , holds. The proof is parallel to that of Lemma 7.3 and is based on Gershgorin’s Theorem and thefact that Λ j p ρ , ρ q ă Λ p ρ , ρ q “ j “ , , ..., | M | . One can use the points identified in the previous lemma to construct further B B -determined p Y, M q -harmonic functions. Lemma 8.3.
Let ρ ,j , f j , j “ , , ..., | M | , be as in Lemma 8.2. Then rp ρ ,j , ρ ,j , f j q , ¨s , j “ , , ..., | M | , are B B -determined p Y, M q -harmonic.Proof. By definition, p ρ ,j , ρ ,j q P H βα and A p ρ ,j , ρ ,j q f j “ f j . Again, A p β, β q “ A p β, β q for all β follows from and p p β, β, m q “ p p β, β q and the definitions of A and A . Then A p ρ ,j , ρ ,j q f j “ A p ρ ,j , ρ ,j q f j “ f j , i.e., p ρ ,j , ρ ,j , f q P H (i.e., the characteristic surfaceof B , see (18)). This and Proposition 2.7 imply that rp ρ ,j , ρ ,j , f j q , ¨s is p Y, M q -harmonic.That it is B B -determined follows from | ρ ,j | ă rp β, α, d q , ¨s is complex valued for any p β, α, d q P H with complex componentsand such points and the functions they define can also be used to improve the approximation;see the next section for an example. The next proposition gives an upper bound on therelative error of an approximation of P p y,m q p τ ă 8q in terms of the values the approximationtakes on the boundary B B ; it covers cases when complex valued p β, α, d q P H is used in theconstruction of the approximation. For any z P C , let ℜ p z q denote its real part. Proposition 8.2.
Let h : Z ˆ Z ` ÞÑ C be B B -determined and p Y, M q -harmonic. Then max p y,m qP B ˆ M | ℜ p h qp y, m q ´ P p y,m q p τ ă 8q| P p y,m q p τ ă 8q ď c ˚ (93) where c ˚ . “ max y PB B,m P M | h p y, m q ´ | . (94)The proof is similar to that of Proposition 7.2: Proof.
That h is B B -determined p Y, M q -harmonic implies the same for its real and imaginaryparts. For any complex number z we have | ℜ p z q ´ | ď | z ´ | ; these and (94) givemax y PB B,m P M | ℜ p h qp y , m q ´ | ď c ˚ . Then p ´ c ˚ q t τ ă8u ď ℜ p h qp Y τ , M τ q t τ ă8u ď p ` c ˚ q t τ ă8u . Applying E p y,m q r¨s to all terms above implies (93).36 Numerical example
This section demonstrates the performance of our approximation results on a numericalexample. For parameter values P , λ p¨q , µ p¨q and µ p¨q we take those listed in (16) and (17),for which | M | “ . We know by Proposition 2.6 that for P as in (16), A p β, α q has distinctpositive eigenvalues for p β, α q P R ,o ` . Furthermore, the rates (17) satisfy λ p m q ă µ p m q , µ p m q for all m P M , therefore, the stability assumption (1) is also satisfied. Computing the rightside of (27) at p ρ , q shows that the parameter values (16) and (17) satisfy (27). Therefore:1. By Proposition 7.1, the function h , is well defined and B B -determined and p Y, M q -harmonic. Furthermore, we know by Lemma 8.1 that there are ρ ,j , j “ , , ..., | M | ,such that 0 ă ρ ,j ă ρ and p ρ ,j , q P H βα for all j . We solve p p ρ ,j , α q “ α for the parameter values assumed in the section and verify that Assumption 2holds with m j “ | M | for all j ; this and Proposition 8.1 imply that the p Y, M q -harmonic B B -determined function h defined in (91) and satisfying (92) is well defined.2. Propositions 2.8, 2.9 and Lemma 7.2 apply and give the B B -determined p Y, M q -harmonicfunction h ρ “ rp ρ , ρ , d q , ¨s ,3. Lemmas 8.2 and 8.3 apply and give the B B -determined p Y, M q -harmonic functions h ρ ,j “ rp ρ ,j , ρ ,j , f j q , ¨s , j “ , , .... | M | .In addition to these functions, we can fix an integer K ą
0, and construct K ¨ | M | further p Y, M q -harmonic functions of the form h k,j . “ | M | ÿ l “ b k,j p l qrp β k,j , α k,j,l , d k,j,l q , ¨s , (95)for k “ , , ..., K , β k,j and j “ , , ...., | M | , as follows:1. Set α k,j, “ R e ik πK ` , R P p , q to be determined below; note that α k,j, depends onlyon k ; including j as an index simplifies notation in (95) and below.2. For each k , β k,j , j “ , , ...., | M | , are the β -roots of p p α k,j, , β q “ | β | ă α k,j,l , l “ , , ..., | M | , are the α -roots of p p β k,j , α q “
0; (97)with | α | ă α k,j, .4. d k,j,l is an eigenvector of A p β k,j , α k,j,l q i,e., p β k,j , α k,j,l , d k,j,l q P H ,5. for each p k, j q the vector b k,j solves | M | ÿ l “ b k,j p l q c p β k,j , α k,j,l q “ , (98)37here b k,j p l q is the l th component of the vector b k,j . For h k,j , k “ , , ..., K and j “ , , ... | M | to be well defined, p Y, M q -harmonic and B B -determined we need 1) for each k , the equation(96) needs to have at least | M | β -roots with absolute value less than 1; 2) for each k and j , the equation (97) needs to have at least | M | solutions different from α k,j, with absolutevalue less than 1; 3) for each k and j the equation (98) needs to have a nontrivial solution b k,j . Here we have two parameters to set: K and R ; for the purposes of this numericalexample we set R “ .
7, and K “
5. Upon solving (96), (97) and (98) with these parametervalues we observe that they have sufficient number of solutions for h k,j to be well defined and p Y, M q -harmonic and B B -determined.We have now 1 ` | M | , B B -determined p Y, M q -harmonic functions to construct our ap-proximation of P p y,m q p τ ă 8q ; the approximation will be of the form h a,K . “ ℜ p h a ˚ ,K q , h a ˚ ,K . “ h ` φ h ρ ` | M | ÿ j “ φ j rp ρ ,j , ρ ,j , d ,j q , ¨s ` K, | M | ÿ k “ ,j “ φ j,k h j,k , (99)where φ j and φ j,k are C valued coefficients to be chosen so that h a,K | B B is as close to 1 aspossible. As in [10, Section 8.2], one simple way to do this is to choose these p K ` q| M | coefficients so that h a,K p y, y, m q “ y “ , , , .., K and m P M . This defines a p K ` q| M | ˆ p K ` q| M | system; for our parameter values ( K “ | M | “
3) this is an 18 ˆ φ j and φ j,k are determinedthrough this solution, an upper bound on the approximation relative error can be computedvia Proposition 8.2; it suffices to compute c ˚ of (94); for h a ˚ ,K of (99) it turns out to be c ˚ “ . h a,K approximates P p y,m q p τ ă 8q with relative error boundedby this quantity. By Theorem 6.1 we know that P p T n p x n q ,m q p τ ă 8q approximates P p x n ,m q p τ n ă τ q with vanishing relative error for x n “ t nx u , x p q ą
0; it follows from these that h a,K p n ´ x n p q , x n p qq will approximate P p x n ,m q p τ n ă τ q with relative error bounded by c ˚ for n large. Let us see how well this approximation works in practice. Figure 6 gives the levelcurves of ´ log p h a,K p n ´ x p q , x p q , qq and ´ log P p x,m q p τ n ă τ q ; P p x,m q p τ n ă τ q is computedby iterating the harmonic equation satisfied by this probability; for n “
60, this iterationconverges in less than 1000 steps. As can be seen, and agreeing with the analysis above,these lines completely overlap except for a narrow region around the origin.38 x x Figure 6: Level curves of ´ log p h a,K p n ´ x p q , x p q , qq and ´ log P p x,m q p τ n ă τ q x x Figure 7: The relative error | log p h a,K p n ´ x p q ,x p q , q´ log P x p τ n ă τ q|| log P x p τ n ă τ q| | log p h a,K p n ´ x p q , x p q , qq ´ log P p x,m q p τ n ă τ q|| log P p x,m q p τ n ă τ q| , we see that it is virtually 0 except for the same region around 0 where it is bounded by 0 . . This narrow layer of where the relative error spikes corresponds to the region 1 ´ x p q ă log p ρ q{ log p ρ q identified in Theorem 6.1.
10 Comparison with earlier works
The present work shows how one can approximate the probability P p x,m q p τ n ă τ q by P p y,m q p τ ă 8q with exponentially vanishing relative error and constructs analytical approxi-mation formulas for the latter. This is done by extending the approach of [10, 11] to Markovmodulated dynamics. In this section, we compare the analysis of the modulated case treatedin this work with the non-modulated two tandem case treated in [10, 11] and the non-modulated two dimensional simple random walk treated in [12]. Harmonic functions
The nonmodulated analysis uses functions of the form y ÞÑ rp β, α q , y s “ β y p q´ y p q α y p q where p β, α q are chosen from the roots of a characteristicpolynomial of second order associated with the process Y . Markov modulation brings anadditional state variable m , leading to functions of the form p y, m q ÞÑ rp β, α, d q , p y, m qs “ β p y p q´ y p q α y p q d p m q . The characteristic surface is now defined in terms of eigenvalue andeigenvector equations of a characteristic matrix depending on p β, α q P C . Geometry of the characteristic surface
The characteristic surface in [10, 11, 12] is the1-level curve of a rational function which can be represented as a second degree polynomialin each of the β, α variables; the projection of the characteristic surface to R ` is a smoothclosed curve bounding a convex region. Conjugate points on this curve come in pairs andhave elementary formulas. The characteristic curve in the modulated case is the 0-levelcurve of the characteristic polynomial of a characteristic matrix and can be represented asa 2 | M | degree polynomial in each of the variables; its projection to R ` consists of | M | components, one for each eigenvalue Λ j of the characteristic matrix. The error analysis isbased on the level curve of the largest eigenvalue while the computation of P p y,m q p τ ă 8q uses points on all components. There are in general no simple formulas for the roots of apolynomial greater than degree 4 and the formulas for degree 4 are fairly complex; therefore,for | M | ě Assumptions
We use the point p ρ , q and its conjugate p ρ , α ˚ q lying on L to define p Y, M q -superharmonic functions to use in our limit analysis. The existence of p ρ , q P L ,follows from the stability assumption (1). The identification of the conjugate point p ρ , α ˚ q requires the additional assumption (27) ensuring α ˚ ă
1. A similar assumption is not neededin the non-modulated tandem case, because when there is no modulation, i.e., when | M | “ p ρ , q is p ρ , ρ q and ρ ă r ă ρ ρ , where r is utilization rate of the whole system.The assumption ρ ‰ ρ (see (28)) generalizes the assumption µ ‰ µ from the non-modulated tandem case and the parallel case treated in [11, 12]. The computation of P p y,m q p τ ă 8q needs progressively more general versions of this assumption (see (77), Remark7.2 and Assumption 2). Analysis
The approximation error analysis in the non-modulated case is based on thesubsolutions of a limit HJB equation and Y -harmonic functions. These works use thesesubsolutions to construct supermartingales which are then used to find upper bounds onerror probabilities. In this work we construct the supermartingales directly using p Y, M q - super harmonic functions constructed from points on the characteristic surface. Because Y has one less constraint compared to X , these functions can be subharmonic on the boundarywhere Y is not constrained. To overcome this, we introduce a decreasing term to the definitionof the supermartingale.In the tandem case there is an explicit formula for P y p τ ă 8q ; this formula is useddirectly in the analysis of the error probability. There is obviously no explicit formula for thecorresponding probability in the Markov modulated case. Instead, we derive an upper boundon it in Section 3 using again p Y, M q -superharmonic functions; this upperbound is used inthe error analysis of Section 6. Computation of the limit probability
In the non-modulated tandem case treated in[11], P y p τ ă 8q can be represented exactly as a linear combination of h ρ and h ρ ; so thecomputation of P y p τ ă 8q is trivial for the nonmodulated two dimensional tandem walk. Inthe parallel case treated in [12], P y p τ ă 8q can be represented exactly as a linear combinationof h ρ and h r when r “ ρ ρ ; when this doesn’t hold [12] develops approximations of P y p τ ă 8q from harmonic functions constructed from conjugate points on the characteristicsurface, which is an application of the principle of superposition. For the modulated case weuse the same principle but Markov modulation complicates the construction of the functionsused in the approximation. The identification of the points on the characteristic surfacerequires the solution of 2 | M | degree polynomial equations (first the α component is fixed toidentify possible β components; then for each of the identified β ’s, the polynomial is solved in α to find the relevant conjugate points). Eigenvectors corresponding to these points are thencomputed and finally we solve a linear equation to find the coefficients of the exponentialfunctions (see, for example, the b k,j vector in (95) and (98)). The corresponding process istrivial when there is no modulation. In [11] and [12] the function h ρ plays a central role inthe approximation of P y p τ ă 8q because it equals approximately 1 away from the origin; dueto Markov modulation there can be in general no function constructed from a single pointand its conjugates that takes a fixed value on B B. To deal with this, we use an appropriatelinear combination of functions constructed from multiple points and their conjugates on thecharacteristic surface so that the linear combination takes the value 1 away from the origin(Proposition 8.1).
11 Conclusion
The current work develops approximate formulas for the exit probability of the two dimen-sional tandem walk with modulated dynamics. Our main approximation Theorem 6.1 says41hat P p T n p x n q ,m q p τ ă 8q approximates P p x n ,m q p τ n ă τ q with relative error vanishing expo-nentially fast with n . To compute the exit probability, we first construct B B -determined p Y, M q -harmonic functions from single and conjugate points on the corresponding character-istic surface and then with their linear combinations, approximate the boundary value 1 ofthe harmonic function P p y,m q p τ ă 8q . In the non-modulated tandem case treated in [10], theprobability P y p τ ă 8q can be represented in any dimension exactly using harmonic functionsconstructed from points on the characteristic surface. As is seen in the present work, evendimension two entails considerable difficulties. Whether an extension to higher dimensions ispossible is a question we would like to tackle in future work.The work [10] gives a formula for P y p τ ă 8q for the non-modulated tandem walk when ρ “ ρ based on harmonic functions with polynomial terms. Whether similar computationscan be carried out for P p y,m q p τ ă 8q in the modulated case when ρ “ ρ is another questionfor future research.The assumption (27) plays a key role in our analysis; it ensures that various functionssuch as h ρ whose construction involves the point p ρ , α ˚ q remain bounded on B B. We thinkthat new ideas will be needed to treat the case when (27) doesn’t hold; this remains for futurework.The computations and the error analysis in the present work depend on the dynamics ofthe process and the geometry of the exit boundary. A significant problem for future researchis to extend these to other dynamics in two or higher dimensions and to other exit boundaries.The simple random walk dynamics (i.e., increments p , q , p´ , q , p , q and p , ´ q ) and therectangular exit boundary appear to be the most natural to study in immediate future work. A Two lemmas
For a square matrix G , let G i,j denote the matrix obtained by removing the i th row and j th column of G . Lemma A.1.
For n P t , , , ... u , suppose G is an n ˆ n irreducible and aperiodic matrixwith nonnegative entries. Then det ` p Λ p G q I ´ G q i,i ˘ ą for all i P t , , ..., n u , where I isthe n ˆ n identity matrix.Proof. The argument is the same for all i P t , , ..., n u ; so it suffices to argue for i “ . Suppose the claim is not true anddet ` p Λ p G q I ´ G q , ˘ ď . (100)Consider the function u ÞÑ g p u q “ det ` p u I ´ G q , ˘ , u ě . The multilinearity and continuityof det implies lim u Õ8 g p u q “ 8 . This implies that if (100) is true there must be u ě Λ p G q such that det ` p u I ´ G q , ˘ “ . (101)The matrix G , is nonnegative, therefore, it has a largest eigenvalue Λ p G , q with an eigen-vector v ě
0. The equality (101) impliesΛ p G , q ě u ě Λ p G q . (102)That G is irreducible and aperiodic implies that G n is strictly positive; its largest eigenvalueis Λ p G n q “ Λ p G q n . p G n q , has strictly positive entries and therefore its largest eigenvalue Λ pp G n q , q has an eigenvalue v with strictly positive entries. For two vectors x, y P R d , let x ě y and x ą y denote componentwise comparison. The inequality p G n q , ě p G , q n implies p G n q , v ě Λ p G , q n v . (103)On the other hand Λ pp G n q , q “ sup t c : D x P R n ´ ` , p G n q , x ě cx u , (104)(see [5, Proof of Theorem 1, Chapter 16]). This and (103) implyΛ pp G n q , q ě Λ p G , q n . (105)Define v “ r v s P R n ; it follows from p G n q , v “ Λ pp G n q , q v , the strict posi-tivity of the components of G n and v that one can choose δ ą G n v ą ` Λ pp G n q , ` δ ˘ v ;This and Λ p G n q “ sup t c : D x P R n ` , G n x ě cx u imply Λ p G n q ą Λ ` p G n q , ˘ . The last inequality, (105) and (102) implyΛ p G q n “ Λ p G n q ą Λ pp G n q , q ě Λ p G , q n ě Λ p G q n , which is a contradiction.In our analysis we need the following fact from [9]; its proof is elementary and followsfrom the multilinearity of the determinant function and the previous lemma. Lemma A.2.
Let G be an aperiodic and irreducible transition matrix. Then the row vectorwhose i th component equals det ` p I ´ G q i,i ˘ is the unique (upto scaling by a positive number)left eigenvector associated with the eigenvalue of G . References [1] Abraham Berman and Robert J Plemmons,
Nonnegative matrices in the mathematicalsciences , vol. 9, Siam, 1994.[2] Paul Dupuis, Ali Devin Sezer, and Hui Wang,
Dynamic importance sampling for queueingnetworks , Annals of Applied Probability (2007), no. 4, 1306–1346.[3] Rick Durrett, Probability: theory and examples, 4th edition , Cambridge University press,2010.[4] Paul Glasserman and Shing-Gang Kou,
Analysis of an importance sampling estimator fortandem queues , ACM Transactions on Modeling and Computer Simulation (TOMACS) (1995), no. 1, 22–42. 435] Peter D Lax, Linear Algebra. Pure and Applied Mathematics , (1996).[6] Shyam Parekh and Jean Walrand,
A quick simulation method for excessive backlogs innetworks of queues , IEEE Transactions on Automatic Control (1989), 54–66.[7] NU Prabhu and Yixin Zhu, Markov-modulated queueing systems , Queueing systems (1989), no. 1-3, 215–245.[8] Denis Serre, Matrices: theory and applications, volume 216 of Graduate Texts in Math-ematics, second edition , 2010.[9] Ali Devin Sezer,
Importance sampling for a Markov modulated queuing network , Stochas-tic Processes and their Applications (2009), no. 2, 491–517.[10] ,
Exit probabilities and balayage of constrained random walks , arXiv preprintarXiv:1506.08674 (2015).[11] ,