Boundary effect in competition processes
aa r X i v : . [ m a t h . P R ] M a y Boundary effect in competition processes
Vadim Shcherbakov ∗ Stanislav Volkov † Abstract
This paper is devoted to studying the long-term behaviour of a continuous timeMarkov chain that can be interpreted as a pair of linear birth processes which evolvewith a competitive interaction; as a special case, they include the famous Lotka-Volterra interaction. Another example of our process is related to urn models withball removals. We show that, with probability one, the process eventually escapes toinfinity by sticking to the boundary in a rather unusual way.
Keywords:
Markov chain, birth-and-death process, competition process, Friedman’s urnmodel, Lyapunov function, martingale.
Subject classification:
In this paper we study the long term behaviour of a continuous time Markov chain (CTMC)with values in Z , where Z + is the set of all non-negative integers, defined on a certainprobability space with probability measure P . The Markov chain jumps only to the nearestneighbours, and we consider two types of transition rates described below. Transition rates of type I.
Given the state ( x , x ) ∈ Z the Markov chain jumps to( x + 1 , x ) with rate λ + α x , ( x , x + 1) with rate λ + α x , ( x − , x ) with rate x g ( x ) if x > , ( x , x −
1) with rate x g ( x ) if x > , (1)where α i , λ i > , i = 1 ,
2, and g i , i = 1 , a competition process with non-linearinteraction (specified by functions g and g ) . ∗ Department of Mathematics, Royal Holloway, University of London, UK. Email address:[email protected] † Centre for Mathematical Sciences, Lund University, Sweden. Email address: [email protected] ransition rates of type II. Given the state ( x , x ) ∈ Z the Markov chain jumps to( x + 1 , x ) with rate λ + α x , ( x , x + 1) with rate λ + α x , ( x − , x ) with rate β x if x > , ( x , x −
1) with rate β x if x > , (2)where α i , λ i ≥ , i = 1 , β i > , i = 1 ,
2. We call the Markov chain withtransition rates of type II a competition process with linear interaction .Competition processes with both non-linear and linear interaction belong to a class ofcompetition processes introduced in [12] as a natural two-dimensional analogue of thebirth-and-death process in Z + . In [12], the competition process was defined as a CTMCwith values in Z , where the transitions are allowed only to the nearest neighbour states(see Section 3 below). This definition was generalised to a multidimensional case in [4], [5].Some basic models of competition processes are discussed in [1]. The term “a competitionprocess” was apparently coined due to the fact that original examples of such processeswere motivated by modelling a competition between populations (e.g. see [12, Examples 1and 2] and references therein). One of the most known competition processes is the onespecified by the famous Lotka-Volterra interaction. In our notation, the Lotka-Volterrainteraction corresponds to functions g i ( z ) = z , i = 1 , λ i = 0, i = 1 , g = g ≡ β = β = 0 in (2), then the cor-responding Markov chain is formed by two independent linear birth processes with immig-ration. Non-zero death rates determine competitive interaction between the components.Therefore, the competition processes of interest can be naturally embedded into a moregeneral technical framework of multivariate Markov processes formulated in terms of locallyinteracting birth-and-death processes. In the absence of interaction components of such aMarkov process evolve as a collection of independent birth-and-death processes, which longterm behaviour is well known. Namely, given a set of transition rates one can, in principle,determine whether the corresponding birth-and-death process is recurrent/positive recur-rent, or transient/explosive, and compute various characteristics of the process. However,the presence of interaction can significantly change the collective behaviour of the system(e.g. see [6], [10] and [13]).Furthermore, note that a discrete time Markov chain, DTMC for short, correspondingto the competition process with linear interaction, can be regarded as an urn model withball removals. In the symmetric case, that is α = α , β = β and λ = λ , this DTMCis similar, in a sense, to Friedman’s urn model. This similarity enabled us to adapt theFreedman’s method for Friedman’s urn model ([3]) in order to obtain a key fact in the2roofs. We discuss this method in detail in Section 2.3.1 below. If both α = α = 0 and λ = λ = 0, then DTMC corresponding to the competition process with linear interactioncoincides with the well-known OK Corral model (see e.g. [7]). If α = α = 0 and λ , λ > i ∗ = ( , if i = 1;1 , if i = 2(i.e. “the other coordinate”). Theorem 1.
Let ξ ( t ) be a competition process with non-linear interaction (transition ratesof type I) specified by functions g , g : [0 , ∞ ) → [0 , ∞ ) . Assume that • g and g are regularly varying functions with indexes ρ > and ρ > respectively,such that g (0) = g (0) = 0 and g ( x ) , g ( x ) > for all x > ; • α i , λ i > , i = 1 , .Then ξ ( t ) is a non-explosive transient CTMC and P ( B ∪ B ) = 1 , where B i = (cid:26) ξ i ( t ) → ∞ , t →∞ ξ i ∗ ( t ) < lim sup t →∞ ξ i ∗ ( t ) = 1 (cid:27) , i = 1 , . Theorem 2.
Let ξ ( t ) be a competition process with linear interaction (transition rates oftype II) specified by parameters α i ≥ , λ i > and β i > , i = 1 , . Then ξ ( t ) is anon-explosive transient CTMC and P ( A S A ) = 1 , where for i = 1 , A i = (cid:26) lim t →∞ ξ i ( t ) = ∞ , lim inf t →∞ ξ i ∗ ( t ) = 0 , lim sup t →∞ ξ i ∗ ( t ) = κ i ⋆ (cid:27) and κ i = κ i ( α i ∗ ) = ( , if α i ∗ > , if α i ∗ = 0 . (3)The proof of each theorem consists of two parts. First, we show that, with probabilitybounded below uniformly over the initial position on the coordinate axes, the process sticksto the boundary of the quarter plane in the following way. Namely, one of the componentsof the process tends to infinity while the other component takes only values 0 and 1 (0, 1and 2 in the special case of Theorem 2) oscillating infinitely often between these values,as described. This is what we call the boundary effect. This step of the proof is thesubject of Lemma 1 (Theorem 1) and Lemma 4 (Theorem 2). In order to prove eachlemma we construct a so-called Lyapunov function for the well-known transience criterion3f a countable Markov chain (e.g. see [9, Theorem 2.5.8]). This allows us to show thatthe Markov chain is confined to a strip along the boundary, as described. Then this fact iscomplemented by an argument based on the Borel-Cantelli lemma that gives the oscillationeffect, i.e. the process transits from one level of the strip to another infinitely often. Inboth cases, this implies that the Markov chain under consideration is transient and, withpositive probability, escapes to infinity in a certain way.Intuitively, it seems rather clear that if the process is already at the boundary, then itprefers to stay near the boundary in the future. We use the Lyapunov function method totransform this intuition into a rigorous argument in Lemmas 1 and 4. Although a directprobabilistic proof might be possible in proving both lemmas, we prefer to use the generalmethod, which can be used in other cases, where a direct probabilistic argument mightbecome cumbersome. For example, this is the case in the model in [10], where a similarboundary effect was originally observed for a pair of interacting birth-and-death processes.In fact, we borrow the idea of the construction of the Lyapunov function from that article.Another key step of the proof of both theorems consists of showing that the processhits the boundary with probability one. Note that in applications of competition processesto population modelling the hitting time is interpreted as the extinction time of one ofthe competing populations. Therefore, determining whether the hitting time is finite is ofinterest in its own right. Sufficient conditions for finiteness of the hitting time and its meanare given in [12] for competition process in Z . These conditions are rather restrictive,which is not surprising as these conditions were obtained for very general assumptions onthe transition characteristic of the competition process. For example, it is possible to usethem only in some special cases of competition processes in Theorems 1 and 2 (see a dis-cussion in Section 3). We use a direct probabilistic argument in order to show finiteness ofthe mean hitting time in the case of the competition process with non-linear interaction inTheorem 1 and also in the case of the competition process with linear interaction in The-orem 2 under assumption α α < β β . However, neither this argument nor results of [12]can be applied in the case of the competition process with linear interaction (Theorem 2)under assumption α α > β β . In this particular case showing that the process hits theboundary almost surely is somewhat reminiscent to showing non-convergence to an un-stable equilibrium in processes with reinforcement (e.g. urn models). Often the method ofstochastic approximation is used to show such non-convergence (see e.g. [11] and referencestherein). Further, showing that the hitting time is finite in this case of the linear modelis similar also to showing that a non-homogeneous random walk exits a cone, where theLyapunov function method proved to be useful (e.g. see [8] and references therein).Although our model is similar to both urn models with ball removals, and to non-homogeneous random walks, we were unable to apply the above research techniques andused a different method instead. Our method is a modification of a method used in [3]for studying Friedman’s urn model. The original method consists of estimating momentsof certain martingales related to the process of interest. The similarity of the competitionprocess with linear interaction with Friedman’s urn model allows us to adapt this idea (see4ection 2.3.1 for details). In what follows, E denotes the expectation with respect to the probability measure P . Lemma 1.
There exists ε > , depending on the model parameters only, such that inf x ≥ P (cid:16) ˜ A | ξ (0) = ( x , (cid:17) ≥ ε and inf x ≥ P (cid:16) ˜ A | ξ (0) = (0 , x ) (cid:17) ≥ ε, where ˜ A i = { ξ i ( t ) → ∞ and ξ i ⋆ ( t ) ∈ { , } , ∀ t ≥ } . Proof of Lemma 1.
We prove only the first bound of the lemma, that is when the processstarts at ξ (0) = ( x , x = x and y = x in the rest of the proof. Given positive numbers ν and µ define the following function on Z \ (0 , f ( x, y ) = x − ν − x − µ , if y = 0; x − ν , if y = 1;1 , if y ≥ . (4)In the rest of the proof of this lemma we assume that0 < ν < µ < min( ρ , ρ ) . (5)Denote G the generator of CTMC ξ ( t ) with transition rates (1). From state ( x, x >
0, transitions are possible only to states ( x + 1 ,
0) and ( x,
1) with rates λ + α x and λ respectively. Therefore, G f ( x,
0) = ( λ + α x ) (cid:0) ( x + 1) − ν − x − ν − ( x + 1) − µ + x − µ (cid:1) + λ x − µ . (6)Given γ >
0, Taylor’s expansion formula shows that( x ± − γ − x − γ = ∓ γx − − γ + o (cid:0) x − − γ (cid:1) (7)for sufficiently large x >
0. Applying this expansion for the polynomial terms on the righthand side of (6) we obtain that G f ( x, ≤ x − ν (cid:0) − α ν + ( α µ + λ ) x − µ + ν + o (1) (cid:1) ≤ , (8)for all sufficiently large x , as 0 < ν < µ . 5ext, given state ( x, x >
0, the Markov chain can jump only to states ( x +1 , x − , x,
2) and ( x, λ + α x , x · g (1), λ + α and g ( x ) · G f ( x,
1) = ( λ + α x ) (cid:0) ( x + 1) − ν − x − ν (cid:1) + xg (1) (cid:0) ( x − − ν − x − ν (cid:1) + ( λ + α ) (cid:0) − x − ν (cid:1) + g ( x ) (cid:0) x − ν − x − µ − x − ν (cid:1) = − g ( x ) x − µ + O (1)by applying the expansion (7). Recall that g is a regularly varying function with index ρ >
0, that is g ( x ) = x ρ l ( x ), where l is a slowly varying function (e.g. see [2] fordefinitions). Since µ < min( ρ , ρ ) (see (5)), we get that g ( x ) x − µ = x ρ − µ l ( x ) → ∞ as x → ∞ . This results in G f ( x, ≤ x . Define the following stopping time σ = inf( t : ξ ( t ) / ∈ { x ≥ N + 1 and y ≤ } ) , where integer N is such that the bounds (8) and (9) hold for all x > N . These boundsimply that the random process Z ( t ) := f ( ξ ( t ∧ σ )) is a supermartingale. Since Z ( t ) ≥ Z ( t ) converges almost surely to a finite limit Z ∞ . Next, note that onthe event { σ = ∞} we must have ξ ( t ) → ∞ , otherwise, if lim sup t →∞ ξ ( t ) = A < ∞ ,then Z ( t ) cannot converge due to the fact that f is not constant on set { , , . . . , A }×{ , } which is irreducible for the chain. Consequently, Z ∞ = ( f ( N,
0) = N − ν − N − µ or f ( N,
1) = N − ν , if σ < ∞ , , if σ = ∞ . Assume that the initial position of the process is ( x, x ≥ N + 1. By the optionalstopping theorem( N − ν − N − µ ) P ( σ < ∞ ) ≤ E ( Z ∞ ) ≤ Z (0) = f ( x,
0) = x − ν − x − µ , so that P ( σ < ∞ ) ≤ f ( x , f ( N, ≤ f ( N + 1 , f ( N,
0) = 1 − ε ′ < ε ′ >
0, due to the monotonicity of the function x − ν − x − µ for positive x . Thus,if ξ (0) = ( x, x ≥ N + 1, then, with probability at least ε ′ , the process ξ ( t ) staysin set { N + 1 , N + 2 , . . . } × { , } forever. Further, for each initial position ( x, x ∈ { , , . . . , N } , with a strictly positive probability, the process reaches state ( N + 1 , { y = 0 , } ∈ Z (e.g. by just jumping only to the right). Consequently, P ( σ = ∞| ξ ( t ) = ( x, x ≥
0. On this event ξ ( t ) → ∞ a.s. 6 emma 2 (Lemma 7.3.6 in [9]) . Let Y t ≥ , t ≥ , be a process adapted to a filtration G t , t ≥ , and let T be a stopping time. Suppose that there exists ε > such that E [ d Y t |G t − ] ≤ − ε dt on { t ≤ T } . Then E [ T |G ] ≤ Y /ε . Lemma 3.
Define τ = inf { t : ξ ( t ) = 0 or ξ ( t ) = 0 } . Then τ is a.s. finite.Proof of Lemma 3. It is easy to see that the infinitesimal mean jump of component ξ i ( t )computed as E ( ξ i ( t + dt ) − ξ i ( t ) | ξ ( t ) = ( x , x )) = ( λ i + ( α i − g i ( x i ⋆ )) x i ) dt + o ( dt ) , i = 1 , , is negative and bounded away from zero in domain { x i ≥ , x i ⋆ ≥ C i ⋆ } , i = 1 ,
2, whereboth C and C are large enough. Now Lemma 2 yields that in a finite mean time theMarkov chain hits the boundary. Remark 1.
Note that in the case of the competition process with Lotka-Volterra interac-tion (mentioned in the introduction), the lemma follows from [12, Theorem 5].Let us finish the proof of the theorem. Let T j be the duration of j -th visit to set D N = { x > N, x ≤ } ∪ { x ≤ , x > N } , where N is chosen in the proof of Lemma 1.This lemma yields that P ( T j < ∞ ) ≤ − ε on { T j − < ∞} . Consequently, with probabilityone, T j < ∞ only for finitely many j , and the process eventually confines to set D N .Finally, suppose for definiteness that the absorbing set is { x > N, x ≤ } . Since thedrift of ξ ( t ) at x = 1 is directed down, the process eventually jumps from level x = 1 tolevel x = 0. On the other hand, the process cannot stay forever at axis x = 0 as λ > x = 0 and x = 1 asclaimed. Theorem 1 is proved. We start with the following lemma which is similar to Lemma 1.
Lemma 4.
There exists ε > , depending on the model parameters only, such that inf x ≥ P (cid:16) ˜ A | ξ (0) = ( x , (cid:17) ≥ ε and inf x ≥ P (cid:16) ˜ A | ξ (0) = (0 , x ) (cid:17) ≥ ε, where ˜ A i = ( ξ i ( t ) → ∞ and ξ i ⋆ ( t ) ∈ { , } , ∀ t ≥ , if α i > ,ξ i ( t ) → ∞ and ξ i ⋆ ( t ) ∈ { , , } , ∀ t ≥ , if α i = 0 . roof. Denote x = x and y = x for simplicity of notations. We prove the lemma only inthe case ξ (0) = ( x, α >
0. Consider function f defined in (4) with parameters µ and ν such that 0 < ν < µ < . Let G be the generator of the competition process with linear interaction. Given x > x,
0) are possible only to states ( x +1 ,
0) and ( x, λ + α x and λ respectively. Using equation (7) we obtain that G f ( x,
0) = ( λ + α x ) (cid:0) ( x + 1) − ν − x − ν − ( x + 1) − µ + x − µ (cid:1) + λ x − µ = − να x − ν + ( µα + λ ) x − µ + o (cid:0) x − ν (cid:1) + o (cid:0) x − µ (cid:1) ≤ , (10)for sufficiently large x >
0, as ν < µ .Now, given that x >
0, the transitions from state ( x,
1) to states ( x + 1 , x − , x,
2) and ( x,
0) occur with rates λ + α x , β , λ + α and β x respectively. Therefore,using equation (7) one more time we obtain that G f ( x,
1) = ( λ + α x ) (cid:0) ( x + 1) − ν − x − ν (cid:1) + β (cid:0) ( x − − ν − x − ν (cid:1) + ( λ + α ) (cid:0) − x − ν (cid:1) − β x − µ ≤ − να x − ν + λ + α − β x − µ + o ( x − ν ) ≤ , (11)for all sufficiently large x , as µ < N > σ = inf( t : ξ ( t ) / ∈ { x > N, y = 0 , } ) . Assume that N is so large that the bounds (10)) and (11) hold for x > N . Then Z ( t ) = f ( ξ ( t ∧ σ )) is a non-negative supermartingale. The proof can be finished by using theargument based on the optional stopping theorem, in a manner similar to the proof ofLemma 1.Assume now that α = 0. In this case, instead of function (4) we consider the followingfunction g ( x, y ) = x − x − λ /λ x ln x + x ln x , if y = 0; x − x , if y = 1; x , if y = 2;1 , if y ≥ . G g ( x, ≤ − λ x ln x + O (cid:18) x ln x (cid:19) ≤ , G g ( x, ≤ − β λ /λ ln x + O (cid:18) x (cid:19) ≤ , G g ( x, ≤ − β x ln x + O (1) ≤ , (12)for all sufficiently large x . The rest of the proof is analogous to the proof in case α > Lemma 5.
Define τ = inf { t : ξ ( t ) = 0 or ξ ( t ) = 0 } . Then τ is a.s. finite. Lemma 5 is proved in Section 2.3. Similarly to the proof of Theorem 1, it follows fromLemma 4 and Lemma 5 that, with probability 1, the process eventually confines either toset { x ≤ κ } , or to set { x ≤ κ } , where κ i , i = 1 , { x ≤ κ } and consider the following two cases.First, suppose that α >
0, so that κ = 1. In this case the process cannot stay forever atline x = 0. Indeed, let ( a j , , j ≥ x = 0. The probability of jump ( a j , → ( a j ,
1) can be boundedbelow by O (1) / ( a + j ) (for instance, consider the worst case scenario, when the processalways jumps to the right); therefore, by the conditional Borel-Cantelli lemma, there areinfinitely many jumps from line x = 0 to line x = 1. Combining this with Lemma 5, or,simply noting that the probability of a jump from line x = 1 to line x = 0 is boundedbelow (it tends to β / ( α + β + β ) as x → ∞ ) one can conclude that the process cannotstay forever at line x = 1 as well; hence, it goes to infinity oscillating between lines x = 0and x = 1, as claimed.Finally, suppose that α = 0 in which case κ = 2. The probability of transition( x , → ( x ,
1) is equal to λ / ( λ + λ ) for all x , so that the Markov chain cannot foreverstay at x = 0. Similarly to the above, let ( a j , , j ≥ x = 1. The probability of jump ( a j , → ( a j ,
2) canbe bounded below by O (1) / ( a + j ). Again, by the conditional Borel-Cantelli lemma, thereare infinitely many jumps from line x = 1 to line x = 2. Combining this with Lemma 5,or, simply noting that probabilities of jumps both from line x = 2 to line x = 1, and fromline x = 1 to line x = 0, are bounded below by constants, we obtain that the processgoes to infinity oscillating between lines x = 0 and x = 2, as described. Note that each of the following lines x = α x + λ β (line l ) and x = β x − λ α (line l )divides Z into two parts. The infinitesimal drift of ξ ( t ) is negative above the line l ,9nd positive below it. Similarly, the infinitesimal drift of ξ ( t ) is negative below line l and positive above it. There are two cases of mutual location of lines l and l , namely, α α < β β and α α > β β .If α α < β β , then line l is located above line l in the positive quarter plane. Bothprocess components have negative drift in the domain between the lines. Moreover, the driftof one of the process components remains negative outside the negative cone. Consequently,with probability 1, the process eventually hits the axes. The formal proof is similar to thecase of competition processes with non-linear interaction in Theorem 1, therefore we skipthe details. In addition, finiteness of the hitting time in this case follows from results in [12](see Section 3).The case α α ≥ β β is different from the previously considered cases. In order toexplain this, assume for a moment that α α > β β . Then there is a positive drift in both coordinates in the domain between lines l and l . If the process starts outside the domain,where the drift of the smallest component is strictly negative, then this component becomeszero in a finite mean time by the same reasoning as in all previous cases. However, if theinitial position of the process is inside the domain, then one has to show that the processeventually leaves the domain.The proof of the lemma in this case is given in Section 2.3.2. The proof is based on anappropriately modified method used in [3] for analysis of Friedman’s urn model. The mainidea of the original method is explained in Section 2.3.1. In this section we explain the main idea of Freedman’s method for Friedman’s urn model.First, recall that Friedman’s urn model with parameters α ≥ β ≥ W n , B n ) ∈ R \ (0 ,
0) evolving as follows. Given ( W n , B n ) = ( W, B ) the Markovchain jumps to ( W + α, B + β ) with probability W/ ( W + B ), and to ( W + β, B + α ) withprobability B/ ( W + B ). In order to demonstrate the main idea of the method we are goingto consider another Markov chain (the auxiliary process) instead. The auxiliary processis a DTMC ( X n , Y n ) ∈ Z \ (0 ,
0) evolving as follows. Given ( X n , Y n ) = ( x, y ) it jumpsto states ( x + 1 , y ) and ( x, y + 1) with probabilities αx + βy ( α + β )( x + y ) and αy + βx ( α + β )( x + y ) respectively.Similar to competition processes with linear interaction, DTMC ( X n , Y n ) takes values inthe integer quarter plane and jumps to the nearest neighbour states. There is also a certainsimilarity between transition probabilities of ( X n , Y n ) and the competition processes withlinear interaction, although the interaction between X n and Y n can now be regarded ascooperative rather than competitive. Furthermore, the auxiliary process and Friedman’surn model are closely related, since W n = αX n + βY n B n = βX n + αY n .
10n other words, X n ( Y n resp.) can be viewed as the number of times a white (black resp.)colour has been picked up in Friedman’s urn model by time n . Without loss of generality,we apply the Freedman’s method to the auxiliary process ( X n , Y n ). Given α ≥ β ≥ ρ = α − βα + β . (13)Theorem 3 below describes the asymptotic behaviour of the auxiliary process under certainassumptions. The theorem is almost a verbatim copy of a part of Theorem 3.1 in [3] for theoriginal Friedman’s urn model with parameters α and β . We state and prove the theoremfor the auxiliary process for the following reason. There is certain similarity between ourcompetition process and the auxiliary process, which allows to adapt the idea of the proof ofTheorem 3 for our purposes, therefore we provide the proof here for the readers convenience. Theorem 3. If ρ > / then n − ρ ( X n − Y n ) converges almost surely to a non-trivial randomvariable.Proof. Define the difference between the components X n and Y n as U n = X n − Y n , andtheir total amount as S n = X n + Y n ; note that S n = S + n . We have E ( U n +1 | U n ) = U n (cid:18) α − β ( α + β ) S n (cid:19) = U n (cid:18) α − βs + ( α + β ) n (cid:19) E (cid:0) U n +1 | U n (cid:1) = U n (cid:18) α − β )( α + β ) S n (cid:19) + 1 = U n (cid:18) α − β ) s + ( α + β ) n (cid:19) + 1 , (14)where s = ( α + β ) S . Denote a n ( j ) = (cid:18) α − β ) js + ( α + β ) n (cid:19) , j = 1 , . In these notations we get that E ( U n +1 | U n ) = U n a n (1) , E (cid:0) U n +1 | U n (cid:1) = U n a n (2) + 1 . (15)The first equation in the preceding display means that Z n := U n n − Y k =0 a − k (1) , n ≥ , (16)is a martingale. The second equation gives E (cid:0) U n +1 (cid:1) = E (cid:0) U n (cid:1) a n (2) + 1 . Using this identity recursively, we arrive at the following equation E (cid:0) U n +1 (cid:1) = U + n X j =0 j Y k =0 a − k (2) ! n Y k =0 a k (2) . m Y k =0 a k ( j ) = ( C j + o (1)) m jρ , j = 1 , , for some C , C >
0, so that ∞ X j =0 j Y k =0 a − k (2) < ∞ , as ρ > /
2. Consequently, sup n n − ρ E ( U n ) < ∞ . Now Doob’s convergence theorem impliesthat martingale Z n defined in (16) converges almost surely to a finite limit as n → ∞ .Theorem 3 is thus proved. α α > β β Proof in the symmetric case.
We start with the symmetric case λ = λ = λ , α = α = α and β = β = β , where α > β , in order to provide an intuition for the way how the proofworks. Denote by ζ ( n ) = ( ζ ( n ) , ζ ( n )) ∈ Z , n ∈ Z + , the DTMC corresponding toCTMC ξ ( t ). Let {F n } ∞ n =1 be the standard natural filtration associated with the Markovchain ζ ( n ). Define S n = ζ ( n ) + ζ ( n ) , U n = ζ ( n ) − ζ ( n ) , τ = min { m : ζ ( m ) = 0 or ζ ( m ) = 0 } . (17)Assume that P ( τ = ∞ ) > E (cid:0) U n +1 |F n (cid:1) = U n (cid:18) α + β )2 λ + ( α + β ) S n (cid:19) + 1 on the event { τ > n } . (18) Remark 2.
This expression is quite similar to the second equation in (14); the fundamentaldifference is that the sum of the components, i.e. S n , is now a random process. This is incontrast to both the auxiliary process and Friedman’s urn model (as well as to other urnmodels without ball removals), where the sum of the components is a deterministic, usuallylinear, function of n . The main idea of what follows below is that the long-term behaviourof S n can be effectively controlled due to its simple asymptotic behaviour.Trivially, S n +1 − S n = ± P ( S n +1 = S n + 1 | S n ) = λ + αS n λ + ( α + β ) S n , P ( S n +1 = S n − | S n ) = βS n λ + ( α + β ) S n . The preceding display shows that the long term behaviour of S n is similar to a homogeneoussimple random walk that jumps right and left with probabilities αα + β and βα + β respectively.Therefore, the strong law of large numbers, with some variations , implies that for any ε, δ > N such that P ( S n ∈ [( ρ − δ ) n, ( ρ + δ ) n ] , ∀ n ≥ N ) ≥ − ε, (19) A rigorous proof can be found further in Lemma 6. ρ is defined in (13).Further, fix some δ > ρ + δ < ε >
0; according to (19)there exists an N = N ( ε ) so large that σ N = min ( n > N : S n / ∈ [( ρ − δ ) n, ( ρ + δ ) n ]) . satisfies P ( σ N = ∞ ) ≥ − ε. (20)It follows from equation (18) and the definition of σ N that E (cid:0) U n +1 |F n (cid:1) ≥ U n b n on { N ≤ n < min ( σ N , τ ) } , (21)where b n = 1 + 2( α + β )2 λ + ( α + β )( ρ + δ ) n = 1 + 2 n − ρ + δ + O (cid:0) n − (cid:1) . Iterating (21) gives that E (cid:0) U n +1 |F N (cid:1) ≥ b n b n − . . . b N +1 b N U N on { N ≤ n < min ( σ N , τ ) } . Assume n ≥ N everywhere below. Then n Y k = N b k = n Y k = N e k − ρ + δ + O ( k ) = e P nk = N h k − ρ + δ + O ( k ) i ≥ C · n ρ + δ for some C = C ( N ) >
0, so that E (cid:0) U n +1 |F N (cid:1) ≥ C · n ρ + δ { n< min( σ,τ ) } . Dividing both sides by n and taking the expectation gives E (cid:18) U n +1 n (cid:19) ≥ C n ρ + δ − P ( n < min( σ N , τ )) . The left hand side of the preceding display is uniformly bounded in n , as | U n | ≤ | U | + n for all n ≥
0. On the other hand, bound ρ + δ < n ρ + δ − → ∞ as n → ∞ .Therefore, if lim n →∞ P ( n < min( σ N , τ )) = P ( σ N = ∞ , τ = ∞ ) >
0, as asserted, then weget a contradiction. Consequently, P ( σ N = ∞ , τ = ∞ ) = 0 and P ( τ = ∞ ) ≤ P ( σ N = ∞ , τ = ∞ ) + P ( σ N < ∞ ) ≤ ε by (20). Since ε > P ( τ = ∞ ) = 0.In turn, this means that the process hits the axes in a finite time a.s., as claimed.13e are now going to extend the above argument on the general case. Without loss ofgenerality assume from now on that α ≥ α . (22)Let us find the asymptotic equilibrium direction for ζ ( n ), which will be shown to beunstable later in the proof. Indeed, if we assume that both λ = 0 and λ = 0 (theycontribute very little to the birth rates when x and y are large) then the slope of the driftof the vector field corresponding to our system is given by α y − β xα x − β y . It coincides with the slope of the vector ( x, y ) if and only if x = ry where r solves1 r = α − rβ rα − β ⇐⇒ β r + ( α − α ) r − β = 0 , (23)Since x, y ≥
0, we choose r to be the positive root of (23) which can be written as r = − ( α − α ) + D β , where D = p ( α − α ) + 4 β β = p ( α + α ) + 4( β β − α α ) . Note that equation (23) can be rewritten as follows r = β + rα α + rβ . (24)Define the following variables R ( x, y ) = ( α + β ) x + ( α + β ) y + λ + λ ,R n = R ( ζ ( n )) , (25)and U ( x, y ) = x − ry − d,U n = U ( ζ ( n )) , (26)where d = − λ − λ r ) + α + β r α + β r ) (27)Assume that x, y >
0. Then E ( U n +1 | ζ ( n ) = ( x, y )) = ( U n + 1) λ + α xR n + ( U n − β yR n + ( U n − r ) λ + α yR n + ( U n + r ) β xR n = U n + 2( α + rβ ) U n x − β + rα α + rβ y − dR n + 2 U n Q + Q ( x, y ) R n = U n (cid:20) α + rβ ) R n (cid:21) + 2 U n Q + Q ( x, y ) R n . (28)14here we used equation (24) to rewrite the second term in the third line of the precedingdisplay and used the notations Q := d ( α + β r ) + λ − rλ and Q ( x, y ) := ( r β + α ) x + ( β + r α ) y. Consequently,2 U n Q + Q ( x, y ) = 2( α + β r ) (cid:18) d + 2( λ − λ r ) + α + β r α + β r ) (cid:19) x + (cid:0) β + α r − rd ( α + β r ) − r ( λ − λ r ) (cid:1) y + Q , where Q = λ + λ r − d ( λ − λ r ) − d ( α + β r ). Note that the coefficient in front of x on the right hand side of the preceding equation is equal to 0 by the definition of d in (27).Further, using again the definition of d , we simplify the coefficient in front of y and arriveat the following equation2 U n Q + Q ( x, y ) = ( β + α r + α r + β r ) y + Q . Note that β + α r + α r + β r >
0, therefore,2 U n Q + Q ( x, y ) > , (29)for all y ≥ y , where y is a value depending on the model parameters.Equations (28) and (29) imply that E ( U n +1 | ζ ( n ) = ( x, y )) ≥ U n (cid:18) α + rβ ) R n (cid:19) , if x > y ≥ y . (30)Our next goal is to obtain an upper bound for the total transition rate R n . Define S ( x, y ) = ( α α + β β + 2 α β ) x + ( α α + β β + 2 α β ) y,S n = S ( ζ ( n )) , (31) T ( x, y ) = β x + ( rβ + α − α ) y,T n = T ( ζ ( n ))and ˜ ρ = α α − β β = ( α − rβ )( α + rβ ) > . (32) Remark 3.
Note that U n , defined by (26), functions as a measure of departure fromthe equilibrium; R n is the common denominator, S n is the (almost) constant drift term(see (34)), while T n is some sort of a remainder, up to a multiplying coefficient, as it willbe clear later in the proof. 15ow we want to write R ( x, y ) defined in (25) as a linear combination of S ( x, y ), T ( x, y ),and an extra constant. In order to find the unknown coefficients, observe that both S and T are linear in x and y with S (0 ,
0) = T (0 ,
0) = 0. Therefore, R ( x, y ) = λ + λ + k S ( x, y ) + l T ( x, y ) where k and l can be found by solving the elementary system of linear equations ( ∂R ( x,y ) ∂x = k ∂S ( x,y ) ∂x + l ∂T ( x,y ) ∂x∂R ( x,y ) ∂y = k ∂S ( x,y ) ∂y + l ∂T ( x,y ) ∂y , yielding k = α + rβ ˜ ρ > , l = − ( α α + 2 α β + β β ) r + α α + 2 α β + β β ˜ ρ < . Hence, R n = ( λ + λ ) + α + rβ ˜ ρ S n + lT n . (33)The next statement is probably known, but just in case we present its proof here aswell. Lemma 6.
Suppose that we are given a process Z n adapted to the filtration F n such that | Z n +1 − Z n | ≤ B for all n and a ≤ E ( Z n +1 − Z n |F n ) ≤ a + σZ n for some constants B > , a > and σ ≥ . Then Z n /n → a a.s.Proof. Fix an ε > Z n = Z n − an . Then ˆ Z n is a submartingale with jumps boundedby B + a , and hence by Azuma-Hoeffding inequality P ( ˆ Z n − ˆ Z ≤ − εn ) ≤ exp (cid:26) − ε n B + a ) (cid:27) and by Borel-Cantelli lemma the event { ˆ Z n /n ≤ − ε + ˆ S /n } occurs finitely often. Since ε > Z /n → n →∞ ˆ Z n /n ≥ n →∞ Z n /n ≥ a .Next, define ¯ Z n = ˆ Z n − n X i =1 σ max { , Z n } . On the event { Z n ≥ } we have E ( ¯ Z n +1 − ¯ Z n |F n ) = 0. Fix a large N and consider ¯ Z n ∧ τ N where τ N = inf { n ≥ N : Z n < } . Then ¯ Z n ∧ τ N is a martingale for n ≥ N with jumpsbounded by B + a + 1, and applying Azuma-Hoeffding inequality again we get P ( (cid:12)(cid:12) ¯ Z n ∧ τ N − ¯ Z N (cid:12)(cid:12) ≥ εn ) ≤ (cid:26) − ε ( n − N )2( B + a + 1) (cid:27) ε >
0. By an argument similar to the first part of the proof, this implies thatlim n →∞ ¯ Z n ∧ τ N /n = 0 a.s. However, the first part of the proof implies that τ N = ∞ for all butfinitely many N ’s a.s. Hence lim n →∞ ¯ Z n /n = 0 a.s. Now, the fact that lim inf n →∞ Z n /n ≥ a gives us that P ni =1 σ max { ,Z n } ≤ O (log n ) so that ¯ Z n − ˆ Z n = o ( n ) thus implying the statementof the lemma. Proposition 1.
Consider S n and ˜ ρ defined in (31) and (32) respectively. Then lim n →∞ S n n =˜ ρ a.s.Proof of Proposition 1. Note that the jumps of S n are bounded (they can take at most fourdistinct values). The expected drift of S n is given by E ( S n +1 − S n | ζ ( n ) = ( x, y )) = ( α α + β β )(( α − β ) x + ( α − β ) y ) R n + 2 α β ( α x − β y ) + α β ( α y − β x ) R n + ( λ + λ )( α α + β β ) + 2 λ α β + 2 λ α β R n An easy algebraic computation gives that the sum of terms with x in the first and thesecond numerators on the right hand side of the preceding display is equal to ˜ ρ ( α + β ) x .Similarly, the sum of all terms with y in the same numerators is equal to ˜ ρ ( α + β ) y .Rearranging all terms with λ and λ in the last numerator of the same display gives thefollowing quantity ˜ ρ ( λ + λ ) + 2 λ β ( α + β ) + 2 λ β ( α + β ) . Thus, we obtain that E ( S n +1 − S n | ζ ( n ) = ( x, y )) = ˜ ρ + 2 λ β ( α + β ) + 2 λ β ( α + β ) R n ≥ ˜ ρ > . (34)Note that R ( x, y ) ≥ ( x + y ) min { β , β } and S ( x, y ) ≤ ( x + y ) max { α α + β β + 2 α β , α α + β β + 2 α β } and since β , β > R ( x, y ) ≥ C S ( x, y ) for some positive constant C , so that R n ≥ C S n . Now the result follows from Lemma 6 with a = ˜ ρ . Corollary 1.
Let κ = lim inf n →∞ T n n . Then P ( κ >
0) = 1 .Proof.
Similarly to the preceding proof, T ( x, y ) ≥ ( x + y ) min( β , rβ + α − α ) ≥ ( x + y ) β min(1 , r )since α − α ≥
0, and thus T n ≥ C S n for some C >
0. Hence, by Proposition 1,lim inf n →∞ T n n ≥ C lim inf n →∞ S n n = C ˜ ρ > . roposition 2. For every δ > and ε > there exists N such that P (cid:18) R n ≤ α + rβ δ n, ∀ n ≥ N (cid:19) ≥ − ε. Proof of Proposition 2.
Using equation (33), Proposition 1 and Corollary 1 we obtain thatfor sufficiently small δ >
0, sufficiently large n and any fixed εR n = ( λ + λ ) + ( α + rβ ) S n ˜ ρ − lT n ≤ ( λ + λ ) + ( α + rβ )(1 + δ ) n + l κ n, with probability at least 1 − ε . Recall that κ > l <
0. Let δ > λ + λ ) + ( α + rβ )(1 + δ ) n + l κ n ≤ ( α + rβ )(1 − δ ) n ≤ ( α + rβ ) n δ . (35)Thus, we obtain that, with probability at least 1 − ε , R n ≤ α + rβ δ n, for all sufficiently large n , as claimed.The rest of the proof is similar to the symmetric case, and we are going to explainbriefly some minor modifications required. First, define τ = min { n : ζ ( n ) = 0 or ζ ( n ) < y } , (36)where y is such that thr bound (30) holds. Then, assume that P ( τ = ∞ ) = 0 and arriveat a contradiction. To this end, fix δ > N > η N = min (cid:26) n ≥ N : R n > α + rβ δ n (cid:27) . Assume that N is sufficiently large, so that probability P ( η N = ∞ ) is sufficiently close to 1to ensure that P ( η N = ∞ , τ = ∞ ) >
0. Then Proposition 2 implies that E (cid:0) U n +1 |F n (cid:1) ≥ U n a n on { n < min ( η N , τ ) } , where a n = 1 + 2(1 + δ ) n . Similarly to the symmetric case, it can be shown by using the inequality in the precedingdisplay that P ( η N = ∞ , τ = ∞ ) = 0. This contradicts the assumption that P ( τ = ∞ ) > ζ ( τ ) > < ζ ( τ ) = y −
1. In this case, observe thatthe probability of hitting the horizontal axis { ( x, , x ∈ Z + } is bounded below uniformlyover starting location ( x, y − x ≥
1. Indeed, P ( ζ ( τ + y −
1) = ( x, | ζ ( τ ) = ( x, y − y − Y k =1 β xλ + λ + ( α + β ) x + ( α + β )( y − k ) ≥ y − Y k =0 β λ + λ + α + β + ( α + β )( y − k ) = Const( λ , λ , α , α , β , β , y ) > x ≥
1. Consequently, with probability one, the process eventually hits the boundary.18 .3.3 Proof of Lemma 5 in case α α = β β The proof will be very similar to the case α α > β β , so we provide only its sketch. Let S ( x, y ), R ( x, y ), S n , R n and ˜ ρ be the same as in the previous section. Note that ˜ ρ = 0 inthis case, so we need to find a replacement for Lemma 6.Observe that due to the fact that α α = β β we have α > α > β β > S ( x, y ) = 2 α ( α + β ) x + 2 α ( α + β ) y, R ( x, y ) = ( α + β ) x + ( α + β ) y + λ + λ so that R ( x, y ) ≥ S ( x,y )max { α , α } . Then (34) becomes E ( S n +1 − S n |F n ) = 2 λ β ( α + β ) + 2 λ β ( α + β ) R n ∈ (cid:20) , C S n (cid:21) for some C ≥
0. Therefore, S n can be majorized by a Lamperti random walk (see [9]) andhence by Theorem 3.2.7 in [9] we get thatlim sup n →∞ log S n log n ≤ / P (cid:0) R n ≤ n / δ , ∀ n ≥ N (cid:1) ≥ − ε and by setting δ = 1 /
6, on the event R n ≤ n / the RHS of (30) becomes U n (cid:18) α + rβ ) n / (cid:19) leading to contradiction similarly to the case α α > β β . In this section we recall the definition of the competition process from [12] and brieflyanalyse the applicability of some theorems from that paper to competition processes inours.Recall that the competition process in [12] is defined as a CTMC X ( t ) = ( X ( t ) , x ( t )) ∈ Z that evolves as follows. Given the state ( x , x ) ∈ Z , the CTMC jumps to( x + 1 , x ) with rate a ( x , x ) , ( x , x + 1) with rate b ( x , x ) , ( x − , x ) with rate c ( x , x ) if x > , ( x , x −
1) with rate d ( x , x ) if x > , ( x − , x + 1) with rate e ( x , x ) if x > , ( x + 1 , x −
1) with rate f ( x , x ) if x > , (37)19here a ( x , x ) , . . . , f ( x , x ) ≥
0. Following [12], let us assume that the Markov chain isregular in a sense that there exists exactly one associated transition matrix. For simplicity,we assume in addition that Markov chain X ( t ) is irreducible, although in general theremight be absorption states.Define the following quantities r k = max x ,x > x + x = k [ a ( x , x ) + b ( x , x )] , (38) s k = min x ,x > x + x = k [ c ( x , x ) + d ( x , x )] ,τ = inf( t ≥ X ( t ) = 0 or X ( t ) = 0) . It follows from Theorem 2 in [12] that A := ∞ X k =2 s . . . s k r . . . r k = ∞ , (39)is a sufficient condition for hitting time τ to be finite almost surely.Consider, for simplicity, the competition process with linear interaction (with transitionrates of type 2 defined in (2)) in the symmetric case, that is α i = α, β i = β, λ i = λ, i = 1 , r k = 2 λ + αk and s k = βk, and it is easy to see that if α ≤ β then s . . . s k r . . . r k ≥ ( C (cid:0) βα (cid:1) k , if α < β, C k λ/α , if α = β, for some C , C > k . Consequently, if α < β or α = β < λ then A = ∞ ; hence τ is almost surely finite. However, if α > β or α = β ≥ λ , then the resultsof [12] are not applicable.Further, we are going to compare the long term behaviour of two simple competitionprocesses. One process of interest is the competition process X ( t ) given in Example 2in [12]. This process is specified by the following choice of transition rates in (37) a ( x , x ) ≡ a, b ( x , x ) ≡ b, c ( x , x ) = γx ,d ( x , x ) = δx , e ( x , x ) = εx x , f ( x , x ) ≡ , (40)where a, b, γ, δ, ε >
0. The other process is a special case of the competition processwith linear interaction which transition rates are specified by parameters α = α = 0, β = δ, β = γ, λ = a, λ = b >
0. In the introduction we interpreted such competitionprocess as the OK Corral model with “resurrection”.Interactions in these processes are different. However, their behaviours inside thequarter plane are quite similar. Indeed, the mean drift of each of these processes in-side the domain are directed towards the axes. Further, quantities r k = a + b , k ≥
1, and20 k = k min { γ, δ } , k ≥
1, are the same for both processes. Now either [12, Theorem 2], orthe argument based on [9, Lemma 7.3.6] (similar to Lemma 3) imply that τ < ∞ a.s. inboth cases.At the same time, these processes evolve differently, because of the difference in thetransition rates on the boundary. The process with rates given by (40) has a strong meandrift towards the origin, while an OK Corral type process jumps away with constant rate,as its death rates on the boundary are zero. This seemingly small change results in quitesubstantial difference in the long term behaviour of the processes. Indeed, define ˜ r k and ˜ s k by the same formula as r k and s k in (38) by taking the maximum (minimum resp.) overthe set x , x ≥
0, that is, now we include the boundary states ( k,
0) and (0 , k ). Theorem 4in [12] states that ˜ A = ∞ X k =1 ˜ r . . . ˜ r k − ˜ s . . . ˜ s k < ∞ is a sufficient condition for the competition process with transition rates (37) to be positiverecurrent, implying that the process governed by (40) is positive recurrent. Indeed, ˜ r k = r k = a + b > s k = s k = k min { γ, δ } , k ≥
1, so,˜ A = 1 a + b ∞ X k =1 k ! (cid:18) a + b min { γ, δ } (cid:19) k < ∞ . (Note also that positive recurrence of this process follows from the Foster criterion forpositive recurrence with Lyapunov function f ( x , x ) = x + x , but we skip further details).At the same time Theorem 4 from [12] is not applicable to the OK Corall model with“resurrection”, as ˜ s k = 0, while our Theorem 2 shows that this process is transient andescapes to infinity in the only possible way, i.e. along the boundary, as described.
Acknowledgement
SV research is partially supported by the Swedish Research Council grant VR2014-5147.We thank Mikhail Menshikov and Svante Janson for helpful discussions.
References [1] Anderson, W. (1991). Continuous time Markov chains: an application oriented ap-proach. Springer Verlag.[2] Bingham, N. H., Goldie, C. M., and Teugels, J.L. (1987). Regular Variation. CambridgeUniversity Press.[3] Freedman, David A. (1965). Bernard Friedman’s urn.
Ann. Math. Statist. , pp. 956–970. 214] Iglehart, D.L. (1964). Reversible Competition Processes. Z. Wahrseheinliehkeitsthe-orie , pp. 314–331.[5] Iglehart, D.L. (1964). Multivariate competition processes. Ann. Math. Statist. ,pp. 350-361.[6] Janson, S., Shcherbakov, V. and Volkov, S. (2019). Long term behaviour of a reversiblesystem of interacting random walks. Journal of Statistical Physics , N1, pp. 71–96.[7] Kingman, J. F. C., and Volkov, S. E. (2003). Solution to the OK Corral model viadecoupling of Friedman’s urn.
J. Theoret. Probab. , pp. 267–276.[8] MacPhee, M.I., Menshikov, M.V., and Wade, A.R. (2010). Angular asymptotics formulti-dimensional non-homogeneous random walks with asymptotically zero drift. Markov Process. Related Fields , Issue 2, pp. 351–388.[9] Menshikov, M.V., Popov, S. and Wade, A.R. (2017). Non-homogeneous Random Walks:Lyapunov Function Methods for Near-Critical Stochastic Systems. Cambridge Univer-sity Press.[10] Menshikov, M. and Shcherbakov, V. (2018). Long term behaviour of two interactingbirth-and-death processes. Markov Process. Related Fields , Issue 1, pp. 85–106.[11] Pemantle, R.(2007). A survey of random processes with reinforcement. ProbabilitySurveys , pp. 1–79.[12] Reuter, G. E. H. (1961). Competition processes. In: Neyman J. (Ed.) Proceedingsof The Fourth Berkeley Symposium on Mathematical Statistics and Probability, v.II:Contributions to Probability Theory. University of California Press, Berkeley.[13] Shcherbakov, V. and Volkov, S. (2015). Long term behaviour of locally interactingbirth-and-death processes. Journal of Statistical Physics158