Non-Markovian state dependent networks in critical loading
aa r X i v : . [ m a t h . P R ] D ec Non-Markovian state dependent networks in criticalloading
Chihoon LeeDepartment of StatisticsColorado State UniversityAnatolii A. PuhalskiiUniversity of Colorado Denver andInstitute for Problems in Information TransmissionJune 23, 2018
Abstract
We establish heavy traffic limit theorems for queue-length processes in critically loadedsingle class queueing networks with state dependent arrival and service rates. A distin-guishing feature of our model is non-Markovian state dependence. The limit stochasticprocess is a continuous-path reflected process on the nonnegative orthant. We give anapplication to generalised Jackson networks with state-dependent rates.
Keywords: State dependent networks, non-Markovian networks, diffusion approximation,weak convergenceAMS Subject Classifications:
Primary 60F17; secondary 60K25, 60K30, 90B15
Queueing systems with arrival and (or) service rates depending on the system’s state arisein various application areas which include, among others, manufacturing, storage, serviceengineering, and communication and computer networks. Longer queues may lead to cus-tomers being discouraged to join the queue, or to faster processing, e.g., when human serversare involved, state-dependent features are present in congestion control protocols in com-munication networks, such as TCP (cf. [1, 2, 9, 11, 16] and references therein for moredetail). 1n this paper, we consider an open network of single server queues where the arrivaland service rates depend on the queue lengths. More specifically, the network comprises K single-server stations indexed 1 through K . Each station has an infinite capacity buffer andthe jobs are processed according to the first-in-first-out discipline. The arrivals of jobs atthe stations occur both externally, from the outside and internally, from the other stations.Upon service completion at a station, a customer is either routed to another station or exitsthe network. Any customer entering the network eventually leaves it. A distinguishingfeature of the model is non-Markovian state dependence: the arrival and service processesare assumed to be time-changed “primitive” processes, where the time change is given bythe integral of the queue-length-dependent rate.Our goal is to obtain limit theorems in critical loading for the queue length processes akinto standard diffusion approximation results available for generalised Jackson networks, seeReiman [15]. To this end, we consider a sequence of networks with similar structure indexedby n ∈ IN . The limits are taken as n → ∞ . The critical loading condition is assumedto hold in the limit. We show that if the network primitives satisfy limit theorems withcontinuous-path limits, then the multidimensional queue-length processes, when suitablyscaled and normalised, converge to a reflected continuous-path process on the nonnegativeorthant. If the limits of the primitives are diffusion processes, the limit stochastic processis a reflected diffusion with a state dependent drift and diffusion. We give an applicationto generalised Jackson networks with state-dependent rates thus providing an extension ofReiman’s [15] results. In addition, we establish the existence and uniqueness of the solutionto the equations governing the network.The results on the heavy traffic asymptotics in critical loading for state-dependent ratesavailable in the literature are mostly confined to the case of diffusion limits for Markovianmodels (which assume state dependent Poisson arrival processes and exponential servicetimes), see Chapter 8 of Kushner [9], Mandelbaum and Pats [11], and Yamada [17]. AMarkovian closed network with state dependent rates has been considered in Krichagina [8].Some non-Markovian models (with Poissonish arrivals) have been treated in Section 7 ofYamada [17].A different class of results on diffusion approximation concerns queueing systems modeledon the many-server queue with a large number of servers. In such a system the service ratedecreases to zero gradually with the number in the system (whereas in the model consideredhere it has a jump at zero, see (2.1d)), so the limit process is an unconstrained diffusion,see, Massey, Mandelbaum, and Reiman [10], Pang, Talreja, and Whitt [12], and referencestherein. We do not consider those set-ups in this paper.The exposition is organised as follows. In the next section, we provide a precise descrip-tion of the model and introduce the hypotheses required for our main result. The statementof the limit theorem for the queue-length process and its proof are contained in Section 3.In Section 4 an application to state-dependent generalised Jackson networks is presented.The appendix contains a proof of the pathwise queue-length construction underlying thedefinition of the model. 2ome notational conventions are in order. We use ⇒ to represent convergence in dis-tribution of random elements with values in an appropriate metric space, all vectors areunderstood as column vectors, | x | denotes the Euclidean length of a vector x , its compo-nents are denoted by x i , unless mentioned otherwise, superscript T is used to denote thetranspose, 1 A stands for the indicator function of an event A , δ ij represents Kronecker’sdelta, ⌊ a ⌋ denotes the integer part of a real number a . Let (Ω , F , P ) be a probability space where all random variables considered in this paper areassumed to be defined. For the n -th network and for i ∈ IK , where IK = { , , . . . , K } , let A ni ( t ) represent the cumulative number of customers that arrive at station i from outsidethe network during the time interval [0 , t ], and let D ni ( t ) represent the cumulative numberof customers that are served at station i for the first t units of busy time of that sta-tion. Let J ⊆ IK represent the set of stations with actual arrivals so that A ni ( t ) = 0 if i
6∈ J . We call A n = ( A ni , i ∈ IK ) and D n = ( D ni , i ∈ IK ), where A ni = ( A ni ( t ) , t ≥ D ni = ( D ni ( t ) , t ≥ n -th network, re-spectively. We associate with the stations of the network the processes Φ ni = (Φ nij , j ∈ IK ), i ∈ IK , where Φ nij = (Φ nij ( m ) , m = 1 , , . . . ), and Φ nij ( m ) denotes the cumulative num-ber of customers among the first m customers that depart station i which go directly tostation j . The process Φ n = (Φ nij , i, j ∈ IK ) is referred to as the routing process. We con-sider the processes A ni , D ni and Φ ni as random elements of the respective Skorohod spaces D ([0 , ∞ ) , IR ) , D ([0 , ∞ ) , IR ) and D ([0 , ∞ ) , IR K ); accordingly, A n , D n and Φ n are regardedas random elements of D ([0 , ∞ ) , IR K ) , D ([0 , ∞ ) , IR K ) and D ([0 , ∞ ) , IR K × K ), respectively.Throughout, S will be used to denote the K -dimensional non-negative orthant IR K + .We now introduce the equations which specify the network. Let λ ni , µ ni , where i ∈ IK , beBorel functions mapping S to IR + , with λ ni ( x ) = 0 if i
6∈ J , and let λ n = ( λ n , . . . , λ nK ) and µ n = ( µ n , . . . , µ nK ) . These functions have the meaning of state-dependent arrival and servicerates. Let N A,ni = ( N A,ni ( t ) , t ≥
0) and N D,ni = ( N D,ni ( t ) , t ≥
0) represent nondecreasing Z + -valued processes with trajectories in D ([0 , ∞ ) , IR ) and with N A,ni (0) = N D,ni (0) = 0. Wedefine N A,ni ( t ) = ⌊ t ⌋ if i
6∈ J . (The latter is but a convenient convention. Since λ ni ( x ) = 0if i
6∈ J , the process N A,ni is immaterial, as the equations below show.) The state of thenetwork at time t is represented by Q n ( t ) = ( Q n ( t ) , . . . , Q nK ( t )) , where Q ni ( t ) represents the3umber of customers at station i at time t . It is assumed to satisfy a.s. the equations: Q ni ( t ) = Q ni (0) + A ni ( t ) + B ni ( t ) − D ni ( t ) , (2.1a) A ni ( t ) = N A,ni (cid:18)Z t λ ni ( Q n ( s )) ds (cid:19) , (2.1b) B ni ( t ) = K X j =1 Φ nji (cid:0) D nj ( t ) (cid:1) , (2.1c) D ni ( t ) = N D,ni (cid:18)Z t µ ni ( Q n ( s ))1 { Q ni ( s ) > } ds (cid:19) , (2.1d)where t ≥ i ∈ IK . The random quantities in (2.1a)–(2.1d) have the following inter-pretation: Q ni (0) ∈ Z K + is the initial queue length at station i ; A ni ( t ) , B ni ( t ) , D ni ( t ) representthe cumulative number of exogenous arrivals at station i during the time interval [0 , t ], thecumulative number of endogenous arrivals at station i during the time interval [0 , t ], and thecumulative number of departures from station i during the time interval [0 , t ], respectively. Let P = ( p ij , i, j ∈ IK ) be a substochastic matrix, R = I − P T , and p i = ( p ij , j ∈ IK ) . Wedenote Q n (0) = Q n (0) √ n , N A,ni ( t ) = N A,ni ( nt ) − nt √ n ,N D,ni ( t ) = N D,ni ( nt ) − nt √ n , Φ ni ( t ) = Φ ni ( ⌊ nt ⌋ ) − p i nt √ n ,N A,ni =( N A,ni ( t ) , t ≥ , N A,n =( N A,ni , i ∈ IK ) ,N D,ni =( N D,ni ( t ) , t ≥ , N D,n =( N D,ni , i ∈ IK ) , Φ ni =(Φ ni ( t ) , t ≥ , Φ n =(Φ ni , i ∈ IK ) . We will need the following conditions.(A0) For each n ∈ IN and each i ∈ J , lim sup t →∞ N A,ni ( t ) /t < ∞ a.s.(A1) The spectral radius of matrix P is strictly less than 1.(A2) For each i ∈ IK , sup n ∈ IN sup x ∈ S (cid:18) λ ni ( nx ) n (1 + | x | ) + µ ni ( nx ) n (1 + | x | ) (cid:19) < ∞ . λ i ( x ) and µ i ( x ) such that λ ni ( nx ) n → λ i ( x ) , µ ni ( nx ) n → µ i ( x )uniformly on compact subsets of S , as n → ∞ . Furthermore, for x ∈ S , λ ( x ) − Rµ ( x ) = 0 . (A4) There exists a Lipschitz-continuous function a ( x ) such that1 √ n ( λ n ( √ nx ) − Rµ n ( √ nx )) → a ( x )as n → ∞ uniformly on compact subsets of S .(A5) As n → ∞ , ( Q n (0) , N A,n , N
D,n , Φ n ) ⇒ ( X , W A , W D , W Φ )where X is a random K -vector, W A , W D , and W Φ are continuous-path stochasticprocesses with trajectories in respective spaces D ([0 , ∞ ) , IR K ), D ([0 , ∞ ) , IR K ), and D ([0 , ∞ ) , IR K × K ) .Condition (A0) is needed to ensure the existence of a unique strong solution to the systemof equations (2.1a)–(2.1d), see Lemma 2.1. It is almost a consequence of condition (A5) inthat the latter implies that lim n →∞ N A,ni ( nt ) / ( nt ) = 1 in probability. Part (A1) is essentiallyan assumption that the network is open. It implies the existence of a regular Skorohod mapassociated with the network data (see Proposition 3.2) which is a key element of the proofof the main result. The requirement λ ( x ) = Rµ ( x ) in (A3) together with condition (A4)defines a critically loaded heavy traffic regime. Condition (A5) is the assumption on theprimitives. The components of W A corresponding to i
6∈ J vanish. Conditions (A2)–(A4)are fulfilled if the following expansions hold: λ n ( x ) = nλ ( x/n ) + √ nλ ( x/ √ n ) and µ n ( x ) = nµ ( x/n ) + √ nµ ( x/ √ n ), where λ , λ , µ , and µ are nonnegative bounded continuousfunctions such that λ ( x ) = Rµ ( x ) . If the above functions are constant, then one obtainsthe standard critical loading condition that ( λ n − Rµ n ) / √ n → λ − µ as n → ∞ , cf. Reiman[15].Most of the results on diffusion approximation in critical loading (see, e.g., Harrison andReiman [5], Kushner [9]) formulate the heavy traffic condition in terms of rates that are O (1) and then consider scaled processes where time is scaled up by a factor of n while spaceis scaled down by a factor of √ n . In the scaling considered here (cf. Mandelbaum and Pats[11], Yamada [17]) the time parameter is left unchanged and the factor of n is absorbed in thearrival and service rates. This is more convenient notationally, however, in the applicationto generalised Jackson networks in Section 4 we work with the conventional scaling.5 emma 2.1. Let conditions (A0) and (A2) hold. Then equations (2.1a) – (2.1d) admit aunique strong solution Q n , which is a Z K + -valued stochastic process. The proof is provided in the appendix. Intuitively, the linear growth condition (A2) onthe total arrival rate, together with the asymptotic bounds (A0), implies that the samplepaths of the Q ni are nonexplosive. We assume conditions (A0)–(A5) throughout this section. We introduce the “centered”processes as follows: For i ∈ IK and t ≥ M ni ( t ) = M A,ni ( t ) + M B,ni ( t ) − M D,ni ( t ) , (3.2a)where M A,ni ( t ) = N A,ni (cid:18)Z t λ ni ( Q n ( s )) ds (cid:19) − Z t λ ni ( Q n ( s )) ds, (3.2b) M B,ni ( t ) = K X j =1 (cid:0) Φ nji (cid:0) D nj ( t ) (cid:1) − p ji D nj ( t ) (cid:1) , (3.2c)and M D,ni ( t ) = N D,ni (cid:18)Z t µ ni ( Q n ( s ))1 { Q ni ( s ) > } ds (cid:19) − Z t µ ni ( Q n ( s ))1 { Q ni ( s ) > } ds + K X j =1 p ji (cid:16) N D,nj (cid:18)Z t µ nj ( Q n ( s ))1 { Q nj ( s ) > } ds (cid:19) − Z t µ nj ( Q n ( s ))1 { Q nj ( s ) > } ds (cid:17) . (3.2d)We can rewrite the evolution (2.1a) as Q ni ( t ) = Q ni (0) + Z t " λ ni ( Q n ( s )) + K X j =1 p ji µ nj ( Q n ( s )) − µ ni ( Q n ( s )) ds + M ni ( t ) + [ RY n ( t )] i , where Y n ( t ) = ( Y ni ( t ) , i ∈ IK ) and Y ni ( t ) = Z t { Q ni ( s )=0 } µ ni ( Q n ( s )) ds, i ∈ IK, (3.3)Note that ( Y ni ( t ) , t ≥
0) is a continuous-path non-decreasing process with Y ni (0) = 0, whichincreases only when Q ni ( t ) = 0, i.e., R ∞ { Q ni ( t ) =0 } dY ni ( t ) = 0 a.s. Set a n ( x ) = λ n ( x ) − Rµ n ( x ) . (3.4)6hen the state evolution can be expressed succinctly by the following vector equation: Q n ( t ) = Q n (0) + Z t a n ( Q n ( s )) ds + M n ( t ) + RY n ( t ) , t ≥ . (3.5)The latter dynamic can equivalently be described in terms of a Skorohod map as describedbelow. Definition 3.1.
Let ψ ∈ D ([0 , ∞ ) , IR K ) be given with ψ (0) ∈ S . Then the pair ( φ, η ) ∈ D ([0 , ∞ ) , IR K ) × D ([0 , ∞ ) , IR K ) solves the Skorohod problem for ψ with respect to S and R if the following hold: (i) φ ( t ) = ψ ( t ) + Rη ( t ) ∈ S , for all t ≥ for i ∈ IK, (a) η i (0) = 0 , (b) η i is non-decreasing, and (c) η i can increase only when φ is on the i th face of S , that is, R ∞ { φ i ( s ) =0 } dη i ( s ) = 0 . Let D S ([0 , ∞ ) , IR K ) = { ψ ∈ D ([0 , ∞ ) , IR K ) : ψ (0) ∈ S } . If the Skorohod problem has aunique solution on a domain D ⊂ D S ([0 , ∞ ) , IR K ), we define the Skorohod map Γ on D byΓ( ψ ) = φ . The following result (see Dupuis and Ishii [3], Harrison and Reiman [5]) yields the regularityof the Skorohod map and is a consequence of Assumption (A1).
Proposition 3.2.
The Skorohod map Γ is well defined on D S ([0 , ∞ ) , IR K ) and is Lipschitzcontinuous in the following sense: There exists a constant L > such that for all T > and ψ , ψ ∈ D S ([0 , ∞ ) , IR K ) , sup t ∈ [0 ,T ] | Γ( ψ )( t ) − Γ( ψ )( t ) | ≤ L sup t ∈ [0 ,T ] | ψ ( t ) − ψ ( t ) | . As a consequence, both φ and η are continuous functions of ψ . (Note that matrix R isinvertible under the hypotheses.)The dynamic in (3.5) can now be equivalently described in terms of the Skorohod map asfollows: Q n ( t ) = Γ (cid:18) Q n (0) + Z · a n ( Q n ( s )) ds + M n ( · ) (cid:19) ( t ) , for t ≥ . (3.6)The Lipschitz continuity of the Skorohod map and of the function a ( x ) imply that theequation X ( t ) = Γ (cid:0) X + Z · a ( X ( s )) ds + M ( · ) (cid:1) ( t ) , (3.7)where M i ( t ) = W Ai ( λ i (0) t ) + K X j =1 W Φ ji ( µ j (0) t ) − K X j =1 ( δ ij − p ji ) W Dj ( µ j (0) t ) , (3.8)7as a unique strong solution.We now state the main result of this paper. For t ≥ i ∈ IK , let X ni ( t ) = Q ni ( t ) / √ n .We also define X = ( X ( t ) , t ≥ X n = (( X ni ( t ) , i = 1 , , . . . , K ) , t ≥
0) .
Theorem 3.3.
Let conditions (A0)–(A5) hold. Then X n ⇒ X , as n → ∞ . To prove this theorem, we first establish certain tightness results. Recall that a sequence V n of stochastic processes with trajectories in a Skorohod space is said to be C -tight if thesequence of the laws of the V n is tight, and if all limit points of the sequence of the laws ofthe V n are laws of continuous-path processes (see, e.g., Definition 3.25 and Proposition 3.26in Chapter VI of Jacod and Shiryaev [7]). Lemma 3.4.
The sequence of processes ( M n ( t ) / √ n, t ≥ is C -tight.Proof. By (2.1a) – (2.1d), K X i =1 Q ni ( t ) ≤ K X i =1 Q ni (0) + K X i =1 A ni ( t ) = K X i =1 Q ni (0) + K X i =1 N A,ni (cid:18)Z t λ ni ( Q n ( s )) ds (cid:19) . Therefore, for suitable
H >
0, on recalling (A2) and denoting Z ni ( t ) = Q ni ( t ) /n , K X i =1 Z ni ( t ) ≤ K X i =1 Z ni (0) + K X i =1 sup y ≥ n y N A,ni ( y ) (cid:18) Z t n λ ni ( nZ n ( s )) ds (cid:19) ≤ K X i =1 Z ni (0) + K X i =1 sup y ≥ n y N A,ni ( y ) H Z t (1 + K X i =1 Z ni ( s )) ds ! . By Gronwall’s inequality (cf. p.498 in Ethier and Kurtz [4]), K X i =1 Z ni ( t ) ≤ (cid:0) K X i =1 Z ni (0) + K X i =1 sup y ≥ n y N A,ni ( y )(1 + Ht ) (cid:1) exp (cid:0) H K X i =1 sup y ≥ n y N A,ni ( y ) t (cid:1) . By (A5), N A,ni ( y ) /y → y → ∞ and n → ∞ and P Ki =1 Z ni (0) → n → ∞ . Therefore,lim r →∞ lim sup n →∞ P (sup s ≤ t K X i =1 Z ni ( s ) > r ) = 0 . (3.9)It follows by (A2) thatlim r →∞ lim sup n →∞ P ( Z t (cid:0) n λ ni ( Q ni ( s )) + 1 n µ ni ( Q ni ( s )) (cid:1) ds > r ) = 0 (3.10)8nd that, for δ > ǫ > T > δ → lim sup n →∞ P (cid:0) sup t ∈ [0 ,T ] Z t + δt (cid:0) n λ ni ( Q ni ( s )) + 1 n µ ni ( Q ni ( s )) (cid:1) ds > ǫ (cid:1) = 0 . (3.11)We have that, for γ > δ > ǫ > T >
0, and r > P ( sup s,t ∈ [0 ,T ]: | s − t |≤ δ | √ n M A,ni ( t ) − √ n M A,ni ( s ) | > γ ) ≤ P (cid:0)Z T n λ ni ( Q ni ( s )) ds > r (cid:1) + P (cid:0) sup t ∈ [0 ,T ] Z t + δt n λ ni ( Q ni ( s )) ds > ǫ (cid:1) + P ( sup s,t ∈ [0 ,r ]: | s − t |≤ ǫ | N A,ni ( t ) − N A,ni ( s ) | > γ ) . By (A5), (3.10), and (3.11),lim δ → lim sup n →∞ P ( sup s,t ∈ [0 ,T ]: | s − t |≤ δ | √ n M A,ni ( t ) − √ n M A,ni ( s ) | > γ ) = 0 . Hence, the sequences of processes ( M A,ni ( t ) / √ n, t ≥
0) are C -tight. A similar argumentshows that the sequences of processes ( M D,ni ( t ) / √ n, t ≥
0) and ( M Φ ,ni ( t ) / √ n, t ≥
0) are C -tight, so the sequence of processes ( M n ( t ) / √ n, t ≥
0) is C -tight.Next, we identify the limit points of M n = ( M n ( t ) / √ n, t ≥ Lemma 3.5.
The sequence of processes M n converges in distribution, as n → ∞ , to M .Proof. From Lemma 3.4, M n ( t ) /n → H ′ , for all n and x , | a n ( nx ) | ≤ H ′ n (1 + | x | ) . By (3.9), the sequenceof processes ( R t (1 /n ) a n ( Q n ( s )) ds, t ≥
0) is C -tight . By (3.5), the fact that M n ( t ) /n → Q n ( t ) /n, t ≥
0) is C -tight and every its limitin distribution ( q ( t ) , t ≥
0) satisfies the equation q ( t ) = Γ (cid:18)Z · ( λ ( q ( s )) − Rµ ( q ( s ))) ds (cid:19) ( t ) . Since by (A3), λ ( x ) − Rµ ( x ) = 0, we must have that q ( t ) = 0, which implies that thesequence Q ni ( t ) /n tends to zero as n → ∞ in probability uniformly on bounded intervals.Since Y n is expressed as a continuous function of ( Q n , M n ), we have that Y n ( t ) /n → i ∈ IK ,1 n Z t µ ni ( Q n ( s ))1 { Q ni ( s )=0 } ds → n → ∞ . (3.12)9e also have by (A3) that1 n Z t λ ni ( Q n ( s )) ds → λ i (0) t in probability as n → ∞ (3.13a)and 1 n Z t µ ni ( Q n ( s )) ds → µ i (0) t in probability as n → ∞ . (3.13b)Since by (A5), N D,ni ( nt ) /n → t in probability as n → ∞ , by (2.1d), (A4), (3.12), and(3.13b), D ni ( t ) n → µ i (0) t in probability as n → ∞ . (3.14)The convergences in (A5), (3.13a), (3.13b), and (3.14) imply if one recalls the definitions in(3.2b), (3.2c), and (3.2d) that the ( M A,n / √ n, M B,n / √ n, M D,n / √ n ) converge in distributionto ( M A , M B , M D ) , where M Ai ( t ) = W Ai ( λ i (0) t ), M Bi ( t ) = P Kj =1 W Φ ji ( µ j (0) t ), M Di ( t ) = P Kj =1 ( δ ij − p ji ) W Dj ( µ j (0) t ), so, by (3.2a) and (3.8), the M n converge in distribution to M . Proof of Theorem 3.3.
We note that by (3.6), X n ( t ) = Γ (cid:18) X n (0) + Z · √ n a n ( √ nX n ( s )) ds + M n ( · ) (cid:19) ( t ) , for t ≥ . (3.15)By the Lipschitz continuity of the Skorohod map, (3.4), and (A2), for T >
H > t ∈ [0 ,T ] | X n ( t ) | ≤ | X n (0) | + L Z t √ n | a n ( √ nX n ( s )) | ds + 1 √ n sup t ∈ [0 ,T ] | M n ( t ) |≤ | X n (0) | + LH Z t (1 + | X n ( s ) | ) ds + 1 √ n sup t ∈ [0 ,T ] | M n ( t ) | . Gronwall’s inequality, the convergence of the X n (0), and Lemma 3.5 yieldlim r →∞ lim sup n →∞ P ( sup t ∈ [0 ,T ] | X n ( t ) | > r ) = 0 , so, by (3.4) and (A4), the sequence of processes ( R t a n ( √ nX n ( s )) / √ n ds , t ≥
0) is C -tight.By (3.15), the convergence of the X n (0), Lemma 3.5, (A4), Prohorov’s theorem, and thecontinuity of the Skorohod map, the sequence of processes ( X n ( t ) , t ≥
0) is C -tight andevery limit point ( ˜ X ( t ) , t ≥
0) for convergence in distribution satisfies the equation˜ X ( t ) = Γ (cid:18) X (0) + Z · a ( ˜ X ( s )) ds + M ( · ) (cid:19) ( t ) , for t ≥ . The uniqueness of a solution to the Skorohod problem implies that ˜ X ( t ) = X ( t ) .10 Generalised Jackson networks with state-dependentrates
In this section, we consider an application to the setting of generalised Jackson networks inconventional scaling. Suppose as given mutually independent sequences of i.i.d. nonnegativerandom variables { u ij ( n ) , i ≥ } , { v ik ( n ) , i ≥ } for j ∈ J ⊆ IK and k ∈ IK . For the n th network, the random variable u ij ( n ) represents the i th exogenous interarrival time atstation j , while v ik ( n ) is the i th service time at station k . The quantities p ij represent theprobabilities of a job leaving station i being routed directly to station j , which are heldconstant. The routing decisions, interarrival and service times, and the initial queue lengthvector are mutually independent.We define µ nk = ( E [ v k ( n )]) − > , s nk = Var ( v k ( n )) ≥ , k ∈ IK, and λ nj = ( E [ u j ( n )]) − > , a nj = Var ( u j ( n )) ≥ , j ∈ J , with all of these terms assumed finite and the set J nonempty. It is convenient to let λ nj = 1and a nj = 0 for j
6∈ J .Let ˆ N ˆ A,nj ( t ) = max { i ′ : P i ′ i =1 u ij ( n ) ≤ t } for j ∈ J and ˆ N ˆ D,nk ( t ) = max { i ′ : P i ′ i =1 v ik ( n ) ≤ t } for k ∈ IK . We may interpret the process ( ˆ N ˆ A,nj ( t ) , t ≥
0) as a nominal arrival processand the random variables v ik ( n ) as the amounts of work needed to serve the jobs. Supposethat arrivals are speeded up (or delayed) by a function ˆ λ ni ( x ), where i ∈ J , and the serviceis performed at rate ˆ µ nk ( x ), where k ∈ IK , when the queue length vector is x . As in Section3, we let ˆ N ˆ A,ni ( t ) = ⌊ t ⌋ and ˆ λ ni ( x ) = 0 for i
6∈ J . In analogy with (2.1a)-(2.1d) the queuelengths at the stations at time t , which we denote by ˆ Q ni ( t ), are assumed to satisfy theequations ˆ Q ni ( t ) = ˆ Q ni (0) + ˆ A ni ( t ) + ˆ B ni ( t ) − ˆ D ni ( t ) , ˆ A ni ( t ) = ˆ N ˆ A,ni (cid:18)Z t ˆ λ ni ( ˆ Q n ( s )) ds (cid:19) , ˆ B ni ( t ) = K X j =1 ˆΦ nji (cid:0) ˆ D nj ( t ) (cid:1) , ˆ D ni ( t ) = ˆ N ˆ D,ni (cid:18)Z t ˆ µ ni ( ˆ Q n ( s ))1 { ˆ Q ni ( s ) > } ds (cid:19) , where ˆΦ nji ( m ) = m X l =1 χ nji ( l ) , with { ( χ nji ( l ) , i = 1 , , . . . , K ) , l = 1 , , . . . } being indicator random variables which aremutually independent for different j and l and are such that P ( χ nji ( l ) = 1) = p ji .11f we introduce the random variables Q ni ( t ) = ˆ Q ni ( nt ), A ni ( t ) = ˆ A ni ( nt ), B ni ( t ) = ˆ B ni ( nt ), D ni ( t ) = ˆ D ni ( nt ), N A,ni ( t ) = ˆ N ˆ A,ni ( t/λ ni ), N D,ni ( t ) = ˆ N ˆ D,ni ( t/µ ni ), and Φ nji ( m ) = ˆΦ nji ( m ),and functions λ ni ( x ) = nλ ni ˆ λ ni ( x ) and µ ni ( x ) = nµ ni ˆ µ ni ( x ), then we can see that they satisfyequations (2.1a)–(2.1d). Condition (A0) holds as N A,ni ( t ) /t → N D,ni ( t ) /t → t → ∞ .If we also assume that ˆ Q n (0) / √ n ⇒ X , that, for k ∈ IK and j ∈ J , µ nk → µ k , s nk → s k ,λ nj → λ j , a nj → a j , as n → ∞ , and thatmax k ∈ IK sup n ≥ E ( v k ( n )) ǫ + max j ∈J sup n ≥ E ( u j ( n )) ǫ < ∞ for some ǫ > , then condition (A5) holds with W Aj = √ a j λ j B Aj for j ∈ J , W Aj ( t ) = 0 for j / ∈ J , and W Dk = √ s k µ k B Dk for k ∈ IK , where B Aj and B Dk are independent standard Brownian motions,with Φ i being a K -dimensional Brownian motion with covariance matrix E Φ ik ( t )Φ ij ( t ) =( p ij δ jk − p ij p ik ) t , and with processes B Aj , B Dk , and Φ i being mutually independent.Let us assume that the following versions of conditions (A2)–(A4) hold: [ ( A
2) For each i ∈ IK , sup n ∈ IN sup x ∈ S ˆ λ ni ( nx )1 + | x | + ˆ µ ni ( nx )1 + | x | ! < ∞ , [ ( A
3) There exist continuous functions ˆ λ i ( x ) and ˆ µ i ( x ) such thatˆ λ ni ( nx ) → ˆ λ i ( x ) , ˆ µ ni ( nx ) → ˆ µ i ( x )uniformly on compact subsets of S , as n → ∞ . Furthermore, for x ∈ S , λ ( x ) − Rµ ( x ) = 0 , where λ i ( x ) = λ i ˆ λ i ( x ) and µ i ( x ) = µ i ˆ µ i ( x ) , [ ( A
4) There exists a Lipschitz-continuous function ˆ a ( x ) such that √ n ( λ n ( √ nx ) − Rµ n ( √ nx )) → ˆ a ( x )as n → ∞ uniformly on compact subsets of S , where λ ni ( x ) = λ ni ˆ λ ni ( x ) and µ ni ( x ) = µ ni ˆ µ ni ( x ) . 12hen the process M in (3.7) and (3.8) is a K -dimensional Brownian motion with covari-ance matrix A which has entries A ii = ˆ λ i (0) λ i a i + ˆ µ i (0) µ i s i (1 − p ii ) + K X j =1 ˆ µ j (0) µ j p ji (1 − p ji + p ji µ j s j ) for i ∈ IK, and A ij = − " ˆ µ i (0) µ i s i p ij + ˆ µ j (0) µ j s j p ji + K X k =1 ˆ µ k (0) µ k p ki p kj (1 − µ k s k ) for 1 ≤ i < j ≤ K. An application of Theorem 3.3 yields the following result.
Corollary 4.1.
If, in addition to the assumed hypotheses, condition (A1) holds, then theprocesses ( ˆ Q n ( nt ) / √ n, t ≥ converge in distribution to the process ( X ( t ) , t ≥ with X ( t ) = Γ (cid:16) X + Z · ˆ a ( X ( s )) ds + A / B ( · ) (cid:1) ( t ) , where B ( · ) is a K -dimensional standard Brownian motion. Remark 4.2.
The conditions on the asymptotics of the arrival and service rates essen-tially boil down to the assumptions that the following expansions hold: λ n ( x ) = λ ( x/n ) + λ ( x/ √ n ) / √ n and µ n ( x ) = µ ( x/n ) + µ ( x/ √ n ) / √ n with suitable functions λ , λ , µ , and µ . Remark 4.3.
If, in addition, the assumption of unit rates is made, that is ˆ λ nj ( x ) = 1 for j ∈ J and ˆ µ nk ( x ) = 1 for k ∈ IK , then the limit process is a K -dimensional reflectedBrownian motion on the positive orthant with infinitesimal drift ˆ a (0) and covariance matrix A , and the reflection matrix R = I − P T , as in Theorem 1 of Reiman [15]. Remark 4.4.
In order to extend applicability, one may consider independent sequences ofweakly dependent random variables { u ij ( n ) , i ≥ } , { v ik ( n ) , i ≥ } for j ∈ J ⊆ IK and k ∈ IK . Under suitable moment and mixing conditions which imply the invariance principle,cf., e.g., Herrndorf [6], Peligrad [13], Jacod and Shiryaev [7], Corollary 4.1 continues tohold. ppendix Proof of Lemma 2.1.
The proof is an adaptation of the one in Puhalskii and Simon [14,Lemma 2.1] and employs the approach of Ethier and Kurtz [4, Theorem 4.1, p.327]. Let θ n ( x ) = 1 + K X i =1 ( µ ni ( x ) + λ ni ( x )) , ˆ µ ni ( x ) = µ ni ( x ) θ n ( x ) , ˆ λ ni ( x ) = λ ni ( x ) θ n ( x ) , and τ n ( t ) = inf { s : Z s θ n ( Q n ( u )) du > t } . We note that τ n ( t ) is finite-valued, differentiable, dτ n ( t ) /dt = 1 /θ n ( Q n ( τ n ( t )) and τ n ( t ) →∞ as t → ∞ . One can see that if the process Q n satisfies a.s. the equations Q ni ( t ) = Q ni (0) + N A,ni (cid:16)Z t λ ni ( Q n ( s )) ds (cid:17) + K X j =1 Φ nji (cid:16) N D,nj (cid:16)Z t µ nj ( Q n ( s ))1 { Q nj ( s ) > } ds (cid:17)(cid:17) − N D,ni (cid:16)Z t µ ni ( Q n ( s ))1 { Q ni ( s ) > } ds (cid:17) , t ≥ , ( A Q n = ( ˆ Q n ( t ) , t ≥
0) defined by ˆ Q n ( t ) = Q n ( τ n ( t )) satisfies a.s. theequations ˆ Q ni ( t ) = ˆ Q ni (0) + N A,ni (cid:16)Z t ˆ λ ni ( ˆ Q n ( s )) ds (cid:17) + K X j =1 Φ nji (cid:16) N D,nj (cid:16)Z t ˆ µ nj ( ˆ Q n ( s ))1 { ˆ Q nj ( s ) > } ds (cid:17)(cid:17) − N D,ni (cid:16)Z t ˆ µ ni ( ˆ Q n ( s ))1 { ˆ Q ni ( s ) > } ds (cid:17) , t ≥ . ( A Z K + -valued process ˆ Q n satisfies a.s. ( A
2) and letˆ τ n ( t ) = inf { s : Z s θ n ( ˆ Q n ( u )) du > t } .
14e show that ˆ τ n ( t ) is well defined for all t a.s. Since by condition (A2), for a suitableconstant L n , θ n ( x ) ≤ L n (1 + x ), we have that Z s θ n ( ˆ Q n ( u )) du ≥ L n Z s
11 + P Ki =1 ˆ Q ni ( u ) du ≥ L n Z s
11 + P Ki =1 ˆ Q ni (0) + P Ki =1 N A,ni ( u ) du, ( A A K X i =1 ˆ Q ni ( t ) ≤ K X i =1 ˆ Q ni (0) + K X i =1 N A,ni (cid:0)Z s ˆ λ ni ( ˆ Q ni ( u )) du (cid:1) and that ˆ λ ni ( x ) ≤ t →∞ N A,ni ( t ) /t < ∞ a.s., the rightmost integral in ( A t → ∞ a.s., so does the leftmost integral, which proves the claim. Inaddition, ˆ τ n ( t ) is differentiable, d ˆ τ n ( t ) /dt = θ n ( ˆ Q n (ˆ τ n ( t )) and ˆ τ n ( t ) → ∞ as t → ∞ a.s. Itfollows that Q n ( t ) = ˆ Q n (ˆ τ n ( t )) satisfies ( A
1) a.s.Thus, existence and uniqueness for ( A
1) holds if and only if existence and uniquenessholds for ( A A
2) follows by recursion on the jumptimes of ˆ Q n . In some more detail, we define the processes ˆ Q n,ℓ = ( ˆ Q n,ℓ ( t ) , t ≥
0) withˆ Q n,ℓ ( t ) = ( ˆ Q n,ℓi ( t ) , i = 1 , , . . . , K ) by ˆ Q n, i ( t ) = ˆ Q ni (0) and, for ℓ = 1 , , . . . , byˆ Q n,ℓi ( t ) = ˆ Q ni (0) + N A,ni (cid:16)Z t ˆ λ ni ( ˆ Q n,ℓ − ( s )) ds (cid:17) + K X j =1 Φ nji (cid:16) N D,nj (cid:16)Z t ˆ µ nj ( ˆ Q n,ℓ − ( s ))1 { ˆ Q n,ℓ − j ( s ) > } ds (cid:17)(cid:17) − N D,ni (cid:16)Z t ˆ µ ni ( ˆ Q n,ℓ − ( s ))1 { ˆ Q n,ℓ − i ( s ) > } ds (cid:17) . Let τ n,ℓ represent the time epoch of the ℓ th jump of ˆ Q n,ℓ with τ n, = 0 . One can see thatˆ Q n, ( t ) = ˆ Q n, (0) if t < τ n, . It follows that ( ˆ Q n, ( t ) , t ≥
0) and ( ˆ Q n, ( t ) , t ≥
0) experiencethe first jump at the same time epoch and the jump size is the same for both processes, so τ n, < τ n, and ˆ Q n, ( t ∧ τ n, ) = ˆ Q n, ( t ) for t < τ n, . We define ˆ Q n ( t ) = ˆ Q n (0) for t < τ n, and ˆ Q n ( t ) = ˆ Q n, ( t ) for τ n, ≤ t < τ n, . Similarly, for an arbitrary ℓ ∈ IN , we obtainthat τ n,ℓ < τ n,ℓ +1 and ˆ Q n,ℓ ( t ∧ τ n,ℓ ) = ˆ Q n,ℓ +1 ( t ) for t < τ n,ℓ +1 . We let ˆ Q n ( t ) = ˆ Q n,ℓ ( t )for τ n,ℓ ≤ t < τ n,ℓ +1 . The process ˆ Q n is defined consistently for t ∈ ∪ ∞ ℓ =1 [ τ n,ℓ − , τ n,ℓ ) . If τ n,ℓ +1 = ∞ for some ℓ , then we let ˆ Q n ( t ) = ˆ Q n,ℓ ( t ) for all t ≥ τ n,ℓ .Suppose that τ n,ℓ < ∞ for all ℓ . Then ˆ Q n ( t ) has been defined for all t < τ n, ∞ =lim ℓ →∞ τ n,ℓ and satisfies ( A
2) for these values of t . We now show that τ n, ∞ = ∞ . The setof the time epochs of the jumps of ˆ Q n is a subset of the set of the time epochs of the jumps15f the process ˜ Q n = ( ˜ Q n ( t ) , t ≥ Q n ( t ) = K X i =1 (cid:18) N A,ni (cid:16)Z t ˆ λ ni ( ˆ Q n,ℓ − ( s )) ds (cid:17) + K X j =1 Φ nji (cid:16) N D,nj (cid:16)Z t ˆ µ nj ( ˆ Q n,ℓ − ( s ))1 { ˆ Q n,ℓ − j ( s ) > } ds (cid:17)(cid:17) + N D,ni (cid:16)Z t ˆ µ ni ( ˆ Q n,ℓ − ( s ))1 { ˆ Q n,ℓ − i ( s ) > } ds (cid:17)(cid:19) . Since the process ˆ Q n has infinitely many jumps, so does the process ˜ Q n . Since ˆ λ ni ( x ) ≤ µ ni ( x ) ≤ nji ( m ) − Φ nji ( m ) ≤ m − m for m ≥ m , the lengths of time between thejumps of ˜ Q n are not less than the lengths of time between the corresponding jumps of the pro-cess (cid:0)P Ki =1 N A,ni ( t ) + P Ki =1 N D,ni ( t ) , t ≥ (cid:1) . The process (cid:0)P Ki =1 N A,ni ( t ) + P Ki =1 N D,ni ( t ) , t ≥ (cid:1) having infinitely many jumps, the time epochs of the jumps of (cid:0)P Ki =1 N A,ni ( t )+ P Ki =1 N D,ni ( t ) , t ≥ (cid:1) tend to infinity as the jump numbers tend to infinity. Thus, τ n, ∞ = ∞ a.s.The provided construction shows that Q n is a suitably measurable function of N A,n , N D,n , and Φ n , so it is a strong solution. We have proved the existence of a strong solutionto ( A
2) . A similar argument establishes uniqueness.
References [1] R. Bekker.
Queues with state-dependent rates . PhD thesis, Technische UniversiteitEindhoven, Eindhoven, 2005.[2] W. K. Chan and L. W. Schruben. Mathematical programming representations for state-dependent queues. In S.J. Mason, R.R. Hill, L. M¨onch, O. Rose, T. Jefferson, and J.W.Fowler, editors,
Proceedings of the 2008 Winter Simulation Conference , pages 495–501.IEEE, 2008.[3] P. Dupuis and H. Ishii. On Lipschitz continuity of the solution mapping to the Skorohodproblem, with applications.
Stochastics , 35:31–62, 1991.[4] S. N. Ethier and T. G. Kurtz.
Markov Processes. Characterization and Convergence .Wiley, 1986.[5] J. M. Harrison and M. I. Reiman. Reflected Brownian motion on an orthant.
Ann.Probab. , 9(2):302–308, 1981.[6] N. Herrndorf. A functional central limit theorem for weakly dependent sequences ofrandom variables.
Ann. Probab. , 12(1):141–153, 1984.[7] J. Jacod and A. N. Shiryaev.
Limit Theorems for Stochastic Processes , volume 288 of
Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathe-matical Sciences] . Springer-Verlag, Berlin, second edition, 2003.168] E. V. Krichagina. Asymptotic analysis of queueing networks (martingale approach).
Stochastics Stochastics Rep. , 40(1-2):43–76, 1992.[9] H. J. Kushner.
Heavy traffic analysis of controlled queueing and communication net-works , volume 47 of
Applications of Mathematics . Springer-Verlag, New York, 2001.[10] A. Mandelbaum, W. A. Massey, and M. Reiman. Strong approximations for Markovianservice networks.
Queueing Syst. , 30:149–201, 1998.[11] A. Mandelbaum and G. Pats. State-dependent stochastic networks. I. Approximationsand applications with continuous diffusion limits.
Ann. Appl. Probab. , 8(2):569–646,1998.[12] G. Pang, R. Talreja, and W. Whitt. Martingale proofs of many-server heavy-trafficlimits for Markovian queues.
Probab. Surv. , 4:193–267, 2007.[13] M. Peligrad. Invariance principles under weak dependence.
J. Multivariate Anal. ,19(2):299–310, 1986.[14] A. Puhalskii and B. Simon. Discrete evolutionary birth-death processes and their largepopulation limits.
Stochastic Models , 2012 (to appear).[15] M. I. Reiman. Open queueing networks in heavy traffic.
Mathematics of OperationsResearch , 9(3):441–458, 1984.[16] W. Whitt. Queues with service times and interarrival times depending linearly andrandomly upon waiting times.
Queueing Systems Theory Appl. , 6(4):335–351, 1990.[17] K. Yamada. Diffusion approximation for open state-dependent queueing networks inthe heavy traffic situation.