Age of Information in a Decentralized Network of Parallel Queues with Routing and Packets Losses
JJOURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 1
Age of Information in a Decentralized Network ofParallel Queues with Routing and Packets Losses
Josu Doncel and Mohamad Assaad
Abstract —The paper deals with Age of Information (AoI)in a network of multiple sources and parallel queues withbuffering capabilities, preemption in service and losses in servedpackets. The queues do not communicate between each other andthe packets are dispatched through the queues according to apredefined probabilistic routing. By making use of the StochasticHybrid System (SHS) method, we provide a derivation of theaverage AoI of a system of two parallel queues (with and withoutbuffer capabilities) and compare the results with those of a singlequeue. We show that known results of packets delay in QueuingTheory do not hold for the AoI. Unfortunately, the complexityof computing the average AoI using the SHS method increaseshighly with the number of queues. We therefore provide anupper bound of the average AoI in a system of an arbitrarynumber of
M/M/ / ( N + 1) ∗ queues and show its tightnessin various regimes. This upper bound allows providing a tightapproximation of the average AoI with a very low complexity.We then provide a game framework that allows each sourceto determine its best probabilistic routing decision. By usingMean Field Games, we provide an analysis of the routing gameframework, propose an efficient iterative method to find therouting decision of each source and prove its convergence tothe desired equilibrium. I. I
NTRODUCTION
Age of Information (AoI) is a relatively new metric thatmeasures the freshness of information in the network. AoIis gaining interest in many areas (e.g. control, communicationnetworks, etc) due to the proliferation of applications in whicha monitor is interested in having timely updates about aprocess of interest. As a typical example, AoI can capture thetimeliness of information in a sensor network where the statusof a sensor is frequently monitored. Since its introduction inthe seminal papers [1], [2], AoI has attracted the attention ofmany researchers in different fields.A main part of the AoI literature focuses on the compu-tation of the average AoI and its minimization, where thechannel/network in which the updates are sent to the monitoris modeled as a queueing system. The computation of AoIin various queueing models have therefore been investigated.For instance, the authors in [2] considered an M/M/1 queue,an M/D/1 queue or a D/M/1 queue model, and the authorsin [3], [4] studied an M/M/2 queue model. Other queueingmodels can also be found in [5], [6], [7], [8], [9]. While inthe aforementioned work, the status updates of the system areassumed to be sent under a predefined transmission policy, theproblem of the design of the update policy has been consideredin some papers as well, e.g. [10], [11],[12]. Furthermore,
J. Doncel is with the University of the Basque Country, Spain, 48940 Leioa,Spain (e-mail:[email protected])M. Assaad is with CentraleSup´elec, Universit´e Paris-Saclay, Laboratoiredes Signaux et Syst`emes 91190 Gif-sur-Yvette, France. the problem of scheduling and random access design withthe aim of minimizing the average age of the network hasbeen considered recently in several papers [13], [14], [15],[16], [17], [18]. Besides, since single server queue models arenot representative of networks in which packets can be sentthrough multiple paths, the average AoI has also been studiedin networks with parallel servers [19], [20], [21], or in morecomplex networks such as multihop systems [22], [23]. Forinstance, the scheduling of a single packet flow in multi-hopqueueing networks was studied in [22], [23]. It was shown thatPreemptive Last Generated First Served (P-LGFS) policy isage-optimal if service times are i.i.d. exponentially distributed.In [24], a multihop scenario in which each node is both asource and a monitor is considered. Fundamental age limitsand near optimal scheduling policies are provided in this work.Recently, the analysis of AoI in a multihop multicast contexthas been studied in [25]. For more detailed and comprehensivereview of recent work on AoI, one can refer to [26].In this paper, the network model is different from the afore-mentioned previous work since we consider multiple sourcesthat can send their status updates through a system of differentparallel queues. The queues are assumed to be decentralizedin the sense that they cannot communicate between each other.The incoming traffic from each source is dispatched throughthe parallel queues according to a predefined probabilisticrouting. Note that we also develop a framework to optimizethis probabilistic routing decision as we will see later on inthis paper. Furthermore, we consider a realistic assumptionthat the transmissions through the parallel queues are notperfect and that packets can be lost. This assumption is quiterealistic in many scenarios, e.g. when the transmission arisesover wireless links (which induces errors and hence packetlosses) or even in wired networks when the service providerbreaks down. We aim to analyze the average AoI of this systemcomposed of parallel queues and, for this purpose, we use thestochastic Hybrid Systems (SHS) method, which is introducedin [27] (we explain it in detail in Section II). A related workto ours is [21], where the authors use the SHS method tocompute the average AoI for a system formed by multiplesources and an arbitrary number of homogeneous M/M/1/1queues (i.e. with no buffer) as well as two heterogeneousM/M/1/1 queues, where in both cases preemption in service isallowed. In our work, we compute the average AoI using theSHS method, including a system formed by two heterogeneousM/M/1/2* queues with preemption in service and packetlosses. Due to the buffering capability at different queues, theanalysis becomes more challenging and complex as comparedto [21]. In addition, we assume that queues are decentralizedin the sense that they do not communicate between each a r X i v : . [ c s . PF ] D ec OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 2 other. This makes our model different than [21], where itis assumed that all the queues know where is the freshestupdate. Besides, we provide an upper bound of the averageAoI in a system of an arbitrary number of
M/M/ / ( N + 1) queues. This allows obtaining an approximation of the AoIwith a low complexity. Finally, we provide in this paper agame framework to optimize the probabilistic routing decisionfor each source, which is to the best of our knowledge has notbeen considered before in the AoI literature.The main contributions of this work are twofold. First, weconsider a system with multiple sources where the packets inservice can be lost and preemption is allowed. The packetsare sent to the parallel queues according to a predefinedprobabilistic routing. We compute the average AoI of a systemwith two parallel queues and we compare its average AoI withthat of a single queue. On one hand, in Table I, we present theobtained results where we compare the following systems: (i)two parallel M/M/1/1 queues, each of them with arrival rate λ/ , service rate µ and loss rate θ/ , (denoted by SERVER-ROUTING), (ii) one M/M/1/1 queue with arrival rate λ/ ,loss rate θ/ and service rate µ (denoted by SERVER-HALF)and (iii) one M/M/1/1 queue with arrival rate λ , loss rate θ and service rate µ (denoted by SERVER-DOUBLE). Onthe other hand, in Table II, we show results in which wecompare the following systems: (i) two parallel M/M/1/2*queues, each of them with arrival rate λ/ , service rate µ and loss rate θ (denoted by QUEUE-ROUTING), (ii) oneM/M/1/3* queue with arrival rate λ/ , service rate µ and lossrate θ/ (denoted by QUEUE-HALF) and (iii) one M/M/1/3*queue with arrival rate λ , service rate µ and loss rate θ (denoted by QUEUE-DOUBLE). The description of M/M/1/3*and M/M/1/2* queues will be provided in Section III. Themain conclusion of this part of the work is that the knownresults of packet delay in Queuing Theory do not hold forthe average AoI. For instance, we know that the delay ofthe systems SERVER-ROUTING and SERVER-HALF is thesame, whereas according to our results, the average AoI ofSERVER-ROUTING is smaller. This property also holds whenwe compare the systems QUEUE-ROUTING and QUEUE-HALF. Besides, we also conclude that the average AoI ofSERVER-ROUTING is very close to the average AoI ofSERVER-DOUBLE and also that the average AoI of QUEUE-ROUTING is very close to the average AoI of QUEUE-DOUBLE.Besides, since the complexity of computing the exact averageAoI with SHS method increases hugely with the number ofparallel queues, the second contribution of this work consistsof providing an upper-bound on the average AoI of a systemcomposed of multiple sources with an arbitrary number ofparallel M/M/1/(N+1)* queues. We also study numericallythe accuracy of the upper bound and we conclude that whenthe arrival rate is large or when there are multiple sources,the upper bound is very tight. The interest of this upperbound lies in the fact that it allows obtaining the averageAoI with a low complexity. The last contribution of this workconsists in using the derived upper bound of the averageAoI in order to optimize the probabilistic routing decision.For instance, we formulate a distributed framework where each source optimizes its own routing decision using GameTheory. By using Mean Field Games, we then provide a mod-ification/simplification of the framework and derived a simpleiterative algorithm allowing each source to find separately itsown routing decision. We also provide a theoretical proof ofthe convergence of the iterative algorithm to the desired fixedpoint (Equilibrium) of the game.The rest of the paper is organized as follows. In Section II,we formulate the problem of calculating the average AoI andwe present how the SHS can be used. In Section III wefocus on the average AoI derivation in the different systemsunder consideration. We present the upper bound of the AoIin Section IV and, finally, we provide the main conclusion ofour work in Section V.II. A O I AND
SHSWe consider a transmitter sending status updates to amonitor. Packet i is generated at time s i and is received bythe monitor at time s (cid:48) i . Hence, we define by N ( t ) the index ofthe last received update at time t , i.e., N ( t ) = max { i | s (cid:48) i ≤ t } ,and the time stamp of the last received update at time t as U ( t ) = s N ( t ) . The AoI, or simply the age, is defined as ∆( t ) = t − U ( t ) . We are interested in calculating the average of the stochasticprocess ∆( t ) , that is, the average age, which is defined as ∆ = lim τ →∞ τ (cid:90) τ ∆( t ) dt. The computation of the average age in a general setting isknown to be a challenging task since the random variables ofthe interarrival times and of the system times are dependent.To overcome this difficulty, the authors in [27] introduce theSHS. For completeness, we describe hereinafter this methodand, for further details one can refer to [28].In SHS, the system is modeled as a hybrid state ( q ( t ) , x ( t )) ,where q ( t ) is a state of a continuous time Markov Chain and x ( t ) is a vector whose component belong to R +0 and capturesthe evolution of the age in the system.A link l of the Markov Chain represents a transition fromtwo states q and q (cid:48) with rate λ l . The interest of SHS is thateach transition l implies a reset mapping in the continuousprocess x . In other words, in each transition l , the vector x istransformed to x (cid:48) using a linear mapping where transformationmatrix is given by A l , that is, we have the following SHStransition for every l : x (cid:48) = xA l . Throughout this paper, wedenote by x (cid:48) i the i-th element of the vector x (cid:48) .Furthermore, each state of the Markov Chain represents theelements of the system whose age increases at unit rate. Inother words, for each state q , we define b q as the vector whoseelements are zero or one. Besides, the evolution of the vector x ( t ) for state q is given by ˙ x ( t ) = xb q .We assume the Markov Chain is ergodic and we denote by π q the stationary distribution of state q . Let L q the set of linksthat get out of state q and L (cid:48) q the set of links that get into state q . The following theorem allows us to characterize the averageAoI: OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 3
Without losses With lossesSingle Source (SERVER-ROUTING) and (SERVER-DOUBLE) (SERVER-ROUTING) and (SERVER-DOUBLE).have equal age for λ small and large. See Figure 4. have equal age always. See Figure 6.Multiple Sources (SERVER-ROUTING) and (SERVER-DOUBLE) (SERVER-ROUTING) and (SERVER-DOUBLE).have almost equal age. See Figure 5. have equal age. See Figure 7. TABLE I: Summary of Average AoI comparison of the systems without buffer (see Section III-A) .Without losses With lossesSingle Source QUEUE-ROUTING and QUEUE-DOUBLE QUEUE-ROUTING and QUEUE-DOUBLEhave equal age for λ small and large. See Figure 11. have equal age. See Figure 13.Multiple Sources QUEUE-ROUTING and QUEUE-HALF QUEUE-ROUTING and QUEUE-DOUBLEhave equal age for λ small. have equal age always. See Figure 14.QUEUE-ROUTING and QUEUE-DOUBLEhave equal age for λ large. See Figure 12. TABLE II: Summary of Average AoI comparison of the systems with buffer (see Section III-B).
Theorem 1 ([27, Thm 4]) . Let v q ( i ) denote the i -th elementof the vector v q . For each state q , if v q is a non-negativesolution of the following system of equations v q (cid:88) l ∈L q λ l = b q π q + (cid:88) l ∈L (cid:48) q λ l v q l A l , (1) then the average AoI is ∆ = (cid:80) q v q (0) . In the following section, we use the above result to charac-terize the average AoI of several systems. In Section IV, weshow that the method under consideration can be also used toobtain an upper-bound on the average AoI of very complexsystems.III. A
VERAGE A O I OF R OUTING S YSTEMS V ERSUS OF A S INGLE Q UEUE
In this section, we aim to study the average AoI for differentconfigurations using the SHS method. We first focus on asystem formed by queues without buffer and then considerseveral cases of queues with buffer. Furthermore, we considerin this section two main scenarios: i) system with single queue,and ii) system with multiple parallel queues. In the latter, weconsider that multiple sources are dispatching their packetsthrough the different parallel queues according to a predefinedprobabilistic routing. This kind of routing policies is used inpractice and is widely considered in routing literature since itcan be implemented without knowing the instantaneous statesof the network or of the servers (this assumption is realistic asin real-life networks such information cannot be known at thesources). In more detail, the routing policy can be explainedas follows. Each source i dispatches its packets according tothe following policy: each job/packet of the source is routed toqueue j with probability p ij . We can see then that the arrivalrate from source i to queue j is λ i p ij . In addition, it is obviousto see that (cid:80) Kj =1 p ij = 1 . Besides, we consider also that thetransmission through the queues is not reliable and that theloss rate of each queue j is denoted by θ j . We also assumethat the queues are decentralized in the sense that they do notcommunicate between each other. A. Queues Without Buffer
In this section, we study the age when the queues donot have a buffer. We first focus on a system formed by l q l → q l (cid:48) λ l x (cid:48) = xA l ¯ v q l A l → λ [ x
0] [ v (0) 0] → (cid:80) k> λ k [ x x ] [ v (0) v (0)] → µ [ x
0] [ v (1) 0] → θ [ x
0] [ v (0) 0] → λ [ x
0] [ v (0) 0] → (cid:80) k> λ k [ x x ] [ v (0) v (0)] TABLE III: Table of transitions of Figure 1an M/M/1/1 queue and then in a system with two parallelM/M/1/1 queues.
1) The M/M/1/1 queue:
We consider a system formed byone M/M/1/1 queue that receives traffic from different n sources when preemption of the packets that are in serviceis permitted. We therefore consider a Poisson arrival processfrom each source and hence the resulting arrival from allsources is a Poisson process and two update arrivals cannotoccur simultaneously. We are interested in calculating theaverage AoI of any source. Without loss of generality, wefocus on source . Thus, we consider that updates arrive tothe system according to a Poisson process, where, without lossof generality, the rate of updates of source are denoted by λ and of the rest of the sources (cid:80) k> λ k . The total arrivalrate in the system is denoted by λ , i.e., λ = (cid:80) nk =1 λ k . Weassume that the service time is exponentially distributed withrate µ . We also assume that an update that is in service is lostwith an exponential time of rate θ .We use the SHS method to compute the age of this system.The continuous state of the SHS is x ( t ) = [ x ( t ) x ( t )] ,where the correspondence between x i (t) and the elements ofthe system is as follows: x is the age at the monitor and x thegeneration time of the packet in service. One can notice that x ( t ) = x ( t ) if the packet/update in service is delivered. Thediscrete state of the SHS is a two-state Markov Chain, where represents that the system is empty and that an update isbeing executed. As explained in the previous section, in eachtransition in the Markov chain, the continuous state of the SHS x changes to x (cid:48) . The Markov Chain is represented in Figure 1and the SHS transitions are given in Table III in which westate explicitly the transitions from x to x (cid:48) .We now explain each transition l : l = 0 There is an arrival of source and the queue is idle.Therefore, the age of the monitor does not change, OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 4
Fig. 1: The SHS Markov Chain for the one M/M/1/1 queuesystem with multiple sources and losses of packets in service.i.e., x (cid:48) = x and the age of the packet in service iszero since there is a fresh update arrived. l = 1 There is an arrival of one of the others sources whenthe queue is idle. Since we are interested in the ageof source 1, an arrival from the rest of the sourcesdoes not bring a fresh update of the status of source and hence it changes the value of x to the age ofthe monitor, that is, the age of the update in servicesatisfies x (cid:48) = x , where x is the age of the monitor.Therefore, when this update ends its service, the ageof the monitor remains unchanged. l = 2 The update under execution ends its service and theage of the monitor is updated by that of this update,i.e., x (cid:48) = x . l = 3 The update that is in service is lost and, therefore,the age of the monitor does not change. l = 4 There is an arrival of source when the queue isin service. For this case, the update in service isreplaced by the fresh one and, therefore, the age ofthe monitor does not change, i.e., x (cid:48) = x , but theage of the update in service changes to zero. l = 5 There is an arrival of another source when the queueis in service. For this case, the update in service isreplaced by the fresh one and the age of the update inservice changes to that of the monitor, i.e., x (cid:48) = x .The stationary distribution of the Markov Chain of Figure 1is π = µ + θλ + µ + θ , π = λλ + µ + θ . Besides, for the state q = 0 , we have that b = [1 , sincethe age of the monitor is the only one that grows at unit rateand the age of the update in service is irrelevant, whereasfor the state q = 1 we have that b = [1 , and the ageof the monitor and of the update in service grow at a unitrate. On the other hand, we have that v = [ v (0) v (1)] and v = [ v (0) v (1)] . From Theorem 4 of [27], we know thatthe age of this system is v (0) + v (0) , where λ v =[ π
0] + µ [ v (1) 0] + θ [ v (0) 0] , ( λ + µ + θ ) v =[ π π ] + λ [ v (0) 0] + (cid:88) k> λ k [ v (0) v (0)]+ λ [ v (0) 0] + (cid:88) k> λ k [ v (0) v (0)] , The above expressions can be written as the following system of equations: λv (0) = π + µv (1) + θv (0) , (2a) ( µ + θ ) v (0) = π + λv (0) , (2b) ( λ + µ + θ ) v (1) = π + (cid:88) k> λ k v (0) + (cid:88) k> λ k v (0) . (2c)The solution to the above system of equations is v (0) = 1 λ + θλ µ − π λ + µ + θv (0) = λλ µ + π λ + µ + θv (1) = (cid:80) k> λ k λ µ + π λ + µ + θ . Therefore, since, from (1), the average AoI for this case is v (0) + v (0) , the next result follows: Proposition 1.
The average AoI of source in the aforemen-tioned system is given by λ + θλ µ + λλ µ . . Remark 1.
We remark that, when θ = 0 , (2) coincides with theresult of Theorem 2a) in [27]. In their model, they consider aMarkov Chain with a single state, but when there are updatesthat are lost their model cannot be considered. Therefore, webelieve that the model presented above is the simplest one tostudy the average AoI using the SHS method when there areupdate losses.2) Two parallel M/M/1/1 queues: We now consider a sys-tem formed by two parallel M/M/1/1 queues receiving trafficfrom different n sources and where preemption of packets inservice is permitted. We aim to calculate the average age ofinformation of source . As in the previous case, the arrivalsare Poisson and the rate of source is denoted by λ andthat of the rest of the sources is denoted by (cid:80) k> λ k . Hence, λ = (cid:80) nk =1 λ k . The packets are dispatched according to apredefined probabilistic routing, where p j (resp. p kj ) is theprobability that a job of source (resp. of another source k (cid:54) = 1 ) is routed to queue j . We denote by λ p and by λ p the arrival rates from source to queue 1 and to queue 2,respectively. Likewise, (cid:80) k> λ k p k and (cid:80) k> λ k p k denotesthe arrival rates from the rest of the sources to queue 1 andto queue 2, respectively. The service rate and the loss rate inqueue j are respectively µ j and θ j , where j = 1 , . We assumethat the queues are decentralized in the sense that they do notcommunicate between each other. Remark 2.
The latter assumption makes the model understudy here different than [21], where it is assumed that theservers know where is the freshest update.
We compute the average AoI using the SHS method. First,we define the continuous state as x ( t ) = [ x ( t ) x ( t ) x ( t )] ,where the correspondence between x i (t) and each element inthe system is as follows: x is the age of the monitor and x OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 5
00 0110 11 451011 1617222301236 7 8 9 1213141518192021
Fig. 2: The SHS Markov Chain for system with two parallelM/M/1/1 queues with multiple sources and losses of packetsin service. l q l → q l (cid:48) λ l x (cid:48) = xA l ¯ v q l A l → λ p [ x v (0) 0 0] → (cid:80) k> λ k p k [ x x ] [ v (0) 0 v (0)] → µ [ x v (2) 0 0] → θ [ x v (0) 0 0] → λ p [ x v (0) 0 0] → (cid:80) k> λ k p k [ x x ] [ v (0) 0 v (0)] → λ p [ x v (0) 0 0] → (cid:80) k> λ k p k [ x x
0] [ v (0) v (0) 0] → µ [ x v (1) 0 0] → θ [ x v (0) 0 0] → λ p [ x v (0) 0 0] → (cid:80) k> λ k p k [ x x
0] [ v (0) v (0) 0] → λ p [ x x ] [ v (0) 0 v (2)] → (cid:80) k> λ k p k [ x x x ] [ v (0) v (0) v (2)] → µ [ x x ] [ v (1) 0 v (2)] → θ [ x x ] [ v (0) 0 v (2)] → λ p [ x x ] [ v (0) 0 v (2)] → (cid:80) k> λ k p k [ x x x ] [ v (0) v (0) v (2)] → λ p [ x x
0] [ v (0) v (1) 0] → (cid:80) k> λ k p k [ x x x ] [ v (0) v (1) v (0)] → µ [ x x
0] [ v (2) v (1) 0] → θ [ x x
0] [ v (0) v (1) 0] → λ p [ x x
0] [ v (0) v (1) 0] → (cid:80) k> λ k p k [ x x x ] [ v (0) v (1) v (0)] TABLE IV: Table of transitions of Figure 2.(resp. x ) is the age if an update of queue 1 (resp. of queue2) is delivered. The discrete state is a Markov Chain with fourstates, represented in Figure 2 and where state k k representsthat in queue 1 there are k updates, with k ∈ { , } , andin queue 2 there are k updates, with k ∈ { , } . We alsorepresent the SHS transitions in Table IV.We now explain each transition l : l = 0 There is an arrival of source to queue 2, when thesystem is empty. Therefore, x and x do not changeand the age of the update in service in queue 2 iszero due to a fresh update arrival. l = 1 There is an arrival of an update of the other sourcesto queue 2 when it is idle. Therefore, we set x (cid:48) = x ,which means that, when this update ends its service,the age of the monitor is again x . l = 2 The update under execution in queue 2 is deliveredand the age of the monitor is updated by that of this update, i.e., x (cid:48) = x . l = 3 The update that is in service in queue 2 is lost and,therefore, the age of the monitor does not changeand queue 2 has no updates. l = 4 There is an arrival of source to queue 2 when it is inservice. For this case, since preemption is permitted,the update in service is replaced by the fresh oneand, therefore, the age of the update in service inqueue 2 changes to zero. l = 5 There is an arrival of another source to queue 2 whenit has an update in service. For this case, the updatein service is replaced by the fresh one and the ageof the update in service in queue 1 changes to thatof the monitor, i.e., x (cid:48) = x .The transitions 6-11 are symmetric to 0-5, respectively. l = 12 When there is an update in queue 2, if an update ofsource arrives to queue 1, the age of the monitorand of the update in queue 2 do not change, whereasthat of queue 1 is set to zero, that is, x (cid:48) = 0 . l = 13 When there is an update in queue 2, if there is anarrival of one of the other sources in queue 1, theage of the update in queue 1 changes to the age ofthe monitor, i.e,. x (cid:48) = x , whereas x and x do notchange. l = 14 When there are updates in both queues, an updateof queue 1 is delivered and the age of the monitorchanges to x , i.e., x (cid:48) = x . l = 15 When there are updates in both queues, if an updateof queue 1 is lost, the age of the monitor does notchange and queue 1 is idle. l = 16 When both queues have updates in service, if anupdate of source arrives to queue 1, we set x (cid:48) to zero and the rest does not change. l = 17 When both queues have updates in service, if anupdate of the other sources arrives to queue 1, weset x (cid:48) to the same as the monitor.The transitions 18-23 are symmetric to 12-17, respec-tively.The stationary distribution of the Markov Chain in Figure 2is given by π k k = ρ k ρ k (1 + ρ )(1 + ρ ) , for k , k = 0 , , where ρ j = λ p j + (cid:80) k> λ k p kj µ j + θ j , j = 1 , . Moreover, we definethe value of b q for each state q ∈ { , , , } as follows: b = [1 0 0] , b = [1 1 0] , b = [1 0 1] and b = [1 1 1] .We also define, for q ∈ { , , , } , the following vector v q = [ v q (0) v q (1) v q (2)] .Let ˆ µ = µ + µ and ˆ θ = θ + θ . The SHS method saysthat the average AoI of this system is (cid:80) q v q (0) , where v q (0) is the solution of the following system of equations: λ v = [ π µ [ v (1) 0 0]+ θ [ v (0) 0 0]+ µ [ v (2) 0 0] + θ [ v (0) 0 0] (3) OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 6 ( λ + µ + θ ) v = [ π π
0] + λ p [ v (0) 0 0]+ (cid:88) k> λ k p k [ v (0) v (0) 0]+ µ [ v (2) v (1) 0]+ θ [ v (0) v (1) 0] + λ p [ v (0) 0 0]+ (cid:88) k> λ k p k [ v (0) v (0) 0] (4) ( λ + µ + θ ) v = [ π π ] + λ p [ v (0) 0 0]+ (cid:88) k> λ k p k [ v (0) 0 v (0)]+ µ [ v (1) 0 v (2)]+ θ [ v (0) 0 v (2)]+ λ p [ v (0) 0 0]+ (cid:88) k> λ k p k [ v (0) 0 v (0)] (5) ( λ + ˆ µ + ˆ θ ) v = [ π π π ] + λ p [ v (0) 0 v (2)]+ (cid:88) k> λ k p k [ v (0) v (0) v (2)]+ λ p [ v (0) 0 v (2)]+ (cid:88) k> λ k p k [ v (0) v (0) v (2)]+ λ p [ v (0) v (1) 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (0)]+ λ p [ v (0) v (1) 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (0)] . (6)Since the first equation has two irrelevant variables andthe second and third ones have one irrelevant variable, theabove expression can be written alternatively as a system of8 equations. Proposition 2.
The average AoI of source in the aforemen-tioned system is given by v (0) + v (0) + v (0) + v (0) ,where for q ∈ { , , , } , v q (0) is the solution of (3) - (6) .3) Average AoI Comparison: We now compare the averageAoI of source for the models we have studied in this section.For this purpose, we consider three systems. The first oneconsists of a single M/M/1/1 queue with arrival rate of source λ / and of the rest of the sources (1 / (cid:80) k> λ k , loss rate θ/ and service rate µ (see Figure 3a). The average AoI ofthis model is represented with a solid line. The second systemwe consider is a single M/M/1/1 queue with arrival rate ofsource λ and of the rest of the sources (cid:80) k> λ k , loss rate θ and service rate µ (see Figure 3b). The average AoI of thismodel is represented with a dotted line. Finally, we consider asystem with two parallel M/M/1/1 queues with arrival rate ofsource equal to λ and of the rest of the sources (cid:80) k> λ k .Besides, we consider that p kj = 1 / for all k = 1 , . . . , n and j = 1 , , the loss rate and the service rate are in both servers θ/ and µ , respectively (see Figure 3c). The average AoI of this model is represented with a dashed line. Our goal is todetermine which system has the smallest average AoI when λ varies and the rest of the parameters are fixed. To thisend, we have solved numerically the systems of equations in(2) and in (3)-(6). We set µ = 1 in these simulations. Whenwe study the system with multiple sources, we consider that (cid:80) k> λ k = 10 , and in the case of losses in the packets inservice, we set θ = 10 .In Figure 4, we plot the average AoI of these systems as afunction of λ when there is a single source and there areno losses in the queues. We observe that the smallest age isachieved for the single M/M/1/1 queue system with servicerate µ . We also observe that the age of the two parallelM/M/1/1 queues is the same as the latter when λ is eithervery small or very large.In Figures 5-7, we show that the average AoI of a systemwith two parallel M/M/1/1 queues is equal to that of a singleserver with service rate µ .In queueing theory, it is known that, among the systemsunder consideration in this section, the one that minimizesthe delay is the single M/M/1/1 queue with service rate µ . Therefore, these illustrations show that the AoI alsoverifies this property. On the other hand, for classical queueingtheory metrics such as delay, the performance of two parallelM/M/1/1 queues coincides with that of a single M/M/1/1 queuewith arrival rate λ/ and loss rate ˆ θ/ . However, as far asaverage AoI is concerned, one can see that, according to thefigures we present in this section, this is not the case (solidand dashed lines do not coincide on these figures). B. Queues With Buffer
We now focus on queues with buffer. For this case, anupdate starts getting served upon its arrival to a queue, if thequeue is idle. However, if the queue is busy, the incomingupdate is put in the last position of the queue and, if the queueis full, the last update of the buffer is replaced by the new one.In this section, we aim to compare the average AoI of a systemwith one queue and buffer size two with that of two parallelqueues with buffer size one. We first compute the average AoIof the former system and then of the latter one.
1) The M/M/1/3* queue:
We concentrate on a systemformed by a queue with a buffer of size two. When an updatearrives and the system is empty, it gets served immediately.However, if an update arrives when another packet is inservice, it replaces the last update in the queue if it is fulland, otherwise, it is put in the last position of the queue. Thissystem will be denoted in the remainder of the paper as theM/M/1/3* queue.When traffic comes from n different sources, we are inter-ested in computing the average AoI of source . Updates ofsource arrive to the queue according to a Poisson processof rate λ and of those of the rest of the sources with rate (cid:80) k> λ k . We assume that the updates that are waiting in thequeue are served according to the FCFS discipline and thatthe service time is exponentially distributed with rate µ , aswell as the update that is in service is lost with exponentiallydistributed time with rate θ . OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 7 μ2μ λ (λ +λ )/2 μμ MonitorMonitorMonitor μ2μμμ
MonitorMonitorMonitor λ λ λ (λ +λ )/2 λ / 2λ / 2λ λ λ / 2λ / 2λ λ θθ/2θ/2θ/2θ/2θ/2θ/2θ (λ +λ )/2(λ +λ )/2 (a) M/M/1/1 queue with arrival rate λ/ . μ2μ λ (λ +λ )/2 μμ MonitorMonitorMonitor μ2μμμ
MonitorMonitorMonitor λ λ λ (λ +λ )/2 λ / 2λ / 2λ λ λ / 2λ / 2λ λ θθ/2θ/2θ/2θ/2θ/2θ/2θ (λ +λ )/2(λ +λ )/2 (b) M/M/1/1 queue with service rate µ . μ2μ λ (λ +λ )/2 μμ MonitorMonitorMonitor μ2μμμ
MonitorMonitorMonitor λ λ λ (λ +λ )/2 λ / 2λ / 2λ λ λ / 2λ / 2λ λ θθ/2θ/2θ/2θ/2θ/2θ/2θ (λ +λ )/2(λ +λ )/2 (c) Two parallel M/M/1/1 queues. Fig. 3: Representation of the models under comparison in Section III-A3 for two sources. -1 A v e r age A o I M/M/1/1 with arrival rate lambda/2M/M/1/1 with service rate 2*mu2 parallel M/M/1/1
Fig. 4: Average AoI of source when λ varies from . to with a single source and without losses ( λ k = 0 for all k > and θ = 0 ). µ = 1 . -1 A v e r age A o I M/M/1/1 with arrival rate lambda/2M/M/1/1 with service rate 2*mu2 parallel M/M/1/1
Fig. 5: Average AoI of source when λ varies from . to with multiple sources and without losses ( (cid:80) k> λ k = 10 and θ = 0 ). µ = 1 .We employ the SHS method to calculate the average AoIof this system. The continuous state is given by x ( t ) =[ x ( t ) x ( t ) x ( t ) x ( t )] , where the correspondence between x i (t) and each element in the system is as follows: x is theage of the monitor, x is the age if the update in service isdelivered and x and x is respectively the age if the updatein the first and second positions of the queue are delivered.The discrete state is a four state Markov Chain, where state k represents that there are k updates present in the system, with k = 0 , , , . The Markov Chain under consideration and theSHS transition are represented respectively in Figure 8 andTable V.We now explain each transition l : l = 0 The system is empty and an update of source -1 A v e r age A o I M/M/1/1 with arrival rate lambda/2M/M/1/1 with service rate 2*mu2 parallel M/M/1/1
Fig. 6: Average AoI of source when λ varies from . to with a single source and losses ( λ k = 0 for all k > and θ = 10 ). µ = 1 . -1 A v e r age A o I M/M/1/1 with arrival rate lambda/2M/M/1/1 with service rate 2*mu2 parallel M/M/1/1
Fig. 7: Average AoI of source when λ varies from . to with multiple sources and losses ( (cid:80) k> λ k = 10 and θ = 10 ). µ = 1 . Fig. 8: The SHS Markov Chain for the M/M/1/3* queue withmultiple sources and losses of packets in service.arrives. The age of the monitor is not modified andwe set x (cid:48) = 0 . l = 1 The system is empty and an update of another sourcearrives. The age of the monitor is not modified and
OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 8 l q l → q l (cid:48) λ l x (cid:48) = xA l ¯ v q l A l → λ [ x v (0) 0 0 0] → (cid:80) k> λ k [ x x v (0) v (0) 0 0] → µ [ x v (1) 0 0 0] → θ [ x v (0) 0 0 0] → λ [ x x v (0) v (1) 0 0] → (cid:80) k> λ k [ x x x
0] [ v (0) v (1) v (1) 0] → µ [ x x v (1) v (2) 0 0] → θ [ x x v (0) v (2) 0 0] → λ [ x x x
0] [ v (0) v (1) v (2) 0] → (cid:80) k> λ k [ x x x x ] [ v (0) v (1) v (2) v (2)] → µ [ x x x
0] [ v (1) v (2) v (3) 0] → θ [ x x x
0] [ v (0) v (2) v (3) 0] → λ [ x x x
0] [ v (0) v (1) v (2) 0] → (cid:80) k> λ k [ x x x x ] [ v (0) v (1) v (2) v (2)] TABLE V: Table of transitions of Figure 8the age x changes to x , i.e., x (cid:48) = x . l = 2 When there is an update getting in service and thequeue is empty. If the update in service is delivered,the age of the monitor changes to x , i.e., x (cid:48) = x . l = 3 The queue is empty and the update in service is lost.For this case, the age of the monitor does not changeand the age of x is replaced by zero. l = 4 The queue is busy and an update of source i arrives.The age of the monitor and that of x are notmodified and we set x (cid:48) = 0 . l = 5 The queue is busy and an update of source i arrives.The age of the monitor and that of x are notmodified and the age x changes to x , i.e., x (cid:48) = x . l = 6 There are two updates in the system and the updatein service is delivered and, therefore, the age of themonitor changes to x and the age x to x , i.e., x (cid:48) = x and x (cid:48) = x . l = 7 There are two updates in the system and the updatein service is lost. For this case, the age of the monitordoes not change, but the age x is replaced by x ,i.e., x (cid:48) = x since the update that was waiting startgetting served. l = 8 There are two updates in the system and an updateof source arrives. The ages of the updates that arepresent in the system do not change and we set x (cid:48) =0 . l = 9 There are two updates in the system and an updateof another source arrives. The ages of the updatesthat are present in the system do not change and theage x changes to x , i.e., x (cid:48) = x . l = 10 The system is full and the update in service isdelivered. For this case, the age of the monitorchanges to x , the age of x to x and the age of x to x . l = 11 The system is full and the update in service is lost.For this case, the age of the monitor does not change,but the age of x changes to x and the age of x to x . l = 12 The system is full and an update of source arrives.The age of the monitor and of that of x and x arenot modified and we set x (cid:48) = 0 . l = 13 The system is full and an update of another sourcearrives. The ages of the monitor, of x and of x do not change, but the age of x is set to x , i.e., x (cid:48) = x .Let λ = (cid:80) nk =1 λ k and ρ = λµ + θ . The stationary distributionof the Markov Chain in Figure 8 is π j = ρ j ρ + ρ + ρ , j = 0 , , , . We now define the vector b q for all state q of the MarkovChain of Figure 8: b = [1 0 0 0] , b = [1 1 0 0] , b =[1 1 1 0] and b = [1 1 1 1] . Besides, for all state q ∈{ , , , } , v q = [ v q (0) v q (1) v q (2) v q (3)] . From Theorem 4in [27], we have that the average AoI of the M/M/1/3* queueis v (0) + v (0) + v (0) + v (0) , where v q (0) is the solutionof the following system of equations: λ v =[ π µ [ v (1) 0 0 0] + θ [ v (0) 0 0 0] (7) ( λ + µ + θ ) v =[ π π λ [ v (0) 0 0 0]+ (cid:88) k> λ k [ v (0) v (0) 0 0]+ µ [ v (1) v (2) 0 0]+ θ [ v (0) v (2) 0 0] . (8) ( λ + µ + θ ) v =[ π π π
0] + λ [ v (0) v (1) 0 0]+ (cid:88) k> λ k [ v (0) v (1) v (1) 0]+ µ [ v (1) v (2) v (3) 0]+ θ [ v (0) v (2) v (3) 0] . (9) ( λ + µ + θ ) v =[ π π π π ] + λ [ v (0) v (1) v (2) 0]+ (cid:88) k> λ k [ v (0) v (1) v (2) v (2)]+ λ [ v (0) v (1) v (2) 0]+ (cid:88) k> λ k [ v (0) v (1) v (2) v (2)] (10)Since the first equation has three irrelevant variables andthe second and third equations have respectively two and oneirrelevant variables, the above expression can be alternativelywritten as a system of 10 equations. Proposition 3.
The average AoI of source in the aforemen-tioned system is given by v (0) + v (0) + v (0) + v (0) , wherefor q ∈ { , , , } , v q (0) is the solution of (7) - (10) .2) Two parallel M/M/1/2* queues: We consider a systemformed by two parallel queues with buffer size equal to oneand n different sources. The packets are dispatched accordingto a predefined probabilistic routing. An update of source arrives to the system with rate λ and it is sent to queue j withprobability p j . Therefore, the arrival rate of source to queue j is λ p j . Besides, an update of the rest of the sources arrivesto the system with rate (cid:80) k> λ k and an update of source k (cid:54) = i is sent to queue j with probability p kj . Therefore, the arrivalrate of the rest of the sources to queue j is (cid:80) k> λ k p kj . Ifan update finds the queue full, it replaces the update that iswaiting in the queue, whereas when the queue is idle, it starts OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 9
00 01 021020 1121 1222 41522 25 28 2901 235 6 7 8 9 101112 131416 17 18 19 20 212423 2726
Fig. 9: The SHS Markov Chain for system with two parallelM/M/1/2* queues with multiple sources and losses of packetsin service.being served immediately. This system will be denoted as twoparallel M/M/1/2* queues.We assume that the service time of queue j is exponentiallydistributed with rate µ j and that updates/packets in service inqueue j are lost with exponential time of rate θ j , j = 1 , .We assume that the queues are decentralized in the sense thatthey do not communicate between each other.We seek to compute the average AoI of this systemusing the SHS method. The continuous state is x ( t ) =[ x ( t ) x ( t ) x ( t ) x ( t ) x ( t )] , where the correspondencebetween x i (t) and each element is as follows: x is the ageof the monitor, x j is the age if the update/packet in servicein queue j is delivered and x j the age if the update thatis waiting for service in queue j is delivered. The discretestate is described by a Markov Chain, where the state k k denotes that there are k updates in queue 1 and k in queue2, with k ∈ { , , } . The Markov Chain we study is depictedin Figure 9. We note that some of the links are unified toavoid heavy notation. The SHS transitions for this model arereported in Appendix C.Let ρ = λ p + (cid:80) k> λ k p k µ + θ and ρ = λ p + λ k p k µ + θ . Thestationary distribution of the Markov Chain in Figure 9 is π k k = ρ k ρ k (1 + ρ + ρ )(1 + ρ + ρ ) , k , k ∈ { , , } . Let ˆ µ = µ + µ , ˆ θ = θ + θ and Q = { , , , , , , , , } . For every q ∈ Q , we define v q = [ v q (0) v q (1) v q (2) v q (3) v q (4)] and the vector b q asfollows: b = [1 0 0 0 0] , b = [1 1 0 0 0] , b = [1 1 1 0 0] , b = [1 0 0 1 0] , b = [1 1 0 1 0] , b = [1 1 1 1 0] , b = [1 0 0 1 1] , b = [1 1 0 1 1] and b = [1 1 1 1 1] .We use the result of Theorem 4 in [27] that shows that theaverage AoI of this system is given by (cid:80) q ∈Q v q (0) , where v q (0) is the solution to the following system of equations: λ v =[ π µ [ v (1) 0 0 0 0]+ θ [ v (0) 0 0 0 0] + µ [ v (3) 0 0 0 0]+ θ [ v (0) 0 0 0 0] (11) ( λ + µ + θ ) v =[ π π λ p [ v (0) 0 0 0 0]+ (cid:88) k> λ k p k [ v (0) v (0) 0 0 0]+ µ [ v (3) v (1) 0 0 0]+ θ [ v (0) v (1) 0 0 0]+ µ [ v (1) v (2) 0 0 0]+ θ [ v (0) v (2) 0 0 0]+ µ [ v (3) v (1) 0 0 0]+ θ [ v (0) v (1) 0 0 0] (12) ( λ + µ + θ ) v =[ π π π µ [ v (3) v (1) v (2) 0 0]+ θ [ v (0) v (1) v (2) 0 0]+ λ p [ v (0) v (1) 0 0 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (1) 0 0]+ λ p [ v (0) v (1) 0 0 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (1) 0 0] (13) ( λ + µ + θ ) v =[ π π λ p [ v (0) 0 0 0 0]+ (cid:88) k> λ k p k [ v (0) 0 0 v (0) 0]+ µ [ v (3) 0 0 v (4) 0]+ θ [ v (0) 0 0 v (4) 0]+ µ [ v (1) 0 0 v (3) 0]+ θ [ v (0) 0 0 v (3) 0] (14) ( λ + ˆ µ + ˆ θ ) v =[ π π π λ p [ v (0) 0 0 v (3) 0]+ (cid:88) k> λ k p k [ v (0) v (0) 0 v (3) 0]+ λ p [ v (0) v (1) 0 0 0]+ (cid:88) k> λ k p k [ v (0) v (1) 0 v (0) 0]+ µ [ v (3) v (1) 0 v (4) 0]+ θ [ v (0) v (1) 0 v (4) 0]+ µ [ v (1) v (2) 0 v (3) 0]+ θ [ v (0) v (2) 0 v (3) 0] (15) OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 10 ( λ + ˆ µ + ˆ θ ) v =[ π π π π λ p [ v (0) v (1) 0 v (3) 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (1) v (3) 0]+ λ p [ v (0) v (1) 0 v (3) 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (1) v (3) 0]+ λ p [ v (0) v (1) v (2) 0 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (2) v (0) 0]+ µ [ v (3) v (1) v (2) v (4) 0]+ θ [ v (0) v (1) v (2) v (4) 0] (16) ( λ + µ + θ ) v =[ π π π ]+ λ p [ v (0) 0 0 v (3) 0]+ (cid:88) k> λ k p k [ v (0) 0 0 v (3) v (3)]+ λ p [ v (0) 0 0 v (3) 0]+ (cid:88) k> λ k p k [ v (0) 0 0 v (3) v (3)]+ µ [ v (1) 0 0 v (3) v (4)]+ θ [ v (0) 0 0 v (3) v (4)] (17) ( λ + ˆ µ + ˆ θ ) v = [ π π π π ]+ λ p [ v (0) v (1) 0 v (3) 0]+ (cid:88) k> λ k p k [ v (0) v (1) 0 v (3) v (3)]+ λ p [ v (0) v (1) v (2) 0 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (2) v (0) 0]+ µ [ v (1) v (2) 0 v (3) v (4)]+ θ [ v (0) v (2) 0 v (3) v (4)] (18) ( λ + ˆ µ + ˆ θ ) v = [ π π π π π ]+ λ p [ v (0) v (1) 0 v (3) v (4)]+ (cid:88) k> λ k p k [ v (0) v (1) v (1) v (3) v (4)]+ λ p [ v (0) v (1) 0 v (3) v (4)]+ (cid:88) k> λ k p k [ v (0) v (1) v (1) v (3) v (4)]+ λ p [ v (0) v (1) v (2) v (3) 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (2) v (3) v (3)]+ λ p [ v (0) v (1) v (2) v (3) 0]+ (cid:88) k> λ k p k [ v (0) v (1) v (2) v (3) v (3)] (19)From the above expressions, if we remove the irrelevantvariables, we obtain a system of 27 equations. Proposition 4.
The average AoI of source in the afore-mentioned system is given by (cid:80) q ∈Q v q (0) , where v q (0) is thesolution of (11) - (19) .3) Age Comparison: We compare the average AoI of themodels presented in this section. We focus on the followingthree systems. First, we consider an M/M/1/3* queue witharrival rate of source equal to λ / and that of the rest of thesources (1 / (cid:80) k> λ k , loss rate θ/ and service rate µ (seeFigure 10a). The average AoI of this model is represented witha solid line. The second system we consider is an M/M/1/3*queue with arrival rate of source equal to λ and that of therest of the sources (cid:80) k> λ k , loss rate θ and service rate µ (see Figure 10b). The average AoI of this model is representedwith a dotted line. We also consider a system with two parallelM/M/1/2* queues with arrival rate of source λ and that ofthe rest of the sources (cid:80) k> λ k . Each of the servers satisfiesthat p k = p k = 1 / for all k = 1 , . . . , n , has a loss rate equalto θ/ and a service rate µ (see Figure 10c). The average AoIof the latter model is represented with a dashed line. We aimto investigate which system has the smallest average AoI when λ varies. Thus, we have solved numerically the systems ofequations in (7)-(10) and of (11)-(19). We set µ = 1 in thesesimulations. When we study the system with multiple sources,we consider that (cid:80) k> λ k = 10 and in the case of packetlosses, we set θ = 10 .We first focus on the average AoI for a single source andwhen there are no packet losses. The evolution of the AoI ofsource with respect to λ is represented in Figure 11. Weobserve that for the M/M/1/3* queue with service rate µ , theaverage AoI coincides with that of the two parallel M/M/1/2*queues when λ is either very small or very large. Anotherinteresting property obtained from these simulations is thatthe average AoI of the M/M/1/3* queue is not monotone in λ . This phenomenon is due to the FCFS discipline and thepresence of the buffer of size 2 and can be interpreted asfollows. For small arrival rates, all the packets will be delivereddirectly without staying too much time in the buffer and thesystem will behave like an M/M/1/1 queue. Therefore, whenthe arrival rate increases and on average there is only onepacket (or less) in the server, the AoI will keep decreasingsince more fresh packets improves the AoI. We can observein this figure that the minimum AoI is achieved when λ = µ (which means that on average we have one packet in theserver as explained above). Then, when the arrival rate keepsincreasing, there will be always packets in the buffer (inaddition to the packet in the server) and the arrived packetswill be delayed, which will increase the AoI. When the arrivalrate grows very large, the packet in the second place in thebuffer will be constantly replaced by the new arrived packet.However, there will be always a delay due the fact that thepacket in the first place of the buffer should wait until thepacket in the server is delivered. In other words, the AoI willconverge to an asymptotic value. However, this asymptoticvalue is greater than the AoI when λ = µ since in that casethere is on average one packet in the system (i.e. the packetis directly served by the server) and hence the packets are notdelayed by the buffer. In addition to these results, we provide OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 11 μ2μ λ (λ +λ )/2 μμ MonitorMonitorMonitor μ2μμμ
MonitorMonitorMonitor λ λ λ (λ +λ )/2 λ / 2λ / 2λ λ λ / 2λ / 2λ λ θθ/2θ/2θ/2θ/2θ/2θ/2θ (λ +λ )/2(λ +λ )/2 (a) M/M/1/2 queue with arrival rate λ/ . μ2μ λ (λ +λ )/2 μμ MonitorMonitorMonitor μ2μμμ
MonitorMonitorMonitor λ λ λ (λ +λ )/2 λ / 2λ / 2λ λ λ / 2λ / 2λ λ θθ/2θ/2θ/2θ/2θ/2θ/2θ (λ +λ )/2(λ +λ )/2 (b) M/M/1/2 queue with service rate µ . μ2μ λ (λ +λ )/2 μμ MonitorMonitorMonitor μ2μμμ
MonitorMonitorMonitor λ λ λ (λ +λ )/2 λ / 2λ / 2λ λ λ / 2λ / 2λ λ θθ/2θ/2θ/2θ/2θ/2θ/2θ (λ +λ )/2(λ +λ )/2 (c) Two parallel M/M/1/2 queues. Fig. 10: Representation of the models under comparison in Section III-B3 for two sources.in Appendix B some results obtained by simulations for singleM/M/1/3* queue. As expected The results assess the accuracyof the SHS method used to evaluate the average AoI.In Figure 12, we study the average AoI for a system withmultiple sources and without losses. We see that the AoI ofthe two parallel M/M/1/2* queues coincides with that of theM/M/1/3* queue with half traffic rate when λ is small, whereasit coincides with that of the M/M/1/3* queue with doubleservice rate when λ is large. It is worth mentioning that weconsider here that the arrival rate of the rest of the sourcesis (cid:80) k> λ k = 10 , which implies that there will be alwayspackets in the server and in the buffer and the system cannotbehave as an M/M/1/1 by changing λ of the first source. Thisexplains why the average AoI decreases with λ until reachinga limiting value and the average AoI does not have the sameshape as in Figure 11.We also study the average AoI with a single source andlosses in Figure 13. For this case, the AoI of the system withtwo parallel M/M/1/2* queues and of an M/M/1/3* queue withdouble service rate coincide when λ is either small or large.In this case, we can see that the average AoI decreases withthe arrival rate λ . This can be explained by the fact that, sincethe packets can get lost, it is better from AoI perspective tohave more arrived packets (even if these packets are delayedin the buffer).Finally, in Figure 14, we show the average age of informa-tion for different values of λ when there are multiple sourcesand losses. This illustration presents that, depending on thevalue of λ , the AoI approaches that of an M/M/1/3* withhalf arrival rate and half loss rate or that of an M/M/1/3* withdouble service rate, as in Figure 14.The main conclusion of these illustrations is that, from anAoI perspective, the M/M/1/3* queue with double service rateis the optimal one among the systems under consideration.Besides, we characterize the instances where the AoI of thetwo parallel queues coincides with the optimal AoI. C. Parallel Queues With Buffer Size
N > We now focus on the study of the average AoI in a systemwith K parallel queues with buffer size N > . We notice thatusing the SHS method leads to the analysis of a Markov Chainwith a number of states equal to K · ( N + 2) . This implies thatthe number of SHS transitions increases at a very high ratewith the number of queues and with the buffer size. As a result,according to (1), the number of equations to be solved so as toobtain the AoI suffers from the curse of dimensionality. Thus,providing an analytical expression of the AoI of source ina system with an arbitrary routing system (with an arbitrary -1 A v e r age A o I M/M/1/3* with arrival rate lambda/2M/M/1/3* with service rate 2*mu2 parallel M/M/1/2* real
Fig. 11: Average AoI comparison when λ varies from . to with a single source and without losses ( λ k = 0 for all k > and θ = 0 ). µ = 1 . -1 A v e r age A o I M/M/1/3* with arrival rate lambda/2M/M/1/3* with service rate 2*mu2 parallel M/M/1/2* real
Fig. 12: Average AoI comparison when λ varies from . to with multiple sources and without losses ( (cid:80) k> λ k = 10 and θ = 0 ). µ = 1 . -1 A v e r age A o I M/M/1/3* with arrival rate lambda/2M/M/1/3* with service rate 2*mu2 parallel M/M/1/2* real
Fig. 13: Average AoI comparison when λ varies from . to with a single source and losses ( λ k = 0 for all k > and θ = 10 ). µ = 1 . OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 12 -1 A v e r age A o I M/M/1/3* with arrival rate lambda/2M/M/1/3* with service rate 2*mu2 parallel M/M/1/2* real
Fig. 14: Average AoI comparison when λ varies from . to with multiple sources and losses ( (cid:80) k> λ k = 10 and θ = 10 ). µ = 1 .number of queues and an arbitrary buffer size) seems to beintractable using the considered method. However, as we willsee in the next section, it is possible to provide an upper-boundon the AoI.IV. U PPER - BOUND ON THE A VERAGE A O I FOR AN A RBITRARY R OUTING S YSTEM
We study the average AoI of a system with
K > parallelqueues with N > buffer size. In this section, we provide anupper bound on the age of information using the SHS methodin a system with a single and multiple sources.We now explain the system we study here. We consider asystem with n sources where, for all i, the updates of source i and of the rest of the sources arrive to the system with rate λ i . We denote by p ij the probability that a job of source i is routed to server j . Hence, λ i = (cid:80) Kj =1 λ i p ij . Besides,the total incoming traffic to the system is denoted by λ , i.e., λ = (cid:80) ni =1 λ i . We assume that the service rate in queue j isexponentially distributed with rate µ j . In the following result,we provide an upper bound of the average AoI of the systemunder study here. Without loss of generality, the result isprovided for source , however one can see that the resultcan be obtained for any source i . The proof of this result isreported in Appendix A. Theorem 2.
For the aforementioned system, the average AoIof source is upper bounded by (cid:80) Kj =1 µ j KN + K (cid:88) j =1 (cid:80) k> λ k p kj + µ j λ j p j . (20)It is also important to note that to obtain the above resultwe consider that when an update completes the service, wecreate a fake update to keep the system full of packets. Thisis the reason why, unlike in the previous section, we are notable to study the influence of the packet losses on the upperbound of the average AoI we provide in Theorem 2. In fact,let us consider that N = 0 as an example. In this case, whena packet in service is lost, we put a false/fake update with thesame age of the lost one in service. Therefore, this fake update -1 A v e r age A o I Real AoIUpper bound
Fig. 15: Upper bound and real average AoI comparison when λ varies from . to for two parallel M/M/1/2* queueswith a single source ( λ = 0 ). -1 A v e r age A o I Real AoIUpper bound
Fig. 16: Upper bound and real average AoI comparison when λ varies from . to for two parallel M/M/1/2* queueswith multiple sources ( λ = 10 ). µ = 1 .modifies the age of the monitor when it is served and deliveredto the monitor. As a result, the fake update will modify theage at the monitor and the system will behave like a systemwith no packet losses (but with a different service rate). Thisexplains why the above result cannot capture the impact ofpacket losses. A. Tightness of the Upper bound
We now aim to explore if the upper bound on the averageAoI is tight for the systems we have studied in Section III. Weconsider µ = 1 and, when we analyze the AoI for multiplesources, we fix the arrival rate of the rest of the sources to ,that is, (cid:80) k> λ k = 10 .We first study in Figures 15-16, a system formed by 2parallel queues with equal arrival rate and service rate, i.e., K = 2 , N = 1 , p = p = p = p = 1 / and µ = µ = µ . For this case, we get from Theorem 2 thatthe upper bound for source is µ (cid:18) (cid:80) k> λ k + 2 µλ (cid:19) . As we observe in Figure 15, the upper bound is tightwhen the arrival rate of source is large enough, whereasin Figure 16, we show that it is always very close to the realage. OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 13 -1 A v e r age A o I Real AoIUpper bound
Fig. 17: Upper bound and real average AoI comparison when λ varies from . to for two parallel M/M/1/2* queueswith a single source ( λ = 0 ). µ = 0 . . -1 A v e r age A o I -1 Real AoIUpper bound
Fig. 18: Upper bound and real average AoI comparison when λ varies from . to for two parallel M/M/1/2* queueswith a single source ( λ = 0 ). µ = 10 .We now focus on the influence of µ on the tightness of theupper bound of the average AoI in a system with two parallelM/M/1/2* queues with a single source. First, we consider µ =0 . in Figure 17 and we show that, when λ is larger than 2,the upper bound is very tight. Then, we consider µ = 10 inFigure 18 and we show that, when λ is larger than 10, theupper bound is accurate. Finally, we consider µ = 100 inFigure 19 and, as it can be observed in this illustration, theupper bound is very close to the real age when λ is largerthan 1000.We also investigate a system formed by one M/M/1/3*queue with arrival rate λ , service rate µ and multiple sourcesin Figure 20. For this case, we have that K = 1 and N = 2 and, therefore, from Theorem 2, the average AoI of source is given by µ (cid:18) (cid:80) k> λ k + µλ (cid:19) . As we see in Figure 20, we show that it is always very closeto the real age for any value of λ . Thus, this plot confirmsthat the upper bound on the average AoI we provide in thispaper is very tight when the arrival rate is large. -1 A v e r age A o I -2 -1 Real AoIUpper bound
Fig. 19: Upper bound and real average AoI comparison when λ varies from . to for two parallel M/M/1/2* queueswith a single source ( λ = 0 ). µ = 100 . -1 A v e r age A o I Real AoIUpper bound
Fig. 20: Upper bound and real average AoI comparison when λ varies from . to for one M/M/1/3* queue with multiplesource ( λ = 10 ). µ = 1 . B. AoI Comparison with a Single M/M/1/1 queue
We now consider a system with a single source which isformed by K homogeneous queues without buffer, i.e., µ j = µ for all j = 1 , . . . , K . According to the result of Theorem 2,the average AoI is upper bounded by Kµ µ K (cid:88) j =1 λ j . (21)We now aim to compare the above expression with theaverage AoI of a single M/M/1/1 queue with preemption ofjobs in service, arrival rate λ/K and service rate µ , whichaccording to Theorem 2(a) in [27] is given by Kλ + 1 µ . In the following result, we compare the above expressions.
Proposition 5.
Let
K > . Then, Kλ + 1 µ > Kµ µ K (cid:88) j =1 λ j . OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 14
Proof.
First, we note that, when λ j = λ/K for all j , we havethat Kµ µ K (cid:88) j =1 λ j = 1 Kµ (cid:18) µK λ (cid:19) = 1 Kµ + Kλ .
Therefore, we aim to show that Kµ + Kλ < µ + Kλ ⇐⇒ K > And the desired result follows since the last expression isalways true.An interesting result is derived from the above proposition.Indeed, when we consider a system formed by K parallelqueues and each of them receives the same arrival rate, sincethe expression (21) provides an upper bound on the averageAoI, this result implies that the average AoI of a singleM/M/1/1 queue with arrival rate λ/K is larger than that ofthe considered system.
C. AoI Comparison with [21]
In Theorem 2 in [21], the author provides the followingexpression of the average AoI of a system with homogeneousparallel queues where the incoming jobs are always sent tothe server with the oldest job: µ (cid:32) K K − (cid:89) i =1 ρi + ρ + 1 ρ + 1 ρ K − (cid:88) l =1 l (cid:89) i =1 ρi + ρ (cid:33) . (22)We now notice that, in our model, the knowledge of thequeue with the oldest job is not considered. Therefore, onemight expect that the average AoI is always smaller in themodel in [21]. In the following result, we consider the regimewhere λ tends to infinity and we compare both models. Proposition 6.
When λ → ∞ , we have that (22) and (20) tend to Kµ . Proof.
The proof is straightforward from (22) and (20).From this result, we conclude that, when λ → ∞ , theimprovement of the average AoI caused by the knowledgeof the state of the queues is negligible. D. Optimization of the Upper Bound
In this section, we provide a framework to minimize theupper bound of the average age of each source i . The objectiveis to find the routing probabilities p ij in such that the age upperbound for each source i is minimized. The problem can beformulated as a game framework. More formally, the problemcan be formulated as min p i (cid:80) Kj =1 µ j KN + K (cid:88) j =1 (cid:80) nl (cid:54) = i λ l p lj + µ j λ i p ij , ∀ i (23)s.t. K (cid:88) j =1 p ij = 1 ∀ i (24) where p i = [ p i , ..., p iK ] . In the sequel, we will characterizesthe Nash Equilibrium (NE) of the above problem and providea solution that achieves the NE. Before defining NE, wefirst introduce the so-called best response set valued function( BR i ) for each source (or player) i , which is given as follows( BR i ) : Given p − i ∆ = ( p , ..., p i − , p i +1 , ... p N ) p i ∈ arg max p (cid:48) i U i ( p (cid:48) i , p − i ) where p (cid:48) i satisfies (cid:80) Kj =1 p (cid:48) ij = 1 and U i ( p (cid:48) i , p − i ) = (cid:80) Kj =1 µ j (cid:18) KN + (cid:80) Kj =1 (cid:80) Nl (cid:54) = i λ l p lj + µ j λ i p ij (cid:19) . In order to showexplicitly the dependence of BR i in p − i , we will use thenotation BR i ( p − i ) to represent the best response set valuedfunction of source i . In other words, the best response consistsof optimizing the utility of each source with respect only toits own action vector (i.e. routing probability p i ).Furthermore, we can also define the sources’ joint best-response function as BR ( p ) = ( BR ( p − ) , ...BR N ( p − N )) We now provide the definition of NE and the relation with thesources’ best response.
Definition 1.
A strategy profile p = ( p , ..., p N ) is a pureNash equilibrium iff ∀ i , ∀ p (cid:48) i , U i ( p (cid:48) i , p − i ) ≥ U i ( p i , p − i ) Definition 2.
A strategy profile is a Nash Equilibrium iff p ∈ BR ( p ) In words, a NE is a fixed point of the BR dynamic. In thesequel, we will therefore show that a fixed point of BR existsand it is unique.It is straightforward to see that U i ( p (cid:48) i , p − i ) is convex withrespect to p i . The best response problem for each source i can be solved easily using the standard Lagrangian technique.The Lagrangian for each source can be written as follows, L i ( p i , δ ) = U i ( p i , p − i ) + δ ( K (cid:88) j =1 p ij − The optimal solution of the above optimization problem foreach source i , i.e. ( p ∗ i , δ ∗ ), can then be obtained by KKT andcomplementarity conditions, i.e. by using ∂L i ∂p ij = 0 ∀ j and δ ∗ ( (cid:80) Kj =1 p ∗ ij −
1) = 0 . After some algebraic manipulations,this leads to the following expression of p ∗ i p ∗ ij = ω ij (cid:80) Kj (cid:48) =1 ω ij (cid:48) where ω ij = (cid:113)(cid:80) Nl (cid:54) = i λ l p lj + µ j .Consequently, a NE is simply the solution of the followingsystem of equations p ∗ ij = (cid:113)(cid:80) Nl (cid:54) = i λ l p ∗ lj + µ j (cid:80) Kj (cid:48) =1 (cid:113)(cid:80) Nl (cid:54) = i λ l p ∗ lj (cid:48) + µ j (cid:48) , ∀ i, j (25) OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 15
We will show numerically later on that the aforementionedsystem of equations has a solution. Before that, we willprovide an analysis in the case of large number of sources,by using mean field analysis, and develop a simple iterativealgorithm allowing each source to find its probabilistic routingvector p i = [ p i , ..., p iK ] T .
1) Mean Field Analysis:
We provide here an analysis inthe case of large number of sources and provide a distributediterative algorithm that converges to the solution of the systemof equations in (25). In order to use mean field tools, indistin-guishable sources should be considered. We therefore considerthat λ l = ¯ λ . We first define the following mean field term foreach queue j : m nij = 1 n n (cid:88) l (cid:54) = i p lj One can notice that m nij → m j when n → ∞ ∀ i , where m j = lim n →∞ n (cid:80) ni =1 p ij .The problem in (23) can be written as min p i (cid:80) Kj =1 µ j n KNn + K (cid:88) j =1 λm nij + µ j n λp ij , ∀ i One can notice also that when n → ∞ , µ j /n → ¯ µ j , where ¯ µ j represents the scaled asymptotic service rate per source. Infact, when the number of sources tends to infinity, the servicerate of each queue must be high enough to serve all sources.When n is very large, the system of equations in (25) tendsto: p ∗ ij = (cid:113) m j + ¯ µ j ¯ λ (cid:80) Kj (cid:48) =1 (cid:113) m j (cid:48) + ¯ µ j (cid:48) ¯ λ , ∀ i, j (26)By summing the previous equations over all source indexes i and taking the limit when n → ∞ , we get m j = (cid:113) m j + ¯ µ j ¯ λ (cid:80) Kj (cid:48) =1 (cid:113) m j (cid:48) + ¯ µ j (cid:48) ¯ λ , ∀ j (27)By making the variable change y j = (cid:113) m j + ¯ µ j ¯ λ , the previousequations can be written as: y j = 1 (cid:80) Kj (cid:48) =1 y j (cid:48) + ¯ µ j ¯ λy j , ∀ j (28) Proposition 7.
If the system of equations in (28) has asolution, then this solution is unique.Proof.
Let’s assume that the system of equations hastwo different solutions y = [ y (1)1 , ..., y (1) K ] T and y =[ y (2)1 , ..., y (2) n ] T . Without loss of generality, we can considerthat there exists ν > such that y ≤ ν y and ∃ at least one j for which y (1) j < νy (2) j and ∃ at least one queue k for which y (1) k = νy (2) k . One can see easily that for any possible vectors y = [ y (1)1 , ..., y (1) K ] T and y = [ y (2)1 , ..., y (2) n ] T obtaining ν tosatisfy the above statement is straightforward. Recall that forqueue k y (2) k = 1 (cid:80) Kj (cid:48) =1 y (2) j (cid:48) + ¯ µ k ¯ λy (2) k By using y ≤ ν y and y (1) j < νy (2) j for at least one queue j , we get y (2) k < ν (cid:80) Kj (cid:48) =1 y (1) j (cid:48) + ν ¯ µ k ¯ λy (1) k = νy (1) k Therefore, we obtain y (2) k < νy (1) k which contradicts the factthat y (2) k = νy (1) k . Consequently, it is not possible to have twodifferent solutions to the system of equations in (28).In order to solve the system of nonlinear equations in(28), we provide an efficient iterative learning algorithm andprove its convergence to the solution of (28). The proposedalgorithm has a reduced complexity and can be implemented ina distributed manner. We first consider the following iterativealgorithm: y j ( t +1) = (1 − α ) y j ( t )+ α (cid:80) Kj (cid:48) =1 y j (cid:48) ( t ) + α ¯ µ j ¯ λy j ( t ) , ∀ j (29)where α is a sufficiently small step size. This algorithm is asimple class of Ishikawa algorithm (using Mann-like iterationbut with constant step size α ; see [29] for details).By definition of y , one can notice that ∀ j y j ∈ S j = (cid:20)(cid:113) ¯ µ j ¯ λ , (cid:113) ¯ µ j ¯ λ (cid:21) . Let S = S × ... ×S K be the set of feasiblevalues of y . In order to ensure that the solution obtained bythe iterative algorithm lies in the feasible set S , we considerthe following projection Π S : ˆ y = Π S ( y ) defined as, ∀ j , if y j < (cid:113) ¯ µ j ¯ λ then ˆ y j = max { y j , (cid:113) ¯ µ j ¯ λ } ; if y j > (cid:113) ¯ µ j ¯ λ then ˆ y j = min { y j , (cid:113) ¯ µ j ¯ λ } ; and ˆ y j = y j otherwise. Usingthe aforementioned projection, the iterative algorithm becomes ˆ y j ( t +1) = Π S (cid:18) (1 − α )ˆ y j ( t )+ α (cid:80) Kj (cid:48) =1 ˆ y j (cid:48) ( t ) + α ¯ µ j ¯ λ ˆ y j ( t ) (cid:19) , ∀ j (30)The following result proves that the algorithm above convergesto the solution of the system of equations in (28), wheneverit exists. Notice that once the algorithm converges, one canobtain m from y from the relation y j = (cid:113) m j + ¯ µ j ¯ λ ∀ j .The routing probabilities for each source i , i.e. p i , can thenbe obtained from (26). Finally, one can see that since theaforementioned algorithm depends only on the average arrivaland service rates, without requiring any information about theinstantaneous status of the network, it can be implementedseparately by each source. Proposition 8.
The algorithm in (30) converges to the solutionof the system of equations in (28).Proof.
We denote by f j ( y ( t )) = (cid:80) Kj (cid:48) =1 y j (cid:48) ( t ) + ¯ µ j ¯ λy j ( t ) . Onecan see that f j ( y ( t )) can be written as f j ( y ( t )) = g ( y ( t )) + g j ( y j ( t )) where ˜ g ( y ( t )) = (cid:80) Kj (cid:48) =1 y j (cid:48) ( t ) and g j ( y j ( t )) = ¯ µ j ¯ λy j ( t ) . We also denote by y ∗ = [ y ∗ , ..., y ∗ K ] ∈ S the solutionof (28). Recall that by using the algorithm in (30), we have y j ( t + 1)) = (1 − α )ˆ y j ( t ) + α (cid:80) Kj (cid:48) =1 ˆ y j (cid:48) ( t ) + α ¯ µ j ¯ λ ˆ y j ( t ) and ˆ y j ( t + 1) = Π S ( y j ( t + 1)) . We can also write the following y j ( t + 1) − y ∗ j = (1 − α )(ˆ y j ( t ) − y ∗ j )+ α (˜ g (ˆ y ( t )) − ˜ g ( y ∗ )) + α t ( g j (ˆ y j ( t )) − g j ( y ∗ j )) (31) OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 16
By using the mean value theorem, ∃ ¯y ( t ) ∈ [ˆ y ( t ) , y ∗ ] suchthat ˜ g (ˆ y ( t )) − ˜ g ( y ∗ ) = ∇ ˜ g | T ¯y ( t ) (ˆ y ( t ) − y ∗ ) Similarly, ∃ ˜y ( t ) = [˜ y ( t ) , ..., ˜ y K ( t )] T ∈ [ˆ y ( t ) , y ∗ ] such that g j (ˆ y j ( t )) − g j ( y ∗ j ) = dg j dy j | ˜ y j ( t ) (ˆ y j ( t ) − y ∗ j ) , ∀ j From all the above, we can therefore write y ( t + 1) − y ∗ asfollows: y ( t + 1) − y ∗ = (1 − α )(ˆ y ( t ) − y ∗ ) + α J t (ˆ y ( t ) − y ∗ ) (32)where J t is a K × K matrix with diagonal elements − (cid:16) (cid:80) Kl =1 ¯ y j ( t ) (cid:17) − ¯ µ j ¯ λ ˜ y j ( t ) and the non-diagonal elements are − (cid:16) (cid:80) Kl =1 ¯ y j ( t ) (cid:17) . We can show easily then that ∀ a ∈ R K a T J t a ≤ , i.e. J t is negative semi definite. Then, by using (cid:107) y ( t + 1) − y ∗ (cid:107) = ( y ( t + 1) − y ∗ ) T ( y ( t + 1) − y ∗ ) , we get (cid:107) y ( t + 1) − y ∗ (cid:107) = (1 − α ) (cid:107) ˆ y ( t ) − y ∗ (cid:107) + α (cid:107) J t (ˆ y ( t ) − y ∗ ) (cid:107) + 2(1 − α t ) α (ˆ y ( t ) − y ∗ ) T J t (ˆ y ( t ) − y ∗ ) (33)Since J is negative semi definite and (cid:107) J t (ˆ y ( t ) − y ∗ ) (cid:107) ≤(cid:107) J t (cid:107) (cid:107) ˆ y ( t ) − y ∗ (cid:107) , we get (cid:107) y ( t + 1) − y ∗ (cid:107) ≤ (cid:18) (1 − α ) + α (cid:107) J t (cid:107) (cid:19) (cid:107) ˆ y ( t ) − y ∗ (cid:107) (34)Recall that ˆ y ∈ S and therefore ˆ y ( t ) (cid:54) = 0 , which implies that ¯ y (cid:54) = 0 and ˜ y (cid:54) = 0 and hence (cid:107) J t (cid:107) is bounded. Therefore, ∃ asufficiently small α such that ∀ t α (cid:107) J t (cid:107) + α − α < − (cid:15) < ,where < (cid:15) < . Consequently, (cid:107) y ( t + 1) − y ∗ (cid:107) ≤ (cid:18) − (cid:15) (cid:19) (cid:107) ˆ y ( t ) − y ∗ (cid:107) Then, by using the inequality (cid:107) ˆ y ( t + 1) − y ∗ (cid:107) ≤ (cid:107) y ( t + 1) − y ∗ (cid:107) (since y ∗ ∈ S ), we get (cid:107) ˆ y ( t + 1) − y ∗ (cid:107) ≤ (cid:18) − (cid:15) (cid:19) (cid:107) ˆ y ( t ) − y ∗ (cid:107) ... ≤ (cid:18) − (cid:15) (cid:19) t +1 (cid:107) ˆ y (0) − y ∗ (cid:107) (35)and (cid:107) ˆ y ( t ) − y ∗ (cid:107) → when t → ∞ . This concludes theproof.From the Proof above, one can see that if α is takensuch that α (cid:107) J t (cid:107) + α − α < then the algorithm in(30) converges to the solution of (28). Since for each j ˆ y j ≥ (cid:113) ¯ µ j ¯ λ and (cid:107) J t (cid:107) ≤ (cid:80) Ki =1 (cid:80) Kj =1 J t ( i, j ) , which impliesthat (cid:107) J t (cid:107) ≤ K (cid:32) (cid:80) Kl =1 (cid:113) ¯ µj ¯ λ (cid:33) + K . Consequently, it issufficient to take α < K (cid:80) Kl =1 (cid:114) ¯ µj ¯ λ + K +1 . Number of Iterations P r obab ili t i e s Converge of the Fixed Point Algorithm p p p p p p Fig. 21: Convergence of the fixed point algorithm defined from(25).
2) Numerical Results:
We provide here an example to shownumerically that the system of equations in (25) has a solution.While in the previous subsection, a detailed mathematicalanalysis is provided to optimize the upper bound in the case oflarge number of sources, we show here numerical results forfinite number of sources. In order to solve (25), we considerthe following iterative algorithm: p ij ( t + 1) = (1 − α ) p ij ( t ) + α (cid:113)(cid:80) Nl (cid:54) = i λ l p lj ( t )+ µ j (cid:80) Kj (cid:48) =1 (cid:113)(cid:80) Nl (cid:54) = i λ l p lj (cid:48) ( t )+ µ j (cid:48) , ∀ i, j . One can see that actuallythis iterative algorithm is similar to (30), or more precisely (30)can be obtained from the above algorithm by using Mean Fieldanalysis (using the derivations in the previous subsection). InFigure 21 we show an illustrative example where we considera system with 6 sources et 10 servers with the followingparameters for the sources λ = 100 , λ = 20 , λ = 50 , λ = λ = 10 and λ = 1000 and for the servers µ = 1 ,µ = 2 , µ = 3 , µ = 5 , µ = 10 , µ = 20 , µ = 50 , µ = 100 , µ = 200 and µ = 1000 . The plots show that(25) has a solution, that p , , p , , p , , p , , p , and p , converge to the solution of (25) and also that in this examplethe algorithm converges quickly. This shows that the proposedalgorithm can converge also for finite number of sources withdifferent arrival rates. Providing a formal proof of convergencewould be an interesting topic and is left for future work.V. C ONCLUSION
In this paper, we studied the average AoI of a system ofmultiple sources and parallel queues using the SHS method.We considered that the queues do not communicate betweeneach other and that the sources send their updates to the queuesaccording to a predefined probabilistic routing scheme. First,we computed the average AoI for the following systems: i)two parallel M/M/1/1 queues, ii) one M/M/1/1 queue withhalf arrival and loss rates, and iii) one M/M/1/1 queue withdouble service rate. Then, we computed the average AoI fortwo parallel M/M/1/2* queues, one M/M/1/3* queue with halfarrival and loss rates, and one M/M/1/3* queue with doubleservice time. We conclude that the average AoI of the systemcomposed of parallel queues is always smaller than that ofone queue with half arrival and loss rates, and can be assmall as that of one queue with double service rate. We also
OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 17 studied the average AoI of a system with an arbitrary numberof heterogeneous M/M/1/(N+1)* queues and we provided anupper bound of AoI that is tight when there are multiplesources. We then provided a framework allowing each sourceto determine its routing decision, by using Game Theoryand best response method. In the contest of large number ofsources, we simplified the game framework by using MeanField Games, provided a simple distributed algorithm andproved its convergence to the desired fixed point.R
EFERENCES[1] S. Kaul, M. Gruteser, V. Rai, and J. Kenney, “Minimizing age ofinformation in vehicular networks,” in , 2011, pp.350–358.[2] S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often shouldone update?” in
Proc. of IEEE INFOCOM , 2012, pp. 2731–2735.[3] C. Kam, S. Kompella, and A. Ephremides, “Effect of message trans-mission diversity on status age,” in
IEEE International Symposium onInformation Theory , 2014, pp. 2411–2415.[4] C. Kam, S. Kompella, G. D. Nguyen, and A. Ephremides, “Effect ofmessage transmission path diversity on status age,”
IEEE Transactionson Information Theory , vol. 62, no. 3, pp. 1360–1374, 2015.[5] S. K. Kaul and R. D. Yates, “Timely updates by multiple sources: Them/m/1 queue revisited,” in
IEEE 54th Annual Conference on InformationSciences and Systems (CISS) , 2020, pp. 1–6.[6] M. Moltafet, M. Leinonen, and M. Codreanu, “On the age of informationin multi-source queueing models,”
IEEE Transactions on Communica-tions, Early Access , 2020.[7] A. Kosta, N. Pappas, V. Angelakis et al. , “Age of information: A newconcept, metric, and tool,”
Foundations and Trends® in Networking ,vol. 12, no. 3, pp. 162–259, 2017.[8] A. Maatouk, M. Assaad, and A. Ephremides, “The age of updates in asimple relay network,” in
Proc. of IEEE Information Theory Workshop(ITW) , 2018.[9] Y.Sun, E.Uysal-Biyikoglu, and S.Kompella, “Age-optimal updates ofmultiple information flows,” arXiv preprint, arXiv:1801.02394 , 2018.[10] R. Talak, S. Karaman, and E. Modiano, “Can determinacy minimize ageof information?” arXiv preprint arXiv:1810.04371 , 2018.[11] Y. Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff,“Update or wait: How to keep your data fresh,”
IEEE Transactions onInformation Theory , vol. 63, no. 11, pp. 7492–7508, 2017.[12] A. Maatouk, S. Kriouile, M. Assaad, and A. Ephremides, “The age ofincorrect information: A new performance metric for status updates,”
To appear in IEEE/ACM Transactions on Networking, arXiv preprintarXiv:1907.06604 , 2020.[13] Y.-P. Hsu, E. Modiano, and L. Duan, “Scheduling algorithms for mini-mizing age of information in wireless broadcast networks with randomarrivals: The no-buffer case,” arXiv e-prints, p. arXiv:1712.07419 , 2017.[14] A. M. Bedewy, Y. Sun, S. Kompella, and N. B. Shroff, “Age-optimalsampling and transmission scheduling in multi-source systems,” arXive-prints, p. arXiv:1812.09463 , 2018.[15] I. Kadota, A. Sinha, E. Uysal-Biyikoglu, R. Singh, and E. Modiano,“Scheduling policies for minimizing age of information in broadcastwireless networks,”
IEEE/ACM Transactions on Networking , vol. 26,no. 6, pp. 2637–2650, 2018.[16] A. Maatouk, M. Assaad, and A. Ephremides, “Minimizing the age ofinformation: NOMA or OMA?” in
Proc. of Infocom AoI Workshop 2019,arXiv e-prints, p. arXiv:1901.03020 , 2019.[17] ——, “Minimizing the age of information in a CSMA environment,” in
Proc. of IEEE WiOPT, arXiv e-prints, p. arXiv:1901.00481 , 2019.[18] ——, “On the age of information in a CSMA environment,”
To appearin IEEE Transactions on Networking , 2020.[19] A. M. Bedewy, Y. Sun, and N. B. Shroff, “Optimizing data freshness,throughput, and delay in multi-server information-update systems,” in
IEEE International Symposium on Information Theory (ISIT) , 2016, pp.2569–2574.[20] Y. Sun, E. Uysal-Biyikoglu, and S. Kompella, “Age-optimal updatesof multiple information flows,” in
IEEE Conference on ComputerCommunications (INFOCOM) Workshops , April 2018, pp. 2281–2285.[21] R. D. Yates, “Status updates through networks of parallel servers,” in
IEEE International Symposium on Information Theory (ISIT) , June 2018,pp. 2281–2285.
00 21
Fig. 22: The SHS Markov Chain for a system formed by K parallel M/M/1/(N+1)* queues with multiple sources. [22] A. M. Bedewy, Y. Sun, and N. B. Shroff, “The age of information inmultihop networks,” IEEE/ACM Transactions on Networking , vol. 27,no. 3, pp. 1248–1257, 2019.[23] ——, “Age-optimal information updates in multihop networks,” in
IEEEInternational Symposium on Information Theory (ISIT) , 2017, pp. 576–580.[24] S. Farazi, A. G. Klein, , and D. R. Brown, “Fundamental bounds on theage of information in general multi-hop interference networks,” in
IEEEConference on Computer Communications (INFOCOM) Workshops ,April 2019, pp. 96–101.[25] A. S. B. Buyukates and S. Ulukus, “Age of information in multihopmulticast networks,”
Journal of Communications and Networks , vol. 21,no. 3, pp. 256–267, 2019.[26] R. D. Yates, Y. Sun, D. R. B. III, S. K. Kaul, E. Modiano,and S. Ulukus, “Age of information: An introduction and survey,” https://arxiv.org/pdf/2007.08564.pdf , 2020.[27] R. D. Yates and S. K. Kaul, “The age of information: Real-timestatus updating by multiple sources,”
IEEE Transactions on InformationTheory , vol. 65, no. 3, pp. 1807–1827, 2019.[28] J. P. Hespanha, “Modelling and analysis of stochastic hybrid systems,”
IEE Proceedings-Control Theory and Applications , vol. 153, no. 5, pp.520–535, 2006.[29] S. Ishikawa, “Fixed points by a new iteration method,” in
Proc. Amer.Math. Soc. , vol. 44, 1974, pp. 147–150. A PPENDIX AP ROOF OF T HEOREM K parallel M/M/1/(N+1)*queues. Without loss of generality, we compute the averageAoI of source 1. The result is of course true for any source i . The arrival rate to queue j from source is λ p j andthat from the rest of the sources is (cid:80) k> λ k p kj for all j = 1 , . . . , K . Let M = N + 1 . The continuous state is givenby a vector of size K ∗ M x ( t ) = [ x ( t ) x ( t ) . . . x M ( t ) x ( t ) x K ( t ) . . .x KM ( t ) ] , where x represents the current age, x j the age if the updatein service in the queue j is delivered and x jl the age if theupdate in the position l − of the queue j is delivered. Thediscrete state is a Markov Chain with a single state. We notethat, when an event (arrival or departure) occurs in queue j , the age of the updates in the rest of the queues is notmodified. This allows us to focus on a queue j to illustrate theMarkov Chain and the SHS transitions, which are presentedrespectively in Figure 22 and Table VI.We now explain each transition l : l = 0 There is an update of source i that arrives to queue j . For this case, the incoming update replaces theupdate in the last position of the queue and the ageof the incoming update is set to zero, i.e., x (cid:48) jM = 0 . l = 1 There is an update of another source that arrives toqueue j . For this case, the incoming update replacesthe update in the last position of the queue and the OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 18 l λ l x x (cid:48) = xA l ¯ v q l A l λ p j [ x . . . x j . . . x jN x jM . . . ] [ x . . . x j . . . x jN . . . ] [ v . . . v j . . . v jN . . . ] (cid:80) k> λ k p kj [ x . . . x j . . . x jN x jM . . . ] [ x . . . x j . . . x jN x jN . . . ] [ v . . . v j . . . v jN v jN . . . ] µ j [ x . . . x j . . . x jN x jM . . . ] [ x j . . . x j . . . x jM x jM . . . ] [ v . . . v j . . . v jM v jM . . . ] TABLE VI: Table of SHS transitions of Figure 22.age of the incoming update is set to the same valueas the age of penultimate update, , i.e., x (cid:48) jM = x jN . Remark 3. If N = 0 , the last update in the queueis an update that is in service. Therefore, when anupdate of other sources arrives to the system, theincoming update replaces the update in service and x (cid:48) j = x . l = 2 The update in service in queue j is delivered andtherefore the age of the monitor changes to x j .Besides, all the elements in the queue move a po-sition ahead, which causes that their ages changerespectively from x j to x j , from x j to x j , . . . and from x jN to x jM . Finally, in the last position ofthe queue, we put a fake update whose age value isset to x jM , that is, the age of the penultimate elementin the queue, i.e., x (cid:48) jM = x jM .As we have just mentioned, when an update of queue i is delivered to the monitor, a fake update is put in the lastposition of the queue. We now explain that these additionalfake updates lead to a larger sojourn time of the incomingupdates, which implies that the overall average AoI is largerand, as a result, the result we obtain provides an upper boundof the real average AoI of the system (i.e. without fakeupdates). Consider that all the updates of queue i are deliveredbefore the arrival of a new update to that queue. According tothe above explained system, the queue is full of fake updatesand, therefore, when a new update arrives to the system, itis enqueued in the last position of the queue. However, ina system without fake updates, a new update would find thequeue empty and would start the service upon arrival. As aresult, we have that the sojourn time of new updates in asystem with fake updates (i.e., the above presented system) isclearly larger than the sojourn time without fake updates.Since the Markov Chain is formed by a single state, thestationary distribution is trivial. We define the vector v =[ v v v v M v . . . v K . . . v KM ] and also b as thevector of size K ∗ M with all ones. From the result ofTheorem 4 in [27] and the above reasoning, we know thatan upper bound of the AoI is given by v , that is, the firstcoordinate of the vector v .In the remainder of the proof, we present the system ofequations that v satisfies and solve it. We first present theequation of the first coordinate of v : K (cid:88) j =1 ( λ p j + (cid:88) k> λ k p kj + µ j ) v =1 + K (cid:88) j =1 (cid:32) ( λ p j + (cid:88) k> λ k p kj ) v + µ j v j (cid:33) , which can be alternatively written as v K (cid:88) j =1 µ j = 1 + K (cid:88) j =1 µ j v j . (36)Let l (cid:48) = l + 1 . We now present that, for all j = 1 , . . . , K and all l = 1 , . . . , M , the following equation is satisfied: K (cid:88) m =1 ( λ p m + (cid:88) k> λ k p km + µ m ) v ml = 1+ (cid:88) m (cid:54) = j ( λ p m + (cid:88) k> λ k p km + µ m ) v ml +( λ p j + (cid:88) k> λ k p kj ) v jl + µ j v ml (cid:48) ⇐⇒ µ j v jl = 1 + µ j v jl (cid:48) (37)Using recursively the last expression for l equals to N ,we get that µ j v j = N − µ j v mN . (38)We now focus on the last position of queue j and theequation that it must satisfy is the following: K (cid:88) m =1 ( λ p m + (cid:88) k> λ k p km + µ m ) v mM = 1+ (cid:88) m (cid:54) = j ( λ p m + (cid:88) k> λ k p km + µ m ) v mM + (cid:88) k> λ k p kj v jN + µ j v jM ⇐⇒ ( λ p j + (cid:88) k> λ k p kj ) v jM = 1 + (cid:88) k> λ k p kj v jN Besides, from (37), for l=N, we have that µ j v jN = 1 + µ j v jM . Using the last two expressions, we get that µ j v jN = 1 + µ j v jM = 1 + µ j (cid:80) k> λ k p kj v jN λ p j + (cid:80) k> λ k p kj . The last expression is equivalent to the following one: µ j v jN (cid:18) − λ p j λ p j + (cid:80) k> λ k p kj (cid:19) = 1+ µ j λ p j + (cid:80) k> λ k p kj ⇐⇒ µ j v jN (cid:18) λ p j λ p j + (cid:80) k> λ k p kj (cid:19) = λ p j + (cid:80) k> λ k p kj + µ j λ p j + (cid:80) k> λ k p kj ⇐⇒ µ j v jN λ p j = λ p j + (cid:88) k> λ k p kj + µ j ⇐⇒ µ j v jN = 1 + (cid:80) k> λ k p kj + µ j λ p j . Using the last expression with (38) and (36), the desiredresult follows for i = 1 . OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 19 -1 A v e r age A o I M/M/1/1 queue, =2 simulationSHS Fig. 23: Simulation results of the M/M/1/1 queue with differentvalues of arrival rates; x-axis in logarithmic scale. -1 A v e r age A o I M/M/1/3 queue, =2 SHSsimulation
Fig. 24: Simulation results of the M/M/1/3* queue withdifferent values of arrival rates; x-axis in logarithmic scale.A
PPENDIX BS IMULATIONS FOR
M/M/1/1
AND
M/M/1/3*
QUEUES
We provide in this section a comparison between the resultsobtained by SHS and those obtained by simulation to show theaccuracy of the SHS method. We consider two different exam-ples with either M/M/1/1 or M/M/1/3* queues. We consider asystem with a single source and without packet losses. We set µ = 2 . As expected, the curves presented in Figures 23 and24 show that, in all the considered cases, the results obtainedby simulation coincide with those of SHS.A PPENDIX CT ABLE OF TRANSITIONS OF F IGURE OURNAL OF COMMUNICATIONS AND NETWORKS, VOL X, NO. X, DATE 20 l q l → q l (cid:48) λ l x (cid:48) = xA l ¯ v q l A l → λ p [ x v (0) 0 0 0 0] (cid:80) k> λ k p k [ x x
0] [ v (0) 0 0 v (0) 0] → µ [ x v (3) 0 0 0 0] θ [ x v (0) 0 0 0 0] → λ p [ x x
0] [ v (0) 0 0 0 0] (cid:80) k> λ k p k [ x x x ] [ v (0) 0 0 v (0) 0] → µ [ x x
0] [ v (3) 0 0 v (4) 0] θ [ x x
0] [ v (0) 0 0 v (4) 0] → λ p [ x x
0] [ v (0) 0 0 v (3) 0] (cid:80) k> λ k p k [ x x x ] [ v (0) 0 0 v (3) v (3)] → λ p [ x v (0) 0 0 0 0] (cid:80) k> λ k p k [ x x v (0) v (0) 0 0 0 0] → µ [ x v (1) 0 0 0 0] θ [ x v (0) 0 0 0 0] → λ p [ x x
0] [ v (0) 0 0 v (3) 0] (cid:80) k> λ k p k [ x x x
0] [ v (0) v (0) 0 0 v (3) 0] → µ [ x x
0] [ v (1) 0 0 v (3) 0] θ [ x x
0] [ v (0) 0 0 v (3) 0] → λ p [ x x x ] [ v (0) 0 0 v (3) v (4)] (cid:80) k> λ k p k [ x x x x ] [ v (0) v (0) 0 0 v (3) v (4)] → µ [ x x x ] [ v (1) 0 0 v (3) v (4)] θ [ x x x ] [ v (0) 0 0 v (3) v (4)] → λ p [ x x v (0) v (1) 0 0 0] (cid:80) k> λ k p k [ x x x
0] [ v (0) v (1) 0 v (0) 0] → µ [ x x v (3) v (1) 0 0 0] θ [ x x v (0) v (1) 0 0 0] → λ p [ x x x
0] [ v (0) v (1) 0 v (3) 0] (cid:80) k> λ k p k [ x x x x ] [ v (0) v (1) 0 v (3) v (3)] → µ [ x x
0] [ v (3) 0 0 v (4) 0] θ [ x x
0] [ v (0) 0 0 v (4) 0] → λ p [ x x x
0] [ v (0) v (1) 0 v (3) 0] (cid:80) k> λ k p k [ x x x x ] [ v (0) v (1) 0 v (3) v (3)] → λ p [ x x v (0) x (1) 0 0 0] (cid:80) k> λ k p k [ x x x v (0) v (1) v (1) 0 0 0] → µ [ x x v (1) v (2) 0 0 0] θ [ x x v (0) x (2) 0 0 0] → λ p [ x x x
0] [ v (0) v (1) 0 v (3) 0] (cid:80) k> λ k p k [ x x x x
0] [ v (0) v (1) v (1) v (3) 0] → µ [ x x x
0] [ v (1) v (2) 0 v (3) 0] θ [ x x x
0] [ v (0) v (2) 0 v (3) 0] → λ p [ x x x x ] [ v (0) v (1) 0 v (3) v (4)] (cid:80) k> λ k p k [ x x x x ] [ v (0) v (1) v (1) v (3) v (4)] → µ [ x x x x ] [ v (1) v (2) 0 v (3) v (4)] θ [ x x x ] [ v (0) v (2) 0 v (3) v (4)] → λ p [ x x v (0) v (1) 0 0 0] (cid:80) k> λ k p k [ x x x x
0] [ v (0) v (1) v (1) 0 0 0] → λ p [ x x x v (0) v (1) v (2) 0 0] (cid:80) k> λ k p k [ x x x x
0] [ v (0) v (1) v (2) v (0) 0] → µ [ x x x v (3) v (1) v (2) 0 0] θ [ x x x v (0) v (1) v (2) 0 0] → λ p [ x x x
0] [ v (0) v (1) 0 v (3) 0] (cid:80) k> λ k p k [ x x x x
0] [ v (0) v (1) v (1) v (3) 0] → λ p [ x x x x
0] [ v (0) v (1) v (2) x (3) 0] (cid:80) k> λ k p k [ x x x x x ] [ v (0) v (1) v (2) v (3) v (3)] → µ [ x x x x
0] [ v (3) v (1) v (2) x (4) 0] θ [ x x x x
0] [ v (0) v (1) v (2) v (4) 0] → λ p [ x x x x ] [ v (0) v (1) 0 v (3) v (4)] (cid:80) k> λ k p k [ x x x x x ] [ v (0) v (1) v (1) v (3) v (4)] → λ p [ x x x x
0] [ v (0) v (1) v (2) v (3) 0] (cid:80) k> λ k p k [ x x x x x ] [ v (0) v (1) v (2) v (3) v (3)](3)]