aa r X i v : . [ c s . PF ] N ov Pass-and-swap queues ∗ C´eline Comte and Jan-Pieter Dorsman † Eindhoven University of Technology, The Netherlands University of Amsterdam, The NetherlandsNovember 25, 2020
Abstract
Order-independent (OI) queues, introduced by Berezner, Kriel, and Krzesinski in 1995, expanded thefamily of multi-class queues that are known to have a product-form stationary distribution by allowingfor intricate class-dependent service rates. This paper further broadens this family by introducing pass-and-swap (P&S) queues, an extension of OI queues where, upon a service completion, the customer thatcompletes service is not necessarily the one that leaves the system. More precisely, we supplement theOI queue model with an undirected graph on the customer classes, which we call a swapping graph, suchthat there is an edge between two classes if customers of these classes can be swapped with one another.When a customer completes service, it passes over customers in the remainder of the queue until it findsa customer it can swap position with, that is, a customer whose class is a neighbor in the graph. In itsturn, the customer that is ejected from its position takes the position of the next customer it can beswapped with, and so on. This is repeated until a customer cannot find another customer to be swappedwith anymore; this customer is the one that leaves the queue. After proving that P&S queues have aproduct-form stationary distribution, we derive a necessary and sufficient stability condition for (opennetworks of) P&S queues that also applies to OI queues. We then study irreducibility properties of closednetworks of P&S queues and derive the corresponding product-form stationary distribution. Lastly, wedemonstrate that closed networks of P&S queues can be applied to describe the dynamics of new andexisting load-distribution and scheduling algorithms in machine pools.
Keywords:
Order-independent queue, product-form stationary distribution, network of queues, quasi-reversibility, first-come-first-served, assign-to-the-longest-idle-server.
Since the pioneering work of Jackson [16] in the 1950s, queueing networks with a product-form stationarydistribution have played a central role in the development of queueing theory [24, 10]. In general, thestationary distribution of a network is said to have a product form if it can be written as the product of thestationary distributions of the queues that compose this network. Further examination of queueing networkswith this property led to several breakthroughs, such as the discovery of BCMP [6] and Kelly [18] networks,which demonstrated the broad applicability of these models. In addition to implying statistical independencebetween queues, this product-form property is appealing for its potential for further performance analysis.In a product-form queueing network, the notion of product form is also relevant on the level of anindividual queue in the sense that, aside from the normalization constant, the stationary distribution ofeach queue is a product of factors, each of which corresponds to a customer in the queue [25]. Dedicatedstudy of this type of product form has gained momentum recently, mainly because of the rising interest in ∗ Preprint submitted to
Queueing Systems: Theory and Applications . † Corresponding author: Jan-Pieter Dorsman ([email protected]). swapping graph on customer classes, such that there is an edge between two classes ifcustomers of these classes can be swapped with one another. Whenever a customer completes service, it scansthe remainder of the queue, passes over subsequent customers that are of a non-swappable class, and swapsroles with the first customer of a swappable class, in the sense that it takes the place of this customer, tostart another round of service. The ejected customer, in turn, scans the rest of the queue, possibly swappingwith yet another customer, and so on. This repeats until a customer cannot find a customer to be swappedwith anymore. This is the customer that leaves the queue.P&S queues have several applications. As an example, we show that they can be used to describe thedynamics of load-distribution and scheduling algorithms in machine pools. To illustrate the diversity ofalgorithms that can be modeled by P&S queues, let us again briefly consider the compatibility graph ofFigure 1. There are two customer classes and three servers. Class 1 is compatible with servers 1 and 3,while class 2 is compatible with servers 2 and 3. These compatibility constraints produce different product-form queueing models depending on their interpretation. In the first variant, each incoming customer isimmediately added to the queues of all its compatible servers and each server applies first-come-first-served(FCFS) service discipline to its assigned customers. A customer may therefore be in service on several serversat a time, in which case its departure rate is the sum of the service rates of these servers. Such compatibilityconstraints may typically result from data locality. If the customers of each class arrive according to anindependent Poisson process and have independent and exponentially distributed service requirements withunit mean, the corresponding queueing model is an OI queue. In a second variant of the model, eachincoming customer is assigned to its compatible server that has been idle the longest, if any, otherwise thejob is kept waiting in a central queue until one of its compatible servers becomes idle. This algorithm iscalled first-come-first-served and assign-to-the-longest-idle-server (FCFS-ALIS). With similar assumptions2n the arrival process and job size distribution as before, it was shown in [3] that, although the correspondingqueueing model is not an OI queue, we again obtain a product-form stationary distribution that is similar tothat obtained under the first variant. These two variants can be interpreted as different forms of redundancyscheduling called cancell-on-completion and cancel-on-start [5], respectively, and we will see that both canbe modeled thanks to P&S queues. Furthermore, the framework of P&S queues allows for extensions indifferent directions that, as far as the authors are aware, were not analyzed in the literature. Going back tothe toy example of Figure 1, it may happen that incoming customers are a priori compatible with all servers,but that they cannot be processed by more than two servers at the same time due to limited parallelism.In this case, we can apply a hybrid algorithm in which each incoming customer is assigned to two serversand may subsequently be processed in parallel on one or both servers. We also consider another extension,called hierarchical load distribution, where incoming customers are successively assigned to several waitinglines before being eventually assigned to one or more servers. Both extensions fall within the framework ofP&S queues, and we will actually see that the P&S mechanism translates into a load-distribution algorithmthat leads to a product-form stationary distribution.Our contributions are as follows. We introduce P&S queues and establish that, although these queuesare a non-trivial generalization of OI queues, the product form of the stationary distribution is preserved;this result is proved by careful inspection of the partial balance equations of the underlying Markov chain.This result paves the way for the performance analysis of several applications, such as those we describebelow, without resorting to scaling regimes. We also provide an easily verifiable necessary and sufficientstability condition for P&S queues that also holds for OI queues. Next to this, we study networks of P&Squeues. By establishing that P&S queues are quasi-reversible [23, 17], we show that open networks of P&Squeues exhibit a product-form stationary distribution under mild conditions on the routing process. We alsostudy irreducibility properties of closed networks of P&S queues and demonstrate that, under particularassumptions, the stationary distribution of such closed networks also has a product form. These closednetworks form a class of independent interest, as we show later that they can be used to model finite-capacity queues with token-based structures, akin to those of [4] and [12, 13].The remainder of the paper is organized as follows. Section 2 recalls results on OI queues that werederived in [7, 19]. The P&S queue is introduced in Section 3, where the product form of its stationarydistribution is also established. After deriving complementary results on open (networks of) P&S queuesin Section 4, we turn to the analysis of closed networks of P&S queues in Section 5. We demonstrate theapplicability of these models to the modeling of resource-management algorithms in Section 6. Section 7concludes the paper.
This section gives an overview of OI queues, introduced in [7] and later studied in [19]. The results of thissection were derived in these two seminal papers and act as a basis for extension in the remainder of thepaper.
We consider a multi-class queue with a finite set I = { , . . . , I } of customer classes. For each i ∈ I , class- i customers enter the queue according to an independent Poisson process with intensity λ i >
0. Customersare queued in their arrival order, with the oldest customer at the head of the queue, and are identified bytheir class. For now, we assume that the queue has an infinite capacity and that each customer leaves thequeue immediately upon service completion.
Microstate and macrostate
We consider two state descriptors of this multi-class queue. The queue microstate represents the classes of customers in the queue in their arrival order. More specifically, amicrostate c ∈ I ∗ has the form c = ( c , . . . , c n ), where n is the total number of customers in the queueand c p is the class of the p -th oldest customer, for each p ∈ { , . . . , n } . In particular, c is the class of3he oldest customer, at the head of the queue. The empty microstate, with n = 0, is denoted by ∅ . Theset I ∗ represents all possible microstates and I n represents the subset of I ∗ consisting of all microstateswith exactly n customers present, for each n ∈ N . To each microstate c ∈ I ∗ , we associate a macrostate | c | = ( | c | , . . . , | c | I ) ∈ N I that only retains the number of present customers of each class, and does not keeptrack of their order in the queue. As a result, for each c ∈ I ∗ and each i ∈ I , the integer | c | i gives thenumber of class- i customers in microstate c . For each x, y ∈ N I , we write x ≤ y if x i ≤ y i for each i ∈ I . Service rates
We now explain the way in which service is provided to customers in an OI queue. Thisis done in such a way that the evolution of the microstate of the queue over time exhibits a memorylessproperty (and thus represents a Markov process). More particularly, the overall service rate in microstate c is denoted by µ ( c ), for each c ∈ I ∗ . This function µ , defined on I ∗ , is called the rate function of the queue.Along with the individual rates of service provided to the customers in the queue, it satisfies the followingtwo conditions. First, the overall rate of service µ ( c ) provided when the queue is in microstate c dependsonly on the number of customers of each class that are present and not on their arrival order. In otherwords, for each c, d ∈ I ∗ , we have µ ( c ) = µ ( d ) whenever | c | = | d | . For this reason, we shall also refer to µ ( c )as µ ( x ) when x is the macrostate corresponding to microstate c . Second, the service rate of each customeris independent of (the number and classes of the) customers that are behind this customer in the queue. Inparticular, for each c = ( c , . . . , c n ) ∈ I ∗ and p ∈ { , . . . , n } , the service rate of the customer in position p inmicrostate c , of class c p , is equal to the increment of the overall service rate induced by the arrival of thiscustomer, denoted by ∆ µ ( c , . . . , c p ) = µ ( c , . . . , c p ) − µ ( c , . . . , c p − ) , where we use the convention that ( c , . . . , c p − ) = ∅ when p = 0. This implies in particular that the function µ is non-decreasing, in the sense that µ ( c , . . . , c n , i ) ≥ µ ( c , . . . , c n ) , ∀ c = ( c , . . . , c n ) ∈ I ∗ , ∀ i ∈ I . The service rate of the first p customers in the queue, given by µ ( c , . . . , c p ) = P pq =1 ∆ µ ( c , . . . , c q ), dependsneither on the classes of the customers in positions p + 1 to n nor even on the total number n of customersin the queue.We set µ ( ∅ ) = 0 since the queue exhibits a zero departure rate when there are no customers in the queue.We additionally assume that µ ( c ) > c = ∅ . In other words, we assume that the oldest customeralways receives a positive service rate, to ensure irreducibility of the Markov process describing the evolutionof the microstate over time. Remark . The definition of OI queues that we presented above is slightly more restrictive than that of[7, 19]. Indeed, in these two papers, the overall service rate function µ is scaled by a factor that depends onthe total number of customers in the queue, while we assume this scaling factor to be equal to one. We omitthis scaling factor for simplicity of notation. However, unless stated otherwise, the results in the sequel ofthis paper can be straightforwardly generalized to account for this factor. Examples
As observed in [7, 19], the framework of OI queues encompasses several classical queueingmodels, such as the FCFS, infinite-server, and processor-sharing queues of BCMP networks [6], multi-serverstations with concurrent classes of customers (MSCCC) [21], and multi-server centers with hierarchicalconcurrency constraints (MSHCC) [20]. In this paper, we will be especially interested in the followingmulti-server queues, introduced in [15] and identified as OI queues in [9].
Example . Consider an infinite-capacity queue with a set I = { , . . . , I } of customerclasses and a set S = { , . . . , S } of servers. All customers have an exponentially-distributed size with unitmean and, for each i ∈ I , class- i customers enter the queue according to a Poisson process with rate λ i > S i ⊆ S . This defines a bipartite compatibility graph betweencustomer classes and servers, in which there is an edge between a class and a server if this server can processcustomers of this class. In the example of Figure 1, customers of classes 1 and 2 can only be processed by4 1 3 2 2 3 3 1 µ µ c = (1 , , , , , , , λ λ Figure 2: A queue state. The color of a server is a visual aid that indicates the class of the customer currentlyin service on this server.servers 1 and 2, respectively, while class-3 customers can be served by both servers. In other words, we have S = { } , S = { } , and S = { , } .Each server applies the FCFS discipline to the customers it can serve, so that each customer is in serviceon all the servers that can process this customer but not the customers that arrived earlier in the queue.For each s ∈ S , the service rate of server s is denoted by µ s >
0. When a class- i customer is in service ona subset T ⊆ S i of its compatible servers, its service rate is P s ∈T µ s . The overall service rate, equal to thesum of the service rates of the servers that can process at least one customer in the queue, is given by µ ( c , . . . , c n ) = X s ∈ S np =1 S cp µ s , ∀ ( c , . . . , c n ) ∈ C . (2.1)For each p ∈ { , . . . , n } , the service rate of the customer in position p is given by∆ µ ( c , . . . , c p ) = µ ( c , . . . , c p ) − µ ( c , . . . , c p − ) = X s ∈S cp \ S p − q =1 S cq µ s . (2.2)One can verify that this multi-server queue is an OI queue. In the example of Figure 2, the queue microstateis c = (1 , , , , , , , µ (1) = µ (1) − µ ( ∅ ) = µ − µ . The second oldest customer, of class 3, is in service on server 2, at rate∆ µ (1 ,
3) = µ (1 , − µ (1) = ( µ + µ ) − µ = µ . The other customers have a zero service rate. If the oldestclass-1 customer completes service, the oldest class-3 customer will be in service on both servers, at rate∆ µ (3) = µ (3) − µ ( ∅ ) = ( µ + µ ) − µ + µ , while the other customers will still have a zero service rate. Example . The following example will be useful in Section 6. Considera multi-server queue in which there is a one-to-one correspondence between classes and servers, in the sensethat I = S (so that I = S ) and S i = { i } for each i ∈ I . An example with I = S = 2 is shown inFigure 3. For each s ∈ S , server s applies FCFS service policy to the customers of class s , while ignoringthe others. Consequently, with s = c p , (2.2) simplifies to ∆ µ ( c , . . . , c p ) = µ s if | ( c , . . . , c p − ) | s = 0 and∆ µ ( c , . . . , c p ) = 0 otherwise. This queue is said to be degenerate because the service rate of one class doesnot depend on the numbers of customers of other classes in the queue.1 21 2 (a) Compatibility graph. µ µ c = (1 , , , , , , , λ λ (b) A queue state. Figure 3: A degenerate multi-server queue with I = S = 2 customer classes and their associated servers. The evolution of the queue microstate leads to a Markov process with state space I ∗ , and this Markovprocess is irreducible. Indeed, for any two microstates c = ( c , . . . , c n ) ∈ I ∗ and d = ( d , . . . , d m ) ∈ I ∗ , the5arkov process can first jump from state c to state ∅ as a result of n transitions corresponding to departuresof the customer at the head of the queue, and then from state ∅ to state d as a result of m transitionscorresponding to customer arrivals.Theorem 2.4 below recalls that this Markov process has a product-form stationary distribution and thatthe OI queue satisfies the quasi-reversibility property (cf. [17, Section 3.2]). This property implies that,when the Markov process associated with the queue microstate is stationary, the departure instants of thecustomers of each class form independent and stationary Poisson processes and that, at every instant, thecurrent queue microstate is independent of the departure instants of the customers of each class prior tothat instant. Quasi-reversibility also implies that an open network of OI queues connected by a Markovianrouting policy has a product-form stationary distribution [17, Theorem 3.7] under mild conditions on therouting process (namely, each customer can become part of any given class with a positive probability andeventually leaves the network with probability one). A similar result holds for closed networks of OI queuesunder some irreducibility assumptions [17, Section 3.4]. The interested reader is referred to [17, Sections3.2 and 3.4] and [24, Chapter 8] for a more complete account on quasi-reversibility. The proof below can befound in [7, 19] but we present it here for ease of later reference when we introduce the P&S queue. Theorem 2.4.
Consider an OI queue with a set I = { , . . . , I } of customer classes, per-class arrival rates λ , . . . , λ I , and a rate function µ . A stationary measure of the Markov process associated with the microstateof this OI queue is of the form π ( c , . . . , c n ) = π ( ∅ )Φ( c ) Y i ∈I λ i | c | i , ∀ ( c , . . . , c n ) ∈ I ∗ , (2.3) where Φ is the balance function of the OI queue, defined on I ∗ by Φ( c , . . . , c n ) = n Y p =1 µ ( c , . . . , c p ) , ∀ ( c , . . . , c n ) ∈ I ∗ , (2.4) and π ( ∅ ) is an arbitrary positive constant. The queue is stable if and only if X c ∈I ∗ Φ( c ) Y i ∈I λ i | c | i < + ∞ , (2.5) in which case the queue is quasi-reversible and the stationary distribution of the Markov process associatedwith its microstate is given by (2.3) with π ( ∅ ) = X c ∈I ∗ Φ( c ) Y i ∈I λ i | c | i ! − . (2.6) Proof.
We will first verify that any measure π of the form (2.3) satisfies the following partial balance equationsin each microstate c = ( c , . . . , c n ) ∈ I ∗ : • Equalize the flow out of microstate c due to a departure with the flow into microstate c due to anarrival (if c = ∅ ): π ( c ) µ ( c ) = π ( c , . . . , c n − ) λ c n . (2.7) • Equalize, for each i ∈ I , the flow out of microstate c due to the arrival of a class- i customer with theflow into microstate c due to the departure of a customer of this class: π ( c ) λ i = n +1 X p =1 π ( c , . . . , c p − , i, c p , . . . , c n ) ∆ µ ( c , . . . , c p − , i ) . (2.8)6his verification will readily imply that the stationary measures of the Markov process associated with thequeue microstate are of the form (2.3) and that the queue, when stable, is quasi-reversible. Indeed, theglobal balance equations of the Markov process associated with the queue microstate follow from the partialbalance equations (2.7) and (2.8) by summation. Moreover, to prove that the queue is quasi-reversible, itsuffices to verify that the stationary measures of the Markov process associated with the queue microstatesatisfy the partial balance equations (2.8). This is a consequence of [17, Equations (3.8) to (3.11)] and of thefact that the customers of each class enter the queue according to an independent and stationary Poissonprocess.We now verify that the measures of the form (2.3) satisfy the partial balance equations (2.7) and (2.8).Equation (2.7) follows immediately from (2.3) and (2.4). The case of (2.8) is more intricate. First observethat the measures π of the form (2.3) satisfy (2.8) if and only if the balance function Φ given by (2.4) satisfiesthe following equation in each microstate c = ( c , . . . , c n ) ∈ I ∗ :Φ( c ) = n +1 X p =1 Φ( c , . . . , c p − , i, c p , . . . , c n ) ∆ µ ( c , . . . , c p − , i ) , ∀ i ∈ I . (2.9)We show that Φ satisfies this equation by induction over the queue length n . For the base step, with n = 0,it suffices to observe that, for each i ∈ I , the right-hand side of (2.9) simplifies to Φ( i ) µ ( i ), which is equalto Φ( ∅ ) by (2.4). Now let n ≥ c ∈ I n − . Let c ∈ I n and i ∈ I .The first n terms in the sum on the right-hand side of (2.9) can be rewritten as follows: n X p =1 Φ( c , . . . , c p − , i, c p , . . . , c n ) ∆ µ ( c , . . . , c p − , i )= 1 µ ( c , . . . , c n , i ) n X p =1 Φ( c , . . . , c p − , i, c p , . . . , c n − ) ∆ µ ( c , . . . , c p − , i ) , = 1 µ ( c , . . . , c n , i ) Φ( c , . . . , c n − ) , = Φ( c , . . . , c n ) µ ( c , . . . , c n ) µ ( c , . . . , c n , i ) , (2.10)where the first and last equalities follow from (2.4) and the order independence of µ , while the second equalityis obtained by applying the induction assumption to microstate ( c , . . . , c n − ) and class i . By (2.4), we alsohave that Φ( c , . . . , c n , i ) ∆ µ ( c , . . . , c n , i ) = Φ( c , . . . , c n ) µ ( c , . . . , c n , i ) − µ ( c , . . . , c n ) µ ( c , . . . , c n , i ) , (2.11)so that summing (2.10) and (2.11) yields (2.9). This concludes the proof by induction.The stability condition (2.5) is equivalent to the statement that P c ∈I ∗ π ( c ) /π ( ∅ ) is finite for any station-ary measure π , which is indeed necessary and sufficient for ergodicity. Equation (2.6) guarantees that thestationary distribution sums to unity. This section contains our first main contribution. Pass-and-swap (P&S) queues, obtained by supplementingOI queues with an additional mechanism when customers complete service, are defined in Section 3.1. Incontrast to signals and negative customers considered for quasi-reversible queues [11], the P&S mechanismoccurs upon a service completion and can move several customers at the same time within the queue. Sec-tion 3.2 shows that both the product-form nature of the stationary measure of the Markov process associatedwith the microstate and the quasi-reversibility property of the queue are preserved by this mechanism.7 .1 Definition
As before, the set of customer classes is denoted by I = { , . . . , I } and we adhere to the state descriptors:the microstate c = ( c , . . . , c n ) ∈ I ∗ gives the classes of customers as they are ordered in the queue and themacrostate | c | = ( | c | , . . . , | c | I ) ∈ N I gives the numbers of customers of each class. Likewise, the customerarrival processes and completion times are as defined in Section 2.1. We however part with the assumptionthat a customer that completes service leaves the queue directly and that all customers behind move forwardone position. With the pass-and-swap mechanism that we will now define, each service completion willpotentially trigger a chain reaction within the queue. More precisely, a customer that completes servicemay take another customer’s position further down the queue and require a new round of service in thisposition. The customer ejected from this position may in turn take the position of another customer furtherdown the queue, and so on. The decision of which customer replaces which other customer is driven by the pass-and-swap mechanism described below. Pass-and-swap mechanism
We supplement the OI queue with a simple graph, that is, an undirectedgraph without loops or multiple edges. This will be called the swapping graph of the queue. The vertices ofthis graph represent the customer classes and, for each i, j ∈ I , when there is an edge between class i andclass j in the graph, a class- i customer can take the position of a class- j customer upon service completion.In that case, we say that (the customers of) classes i and j are mutually swappable . Observe that the graphis undirected, meaning that the swapping relation is symmetric: if class i can be swapped with class j , thenclass j can also be swapped with class i . For each i ∈ I , we let I i ⊆ I denote the set of neighbors of class i in the graph, that is, the set of classes that are swappable with class i .Based on this graph, the pass-and-swap mechanism is defined as follows. A customer whose service iscomplete scans the rest of the queue, passes over subsequent customers that it cannot swap with, and replacesthe first swappable customer, if any. This ejected customer, in turn, scans the rest of the queue and replacesthe first swappable customer afterwards. This is repeated until an ejected customer finds no customers inthe remainder of the queue it can swap with. In this case, the customer leaves the queue.More formally, let c = ( c , . . . , c n ) ∈ I ∗ denote a queue microstate. Assume that the service of thecustomer in some position p ∈ { , . . . , n } completes and let i = c p denote the class of this customer. Ifthere is at least one position q ∈ { p + 1 , . . . , n } such that c q ∈ I i , we let p denote the smallest of thesepositions and i = c p the class of the corresponding customer. The class- i customer that was originally inposition p replaces the class- i customer in position p , and this class- i customer is ejected. If there is atleast one position q ∈ { p + 1 , . . . , n } such that c q ∈ I i , we let p denote the smallest of these positions and i = c p the class of the corresponding customer. The class- i customer that was originally in position p replaces the class- i customer in position p , and this class- i customer is ejected. Going on like this, wedefine recursively p v +1 = min { q ≥ p v + 1 : c q ∈ I c pv } for each v ∈ { , . . . , u − } , where p u ∈ { p, . . . , n } isthe position of the first ejected customer that cannot replace any other subsequent customer in the queue,that is, for which there is no q ∈ { p u + 1 , . . . , n } such that c q ∈ I c pu . This customer is the one that leavesthe queue. The integer u ∈ { , . . . , n − p + 1 } gives the total number of customers that are involved in thetransition, and the microstate reached after this transition is( c , . . . , c p − , c p +1 , . . . , c p − , i , c p +1 , . . . , c p − , i , c p +1 , . . . , c p u − , i u − , c p u +1 , . . . , c n ) . Observe that the transition is recorded as a departure of a class- i u customer and not as a departure of aclass- i customer in general. In the special case where u = 1, the customer that completed service could notreplace any subsequent customer, so that this customer left the queue. An OI queue supplemented with thepass-and-swap mechanism is called a pass-and-swap queue (P&S). The stochastic process keeping track ofthe microstate over time in the P&S queue has the Markov property, just like the OI queue. For both theOI and the P&S queue, however, the stochastic process that describes the macrostate over time does yieldno such Markov property in general. Example . We give a toy example that illustrates the P&S mechanism. More concrete applications willbe described in Section 6. Consider the multi-server queue of Example 2.2 supplemented with the swapping8raph shown in Figure 4a. Class 2 can be swapped with classes 1 and 3 but classes 1 and 3 cannot be swappedwith one another. Assume that the queue is in the state c = (1 , , , , , , ,
2) depicted in Figure 4b, andthat the customer in first position, of class 1, completes service. The corresponding chain reaction is depictedon the same figure by arrows. Class 1 can only be swapped with class 2 and the first subsequent class-2customer is in the fourth position. Therefore, the class-1 customer that completed service is passed along thequeue up until the fourth position and then swapped with the class-2 customer at this position. The ejectedclass-2 customer can be swapped with customers of classes 1 and 3. Therefore, this customer is passed alongthe queue up until the sixth position and swapped with the class-3 customer at this position. We repeatthis with the ejected class-3 customer, which replaces the last class-2 customer, which leaves the queue. Thetransition is recorded as a departure of a class-2 customer and the new queue state is d = (3 , , , , , , (a) Swapping graph. µ µ λ λ (b) A queue state and the transition that occurs upon servicecompletion of the first class-1 customer. Figure 4: Toy example of a P&S queue.
Additional comments
The P&S mechanism is applied instantaneously upon a service completion. Re-placements are performed from the front to the back of the queue, so that, when a customer is ejected fromsome position p ∈ { , . . . , n } , this customer never replaces a customer at a position q ∈ { , . . . , p − } , evenif c q ∈ I c p . Also note that customer classes now have a dual role: they determine not only the service ratereceived by each customer through the rate function µ , but also the chain reaction that happens upon eachservice completion through the swapping graph. One could also associate two classes with each customer,one that determines its service rate and another that determines its swapping relations. Finally, the originalOI queue, in which the customer that completes service is the one that leaves the queue, is obtained byapplying the P&S mechanism based on a swapping graph without edges. Therefore, an OI queue is also aP&S queue, so that the results that we will derive for P&S queues in the sequel also apply to OI queues.The following notation will be useful. We write δ p ( c ) = ( d, i ) if the service completion of the customerin position p in microstate c leads to microstate d and triggers the departure of a class- i customer, foreach n ≥ c ∈ I n , d ∈ I n − , p ∈ { , . . . , n } , and i ∈ I . In Example 3.1 for instance, we have δ ( c ) =((3 , , , , , , , δ p ( c ) = i if we only want to specify thatthe departing customer is of class i . The Markov process associated with the queue microstate on the state space I ∗ is irreducible. Indeed, givenany microstates c = ( c , . . . , c n ) ∈ I ∗ and d = ( d , . . . , d m ) ∈ I ∗ , the Markov process can again jump frommicrostate c to microstate ∅ thanks to n transitions corresponding to departures (for instance, triggered bythe service completion of the customer at the head of the queue), and then from microstate ∅ to microstate d thanks to m transitions corresponding to arrivals.Theorem 3.2 below shows that introducing the P&S mechanism does actually not change the stationarydistribution of this Markov process compared to the original OI queue. In particular, this stationary dis-tribution is independent of the swapping graph. Intuitively, this can be thought of as a consequence of thesymmetric property of the swapping relation. Another implication of Theorem 3.2 is that the consequencesof the quasi-reversibility property stated in Section 2.2 for OI queues also apply to P&S queues. As a result,an open network of P&S queues connected by a random routing process has a product-form stationary dis-tribution. The only peculiarity is that, when a customer completes service in a P&S queue, this customer is9ot necessarily the one that leaves this queue. The case of closed networks is more complicated and will beconsidered in Section 5. The sketch of the proof given below is completed in Appendix A.1. Theorem 3.2.
The results of Theorem 2.4 remain valid if we replace “OI queue” with “P&S queue”.Sketch of proof.
The proof resembles that of Theorem 2.4, except that the second set of partial balanceequations (2.8) has a different form. More specifically, we will verify that any measure π of the form (2.3)satisfies the following partial balance equations in each microstate c = ( c , . . . , c n ) ∈ I ∗ : • Equalize the flow out of microstate c due to a departure with the flow into microstate c due to anarrival (if c = ∅ ): π ( c ) µ ( c ) = π ( c , . . . , c n − ) λ c n . (3.1) • Equalize, for each i ∈ I , the flow out of microstate c due to the arrival of a class- i customer with theflow into microstate c due to the departure of a customer of this class: π ( c ) λ i = X d ∈I n +1 n +1 X p =1 δ p ( d )=( c,i ) π ( d ) ∆ µ ( d , . . . , d p ) . (3.2)Since (3.1) represents the same set of equations as (2.7), it is already known that (2.3) satisfies (3.1). Showingthat (2.3) satisfies (3.2) is equivalent to showing that the balance function Φ in (2.4) satisfies the followingequation in each microstate c = ( c , . . . , c n ) ∈ I ∗ :Φ( c ) = X d ∈I n +1 n +1 X p =1 δ p ( d )=( c,i ) Φ( d ) ∆ µ ( d , . . . , d p ) , ∀ i ∈ I . (3.3)This is shown by induction over the queue length n in Appendix A.1. The rest of the proof follows alongthe same lines as Theorem 2.4. Remark . In this section, we have seen that the introduction of the P&S mechanism to the OI queueinduces a different set of partial balance equations, while product-form properties are retained. Even stronger,Theorem 3.2 shows that replacing (2.8) with (3.2) does not alter the stationary distribution of the queueat all. This begs the question of whether there exist other intra-queue routing mechanisms than the P&Smechanism that also lead to this product-form stationary distribution. A partial answer can be found inthe actual proof of Theorem 3.2, given in Appendix A.1. A careful analysis of this proof suggests thatother routing mechanisms, resulting in a different completion order q , q , . . . , q u as defined in Appendix A.1,could also lead to the stationary measure given in Theorem 2.4, as long as this completion order adheresto an equation of the same form as (A.1). However, identifying such routing mechanisms does not seemstraightforward. We now use Theorem 3.2 to derive further results on the stationary behavior of P&S queues. Section 4.1 givesan alternative stability condition that is simpler to verify than (2.5). The result of this section extends thatobtained for OI queues in the Ph.D. thesis [13] and before that for multi-server queues in [9]. In Section 4.2,we use the quasi-reversibility property to prove that the average service and departures rates of each classare equal to each other and independent of the swapping graph. Recall that, since an OI queue is also aP&S queue, the results of this section readily apply to OI queues.10 .1 Stability condition
Theorem 4.1 below gives a necessary and sufficient condition for the stability of P&S queues. This conditionis simpler than (2.5) as it only compares the per-class arrival rates λ , . . . , λ I to the rate function µ . A firstversion of this theorem was stated in the Ph.D. thesis [13, Theorem 3.4] for OI queues. The proof that wegive in Appendix A.2 is different in that it does not involve Whittle networks [24].To state the stability condition, it is worth recalling that, for each c ∈ I ∗ , the macrostate associated withmicrostate c is the vector | c | = ( | c | , . . . , | c | I ) ∈ N I that counts the number of customers of each class presentin the queue. Also recall that the service rate µ ( c ) depends on the number of customers of each class thatare contained in microstate c but not on their order. This means that, once the macrostate corresponding toa microstate is given, the service rate function is not sensitive to the microstate itself anymore. Therefore,in the sequel, we also refer to µ ( c ) as µ ( | c | ) for simplicity of notation.For each i ∈ I , we let e i denote the I -dimensional vector with one in component i and zero elsewhere.We define the function ¯ µ on the power set of I by¯ µ ( A ) = lim m → + ∞ µ ( me A ) , ∀A ⊆ I , (4.1)where e A = P i ∈A e i for each A ⊆ I . The monotonicity of µ ensures that ¯ µ is well defined, with values in R + ∪ { + ∞} , and is itself a non-decreasing set function. If the overall service rate only depends on the setof active classes, as is the case in the multi-server queue of Examples 2.2 and 3.1, we have ¯ µ ( A ) = µ ( x ) foreach x ∈ N I such that A = { i ∈ I : x i > } , but in general, we may have ¯ µ ( A ) > µ ( x ) for each such x . Theorem 4.1.
Consider a P&S queue with a set I = { , . . . , I } of customer classes, per-class arrival rates λ , . . . , λ I , and a rate function µ . This P&S queue is stable if and only if X i ∈A λ i < ¯ µ ( A ) , ∀A ⊆ I : A 6 = ∅ . (4.2) Proof.
See Appendix A.2.This result is the only one in this paper that cannot be straightforwardly extended to P&S queues withan arbitrary scaling factor as considered in [7, 19]. However, it can be extended to P&S queues with anon-decreasing scaling factor by including this scaling rate into the definition of ¯ µ . We now consider the relation between the service rates and departure rates of customers in the P&S queue.Consider a stable P&S queue, as defined in Section 3.1, and let π denote the stationary distribution of theMarkov process tracking the microstate over time. For each c = ( c , . . . , c n ) ∈ I ∗ and i ∈ I , we also definethe overall departure rate of class i in microstate c to be φ di ( c ) = n X p =1 δ p ( c )= i ∆ µ ( c , . . . , c p ) , (4.3)while the overall service rate of class i in this microstate is defined as φ si ( c ) = n X p =1 c p = i ∆ µ ( c , . . . , c p ) . (4.4)While it is not necessarily true that φ di ( c ) = φ si ( c ), the next proposition states that, for each x ∈ N I and i ∈ I , the overall probability flow out of macrostate x due to a departure of a class- i customer is equal tothe overall probability flow out of macrostate x due to a service completion of a class- i customer. Since the11atter quantity does not depend on the swapping graph, this equality implies that, for each x ∈ N I and i ∈ I ,the overall probability flow out of macrostate x due to a departure of a class- i customer does not depend onthe swapping graph. Upon dividing (4.5) by P c ∈I ∗ : | c | = x π ( c ), we also obtain that the conditional expecteddeparture and service rates of a class given the macrostate are equal to each other. Proposition 4.2.
For each x ∈ N I and i ∈ I , we have X c ∈I ∗ : | c | = x π ( c ) φ di ( c ) = X c ∈I ∗ : | c | = x π ( c ) φ si ( c ) . (4.5) Proof. If x i = 0, the result is immediate since φ di ( c ) = φ si ( c ) = 0 for any microstate c for which | c | = x .For the case x i >
0, note that the stationary distribution of the P&S queue satisfies (3.2), but, becauseof Theorem 2.4, it also satisfies (2.8). Therefore, the right-hand sides of these two equations are equal.Equating these two sides and summing over all microstates c = ( c , . . . , c n ) for which | c | = x − e i , we obtain X c ∈I ∗ : | c | = x − e i X d ∈I ∗ : | d | = x n +1 X p =1 δ p ( d )=( c,i ) π ( d ) ∆ µ ( d , . . . , d p ) = X c ∈I ∗ : | c | = x − e i n +1 X p =1 π ( c , . . . , c p − , i, c p , . . . , c n )∆ µ ( c , . . . , c p − , i ) . Rewriting both sides leads to X c ∈I ∗ : | c | = x n X p =1 δ p ( c )= i π ( c ) ∆ µ ( c , . . . , c p ) = X c ∈I ∗ : | c | = x n X p =1 c p = i π ( c , . . . , c n )∆ µ ( c , . . . , c p ) . Combining this equation with (4.3) and (4.4) finalizes the proof.
Remark . The result of Proposition 4.2 is intuitively not very surprising. If, in a microstate c =( c , . . . , c n ), the completion of customer c p will trigger a departure of customer c q , then, in microstate d = ( c n , . . . , c ), the completion of customer c q will trigger a departure of customer c p , as the swappinggraph is undirected. Moreover, both microstates lead to the same macrostate x . The fact that π ( c ) = π ( d )is offset by the nature of the order-independent service rates, as formalized in the proof of Proposition 4.2. We saw that P&S queues are quasi-reversible, so that stable open networks of P&S queues have a product-form stationary distribution under mild conditions on the routing process. In this section, we consider closed networks of P&S queues in more detail and conclude that, also for closed networks, the stationary distributionhas a product form. In contrast to open networks, this does not follow directly from quasi-reversibility since,in general, the obtained Markov process does not meet the irreducibility assumptions posed in [17, Section3.4]. In Section 5.1, we first consider a closed P&S queue in which the number of customers of each class isfixed and departing customers are appended back to the end of the queue instead of leaving. These resultsare extended to a closed tandem network of two P&S queues in Section 5.2. This tandem network turns outto have rich applications, as we will see in Section 6.
We first consider a closed network that consists of a single P&S queue. In Section 5.1.1, we give an example ofsuch a closed P&S queue to illustrate its dynamics. Section 5.1.2 then gives a more formal description of thismodel, including necessary notation. This section also studies the structure of the Markov process underlyingthis closed P&S queue and establishes sufficient conditions for this Markov process to be irreducible. Providedthat the Markov process is indeed irreducible, Section 5.1.3 provides the stationary distribution of the closedP&S queue and establishes its product-form nature. 12 .1.1 Introductory example
Consider a closed P&S queue with six customer classes ( I = { , , . . . , } ) and the swapping graph shown inFigure 5a. When a class- i customer departs the queue, this customer does not leave the system. Instead, it isappended back to the queue as a class- i customer. For simplicity, we assume that there is a single customerof each class in the queue. More precisely, we assume that the queue starts in microstate c = (1 , , , , , (a) Swapping graph. (b) Initial state. (c) State reached after the service completion of customer 1. (d) State reached after the service completion of customer 2. Figure 5: A closed P&S queue. The rate function need not be specified, as throughout this example we onlyconsider the service completion of the customer at the head of the queue.It is worth noting that, in the three states shown in Figure 5, customer 1 precedes customers 3 and 4,which precede customer 6. This order will be conserved by the P&S mechanism, as the service completion ofcustomer 1 systematically triggers the movement of either customer 3 or customer 4, that in turn will replacecustomer 6. Similarly, customer 2 will always precede customers 4 and 5 that, in their turn, will alwaysprecede customer 6. In the sequel, we will formalize this phenomenon and characterize the communicatingclasses of the Markov process associated with the microstate of a closed P&S queue.
The closed P&S queue inherits virtually all properties and notation from Section 3.1. As mentioned before,the only difference is that, upon a service completion, the customer that would have left if the queue wereopen is, instead, appended back to the end of the queue as a customer of the same class. There is also noexternal arrival process, so that the macrostate of the queue is determined by its initial state and does notchange over time. We therefore let I = { , . . . , I } denote the set of classes of the customers in the initialstate of the queue. The (fixed) macrostate of the queue is denoted by ℓ = ( ℓ , . . . , ℓ I ) ∈ N I and the totalnumber of customers by n = ℓ + . . . + ℓ I . Note that ℓ i > i ∈ I , as we only regard classes ofcustomers present in the queue.In Section 5.1.1, we found that it was possible for an ordering of customers to be preserved by the P&Smechanism. To help formalize this phenomenon, we introduce the notion of a placement order . We firstdefine a placement graph of the queue as an acyclic orientation of its swapping graph, that is, a directed13cyclic graph obtained by assigning an orientation to each edge of the swapping graph. A placement order of the queue is then defined as (the strict partial order associated with) the reachability relationship of oneof its placement graphs. In other words, a strict partial order ≺ on I is said to be a placement order if thereexists a placement graph such that, for each i, j ∈ I with i = j , i ≺ j if and only if there is a directed pathfrom class i to class j in the placement graph. It will be useful later to observe that, for each classes i, j ∈ I that are neighbors in the swapping graph, we have either i ≺ j or j ≺ i .We say that a microstate c = ( c , . . . , c n ) ∈ I n adheres to the placement order if c q ⊀ c p for each p, q ∈ { , . . . , n } such that p < q . Since the placement order is only partial, there may be pairs of classes forwhich neither c p ≺ c q nor c q ≺ c p hold. As a result, adherence is a weaker property than having c p ≺ c q for each p, q ∈ { , . . . , n } such that p < q . As a special case, we mention that adherence allows that c p = c q when p < q . Example . We consider the closed P&S queue of Section 5.1.1. The placement graph in Figure 6 isobtained by orienting the edges of the swapping graph of Figure 5a from bottom to top. All microstates inFigure 5 adhere to the corresponding placement order. For example, the placement graph implies that 1 ≺ j for j ∈ { , , } and 2 ≺ j for j ∈ { , , } , which in turn implies that the customer at the front of the queueis either customer 1 or customer 2. All microstates in Figure 5 indeed satisfy this property. Customers 1and 2 can alternate positions, as neither 1 ≺ ≻ , , , , , , at most one placement order, so that the sets of microstatesthat adhere to different placement orders are disjoint. To prove this, it suffices to observe that, for eachmicrostate that adheres to a placement order, the relative placement of customers within the microstatespecifies the orientation of all edges of the swapping graph, which in turn uniquely defines a placementorder.In the rest of this section and in Section 5.1.3, we focus on the case where the initial microstate ofthe queue does adhere to a placement order. Proposition 5.2 describes the phenomenon encountered inSection 5.1.1 in full generality, while Proposition 5.3 provides a stronger result, assuming that all customersreceive a positive service rate. The case of microstates that do not adhere to a placement order is treated inAppendix A.3. Proposition 5.2.
If the initial microstate of the closed P&S queue adheres to the placement order ≺ , thenany microstate reached by applying the P&S mechanism also adheres to this placement order.Proof. Let c = ( c , . . . , c n ) denote the initial microstate of the queue and assume that c adheres to theplacement order ≺ . Let p ∈ { , . . . , n } such that ∆ µ ( c , . . . , c p ) > p . In the course of this transition, one or more customersare moved from the head towards the tail of the queue, the last one being moved to the last position. We14ow argue that the microstate reached after this transition still adheres to ≺ , after which the propositionfollows immediately, since application of the P&S mechanism only consists of a number of such transitions.We first show that the customer that completes service, of class c p , does not pass over any customer ofa class i such that c p ≺ i . If there is no integer p ′ ∈ { p + 1 , . . . , n } such that c p ≺ c p ′ , the conclusion isimmediate. Now assume that there is such an integer and let q denote the smallest integer in { p + 1 , . . . , n } such that c p ≺ c q . We will show that:(i) classes c p and c q are neighbors in the swapping graph, and(ii) there is no r ∈ { p + 1 , . . . , q − } such that classes c p and c r are neighbors in the swapping graph.By definition of the P&S mechanism, this will imply that the customer that completes service at position p replaces the customer at position q in microstate c , so that, after the transition, the prefix of length q − c , . . . , c p − , c p +1 , . . . , c q − , c p ). By definition of q , this prefix still adheres to theplacement order, and since the rest of the microstate does not change, the complete microstate will as well.The same reasoning can be repeated for each customer that is moved by applying the P&S mechanism.We first prove property (i) by contradiction. Assume that this property is not satisfied. By definition ofthe placement order, this implies that there is a class i ∈ I such that c p ≺ i ≺ c q . Since microstate c adheresto the placement order, this implies that all class- i customers are between positions p and q in microstate c .In particular, there is an r ∈ { p + 1 , . . . , q − } such that c r = i and, therefore, c p ≺ c r , which contradicts theminimality of q . Therefore, property (i) is satisfied. We now prove property (ii), again by contradiction. Ifthis property would not be not satisfied, there would be an r ∈ { p + 1 , . . . , q − } such that classes c p and c r are neighbors in the swapping graph. By definition of the placement order, this implies that either c p ≺ c r or c r ≺ c p . Since p < r and microstate c adheres to the placement order, the only possibility is that c p ≺ c r ,which again contradicts the minimality of q . Therefore, property (ii) is satisfied. Proposition 5.3.
Assume that ∆ µ ( c ) > for each c ∈ I ∗ . All microstates that adhere to the same placementorder and correspond to the same macrostate form a single closed communicating class of the Markov processassociated with the queue microstate.Proof. Given the result of Proposition 5.2, it suffices to show that, for all microstates c = ( c , . . . , c n ) and d = ( d , . . . , d n ) that adhere to the same placement order ≺ and satisfy | c | = | d | = ℓ , microstate d can bereached from microstate c with a positive probability.If c = d , the conclusion is immediate. Now assume that c = d . We will construct a path of microstates c , c , . . . , c K − , c K , with c = c and c K = d , that the queue traverses with a positive probability, providedthat it starts in microstate c . We argue that such a path c , c , . . . , c K is attained by the following algorithm: Step 1:
Set k = 0 and c = c . Step 2:
Determine the smallest integer p ∈ { , . . . , n } such that c kp = d p . Step 3:
Let c k +1 denote the microstate reached when, in microstate c k , the customer in position p completesservice and the P&S mechanism is applied. Step 4:
Set k = k + 1. If c k = d , then K = k and the algorithm terminates. Otherwise, go to step 2.The idea behind this algorithm is as follows. Step 2 identifies the first position in microstate c k at which theclass of the customer does not coincide with that of the customer at the same position in microstate d . Thisposition is denoted by p . Since ( c k , . . . c kp − ) = ( d , . . . , d p − ), the customers in positions 1 to p − r denote the smallest integer in { p + 1 , . . . , n } such that c kr = d p . We willshow in the next paragraph that, thanks to the service completion in step 3, the customer in position r inmicrostate c k is one step closer to (or even attains) position p in microstate c k +1 compared to microstate c k .This suffices to prove that the algorithm terminates. Step 4 makes sure that the two microstates are equalto each other, otherwise it initiates a new P&S transition.We now prove that, if r denotes the smallest integer in { p + 1 , . . . , n } such that c kr = d p , then c k +1 r − = c kr .This is equivalent to proving that the customer in position r in microstate c k is not ejected in the course of15he transition described in step 3. To prove this, it is sufficient to show that, for each q ∈ { p, . . . , r − } ,classes c kq and c kr cannot be swapped with one another, that is, are not neighbors in the swapping graph.Let q ∈ { p, . . . , r − } . The adherence of microstate d to the placement order implies that d q ′ d p foreach q ′ ∈ { p + 1 , . . . , n } . As ( d , . . . , d p − ) = ( c k , . . . , c kp − ) and d p = c kr , this implies that c kq ⊀ c kr . It alsofollows from Lemma 5.2 that microstate c k adheres to the placement order, so that c kr ⊀ c kq . Therefore, wehave c kq ⊀ c kr and c kr ⊀ c kq , which, by definition of a placement order, implies that classes c kq and c kr are notneighbors in the swapping graph. Remark . The assumption in Proposition 5.3, namely that ∆ µ ( c ) > c ∈ I ∗ , is a sufficientcondition for this result to hold but it is not a necessary condition. It is for example worth noting thatthis assumption is not satisfied by the multi-server queue of Example 2.2, yet the closed variant of thisqueue satisfies the conclusions of these two propositions whenever µ and µ are positive. In general, theconstruction of weaker sufficient conditions appears to be challenging since the transition described in step 3of the algorithm in the proof of Proposition 5.3 is not guaranteed to occur with a positive probability whenthere are states c so that ∆ µ ( c ) = 0. We now turn to the stationary distribution of the Markov process underlying the closed P&S queue andestablish its product-form nature. Recall that the initial macrostate of the queue equals ℓ , which cannotchange over time due to the closed nature of the queue. We assume that the initial microstate adheres toa placement order ≺ . Since all subsequent microstates must also adhere to this placement order due toProposition 5.2, we restrict the state space of the Markov process to the state space C that consists of allmicrostates c = ( c , . . . , c n ) that satisfy | c | = ℓ and adhere to the placement order ≺ . The rate function µ and balance function Φ of the queue are assumed to be defined on the whole set I ∗ for simplicity, althoughwe could just as well define them on a subset of I ∗ . Theorem 5.5 below provides the stationary distributionof the closed P&S queue and reveals its product form. Theorem 5.5.
Assume that the Markov process associated with the microstate of the closed P&S queue,with state space C , is irreducible. The stationary distribution of this Markov process is then given by π ( c ) = Φ( c ) P d ∈C Φ( d ) , ∀ c ∈ C , (5.1) where the function Φ is given by (2.4) .Proof. It suffices to show that the function Φ satisfies the balance equations of the Markov process, afterwhich the result follows by normalization. Let c = ( c , . . . , c n ) ∈ C and i = c n . Since a departing customerimmediately re-enters the queue as a customer of the same class, the balance equation for any microstate c ∈ C reads π ( c ) µ ( c ) = X d ∈C n X p =1 δ p ( d )=(( c ,...,c n − ) ,i ) π ( d ) ∆ µ ( d , . . . , d p ) , (5.2)where we write δ p ( d ) = (( c , . . . , c n − ) , i ) if, in the open queue, the service completion of the customer inposition p in state d would lead to state ( c , . . . , c n − ) with a departure of a class- i customer. It follows fromProposition 5.2 that the set C contains all microstates d ∈ I ∗ such that δ p ( d ) = (( c , . . . , c n − ) , i ) for some p ∈ { , . . . , n } . Therefore, it suffices to prove that the balance function Φ satisfiesΦ( c ) µ ( c ) = X d ∈I ∗ n X p =1 δ p ( d )=(( c ,...,c n − ) ,i ) Φ( d ) ∆ µ ( d , . . . , d p ) . (5.3)By applying (3.3) to microstate ( c , . . . , c n − ) and class i , we obtain that Φ( c , . . . , c n − ), as defined in (2.4),is equal to the right-hand side of (5.3). To conclude, it suffices to observe that (2.4) implies Φ( c ) µ ( c ) =Φ( c , . . . , c n − ). 16 emark . According to Proposition 5.3, a sufficient condition for the Markov process considered inTheorem 5.5 to be irreducible is that ∆ µ ( c ) > c ∈ I ∗ . If this process is not irreducible, allsteps of the proof of Theorem 5.5 remain valid, so that the distribution defined by (5.1) is still a stationarydistribution of the Markov process, but it may not be the only one. Since Φ( c ) > c ∈ I ∗ by (2.4),this observation shows that the Markov process considered in Theorem 5.5 always has a positive stationarydistribution, which implies that this process has no transient state, that is, all its communicating classes areclosed. Remark . A variant of Theorem 5.5 can also be derived for closed P&S queues with initial microstates thatdo not adhere to a placement order. We have deferred derivation of this more general result to Appendix A.3to simplify the discussion. Furthermore, Theorem 5.5 will be sufficient for the applications of Section 6.
Now that the product-form of the stationary distribution of a single closed P&S queue has been established,we turn to the study of a closed tandem of two P&S queues. Again, we first explain the model through anintroductory example in Section 5.2.1, after which we formalize the model and describe structural propertiesin Section 5.2.2. We also derive the stationary distribution in Section 5.2.3.
We consider the closed tandem network of two P&S queues depicted in Figure 7. Both queues have the sameset of customer classes and the same swapping graph as the closed queue of Section 5.1. We also assumethat there is a single customer of each class in the network. The routing process is as follows: for each i ∈ I ,if a class- i customer departs from a queue, this customer is routed to the back of the other queue as a class- i customer. As shown in Figure 7a, the initial microstate of the first queue is c = (1 , , , , ,
6) and that ofthe second queue is d = ∅ . Figure 7b shows the state reached after customer 1 (that is, the only customerbelonging to class 1) completes service. As in the introductory example of Section 5.1, this customer replacescustomer 3, and customer 3 replaces customer 6. The difference is that customer 6 is now routed to thesecond queue rather than to the first. Figure 7c shows the state reached after several service completions,each time that of the customer at the head of the first queue.As in Section 5.1.1, the order of customers seems to be preserved by the P&S mechanism. However, theorders of customers in the two queues are reversed. For instance, while customer 6 comes after customers 3and 4 in the first queue (as in Section 5.1.1), in the second queue customer 6 always precedes these customers.We will show that this symmetry in customer orders holds for any closed tandem network of two P&S queues.Before we move on to the analysis, let us motivate this model by giving a glimpse of the applicationsof Section 6. This section is concerned with token-based load-distribution algorithms for server systemswith assignment constraints, in which each job seizes a token upon arrival and releases its token upon6 5 4 3 2 1 (a) Initial state. (b) State reached after the servicecompletion of customer 1. (c) State reached after service com-pletions of several customers at thehead of the first queue. Figure 7: A closed tandem network of two P&S queues. As in Figure 5, the rate function is not specifiedbecause we will only consider the service completion of the customer at the head of a queue.17eparture. The above-defined closed tandem network models the dynamics of tokens as follows. The firstqueue contains available tokens, while the second queue contains tokens held by jobs in the system (eitherwaiting to be served or in service). Service completions in the first and second queues correspond to jobarrivals and departures, respectively, and the P&S mechanism translates to an algorithm to distribute loadacross servers.
Just like in Section 5.1, the queues in the closed tandem network are ordinary P&S queues as describedin Section 3. There are however no external arrivals or departures: the departure process of one queuenow forms the arrival process of the other. In particular, both queues share the same set I = { , . . . , I } ofcustomer classes. The microstate of the first queue is denoted by c = ( c , . . . , c n ) ∈ I ∗ and that of the secondqueue by d = ( d , . . . , d m ) ∈ I ∗ , where n and m are the number of customers present in the first and secondqueue, respectively. We refer to ( c ; d ) = (( c , . . . , c n ); ( d , . . . , d m )) as the microstate of the network. Wefurthermore assume that both queues have the same swapping graph. The rate functions of the two queuesmay however differ: whereas customers in the first queue complete service according to the rate function µ ,the rate function in the second queue is denoted by ν . Because of the closedness of the system, we have thatthe sum of the macrostates of the two queues, denoted by ℓ = | c | + | d | , is constant over time. We refer to thisvector ℓ = ( ℓ , . . . , ℓ I ) as the network macrostate and we assume that ℓ i > i ∈ I , as otherwisewe can simply disregard classes that are not present in the closed tandem network. Macrostates | c | and | d | ,however, do fluctuate over time. We let Φ (resp. Λ) denote the balance function of the first (resp. second)queue. The functions µ , ν , Φ, and Λ are assumed to be defined on I ∗ for simplicity.As mentioned above, both queues have the same swapping graph. Similarly to Section 5.1.2, we define a placement order ≺ of the network by directing the edges of this swapping graph so that a directed acyclicgraph, called the placement graph of the network, arises; we again write that i ≺ j if and only if there existsa directed path from class i to class j in the placement graph. We now say that a network microstate ( c ; d )adheres to the placement order ≺ if the following three conditions are satisfied:(i) ( c , . . . , c n ) adheres to the placement order ≺ in the sense of Section 5.1.2,(ii) ( d m , . . . , d ) adheres to the placement order ≺ in the sense of Section 5.1.2, and(iii) c p ⊁ d q for each p ∈ { , . . . , n } and q ∈ { , . . . , m } .Reversing the order of microstate d in property (ii) is consistent with the observation of Section 5.2.1 thatthe order of customers in the second queue is reversed compared to the first queue. Equivalently, the networkmicrostate ( c ; d ) adheres to the placement order ≺ if and only if the concatenation ( c , . . . , c n , d m , . . . , d ) ofthe sequences ( c , . . . , c n ) and ( d m , . . . , d ) adheres to the placement order ≺ in the sense of Section 5.1.2.For example, the three network microstates shown in Figure 7 adhere to the placement order ≺ defined bythe placement graph of Figure 6. Focusing on the microstate shown in Figure 7c, we have c = (2 , ,
1) and d = (6 , , , , , , , c ; d ) adheres tothe placement order ≺ if and only if microstate ( d ; c ) adheres to the reverse placement order ≻ defined asfollows: for each i, j ∈ I , i ≻ j if and only if j ≺ i .As for the case of a single queue, we focus here on the case where the initial microstate adheres to aplacement order. Propositions 5.8 and 5.9 below are the counterparts of Propositions 5.2 and 5.3 for closedtandem networks of two queues. The proofs of these two propositions rely on the proof of their single-queuecounterparts. Proposition 5.8.
If the initial network microstate adheres to the placement order ≺ , then any microstatereached by applying the P&S mechanism to either of the two queues also adheres to this placement order.Proof. By symmetry, it suffices to prove that, if a network microstate adheres to the placement order ≺ , thenany network microstate reached by a service completion in the first queue also adheres to this placement order.Consider a microstate ( c ; d ) that adheres to the placement order and let c = ( c , . . . , c n ) and d = ( d , . . . , d m ).18et c ′ = ( c ′ , . . . , c ′ n − ) denote the microstate of the first queue right after a service completion in thisqueue and i the class of the customer that departs this queue. The microstate of the network right afterthe transition is ( c ′ ; d ′ ) with d ′ = ( d , . . . , d m , i ). Applying Proposition 5.2 to the first queue yields thatmicrostate ( c ′ , . . . , c ′ n − , i ) adheres to the placement order, from which we can derive that properties (i) and(iii) are satisfied by the new network microstate. Finally, the fact that microstate ( c ; d ) satisfies properties (ii)and (iii) implies that microstate d ′ satisfies property (ii). Proposition 5.9.
Assume that either ∆ µ ( c ) > for each c ∈ I ∗ or ∆ ν ( d ) > for each d ∈ I ∗ (or both).All microstates that adhere to the same placement order and correspond to the same macrostate form a singleclosed communicating class of the Markov process associated with the network microstate.Proof. Without loss of generality, we assume that ∆ µ ( c ) > c ∈ I ∗ . The case where ∆ ν ( d ) > d ∈ I ∗ is solved by exchanging the roles of the two queues. Given the result of Proposition 5.8,it suffices to show that, for all microstates ( c ; d ) and ( c ′ ; d ′ ) that adhere to the same placement order andcorrespond to the same macrostate, microstate ( c ′ ; d ′ ) can be reached from microstate ( c ; d ) with a positiveprobability.Consider two microstates ( c ; d ) and ( c ′ ; d ′ ) that adhere to the same placement order and satisfy | c | + | d | = | c ′ | + | d ′ | . The numbers of customers in microstates c , d , c ′ , and d ′ are denoted by n , m , n ′ , and m ′ ,respectively. We now build a series of transitions that leads from microstate ( c ; d ) to microstate ( c ′ ; d ′ ) witha positive probability.First let ( c ′′ ; ∅ ) denote the microstate reached from microstate ( c ; d ) by having, m times in a row,the customer at the head of the second queue complete service. Proposition 5.8 guarantees that mi-crostate ( c ′′ ; ∅ ) adheres to the placement order so that, by property (i), microstate c ′′ adheres to the place-ment order. Since we assumed that microstate ( c ′ ; d ′ ) adheres to the placement order, we also have thatmicrostate ( c ′ , . . . , c ′ n ′ , d ′ m ′ , . . . , d ′ ) adheres to the placement order. Therefore, it follows from Proposi-tion 5.3 that, if the first queue evolved in isolation, as in Section 5.1, it would be possible to reach microstate( c ′ , . . . , c ′ n ′ , d ′ m ′ , . . . , d ′ ) from microstate c ′′ with positive probability. We can adapt the algorithm in thisproposition to prove that, in the tandem network, microstate (( c ′ , . . . , c ′ n ′ , d ′ m ′ , . . . , d ′ ); ∅ ) can also be reachedfrom microstate ( c ′′ ; ∅ ) with a positive probability: it suffices to add a transition, after step 3, that consistsof the service completion of the (only) customer in the second queue (so that this customer joins the backof the first queue). Once microstate (( c ′ , . . . , c ′ n ′ , d ′ m ′ , . . . , d ′ ); ∅ ) is reached, it suffices to have the customerat the back of the first queue complete service m ′ times in a row. Since a service completion at the finalposition of a queue does not trigger any P&S movement, the network microstate ( c ′ ; d ′ ) is reached, whichconcludes the proof. We now derive the stationary distribution of the Markov process associated with the network microstate. Asin Section 5.1.3, we focus on the special case where the initial microstate adheres to the placement order ≺ ,so that, by Proposition 5.8, all subsequent microstates also adhere to this placement order. Therefore, werestrict the state space of the Markov process to the set Σ that consists of all network microstates ( c ; d ) thatadhere to the placement order ≺ and satisfy | c | + | d | = ℓ , where ℓ denotes the initial network macrostate.Theorem 5.10 below gives the stationary distribution of the Markov process associated with the networkmicrostate. Theorem 5.10.
Assume that the Markov process associated with the microstate of the closed tandem net-work, with state space Σ , is irreducible. The stationary distribution of this Markov process is then given by π ( c ; d ) = 1 G Φ( c )Λ( d ) , ∀ ( c ; d ) ∈ Σ , (5.4) where Φ and Λ are the balance functions of the first and second queues, respectively, and the normalizationconstant G is given by G = X ( c ; d ) ∈ Σ Φ( c )Λ( d ) . (5.5)19 roof. Before writing down the balance equations, we introduce some useful notation. Let X denote thesubset of N I that consists of the vectors x = ( x , . . . , x I ) such that x ≤ ℓ and, for each i, j ∈ I with i ≺ j , x j = 0 whenever x i = 0. This is the set of possible macrostates of the first queue. For each x ∈ X , let C x denote the set of microstates c = ( c , . . . , c n ) ∈ I ∗ that adhere to the placement order and satisfy | c | = x .The set of possible microstates of the first queue is C = S x ∈X ℓ C x . Similarly, let Y denote the subset of N I that consists of the vectors y = ( y , . . . , y I ) such that | y | ≤ ℓ and, for each i, j ∈ I with i ≺ j , y i = 0whenever y j = 0. This is the set of possible macrostates of the second queue. Also, for each y ∈ Y , let D y denote the set of microstates d = ( d , . . . , d m ) such that ( d m , . . . , d ) adheres to the placement order and | d | = y . The set of possible microstates of the second queue is D = S y ∈Y ℓ D y . As a result, the state spaceΣ can be partitioned as follows: Σ = [ x ∈X C x × D ℓ − x = [ y ∈Y C ℓ − y × D y , (5.6)where the symbol × stands for the Cartesian product. In particular, if the microstate of the first queue isequal to c ∈ C , then the set of possible microstates of the second queue is Y ℓ −| c | , and vice versa.To prove the theorem, it suffices to verify that any measure given by (5.4) satisfies the following partialbalance equations in each microstate ( c ; d ) ∈ Σ, with c = ( c , . . . , c n ), d = ( d , . . . , d m ), x = | c | , and y = | d | : • Equalize the flow out of microstate ( c ; d ) due to a service completion at the first queue with the flowinto that microstate due to an arrival at this queue, that is, to a service completion at the second queue(if c = ∅ ): π ( c ; d ) µ ( c ) = X d ′ ∈D y + ecn m +1 X p =1 δ p ( d ′ )=( d,c n ) π (( c , . . . , c n − ); d ′ ) ∆ ν ( d ′ , . . . , d ′ p ) . (5.7) • Equalize the flow out of microstate ( c ; d ) due to a service completion at the second queue with the flowinto that microstate due to an arrival at this queue, that is, to a service completion at the first queue(if d = ∅ ): π ( c ; d ) ν ( d ) = X c ′ ∈C x + edm n +1 X p =1 δ p ( c ′ )=( c,d m ) π ( c ′ ; ( d , . . . , d m − )) ∆ µ ( c ′ , . . . , c ′ p ) , (5.8)We focus on (5.8) because (5.7) follows by symmetry. Assuming that d = ∅ , the main argument consists ofobserving that, since Λ( d ) ν ( d ) = Λ( d , . . . , d m − ), a stationary measure given by (5.4) satisfies (5.8) if andonly if the balance function Φ defined by (2.4) satisfiesΦ( c ) = X c ′ ∈C x + edm n +1 X p =1 δ p ( c ′ )=( c,d m ) Φ( c ′ ) ∆ µ ( c ′ , . . . , c ′ p ) . Up to a normalization constant, the right-hand side of this equation is also that of the partial balanceequation (5.2) applied to microstate ( c , . . . , c n , d m ), since the domains of the outer sums are the same. Theproof of Theorem 5.5 already showed that Φ satisfies this equation. To conclude, it suffices to observe thatthe left-hand side of (5.2) is Φ( c , . . . , c n , d m ) µ ( c , . . . , c n , d m ) = Φ( c ). Remark . Mutatis mutandis, Remarks 5.6 and 5.7 also apply to a closed tandem network of two P&Squeues. In particular, an equivalent of Appendix A.3 can be derived in case the initial network microstatedoes not adhere to a placement order. 20
Application to load distribution and scheduling in machine pools
In the introduction, we already eluded to the fact that P&S queues can be used to model several schedulingalgorithms in machine pools, such as redundancy scheduling and FCFS-ALIS scheduling. We now explainhow P&S queues can be applied to analyze the performance of other load-distribution and scheduling algo-rithms in pools of machines at which jobs arrive to be processed. This machine-pool model can representvarious queueing systems, like the computer clusters or manufacturing systems mentioned in the introduc-tion, in which not every machine is able to fulfill the service requirement of any job. As an introductoryexample, in Section 6.1, we consider a load-distribution algorithm where the assignment decision is basedon the order of tokens stored in a queue at the dispatcher. We will see later that this algorithm is relatedto other load-distribution algorithms considered in the literature. We then explain how the queueing modelthat describes the dynamics of this algorithm can be cast as a closed tandem network of two P&S queueslike that of Section 5.2. In Section 6.2, we introduce a more general framework that encompasses otherload-distribution and scheduling algorithms and then give two prototypical examples of such algorithms. Inall cases, deriving the stationary distribution of the system state is a direct application of the results ofSection 5.2, provided that the associated Markov process satisfies the appropriate irreducibility conditions.
We first consider a load-distribution algorithm whereby each incoming job is either immediately assigned toan available machine, put on hold until a machine becomes available, or blocked. Each incoming job mayhave constraints that restrict the set of machines to which this job can be assigned.
Consider a machine pool that consists of a set S = { , . . . , S } of machines. There is a single dispatcher atwhich jobs arrive to be assigned to a machine. Each machine processes its assigned jobs in FCFS order and,for each s ∈ S , the service time of a job on machine s is exponentially distributed with a positive rate µ s .Each incoming job has a type that determines the set of machines to which this job can be assigned. Theset of job types is denoted by K = { , . . . , K } and, for each k ∈ K , type- k jobs arrive according to a Poissonprocess with rate ν k and can be assigned to any machine within the set S k ⊆ S . Conversely, for each s ∈ S ,we let K s ⊆ K denote the set of job types that can be assigned to machine s (that is, such that s ∈ S k ). Thisdefines a bipartite assignment graph between job types and machines, in which there is an edge betweena type and a machine if the jobs of this type can be assigned to this machine; in this case, we say thatthese jobs are compatible with the machine. In the examples of this section, job types will be identified byletters rather than numbers to avoid confusion in the remainder. In the assignment graph of Figure 8 forinstance, type- A jobs are compatible with machines 1 and 3 and type- B jobs with machines 2 and 3, so that S A = { , } , S B = { , } , K = { A } , K = { B } , and K = { A, B } .Type A Type BMachine 1 Machine 3 Machine 2Figure 8: A bipartite assignment graph between job types and machines. To avoid any confusion in the restof this section, job types are identified by letters rather than numbers in the examples.All present jobs are stored in a single queue in their order of arrival in the pool, with the oldest job atthe head of the queue. To implement the load-distribution algorithm that we will describe below, we couldequivalently have assumed that each machine has its own queue of jobs, and that unassigned jobs are storedat the dispatcher. We will however adhere to the central-queue perspective, as it is more practical from a21odeling perspective. For each s ∈ S , at most ℓ s jobs can be assigned to machine s at a time, includingthe job in service on this machine, for some ℓ s >
0. Machine s is said to be available if there are currentlyless than ℓ s jobs assigned to this machine and unavailable otherwise. Equivalently, for each s ∈ S , we canassume that the job assignments to machine s are controlled via ℓ s tokens, so that a job seizes a token ofmachine s when it is assigned to this machine and releases its token upon service completion. Furthermore,for each k ∈ K , at most ℓ k type- k jobs can be put on hold if they arrive while no compatible machine isavailable, for some ℓ k ≥ k ∈ K , when a type- k job arrives, one of the following events occurs:(i) if one or more machines in the set S k are available, the job is immediately assigned to the machineidentified by the token that has been available the longest (among the tokens that correspond to acompatible machine). In other words, if the available tokens are sorted in their release order in a queueat the dispatcher, with the oldest available token at the head of the queue, the incoming job scans thisqueue from head to tail and seizes the first encountered token that belongs to a compatible machine;(ii) otherwise, if there are currently less than ℓ k type- k jobs on hold, the incoming job is put on hold untilit can be assigned to one of its compatible machines;(iii) otherwise, the job is lost.We now describe what happens upon a service completion. For each s ∈ S , if a job completes service onmachine s , the oldest job of a type in K s on hold, if any, is assigned to machine s , and then machine s startsprocessing the oldest job assigned to this machine. Note that there cannot be a job of a type in K s on holdif machine s was previously available, as in this case this job would have been assigned to machine s earlier.We assume that these operations occur immediately upon a job arrival or a service completion. Remark . The assign-to-the-longest-idle-server (ALIS) algorithm introduced in [2] corresponds to thespecial case where ℓ s = 1 for each s ∈ S and ℓ k = 0 for each k ∈ K , so that a job is blocked whenever allits compatible machines are already busy. It was shown in [2] that, in this case, performance is insensitiveto the job size distribution beyond its mean. A token-based assignment rule that generalizes ALIS to ascenario where ℓ s ≥ s ∈ S and that preserves its insensitivity property was introduced in [12]; thisalgorithm is also a special case of the algorithm that we have just defined. The FCFS-ALIS assignment ruleintroduced in [3] corresponds to the special case where ℓ s = 1 for each s ∈ S and ℓ k = + ∞ for each k ∈ K .In practice, taking larger values for ℓ s for s ∈ S can be beneficial if there is a communication delay betweenthe dispatcher and the machines, such that a job cannot enter in service immediately after it is assigned toa machine. Furthermore, taking unequal values for ℓ k for k ∈ K allows to differentiate service between jobtypes. In the remainder of this section, we shall assume that 1 ≤ ℓ k < + ∞ for each k ∈ K and 1 ≤ ℓ s < + ∞ for each s ∈ S . We now cast the above-mentioned model as a closed tandem network of two P&S queues. To this end, itis worth making a clear distinction between two types of tokens in the machine pool. First observe thatthe entry of type- k jobs can also be described as if regulated via ℓ k tokens: an incoming type- k job seizesone of these tokens when it can enter the system but cannot be assigned immediately to a compatiblemachine. These tokens are called holding tokens. Once a job has claimed such a holding token, this token isonly released when the job is assigned to some compatible machine, or equivalently, when it seizes a tokenspecific to some compatible machine (of which there are ℓ s per machine s ). Such machine-specific tokens arecalled assignment tokens . Both holding and assignment tokens are either available or held by a job and thedynamics of the machine pool are entirely described by the movement of these tokens. Incoming jobs havea preference for assignment tokens over holding tokens in the sense that, for each k ∈ K , an arriving type- k job seizes an assignment token of a machine in S k , if any, and otherwise a holding token of type k , if any.We will show that this preference can be modeled by the P&S mechanism. We first give an overview of theclosed tandem network and then we will detail the dynamics of each queue separately.22 verview We consider a closed tandem network of two P&S queues like the one described in Section 5.2.Customers represent tokens and the set of customer classes is I = K ⊔ S , where ⊔ denotes the disjointunion operator. This set has cardinality I = K + S . Class- s customers represent the assignment tokensof machine s , for each s ∈ S , while class- k customers represent the holding tokens for type- k jobs, for each k ∈ K . The first queue contains the tokens held by jobs present in the machine pool and the second queuecontains the available tokens. The overall number of class- i customers in the network is ℓ i , for each i ∈ I .Both P&S queues have the same swapping graph, so that we obtain a closed tandem network of twoP&S queues like that described in Section 5.2. The placement order is as follows: s ≺ k for each s ∈ S and k ∈ K s (or equivalently, for each k ∈ K and s ∈ S k ). The corresponding placement graph is obtained fromthe assignment graph introduced in Section 6.1.1 by orienting edges from the (classes that correspond to)machines towards the (classes that correspond to) job types. For example, the placement graph associatedwith the assignment graph of Figure 8 is shown in Figure 9a. The swapping graph of the network is simplythe underlying undirected graph of the placement graph. This placement order guarantees that, if a class- k customer is in the first queue, then, for each s ∈ S k , all class- s customers are also in the first queue andprecede this class- k customer. This corresponds to the fact that, in the machine pool, a type- k job can onlybe on hold if all machines in S k are unavailable. We will come back to this interpretation later when wespecify the dynamics of each queue in detail. An example of a network state that adheres to this placementorder is given by a state where all customers are lined up in the second queue, with first those of the classesin K in an arbitrary order, and then those of the classes in S , also in an arbitrary order. We assume that thesystem starts in such a state, which corresponds to an empty machine pool in which all tokens are available. A B (a) Placement graph.
A B (b) Compatibility graph of the firstmulti-server queue. Each server corre-sponds to a machine in the pool.
A B
A B (c) Compatibility graph of the secondmulti-server queue. Each server corre-sponds to a job type in the pool.
Figure 9: Closed tandem network of two P&S queues associated with the assignment graph of Figure 8.Since the servers in the second queue and the holding classes are associated with the job types, they areidentified by letters rather than numbers (in accordance with Figure 8). The class colors are visual aids thathelp distinguish between the classes associated with assignment tokens (in green) and those associated withholding tokens (in orange). The same class and server indexing and color code will be adopted in Figures 10and 11.
First queue
Customers in the first queue represent tokens held by jobs present in the machine pool, eitherassigned to a machine or on hold. This queue is a multi-server queue, like that of Example 3.1, and eachserver represents a machine in the pool. More specifically, the set of servers is S and, for each s ∈ S , theservice rate of server s is equal to the service rate µ s of machine s . Using the same index set S for theset of servers in the first queue and for the set of classes associated with assignment tokens may seem tobe ambiguous at first, but the idea is the same as in the degenerate queue of Example 2.3: there will be aone-to-one correspondence between these servers and these classes. More specifically, for each s ∈ S , a class- s customer can be served by server s and by this server only, so that S s = { s } . For example, the compatibilitygraph of the first queue in the tandem network associated with the assignment graph of Figure 8 is shown In practice, to make sure that the sets K and S remain disjoint, job types and machines can be renumbered if necessary.No confusion will arise from this slight abuse of notation, as we will always be using the letter k to refer to a job type and theletters s and t to refer to machines. In the example, the index of a class will be a number if this class represents the assignmenttokens of a machine and a letter if it represents the holding tokens of a job type.
23n Figure 9b. This graph shows that classes 1, 2, and 3 can only be served by the server with the samenumber. Additionally, for each k ∈ K , the set of servers that can process class- k customers is S k , the set ofmachines to which type- k jobs can be assigned. In the end, for each s ∈ S , server s can process customersof the classes in { s } ∪ K s . Each server processes its assigned customers in FCFS order. Note that, for each k ∈ K , the set S k now plays two roles in the tandem network: it represents the set of servers that can processcustomers of this class in the first queue as well as the set of customer classes that can be swapped withclass- k customers.Recall that, according to the placement order, a class- k customer (representing a holding token) cannotprecede a customer of a class in S k (representing an assignment token) in the first queue. With the abovedefinition of S i for each i ∈ I , this implies that a class- k customer will actually never be in service in thefirst queue (except in the degenerate case, left aside in this discussion, where ℓ s = 0 for some s ∈ S k ). Theonly way that a class- k customer leaves the first queue is if a customer of a class in S k completes service andejects this class- k customer. In the machine pool, this means that a job completes service on a machine in S k and the assignment token released by this job is seized by a type- k job that was on hold (so that thistype- k job releases its holding token). Note that the dynamics of the tandem network would be the same ifthe set of servers that can process class- k customers were a strict subset of S k .We let c = ( c , . . . , c n ) ∈ I ∗ denote the microstate of the first queue. As observed before, the placementorder guarantees that, if there is a customer of a class k ∈ K at a position p ∈ { , . . . , n } in the first queue,then, for each s ∈ S k , each class- s customer is also in the first queue, at a position q ∈ { , . . . , p − } thatprecedes that of the class- k customer. Therefore, the state space of the microstate of the first queue is astrict subset C of the set of sequences c ∈ I ∗ such that | c | ≤ ℓ , where ℓ = ( ℓ , . . . , ℓ I ) is the vector that givesthe maximum number of customers of each class. The overall and per-customer service rates are still givenby (2.1) and (2.2), with the sets S k for k ∈ K and S s for s ∈ S as defined above. Because of the placementorder, (2.2) simplifies to ∆ µ ( c , . . . , c p ) = 0 for each p ∈ { , . . . , n } such that c p ∈ K . Furthermore, theone-to-one correspondence between the servers and the classes associated with assignment tokens guaranteesthat, for each p ∈ { , . . . , n } such that s = c p ∈ S , we have ∆ µ ( c , . . . , c p ) = µ s if | ( c , . . . , c p − ) | s = 0 and∆ µ ( c , . . . , c p ) = 0 otherwise.Now assume that a customer in some position p ∈ { , . . . , n } completes service and let s = c p ∈ S denotethis customer’s class. Because of the P&S mechanism, only one of these two types of transitions can occur:(i) If the first queue contains a customer of a class in K s (necessarily in position at least p + 1 because ofthe placement order), the class- s customer replaces the first of these customers, say of class k ∈ K s ,and this class- k customer joins the second queue. In the machine pool, this means that an assignmenttoken from machine s is released by a departing job and is immediately seized by a type- k job that wason hold; this type- k job releases its holding token which is appended to the queue of available tokens.(ii) If there is no customer with a class in K s in the first queue, the class- s customer leaves this queue andjoins the second queue. In the machine pool, this means that a token from machine s is released by adeparting job and is immediately appended to the queue of available tokens because there is no job ofa type in K s on hold.In both cases, a customer leaves the first queue and is added to the second, meaning that a token is releasedin the machine pool. Examples of transitions are shown in Figure 10 for the machine pool of Figure 8. Inthe state of Figure 10a, all assignment tokens are held by a present job and there is also a type- A job onhold. From Figure 10a to Figure 10b, the oldest class-2 customer completes service in the first queue andjoins the second queue. In the machine pool, this means that a job completes service on machine 2 and itstoken is added to the queue of available tokens because there is not job of a compatible type on hold. This isa transition of type (ii). From Figure 10b to Figure 10c, the oldest class-3 customer completes service in thefirst queue; this customer replaces the class- A customer which joins the second queue. In the machine pool,this means that a job completes service on machine 3 and its token is seized by a type- A job on hold. Thisis a transition of type (i). The transition from Figure 10c to Figure 10d, triggered by a service completionin the second queue, will be commented on later. 24 µ µ µ ν A ν B B A B (a) Initial state. A µ µ µ ν A ν B B A B (b) State reached after the service completion ofthe oldest class-2 customer in the first queue. µ µ µ ν A ν B B A B A (c) State reached after the service completion ofthe oldest class-3 customer in the first queue. µ µ µ ν A ν B A B B A (d) State reached after the service completion ofthe oldest class- B customer in the second queue. Figure 10: Closed tandem network of two P&S queues associated with the machine pool of Figure 8, assumingthat ℓ k = 2 for each k ∈ K and ℓ s = 2 for each s ∈ S . Second queue
We now provide a symmetric description for the second queue, in which customers corre-spond to available tokens. This queue is again a multi-server queue like that of Example 3.1, but the serverscorrespond to job types and not to machines. More specifically, the set of servers is K and, for each k ∈ K ,the service rate of server k is equal to ν k , the arrival rate of type- k jobs in the machine pool. There is aone-to-one correspondence between the servers in this second queue and the classes associated with holdingtokens in the sense that, for each k ∈ K , the set of servers that can process class- k customers is K k = { k } .Also, for each s ∈ S , the set of servers that can process class- s customers is K s , corresponding to the set ofjob types that can seize a token from machine s in the machine pool. For example, the compatibility graphof the second queue in the tandem network associated with the assignment graph of Figure 8 is shown inFigure 9c. Each server processes its assigned customers in FCFS order.Due to the placement order, a class- s customer cannot precede a customer of a class in K s in the secondqueue. With the above definition of K i for each i ∈ I , this implies that a class- s customer will never be inservice in the second queue (except in the degenerate case, left aside in this discussion, where ℓ k = 0 forsome k ∈ K s ); the only way a class- s customer leaves this queue is if a customer of a class in K s completesservice and ejects this class- s customer. In the machine pool, this means that a type- k job enters and seizesa token from machine s . Note that the dynamics would be the same if the set K s of servers that can processclass- s customer were a strict subset of K s .We let d = ( d , . . . , d m ) ∈ I ∗ denote the microstate of the second queue. As observed before, theplacement order guarantees that, if there is a customer of a class s ∈ S at some position p ∈ { , . . . , m } inthe second queue, then, for each k ∈ K s , each class- k customer is also in the second queue, at a position25 ∈ { , . . . , p − } that precedes that of this class- s customer. Therefore, the state space of the microstateof the second queue is a strict subset D of the set of sequences d ∈ I ∗ such that | d | ≤ ℓ . The overall servicerate in this queue is equal to the sum of the arrival rates of the job types that can seize at least one availabletoken, given by ν ( d ) = X k ∈ S mp =1 K dp ν k . (6.1)For each p ∈ { , . . . , m } , the token in position p can be seized or moved by the incoming jobs that arecompatible with this token but not with the older available tokens, and these jobs arrive at rate∆ ν ( d , . . . , d p ) = X k ∈K dp \ S p − q =1 K dq ν k . (6.2)The functions ν and ∆ ν play the same role for the second queue as the functions µ and ∆ µ , given by (2.1)and (2.2), for the first queue. Again because of the placement order, (6.2) simplifies to ∆ ν ( d , . . . , d p ) = 0 foreach p ∈ { , . . . , m } such that d p ∈ S . Additionally, the one-to-one correspondence between the servers andthe classes associated with holding tokens guarantees that, for each p ∈ { , . . . , m } such that k = d p ∈ K ,we have ∆ ν ( d , . . . , d p ) = ν k if | ( d , . . . , d p − ) | k = 0 and ∆ ν ( d , . . . , d p ) = 0 otherwise.Now assume that a customer in some position p ∈ { , . . . , m } completes service. If k = c p ∈ K denotesthis customer’s class, then, because of the P&S mechanism, only one of these two transitions can occur:(i) If the second queue contains a customer of a class in S k (necessarily in position at least p + 1 becauseof the placement order), the class- k customer replaces the first of these customers, say of class s ∈ S k ,and this class- s customer joins the first queue. In the machine pool, this means that an incomingtype- k job seizes an assignment token from machine s because this was the oldest available token of acompatible machine.(ii) If there is no customer of a class in S k in the second queue, the class- k customer leaves this queue andis added to the first queue. In the machine pool, this means that a type- k job enters and does notfind any available token from a machine in S k , so that this job seizes a class- k token and will hold thistoken until it is assigned to a machine in S k .In both cases, a customer leaves the second queue and joins the first, meaning that a token is seized in themachine pool. An example of a type-(i) transition is shown in Figure 10 for the machine pool of Figure 8.From Figure 10c to Figure 10d, the oldest class- B customer completes service in the second queue; thiscustomer replaces the class-2 customer that joins the first queue. In the machine pool, this means that atype-2 job enters and seizes an assignment token corresponding to machine 2. Remark . Both queues are degenerate in the following sense. In the second queue, for each k ∈ K , class k isthe only class within the set K that can be processed by server k . Therefore, if we let A = { k ∈ K : | d | k > } ,then, for each k ∈ A , the service rate of the oldest class- k customer is ν k , irrespective of the order of thecustomers in microstate d (provided that this microstate adheres to the placement order). This implies thatthe relative order of the customers of the classes in K in microstate d modifies neither their service rates northe departure rate of the customers of the classes in S . On the contrary, in general, the relative order ofthe customers of the classes in S modifies their departure rate. A similar remark could be made for the firstqueue by exchanging the roles of the sets K and S . The key insight from Section 6.1 is that we can describe the dynamics of tokens using a closed tandemnetwork of two P&S queues, so that one queue contains tokens held by jobs in the machine pool andthe other available tokens. In Section 6.2.1, we propose a more general framework based on the sameidea. This framework extends the example of Section 6.1 in two ways: it allows not only for more generalcompatibility constraints between customer classes and servers, but also for multiple levels of preferencesbetween customers. Sections 6.2.2 and 6.2.3 give a prototypical example for each extension.26 .2.1 Queueing model
Consider a closed tandem network of two P&S queues like that described in Section 5.2. Let I = { , . . . , I } denote the set of customer classes and ≺ the placement order of this network. Recall that, for each i, j ∈ I such that i ≺ j , class- i customers precede (resp. succeed) class- j customers in the first (resp. second) queue.The swapping graph of the network is simply the underlying undirected graph of the placement graph. Foreach i ∈ I , let ℓ i denote the number of class- i customers in the network. We assume that both P&S queuesare multi-server queues, like that described in Example 3.1, with compatibility graphs that will be describedin the next paragraphs. In the applications we consider, customers represent tokens in a machine pool, sothat the first queue contains tokens held by present jobs and the second queue available tokens.Let S = { , . . . , S } denote the set of servers in the first queue and, for each s ∈ S , µ s the service rateof server s . For each class i ∈ I that is minimal with respect to the placement order ≺ (that is, there is noclass j ∈ I with i ≻ j ), we let S i ⊆ S denote the set of servers that can process class- i customers in the firstqueue. This defines a bipartite graph between the set of minimal classes and the set of servers. The set ofservers that can process non-minimal classes is defined by an ascending recursion over the placement order.More specifically, for each class i ∈ I that is not minimal with respect to the placement order, the set ofservers that can process class- i customers is S i = S j ∈I : j ≺ i S j . Going back to the example of Figure 9, wehave that the set of minimal classes is { , , } ; the sets of servers associated with these classes in the firstqueue are S = { } , S = { } , and S = { } , while the sets of servers associated with the classes that arenot minimal are S A = S ∪ S = { , } and S B = S ∪ S = { , } . The microstate of the first queue isdenoted by c = ( c , . . . , c n ) and the overall and individual service rates in this queue are given by (2.1) and(2.2), respectively.Similarly, we let K = { , . . . , K } denote the set of servers in the second queue and, for each k ∈ K , ν k the service rate of server k . For each class i ∈ I that is maximal with respect to the placement order ≺ (that is, there is no class j ∈ I with i ≺ j ), we let K i ⊆ K denote the set of servers that can process class- i customers. This defines a bipartite graph between the set of maximal classes and the set of servers. Theset of servers that can process non-maximal classes is defined by a descending recursion over the placementorder. More specifically, for each class i ∈ I that is not maximal, the set of servers that can processclass- i customers is K i = S j ∈I : i ≺ j K j . By again considering the example of Figure 9, we have that theset of maximal classes is { A, B } ; the sets of servers associated with these classes in the second queue are K A = { A } and K B = { B } , while the sets of servers associated with the classes that are not maximal are K = K A = { A } , K = K B = { B } , and K = K A ∪ K B = { A, B } . The microstate of the second queue isdenoted by d = ( d , . . . , d m ) and the overall and individual service rates in this queue are given by (6.1) and(6.2), respectively.In this new framework, the placement order describes not only priorities between classes but also compat-ibilities between classes and servers. Using this observation, we will now see that the structure of the closedtandem network can be described more compactly by a mixed graph (that is, a graph with both directedand undirected edges). The mixed graph associated with the model of Figure 9 is shown in Figure 11. Thesubgraph induced in this mixed graph by the set of classes describes the placement order. The subgraphinduced by the set of minimal classes and the set of machines, as shown at the bottom of Figure 11, describesthe compatibilities between the minimal classes and the servers of the first queue. The set of servers thatcan serve a non-minimal class is the union of the sets of servers that can serve the ancestors of this class.Similarly, the subgraph induced by the sets of maximal classes and the set of job types, as shown at the topof Figure 11, describes the compatibilities between the maximal classes and the servers of the second queue.The set of servers that can serve a non-maximal class is the union of the sets of servers that can serve thedescendants of this class. Figures 13 and 15 show more elaborate examples of mixed graphs that will bestudied in Sections 6.2.2 and 6.2.3.As in Section 6.1, some customers may never be in service. In the first queue, the only customers thatcan be in service are those of the classes that are minimal with respect to the placement order. A customerof a class i ∈ I that is not minimal can only leave this queue upon the service completion of a customer ofa minimal class j ∈ I such that j ≺ i . Similarly, only customers of classes that are maximal with respect tothe placement order can be in service in the second queue. A customer of a class i ∈ I that is not maximal27 B j ∈ I such that i ≺ j .Applying the results of Section 5.2.3 allows us to directly derive a closed-form expression for the stationarydistribution of the network microstate. We adopt the notation of this section. In particular, the state spaceof the Markov process associated with the network microstate ( c, d ) is denoted by Σ and characterizedby (5.6). Assuming that this Markov process is irreducible, it follows from Theorem 5.10 that its stationarydistribution is given by π ( c, d ) = 1 G n Y p =1 µ ( c , . . . , c p ) ! m Y p =1 ν ( d , . . . , d p ) ! , ∀ ( c, d ) ∈ Σ , where the constant G follows from normalization.We now consider two examples that illustrate the descriptive power of this framework. Section 6.2.2 givesan extension of the introductory example of Section 6.1 to a machine pool where requests can be distributedover several machines. Section 6.2.3 looks at a token-based hierarchical load-distribution algorithm. As in Section 6.1, we consider a pool that consists of a dispatcher and a set S = { , . . . , S } of machines.The set of job types is denoted by K = { , . . . , K } and, for each k ∈ K , type- k jobs arrive according toan independent Poisson process with rate ν k and have independent and exponentially distributed sizes withunit mean. Depending on its type and on the system state, an incoming job can be assigned to one (or more)machine(s), put on hold, or blocked.The difference with Section 6.1 is that an incoming job is not assigned to a single machine anymore butto a group of machines that will subsequently be able to process this job in parallel. We let T = { , . . . , T } denote the set of group indices and, for each t ∈ T , S t ⊆ S the set of machines that belong to group t and K t ⊆ K the set of job types that can be assigned to group t . Conversely, we let T s ⊆ T denote theset of groups that contain machine s , for each s ∈ S , and T k ⊆ T the set of groups to which type- k jobscan be assigned, for each k ∈ K . This defines a tripartite assignment graph between job types, groups, andmachines, as shown in Figure 12. The introductory example of Section 6.1 corresponds to the special casein which there is a one-to-one correspondence between groups and machines, that is, T = S and S t = { t } for each t ∈ T .A job is said to be assigned to a machine if it is assigned to a group that includes this machine. All presentjobs are still gathered in a single queue in their order of arrival in the pool, and each machine processes itsassigned jobs in FCFS order, while ignoring other jobs. In this way, each assigned job is in service on all themachines to which this job was assigned and no older job was assigned. For each t ∈ T , if a job assigned togroup t is in service on a subset S ′ ⊆ S t of its assigned machines, the departure rate of this job is P s ∈S ′ µ s .28ype A Type BGroup 1 Group 2Machine 1 Machine 3 Machine 2Figure 12: A tripartite assignment graph between job types, groups, and machines. We have S = { , } , S = { , } , K = { A } , and K = { A, B } .The token-based assignment rule introduced in Section 6.1 is generalized as follows. For each t ∈ T , thejob assignment to group t is regulated via ℓ t tokens so that, for each k ∈ K t , an incoming type- k job can beassigned to group t if and only if one of the tokens of this group is available. Additionally, for each k ∈ K ,at most ℓ k type- k jobs can be put on hold if no token of a compatible group is available upon their arrival.In this way, when a type- k job arrives in the machine pool, one of the following events occurs:(i) if one or more tokens of the groups in T k are available, the job seizes the one of these tokens that hasbeen available the longest and is assigned to the corresponding group;(ii) otherwise, if there are currently less than ℓ k type- k jobs on hold, the incoming job is put on hold untilit can be assigned to one of its compatible groups;(iii) otherwise, the job is lost.When a job assigned to group t completes service, this job leaves the machine pool immediately. Its token isseized by the oldest job of a type in K t on hold, if any, otherwise it is added to the queue of available tokens.Furthermore, the machines that were processing this job immediately start processing their next assignedjob, if any. This machine pool can be seen as a generalization of that introduced in [12] to a scenario whereincoming jobs are put on hold (instead of being blocked) in the absence of available compatible tokens.The dynamics of this machine pool can be described by the queueing model of Section 6.2.1 as follows.The set of customer classes is I = K ⊔ T . The placement order is defined by t ≺ k for each t ∈ T and k ∈ K t . The minimal classes are those corresponding to the (assignment tokens of the) machine groups andthe maximal classes are those corresponding to the (holding tokens of the) job types. For each t ∈ T , theset of servers that can process class- t customers in the first queue is S t , the set of machines that belong togroup t . For each k ∈ K , the set of servers that can process class- k customers in the second queue is { k } .The mixed graph associated with the example of Figure 12 is shown in Figure 13. A B .2.3 Hierarchical load distribution
Let H denote a positive integer. We consider a pool of 2 H − machines and denote by S = { , . . . , H − } the set of machines. Jobs arrive according to a Poisson process with a positive rate ν . Each incoming jobis compatible with all machines but will eventually be assigned to and processed by a single machine. Foreach s ∈ S , the service time of a job on machine s is exponentially distributed with a positive rate µ s . Thejob arrivals within the machine pool are regulated via 2 H − H −
1. A jobthat is waiting to be assigned to a machine holds a token numbered from 1 to 2 H − −
1, while, for each s ∈ { , . . . , H − } , a job in service on machine s holds token 2 H − + s −
1. Initially, when the pool is emptyof jobs, all tokens are arranged in ascending order in a queue of available tokens, with token 1 at the head ofthe queue and token 2 H − i ∈ { , . . . , H − − } , token i can be swapped with tokens 2 i and 2 i + 1. An incoming job is lost if no token is available. Conversely, aservice completion triggers the following chain reaction: if token i is released by a job, this token is seizedby the job that holds token ⌊ i/ ⌋ (so that this token is in turn released and can be seized by another job),if any, otherwise it is added to the queue of available tokens.Priorities between tokens can be represented by a perfect binary tree of height H − i ∈ { , . . . , H − − } , the children of node i are nodes 2 i and 2 i + 1. Figure 14 shows an example with H = 3. Leaf nodes correspond to tokens held by jobs in service on a machine. For each h ∈ { , . . . , H − } ,the nodes at depth h in the tree correspond to tokens 2 h − to 2 h −
1. A job holding one of these tokens is H − h steps away from entering in service on a machine. Indeed, if a job holds a token i ∈ { , . . . , H − − } and a token that belongs to the subtree rooted at node i is released, this job will seize either token 2 i ortoken 2 i + 1, thus getting one step closer to entering in service on a machine.12 34 5 6 7Figure 14: A perfect binary tree of height H − I = { , . . . , H − } . For each i ∈ I , there is a single class- i customer which corresponds,in the machine pool, to token i . The placement order is defined as follows: for each i ∈ { , . . . , H − − } , i ≻ i and i ≻ i + 1 (so that the placement graph is obtained by reversing edges in the perfect binary treedefined in the previous paragraph). In particular, if the queue of available tokens is not empty, the customerat the head of this queue is necessarily customer 1. The set of servers in the first queue is S = { , . . . , H − } .The set of minimal customer classes is { H − , H − + 1 , . . . , H − } and, for each s ∈ { , . . . , H − } , class2 H − + s − s . In the second queue, there is a single server of rate ν . The onlymaximal class is class 1 and this class is compatible with this server. The mixed graph associated with theperfect binary tree of Figure 14 is shown in Figure 15.This hierarchical load-distribution strategy could be generalized by considering a perfect a -ary tree with a ≥ In this paper, we introduced pass-and-swap (P&S) queues, an extension of order-independent (OI) queues inwhich, upon a service completion, customers move along the queue and swap position with other customers,depending on compatibilities defined by a so-called swapping graph. We showed that a stable P&S queue isquasi-reversible and that, surprisingly, its product-form stationary distribution is independent of its swappinggraph. We then studied networks of P&S queues. Although deriving the stationary distribution of opennetworks is a straightforward application of quasi-reversibility, the case of closed networks is more intricatebecause the Markov process describing the network microstate over time is not necessarily irreducible. Forclosed networks with one or two queues and a deterministic routing process, we observed that the P&Smechanism allows for the enforcement of priorities between classes, in the sense that a customer cannot leavea queue before all customers of the classes with higher priority leave it. Finally, we showed that such closednetworks describe the dynamics of the loss variants of several token-based load-distribution algorithms, suchas ALIS and FCFS-ALIS.This work suggests that we still do not have a complete picture of all queueing dynamics that lead toa product-form stationary distribution, which leaves open an important avenue for further study. Anotheropen question is formed by the irreducibility of the Markov process underlying closed networks. Whilewe established irreducibility of this Markov process under the condition that, at any point in time, eachcustomer has a positive service rate, the characterization of irreducibility properties of the Markov processunderlying general closed networks, and their impact on the stationary distribution (along with its product-form nature), remains an open question. A different direction of further research entails the applications ofP&S queues. In particular, in Section 6, we regarded applications based on multi-server queues as defined inExample 2.2. Although applications with arbitrary customer-server compatibilities, such as load-balancingand resource-management algorithms in computer systems, form the motivation for this work, we believethat P&S queues can be successfully applied to other systems involving priorities. This would require theuse of more general P&S queues than just multi-server queues.
Acknowledgement
We are thankful to Thomas Bonald and Fabien Mathieu for their useful commentsand for coining the name “pass-and-swap queues”. We also thank Sem Borst for some helpful discussionsand for valuable remarks on an earlier draft of this paper. The research of Jan-Pieter Dorsman is supportedby the Netherlands Organisation for Scientific Research (NWO) through Gravitation-grant NETWORKS-024.002.003. 31 eferences [1] I.J.B.F. Adan, I. Kleiner, R. Righter, and G. Weiss. FCFS parallel service systems and matching models.
Performance Evaluation , 127-128:253–272, 2018.[2] I.J.B.F. Adan and G. Weiss. A loss system with skill-based servers under assign to longest idle serverpolicy.
Probability in the Engineering and Informational Sciences , 26(3):307–321, 2012.[3] I.J.B.F. Adan and G. Weiss. A skill based parallel service system under FCFS-ALIS — steady state,overloads, and abandonments.
Stochastic Systems , 4(1):250–299, 2014.[4] U. Ayesta, T. Bodas, J.L. Dorsman, and I.M. Verloop. A token-based central queue with order-independent service rates. 2020. To appear in
Operations Research .[5] U. Ayesta, T. Bodas, and I.M. Verloop. On a unifying product form framework for redundancy models.
Performance Evaluation , 127-128:93–119, 2018.[6] F. Baskett, K.M. Chandy, R.R. Muntz, and F.G. Palacios. Open, closed, and mixed networks of queueswith different classes of customers.
Journal of the ACM , 22(2):248–260, 1975.[7] S.A. Berezner, C.F. Kriel, and A.E. Krzesinski. Quasi-reversible multiclass queues with order indepen-dent departure rates.
Queueing Systems , 19(4):345–359, 1995.[8] S.A. Berezner and A.E. Krzesinski. Order independent loss queues.
Queueing Systems , 23(1-4):331–335,1996.[9] T. Bonald and C. Comte. Balanced fair resource sharing in computer clusters.
Performance Evaluation ,116:70–83, 2017.[10] R.J. Boucherie and N.M. van Dijk, editors.
Queueing networks: A fundamental approach . InternationalSeries in Operations Research & Management Science. Springer US, 2011.[11] X. Chao. Networks with customers, signals, and product form solution. In
Queueing Networks: AFundamental Approach , International Series in Operations Research & Management Science, pages217–267. Springer, Boston, MA, 2011.[12] C. Comte. Dynamic load balancing with tokens.
Computer Communications , 144:76–88, 2019.[13] C. Comte.
Resource management in computer clusters: algorithm design and performance analysis .phdthesis, Institut Polytechnique de Paris, 2019.[14] K.S. Gardner and R. Righter. Product forms for FCFS queueing models with arbitrary server-jobcompatibilities: an overview.
Queueing Systems , 96(1):3–51, 2020.[15] K.S. Gardner, S. Zbarsky, S. Doroudi, M. Harchol-Balter, E. Hyyti¨a, and A. Scheller-Wolf. Queueingwith redundant requests: Exact analysis.
Queueing Systems , 83(3-4):227–259, 2016.[16] J.R. Jackson. Networks of waiting lines.
Operations Research , 5(4):518–521, 1957.[17] F.P. Kelly.
Reversibility and stochastic networks . Cambridge University Press, 2011.[18] F.P. Kelly and J. Walrand. Networks of quasi-reversible nodes. In
Applied Probability-Computer Science:The Interface Volume 1 , Progress in Computer Science, pages 3–29. Birkh¨auser Boston, 1982.[19] A.E. Krzesinski. Order independent queues. In R.J. Boucherie and N.M. van Dijk, editors,
Queue-ing networks: A fundamental approach , number 154 in International Series in Operations Research &Management Science, pages 85–120. Springer US, 2011.3220] A.E. Krzesinski and R. Schassberger. Product form solutions for multiserver centers with hierarchicalconcurrency constraints.
Probability in the Engineering and Informational Sciences , 6(2):147–156, 1992.Publisher: Cambridge University Press.[21] J.-Y. Le Boudec. A BCMP extension to multiserver stations with concurrent classes of customers.
SIGMETRICS Performance Evaluation Review , 14(1):78–91, 1986.[22] P. Moyal, A. Busic, and J. Mairesse. A product form for the general stochastic matching model. arXiv:1711.02620 [math] , 2020.[23] R.R. Muntz. Poisson departure processes and queueing networks.
IBM Thomas J. Watson ResearchCentre , 1972.[24] R. Serfozo.
Introduction to stochastic networks . Stochastic Modelling and Applied Probability. Springer-Verlag, 1999.[25] N.M. Van Dijk. On practical product form characterizations. In R.J. Boucherie and N.M. van Dijk,editors,
Queueing networks: A fundamental approach , International Series in Operations Research &Management Science, pages 1–83. Springer US, Boston, MA, 2011.
A Appendix
A.1 Proof of Theorem 3.2
Consider a P&S queue as defined in Section 3.1, with a set I = { , . . . , I } of customer classes, per-classarrival rates λ , . . . , λ I , and a rate function µ . Also, for each i ∈ I , let I i ⊆ I denote the set of customerclasses that can be swapped with class i . As announced in the sketch of proof that followed Theorem 3.2,our objective is to prove that the balance function Φ defined by (2.4) satisfies (3.3). Rewriting (3.3) We first need to specify, for each c = ( c , . . . , c n ) ∈ I ∗ and i ∈ I , all transitions that leadto microstate c by the departure of a class- i customer. To this end, we will identify all microstates d ∈ I n +1 and positions p ∈ { , . . . , n + 1 } such that δ p ( d ) = ( c, i ). The following notation will be convenient. For each c = ( c , . . . , c n ) ∈ I ∗ and d = ( d , . . . , d m ) ∈ I ∗ , we let c, d = ( c , . . . , c n , d , . . . , d m ) denote the microstateobtained by concatenation. If d contains a single class- i customer, that is d = ( i ), then we simply write c, i for c, ( i ) and i, c for ( i ) , c . For each sequence c = ( c , . . . , c n ) ∈ I ∗ and positions p, q ∈ { , . . . , n } with p ≤ q ,we let c p...q = ( c p , . . . , c q ). Finally, we adopt the convention that c p...q = ∅ if p > q .Now that all required notation is introduced, we proceed with the identification. Let c = ( c , . . . , c n ) ∈ I ∗ and i ∈ I . Furthermore, we set q = n + 1 and i = i . Moving from tail to head in microstate c (that is, fromposition n to position 1), we determine the positions and classes of the customers that may be involved ina transition that leads to microstate c by the departure of a class- i customer. We now distinguish betweenmultiple cases, based on the total number v of customers that move during the transition: Case v = 1 : A single customer was involved in the transition, namely the customer of class i = i thatleft. By definition of the P&S mechanism, this customer is the one that completed service and itcould not replace any subsequent customer in the queue. Therefore, if microstate c contains anyclass that can be swapped with class i , then the departing customer of class i was necessarily in aposition p ∈ { q + 1 , . . . , n + 1 } before the transition, where q is the largest integer q ∈ { , . . . , n } such that c q ∈ I i . If microstate c does not contain any such customer, we let q = 0. In bothcases, before the departure, the queue could be any microstate of the form d = c ...p − , i , c p...n , where p ∈ { q + 1 , . . . , n + 1 } . Case v = 2 : If two customers were involved in the transition, this means that the departing customer ofclass i was ejected by a second customer whose service was completed. The P&S mechanism and thesymmetric property of the swapping relation impose that this second customer is the one we have just33dentified, in position q , and that q ≥
1. We let i = c q ∈ I i denote the class of this second customer.By the same argument as before, this second customer could be in any position p ∈ { q + 1 , . . . , q } before the transition, where q is the largest integer q ∈ { , . . . , q − } such that c q ∈ I i , if any, and q = 0 otherwise. In both cases, before the departure, the queue could be in any microstate of theform d = c ...p − , i , c p...q − , i , c q +1 ...n , where p ∈ { q + 1 , . . . , q } . Case v = 3 : The departing customer was ejected by a second customer, which was ejected by a third cus-tomer whose service was completed. Pursuing the previous reasoning, we can show that the secondinvolved customer is that of class i = c q , in position q , and the third involved customer is that ofclass i = c q , in position q , assuming that 1 ≤ q < q . Before the transition, this third customercould be in any position p ∈ { q + 1 , . . . , q } , where q is the largest integer q ∈ { , . . . , q − } suchthat c q ∈ I i , if any, and q = 0 otherwise. Before the departure, the queue could be in any microstate d = c ...,p − , i , c p...q − , i , c q +1 ...q − , i , c q +1 ...n , where p ∈ { q + 1 , . . . , q } .Continuing on, we build a decreasing sequence n + 1 = q > q > q > . . . > q u − > q u = 0 of positionsin microstate c using the recursion q v = max { q ≤ q v − − c q ∈ I i v − } for each v ∈ { , . . . , u − } . Therecursion stops when the set { q ≤ q v − − c q ∈ I i v − } is empty, in which case we let u = v and q u = 0.This integer u gives the maximum number of customers that can be involved in the transition (including thedeparting class- i customer). We also define a sequence i = i, i = c q , i = c q , . . . , i u − = c q u − of classes.In the end, the microstates d that lead to microstate c by a departure of a class- i customer are those of theform d = c ...p − , i v , c p...q v − , i v − , c q v +1 ...q v − − , . . . , c q +1 ...q − , i , c q +1 ...q − , i , c q +1 ...n with v ∈ { , . . . , u − } and p ∈ { q v +1 + 1 , q v +1 + 2 , . . . , q v } , where p gives the position of the customer, ofclass i v , whose service was actually completed. This implies that (3.3) can be rewritten as follows:Φ( c ) = u − X v =0 q v X p = q v +1 +1 Φ( c ...p − , i v , c p...q v − , i v − , c q v +1 ...q v − − , i v − ,. . . , c q +1 ...q − , i , c q +1 ...q − , i , c q +1 ...n ) × ∆ µ ( c ...p − , i v ) , (A.1)That the balance function Φ defined by (2.4) satisfies (A.1) is shown in the following lemma, which concludesthe proof of Theorem 3.2. Lemma A.1.
The function Φ defined by (2.4) satisfies (A.1) for each integers n ≥ and u ∈ { , . . . , n + 1 } ,microstate c = ( c , . . . , c n ) ∈ I n , class i ∈ I , and decreasing integer sequence q , q , . . . , q u with q = n + 1 and q u = 0 , where i = i, i = c q , i = c q , . . . , i u − = c q u − .Proof of the lemma. Our proof is by induction on the maximum number u ≥ u :Equation (A.1) is satisfied for each integer n ≥ u −
1, microstate c = ( c , . . . , c n ) ∈ I n , class i ∈ I ,and decreasing integer sequence q , q , . . . , q u with q = n + 1 and q u = 0.Before we proceed to the proof by induction, recall that Φ satisfies the following equation, which is a rewrittenversion of Equation (2.9) shown in the proof of Theorem 2.4:Φ( c ) = n +1 X p =1 Φ( c ...p − , i, c p...n ) ∆ µ ( c ...p − , i ) , ∀ n ≥ , ∀ c = ( c , . . . , c n ) ∈ I n , ∀ i ∈ I . (A.2) Base step.
With u = 1, (A.1) is equivalent to (A.2) with i = i . As we have just mentioned, it was alreadyshown that Φ satisfies this equation. 34 nduction step. Now let u ≥ u ′ ∈ { , , . . . , u − } .Consider an integer n ≥ u −
1, a microstate c = ( c , . . . , c n ) ∈ I n , a class i ∈ I , and a decreasing integersequence q , q , . . . , q u with q = n + 1 and q u = 0. Also let i = i, i = c q , i = c q , . . . , i u − = c q u − . Wefirst apply (A.2) to microstate c and class i = i and split the sum into two parts to obtainΦ( c ) = q X p =1 Φ( c ...p − , i , c p...n ) ∆ µ ( c ...p − , i ) + n +1 X p = q +1 Φ( c ...p − , i , c p...n ) ∆ µ ( c ...p − , i ) . Using the definition (2.4) of Φ and the fact that µ is order independent, we rewrite the first sum differently:Φ( c ) = n Y p = q µ ( c ...p , i ) ! q X p =1 Φ( c ...p − , i , c p...q − ) ∆ µ ( c ...p − , i )+ n +1 X p = q +1 Φ( c ...p − , i , c p...n ) ∆ µ ( c ...p − , i ) . (A.3)But applying (A.2) to microstate c ...q − and class i yieldsΦ( c ...q − ) = q X p =1 Φ( c ...p − , i , c p...q − ) ∆ µ ( c ...p − , i ) , so that (A.3) can be rewritten asΦ( c ) = n Y p = q µ ( c ...p , i ) ! Φ( c ...q − ) + n +1 X p = q +1 Φ( c ...p − , i , c p...n ) ∆ µ ( c ...p − , i ) . (A.4)Now we apply the induction assumption to the positive integer u ′ = u −
1, with the integer n ′ = q −
1, themicrostate c ′ = c ...q − , the class i , the decreasing sequence q ′ = q = n ′ + 1, q ′ = q , . . . , q ′ u − = q u − , q ′ u ′ = q ′ u − = q u = 0, and the indices i ′ = i , i ′ = i , . . . , i ′ u ′ − = i ′ u − = i u − . We can verify that n ′ ≥ u ′ − q , q , . . . , q u is decreasing with q u = 0, so that q ≥ q + 1 ≥ q + 2 ≥ . . . ≥ q u + ( u −
1) = u −
1. For this setting, (A.1) implies thatΦ( c ...q − ) = u − X v =1 q v X p = q v +1 +1 Φ( c ...p − , i v , c p...q v − , i v − , c q v +1 ...q v − − , i v − ,. . . , c q +1 ...q − , i , c q +1 ...q − , i , c q +1 ...q − ) × ∆ µ ( c ...p − , i v ) . Note that the first sum ranges from 1 to u −
1, and not from 0 to u ′ − u −
2, as a result of rewriting.Doing the substitution in (A.4) yieldsΦ( c ) = n Y p = q µ ( c ...p , i ) ! u − X v =1 q v X p = q v +1 +1 Φ( c ...p − , i v , c p...q v − , i v − , c q v +1 ...q v − − , i v − ,. . . , c q +1 ...q − , i , c q +1 ...q − , i , c q +1 ...q − ) × ∆ µ ( c ...p − , i v )+ n +1 X p = q +1 Φ( c ...p − , i , c p...n ) ∆ µ ( c ...p − , i ) .
35e again apply (2.4) and the fact that µ is order independent to move the product back into the first sum,so that we obtain Φ( c ) = u − X v =1 q v X p = q v +1 +1 Φ( c ...p − , i v , c p...q v − , i v − , c q v +1 ...q v − − , i v − ,. . . , c q +1 ...q − , i , c q +1 ...q − , i , c q +1 ...n ) × ∆ µ ( c ...p − , i v )+ n +1 X p = q +1 Φ( c ...p − , i , c p...n ) ∆ µ ( c ...p − , i ) . We conclude by observing that the second sum corresponds to the missing term v = 0 in the first sum. A.2 Proof of Theorem 4.1
We first prove that (4.2) is a necessary condition for stability by arguing that the Markov process describingthe microstate of the queue over time cannot be ergodic in the absence of this condition. Then, we provethat this condition is sufficient, by comparing the P&S queue to a degenerate queue with pessimistic servicerates.
Necessary condition
Assume that there is a non-empty set
A ⊆ I such that ¯ µ ( A ) ≤ P i ∈A λ i . Since µ isnon-decreasing, this means that µ ( x ) ≤ P i ∈A λ i for each x ∈ N I such that { i ∈ I : x i > } ⊆ A . Combiningthis inequality with (2.4) yields that, for any such x , and for each c ∈ I ∗ such that | c | = x , we haveΦ( c ) ≥ (cid:18) P i ∈A λ i (cid:19) | c | + ... + | c | I = (cid:18) P i ∈A λ i (cid:19) x + ... + x I , which implies X c ∈I ∗ : | c | = x Φ( c ) Y i ∈I λ i | c | i ≥ X c ∈I ∗ : | c | = x Y i ∈A λ i P j ∈A λ j ! x i = (cid:18) x + . . . + x I x , . . . , x I (cid:19) Y i ∈A λ i P j ∈A λ j ! x i . (A.5)It follows that X c ∈I ∗ Φ( c ) Y i ∈I λ i | c | i = X x ∈ N I X c ∈I ∗ : | c | = x Φ( c ) Y i ∈I λ i | c | i , ≥ X x ∈ N I : { i ∈I : x i > }⊆A X c ∈I ∗ : | c | = x Φ( c ) Y i ∈I λ i | c | i , ≥ X x ∈ N I : { i ∈I : x i > }⊆A (cid:18) x + . . . + x I x , . . . , x I (cid:19) Y i ∈A λ i P j ∈A λ j ! x i , = ∞ X n =0 X i ∈A λ i P j ∈A λ j ! n . In the first inequality, we restricted the outer sum so that we can apply (A.5). In the final equality, we usedthe multinomial theorem, stating that, for each positive integers n and N and reals ρ , ρ , . . . , ρ N , we have( ρ + . . . + ρ N ) n = X x + ... + x N = n (cid:18) nx , . . . , x N (cid:19) N Y i =1 ρ x i i . Since P i ∈A λ i P j ∈A λ j = 1, the final expression amounts to infinity, so that P c ∈I ∗ Φ( c ) Q i ∈I λ i | c | i = ∞ . Thisensures that (2.5) is not satisfied, so that the Markov process on I ∗ cannot be ergodic.36 ufficient condition Assuming that (4.2) is satisfied, we prove stability in two steps. We first introducea second P&S queue with the same set I of classes and arrival rates λ , . . . , λ I as the original P&S queue,but with a rate function ˆ µ such that ˆ µ ( x ) ≤ µ ( x ) for each x ∈ N I . We will refer to this second P&S queueas the degenerate P&S queue, as the service rate received by the customers of each class only depends onthe number of customers of this class. Then we will show that the degenerate P&S queue is stable. Sincethe degenerate P&S queue has more pessimistic service rates than the original P&S queue, this also impliesthat the original P&S queue is stable, as we will see below.We first introduce several quantities that will be useful to define the degenerate P&S queue. Since ¯ µ satisfies (4.2), there exists an m ∈ N such that X i ∈A λ i < µ ( me A ) , ∀A ⊆ I : A 6 = ∅ . We can also find ˆ λ = (ˆ λ , . . . , ˆ λ I ) ∈ R I + such that λ i < ˆ λ i for each i ∈ I , and X i ∈A ˆ λ i < µ ( me A ) , ∀A ⊆ I : A 6 = ∅ . For instance, we can chooseˆ λ i = λ i + 12 min A⊆I : i ∈A (cid:18) µ ( me A ) − P j ∈A λ j |A| (cid:19) , ∀ i ∈ I . Finally, we let δ = 1 I min min x ∈ N I \{ } ( µ ( x )) , min A⊆I : A6 = ∅ µ ( me A ) − X i ∈A ˆ λ i !! . (A.6)The definitions of µ and ˆ λ guarantee that δ >
0. In the degenerate P&S queue, δ will be the service rate ofthe customer classes that have less than m present customers. As we will see later, choosing this value of δ ensures that the service rate of the degenerate queue is always smaller or equal to that of the original queue.The degenerate P&S queue is defined as follows. Just like the original P&S queue, the set of customerclasses is I = { , . . . , I } and the per-class arrival rates are λ , . . . , λ I . But the rate function ˆ µ of this newqueue is defined on N I by ˆ µ ( x ) = P i ∈I ˆ µ i ( x i ), withˆ µ i ( x i ) = x i = 0,min( δ, ˆ λ i ) if x i = 1 , , . . . , m − λ i if x i = m, m + 1 , . . . In this way, for each i ∈ I , the oldest class- i customer is served at rate min( δ, ˆ λ i ) and the m -th oldest class- i customer is served at rate max(ˆ λ i − δ, i customers is zero. It follows that,for each x ∈ N I \ { } , we have ˆ µ ( x ) ≤ µ ( x ). Indeed, • if x i < m for each i ∈ I , then ˆ µ ( x ) ≤ X i ∈I : x i > δ ≤ X i ∈I : x i > I µ ( x ) ≤ µ ( x ) , where the first inequality follows from the definition of ˆ µ and the second holds by (A.6); • otherwise, with A = { i ∈ I : x i ≥ m } , we have A 6 = ∅ and x ≥ me A , so thatˆ µ ( x ) ≤ X i ∈A ˆ λ i + X i ∈I\A : x i > δ ≤ X i ∈A ˆ λ i + X i ∈I\A : x i > µ ( me A ) − P j ∈A ˆ λ j I ≤ µ ( me A ) ≤ µ ( x ) , where the first inequality follows from the definition of ˆ µ , the second holds by (A.6), and the fourthfollows from the monotonicity of µ . 37e let ˆΦ denote the balance function of the degenerate P&S queue, as defined in (2.4). Theorems 2.4 and3.2 now guarantee that the original P&S queue is stable whenever the degenerate P&S queue is. Moreparticularly, by (2.4), we have for each ( c , . . . , c n ) ∈ I ∗ :Φ( c , . . . , c n ) = n Y p =1 µ ( c , . . . , c p ) ≤ n Y p =1 µ ( c , . . . , c p ) = ˆΦ( c , . . . , c n ) , which implies that X c ∈I ∗ Φ( c ) Y i ∈I λ i | c | i ≤ X c ∈I ∗ ˆΦ( c ) Y i ∈I λ i | c | i . Therefore, according to (2.5), the original P&S queue is stable whenever the degenerate P&S queue is. Ittherefore only remains to show that the degenerate P&S queue is stable. To prove this, we first write: X c ∈I ∗ ˆΦ( c ) Y i ∈I λ i | c | i = X x ∈{ , ,...,m − } I X c ∈I ∗ : | c | = x ˆΦ( c ) Y i ∈I λ ix i + X A⊆I : A6 = ∅ X x ∈ N I : x i ≥ m, ∀ i ∈A ,x i A ⊆ I , we have X x ∈ N I : x i ≥ m, ∀ i ∈A ,x i As mentioned in Remark 5.7, a product-form stationary distribution can also be found for closed P&S queuesin which the initial microstate does not adhere to a placement order. To do so, we first associate, with eachclosed P&S queue, another closed P&S queue. We call this other queue the associated isomorphic queue .This isomorphic queue has the same dynamics as the original queue but its set of customer classes is different.The initial microstate of this isomorphic queue does adhere to a placement order by construction, so thatPropositions 5.2 and 5.3 and Theorem 5.5 can be applied. This in its turn leads to a stationary distributionfor the original closed P&S queue in the general case.38 .3.1 The isomorphic queue We first define, for any closed P&S queue, its associated isomorphic queue. If the initial microstate of theoriginal queue contains a single customer of each class, as in the example of Section 5.1.1, its associatedisomorphic queue is the queue itself. We now describe how the isomorphic queue is constructed if the initialmicrostate of the original queue contains two or more customers of the same class.Let c = ( c , . . . , c n ) denote the initial microstate of the queue and consider a class i ∈ I and twopositions p, q ∈ { , . . . , n } such that c p = c q = i and p < q . We introduce an extra class j (so that I is replaced with I ∪ { j } ) that has the same characteristics as class i . More specifically, we impose that∆ µ ( d , . . . , d m , i ) = ∆ µ ( d , . . . , d m , j ) for each d = ( d , . . . , d m ) ∈ I ∗ . In the swapping graph, for each k ∈ I \ { i, j } , we add an edge between classes j and k if and only if there is an edge between classes i and k .Moving towards a setting where all customers have different classes, we alter the initial microstate c bychanging the class of the customer in position q from i to j . While c p and c q are not equal anymore, thedefinition of class j guarantees that the dynamics of the queue remain the same. This procedure can berepeated with newly selected class i and positions p and q as long as there are at least two customers withthe same class in microstate c . The queue obtained once all customers have different classes is called theisomorphic queue. If ¯ c is the initial microstate of the isomorphic queue obtained by repeating this procedure,we say that microstate ¯ c in the isomorphic queue corresponds to microstate c in the original queue. Example A.2 . We now illustrate the construction of an isomorphic queue by means of the closed P&S queuedepicted in Figure 16. This queue has six customers belonging to three classes. There are two class-1customers, three class-2 customers, and one class-3 customer. The initial microstate of the queue, shown inFigure 16b, is ( c , c , c , c , c , c ) = (1 , , , , , c = c = 1, we introduce an extra class 1 ′ suchthat ∆ µ ( d , . . . , d m , 1) = ∆ µ ( d , . . . , d m , ′ ) for each microstate d = ( d , . . . , d m ) ∈ I ∗ , with I = { , , } .Moreover, in the swapping graph, we add edges between class 1 ′ and classes 2 and 3. Finally, we change theclass of the customer in position 3 to 1 ′ . This procedure has no effect on the future dynamics of the queuebut the customers in positions 1 and 3 are now the only members of their respective classes. The result isnot yet an isomorphic queue since, for example, the customers in positions 2 and 4 are both of class 2. Wetherefore iterate this procedure, changing the class of the customer in position 4 into class 2 ′ and adding anedge between class 2 ′ and classes 1, 1 ′ , and 3 in the swapping graph. After this action, only the customersin positions 2 and 5 belong to the same class. Changing the class of the customer in position 5 to an extraclass 2 ′′ , with the same characteristics as class 2, yields the isomorphic queue shown in Figure 16. Microstate(1 , , ′ , ′ , ′′ , 3) in the isomorphic queue now corresponds to microstate (1 , , , , , 3) in the original queue.While the isomorphic queue has by construction the same dynamics as the original P&S queue, it hasthe following useful property. Lemma A.3. Every microstate of an isomorphic queue adheres to a unique placement order.Proof. To prove this lemma, we show how to construct a placement graph for any microstate, which definesthe placement order to which the state adheres, and moreover show that it is unique. Recall that a placementgraph is an acyclic orientation of the swapping graph, and consists of as many vertices as there are customersin the isomorphic queue. Therefore, to construct the placement graph, the edges of the swapping graph needto be given an orientation. Recall that all customer classes appear exactly once in a state of an isomorphicqueue. Therefore, we have that, for any two customer classes i and j for which an edge ( i, j ) exists in theswapping graph, the placement order ≺ of the placement graph to be constructed should satisfy i ≺ j or j ≺ i , depending on whether or not the class- i customer is nearer to the front of the queue than the class- j customer. Orienting each edge accordingly yields a directed graph. Due to the transitivity of the order ofcustomers in the microstate, this directed graph must be acyclic, and therefore it is a placement graph towhich the microstate adheres. Note that, if any edge’s orientation corresponding to this placement orderwould be flipped, the corresponding pair of positions in the microstate will violate the new orientation. Thisproves the uniqueness, finalizing the proof. 39 23 (a) Swapping graph of the original queue. (b) A state of the original queue. ′ ′ ′′ (c) Swapping graph of the isomorphic queue. ′′ ′ ′ (d) State of the isomorphic queuecorresponding to Figure 16b. Figure 16: A closed P&S queue and its isomorphic queue. The colors and shades are visual aids that helpdistinguish classes. A.3.2 Stationary distribution Now that the isomorphic queue has been introduced, we can derive the stationary distribution for a closedP&S queue of which the initial microstate is not necessarily adhering. We can do this because the initialmicrostate of the isomorphic queue does necessarily adhere to a placement order, say ≺ . Let ¯ ℓ be themacrostate of the isomorphic queue corresponding to the initial macrostate ℓ of the closed P&S queue. Wedenote by ¯ C the set of microstates ¯ c in the isomorphic queue that adhere to ≺ and satisfy | ¯ c | = ¯ ℓ . Also let C denote the set of microstates of the original closed P&S queue to which the microstates in ¯ C correspond.Finally, for each c ∈ C , let ¯ C c ⊂ ¯ C denote the set of microstates ¯ c ∈ ¯ C in the isomorphic queue that correspondto microstate c in the original queue. Note that ¯ C c may consist of multiple elements and that, consideringall c ∈ C , the sets ¯ C c form a partition of ¯ C . For example, a microstate (1 , , , 3) of a closed P&S queuemay have corresponding microstates (1 , , ′ , 3) and (1 , ′ , , 3) in the isomorphic queue, both adhering to thesame placement order. Importantly, since all microstates in C correspond to the same macrostate ℓ , the sets¯ C c for all c ∈ C have the same cardinality. These definitions allow us to derive the stationary distribution ofthe Markov process associated with the microstate of the original P&S queue. Theorem A.4. The results of Theorem 5.5 remain valid if C refers to the set of microstates of the originalP&S queue to which the isomorphic microstates in ¯ C correspond.Proof. By Lemma A.3, the initial microstate of the isomorphic queue must adhere to a placement orderthat we denote by ≺ . Therefore, applying Theorem 5.5 to the isomorphic queue shows that the stationarydistribution of the Markov process associated with its microstate is given by¯ π (¯ c ) = ¯Φ(¯ c ) P ¯ d ∈ ¯ C ¯Φ( ¯ d ) = ¯Φ(¯ c ) P d ∈C P ¯ d ∈ ¯ C d ¯Φ( ¯ d ) , ∀ ¯ c ∈ ¯ C , where ¯Φ(¯ c ) = Φ( c ) for each c ∈ C and ¯ c ∈ ¯ C c .By construction of the isomorphic queue, the dynamics of the original P&S queue and its isomorphicqueue are the same. As such, the stationary probability of the original queue residing in microstate c is equal40o the stationary probability of the isomorphic queue residing in any microstate of ¯ C c , leading to: π ( c ) = X ¯ c ∈ ¯ C c ¯ π (¯ c ) = P ¯ c ∈ ¯ C c ¯Φ(¯ c ) P d ∈C P ¯ d ∈ ¯ C d ¯Φ( ¯ d ) , ∀ c ∈ C . Equation (5.1) follows by recalling that ¯Φ(¯ c ) = Φ( c ) for each ¯ c ∈ ¯ C c and that all sets ¯ C c have the samecardinality. Remark A.5 . By Remark 5.6, the isomorphic queue cannot have transient states. Since an isomorphic queuewith identical dynamics can be constructed for any closed P&S queue, this implies that the Markov processassociated with the microstate of any closed P&S queue, regardless of any adherence of its initial state,cannot have transient states either. As a result, Theorem A.4 now implies a partition of the complete statespace I ∗ in closed communicating classes, each of which corresponds to a set ¯ C defined by a combination ofa macrostate ¯ ℓ and a particular placement order ≺≺