Kuang Xu

On the Power of (Even a Little) Resource Pooling

We propose and analyze a multi-server model that captures a performance trade-off between centralized and distributed processing. In our model, a fraction p of an available resource is deployed in a centralized manner (e.g., to serve a most-loaded station) while the remaining fraction 1 − p is allocated to local servers that can only serve requests addressed specifically to their respective stations. Using a fluid model approach, we demonstrate a surprising phase transition in the steady-state delay scaling, as p changes: in the limit of a large number of stations, and when any amount of centralization is available (p > 0), the average queue length in steady state scales as log11−p11−λ when the traffic intensity λ goes to 1. This is exponentially smaller than the usual M/M/1-queue delay scaling of 11−λ, obtained when all resources are fully allocated to local stations (p = 0). This indicates a strong qualitative impact of even a small degree of resource pooling. We prove convergence to a fluid limit, and ...

On the power of (even a little) centralization in distributed processing

We propose and analyze a multi-server model that captures a performance trade-off between centralized and distributed processing. In our model, a fraction p of an available resource is deployed in a centralized manner (e.g., to serve a most loaded station) while the remaining fraction 1-p is allocated to local servers that can only serve requests addressed specifically to their respective stations. Using a fluid model approach, we demonstrate a surprising phase transition in steady-state delay, as p changes: in the limit of a large number of stations, and when any amount of centralization is available (p>0), the average queue length in steady state scales as log 1/1-p 1/1-λ when the traffic intensity λ goes to 1. This is exponentially smaller than the usual M/M/1-queue delay scaling of 1/1-λ, obtained when all resources are fully allocated to local stations (p=0). This indicates a strong qualitative impact of even a small degree of centralization. We prove convergence to a fluid limit, and characterize both the transient and steady-state behavior of the finite system, in the limit as the number of stations N goes to infinity. We show that the queue-length process converges to a unique fluid trajectory (over any finite time interval, as N → ∞), and that this fluid trajectory converges to a unique invariant state vI, for which a simple closed-form expression is obtained. We also show that the steady-state distribution of the N-server system concentrates on vI as N goes to infinity.

Queueing system topologies with limited flexibility

We study a multi-server model with n flexible servers and rn queues, connected through a fixed bipartite graph, where the level of flexibility is captured by the average degree, d(n), of the queues. Applications in content replication in data centers, skill-based routing in call centers, and flexible supply chains are among our main motivations. We focus on the scaling regime where the system size n tends to infinity, while the overall traffic intensity stays fixed. We show that a large capacity region (robustness) and diminishing queueing delay (performance) are jointly achievable even under very limited flexibility (d(n) l n). In particular, when d(n) gg ln n , a family of random-graph-based interconnection topologies is (with high probability) capable of stabilizing all admissible arrival rate vectors (under a bounded support assumption), while simultaneously ensuring a diminishing queueing delay, of order ln n/ d(n), as n-> ∞. Our analysis is centered around a new class of virtual-queue-based scheduling policies that rely on dynamically constructed partial matchings on the connectivity graph.

Queueing with future information

We study an admissions control problem, where a queue with service rate 1 - <i>p</i> receives incoming jobs at rate λ ε (1?<i>p</i>, 1), and the decision maker is allowed to redirect away jobs up to a rate of <i>p</i>, with the objective of minimizing the time-average queue length. We show that the amount of information about the future has a significant impact on system performance, in the heavy-traffic regime. When the future is unknown, the optimal average queue length diverges at rate ~ log _1/1-<i>p</i> 1/1-λ, as λ → 1. In sharp contrast, when all future arrival and service times are revealed beforehand, the optimal average queue length converges to a finite constant, (1 - <i>p</i>)/<i>p</i>, as λ → 1. We further show that the finite limit of (1 - <i>p</i>)/<i>p</i> can be achieved using only a finite lookahead window starting from the current time frame, whose length scales as O(log 1/1-λ), as λ → 1. This leads to the conjecture of an interesting duality between queuing delay and the amount of information about the future.

On the Capacity of Information Processing Systems

We propose and analyze a family of information processing systems, where a finite set of experts or servers are employed to extract information about a stream of incoming jobs. Each job is associated with a hidden label drawn from some prior distribution. An inspection by an expert produces a noisy outcome that depends both on the job’s hidden label and the type of the expert, and occupies the expert for a finite time duration. A decision maker’s task is to dynamically assign inspections so that the resulting outcomes can be used to accurately recover the labels of all jobs, while keeping the system stable. Among our chief motivations are applications in crowd-sourcing, diagnostics, and experiment designs, where one wishes to efficiently discover the nature of a large number of items, using a finite pool of computational resources or human agents. We focus on the capacity of such an information processing system. Given a level of accuracy guarantee, we ask how many experts are needed in order to stabilize the system, and through what inspection architecture. Our main result provides an adaptive inspection policy that is asymptotically optimal in the following sense: the ratio between the required number of experts under our policy and the theoretical optimal converges to one, as the probability of error in label recovery tends to zero.

Flexible Queueing Architectures

We study a multiserver model with n flexible servers and n queues, connected through a bipartite graph, where the level of flexibility is captured by an upper bound on the graph’s average degree, dn. Applications in content replication in data centers, skill-based routing in call centers, and flexible supply chains are among our main motivations. We focus on the scaling regime where the system size n tends to infinity, while the overall traffic intensity stays fixed. We show that a large capacity region and an asymptotically vanishing queueing delay are simultaneously achievable even under limited flexibility (dn ≪ n). Our main results demonstrate that, when dn ≫ ln n, a family of expander-graph-based flexibility architectures has a capacity region that is within a constant factor of the maximum possible, while simultaneously ensuring a diminishing queueing delay for all arrival rate vectors in the capacity region. Our analysis is centered around a new class of virtual-queue-based scheduling policies that...

Using Future Information to Reduce Waiting Times in the Emergency Department via Diversion

The development of predictive models in healthcare settings has been growing; one such area is the prediction of patient arrivals to the emergency department (ED). The general premise behind these works is that such models may be used to help manage an ED that consistently faces high congestion. In this work, we propose a class of proactive policies that utilize future information of potential patient arrivals to effectively manage admissions into an ED while reducing waiting times for patients who are eventually treated. Instead of the standard strategy of waiting for queues to build before diverting patients, the proposed policy utilizes the predictions to identify when congestion is going to increase and proactively diverts patients before things get “too bad.” We demonstrate that the proposed policy provides delay improvements over standard policies used in practice. We also consider the impact of errors in the information provided by the predictive models and find that even with noisy predictions, ou...

Necessity of Future Information in Admission Control

We study the necessity of predictive information in a class of queueing admission control problems, where a system manager is allowed to divert incoming jobs up to a fixed rate, in order to minimize the queueing delay experienced by the admitted jobs. Spencer et al. (2014) show that the systems delay performance can be significantly improved by having access to future information in the form of a lookahead window, during which the times of future arrivals and services are revealed. They prove that, while delay under an optimal online policy diverges to infinity in the heavy-traffic regime, it can stay bounded by making use of future information. However, the diversion polices of Spencer et al. (2014) require the length of the lookahead window to grow to infinity at a non-trivial rate in the heavy-traffic regime, and it remained open whether substantial performance improvement could still be achieved with less future information. We resolve this question to a large extent by establishing an asymptotically tight lower bound on how much future information is necessary to achieve superior performance, which matches the upper bound of Spencer et al. (2014) up to a constant multiplicative factor. Our result hence demonstrates that the systems heavy-traffic delay performance is highly sensitive to the amount of future information available. Our proof is based on analyzing certain excursion probabilities of the input sample paths, and exploiting a connection between a policys diversion decisions and subsequent server idling, which may be of independent interest for related dynamic resource allocation problems.

Queuing with future information

We study an admissions control problem, where a queue with service rate

THE OPTIMAL ADMISSION THRESHOLD IN OBSERVABLE QUEUES WITH STATE DEPENDENT PRICING

1-p