Shaler Stidham

Individual versus Social Optimization in Exponential Congestion Systems

We consider a stochastic congestion system modeled as a birth-death process. Customers arrive from a Poisson process. The departure rate when i customers are in the system is non-decreasing, concave, and bounded above in i. The cost structure consists of a linear holding cost and a random reward received when a customer enters the system. The system can be controlled by deciding which customers will enter. Our main result extending those of Naor, Yechiali, and Knudsen is that, with or without discounting and for a finite or infinite time horizon, the individually optimal rule calls for the customer to enter the system whenever the socially optimal rule does. We also study the properties of the optimal congestion toll, which induces customers acting in their own interest to follow a socially optimal rule.

The Allocation of Interstage Buffer Capacities in Production Lines

Abstract A flow-shop-type production line where the stations are subject to breakdown is modeled as a series of queues. The objective is to find the allocation of interstage buffer capacities that maximizes total profit. The stations, which are modeled as single-server queueing systems, have completion-time distributions of two-stage Coxian type. After a standard transformation to a phase-type state representation, the new system gives rise to a Markov chain. The balance equations for this chain are solved by successive approximations to find the steady-state probability distribution of the number of items at each station, once the buffer capacities are given. A search procedure has been employed to find the optimal buffer capacities.

The Relation between Customer and Time Averages in Queues

Brumelle has generalized the queueing formula L = λW to H = λG, where λ is the arrival rate and H and G are respectively time and customer averages of some queue statistics which have a certain relationship to each other but are otherwise arbitrary. Stidham has developed a simple proof of L = λW for each sample path, in which the only requirement is that λ and W be finite. In this note it is shown that Stidhams proof applies directly to the more general case of H = λG, provided λ and G are finite and a simple technical assumption is satisfied. The result is used to obtain time average probabilities in the queue GI/M/c/K. Finally, a counterexample is given to demonstrate that the technical assumption is not superfluous, even in the special case where H and G can be interpreted, respectively, as the time average number of units in the system and the average time spent by a unit in the system, as is the case with both L = λW and the application to the queue GI/M/c/K.

Control of arrivals to two queues in series

Abstract We consider two queues in series with input to each queue, which can be controlled by accepting or rejecting arriving customers. The objective is to maximize the discounted or average expected net benefit over a finite or infinite horizon, where net benefit is composed of (random) rewards for entering customers minus holding costs assessed against the customers at each queue. Provided that it costs more to hold a customer at the first queue than at the second, we show that an optimal policy is monotonic in the following senses: Adding a customer to either queue makes it less likely that we will accept a new customer into either queue; moreover moving a customer from the first queue to the second makes it more (less) likely that we will accept a new customer into the first (second) queue. Our model has policy implications for flow control in communication systems, industrial job shops, and traffic-flow systems. We comment on the relation between the control policies implied by our model and those proposed in the communicationa literature.

A Note On Transfer Lines With Unreliable Machines, Random Processing Times, and Finite Buffers

Abstract We comment on a recent paper by Gershwin and Herman, with particular attention to their interpretation of the blocking phenomenon in transfer lines with finite buffers.

Clearing Systems and s, S Inventory Systems with Nonlinear Costs and Positive Lead Times

Stochastic clearing systems model many different applications, including certain bulk server queues, shuttle buses, and buffers in communication systems. We extend the characterization of the optimal parameters of a generalized stochastic clearing system Stidham, S., Jr. 1977. Cost models for stochastic clearing systems. Opns. Res.25 100-127. from piece-wise linear to general convex cost-rate functions, and discuss applications to continuous-review and periodic-review s, S inventory models with nonlinear holding and storage costs and/or nonzero lead times. As in Stidham, we show that log-concavity of the sojourn function associated with the input process guarantees that the necessary conditions are also sufficient for optimality of the clearing parameters, which correspond to s and S in inventory examples. Our conditions are weaker and our model more general than those found in the previous literature on s, S inventory models.

Sample-Path Analysis of Queues

In this paper we provide a survey of sample-path methods in queueing theory, particularly in connection with “distribution-free” analysis such as (i) relations between customer averages and time averages, such as L = λW (Little’s formula); (ii) relations between the stationary distribution of a process and an imbedded process; and (iii) the phenomenon of insensitivity.

Forward recursion for Markov decision processes with skip-free-to-the-right transitions. Part I. Theory and algorithm

We consider a Markovian decision process with countable state space states 0, 1, 2,... which is skip-free to the right a transition from i to j is impossible if j >i + 1. In this type of system it is easy to calculate by forward recursion the maximal total expected reward going from state 0 to state i; the same can be done, of course, for the case where a constant g is subtracted from the one-period reward function g-revised reward. Let -wgi be the maximal total expected g-revised reward going from state 0 to state i. We show that wg· satisfies the average-reward optimality equation. If wg· satisfies a growth condition, then g = g*, the maximal average reward. For all other g, the function wg increases or decreases so fast that this cannot be the case. Thus, in principle the solution wg can be used to check if g g*, which suggests a method for approximating g* and an associated average-return optimal policy. We develop an efficient algorithm based on this idea. In a companion paper we shall show how the algorithm, or modifications of it, can be applied to some special cases, such as control of arrivals to a queue, control of the service rate, and controlled random walks.

On the convergence of successive approximations in dynamic programming with non-zero terminal reward

This paper considers the convergence of the finite-horizon optimal value functions of dynamic programming to the infinite-horizon optimal value function, when there is a non-zero terminal-reward function. The model and methods follow closely these used by Schal in a recent paper, in which a terminal reward of zero was assumed. We first present convergence conditions that are direct extensions of Schals, then related conditions in which the terminal-reward function is an upper or lower bound for the infinite-horizon optimal value function. Some applications to problems in queueing control are mentioned briefly. We also comment on the relation between our conditions and the more restrictive conditions of strongly convergent and contractive models, and present a very general result concerning uniqueness of the solution to the infinite-horizon optimality equation.This paper considers the convergence of the finite-horizon optimal value functions of dynamic programming to the infinite-horizon optimal value function, when there is a non-zero terminal-reward function. The model and methods follow closely these used by Schäl in a recent paper, in which a terminal reward of zero was assumed. We first present convergence conditions that are direct extensions of Schäls, then related conditions in which the terminal-reward function is an upper or lower bound for the infinite-horizon optimal value function. Some applications to problems in queueing control are mentioned briefly. We also comment on the relation between our conditions and the more restrictive conditions of strongly convergent and contractive models, and present a very general result concerning uniqueness of the solution to the infinite-horizon optimality equation.ZusammenfassungIn der Arbeit wird die Konvergenz von Wertfunktionen dynamischer Optimierungsprobleme mit endlichem Planungshorizont gegen die Wertfunktionen bei unendlichem Planungshorizont betrachtet, wobei die Endauszahlung verschieden von Null ist. Modell und Vorgehensweise lehnen sich an entsprechende Resultate von Schäl an für den Fall, daß die Endauszahlung Null ist. Zunächst werden Konvergenzbedingungen angegeben, welche unmittelbare Erweiterungen der Schälschen Ergebnisse sind, gefolgt von Bedingungen, bei denen die Endauszahlung eine obere oder untere Schranke für die Endauszahlung bei unendlichem Planungshorizont ist. Einige Anwendungen auf Probleme der Steuerung von Warteschlangen werden erwähnt. Ferner werden der Zusammenhang zwischen unseren Bedingungen und den restriktiveren Bedingungen bei stark konvergenten und Kontraktions-Modellen erläutert und ein sehr allgemeines Modell über die Eindeutigkeit der Lösung der Optimalitätsgleichung bei unendlichem Planungshorizont angegeben.