Arka P. Ghosh

Diffusion approximations for controlled stochastic networks: An asymptotic bound for the value function

We consider the scheduling control problem for a family of unitary networks under heavy traffic, with general interarrival and service times, probabilistic routing and infinite horizon discounted linear holding cost. A natural nonanticipativity condition for admissibility of control policies is introduced. The condition is seen to hold for a broad class of problems. Using this formulation of admissible controls and a time-transformation technique, we establish that the infimum of the cost for the network control problem over all admissible sequencing control policies is asymptotically bounded below by the value function of an associated diffusion control problem (the Brownian control problem). This result provides a useful bound on the best achievable performance for any admissible control policy for a wide class of networks.

Optimal buffer size for a stochastic processing network in heavy traffic

Abstract We consider a one-dimensional stochastic control problem that arises from queueing network applications. The state process corresponding to the queue-length process is given by a stochastic differential equation which reflects at the origin. The controller can choose the drift coefficient which represents the service rate and the buffer size b>0. When the queue length reaches b, the new customers are rejected and this incurs a penalty. There are three types of costs involved: A “control cost” related to the dynamically controlled service rate, a “congestion cost” which depends on the queue length and a “rejection penalty” for the rejection of the customers. We consider the problem of minimizing long-term average cost, which is also known as the ergodic cost criterion. We obtain an optimal drift rate (i.e. an optimal service rate) as well as the optimal buffer size b*>0. When the buffer size b>0 is fixed and where there is no congestion cost, this problem is similar to the work in Ata, Harrison and Shepp (Ann. Appl. Probab. 15, 1145–1160, 2005). Our method is quite different from that of (Ata, Harrison and Shepp (Ann. Appl. Probab. 15, 1145–1160, 2005)). To obtain a solution to the corresponding Hamilton–Jacobi–Bellman (HJB) equation, we analyze a family of ordinary differential equations. We make use of some specific characteristics of this family of solutions to obtain the optimal buffer size b*>0.

Ergodic Rate Control Problem for Single Class Queueing Networks

We consider critically loaded single class queueing networks with infinite buffers in which arrival and service rates are state (i.e., queue length) dependent and may be dynamically controlled. An optimal rate control problem for such networks with an ergodic cost criterion is studied. It is shown that the value function (i.e., optimum value of the cost) of the rate control problem for the network converges, under a suitable heavy traffic scaling limit, to that of an ergodic control problem for certain controlled reflected diffusions. Furthermore, we show that near optimal controls for limit diffusion models can be used to construct asymptotically near optimal rate control policies for the underlying physical queueing networks. The expected cost per unit time criterion studied here is given in terms of an unbounded holding cost and a linear control cost (“cost for effort”). Time asymptotics of a related uncontrolled model are studied as well. We establish convergence of invariant measures of scaled queue ...

On a directionally reinforced random walk

We consider a generalized version of a directionally reinforced random walk, which was originally introduced by Mauldin, Monticino, and von Weizsacker in [20]. Our main result is a stable limit theorem for the position of the random walk in higher dimensions. This extends a result of Horvath and Shao [13] that was previously obtained in dimension one only (however, in a more stringent functional form).

Scheduling control for Markov-modulated single-server multiclass queueing systems in heavy traffic

This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite-state Markov process that modulates the arrival and service rates in the system. Various cases are considered: fast changing environment, fixed environment, and slowly changing environment. In all cases, the arrival rates are environment dependent, whereas the service rates are environment dependent when the environment Markov process is changing fast, and are assumed to be constant in the other two cases. In each of the cases, using weak convergence analysis, in particular functional limit theorems for Poisson processes and ergodic Markov processes, it is shown that an appropriate “averaged” version of the classical

Controlled stochastic networks in heavy traffic: Convergence of value functions

c\mu

On the range of the transient frog model on ℤ

cμ-policy (the priority policy that favors classes with higher values of the product of holding cost