Asaf Cohen

Bandit Problems with Lévy Processes

Bandit problems model the trade-off between exploration and exploitation in various decision problems. We study two-armed bandit problems in continuous time, where the risky arm can have two types: High or Low; both types yield stochastic payoffs generated by a Levy process. We show that the optimal strategy is a cut-off strategy and we provide an explicit expression for the cut-off and for the optimal payoff.

Asymptotically optimal control for a multiclass queueing model in the moderate deviation heavy traffic regime

A multi-class single-server queueing model with finite buffers, in which scheduling and admission of customers are subject to control, is studied in the moderate deviation heavy traffic regime. A risk-sensitive cost set over a finite time horizon

A Differential Game for a Multiclass Queueing Model in the Moderate-Deviation Heavy-Traffic Regime

[0,T]

Parameter Estimation: The Proper Way to Use Bayesian Posterior Processes with Brownian Noise

is considered. The main result is the asymptotic optimality of a control policy derived via an underlying differential game. The result is the first to address a queueing control problem at the moderate deviation regime that goes beyond models having the so called pathwise minimality property. Moreover, despite the well known fact that an optimal control over a finite time interval is generically of a nonstationary feedback type, the proposed policy forms a stationary feedback, provided

Bandit problems with Levy payoff processes

T

Risk Sensitive Control of the Lifetime Ruin Problem

is sufficiently large.

Rate Control under Heavy Traffic with Strategic Servers

We study a differential game that governs the moderate-deviation heavy-traffic asymptotics of a multiclass single-server queueing control problem with a risk-sensitive cost. We consider a cost set on a finite but sufficiently large time horizon, and show that this formulation leads to stationary feedback policies for the game. Several aspects of the game are explored, including its characterization via a (one-dimensional) free boundary problem, the semi-explicit solution of an optimal strategy, and the specification of a saddle point. We emphasize the analogy to the well-known Harrison-Taksar free boundary problem which plays a similar role in the diffusion-scale heavy-traffic literature.

Serve the shortest queue and Walsh Brownian motion.

This paper studies a problem of Bayesian parameter estimation for a sequence of scaled counting processes whose weak limit is a Brownian motion with an unknown drift. The main result of the paper is that the limit of the posterior distribution processes is, in general, not equal to the posterior distribution process of the mentioned Brownian motion with the unknown drift. Instead, it is equal to the posterior distribution process associated with a Brownian motion with the same unknown drift and a different standard deviation coefficient. The difference between the two standard deviation coefficients can be arbitrarily large. The characterization of the limit of the posterior distribution processes is then applied to a family of stopping time problems. We show that the proper way to find asymptotically optimal solutions to stopping time problems w.r.t.~the scaled counting processes is by looking at the limit of the posterior distribution processes rather than by the naive approach of looking at the limit of the scaled counting processes themselves. The difference between the performances can be arbitrarily large.