Marina A. Epelman

Call Center Staffing with Simulation and Cutting Plane Methods

We present an iterative cutting plane method for minimizing staffing costs in a service system subject to satisfying acceptable service level requirements over multiple time periods. We assume that the service level cannot be easily computed, and instead is evaluated using simulation. The simulation uses the method of common random numbers, so that the same sequence of random phenomena is observed when evaluating different staffing plans. In other words, we solve a sample average approximation problem. We establish convergence of the cutting plane method on a given sample average approximation. We also establish both convergence, and the rate of convergence, of the solutions to the sample average approximation to solutions of the original problem as the sample size increases. The cutting plane method relies on the service level functions being concave in the number of servers. We show how to verify this requirement as our algorithm proceeds. A numerical example showcases the properties of our method, and sheds light on when the concavity requirement can be expected to hold.

Optimizing Call Center Staffing Using Simulation and Analytic Center Cutting-Plane Methods

We consider the problem of minimizing staffing costs in an inbound call center, while maintaining an acceptable level of service in multiple time periods. The problem is complicated by the fact that staffing level in one time period can affect the service levels in subsequent periods. Moreover, staff schedules typically take the form of shifts covering several periods. Interactions between staffing levels in different time periods, as well as the impact of shift requirements on the staffing levels and cost, should be considered in the planning. Traditional staffing methods based on stationary queueing formulas do not take this into account. We present a simulation-based analytic center cutting-plane method to solve a sample average approximation of the problem. We establish convergence of the method when the service-level functions are discrete pseudoconcave. An extensive numerical study of a moderately large call center shows that the method is robust and, in most of the test cases, outperforms traditional staffing heuristics that are based on analytical queueing methods.

Condition number complexity of an elementary algorithm for computing a reliable solution of a conic linear system

Abstract.A conic linear system is a system of the form¶¶(FPd)Ax=b¶x∈CX,¶¶where A:X?Y is a linear operator between n- and m-dimensional linear spaces X and Y, b∈Y, and CX⊂X is a closed convex cone. The data for the system is d=(A,b). This system is “well-posed” to the extent that (small) changes in the data d=(A,b) do not alter the status of the system (the system remains feasible or not). Renegar defined the “distance to ill-posedness,”ρ(d), to be the smallest change in the data Δd=(ΔA,Δb) needed to create a data instance d+Δd that is “ill-posed,” i.e., that lies in the intersection of the closures of the sets of feasible and infeasible instances d′=(A′,b′) of (FP(·)). Renegar also defined the condition number ?(d) of the data instance d as the scale-invariant reciprocal of ρ(d) : ?(d)=.¶In this paper we develop an elementary algorithm that computes a solution of (FPd) when it is feasible, or demonstrates that (FPd) has no solution by computing a solution of the alternative system. The algorithm is based on a generalization of von Neumann’s algorithm for solving linear inequalities. The number of iterations of the algorithm is essentially bounded by¶¶O( ?(d)2ln(?(d)))¶¶where the constant depends only on the properties of the cone CX and is independent of data d. Each iteration of the algorithm performs a small number of matrix-vector and vector-vector multiplications (that take full advantage of the sparsity of the original data) plus a small number of other operations involving the cone CX. The algorithm is “elementary” in the sense that it performs only a few relatively simple computations at each iteration.¶The solution of the system (FPd) generated by the algorithm has the property of being “reliable” in the sense that the distance from to the boundary of the cone CX, dist(,∂CX), and the size of the solution, ∥∥, satisfy the following inequalities:¶¶∥∥≤c1?(d),dist(,∂CX)≥c2, and ≤c3?(d),¶¶where c1, c2, c3 are constants that depend only on properties of the cone CX and are independent of the data d (with analogous results for the alternative system when the system (FPd) is infeasible).

CoSIGN: A Parallel Algorithm for Coordinated Traffic Signal Control

The problem of finding optimal coordinated signal timing plans for a large number of traffic signals is a challenging problem because of the exponential growth in the number of joint timing plans that need to be explored as the network size grows. In this paper, the game-theoretic paradigm of fictitious play to iteratively search for a coordinated signal timing plan is employed, which improves a system-wide performance criterion for a traffic network. The algorithm is robustly scalable to realistic-size networks modeled with high-fidelity simulations. Results of a case study for the city of Troy, MI, where there are 75 signalized intersections, are reported. Under normal traffic conditions, savings in average travel time of more than 20% are experienced against a static timing plan, and even against an aggressively tuned automatic-signal-retiming algorithm, savings of more than 10% are achieved. The efficiency of the algorithm stems from its parallel nature. With a thousand parallel CPUs available, the algorithm finds the plan above under 10 min, while a version of a hill-climbing algorithm makes virtually no progress in the same amount of wall-clock computational time

Stochastic programming for off-line adaptive radiotherapy

In intensity-modulated radiotherapy (IMRT), a treatment is designed to deliver high radiation doses to tumors, while avoiding the healthy tissue. Optimization-based treatment planning often produces sharp dose gradients between tumors and healthy tissue. Random shifts during treatment can cause significant differences between the dose in the “optimized” plan and the actual dose delivered to a patient. An IMRT treatment plan is delivered as a series of small daily dosages, or fractions, over a period of time (typically 35 days). It has recently become technically possible to measure variations in patient setup and the delivered doses after each fraction. We develop an optimization framework, which exploits the dynamic nature of radiotherapy and information gathering by adapting the treatment plan in response to temporal variations measured during the treatment course of a individual patient. The resulting (suboptimal) control policies, which re-optimize before each fraction, include two approximate dynamic programming schemes: certainty equivalent control (CEC) and open-loop feedback control (OLFC). Computational experiments show that resulting individualized adaptive radiotherapy plans promise to provide a considerable improvement compared to non-adaptive treatment plans, while remaining computationally feasible to implement.

Sampled fictitious play for approximate dynamic programming

Sampled fictitious play (SFP) is a recently proposed iterative learning mechanism for computing Nash equilibria of non-cooperative games. For games of identical interests, every limit point of the sequence of mixed strategies induced by the empirical frequencies of best response actions that players in SFP play is a Nash equilibrium. Because discrete optimization problems can be viewed as games of identical interests wherein Nash equilibria define a type of local optimum, SFP has recently been employed as a heuristic optimization algorithm with promising empirical performance. However, there have been no guarantees of convergence to a globally optimal Nash equilibrium established for any of the problem classes considered to date. In this paper, we introduce a variant of SFP and show that it converges almost surely to optimal policies in model-free, finite-horizon stochastic dynamic programs. The key idea is to view the dynamic programming states as players, whose common interest is to maximize the total multi-period expected reward starting in a fixed initial state. We also offer empirical results suggesting that our SFP variant is effective in practice for small to moderate sized model-free problems.

A sampled fictitious play based learning algorithm for infinite horizon Markov decision processes

Using Sampled Fictitious Play (SFP) concepts, we develop SFPL: Sampled Fictitious Play Learning — a learning algorithm for solving discounted homogeneous Markov Decision Problems where the transition probabilities are unknown and need to be learned via simulation or direct observation of the system in real time. Thus, SFPL simultaneously updates the estimates of the unknown transition probabilities and the estimates of optimal value and optimal action in the observed state. In the spirit of SFP, the action after each transition is selected by sampling from the empirical distribution of previous optimal action estimates for the current state. The resulting algorithm is provably convergent. We compare its performance with other learning methods, including SARSA and Q-learning.

Assignment of swimmers to dual meet events

Every fall, thousands of high school swimming coaches across the country begin the arduous process of preparing their athletes for competition. With a grueling practice schedule and a dedicated group of athletes, a coach can hone the squad into a cohesive unit, poised for any competition. However, oftentimes all preparation is in vain, as coaches assign swimmers to events with a lineup that is far from optimal. This paper provides a model that may help a high school (or other level) swim team coach make these assignments. Following state and national guidelines for swim meets, we describe a binary integer model that determines an overall assignment that maximizes the total number of points scored by the squad based on the times for swimmers on the squad and for the expected opponent.

Simplex Algorithm for Countable-State Discounted Markov Decision Processes.

We consider discounted Markov decision processes (MDPs) with countably-infinite state spaces, finite action spaces, and unbounded rewards. Typical examples of such MDPs are inventory management and queueing control problems in which there is no specific limit on the size of inventory or queue. Existing solution methods obtain a sequence of policies that converges to optimality in value but may not improve monotonically, ie., a policy in the sequence may be worse than preceding policies. Our proposed approach considers countably-infinite linear programming (CILP) formulations of the MDPs (a CILP is defined as a linear program (LP) with countably-infinite numbers of variables and constraints). Under standard assumptions for analyzing MDPs with countably-infinite state spaces and unbounded rewards, we extend the major theoretical extreme point and duality results to the resulting CILPs. Under additional mild assumptions, which are satisfied by several applications of interest, we present a simplex-type algor...

VMATc: VMAT with constant gantry speed and dose rate

This article considers the treatment plan optimization problem for Volumetric Modulated Arc Therapy (VMAT) with constant gantry speed and dose rate (VMATc). In particular, we consider the simultaneous optimization of multi-leaf collimator leaf positions and a constant gantry speed and dose rate. We propose a heuristic framework for (approximately) solving this optimization problem that is based on hierarchical decomposition. Specifically, an iterative algorithm is used to heuristically optimize dose rate and gantry speed selection, where at every iteration a leaf position optimization subproblem is solved, also heuristically, to find a high-quality plan corresponding to a given dose rate and gantry speed. We apply our framework to clinical patient cases, and compare the resulting VMATc plans to idealized IMRT, as well as full VMAT plans. Our results suggest that VMATc is capable of producing treatment plans of comparable quality to VMAT, albeit at the expense of long computation time and generally higher total monitor units.