Archis Ghate

A stochastic control formalism for dynamic biologically conformal radiation therapy

State-of-the-art methods for optimizing cancer treatment over several weeks of external beam radiotherapy take a static–deterministic view of the treatment planning process, mainly focusing on spatial distribution of dose. Recent progress in quantitative functional imaging as well as mathematical models of tumor response to radiotherapy is increasingly enabling treatment planners to monitor/predict a patient’s biological response over weeks of treatment. In this paper we introduce dynamic biologically conformal radiation therapy (DBCRT), a mathematical framework intended to exploit these emerging technological and biological modeling advances to design patient-specific radiation treatment strategies that dynamically adapt to the spatiotemporal evolution of a patient’s biological response over several treatment sessions in order to achieve the best possible health outcome. More specifically, we propose a discrete-time stochastic control formalism where we use the patient’s biological condition to model the system state and the beam intensities as controls. Three approximate control schemes are then applied and compared for efficiency. Numerical simulations on test cases show that DBCRT results in a 64–98% improvement in treatment efficacy as compared to the more conventional static–deterministic approach.

Adaptive search with stochastic acceptance probabilities for global optimization

We present an extension of continuous domain Simulated Annealing. Our algorithm employs a globally reaching candidate generator, adaptive stochastic acceptance probabilities, and converges in probability to the optimal value. An application to simulation-optimization problems with asymptotically diminishing errors is presented. Numerical results on a noisy protein-folding problem are included.

A Markov decision process approach to multi-category patient scheduling in a diagnostic facility

OBJECTIVES To develop a mathematical model for multi-category patient scheduling decisions in computed tomography (CT), and to investigate associated tradeoffs from economic and operational perspectives. METHODS We modeled this decision-problem as a finite-horizon Markov decision process (MDP) with expected net CT revenue as the performance metric. The performance of optimal policies was compared with five heuristics using data from an urban hospital. In addition to net revenue, other patient-throughput and service-quality metrics were also used in this comparative analysis. RESULTS The optimal policy had a threshold structure in the two-scanner case - it prioritized one type of patient when the queue-length for that type exceeded a threshold. The net revenue gap between the optimal policy and the heuristics ranged from 5% to 12%. This gap was 4% higher in the more congested, single-scanner system than in the two-scanner system. The performance of the net revenue maximizing policy was similar to the heuristics, when compared with respect to the alternative performance metrics in the two-scanner case. Under the optimal policy, the average number of patients that were not scanned by the end of the day, and the average patient waiting-time, were both nearly 80% smaller in the two-scanner case than in the single-scanner case. The net revenue gap between the optimal policy and the priority-based heuristics was nearly 2% smaller as compared to the first-come-first-served and random selection schemes. Net revenue was most sensitive to inpatient (IP) penalty costs in the single-scanner system, whereas to IP and outpatient revenues in the two-scanner case. CONCLUSIONS The performance of the optimal policy is competitive with the operational and economic metrics considered in this paper. Such a policy can be implemented relatively easily and could be tested in practice in the future. The priority-based heuristics are next-best to the optimal policy and are much easier to implement.

Dynamic Lot-sizing in Sequential Online Retail Auctions

Retailers often conduct non-overlapping sequential online auctions as a revenue generation and inventory clearing tool. We build a stochastic dynamic programming model for the sellers lot-size decision problem in these auctions. The model incorporates a random number of participating bidders in each auction, allows for any bid distribution, and is not restricted to any specific price-determination mechanism. Using stochastic monotonicity/stochastic concavity and supermodularity arguments, we present a complete structural characterization of optimal lot-sizing policies under a second order condition on the single-auction expected revenue function. We show that a monotone staircase with unit jumps policy is optimal and provide a simple inequality to determine the locations of these staircase jumps. Our analytical examples demonstrate that the second order condition is met in common online auction mechanisms. We also present numerical experiments and sensitivity analyses using real online auction data.

Discrete Hit-and-Run for Sampling Points from Arbitrary Distributions Over Subsets of Integer Hyperrectangles

We consider the problem of sampling a point from an arbitrary distribution π over an arbitrary subset S of an integer hyperrectangle. Neither the distribution π nor the support set S are assumed to be available as explicit mathematical equations, but may only be defined through oracles and, in particular, computer programs. This problem commonly occurs in black-box discrete optimization as well as counting and estimation problems. The generality of this setting and high dimensionality of S precludes the application of conventional random variable generation methods. As a result, we turn to Markov chain Monte Carlo (MCMC) sampling, where we execute an ergodic Markov chain that converges to π so that the distribution of the point delivered after sufficiently many steps can be made arbitrarily close to π. Unfortunately, classical Markov chains, such as the nearest-neighbor random walk or the coordinate direction random walk, fail to converge to π because they can get trapped in isolated regions of the support set. To surmount this difficulty, we propose discrete hit-and-run (DHR), a Markov chain motivated by the hit-and-run algorithm known to be the most efficient method for sampling from log-concave distributions over convex bodies in Rn. We prove that the limiting distribution of DHR is π as desired, thus enabling us to sample approximately from π by delivering the last iterate of a sufficiently large number of iterations of DHR. In addition to this asymptotic analysis, we investigate finite-time behavior of DHR and present a variety of examples where DHR exhibits polynomial performance.

Optimal fractionation in radiotherapy with multiple normal tissues

The goal in radiotherapy is to maximize the biological effect (BE) of radiation on the tumour while limiting its toxic effects on healthy anatomies. Treatment is administered over several sessions to give the normal tissue time to recover as it has better damage-repair capabilities than tumour cells. This is termed fractionation. A key problem in radiotherapy involves finding an optimal number of treatment sessions (fractions) and the corresponding dosing schedule. A major limitation of existing mathematically rigorous work on this problem is that it includes only a single normal tissue. Since essentially no anatomical region of interest includes only one normal tissue, these models may incorrectly identify the optimal number of fractions and the corresponding dosing schedule. We present a formulation of the optimal fractionation problem that includes multiple normal tissues. Our model can tackle any combination of maximum dose, mean dose and dose-volume type constraints for serial and parallel normal tissues as this is characteristic of most treatment protocols. We also allow for a spatially heterogeneous dose distribution within each normal tissue. Furthermore, we do not a priori assume that the doses are invariant across fractions. Finally, our model uses a spatially optimized treatment plan as input and hence can be seamlessly combined with any treatment planning system. Our formulation is a mixed-integer, non-convex, quadratically constrained quadratic programming problem. In order to simplify this computationally challenging problem without loss of optimality, we establish sufficient conditions under which equal-dosage or single-dosage fractionation is optimal. Based on the prevalent estimates of tumour and normal tissue model parameters, these conditions are expected to hold in many types of commonly studied tumours, such as those similar to head-and-neck and prostate cancers. This motivates a simple reformulation of our problem that leads to a closed-form formula for the dose per fraction. We then establish that the tumour-BE is quasiconcave in the number of fractions; this ultimately helps in identifying the optimal number of fractions. We perform extensive numerical experiments using 10 head-and-neck and prostate test cases to uncover several clinically relevant insights.

A two-variable linear program solves the standard linear-quadratic formulation of the fractionation problem in cancer radiotherapy

The standard formulation of the fractionation problem with multiple organs-at-risk based on the linear-quadratic dose-response model requires the solution of a nonconvex quadratically constrained quadratic program. Existing literature therefore uses heuristic methods without any analyses about solution quality. There is no known method that is guaranteed to find an optimal solution. We prove that this formulation of the fractionation problem can in fact be solved to optimality by instead solving a two-variable linear program with a few constraints.

A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes

Nonstationary infinite-horizon Markov decision processes (MDPs) generalize the most well-studied class of sequential decision models in operations research, namely, that of stationary MDPs, by relaxing the restrictive assumption that problem data do not change over time. Linear programming (LP) has been very successful in obtaining structural insights and devising solution methods for stationary MDPs. However, an LP approach for nonstationary MDPs is currently missing. This is because the LP formulation of a nonstationary infinite-horizon MDP includes countably infinite variables and constraints, and research on such infinite-dimensional LPs has traditionally faced several hurdles. For instance, duality results may not hold; an extreme point may not be a basic feasible solution; and in the context of a simplex algorithm, a pivot operation may require infinite data and computations, and a sequence of improving extreme points need not converge in value to optimal. In this paper, we tackle these challenges and establish (1) weak and strong duality, (2) complementary slackness, (3) a basic feasible solution characterization of extreme points, (4) a one-to-one correspondence between extreme points and deterministic Markovian policies, and (5) we devise a simplex algorithm for an infinite-dimensional LP formulation of nonstationary infinite-horizon MDPs. Pivots in this simplex algorithm use finite data, perform finite computations, and generate a sequence of improving extreme points that converges in value to optimal. Moreover, this sequence of extreme points gets arbitrarily close to the set of optimal extreme points. We also prove that decisions prescribed by these extreme points are eventually exactly optimal in all states of the nonstationary infinite-horizon MDP in early periods.

A Shadow Simplex Method for Infinite Linear Programs

We present a simplex-type algorithm---that is, an algorithm that moves from one extreme point of the infinite-dimensional feasible region to another, not necessarily adjacent, extreme point---for solving a class of linear programs with countably infinite variables and constraints. Each iteration of this method can be implemented in finite time, whereas the solution values converge to the optimal value as the number of iterations increases. This simplex-type algorithm moves to an adjacent extreme point and hence reduces to a true infinite-dimensional simplex method for the important special cases of nonstationary infinite-horizon deterministic and stochastic dynamic programs.

Multi-class, multi-resource advance scheduling with no-shows, cancellations and overbooking

We investigate a class of scheduling problems where dynamically and stochastically arriving appointment requests are either rejected or booked for future slots. A customer may cancel an appointment. A customer who does not cancel may fail to show up. The planner may overbook appointments to mitigate the detrimental effects of cancellations and no-shows. A customer needs multiple renewable resources. The system receives a reward for providing service; and incurs costs for rejecting requests, appointment delays, and overtime. Customers are heterogeneous in all problem parameters. We provide a Markov decision process (MDP) formulation of these problems. Exact solution of this MDP is intractable. We show that this MDP has a weakly coupled structure that enables us to apply an approximate dynamic programming method rooted in Lagrangian relaxation, affine value function approximation, and constraint generation. We compare this method with a myopic scheduling heuristic on eighteen hundred problem instances. Our experiments show that there is a statistically significant difference in the performance of the two methods in 77% of these instances. Of these statistically significant instances, the Lagrangian method outperforms the myopic method in 97% of the instances. HighlightsProposes a large class of advance scheduling problems with no-shows, cancellations, and overbooking.Provides a Markov decision process model for these problems and shows that it is weakly coupled.Applies an approximate dynamic programming approach rooted in Lagrangian relaxation.