Daniel Adelman

A Price-Directed Approach to Stochastic Inventory/Routing

We consider a new approach to stochastic inventory/routing that approximates the future costs of current actions using optimal dual prices of a linear program. We obtain two such linear programs by formulating the control problem as a Markov decision process and then replacing the optimal value function with the sum of single-customer inventory value functions. The resulting approximation yields statewise lower bounds on optimal infinite-horizon discounted costs. We present a linear program that takes into account inventory dynamics and economics in allocating transportation costs for stochastic inventory routing. On test instances we find that these allocations do not introduce any error in the value function approximations relative to the best approximations that can be achieved without them. Also, unlike other approaches, we do not restrict the set of allowable vehicle itineraries in any way. Instead, we develop an efficient algorithm to both generate and eliminate itineraries during solution of the linear programs and control policy. In simulation experiments, the price-directed policy outperforms other policies from the literature.

An Approximate Dynamic Programming Approach to Network Revenue Management with Customer Choice

We consider a network revenue management problem where customers choose among open fare products according to some prespecified choice model. Starting with a Markov decision process (MDP) formulation, we approximate the value function with an affine function of the state vector. We show that the resulting problem provides a tighter bound for the MDP value than the choice-based linear program. We develop a column generation algorithm to solve the problem for a multinomial logit choice model with disjoint consideration sets (MNLD). We also derive a bound as a by-product of a decomposition heuristic. Our numerical study shows the policies from our solution approach can significantly outperform heuristics from the choice-based linear program.

Dynamic Bid Prices in Revenue Management

We formally derive the standard deterministic linear program (LP) for bid-price control by making an affine functional approximation to the optimal dynamic programming value function. This affine functional approximation gives rise to a new LP that yields tighter bounds than the standard LP. Whereas the standard LP computes static bid prices, our LP computes a time trajectory of bid prices. We show that there exist dynamic bid prices, optimal for the LP, that are individually monotone with respect to time. We provide a column generation procedure for solving the LP within a desired optimality tolerance, and present numerical results on computational and economic performance.

Relaxations of Weakly Coupled Stochastic Dynamic Programs

We consider a broad class of stochastic dynamic programming problems that are amenable to relaxation via decomposition. These problems comprise multiple subproblems that are independent of each other except for a collection of coupling constraints on the action space. We fit an additively separable value function approximation using two techniques, namely, Lagrangian relaxation and the linear programming (LP) approach to approximate dynamic programming. We prove various results comparing the relaxations to each other and to the optimal problem value. We also provide a column generation algorithm for solving the LP-based relaxation to any desired optimality tolerance, and we report on numerical experiments on bandit-like problems. Our results provide insight into the complexity versus quality trade-off when choosing which of these relaxations to implement.

Price-Directed Replenishment of Subsets: Methodology and Its Application to Inventory Routing

The idea of price-directed control is to use an operating policy that exploits optimal dual prices from a mathematical programming relaxation of the underlying control problem. We apply it to the problem of replenishing inventory to subsets of products/locations, such as in the distribution of industrial gases, so as to minimize long-run time average replenishment costs. Given a marginal value for each product/location, whenever there is a stockout the dispatcher compares the total value of each feasible replenishment with its cost, and chooses one that maximizes the surplus. We derive this operating policy using a linear functional approximation to the optimal value function of a semi-Markov decision process on continuous spaces. This approximation also leads to a math program whose optimal dual prices yield values and whose optimal objective value gives a lower bound on system performance. We use duality theory to show that optimal prices satisfy several structural properties and can be interpreted as estimates of lowest achievable marginal costs. On real-world instances, the price-directed policy achieves superior, near optimal performance as compared with other approaches.

Price-Directed Control of a Closed Logistics Queueing Network

Motivated by one of the leading intermodal logistics suppliers in the United States, we consider an internal pricing mechanism for managing a fleet of service units (shipping containers) flowing in a closed queueing network. Nodes represent geographic locations, and arcs represent travel between them. Customer requests for arcs arrive over time, and the problem is to find an accept/reject policy that maximizes the long-run time average reward rate from accepting requests. We formulate the problem as a semi-Markov decision process and give a simple linear program that provides an upper bound on the optimal reward rate. Using Palm calculus, we derive a nonlinear program that approximately captures queueing and stockout effects on the network. Using its optimal Lagrange multipliers, we construct a simple functional approximation to the dynamic programming value function. The resulting policy is computationally efficient and produces superior economic performance as compared with other policies. Furthermore, it provides a methodologically grounded solution to the firms internal pricing problem.

Computing Near-Optimal Policies in Generalized Joint Replenishment

We provide a practical methodology for solving the generalized joint replenishment (GJR) problem, based on a mathematical programming approach to approximate dynamic programming. We show how to automatically generate a value function approximation basis built upon piecewise-linear ridge functions by developing and exploiting a theoretical connection with the problem of finding optimal cyclic schedules. We provide a variant of the algorithm that is effective in practice, and we exploit the special structure of the GJR problem to provide a coherent, implementable framework.

Dynamic Capacity Allocation to Customers Who Remember Past Service

We study the problem faced by a supplier deciding how to dynamically allocate limited capacity among a portfolio of customers who remember the fill rates provided to them in the past. A customers order quantity is positively correlated with past fill rates. Customers differ from one another in their contribution margins, their sensitivities to the past, and in their demand volatilities. By analyzing and comparing policies that ignore goodwill with ones that account for it, we investigate when and how customer memory effects impact supplier profits. We develop an approximate dynamic programming policy that dynamically rationalizes the fill rates the firm provides to each customer. This policy achieves higher rewards than margin-greedy and Lagrangian policies and yields insights into how a supplier can effectively manage customer memories to its advantage. This paper was accepted by Martin Lariviere, operations management.

An Infinite-Dimensional Linear Programming Algorithm for Deterministic Semi-Markov Decision Processes on Borel Spaces

We devise an algorithm for solving the infinite-dimensional linear programs that arise from general deterministic semi-Markov decision processes on Borel spaces. The algorithm constructs a sequence of approximate primal-dual solutions that converge to an optimal one. The innovative idea is to approximate the dual solution with continuous piecewise linear ridge functions that naturally represent functions defined on a high-dimensional domain as linear combinations of functions defined on only a single dimension. This approximation gives rise to a primal/dual pair of semi-infinite programs, for which we show strong duality. In addition, we prove various properties of the underlying ridge functions.

Duality and Existence of Optimal Policies in Generalized Joint Replenishment

We establish a duality theory for a broad class of deterministic inventory control problems on continuous spaces that includes the classical joint replenishment problem and inventory routing. Using this theory, we establish the existence of an optimal policy, which has been an open question. We show how a primal-dual pair of infinite dimensional linear programs encode both cyclic and noncyclic schedules, and provide various results regarding cyclic schedules, including an example showing that they need not be optimal.