Belgacem Bouzaiene-Ayari

Approximate dynamic programming for high dimensional resource allocation problems

There are wide arrays of discrete resource allocation problems (buffers in manufacturing, complex equipment in electric power, aircraft and locomotives in transportation) which need to be solved over time, under uncertainty. These can be formulated as dynamic programs, but typically exhibit high dimensional state, action and outcome variables (the three curses of dimensionality). For example, we have worked on problems where the dimensionality of these variables is in the ten thousand to one million range. We describe an approximation methodology for this problem class, and summarize the problem classes where the approach seems to be working well, and research challenges that we continue to face.

The Effect of Robust Decisions on the Cost of Uncertainty in Military Airlift Operations

There are a number of sources of randomness that arise in military airlift operations. However, the cost of uncertainty can be difficult to estimate, and is easy to overestimate if we use simplistic decision rules. Using data from Canadian military airlift operations, we study the effect of uncertainty in customer demands as well as aircraft failures, on the overall cost. The system is first analyzed using the types of myopic decision rules widely used in the research literature. The performance of the myopic policy is then compared to the results obtained using robust decisions that account for the uncertainty of future events. These are obtained by modeling the problem as a dynamic program, and solving Bellman’s equations using approximate dynamic programming. The experiments show that even approximate solutions to Bellman’s equations produce decisions that reduce the cost of uncertainty.

Modeling Bus Stops in Transit Networks: A Survey and New Formulations

In this paper, we undertake a detailed study of the bus stop problem in congested transit networks. In the first part of the paper, we present and discuss the bus stop models existing in the literature. In the second part, we propose a new general model for which we prove a number of good properties and we give equivalent formulations. Then, we examine two special cases of the general model. In the first case, the line capacities are considered limited and therefore they can not be exceeded by the on-board passenger flows. In the second case, the strict capacity constraints are relaxed in order to obtain a stop model that can be easily integrated into an assignment model to predict the global passenger behavior in transit networks.

Approximate dynamic programming in transportation and logistics: a unified framework

Deterministic optimization has enjoyed a rich place in transportation and logistics, where it represents a mature field with established modeling and algorithmic strategies. By contrast, sequential stochastic optimization models (dynamic programs) have been plagued by the lack of a common modeling framework, and by algorithmic strategies that just do not seem to scale to real-world problems in transportation. This paper is designed as a tutorial of the modeling and algorithmic framework of approximate dynamic programming; however, our perspective on approximate dynamic programming is relatively new, and the approach is new to the transportation research community. We present a simple yet precise modeling framework that makes it possible to integrate most algorithmic strategies into four fundamental classes of policies, the design of which represents approximate solutions to these dynamic programs. The paper then uses problems in transportation and logistics to indicate settings in which each of the four classes of policies represents a natural solution strategy, highlighting the fact that the design of effective policies for these complex problems will remain an exciting area of research for many years. Along the way, we provide a link between dynamic programming, stochastic programming and stochastic search.

Chapter 5 Dynamic Models for Freight Transportation

Publisher Summary Dynamic models arise in a vast array of transportation applications because of the need to capture the evolution of activities over time. This chapter focuses on modeling the organization and flow of information and decisions, in the context of freight transportation problems that involve the management of people and equipment to serve the needs of customers. The timing of the flow of capital is becoming an increasingly important dimension of freight transportation, but there has been virtually no formal research on the topic. Modeling the timing of physical activities, by contrast, dates to the 1950s. These models introduce a range of modeling and algorithmic challenges that have been studied since the early years of operations research models. Dynamic models have a number of applications. A dynamic process can be simulated to better understand the way for operating a system under more realistic settings. One application that often arises at many companies is a desire to develop models to help run their operations in real time.

From Single Commodity to Multiattribute Models for Locomotive Optimization: A Comparison of Optimal Integer Programming and Approximate Dynamic Programming

We present a general optimization framework for locomotive models that captures different levels of detail, ranging from single and multicommodity flow models that can be solved using commercial integer programming solvers, to a much more detailed multiattribute model that we solve using approximate dynamic programming (ADP). Both models have been successfully implemented at Norfolk Southern for different planning applications. We use these models, presented using a common notational framework, to demonstrate the scope of different modeling and algorithmic strategies, all of which add value to the locomotive planning problem. We demonstrate how ADP can be used for both deterministic and stochastic models that capture locomotives and trains at a very high level of detail.

Locomotive Planning at Norfolk Southern: An Optimizing Simulator Using Approximate Dynamic Programming

For decades, locomotive planning has been approached using the classical tools of mathematical programming; the result has been very large-scale integer programming models that are beyond the capabilities of modern solvers but still require a host of simplifying assumptions that limit their use for analyzing important planning problems. The primary interest of Norfolk Southern was in developing a model that could assist it with fleet sizing. However, the cumulative effect of the simplifications required to produce a practical integer programming formulation resulted in models that underestimated the required fleet. We use the modeling and algorithmic framework of approximate dynamic programming, which uses an intuitive balance of simulation and optimization with feedback learning, to produce a highly detailed model that calibrates accurately against historical metrics. The result was a model that can be used to plan fleet size and mix, be sensitive to a wide range of operating parameters, and adapt to many scenarios.

Parallel Nonstationary Direct Policy Search for Risk-Averse Stochastic Optimization

This paper presents an algorithmic strategy to nonstationary policy search for finite-horizon, discrete-time Markovian decision problems with large state spaces, constrained action sets, and a risk-sensitive optimality criterion. The methodology relies on modeling time-variant policy parameters by a nonparametric response surface model for an indirect parametrized policy motivated by Bellman’s equation. The policy structure is heuristic when the optimization of the risk-sensitive criterion does not admit a dynamic programming reformulation. Through the interpolating approximation, the level of nonstationarity of the policy, and consequently, the size of the resulting search problem can be adjusted. The computational tractability and the generality of the approach follow from a nested parallel implementation of derivative-free optimization in conjunction with Monte Carlo simulation. We demonstrate the efficiency of the approach on an optimal energy storage charging problem, and illustrate the effect of the...

Strategic, Tactical and Real-Time Planning of Locomotives at Norfolk Southern Using Approximate Dynamic Programming

Locomotive planning has been a popular application of classical optimization models for decades, but with very few success stories. There are a host of complex rules governing how locomotives should be used. In addition, it is necessary to simultaneously manage locomotive inventories by balancing the need for holding power against the need for power at other yards. At the same time, we have to plan the need to return foreign power, and move power to maintenance facilities for scheduled FRA appointments. An additional complication arises as a result of the high level of uncertainty in transit times and delays due to yard processing, and as a result we may have to plan additional inventories in order to move outbound trains on time despite inbound delays. We describe a novel modeling and algorithmic strategy known as approximate dynamic programming, which can also be described as a form of “optimizing simulator” which uses feedback learning to plan locomotive movements in a way that closely mimics how humans plan real-world operations. This strategy can be used for strategic and tactical planning, and can also be adapted to real-time operations. We describe the strategy, and summarize experiences at Norfolk Southern with a strategic planning system.

A Probability Model for Grid Faults Using Incomplete Information

Utilities face the challenge of responding to power outages due to storms and ice damage, but most power grids are not equipped with sensors to pinpoint the precise location of fault causing the outage. Instead, utilities have to depend primarily on phone calls (trouble calls) from customers who have lost power. This paper presents an information model of the grid in the presence of outages; the developed model is used to estimate the probability of power line faults causing the outages. However, the computational complexity of the problem grows exponentially with the number of power lines in the grid. Thus, several methods are proposed for handling the combinatorial growth of events and the behavior is demonstrated using the data of a real-power grid. Performance results show that power line fault detection can be achieved with high accuracy, even with a very low percentage of customers calling to report an outage.