Alexander Shapiro

Lectures on Stochastic Programming: Modeling and Theory

“SPbook”2009/5/4page iiiiiiiiiiDarinka DentchevaDepartment of Mathematical SciencesStevens Institute of TechnologyHoboken, NJ 07030, USAAndrzej Ruszczynski´Department of Management Science and Information SystemsRutgers UniversityPiscataway, NJ 08854, USAAlexander ShapiroSchool of Industrial and Systems EngineeringGeorgia Institute of TechnologyAtlanta, GA 30332, USAThe authors dedicate this book:to Julia, Benjamin, Daniel, Natan and Yael;to Tsonka, Konstatin and Marek;and to the Memory of Feliks, Maria, and Dentcho.

The Sample Average Approximation Method for Stochastic Discrete Optimization

In this paper we study a Monte Carlo simulation--based approach to stochastic discrete optimization problems. The basic idea of such methods is that a random sample is generated and the expected value function is approximated by the corresponding sample average function. The obtained sample average optimization problem is solved, and the procedure is repeated several times until a stopping criterion is satisfied. We discuss convergence rates, stopping rules, and computational complexity of this procedure and present a numerical example for the stochastic knapsack problem.

A stochastic programming approach for supply chain network design under uncertainty

This paper proposes a stochastic programming model and solution algorithm for solving supply chain network design problems of a realistic scale. Existing approaches for these problems are either restricted to deterministic environments or can only address a modest number of scenarios for the uncertain problem parameters. Our solution methodology integrates a recently proposed sampling strategy, the sample average approximation (SAA) scheme, with an accelerated Benders decomposition algorithm to quickly compute high quality solutions to large-scale stochastic supply chain design problems with a huge (potentially infinite) number of scenarios. A computational study involving two real supply chain networks are presented to highlight the significance of the stochastic model as well as the efficiency of the proposed solution strategy.

Robust Stochastic Approximation Approach to Stochastic Programming

In this paper we consider optimization problems where the objective function is given in a form of the expectation. A basic difficulty of solving such stochastic optimization problems is that the involved multidimensional integrals (expectations) cannot be computed with high accuracy. The aim of this paper is to compare two computational approaches based on Monte Carlo sampling techniques, namely, the stochastic approximation (SA) and the sample average approximation (SAA) methods. Both approaches, the SA and SAA methods, have a long history. Current opinion is that the SAA method can efficiently use a specific (say, linear) structure of the considered problem, while the SA approach is a crude subgradient method, which often performs poorly in practice. We intend to demonstrate that a properly modified SA approach can be competitive and even significantly outperform the SAA method for a certain class of convex stochastic problems. We extend the analysis to the case of convex-concave stochastic saddle point problems and present (in our opinion highly encouraging) results of numerical experiments.

Monte Carlo Sampling Methods

Abstract In this chapter we discuss Monte Carlo sampling methods for solving large scale stochastic programming problems. We concentrate on the “exterior” approach where a random sample is generated outside of an optimization procedure, and then the constructed, so-called sample average approximation (SAA), problem is solved by an appropriate deterministic algorithm. We study statistical properties of the obtained SAA estimators. The developed statistical inference is incorporated into validation analysis and error estimation. We describe some variance reduction techniques which may enhance convergence of sampling based estimates. We also discuss difficulties in extending this methodology to multistage stochastic programming. Finally, we briefly discuss the SAA method applied to stochastic generalized equations and variational inequalities.

Convex Approximations of Chance Constrained Programs

We consider a chance constrained problem, where one seeks to minimize a convex objective over solutions satisfying, with a given close to one probability, a system of randomly perturbed convex constraints. This problem may happen to be computationally intractable; our goal is to build its computationally tractable approximation, i.e., an efficiently solvable deterministic optimization program with the feasible set contained in the chance constrained problem. We construct a general class of such convex conservative approximations of the corresponding chance constrained problem. Moreover, under the assumptions that the constraints are affine in the perturbations and the entries in the perturbation vector are independent-of-each-other random variables, we build a large deviation-type approximation, referred to as “Bernstein approximation,” of the chance constrained problem. This approximation is convex and efficiently solvable. We propose a simulation-based scheme for bounding the optimal value in the chance constrained problem and report numerical experiments aimed at comparing the Bernstein and well-known scenario approximation approaches. Finally, we extend our construction to the case of ambiguous chance constrained problems, where the random perturbations are independent with the collection of distributions known to belong to a given convex compact set rather than to be known exactly, while the chance constraint should be satisfied for every distribution given by this set.

ON THE MULTIVARIATE ASYMPTOTIC DISTRIBUTION OF SEQUENTIAL CHI-SQUARE STATISTICS

The multivariate asymptotic distribution of sequential Chi-square test statistics is investigated. It is shown that: (a) when sequential Chi-square statistics are calculated for nested models on the same data, the statistics have an asymptotic intercorrelation which may be expressed in closed form, and which is, in many cases, quite high; and (b) sequential Chi-squaredifference tests are asymptotically independent. Some Monte Carlo evidence on the applicability of the theory is provided.

The empirical behavior of sampling methods for stochastic programming

We investigate the quality of solutions obtained from sample-average approximations to two-stage stochastic linear programs with recourse. We use a recently developed software tool executing on a computational grid to solve many large instances of these problems, allowing us to obtain high-quality solutions and to verify optimality and near-optimality of the computed solutions in various ways.

The Sample Average Approximation Method Applied to Stochastic Routing Problems: A Computational Study

The sample average approximation (SAA) method is an approach for solving stochastic optimization problems by using Monte Carlo simulation. In this technique the expected objective function of the stochastic problem is approximated by a sample average estimate derived from a random sample. The resulting sample average approximating problem is then solved by deterministic optimization techniques. The process is repeated with different samples to obtain candidate solutions along with statistical estimates of their optimality gaps.We present a detailed computational study of the application of the SAA method to solve three classes of stochastic routing problems. These stochastic problems involve an extremely large number of scenarios and first-stage integer variables. For each of the three problem classes, we use decomposition and branch-and-cut to solve the approximating problem within the SAA scheme. Our computational results indicate that the proposed method is successful in solving problems with up to 21694 scenarios to within an estimated 1.0% of optimality. Furthermore, a surprising observation is that the number of optimality cuts required to solve the approximating problem to optimality does not significantly increase with the size of the sample. Therefore, the observed computation times needed to find optimal solutions to the approximating problems grow only linearly with the sample size. As a result, we are able to find provably near-optimal solutions to these difficult stochastic programs using only a moderate amount of computation time.

Optimization Problems with Perturbations: A Guided Tour

This paper presents an overview of some recent, and significant, progress in the theory of optimization problems with perturbations. We put the emphasis on methods based on upper and lower estimates of the objective function of the perturbed problems. These methods allow one to compute expansions of the optimal value function and approximate optimal solutions in situations where the set of Lagrange multipliers is not a singleton, may be unbounded, or is even empty. We give rather complete results for nonlinear programming problems and describe some extensions of the method to more general problems. We illustrate the results by computing the equilibrium position of a chain that is almost vertical or horizontal.