Reuven Y. Rubinstein

The Cross-Entropy Method for Combinatorial and Continuous Optimization

We present a new and fast method, called the cross-entropy method, for finding the optimal solution of combinatorial and continuous nonconvex optimization problems with convex bounded domains. To find the optimal solution we solve a sequence of simple auxiliary smooth optimization problems based on Kullback-Leibler cross-entropy, importance sampling, Markov chain and Boltzmann distribution. We use importance sampling as an important ingredient for adaptive adjustment of the temperature in the Boltzmann distribution and use Kullback-Leibler cross-entropy to find the optimal solution. In fact, we use the mode of a unimodal importance sampling distribution, like the mode of beta distribution, as an estimate of the optimal solution for continuous optimization and Markov chains approach for combinatorial optimization. In the later case we show almost surely convergence of our algorithm to the optimal solution. Supporting numerical results for both continuous and combinatorial optimization problems are given as well. Our empirical studies suggest that the cross-entropy method has polynomial in the size of the problem running time complexity.

Optimization of computer simulation models with rare events

Discrete event simulation systems (DESS) are widely used in many diverse areas such as computer-communication networks, flexible manufacturing systems, project evaluation and review techniques (PERT), and flow networks. Because of their complexity, such systems are typically analyzed via Monte Carlo simulation methods. This paper deals with optimization of complex computer simulation models involving rare events. A classic example is to find an optimal (s, S) policy in a multi-item, multicommodity inventory system, when quality standards require the backlog probability to be extremely small. Our approach is based on change of the probability measure techniques, also called likelihood ratio (LR) and importance sampling (IS) methods. Unfortunately, for arbitrary probability measures the LR estimators and the resulting optimal solution often tend to be unstable and may have large variances. Therefore, the choice of the corresponding importance sampling distribution and in particular its parameters in an optimal way is an important task. We consider the case where the IS distribution comes from the same parametric family as the original (true) one and use the stochastic counterpart method to handle simulation based optimization models. More specifically, we use a two-stage procedure: at the first stage we identify (estimate) the optimal parameter vector at the IS distribution, while at the second stage we estimate the optimal solution of the underlying constrained optimization problem. Particular emphasis will be placed on estimation of rare events and on integration of the associated performance function into stochastic optimization programs. Supporting numerical results are provided as well.

Sensitivity analysis and performance extrapolation for computer simulation models

We present a method for deriving sensitivities of performance measures for computer simulation models. We show that both the sensitivities (derivatives, gradients, Hessians, etc.) and the performance measure can be estimated simultaneously from the same simulation. Our method is based on probability measure transformations derived from the efficient score. We also present a rather general procedure from which perturbation analysis and our method can be viewed as particular cases. Applications to reliability models and stochastic shortest path networks are given.

Monte Carlo optimization, simulation, and sensitivity of queueing networks

Antithetic and Common Random Variables in Simulation of Complex Stochastic Systems. Multidimensional Control Variates in Monte Carlo Simulation. Stochastic Optimization Via Stochastic Approximation. Perturbation Analysis for Sensitivity and Optimization of Complex Queueing Networks. Monte Carlo Optimization. Appendixes. References. Index.

Application of the Cross-Entropy Method to the Buffer Allocation Problem in a Simulation-Based Environment

The buffer allocation problem (BAP) is a well-known difficult problem in the design of production lines. We present a stochastic algorithm for solving the BAP, based on the cross-entropy method, a new paradigm for stochastic optimization. The algorithm involves the following iterative steps: (a) the generation of buffer allocations according to a certain random mechanism, followed by (b) the modification of this mechanism on the basis of cross-entropy minimization. Through various numerical experiments we demonstrate the efficiency of the proposed algorithm and show that the method can quickly generate (near-)optimal buffer allocations for fairly large production lines.

Cross-entropy and rare events for maximal cut and partition problems

We show how to solve the maximal cut and partition problems using a randomized algorithm based on the <i>cross-entropy</i> method. For the maximal cut problem, the proposed algorithm employs an auxiliary Bernoulli distribution, which transforms the original deterministic network into an associated stochastic one, called the <i>associated stochastic network</i> (ASN). Each iteration of the randomized algorithm for the ASN involves the following two phases:(1) Generation of random cuts using a multidimensional <i>Ber</i>(<b>p</b>) distribution and calculation of the associated cut lengths (objective functions) and some related quantities, such as rare-event probabilities.(2) Updating the parameter vector <b>p</b> on the basis of the data collected in the first phase.We show that the <i>Ber</i>(<b>p</b>) distribution converges in distribution to a degenerated one, <i>Ber</i>(<b>p</b><inf><i>d</i></inf>^*), <b>p</b><inf><i>d</i></inf>^* = (<i>pd</i>,1,...,<i>pd,n</i>) in the sense that someelements of <b>p</b><inf><i>d</i></inf>^*, will be unities and the rest zeros. The unity elements of <b>p</b><inf><i>d</i></inf>^* uniquely define a cut which will be taken as the estimate of the maximal cut. A similar approach is used for the partition problem. Supporting numerical results are given as well. Our numerical studies suggest that for the maximal cut and partition problems the proposed algorithm typically has polynomial complexity in the size of the network.

Combinatorial Optimization, Cross-Entropy, Ants and Rare Events

We show how to solve network combinatorial optimization problems using a randomized algorithm based on the cross-entropy method. The proposed algorithm employs an auxiliary random mechanism, like a Markov chain, which converts the original deterministic network into an associated stochastic one, called the associated stochastic network (ASN). Depending on a particular problem, we introduce the randomness in ASN by making either the nodes or the edges of the network random. Each iteration of the randomized algorithm based on the ASN involves the following two phases: 1. Generation of trajectories using the random mechanism and calculation of the associated path (objective functions) and some related quantities, such as rare-event probabilities. 2. Updating the parameters associated with the random mechanism, like the probability matrix P of the Markov chain, on the basis of the data collected at first phase. We show that asymptotically the matrix P converges to a degenerated one P* d in the sense that at each row of the MC P* d only a single element equals unity, while the remaining elements in each row are zeros. Moreover, the unity elements of each row uniquely define the optimal solution. We also show numericaly that for a finite sample the algorithm converges with very high probability to a very small subset of the optimal values. We finally show that the proposed method can also be used for noisy networks, namely where the deterministic edge distances in the network are replaced by random variables with unknown expected values. Supporting numerical results are given as well. Our numerical studies suggest that the proposed algorithm typically has polynomial complexity in the size of the network.

Optimization and sensitivity analysis of computer simulation models by the score function method

Abstract This paper surveys some recent results on the score function (SF) method. This method is suitable for performance evaluation, sensitivity analysis, and optimization of complex discrete-event systems such as non-Markovian queueing systems.

Efficiency of Multivariate Control Variates in Monte Carlo Simulation

This paper considers some statistical aspects of applying control variates to achieve variance reduction in the estimation of a vector of response variables in Monte Carlo simulation. It gives a result that quantifies the loss in variance reduction caused by the estimation of the optimal control matrix. For the one-dimensional case, we derive analytically the optimal size of the vector of control variates under specific assumptions on the covariance matrix. For the multidimensional case, our numerical results show that good variance reduction is achieved when the number of control variates is relatively small approximately of the same order as the number of unknown parameters. Finally, we give some recommendations for future research.

Heavy tails, importance sampling and cross-entropy

ABSTRACT We consider the problem of estimating ℙ(Y 1 + … + Y n > x) by importance sampling when the Y i are i.i.d. and heavy-tailed. The idea is to exploit the cross-entropy method as a tool for choosing good parameters in the importance sampling distribution; in doing sso, we use the asymptotic description that given ℙ(Y 1 + … + Y n > x), n − 1 of the Y i have distribution F and one the conditional distribution of Y given Y > x. We show in some specific parametric examples (Pareto and Weibull) how this leads to precise answers which, as demonstrated numerically, are close to being variance minimal within the parametric class under consideration. Related problems for M/G/1 and GI/G/1 queues are also discussed.