Stephen E. Chick

Simulation Budget Allocation for Further Enhancing theEfficiency of Ordinal Optimization

Ordinal Optimization has emerged as an efficient technique for simulation and optimization. Exponential convergence rates can be achieved in many cases. In this paper, we present a new approach that can further enhance the efficiency of ordinal optimization. Our approach determines a highly efficient number of simulation replications or samples and significantly reduces the total simulation cost. We also compare several different allocation procedures, including a popular two-stage procedure in simulation literature. Numerical testing shows that our approach is much more efficient than all compared methods. The results further indicate that our approach can obtain a speedup factor of higher than 20 above and beyond the speedup achieved by the use of ordinal optimization for a 210-design example.

Selecting a Selection Procedure

Selection procedures are used in a variety of applications to select the best of a finite set of alternatives. “Best” is defined with respect to the largest mean, but the mean is inferred with statistical sampling, as in simulation optimization. There are a wide variety of procedures, which begs the question of which selection procedure to select. The main contribution of this paper is to identify, through extensive experimentation, the most effective selection procedures when samples are independent and normally distributed. We also (a) summarize the main structural approaches to deriving selection procedures, (b) formalize new sampling allocations and stopping rules, (c) identify strengths and weaknesses of the procedures, (d) identify some theoretical links between them, and (e) present an innovative empirical test bed with the most extensive numerical comparison of selection procedures to date. The most efficient and easiest to control procedures allocate samples with a Bayesian model for uncertainty about the means and use new adaptive stopping rules proposed here.

New Procedures to Select the Best Simulated System Using Common Random Numbers

Although simulation is widely used to select the best of several alternative system designs, and common random numbers is an important tool for reducing the computation effort of simulation experiments, there are surprisingly few tools available to help a simulation practitioner select the best system when common random numbers are employed. This paper presents new two-stage procedures that use common random numbers to help identify the best simulated system. The procedures allow for screening and attempt to allocate additional replications to improve the value of information obtained during the second stage, rather than determining the number of replications required to provide a given probability of correct selection guarantee. The procedures allow decision makers to reduce either the expected opportunity cost associated with potentially selecting an inferior system, or the probability of incorrect selection. A small empirical study indicates that the new procedures outperform several procedures with respect to several criteria, and identifies potential areas for further improvement.

Supply Chain Coordination and Influenza Vaccination

Annual influenza outbreaks incur great expenses in both human and monetary terms, and billions of dollars are being allocated for influenza pandemic preparedness in an attempt to avert even greater potential losses. Vaccination is a primary weapon for fighting influenza outbreaks. The influenza vaccine supply chain has characteristics that resemble the newsvendor problem but possesses several characteristics that distinguish it from many other supply chains. Differences include a nonlinear value of sales (caused by the nonlinear health benefits of vaccination that are due to infection dynamics) and vaccine production yield issues. We show that production risks, taken currently by the vaccine manufacturer, lead to an insufficient supply of vaccine. Several supply contracts that coordinate buyer (governmental public health service) and supplier (vaccine manufacturer) incentives in many other industrial supply chains cannot fully coordinate the influenza vaccine supply chain. We design a variant of the cost-sharing contract and show that it provides incentives to both parties so that the supply chain achieves global optimization and hence improves the supply of vaccines.

Bayesian analysis for simulation input and output

The paper summarizes some important results at the intersection of the elds of Bayesian statistics and stochastic simulation. Two statistical analysis issues for stochastic simulation are discussed in further detail from a Bayesian perspective. First, a review of recent work in input distribution selection is presented. Then, a new Bayesian formulation for the problem of output analysis for a single system is presented. A key feature is analyzing simulation output as a random variable whose parameters are an unknown function of the simulation’s inputs. The distribution of those parameters is inferred from simulation output via Bayesian response-surface methods. A brief summary of Bayesian inference and decision making is included for reference.

Economic Analysis of Simulation Selection Problems

Ranking and selection procedures are standard methods for selecting the best of a finite number of simulated design alternatives based on a desired level of statistical evidence for correct selection. But the link between statistical significance and financial significance is indirect, and there has been little or no research into it. This paper presents a new approach to the simulation selection problem, one that maximizes the expected net present value of decisions made when using stochastic simulation. We provide a framework for answering these managerial questions: When does a proposed system design, whose performance is unknown, merit the time and money needed to develop a simulation to infer its performance? For how long should the simulation analysis continue before a design is approved or rejected? We frame the simulation selection problem as a “stoppable” version of a Bayesian bandit problem that treats the ability to simulate as a real option prior to project implementation. For a single proposed system, we solve a free boundary problem for a heat equation that approximates the solution to a dynamic program that finds optimal simulation project stopping times and that answers the managerial questions. For multiple proposed systems, we extend previous Bayesian selection procedures to account for discounting and simulation-tool development costs.

Sequential Sampling to Myopically Maximize the Expected Value of Information

Statistical selection procedures are used to select the best of a finite set of alternatives, where “best” is defined in terms of each alternatives unknown expected value, and the expected values are inferred through statistical sampling. One effective approach, which is based on a Bayesian probability model for the unknown mean performance of each alternative, allocates samples based on maximizing an approximation to the expected value of information (EVI) from those samples. The approximations include asymptotic and probabilistic approximations. This paper derives sampling allocations that avoid most of those approximations to the EVI but entails sequential myopic sampling from a single alternative per stage of sampling. We demonstrate empirically that the benefits of reducing the number of approximations in the previous algorithms are typically outweighed by the deleterious effects of a sequential one-step myopic allocation when more than a few dozen samples are allocated. Theory clarifies the derivation of selection procedures that are based on the EVI.

Selecting the best system: a decision-theoretic approach

The problem of selecting the best system from a nite set of alternatives is considered from a Bayesian decision-theoretic perspective. The framework presented is quite general, and permits selection from two or more systems, with replications that use either independent or common random numbers, with unknown mean and covariance for the output, and permits Gaussian or non-Gaussian simulation output. For the case of unknown means and variance with common random numbers, the framework provides a probability of correct selection that does not suer from problems associated with the Bonferroni inequality. We indicate some criteria for which the Bayesian approach and other approaches are in general agreement, or disagreement. The probability of correct selection can be calculated either by quadrature or by Monte Carlo simulation from the posterior distribution of the parameters of the statistical distribution of the simulation output. We also comment on expected-value decision-making versus optimization criteria based on other functionals of the distribution of the output.

Integrating techniques from statistical ranking into evolutionary algorithms

Many practical optimization problems are subject to uncertain fitness evaluations. One way to reduce the noise is to average over multiple samples of the fitness function in order to evaluate a single individual. This paper proposes a general way to integrate statistical ranking and selection procedures into evolutionary algorithms. The proposed procedure focuses sampling on those individuals that are crucial for the evolutionary algorithm, and distributes samples in a way that efficiently reduces uncertainty. The goal is to drastically reduce the number of evaluations required for a proper operation of the evolutionary algorithm in noisy environments.

Analysis and simulation of a stochastic, discrete-individual model of STD transmission with partnership concurrency

Deterministic differential equation models indicate that partnership concurrency and non-homogeneous mixing patterns play an important role in the spread of sexually transmitted infections. Stochastic discrete-individual simulation studies arrive at similar conclusions, but from a very different modeling perspective. This paper presents a stochastic discrete-individual infection model that helps to unify these two approaches to infection modeling. The model allows for both partnership concurrency, as well as the infection, recovery, and reinfection of an individual from repeated contact with a partner, as occurs with many mucosal infections. The simplest form of the model is a network-valued Markov chain, where the networks nodes are individuals and arcs represent partnerships. Connections between the differential equation and discrete-individual approaches are constructed with large-population limits that approximate endemic levels and equilibrium probability distributions that describe partnership concurrency. A more general form of the discrete-individual model that allows for semi-Markovian dynamics and heterogeneous contact patterns is implemented in simulation software. Analytical and simulation results indicate that the basic reproduction number R(0) increases when reinfection is possible, and the epidemic rate of rise and endemic levels are not related by 1-1/R(0), when partnerships are not point-time processes.