W. David Kelton

Statistical design and analysis

After a general simulation model is built, coded, verified, and validated, it is used to learn about the system(s) under study, requiring careful prior design of the runs and appropriate analysis of their output. This paper surveys methods for effective and efficient design and analysis of simulation experiments.

Confidence Intervals for Steady-State Simulations: I. A Survey of Fixed Sample Size Procedures

We consider the problem of constructing a confidence interval for the steady-state mean of a stochastic process by means of simulation, and study the five main methods which have been proposed replication, batch means, autoregressive representation, spectrum analysis, and regeneration cycles for the case when the length of the simulation is fixed in advance. Comparing the performances of these methods on stochastic models with known steady-state means, we find that the simulator should exercise caution in interpreting the results from a simulation of fixed length, and that the length of a simulation adequate for acceptable performance is highly model-dependent. We also investigate possible sources of error for the methods, and conclude that variance estimator bias is more important than point estimator bias in confidence interval coverage degradation.

Random initialization methods in simulation

In a steady-state simulation, the initial conditions often bias the estimators, sometimes severely. A common method for ameliorating this is to delete an initial portion of the run, perhaps entailing substantial data loss. We investigate feasible methods for initializing such simulations that lead to lower estimator bias or, alternatively, less requisite deletion. Deterministic and stochastic initialization rules are compared, with appropriately chosen stochastic rules being preferable in terms of several measures of quality of point estimators and confidence intervals. Forms for the initial distribution are suggested by the maximum entropy principle, the parameters of which may be specified from short pilot runs. These initialization rules are tested on a range of models with analytical tractability varying From complete to none.

An Analytical Evaluation of Alternative Strategies in Steady-State Simulation

We consider the simple technique of making independent and probabilistically identical simulation replications to estimate the steady-state mean of a stochastic process. For a fixed total sample size, using a first-order autoregressive model with high autocorrelation and time-dependent distributions, we analytically quantify and investigate the effects of the number of replications and of initial deletions of output on several different measures of point estimator and confidence interval quality. This analysis leads to general recommendations on the choice of replications and deletion. In particular, we conclude that deletion of some amount of the initial output in a replication can be an effective and efficient method of dealing with initialization bias; this conclusion differs from several previous studies on the efficacy of deletion.

Transient exponential-Erlang queues and steady-state simulation

The transient probabilistic structure of M/Em/1 and Em/M/1 queues initialized in an arbitrary deterministic state is derived in discrete time. Computational algorithms for obtaining the required probabilities are provided, and their application in calculating a variety of system performance measures is illustrated. The results are used to investigate the question of initializing simulations of systems such as these to promote rapid convergence to steady state, if that is the object of the simulation. These results are consistent with earlier studies for transient queueing systems, such as the M/M/s, but allow greater flexibility in specification of interarrival or service-time models inherent in the Erlang distributions.

ADVERTISING AND INTRAINDUSTRY BRAND SHIFT IN THE U.S. BREWING INDUSTRY

The brewing industry has constituted a focus of interest in industrialorganization economics since World War II, due to its rapidly increasing concentration, possibly large plant-level scale economies, and marked product differentiation accompanied by relatively high firm advertising intensities. Scherer [ 19, p. 1 10] wrote concerning the dramatic postwar structural change which has characterized this industry:

The transient behavior of the M / E k /2 queue and steady-state simulation

Abstract The probabilistic structure for the transient M / E k /2 queue is derived in discrete time, where E k denotes a k -Erlang distribution. This queue has a two-dimensional state-space. Expressions for the expected delay in queue are formulated in terms of transition probabilities. Results are numerically evaluated for a few cases. The convergence behavior is similar to that seen in previous work on queues with one-dimensional state spaces. The implications for initialization of steady-state simulations are discussed.

Hypothesis Tests for Markov Process Models Estimated from Aggregate Frequency Data

Abstract When the only data available for estimating the transition probabilities of a Markov chain are state occupation frequencies (rather than interstate transition frequencies), a least squares estimation technique and an accompanying hypothesis-testing methodology are proposed. This general hypothesis-testing procedure is used to develop three tests for adequacy of the simple stationary model. Null hypotheses of a zero-order process, stationarity, and homogeneity are considered. The distributions of the test statistics are investigated in a factorially designed Monte Carlo study. In general, it is found that treating the test statistics as having F distributions with appropriate degrees of freedom under the null hypothesis of interest leads to rejection proportions close to the desired levels. Additional Monte Carlo results indicate favorable power of the proposed tests.

Designing computer simulation experiments

This tutorial focuses on that part of a simulation study concerning setting the various specific model parameters and the experimental conditions under which the model will be exercised. Other issues in setting up and designing simulation experiments, such as variance reduction, ranking and selection, and optimization, are also mentioned. The focus is on careful choice of such parameters beforehand, with an eye toward the statistical analysis of the simulations results. Output analysis is not treated per se, being covered in another tutorial in this conference (by Gordon Clark), but the design and analysis activities must be done hand in hand.

Replication splitting and variance for simulating discrete-parameter stochastic processes

Previous results for stationary continuous-time processes concerning allocation of a fixed amount of simulation effort across independent replications are extended both to stationary and certain non-stationary discrete-time processes. In particular, in the presence of positive autocorrelation, variance is reduced if more short replications are designed. The magnitude, however, of the variance reduction is not great as long as the computation budget is not tight, suggesting that a good strategy is to design for a moderate number of replications in any case, which also mitigates potential bias problems.