Ernest C. Houck

Correlated simulation experiments in first-order response surface design

The collection of mathematical models, experimental strategies, and statistical inference referred to as response surface methodology RSM has been used in the empirical exploration of a wide variety of systems, particularly industrial situations in which a large number of variables influence the system response of interest. This paper examines experimental strategies for implementing RSM procedures in a simulation environment. Of particular interest is the question of how to best assign the pseudorandom number streams that drive the simulation to the experimental points when the objective is to estimate a first-order response surface model. We present general results for factorial and fractional-factorial plans where each factor is present at two levels. For this class of response surface designs, we consider three strategies for the assignment of pseudorandom number streams to experimental points: i the use of a unique set of streams at each design point; ii the assignment of a common set of streams to all experiments; and iii the simultaneous use of common and antithetic stream sets by the use of design blocking. We base our analysis of these correlation induction strategies on variance criteria commonly employed in response surface design, including: generalized variance, prediction variance, integrated variance, and variance of slopes. Our findings show that the simultaneous use of common and antithetic stream sets is the preferred correlation induction strategy, but that no one assignment procedure is uniformly superior for all four criteria. Our results provide a basis for selecting among the three correlation induction strategies.

Simulation designs and correlation induction for reducing second-order bias in first-order response surfaces

Construction of simulation designs for the estimation of response surface metamodels is often based on optimal design theory. Underlying such designs is the assumption that the postulated model provides the correct representation of the simulated response. As a result, the location of design points and the assignment of pseudorandom number streams to these experiments are determined through the minimization of some function of the covariance matrix of the model coefficient estimators. In contrast, we assume that the postulated model may be incorrect. Attention is therefore directed to the development of simulation designs that offer protection against the bias due to possible model misspecification as well as error variance. The particular situation examined is the estimation of first-order response surface models in the presence of polynomials of order two. Traditional two-level factorial plans combined with one of three pseudorandom number assignment strategies define the simulation designs. Specification of the factor settings for these experimental plans are based on two integrated mean squared error criteria of particular interest in response surface studies. For both design criteria, comparisons of the optimal designs across the three assignment strategies are presented to assist experimenters in the selection of an appropriate simulation design.

A sequential experimental design procedure for the estimation of first- and second-order simulation metamodels

Simulation metamodels find apphcation m the study of complex systems that cannot be solved analytically. These metamodels represent efficient tools for studying the characteristics of the more comphcated simulation model, provide needed insight into the problems of computer model validation and verification, and allow for the prediction of both system performance and op’umum operating conditions. This article presents a procedure for the construction of sequential simulation designs for the estimation of response surface metamodels. The first set of experiments is defined as a fractional two-level factorial design augmented with rephcated center points. Information from these experiments 1s used to estimate the levels of the factorial design points that constitute the second stage of experimentation If observations on this two-stage, first-order design suggest the presence of unfitted quadratic terms, a thn-d set of observations corresponding to the axial portion of a central composite design M taken to allow for the estimation of a second-order metamodel, Two types of performance criteria are considered in the specdlcation of the factor settings in the second and third stages: (1) minimizing errors associated with predicting the response variable and (2) mimmizmg errors involved with estimating the response function slopes. Additionally, three methods of assigning random number streams to the stochastic components of the simulation model are considered: (1) independent streams, (2) common streams, and (3) the assignment rule blocking strategy, An example illustrating the use of the sequential design procedure is presented, and a Monte Carlo study investigates the performance of the two variance reduction techniques (common streams and the assignment rule) relatlve to independent stream sets. Empirical results mdlcate a preference for the assignment rule strategy for the estimation of both firstand second-order metamodels,

Pseudorandom Number Assignment in Quadratic Response Surface Designs

Abstract This paper examines three general strategies for the assignment of pseudorandom number streams to simulation experiments in quadratic response surface designs. Comparisons of these variance-reduction schemes are presented for four design criteria that collectively address the two goals of response surface methodology: prediction and optimization.

Combining regression diagnostics with simulation metamodels

Abstract In a 1983 article in this journal Kleijnen proposed a lack-of-fit cross-validation test for validating regression metamodels in computer simulation. His procedure utilizes additional information frequently available in simulation to obtain independent estimates of the response variance at each design point. Kleijnen notes that one drawback of his strategy is the need to run a large number of regressions. We demonstrate that the statistic upon which this inference procedure is based is analogous to the R-Student statistic commonly used in regression diagnostics and show that only a single regression need be computed to implement the test. Kleijnens procedure is then extended to include weighted least squares, and again, major computational simplifications are identified. For both ordinary least squares and weighted least squares, extensions of modern regression diagnostic tools also are developed to supplement traditional inferential model building strategies.

Some optimal simulation designs for estimating quadratic response surface functions

Presents some experimental design strategies for simulation studies involving the estimation of quadratic response surfaces. Optimal design plans are developed in four common second-order design classes (central composite, Box-Behnken, three-level factorial, and small composite designs) using a criterion which incorporates both the bias and variance of the predicted response variable. Three methods of assigning pseudorandom number streams to design points are considered: independent streams, common streams, and the simultaneous use of common and antithetic streams in an orthogonally blockable experimental design. Each method uses independent streams for replications of design points. The findings indicate that carefully planned experimental designs can substantially improve the estimation of quadratic response surface models.<<ETX>>

Sequential experimental designs for simulation metamodeling

A procedure is developed for the construction of sequential simulation designs for the estimation of firstand second-order response surface met amodela. The first stage of experimentation involves the use of a fractional two-level factorial design augmented with replicated center points. Information obtained from this experimental design is used to estimate the “optimal” location of the factorial design points for the second stage of experimentation. Two types of performance criteria are considered in the specification of the factor settings: (1) integrated mean squared error of the predicted response variable, and (2) integrated mean squared error of the response function slopes. Additional data is collected in the second stage using a different fraction of the t we-level factorial design. If quadratic curvature is indicated, a third stage of experimentation is performed to collect data for the axial portion of a central composite design. Two performance criteria are considered in the specification of the optimal azial levels: (1) integrated variance error of the predicted response variable, and (2) integrated variance error of the response function slopes. The selection of factor levels in the second and third stages also depends on the strategy used in assigning random number streams to the stochastic components of the simulation model. We investigate three assignment methods (independent streams, common streams, and the assignment rule blocking strategy), and we develop sequential design plans for each strategy.

An RSM investigation of the profit potential of customer service variables in physical distribution

This paper presents a methodology for examining the profit-generating potential of customer service variables. Through the use of a case study, the proposed methodology is shown to provide a means for estimating the optimum customer service mix and assessing the sensitivity of the profit-customer service relationship. Results of the investigation indicate that service levels can be adjusted so as to enhance the profit performance of the firm.