Timothy W. Simpson

Metamodels for Computer-based Engineering Design: Survey and recommendations

The use of statistical techniques to build approximations of expensive computer analysis codes pervades much of today’s engineering design. These statistical approximations, or metamodels, are used to replace the actual expensive computer analyses, facilitating multidisciplinary, multiobjective optimization and concept exploration. In this paper, we review several of these techniques, including design of experiments, response surface methodology, Taguchi methods, neural networks, inductive learning and kriging. We survey their existing application in engineering design, and then address the dangers of applying traditional statistical techniques to approximate deterministic computer analysis codes. We conclude with recommendations for the appropriate use of statistical approximation techniques in given situations, and how common pitfalls can be avoided.

Kriging models for global approximation in simulation-based multidisciplinary design optimization

Response surface methods have been used for a variety of applications in aerospace engineering, particularly in multidisciplinary design optimization. We investigate the use of kriging models as alternatives to traditional second-order polynomial response surfaces for constructing global approximations for use in a real aerospace engineering application, namely, the design of an aerospike nozzle. Our objective is to examine the difeculties in building and using kriging models to create accurate global approximations to facilitate multidisciplinary design optimization. Error analysis of the response surface and kriging models is performed, along with a graphical comparison of the approximations. Four optimization problems are also formulated and solved using both sets of approximation models to gain insight into their use for multidisciplinary design optimization. We end that the kriging models, which use only a constant “global” model and a Gaussian correlation function, yield global approximations that are slightly more accurate than the response surface models.

Product family design and platform-based product development: a state-of-the-art review

Product family design and platform-based product development has received much attention over the last decade. This paper provides a comprehensive review of the state-of-the-art research in this field. A decision framework is introduced to reveal a holistic view of product family design and platform-based product development, encompassing both front-end and back-end issues. The review is organized according to various topics in relation to product families, including fundamental issues and definitions, product portfolio and product family positioning, platform-based product family design, manufacturing and production, as well as supply chain management. Major challenges and future research directions are also discussed.

Use of kriging models to approximate deterministic computer models

The use of kriging models for approximation and metamodel-based design and optimization has been steadily on the rise in the past decade. The widespread usage of kriging models appears to be hampered by (1) the lack of guidance in selecting the appropriate form of the kriging model, (2) computationally efficient algorithms for estimating the model’s parameters, and (3) an effective method to assess the resulting model’s quality. In this paper, we compare (1) Maximum Likelihood Estimation (MLE) and Cross-Validation (CV) parameter estimation methods for selecting a kriging model’s parameters given its form and (2) and an R 2 of prediction and the corrected Akaike Information Criterion for assessing the quality of the created kriging model, permitting the comparison of different forms of a kriging model. These methods are demonstrated with six test problems. Finally, different forms of kriging models are examined to determine if more complex forms are more accurate and easier to fit than simple forms of kriging models for approximating computer models.

Product platform design and customization: Status and promise

In an effort to improve customization for todays highly competitive global marketplace, many companies are utilizing product families and platform-based product development to increase variety, shorten lead times, and reduce costs. The key to a successful product family is the product platform from which it is derived either by adding, removing, or substituting one or more modules to the platform or by scaling the platform in one or more dimensions to target specific market niches. This nascent field of engineering design has matured rapidly in the past decade, and this paper provides a comprehensive review of the flurry of research activity that has occurred during that time to facilitate product family design and platform-based product development for mass customization. Techniques for identifying platform leveraging strategies within a product family are reviewed along with metrics for assessing the effectiveness of product platforms and product families. Special emphasis is placed on optimization approaches and artificial intelligence techniques to assist in the process of product family design and platform-based product development. Web-based systems for product platform customization are also discussed. Examples from both industry and academia are presented throughout the paper to highlight the benefits of product families and product platforms. The paper concludes with a discussion of potential areas of research to help bridge the gap between planning and managing families of products and designing and manufacturing them.

Comparative studies of metamodeling techniques under multiple modeling criteria

Despite the advances in computer capacity, the enormous computational cost of complex engineering simulations makes it impractical to rely exclusively on simulation for the purpose of design optimization. To cut down the cost, surrogate models, also known as metamodels, are constructed from and then used in lieu of the actual simulation models. In the paper, we systematically compare four popular metamodeling techniques —Polynomial Regression, Multivariate Adaptive Regression Splines, Radial Basis Functions, and Kriging —based on multiple performance criteria using fourteen test problems representing different classes of problems. Our objective in th is study is to investigate the advantages and disadvantages these four metamodeling techniques using multiple modeling criteria and multiple test problems rather than a single measure of merit and a single test problem.

COMPARISON OF RESPONSE SURFACE AND KRIGING MODELS FOR MULTIDISCIPLINARY DESIGN OPTIMIZATION

In this paper, we compare and contrast the use of second-order response surface models and kriging models for approximating non-random, deterministic computer analyses. After reviewing the response surface method for constructing polynomial approximations, kriging is presented as an alternative approximation method for the design and analysis of computer experiments. Both methods are applied to the multidisciplinary design of an aerospike nozzle which consists of a computational fluid dynamics model and a finite-element model. Error analysis of the response surface and kriging models is performed along with a graphical comparison of the approximations, and four optimization problems are formulated and solved using both sets of approximation models. The second-order response surface models and kriging models—using a constant underlying global model and a Gaussian correlation function—yield comparable results. NOMENCLATURE

Analysis of support vector regression for approximation of complex engineering analyses

A variety of metamodeling techniques have been developed in the past decade to reduce the computational expense of computer-based analysis and simulation codes. Metamodeling is the process of building a model of a model to provide a fast surrogate for a computationally expensive computer code. Common metamadeling techniques include response surface methodology, kriging, radial basis functions, and multivariate adaptive regression splines. In this paper, we investigate support vector regression (SVR) as an alternative technique for approximating complex engineering analyses. The computationally efficient theory behind SVR is reviewed, and SVR approximations are compared against the aforementioned four mefamodeling techniques using a test bed of 26 engineering analysis functions. SVR achieves more accurate and more robust function approximations than the four metamodeling techniques, and shows great potential for metamodeling applications, adding to the growing body of promising empirical performance of SVR.

Computationally Inexpensive Metamodel Assessment Strategies

In many scientific and engineering domains, it is common to analyze and simulate complex physical systems using mathematical models. Although computing resources continue to increase in power and speed, computer simulation and analysis codes continue to grow in complexity and remain computationally expensive, limiting their use in design and optimization. Consequently, many researchers have developed different metamodeling strategies to create inexpensive approximations of computationally expensive computer simulations. These approximations introduce a new element of uncertainty during design optimization, and there is a need to develop efficient methods to assess metamodel validity. We investigate computationally inexpensive assessment methods for metamodel validation based on leave-k-out cross validation and develop guidelines for selecting k for different types of metamodels. Based on the results from two sets of test problems, k = 1 is recommended for leave-k-out cross validation of low-order polynomial and radial basis function metamodels, whereas k=0.1N or N is recommended for kriging metamodels, where N is the number of sample points used to construct the metamodel.

Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come - Or Not

The use of metamodeling techniques in the design and analysis of computer experiments has progressed remarkably in the past two decades, but how far have we really come? This is the question that we investigate in this paper, namely, the extent to which the use of metamodeling techniques in multidisciplinary design optimization have evolved in the two decades since the seminal paper on Design and Analysis of Computer Experiments by Sacks et al. As part of this review, we examine the motivation for advancements in metamodeling techniques from both a historical perspective and the research itself. Based on current thrusts in the field, we emphasize multi-level/multi-fidelity approximations and ensembles of metamodels, as well as the availability of metamodels within commercial software and for design space exploration and visualization in this review. Our closing remarks offer insight into future research directions – nearly the same ones that have motivated us in the past.