Amos H. C. Ng

An empirical comparison of metamodeling strategies in noisy environments

Metamodeling plays an important role in simulation-based optimization by providing computationally inexpensive approximations for the objective and constraint functions. Additionally metamodeling can also serve to filter noise, which is inherent in many simulation problems causing optimization algorithms to be mislead. In this paper, we conduct a thorough statistical comparison of four popular metamodeling methods with respect to their approximation accuracy at various levels of noise. We use six scalable benchmark problems from the optimization literature as our test suite. The problems have been chosen to represent different types of fitness landscapes, namely, bowl-shaped, valley-shaped, steep ridges and multi-modal, all of which can significantly influence the impact of noise. Each metamodeling technique is used in combination with four different noise handling techniques that are commonly employed by practitioners in the field of simulation-based optimization. The goal is to identify the metamodeling strategy, i.e. a combination of metamodeling and noise handling, that performs significantly better than others on the fitness landscapes under consideration. We also demonstrate how these results carry over to a simulation-based optimization problem concerning a scalable discrete event model of a simple but realistic production line.

Quantitative analysis of a conceptual system dynamics maintenance performance model using multi-objective optimisation

Abstract This paper presents a quantitative analysis of a conceptual, system dynamics (SD) model by the application of multi-objective optimisation (MOO). The SD model investigates the strategic development of maintenance performance, using a system view of maintenance costs, while the execution of MOO evaluates multiple simulation runs, seeking the simultaneous trade-off solutions of the three conflicting objectives: maximise availability, minimise maintenance costs, and minimise maintenance consequential costs. The study explores three scenarios that represent companies at different states of developed maintenance performance. The application of this integrated, simulation-based optimisation approach reveals multiple analyses of system behaviour of the SD model, which are presented in a compact format to a decision-maker. Actually, notwithstanding the application to a conceptual model, the study results make explicit the nonlinearity between invested maintenance cost and its consequent effects. Furthermore, the approach demonstrates the contribution to the process of strengthening the usefulness of the conceptual maintenance performance model.