Songqing Shan

Mode-pursuing sampling method for global optimization on expensive black-box functions

The presence of black-box functions in engineering design, which are usually computation-intensive, demands efficient global optimization methods. This article proposes a new global optimization method for black-box functions. The global optimization method is based on a novel mode-pursuing sampling method that systematically generates more sample points in the neighborhood of the function mode while statistically covering the entire search space. Quadratic regression is performed to detect the region containing the global optimum. The sampling and detection process iterates until the global optimum is obtained. Through intensive testing, this method is found to be effective, efficient, robust, and applicable to both continuous and discontinuous functions. It supports simultaneous computation and applies to both unconstrained and constrained optimization problems. Because it does not call any existing global optimization tool, it can be used as a standalone global optimization method for inexpensive problems as well. Limitations of the method are also identified and discussed.

Reliable design space and complete single-loop reliability-based design optimization

Reliability-based design optimization (RBDO) has been intensively studied due to its significance and its conceptual and mathematical complexity. This paper proposes a new method for RBDO on the basis of the concept of reliable design space (RDS), within which any design satisfies the reliability requirements. Therefore, a RBDO problem becomes a simple, deterministic optimization problem constrained by RDS rather than its deterministic feasible space. The RDS is found in this work by using the partial derivatives at the current design point as an approximation of the derivatives at its corresponding most probable point (MPP) on the limit state function. This work completely resolves the double loop in RBDO and turns RBDO into a simple optimization problem. Well-known problems from the literature are selected to illustrate the steps of the approach and for result comparison. Discussions will also be given on the limitation of the proposed method, which is shown to be a common limitation overlooked by the research community on RBDO.

Metamodeling for High Dimensional Simulation-Based Design Problems

Computational tools such as finite element analysis and simulation are widely used in engineering, but they are mostly used for design analysis and validation. If these tools can be integrated for design optimization, it will undoubtedly enhance a manufacturers competitiveness. Such integration, however, faces three main challenges: (1) high computational expense of simulation, (2) the simulation process being a black-box function, and (3) design problems being high dimensional. In the past two decades, metamodeling has been intensively developed to deal with expensive black-box functions, and has achieved success for low dimensional design problems. But when high dimensionality is also present in design, which is often found in practice, there lacks of a practical method to deal with the so-called high dimensional, expensive, and black-box (HEB) problems. This paper proposes the first metamodel of its kind to tackle the HEB problem. This paper integrates the radial basis function with high dimensional model representation into a new model, RBF-HDMR. The developed RBF-HDMR model offers an explicit function expression, and can reveal (1 ) the contribution of each design variable, (2) inherent linearity/nonlinearity with respect to input variables, and (3) correlation relationships among input variables. An accompanying algorithm to construct the RBF - HDMR has also been developed. The model and the algorithm fundamentally change the exponentially growing computation cost to be polynomial. Testing and comparison confirm the efficiency and capability of RBF-HDMR for HEB problems.

An Efficient Pareto Set Identification Approach for Multiobjective Optimization on Black-Box Functions

Both multiple objectives and computation-intensive black-box functions often exist simultaneously in engineering design problems. Few of existing multiobjective optimization approaches addresses problems with expensive black-box functions. In this paper, a new method called the Pareto set pursuing (PSP) method is developed. By developing sampling guidance functions based on approximation models, this approach progressively provides a designer with a rich and evenly distributed set of Pareto optimal points. This work describes PSP procedures in detail. From testing and design application, PSP demonstrates considerable promises in efficiency, accuracy, and robustness. Properties of PSP and differences between PSP and other approximation-based methods are also discussed. It is believed that PSP has a great potential to be a practical tool for multiobjective optimization problems.

Reliability Assessment Using Discriminative Sampling and Metamodeling

Reliability assessment is the foundation for reliability engineering and reliability-based design optimization. It has been a difficult task, however, to perform both accurate and efficient reliability assessment after decades of research. This work proposes an innovative method that deviates significantly from conventional methods. It applies a discriminative sampling strategy to directly generate more points close to or on the limit state. A sampling guidance function was developed for such a strategy. Due to the dense samples in the neighborhood of the limit state, a kriging model can be built which is especially accurate near the limit state. Based on the kriging model, reliability assessment can be performed. The proposed method is tested by using well-known problems in the literature. It is found that it can efficiently assess the reliability for problems of single failure region and has a good performance for problems of multiple failure regions. The features and limitations of the method are also discussed, along with the comparison with the importance sampling (IS) based assessment methods.

Design Space Reduction for Multi-Objective Optimization and Robust Design Optimization Problems

Modern engineering design often involves computation-intensive simulation processes and multiple objectives. Engineers prefer an efficient optimization method that can provide them insights into the problem, yield multiple good or optimal design solutions, and assist decision-making. This work proposed a rough-set based method that can systematically identify regions (or subspaces) from the original design space for multiple objectives. In the smaller regions, any design solution (point) very likely satisfies multiple design goals. Engineers can pick many design solutions from or continue to search in those regions. Robust design optimization (RDO) problems can be formulated as a biobjective optimization problem and thus in this work RDO is considered a special case of multi-objective optimization (MOO). Examples show that the regions can be efficiently identified. Pareto-optimal frontiers generated from the regions are identical with those generated from the original design space, which indicates that important design information can be captured by only these regions (subspaces). Advantages and limitations of the proposed method are discussed.

Failure Surface Frontier for Reliability Assessment on Expensive Performance Function

This work proposes a novel concept of failure surface frontier (FSF), which is a hyper-surface consisting of the set of non-dominated failure points on the limit states of a failure region. Assumptions, definitions, and benefits of FSF are described first in detail. It is believed that FSF better represents the limit states for reliability assessment (RA) than conventional linear or quadratic approximations on the most probable point. Then, a discriminative sampling based algorithm is proposed to identify FSF, based on which the reliability can be directly assessed for expensive performance functions. Though an approximation model is employed to approximate the limit states, it is only used as a guide for sampling and a supplementary tool for RA. Test results on well-known problems show that FSF-based RA on expensive performance functions achieves high accuracy and efficiency, when compared with the state-of-the-art results archived in literature. Moreover, the concept of FSF and proposed RA algorithm are proved to be applicable to problems of multiple failure regions, multiple most probable points, or failure regions of extremely small probability.

Integrating Least Square Support Vector Regression and Mode Pursuing Sampling Optimization for Crashworthiness Design

Many metamodeling techniques have been developed in the past two decades to reduce the computational cost of design evaluation. With the increasing scale and complexity of engineering problems, popular metamodeling techniques including artificial neural network (ANN), Polynomial regression (PR), Kriging (KG), radial basis functions (RBF), and multivariate adaptive regression splines (MARS) face difficulties in solving highly nonlinear problems, such as the crashworthiness design. Therefore, in this work, we integrate the least support vector regression (LSSVR) with the mode pursuing sampling (MPS) optimization method and applied the integrated approach for crashworthiness design. The MPS is used for generating new samples which are concentrated near the current local minima at each iteration and yet still statistically cover the entire design space. The LSSVR is used for establishing a more robust metamodel from noisy data. Therefore, the proposed method integrates the advantages of both the LSSVR and MPS to more efficiently achieve reasonably accurate results. In order to verify the proposed method, well-known highly nonlinear functions are used for testing. Finally, the proposed method is applied to three typical crashworthiness optimization cases. The results demonstrate the potential capability of this method in the crashworthiness design of vehicles. [DOI: 10.1115/1.4003840]

Survey of Modeling and Optimization Strategies for High-Dimensional Design Problems

The integration of optimization methodologies with computational analyses/simulations has profound impact on the product design. Such integration, however, faces multiple challenges. The most eminent challenges arise from high-dimensional problems, expensive analysis/simulation functions, and unknown function properties (i.e., black-box functions). The merge of these three challenges severely aggravates the difficulty and becomes a major hurdle for design optimization. This paper provides a survey on modeling and optimization strategies for high-dimensional design problems with expensive black-box functions. This survey screens out 191 references including multiple historical reviews on relevant subjects from about 1000 papers in Statistics, Mathematics, Chemistry, Physics, Computer Science, and various engineering disciplines. The survey has been performed in three areas: strategies tackling high-dimensionality of problems, model approximation techniques, and optimization strategies for expensive functions. Major contributions in each area are discussed and presented in an organized manner. The survey also exposes that modeling and optimization strategies for high-dimensional expensive black-box function are scarce and sporadic, partially due to our lack of understanding of high dimensional spaces and the difficulty of the problem itself. Based on the review results, observations and summaries will be given. The authors will offer our perspectives on the challenges and directions for future research.

Development of Adaptive RBF-HDMR Model for Approximating High Dimensional Problems

Modeling or approximating high dimensional, computationally-expensive, black-box problems faces an exponentially increasing difficulty, the “curse-of-dimensionality”. This paper proposes a new form of high-dimensional model representation (HDMR) by integrating the radial basis function (RBF). The developed model, called RBF-HDMR, naturally explores and exploits the linearity/nonlinearity and correlation relationships among variables of the underlying function that is unknown or computationally expensive. This work also derives a lemma that supports the divide-and-conquer and adaptive modeling strategy of RBF-HDMR. RBF-HDMR circumvents or alleviates the “curse-of-dimensionality” by means of its explicit hierarchical structure, adaptive modeling strategy tailored to inherent variable relation, sample reuse, and a divide-and-conquer space-filling sampling algorithm. Multiple mathematical examples of a wide scope of dimensionalities are given to illustrate the modeling principle, procedure, efficiency, and accuracy of RBF-HDMR.Copyright