Qiao Ge

Combining screening and metamodel-based methods: An efficient sequential approach for the sensitivity analysis of model outputs

Abstract Sensitivity analysis (SA) is able to identify the most influential parameters of a given model. Application of SA is usually critical for reducing the complexity in the subsequent model calibration and use. Unfortunately it is hardly applied, especially when the model is in the form of a computationally expensive black-box computer program. A possible solution concerns applying SA to the metamodel (i.e., an approximation of the computationally expensive model) instead. Among the other options, the use of Gaussian process metamodels (also known as Kriging metamodels) has been recently proposed for the SA of computationally expensive traffic simulation models. However, the main limitation of this approach is its dependence on the model dimensionality. When the model is high-dimensional, the estimation of the Kriging metamodel may still be problematic due to its high computational cost. In order to overcome this problem, in the present paper, the Kriging-based approach has been combined with the quasi-optimized trajectory based elementary effects (quasi-OTEE) approach for the SA of high-dimensional models. The quasi-OTEE SA is used first to screen the influential and non-influential parameters of a high-dimensional model; then the Kriging-based SA is used to calculate the variance-based sensitivity indices, and to rank the most influential parameters in a more accurate way. The application of the proposed sequential SA is illustrated with several numerical experiments. Results show that the method can properly identify the most influential parameters and their ranks, while the number of model evaluations is considerably less than the variance-based SA (e.g., in one of the tests the sequential SA requires over 50 times less model evaluations than the variance-based SA).

An Exploratory Study of Two Efficient Approaches for the Sensitivity Analysis of Computationally Expensive Traffic Simulation Models

One of the main challenges arising when calibrating a complex traffic simulation model concerns the selection of the most important input parameters. The quasi-optimized trajectory-based elementary effects (quasi-OTEE) and the Kriging-based sensitivity analysis (SA) are two recently developed efficient approaches for the SA of computationally expensive simulation models. In this paper, two experimental studies using two different traffic simulation models (i.e., Aimsun and VISSIM) are presented to compare these two approaches and to better understand their advantages and disadvantages. Results show that both approaches are able to identify, to a good degree, the important parameters. In particular, the quasi-OTEE is better for screening the parameters, whereas the Kriging-based SA has higher precision in ranking the parameters. These findings suggest the following rule of thumb for the SA of computationally expensive traffic simulation models: the quasi-OTEE SA can be used first to screen the parameters and to decide which parameters to discard. Then, the Kriging-based SA can be used to refine the analysis and calculate first-order indexes to identify the correct rank of the important parameters.

Extending Morris method for qualitative global sensitivity analysis of models with dependent inputs

Abstract Global Sensitivity Analysis (GSA) can help modelers to better understand the model and manage the uncertainty. However, when the model itself is rather sophisticated, especially when dependence exists among model inputs, it could be difficult or even unfeasible to perform quantitative GSA directly. In this paper, a non-parametric approach is proposed for screening model inputs. It extends the classic Elementary Effects (i.e., Morris) method, which is widely used for screening independent inputs, to enable the screening of dependent model inputs. The performance of the proposed method is tested with three numerical experiments, and the results are cross-compared with those from the variance-based GSA. It is found that the proposed method can properly identify the influential and non-influential inputs from a complex model with several independent and dependent inputs. Furthermore, compared with the variance-based GSA, the proposed screening method only needs a few model runs, while the screening accuracy is well maintained. Therefore, it can be regarded as a practical tool for the initial GSA of high dimensional and computationally expensive models with dependent inputs.

Comprehensive Approach for the Sensitivity Analysis of High-Dimensional and Computationally Expensive Traffic Simulation Models

The reliability of traffic model results is strictly connected to the quality of its calibration. A challenge arising in this context concerns the selection of the most influential input parameters. A model sensitivity analysis should be used with this aim. However, because of the limitations of time and computational resources, a proper sensitivity analysis is rarely performed in common practice. A recent study introduced a methodology based on Gaussian process metamodels for the sensitivity analysis of computationally expensive traffic simulation models. The main limitation was a dependence on model dimensionality. When the model has more than about 15 to 20 parameters, estimation of a Gaussian process metamodel (also known as a Kriging metamodel) may become problematic. In this paper, the Kriging-based approach is coupled with a recently developed approach, quasi-optimized trajectory-based elementary effects (quasi-OTEE), for the sensitivity analysis of computationally expensive models. The quasi-OTEE sensitivity analysis can be used to identify the whole subset of sensitive parameters of a high-dimensional model, and the Kriging-based sensitivity analysis can then be used to refine the analysis and to rank the different parameters of the subset in a more reliable way. Application of this new sequential sensitivity analysis method is illustrated with the Wiedemann-74 car-following model. Results show that the new method requires 40 times fewer model evaluations than a standard variance-based sensitivity analysis to identify the influential parameters and their ranks.