Saman Razavi

Numerical assessment of metamodelling strategies in computationally intensive optimization

Metamodelling is an increasingly more popular approach for alleviating the computational burden associated with computationally intensive optimization/management problems in environmental and water resources systems. Some studies refer to the metamodelling approach as function approximation, surrogate modelling, response surface methodology or model emulation. A metamodel-enabled optimizer approximates the objective (or constraint) function in a way that eliminates the need to always evaluate this function via a computationally expensive simulation model. There is a sizeable body of literature developing and applying a variety of metamodelling strategies to various environmental and water resources related problems including environmental model calibration, water resources systems analysis and management, and water distribution network design and optimization. Overall, this literature generally implies metamodelling yields enhanced solution efficiency and (almost always) effectiveness of computationally intensive optimization problems. This paper initially develops a comparative assessment framework which presents a clear computational budget dependent definition for the success/failure of the metamodelling strategies, and then critically evaluates metamodelling strategies, through numerical experiments, against other common optimization strategies not involving metamodels. Three different metamodel-enabled optimizers involving radial basis functions, kriging, and neural networks are employed. A robust numerical assessment within different computational budget availability scenarios is conducted over four test functions commonly used in optimization as well as two real-world computationally intensive optimization problems in environmental and water resources systems. Numerical results show that metamodelling is not always an efficient and reliable approach to optimizing computationally intensive problems. For simpler response surfaces, metamodelling can be very efficient and effective. However, in some cases, and in particular for complex response surfaces when computational budget is not very limited, metamodelling can be misleading and a hindrance, and better solutions are achieved with optimizers not involving metamodels. Results also demonstrate that neural networks are not appropriate metamodelling tools for limited computational budgets while metamodels employing kriging and radial basis functions show comparable overall performance when the available computational budget is very limited.

What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models

Sensitivity analysis is an essential paradigm in Earth and Environmental Systems modeling. However, the term “sensitivity” has a clear definition, based in partial derivatives, only when specified locally around a particular point (e.g., optimal solution) in the problem space. Accordingly, no unique definition exists for “global sensitivity” across the problem space, when considering one or more model responses to different factors such as model parameters or forcings. A variety of approaches have been proposed for global sensitivity analysis, based on different philosophies and theories, and each of these formally characterizes a different “intuitive” understanding of sensitivity. These approaches focus on different properties of the model response at a fundamental level and may therefore lead to different (even conflicting) conclusions about the underlying sensitivities. Here we revisit the theoretical basis for sensitivity analysis, summarize and critically evaluate existing approaches in the literature, and demonstrate their flaws and shortcomings through conceptual examples. We also demonstrate the difficulty involved in interpreting “global” interaction effects, which may undermine the value of existing interpretive approaches. With this background, we identify several important properties of response surfaces that are associated with the understanding and interpretation of sensitivities in the context of Earth and Environmental System models. Finally, we highlight the need for a new, comprehensive framework for sensitivity analysis that effectively characterizes all of the important sensitivity-related properties of model response surfaces.

A New Formulation for Feedforward Neural Networks

Feedforward neural network is one of the most commonly used function approximation techniques and has been applied to a wide variety of problems arising from various disciplines. However, neural networks are black-box models having multiple challenges/difficulties associated with training and generalization. This paper initially looks into the internal behavior of neural networks and develops a detailed interpretation of the neural network functional geometry. Based on this geometrical interpretation, a new set of variables describing neural networks is proposed as a more effective and geometrically interpretable alternative to the traditional set of network weights and biases. Then, this paper develops a new formulation for neural networks with respect to the newly defined variables; this reformulated neural network (ReNN) is equivalent to the common feedforward neural network but has a less complex error response surface. To demonstrate the learning ability of ReNN, in this paper, two training methods involving a derivative-based (a variation of backpropagation) and a derivative-free optimization algorithms are employed. Moreover, a new measure of regularization on the basis of the developed geometrical interpretation is proposed to evaluate and improve the generalization ability of neural networks. The value of the proposed geometrical interpretation, the ReNN approach, and the new regularization measure are demonstrated across multiple test problems. Results show that ReNN can be trained more effectively and efficiently compared to the common neural networks and the proposed regularization measure is an effective indicator of how a network would perform in terms of generalization.

Toward understanding nonstationarity in climate and hydrology through tree ring proxy records

Natural proxy records of hydroclimatic behavior, such as tree ring chronologies, are a rich source of information of past climate-driven nonstationarities in hydrologic variables. In this study, we investigate tree ring chronologies that demonstrate significant correlations with streamflows, with the objective of identifying the spatiotemporal patterns and extents of nonstationarities in climate and hydrology, which are essentially representations of past “climate changes.” First and second-order nonstationarities are of particular interest in this study. As a prerequisite, we develop a methodology to assess the consistency and credibility of a regional network of tree ring chronologies as proxies for hydrologic regime. This methodology involves a cluster analysis of available tree ring data to understand and evaluate their dependence structure, and a regional temporal-consistency plot to assess the consistency of different chronologies over time. The major headwater tributaries of the Saskatchewan River basin (SaskRB), the main source of surface water in the Canadian Prairie Provinces, are used as the case study. Results indicate that stationarity might never have existed in the hydrology of the region, as the statistical properties of annual paleo-hydrologic proxy records across the basin, i.e., the mean and autocorrelation structure, have consistently undergone significant changes (nonstationarities) at different points in the history of the region. The spatial pattern of the changes in the mean statistic has been variable with time, indicating a time-varying cross-correlation structure across the tributaries of the SaskRB. Conversely, the changes in the autocorrelation structure across the basin have been in harmony over time. The results demonstrate that the 89 year period of observational record in this region is a poor representation of the long-term properties of the hydrologic regime, and shorter periods, e.g., 30 year periods, are by no means representative. This paper highlights the need to broaden the understanding of hydrologic characteristics in any basin beyond the limited observational records, as an improved understanding is essential for more reliable assessment and management of available water resources.

A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory

Computer simulation models are continually growing in complexity with increasingly more factors to be identified. Sensitivity Analysis (SA) provides an essential means for understanding the role and importance of these factors in producing model responses. However, conventional approaches to SA suffer from (1) an ambiguous characterization of sensitivity, and (2) poor computational efficiency, particularly as the problem dimension grows. Here, we present a new and general sensitivity analysis framework (called VARS), based on an analogy to “variogram analysis,” that provides an intuitive and comprehensive characterization of sensitivity across the full spectrum of scales in the factor space. We prove, theoretically, that Morris (derivative-based) and Sobol (variance-based) methods and their extensions are special cases of VARS, and that their SA indices can be computed as by-products of the VARS framework. Synthetic functions that resemble actual model response surfaces are used to illustrate the concepts, and show VARS to be as much as two orders of magnitude more computationally efficient than the state-of-the-art Sobol approach. In a companion paper, we propose a practical implementation strategy, and demonstrate the effectiveness, efficiency, and reliability (robustness) of the VARS framework on real-data case studies.

A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application

Based on the theoretical framework for sensitivity analysis called “Variogram Analysis of Response Surfaces” (VARS), developed in the companion paper, we develop and implement a practical “star-based” sampling strategy (called STAR-VARS), for the application of VARS to real-world problems. We also develop a bootstrap approach to provide confidence level estimates for the VARS sensitivity metrics and to evaluate the reliability of inferred factor rankings. The effectiveness, efficiency, and robustness of STAR-VARS are demonstrated via two real-data hydrological case studies (a 5-parameter conceptual rainfall-runoff model and a 45-parameter land surface scheme hydrology model), and a comparison with the “derivative-based” Morris and “variance-based” Sobol approaches are provided. Our results show that STAR-VARS provides reliable and stable assessments of “global” sensitivity across the full range of scales in the factor space, while being 1–2 orders of magnitude more efficient than the Morris or Sobol approaches.

Evaluation of Integrated Multisatellite Retrievals for GPM (IMERG) over Southern Canada against Ground Precipitation Observations: A Preliminary Assessment

AbstractThe Global Precipitation Measurement (GPM) mission offers new opportunities for modeling a range of physical/hydrological processes at higher resolutions, especially for remote river systems where the hydrometeorological monitoring network is sparse and weather radar is not readily available. In this study, the recently released Integrated Multisatellite Retrievals for GPM [version 03 (V03) IMERG Final Run] product with high spatiotemporal resolution of 0.1° and 30 min is evaluated against ground-based reference measurements (at the 6-hourly, daily, and monthly time scales) over different terrestrial ecozones of southern Canada within a 23-month period from 12 March 2014 to 31 January 2016. While IMERG and ground-based observations show similar regional variations of mean daily precipitation, IMERG tends to overestimate higher monthly precipitation amounts over the Pacific Maritime ecozone. Results from using continuous as well as categorical skill metrics reveal that IMERG shows more satisfactory...

Adaptive Neural Networks for Flood Routing in River Systems

Abstract A methodology based on adaptive ANN models is proposed for flood routing in river systems. The proposed methodology is capable of modeling both converging and diverging river networks. A Multilayer Perceptron Network (MLP), a Recurrent Neural Network (RNN), a Time Delay Neural Network (TDNN) and a Time Delay Recurrent Neural Network (TDRNN) are applied in this study. An Adaptive training procedure based on the Forgetting Factor (FF) approach is used to train ANNs models. The methodology provides a lead time equal to travel time for the flood estimation downstream of the river. The performances of the models are tested within the two distinctive parts of the Karoon River in south-west Iran. The first case study uses synthetic floods generated by the HEC-RAS hydraulic model; the second one uses observed floods. Besides, the Muskingum routing method is used in the second case study to be compared with the results of ANN models. Overall, the results demonstrated that the proposed methodology performs well considering goodness-of-fit criteria. Moreover, the dynamic neural networks outperform the static MLP and the Muskingum model.

Progressive Latin Hypercube Sampling

Efficient sampling strategies that scale with the size of the problem, computational budget, and users needs are essential for various sampling-based analyses, such as sensitivity and uncertainty analysis. In this study, we propose a new strategy, called Progressive Latin Hypercube Sampling (PLHS), which sequentially generates sample points while progressively preserving the distributional properties of interest (Latin hypercube properties, space-filling, etc.), as the sample size grows. Unlike Latin hypercube sampling, PLHS generates a series of smaller sub-sets (slices) such that (1) the first slice is Latin hypercube, (2) the progressive union of slices remains Latin hypercube and achieves maximum stratification in any one-dimensional projection, and as such (3) the entire sample set is Latin hypercube. The performance of PLHS is compared with benchmark sampling strategies across multiple case studies for Monte Carlo simulation, sensitivity and uncertainty analysis. Our results indicate that PLHS leads to improved efficiency, convergence, and robustness of sampling-based analyses. A new sequential sampling strategy called PLHS is proposed for sampling-based analysis of simulation models.PLHS is evaluated across multiple case studies for Monte Carlo simulation, sensitivity and uncertainty analysis.PLHS provides better performance compared with the other sampling strategies in terms of convergence rate and robustness.PLHS can be used to monitor the performance of the associated sampling-based analysis and to avoid over- or under-sampling.

Correlation and causation in tree‐ring‐based reconstruction of paleohydrology in cold semiarid regions

This paper discusses ways in which the tree-ring based reconstruction of paleohydrology can be better understood and better utilized to support water resource management, especially in cold semi-arid regions. The relationships between tree growth, as represented by tree-ring chronologies (TRCs), runoff (Q), precipitation (P), and evapotranspiration (ET) are discussed and analyzed within both statistical and hydrological contexts. Data from the Oldman River Basin (OMRB), Alberta, Canada, are used to demonstrate the relevant issues. Instrumental records of Q and P data were available while actual ET was estimated using a lumped conceptual hydrological model developed in this study. Correlation analysis was conducted to explore the relationships between TRCs and each of Q, P, and ET over the entire historical record (globally) as well as locally in time within the wet and dry subperiods. Global and local correlation strengths and linear relationships appear to be substantially different. This outcome particularly affects tree-ring based inferences about the hydrology of wet and dry episodes when reconstructions are made using regression models. Important findings include: (i) reconstruction of paleoQ may not be as credible as paleoP and paleoET; (ii) a moving average window of P and ET larger than one year might be necessary for reconstruction of these variables; and (iii) the long term mean of reconstructed P, Q, and ET leads us to conclude that there is uncertainty about the past climate. And finally, we suggest using the topographic index to pre-judge side suitability for dendrohydrological analysis. This article is protected by copyright. All rights reserved.