Bryan A. Tolson

Dynamically dimensioned search algorithm for computationally efficient watershed model calibration

[1] A new global optimization algorithm, dynamically dimensioned search (DDS), is introduced for automatic calibration of watershed simulation models. DDS is designed for calibration problems with many parameters, requires no algorithm parameter tuning, and automatically scales the search to find good solutions within the maximum number of user-specified function (or model) evaluations. As a result, DDS is ideally suited for computationally expensive optimization problems such as distributed watershed model calibration. DDS performance is compared to the shuffled complex evolution (SCE) algorithm for multiple optimization test functions as well as real and synthetic SWAT2000 model automatic calibration formulations. Algorithms are compared for optimization problems ranging from 6 to 30 dimensions, and each problem is solved in 1000 to 10,000 total function evaluations per optimization trial. Results are presented so that future modelers can assess algorithm performance at a computational scale relevant to their modeling case study. In all four of the computationally expensive real SWAT2000 calibration formulations considered here (14, 14, 26, and 30 calibration parameters), results show DDS to be more efficient and effective than SCE. In two cases, DDS requires only 15–20% of the number of model evaluations used by SCE in order to find equally good values of the objective function. Overall, the results also show that DDS rapidly converges to good calibration solutions and easily avoids poor local optima. The simplicity of the DDS algorithm allows for easy recoding and subsequent adoption into any watershed modeling application framework.

Numerical assessment of metamodelling strategies in computationally intensive optimization

Metamodelling is an increasingly more popular approach for alleviating the computational burden associated with computationally intensive optimization/management problems in environmental and water resources systems. Some studies refer to the metamodelling approach as function approximation, surrogate modelling, response surface methodology or model emulation. A metamodel-enabled optimizer approximates the objective (or constraint) function in a way that eliminates the need to always evaluate this function via a computationally expensive simulation model. There is a sizeable body of literature developing and applying a variety of metamodelling strategies to various environmental and water resources related problems including environmental model calibration, water resources systems analysis and management, and water distribution network design and optimization. Overall, this literature generally implies metamodelling yields enhanced solution efficiency and (almost always) effectiveness of computationally intensive optimization problems. This paper initially develops a comparative assessment framework which presents a clear computational budget dependent definition for the success/failure of the metamodelling strategies, and then critically evaluates metamodelling strategies, through numerical experiments, against other common optimization strategies not involving metamodels. Three different metamodel-enabled optimizers involving radial basis functions, kriging, and neural networks are employed. A robust numerical assessment within different computational budget availability scenarios is conducted over four test functions commonly used in optimization as well as two real-world computationally intensive optimization problems in environmental and water resources systems. Numerical results show that metamodelling is not always an efficient and reliable approach to optimizing computationally intensive problems. For simpler response surfaces, metamodelling can be very efficient and effective. However, in some cases, and in particular for complex response surfaces when computational budget is not very limited, metamodelling can be misleading and a hindrance, and better solutions are achieved with optimizers not involving metamodels. Results also demonstrate that neural networks are not appropriate metamodelling tools for limited computational budgets while metamodels employing kriging and radial basis functions show comparable overall performance when the available computational budget is very limited.

First-order reliability method for estimating reliability, vulnerability, and resilience

Reliability, vulnerability, and resilience provide measures of the frequency, magnitude, and duration of the failure of water resources systems, respectively. Traditionally, these measures have been estimated using simulation. However, this can be computationally intensive, particularly when complex system-response models are used, when many estimates of the performance measures are required, and when persistence among the data needs to be taken into account. In this paper, an efficient method for estimating reliability, vulnerability, and resilience, which is based on the First-Order Reliability Method (FORM), is developed and demonstrated for the case study of managing water quality in the Willamette River, Oregon. Reliability, vulnerability, and resilience are determined for different dissolved oxygen (DO) standards. DO is simulated using a QUAL2EU water quality response model that has recently been developed for the Oregon Department of Environmental Quality (ODEQ) as part of the Willamette River Basin Water Quality Study (WRBWQS). The results obtained indicate that FORM can be used to efficiently estimate reliability, vulnerability, and resilience.

A New Formulation for Feedforward Neural Networks

Feedforward neural network is one of the most commonly used function approximation techniques and has been applied to a wide variety of problems arising from various disciplines. However, neural networks are black-box models having multiple challenges/difficulties associated with training and generalization. This paper initially looks into the internal behavior of neural networks and develops a detailed interpretation of the neural network functional geometry. Based on this geometrical interpretation, a new set of variables describing neural networks is proposed as a more effective and geometrically interpretable alternative to the traditional set of network weights and biases. Then, this paper develops a new formulation for neural networks with respect to the newly defined variables; this reformulated neural network (ReNN) is equivalent to the common feedforward neural network but has a less complex error response surface. To demonstrate the learning ability of ReNN, in this paper, two training methods involving a derivative-based (a variation of backpropagation) and a derivative-free optimization algorithms are employed. Moreover, a new measure of regularization on the basis of the developed geometrical interpretation is proposed to evaluate and improve the generalization ability of neural networks. The value of the proposed geometrical interpretation, the ReNN approach, and the new regularization measure are demonstrated across multiple test problems. Results show that ReNN can be trained more effectively and efficiently compared to the common neural networks and the proposed regularization measure is an effective indicator of how a network would perform in terms of generalization.

Battle of the Water Networks II

The Battle of the Water Networks II (BWN-II) is the latest of a series of competitions related to the design and operation of water distribution systems (WDSs) undertaken within the Water Distribution Systems Analysis (WDSA) Symposium series. The BWN-II problem specification involved a broadly defined design and operation problem for an existing network that has to be upgraded for increased future demands, and the addition of a new development area. The design decisions involved addition of new and parallel pipes, storage, operational controls for pumps and valves, and sizing of backup power supply. Design criteria involved hydraulic, water quality, reliability, and environmental performance measures. Fourteen teams participated in the Battle and presented their results at the 14th Water Distribution Systems Analysis conference in Adelaide, Australia, September 2012. This paper summarizes the approaches used by the participants and the results they obtained. Given the complexity of the BWN-II problem and the innovative methods required to deal with the multiobjective, high dimensional and computationally demanding nature of the problem, this paper represents a snap-shot of state of the art methods for the design and operation of water distribution systems. A general finding of this paper is that there is benefit in using a combination of heuristic engineering experience and sophisticated optimization algorithms when tackling complex real-world water distribution system design problems

Optimizing hydrological consistency by incorporating hydrological signatures into model calibration objectives

The simulated outcome of a calibrated hydrologic model should be hydrologically consistent with the measured response data. Hydrologic modelers typically calibrate models to optimize residual-based goodness-of-fit measures, e.g., the Nash-Sutcliffe efficiency measure, and then evaluate the obtained results with respect to hydrological signatures, e.g., the flow duration curve indices. The literature indicates that the consideration of a large number of hydrologic signatures has not been addressed in a full multiobjective optimization context. This research develops a model calibration methodology to achieve hydrological consistency using goodness-of-fit measures, many hydrological signatures, as well as a level of acceptability for each signature. The proposed framework relies on a scoring method that transforms any hydrological signature to a calibration objective. These scores are used to develop the hydrological consistency metric, which is maximized to obtain hydrologically consistent parameter sets during calibration. This consistency metric is implemented in different signature-based calibration formulations that adapt the sampling according to hydrologic signature values. These formulations are compared with the traditional formulations found in the literature for seven case studies. The results reveal that Pareto dominance-based multiobjective optimization yields the highest level of consistency among all formulations. Furthermore, it is found that the choice of optimization algorithms does not affect the findings of this research.

Progress in integrating remote sensing data and hydrologic modeling

Remote sensing and hydrologic modeling are two key approaches to evaluate and predict hydrology and water resources. Remote sensing technologies, due to their ability to offer large-scale spatially distributed observations, have opened up new opportunities for the development of fully distributed hydrologic and land-surface models. In general, remote sensing data can be applied to land-surface and hydrologic modeling through three strategies: model inputs (basin information, boundary conditions, etc.), parameter estimation (model calibration), and state estimation (data assimilation). There has been an intensive global research effort to integrate remote sensing and land/hydrologic modeling over the past few decades. In particular, in recent years significant progress has been made in land/hydrologic remote sensing data assimilation. Hence there is a demand for an up-to-date review on these efforts. This paper presents an overview of research efforts to combine hydrologic/land models and remote sensing products (mainly including precipitation, surface soil moisture, snow cover, snow water equivalent, leaf area index, and evapotranspiration) over the past decade. This paper also discusses the major challenges remaining in this field, and recommends the directions for further research efforts.

Pareto archived dynamically dimensioned search with hypervolume-based selection for multi-objective optimization

Pareto archived dynamically dimensioned search (PA-DDS) is a parsimonious multi-objective optimization algorithm with only one parameter to diminish the users effort for fine-tuning algorithm parameters. This study demonstrates that hypervolume contribution (HVC) is a very effective selection metric for PA-DDS and Monte Carlo sampling-based HVC is very effective for higher dimensional problems (five objectives in this study). PA-DDS with HVC performs comparably to algorithms commonly applied to water resources problems (ϵ-NSGAII and AMALGAM under recommended parameter values). Comparisons on the CEC09 competition show that with sufficient computational budget, PA-DDS with HVC performs comparably to 13 benchmark algorithms and shows improved relative performance as the number of objectives increases. Lastly, it is empirically demonstrated that the total optimization runtime of PA-DDS with HVC is dominated (90% or higher) by solution evaluation runtime whenever evaluation exceeds 10 seconds/solution. Therefore, optimization algorithm runtime associated with the unbounded archive of PA-DDS is negligible in solving computationally intensive problems.

A benchmarking framework for simulation-based optimization of environmental models

Simulation models assist with designing and managing environmental systems. Linking such models with optimization algorithms yields an approach for identifying least-cost solutions while satisfying system constraints. However, selecting the best optimization algorithm for a given problem is non-trivial and the community would benefit from benchmark problems for comparing various alternatives. To this end, we?propose a set of six guidelines for developing effective benchmark problems for simulation-based optimization.The proposed guidelines were used to investigate problems involving sorptive landfill liners for containing and treating hazardous waste. Two solution approaches were applied to these types of problems for the first time - a pre-emptive (i.e. terminating simulations early when appropriate) particle swarm optimizer (PSO), and a hybrid discrete variant of the dynamically dimensioned search algorithm (HD-DDS). Model pre-emption yielded computational savings of up to 70% relative to non-pre-emptive counterparts. Furthermore, HD-DDS often identified globally optimal designs while incurring minimal computational expense, relative to alternative algorithms. Results also highlight the usefulness of organizing decision variables in terms of cost values rather than grouping by material type.

Calibrating Environment Canada's MESH Modelling System over the Great Lakes Basin

Abstract This paper reports on recent progress towards improved predictions of a land surface-hydrological modelling system, Modélisation Environmentale–Surface et Hydrologie (MESH), via its calibration over the Laurentian Great Lakes Basin. Accordingly, a “global” calibration strategy is utilized in which parameters for all land class types are calibrated simultaneously to a number of sub-basins and then validated in time and space. Model performance was evaluated based on four performance metrics, including the Nash-Sutcliffe (NS) coefficient and simulated compared with observed hydrographs. Results from two calibration approaches indicate that in the model validation mode, the global strategy generates better results than an alternative calibration strategy, referred to as the “individual” strategy, in which parameters are calibrated individually to a single sub-basin with a dominant land type and then validated in a different sub-basin with the same dominant land type. The global calibration strategy was relatively successful despite the large number of calibration parameters (51) and relatively small number of model evaluations (1000) used in the automatic calibration procedure. The NS values for spatial validation range from 0.10 to 0.72 with a median of 0.41 for the 15 sub-basins considered. Results also confirm that a careful model calibration and validation is needed before any application of the model.