Mark Thyer

Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: A case study using Bayesian total error analysis

The lack of a robust framework for quantifying the parametric and predictive uncertainty of conceptual rainfall-runoff (CRR) models remains a key challenge in hydrology. The Bayesian total error analysis (BATEA) methodology provides a comprehensive framework to hypothesize, infer, and evaluate probability models describing input, output, and model structural error. This paper assesses the ability of BATEA and standard calibration approaches (standard least squares (SLS) and weighted least squares (WLS)) to address two key requirements of uncertainty assessment: (1) reliable quantification of predictive uncertainty and (2) reliable estimation of parameter uncertainty. The case study presents a challenging calibration of the lumped GR4J model to a catchment with ephemeral responses and large rainfall gradients. Postcalibration diagnostics, including checks of predictive distributions using quantile-quantile analysis, suggest that while still far from perfect, BATEA satisfied its assumed probability models better than SLS and WLS. In addition, WLS/SLS parameter estimates were highly dependent on the selected rain gauge and calibration period. This will obscure potential relationships between CRR parameters and catchment attributes and prevent the development of meaningful regional relationships. Conversely, BATEA provided consistent, albeit more uncertain, parameter estimates and thus overcomes one of the obstacles to parameter regionalization. However, significant departures from the calibration assumptions remained even in BATEA, e.g., systematic overestimation of predictive uncertainty, especially in validation. This is likely due to the inferred rainfall errors compensating for simplified treatment of model structural error.

Probabilistic optimization for conceptual rainfall-runoff models : A comparison of the shuffled complex evolution and simulated annealing algorithms

Automatic optimization algorithms are used routinely to calibrate conceptual rainfall-runoff (CRR) models. The goal of calibration is to estimate a feasible and unique (global) set of parameter estimates that best fit the observed runoff data. Most if not all optimization algorithms have difficulty in locating the global optimum because of response surfaces that contain multiple local optima with regions of attraction of differing size, discontinuities, and long ridges and valleys. Extensive research has been undertaken to develop efficient and robust global optimization algorithms over the last 10 years. This study compares the performance of two probabilistic global optimization methods: the shuffled complex evolution algorithm SCE-UA, and the three-phase simulated annealing algorithm SA-SX. Both algorithms are used to calibrate two parameter sets of a modified version of Boughtons [1984] SFB model using data from two Australian catchments that have low and high runoff yields. For the reduced, well-identified parameter set the algorithms have a similar efficiency for the low-yielding catchment, but SCE-UA is almost twice as robust. Although the robustness of the algorithms is similar for the high-yielding catchment, SCE-UA is six times more efficient than SA-SX. When fitting the full parameter set the performance of SA-SX deteriorated markedly for both catchments. These results indicated that SCE-UAs use of multiple complexes and shuffling provided a more effective search of the parameter space than SA-SXs single simplex with stochastic step acceptance criterion, especially when the level of parameterization is increased. Examination of the response surface for the low-yielding catchment revealed some reasons why SCE-UA outperformed SA-SX and why probabilistic optimization algorithms can experience difficulty in locating the global optimum.

Modeling long‐term persistence in hydroclimatic time series using a hidden state Markov Model

A hidden state Markov (HSM) model is developed as a new approach for generating hydroclimatic time series with long-term persistence. The two-state HSM model is motivated by the fact that the interaction of global climatic mechanisms produces alternating wet and dry regimes in Australian hydroclimatic time series. The HSM model provides an explicit mechanism to stochastically simulate these quasi-cyclic wet and dry periods. This is conceptually sounder than the current stochastic models used for hydroclimatic time series simulation. Models such as the lag-one autoregressive (AR(1))) model have no explicit mechanism for simulating the wet and dry regimes. In this study the HSM model was calibrated to four long-term Australian hydroclimatic data sets. A Markov Chain Monte Carlo method known as the Gibbs sampler was used for model calibration. The results showed that the locations significantly influenced by tropical weather systems supported the assumptions of the HSM modeling framework and indicated a strong persistence structure. In contrast, the calibration of the AR(1) model to these data sets produced no statistically significant evidence of persistence.

Quantifying parameter uncertainty in stochastic models using the Box-Cox transformation

Abstract The Box–Cox transformation is widely used to transform hydrological data to make it approximately Gaussian. Bayesian evaluation of parameter uncertainty in stochastic models using the Box–Cox transformation is hindered by the fact that there is no analytical solution for the posterior distribution. However, the Markov chain Monte Carlo method known as the Metropolis algorithm can be used to simulate the posterior distribution. This method properly accounts for the nonnegativity constraint implicit in the Box–Cox transformation. Nonetheless, a case study using the AR(1) model uncovered a practical problem with the implementation of the Metropolis algorithm. The use of a multivariate Gaussian jump distribution resulted in unacceptable convergence behaviour. This was rectified by developing suitable parameter transformations for the mean and variance of the AR(1) process to remove the strong nonlinear dependencies with the Box–Cox transformation parameter. Applying this methodology to the Sydney annual rainfall data and the Burdekin River annual runoff data illustrates the efficacy of these parameter transformations and demonstrate the value of quantifying parameter uncertainty.

Comparison of joint versus postprocessor approaches for hydrological uncertainty estimation accounting for error autocorrelation and heteroscedasticity

The paper appraises two approaches for the treatment of heteroscedasticity and autocorrelation in residual errors of hydrological models. Both approaches use weighted least squares (WLS), with heteroscedasticity modeled as a linear function of predicted flows and autocorrelation represented using an AR(1) process. In the first approach, heteroscedasticity and autocorrelation parameters are inferred jointly with hydrological model parameters. The second approach is a two-stage “postprocessor” scheme, where Stage 1 infers the hydrological parameters ignoring autocorrelation and Stage 2 conditionally infers the heteroscedasticity and autocorrelation parameters. These approaches are compared to a WLS scheme that ignores autocorrelation. Empirical analysis is carried out using daily data from 12 US catchments from the MOPEX set using two conceptual rainfall-runoff models, GR4J, and HBV. Under synthetic conditions, the postprocessor and joint approaches provide similar predictive performance, though the postprocessor approach tends to underestimate parameter uncertainty. However, the MOPEX results indicate that the joint approach can be nonrobust. In particular, when applied to GR4J, it often produces poor predictions due to strong multiway interactions between a hydrological water balance parameter and the error model parameters. The postprocessor approach is more robust precisely because it ignores these interactions. Practical benefits of accounting for error autocorrelation are demonstrated by analyzing streamflow predictions aggregated to a monthly scale (where ignoring daily-scale error autocorrelation leads to significantly underestimated predictive uncertainty), and by analyzing one-day-ahead predictions (where accounting for the error autocorrelation produces clearly higher precision and better tracking of observed data). Including autocorrelation into the residual error model also significantly affects calibrated parameter values and uncertainty estimates. The paper concludes with a summary of outstanding challenges in residual error modeling, particularly in ephemeral catchments.

A strategy for diagnosing and interpreting hydrological model nonstationarity

This paper presents a strategy for diagnosing and interpreting hydrological nonstationarity, aiming to improve hydrological models and their predictive ability under changing hydroclimatic conditions. The strategy consists of four elements: (i) detecting potential systematic errors in the calibration data; (ii) hypothesizing a set of “nonstationary” parameterizations of existing hydrological model structures, where one or more parameters vary in time as functions of selected covariates; (iii) trialing alternative stationary model structures to assess whether parameter nonstationarity can be reduced by modifying the model structure; and (iv) selecting one or more models for prediction. The Scott Creek catchment in South Australia and the lumped hydrological model GR4J are used to illustrate the strategy. Streamflow predictions improve significantly when the GR4J parameter describing the maximum capacity of the production store is allowed to vary in time as a combined function of: (i) an annual sinusoid; (ii) the previous 365 day rainfall and potential evapotranspiration; and (iii) a linear trend. This improvement provides strong evidence of model nonstationarity. Based on a range of hydrologically oriented diagnostics such as flow-duration curves, the GR4J model structure was modified by introducing an additional calibration parameter that controls recession behavior and by making actual evapotranspiration dependent only on catchment storage. Model comparison using an information-theoretic measure (the Akaike Information Criterion) and several hydrologically oriented diagnostics shows that the GR4J modifications clearly improve predictive performance in Scott Creek catchment. Based on a comparison of 22 versions of GR4J with different representations of nonstationarity and other modifications, the model selection approach applied in the exploratory period (used for parameter estimation) correctly identifies models that perform well in a much drier independent confirmatory period.

Goulburn River experimental catchment data set

(651 km 2 ) and Krui (562 km 2 ) subcatchments in the northern half of this experimental catchment with a few monitoring sites located in the south. The data set comprises soil temperature and moisture profile measurements from 26 locations; meteorological data from two automated weather stations (data from a further three stations are available from other sources) including precipitation, atmospheric pressure, air temperature and relative humidity, wind speed and direction, soil heat flux, and up- and down-welling shortand long-wave radiation; streamflow observations at five nested locations (data from a further three locations are available from other sources); a total of three surface soil moisture maps across a 40 km � 50 km region in the north from � 200 measurement locations during intensive field campaigns; and a high-resolution digital elevation model (DEM) of a 175-ha microcatchment in the Krui catchment. These data are available on the World Wide Web at http://www.sasmas.unimelb.edu.au.

There are no hydrological monsters, just models and observations with large uncertainties!

Abstract Catchments that do not behave in the way the hydrologist expects, expose the frailties of hydrological science, particularly its unduly simplistic treatment of input and model uncertainty. A conceptual rainfall–runoff model represents a highly simplified hypothesis of the transformation of rainfall into runoff. Sub-grid variability and mis-specification of processes introduce an irreducible model error, about which little is currently known. In addition, hydrological observation systems are far from perfect, with the principal catchment forcing (rainfall) often subject to large sampling errors. When ignored or treated simplistically, these errors develop into monsters that destroy our ability to model certain catchments. In this paper, these monsters are tackled using Bayesian Total Error Analysis, a framework that accounts for user-specified sources of error and yields quantitative insights into how prior knowledge of these uncertainties affects our ability to infer models and use them for predictive purposes. A case study involving a catchment with an apparent water balance anomaly (a hydrological monstrosity!) illustrates these concepts. It is found that, in the absence of additional information, the rainfall–runoff record is insufficient to explain this anomaly – it could be due to a large export of groundwater, systematic overestimation of catchment rainfall of the order of 40%, or a conspiracy of these factors. There is “no free lunch” in hydrology. The rainfall–runoff record on its own is insufficient to decompose the different sources of uncertainty affecting calibration, testing and prediction, and hydrological monstrosities will persist until additional independent knowledge of uncertainties is obtained. Citation Kuczera, G., Renard, B., Thyer, M. & Kavetski, D. (2010) There are no hydrological monsters, just models and observations with large uncertainties! Hydrol. Sci. J. 55(6), 980–991.

A hidden Markov model for modelling long-term persistence in multi-site rainfall time series 1. Model calibration using a Bayesian approach

A Bayesian approach for calibrating a hidden Markov model (HMM) to long-term multi-site rainfall time series is presented. Using a HMM approach for simulating long-term persistence is attractive because it has an explicit mechanism to produce long-term wet and dry periods which are a feature of many long-term hydrological time series. The ability to fully evaluate parameter uncertainty for the multi-site HMM represents an advance in the stochastic modelling of long-term persistence in multi-site hydrological time series. The challenges in applying the Bayesian Markov chain Monte Carlo (MCMC) method known as the Gibbs sampler to infer the posterior distribution of the multi-site HMM parameters are fully outlined. The specification of appropriate prior distributions was found to be crucial for the successful implementation of the Gibbs sampler. It is described how using synthetic data led to the development of an appropriate prior specification. Further synthetic data analysis showed how the benefits of space-for-time substitution for identifying the long-term persistence structure are dependent on the spatial correlation that exists in multi-site data. A methodology for handling missing data is also described. This study highlights the important role of the priors in Bayesian analysis using MCMC methods by illustrating that misleading inferences can result if the priors are inappropriately specified.

A hidden Markov model for modelling long-term persistence in multi-site rainfall time series. 2. Real data analysis

Abstract The hidden Markov model (HMM) provides an attractive framework for modelling long-term persistence in hydrological data because it can produce time series with long-term wet and dry periods. In this study, the Bayesian calibration procedure for the multi-site HMM developed by Thyer and Kuczera [J. Hydrol. (2003)] is used to calibrate the model to multi-site rainfall data from the Warragamba, Central Coast and Williams River catchment regions—all important water supply catchments located on the east coast of Australia. This methodology is used to verify the majority of the HMM assumptions. The results for the Warragamba and Central Coast catchment region provided strong evidence that a model with a two-state persistence structure was more consistent with the data than a one state model with no persistence. These findings may have considerable implications for water resources management and drought risk assessment in both these regions. In addition, the results suggested that the multi-site framework exploits space-for-time substitution and the sampling of missing data to better identify the long-term persistence structure. For the Williams River catchment rainfall data difficulties were experienced with achieving convergence of the calibration procedure because of bimodal posterior distributions. While the results suggested a two-state persistence structure exists for the Williams, the difficulties indicate there is scope for further refinement of the implementation of the HMM concept.