Grey S. Nearing

The quantity and quality of information in hydrologic models

The role of models in science is to facilitate predictions from hypotheses. Although the idea that models provide information is widely reported and has been used as the basis for model evaluation, benchmarking, and updating strategies, this intuition has not been formally developed and current benchmarking strategies remain ad hoc at a fundamental level. Here we interpret what it means to say that a model provides information in the context of the formal inductive philosophy of science. We show how information theory can be used to measure the amount of information supplied by a model, and derive standard model benchmarking and evaluation activities in this context. We further demonstrate that, via a process of induction, dynamical models store information from hypotheses and observations about the systems that they represent, and that this stored information can be directly measured.

The impact of vertical measurement depth on the information content of soil moisture times series data

Using a decade of ground-based soil moisture observations acquired from the United States Department of Agricultures Soil Climate Analysis Network (SCAN), we calculate the mutual information (MI) content between multiple soil moisture variables and near-future vegetation condition to examine the existence of emergent drought information in vertically integrated (surface to 60 cm) soil moisture observations (theta(0-60) ([cm])) not present in either superficial soil moisture observations (theta(5) ([cm])) or a simple low-pass transformation of theta(5). Results suggest that while theta(0-60) is indeed more valuable than theta(5) for predicting near-future vegetation anomalies, the enhanced information content in theta(0-60) soil moisture can be effectively duplicated by the low-pass transformation of theta(5). This implies that, for drought monitoring applications, the shallow vertical penetration depth of microwave-based theta(5) retrievals does not represent as large a practical limitation as commonly perceived.

Estimating information entropy for hydrological data: One‐dimensional case

There has been a recent resurgence of interest in the application of Information Theory to problems of system identification in the Earth and Environmental Sciences. While the concept of entropy has found increased application, little attention has yet been given to the practical problems of estimating entropy when dealing with the unique characteristics of two commonly used kinds of hydrologic data: rainfall and runoff. In this paper, we discuss four important issues of practical relevance that can bias the computation of entropy if not properly handled. The first (zero effect) arises when precipitation and ephemeral streamflow data must be viewed as arising from a discrete-continuous hybrid distribution due to the occurrence of many zero values (e.g., days with no rain/no runoff). Second, in the widely used bin-counting method for estimation of PDFs, significant error can be introduced if the bin width is not carefully selected. The third (measurement effect) arises due to the fact that continuously varying hydrologic variables can typically only be observed discretely to some degree of precision. The Fourth (skewness effect) arises when the distribution of a variable is significantly skewed. Here we present an approach that can deal with all four of these issues, and test them with artificially generated and real hydrological data. The results indicate that the method is accurate and robust.

Performance Metrics, Error Modeling, and Uncertainty Quantification

AbstractA common set of statistical metrics has been used to summarize the performance of models or measurements—the most widely used ones being bias, mean square error, and linear correlation coefficient. They assume linear, additive, Gaussian errors, and they are interdependent, incomplete, and incapable of directly quantifying uncertainty. The authors demonstrate that these metrics can be directly derived from the parameters of the simple linear error model. Since a correct error model captures the full error information, it is argued that the specification of a parametric error model should be an alternative to the metrics-based approach. The error-modeling methodology is applicable to both linear and nonlinear errors, while the metrics are only meaningful for linear errors. In addition, the error model expresses the error structure more naturally, and directly quantifies uncertainty. This argument is further explained by highlighting the intrinsic connections between the performance metrics, the erro...

Benchmarking of a Physically Based Hydrologic Model

AbstractThe concepts of model benchmarking, model agility, and large-sample hydrology are becoming more prevalent in hydrologic and land surface modeling. As modeling systems become more sophisticated, these concepts have the ability to help improve modeling capabilities and understanding. In this paper, their utility is demonstrated with an application of the physically based Variable Infiltration Capacity model (VIC). The authors implement VIC for a sample of 531 basins across the contiguous United States, incrementally increase model agility, and perform comparisons to a benchmark. The use of a large-sample set allows for statistically robust comparisons and subcategorization across hydroclimate conditions. Our benchmark is a calibrated, time-stepping, conceptual hydrologic model. This model is constrained by physical relationships such as the water balance, and it complements purely statistical benchmarks due to the increased physical realism and permits physically motivated benchmarking using metrics...

The Impact of Vertical Measurement Depth on the Information Content of Soil Moisture for Latent Heat Flux Estimation

AbstractThis study aims to identify the impact of vertical support on the information content of soil moisture (SM) for latent heat flux estimation. This objective is achieved via calculation of the mutual information (MI) content between multiple soil moisture variables (with different vertical supports) and current/future evaporative fraction (EF) using ground-based soil moisture and latent/sensible heat flux observations acquired from the AmeriFlux network within the contiguous United States. Through the intercomparison of MI results from different SM–EF pairs, the general value (for latent heat flux estimation) of superficial soil moisture observations , vertically integrated soil moisture observations , and vertically extrapolated soil moisture time series [soil wetness index (SWI) from a simple low-pass transformation of ] are examined. Results suggest that, contrary to expectations, 2-day averages of and have comparable mutual information with regards to EF. That is, there is no clear evidence that...

Ensembles vs. information theory: supporting science under uncertainty

Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.

Nonparametric triple collocation

Triple collocation has found widespread application in the hydrological sciences because it provides information about the errors in our measurements without requiring that we have any direct access to the true value of the variable being measured. Triple collocation derives variance-covariance relationships between three or more independent measurement sources and an indirectly observed truth variable in the case where the measurement operators are additive. We generalize that theory to arbitrary observation operators by deriving nonparametric analogues to the total error and total correlation statistics as integrations of divergences from conditional to marginal probability ratios. The nonparametric solution to the full measurement problem is under-determined, and we therefore retrieve conservative bounds on the theoretical total nonparametric error and correlation statistics. We examine the application of both linear and nonlinear triple collocation to synthetic examples and to a real-data test case related to evaluating space-borne soil moisture retrievals using sparse monitoring networks and dynamical process models.

A ranking of hydrological signatures based on their predictability in space

Hydrological signatures are now used for a wide range of purposes, including catchment classification, process exploration and hydrological model calibration. The recent boost in the popularity and number of signatures has however not been accompanied by the development of clear guidance on signature selection. Here we propose that exploring the predictability of signatures in space provides important insights into their drivers, their sensitivity to data uncertainties, and is hence useful for signature selection. We use three complementary approaches to compare and rank 15 commonly‐used signatures, which we evaluate in 671 US catchments from the CAMELS data set (Catchment Attributes and MEteorology for Large‐sample Studies). Firstly, we employ machine learning (random forests) to explore how attributes characterizing the climatic conditions, topography, land cover, soil and geology influence (or not) the signatures. Secondly, we use simulations of a conceptual hydrological model (Sacramento) to benchmark the random forest predictions. Thirdly, we take advantage of the large sample of CAMELS catchments to characterize the spatial auto‐correlation (using Morans I) of the signature field. These three approaches lead to remarkably similar rankings of the signatures. We show i) that signatures with the noisiest spatial pattern tend to be poorly captured by hydrological simulations, ii) that their relationship to catchments attributes are elusive (in particular they are not correlated to climatic indices) and iii) that they are particularly sensitive to discharge uncertainties. We suggest that a better understanding of their drivers and better characterization of their uncertainties would increase their value in hydrological studies.