Steven Vincent Weijs

Kullback–Leibler Divergence as a Forecast Skill Score with Classic Reliability–Resolution–Uncertainty Decomposition

Abstract This paper presents a score that can be used for evaluating probabilistic forecasts of multicategory events. The score is a reinterpretation of the logarithmic score or ignorance score, now formulated as the relative entropy or Kullback–Leibler divergence of the forecast distribution from the observation distribution. Using the information–theoretical concepts of entropy and relative entropy, a decomposition into three components is presented, analogous to the classic decomposition of the Brier score. The information–theoretical twins of the components uncertainty, resolution, and reliability provide diagnostic information about the quality of forecasts. The overall score measures the information conveyed by the forecast. As was shown recently, information theory provides a sound framework for forecast verification. The new decomposition, which has proven to be very useful for the Brier score and is widely used, can help acceptance of the logarithmic score in meteorology.

Accounting for Observational Uncertainty in Forecast Verification: An Information-Theoretical View on Forecasts, Observations, and Truth

AbstractRecently, an information-theoretical decomposition of Kullback–Leibler divergence into uncertainty, reliability, and resolution was introduced. In this article, this decomposition is generalized to the case where the observation is uncertain. Along with a modified decomposition of the divergence score, a second measure, the cross-entropy score, is presented, which measures the estimated information loss with respect to the truth instead of relative to the uncertain observations. The difference between the two scores is equal to the average observational uncertainty and vanishes when observations are assumed to be perfect. Not acknowledging for observation uncertainty can lead to both overestimation and underestimation of forecast skill, depending on the nature of the noise process.

Could electrical conductivity replace water level in rating curves for alpine streams

Streamflow time series are important for inference and understanding of the hydrological processes in alpine watersheds. Because streamflow is expensive to continuously measure directly, it is usually derived from measured water levels, using a rating curve modeling the stage-discharge relationship. In alpine streams, this practice is complicated by the fact that the streambed constantly changes due to erosion and sedimentation by the turbulent mountain streams. This makes the stage-discharge relationship dynamic, requiring frequent discharge gaugings to have reliable streamflow estimates. During an ongoing field study in the Val Ferret watershed in the Swiss Alps, 93 streamflow values were measured in the period 2009–2011 using salt dilution gauging with the gulp injection method. The natural background electrical conductivity in the stream, which was measured as by-product of these gaugings, was shown to be a strong predictor for the streamflow, even marginally outperforming water level. Analysis of the residuals of both predictive relations revealed errors in the gauged streamflows. These could be corrected by filtering disinformation from erroneous calibration coefficients. In total, extracting information from the auxiliary data enabled to reduce the uncertainty in the rating curve, as measured by the root-mean-square error in log-transformed streamflow relative to that of the original stage-discharge relationship, by 43.7%.

HydroZIP : How hydrological knowledge can be used to improve compression of hydrological data

From algorithmic information theory, which connects the information content of a data set to the shortest computer program that can produce it, it is known that there are strong analogies between compression, knowledge, inference and prediction. The more we know about a data generating process, the better we can predict and compress the data. A model that is inferred from data should ideally be a compact description of those data. In theory, this means that hydrological knowledge could be incorporated into compression algorithms to more efficiently compress hydrological data and to outperform general purpose compression algorithms. In this study, we develop such a hydrological data compressor, named HydroZIP, and test in practice whether it can outperform general purpose compression algorithms on hydrological data from 431 river basins from the Model Parameter Estimation Experiment (MOPEX) data set. HydroZIP compresses using temporal dependencies and parametric distributions. Resulting file sizes are interpreted as measures of information content, complexity and model adequacy. These results are discussed to illustrate points related to learning from data, overfitting and model complexity.

Controls on the diurnal streamflow cycles in two subbasins of an alpine headwater catchment

In high-altitude alpine catchments, diurnal streamflow cycles are typically dominated by snowmelt or ice melt. Evapotranspiration-induced diurnal streamflow cycles are less observed in these catchments but might happen simultaneously. During a field campaign in the summer 2012 in an alpine catchment in the Swiss Alps (Val Ferret catchment, 20.4 km2, glaciarized area: 2%), we observed a transition in the early season from a snowmelt to an evapotranspiration-induced diurnal streamflow cycle in one of two monitored subbasins. The two different cycles were of comparable amplitudes and the transition happened within a time span of several days. In the second monitored subbasin, we observed an ice melt-dominated diurnal cycle during the entire season due to the presence of a small glacier. Comparisons between ice melt and evapotranspiration cycles showed that the two processes were happening at the same times of day but with a different sign and a different shape. The amplitude of the ice melt cycle decreased exponentially during the season and was larger than the amplitude of the evapotranspiration cycle which was relatively constant during the season. Our study suggests that an evapotranspiration-dominated diurnal streamflow cycle could damp the ice melt-dominated diurnal streamflow cycle. The two types of diurnal streamflow cycles were separated using a method based on the identification of the active riparian area and measurement of evapotranspiration.

Toward a New Approach for Hydrological Modeling: A Tool for Sustainable Development in a Savanna Agro-System

Agriculture in Tambarga, a small, remote village in the landlocked country of Burkina Faso, is dependent on the seasonally variable local hydrology. Extreme seasonal and spatial variability of rainfall significantly impacts the livelihood of farmers, who depend mainly on rainfed agriculture. This dependence on rainfed production makes them particularly vulnerable to meteorological conditions, and they continually experience food insecurity. The groundwater is promising as storage to mitigate effects of drought. However, because of its interaction with the various hydrological components, we need to better understand all the processes to fully assess the impacts of possible solutions. Hydrological and meteorological data were collected over a two-and-a-half-year period in the catchment adjacent to the village (area = 3.5 km²) to address these issues. The field studies show that the major portion of storm runoff was generated in the upper savanna basin, while baseflow appears to be mostly originating from the downstream agricultural field. The seasonal cycle of groundwater appears to control the stream flow and therefore, the continuous flow over the entire stream occurred when the water tables became interconnected and surfaced the ground level. Additionally, this paper discusses water management scenarios (open dam, deeper wells and buried dam) for agricultural purposes using a simple and comprehensive hydrological model. Simulations based on reducing evaporation rate by keeping the water underground present a solution that could improve agricultural production, and therefore, reduce vulnerability of Tambarga’s farmers to climate change.

Entropy Ensemble Filter: A Modified Bootstrap Aggregating (Bagging) Procedure to Improve Efficiency in Ensemble Model Simulation

Over the past two decades, the Bootstrap AGGregatING (bagging) method has been widely used for improving simulation. The computational cost of this method scales with the size of the ensemble, but excessively reducing the ensemble size comes at the cost of reduced predictive performance. The novel procedure proposed in this study is the Entropy Ensemble Filter (EEF), which uses the most informative training data sets in the ensemble rather than all ensemble members created by the bagging method. The results of this study indicate efficiency of the proposed method in application to synthetic data simulation on a sinusoidal signal, a sawtooth signal, and a composite signal. The EEF method can reduce the computational time of simulation by around 50% on average while maintaining predictive performance at the same level of the conventional method, where all of the ensemble models are used for simulation. The analysis of the error gradient (root mean square error of ensemble averages) shows that using the 40% most informative ensemble members of the set initially defined by the user appears to be most effective.

Application of Entropy Ensemble Filter in Neural Network Forecasts of Tropical Pacific Sea Surface Temperatures

Recently, the Entropy Ensemble Filter (EEF) method was proposed to mitigate the computational cost of the Bootstrap AGGregatING (bagging) method. This method uses the most informative training data sets in the model ensemble rather than all ensemble members created by the conventional bagging. In this study, we evaluate, for the first time, the application of the EEF method in Neural Network (NN) modeling of El Nino-southern oscillation. Specifically, we forecast the first five principal components (PCs) of sea surface temperature monthly anomaly fields over tropical Pacific, at different lead times (from 3 to 15 months, with a three-month increment) for the period 1979–2017. We apply the EEF method in a multiple-linear regression (MLR) model and two NN models, one using Bayesian regularization and one Levenberg-Marquardt algorithm for training, and evaluate their performance and computational efficiency relative to the same models with conventional bagging. All models perform equally well at the lead time of 3 and 6 months, while at higher lead times, the MLR model’s skill deteriorates faster than the nonlinear models. The neural network models with both bagging methods produce equally successful forecasts with the same computational efficiency. It remains to be shown whether this finding is sensitive to the dataset size.