James D. Brown

The Ensemble Verification System (EVS): A software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations

Ensemble forecasting is widely used in meteorology and, increasingly, in hydrology to quantify and propagate uncertainty. In practice, ensemble forecasts cannot account for every source of uncertainty, and many uncertainties are difficult to quantify accurately. Thus, ensemble forecasts are subject to errors, which may be correlated in space and time and may be systematic. Ensemble verification is necessary to quantify these errors, and to better understand the sources of predictive error and skill in particular modeling situations. The Ensemble Verification System (EVS) is a flexible, user-friendly, software tool that is designed to verify ensemble forecasts of numeric variables, such as temperature, precipitation and streamflow. It can be applied to forecasts from any number of discrete locations, which may be issued with any frequency and lead time. The EVS can also produce and verify forecasts that are aggregated in time, such as daily precipitation totals based on hourly forecasts, and can aggregate verification statistics across several discrete locations. This paper is separated into four parts. It begins with an overview of the EVS and the structure of the Graphical User Interface. The verification metrics available in the EVS are then described. These include metrics that verify the forecast probabilities and metrics that verify the ensemble mean forecast. Several new verification metrics are also presented. Following a description of the Application Programming Interface, the procedure for adding a new metric to the EVS is briefly outlined. Finally, the EVS is illustrated with two examples from the National Weather Service (NWS), one focusing on ensemble forecasts of precipitation from the NWS Ensemble Pre-Processor and one focusing on ensemble forecasts of streamflow from the NWS Ensemble Streamflow Prediction system. The conclusions address future enhancements to, and applications of, the EVS.

A Nonparametric Postprocessor for Bias Correction of Hydrometeorological and Hydrologic Ensemble Forecasts

Abstract This paper describes a technique for quantifying and removing biases from ensemble forecasts of hydrometeorological and hydrologic variables. The technique makes no a priori assumptions about the distributional form of the variables, which is often unknown or difficult to model parametrically. The aim is to estimate the conditional cumulative distribution function (ccdf) of the observed variable given a (possibly biased) real-time ensemble forecast. This ccdf represents the “true” probability distribution of the forecast variable, subject to sampling uncertainties. In the absence of a known distributional form, the ccdf should be estimated nonparametrically. It is noted that the probability of exceeding a threshold of the observed variable, such as flood stage, is equivalent to the expectation of an indicator variable defined for that threshold. The ccdf is then modeled through a linear combination of the indicator variables of the forecast ensemble members. The technique is based on Bayesian opt...

Verification of Precipitation Forecasts from NCEP’s Short-Range Ensemble Forecast (SREF) System with Reference to Ensemble Streamflow Prediction Using Lumped Hydrologic Models

AbstractPrecipitation forecasts from the Short-Range Ensemble Forecast (SREF) system of the National Centers for Environmental Prediction (NCEP) are verified for the period April 2006–August 2010. Verification is conducted for 10–20 hydrologic basins in each of the following: the middle Atlantic, the southern plains, the windward slopes of the Sierra Nevada, and the foothills of the Cascade Range in the Pacific Northwest. Mean areal precipitation is verified conditionally upon forecast lead time, amount of precipitation, season, forecast valid time, and accumulation period. The stationary block bootstrap is used to quantify the sampling uncertainties of the verification metrics. In general, the forecasts are more skillful for moderate precipitation amounts than either light or heavy precipitation. This originates from a threshold-dependent conditional bias in the ensemble mean forecast. Specifically, the forecasts overestimate low observed precipitation and underestimate high precipitation (a type-II cond...

Assessing the Skill of Medium-Range Ensemble Precipitation and Streamflow Forecasts from the Hydrologic Ensemble Forecast Service (HEFS) for the Upper Trinity River Basin in North Texas

AbstractTo issue early warnings for the public to act, for emergency managers to take preventive actions, and for water managers to operate their systems cost-effectively, it is necessary to maximi...