Stuart S. Schwartz

Sampling Uncertainty and Confidence Intervals for the Brier Score and Brier Skill Score

Abstract For probability forecasts, the Brier score and Brier skill score are commonly used verification measures of forecast accuracy and skill. Using sampling theory, analytical expressions are derived to estimate their sampling uncertainties. The Brier score is an unbiased estimator of the accuracy, and an exact expression defines its sampling variance. The Brier skill score (with climatology as a reference forecast) is a biased estimator, and approximations are needed to estimate its bias and sampling variance. The uncertainty estimators depend only on the moments of the forecasts and observations, so it is easy to routinely compute them at the same time as the Brier score and skill score. The resulting uncertainty estimates can be used to construct error bars or confidence intervals for the verification measures, or perform hypothesis testing. Monte Carlo experiments using synthetic forecasting examples illustrate the performance of the expressions. In general, the estimates provide very reliable inf...

Effective Curve Number and Hydrologic Design of Pervious Concrete Storm-Water Systems

The effective use of pervious concrete in environmental site design requires consistent design procedures integrating the structural and material properties of the pervious concrete pavement with hydrologic performance of the pervious concrete system. Design procedures to size pervious concrete storm-water systems are presented based on criteria for freeze-thaw protection and drawdown reliability. Hydrologic performance criteria are quantified by an effective curve number, estimated from simulated routing of design storm hydrographs using standard storm-water computations. Combining operational design criteria with the evaluation of hydrologic perfor- mance criteria, as an effective curve number, integrates pervious concrete systems with traditional storm-water management practice and emerging standards for environmental site design.

Bias and variance of planning level estimates of pollutant loads

Planning level techniques typically use the product of runoff volume and a characteristic concentration to estimate mean annual contaminant loads when monitoring data are inadequate or unavailable. In contrast to the extensive literature on sampling properties, bias, and precision of loads estimated from monitoring data, the unconstrained and often inconsistent alternatives for choosing “representative” runoff volumes and concentrations for use in planning level estimates limit the opportunities of generalizing analytical results on the properties of these estimators. The ease with which these simple load estimates can be calculated belies their inherent uncertainty, motivating this examination of their bias and variability. The mean and variance of planning level load estimators are derived both under mild parametric assumptions and using a distribution free approximation. Common use of the mean, median, or geometric mean of event concentrations is shown to result, in general, in biased estimates of the mean annual load. Sensitivity analysis of the mean and variance demonstrates the need to incorporate the relative variance as well as the correlation of cumulative discharge and characteristic concentration in planning level load estimates. While analogous to load estimation from monitoring data, the results presented here are distinct and unrelated to retransformation or sampling biases that have been well documented in the river load literature. Substantive implications for regional assessments, planning, and watershed management are illustrated with a simple example drawn from Chesapeake Bay.

Automating Recession Curve Displacement Recharge Estimation.

Recharge estimation is an important and challenging element of groundwater management and resource sustainability. Many recharge estimation methods have been developed with varying data requirements, applicable to different spatial and temporal scales. The variability and inherent uncertainty in recharge estimation motivates the recommended use of multiple methods to estimate and bound regional recharge estimates. Despite the inherent limitations of using daily gauged streamflow, recession curve displacement methods provide a convenient first-order estimate as part of a multimethod hierarchical approach to estimate watershed-scale annual recharge. The implementation of recession curve displacement recharge estimation in the United States Geologic Survey (USGS) RORA program relies on the subjective, operator-specific selection of baseflow recession events to estimate a gauge-specific recession index. This paper presents a parametric algorithm that objectively automates this tedious, subjective process, parameterizing and automating the implementation of recession curve displacement. Results using the algorithm reproduce regional estimates of groundwater recharge from the USGS Appalachian Valley and Piedmont Regional Aquifer-System Analysis, with an average absolute error of less than 2%. The algorithm facilitates consistent, completely automated estimation of annual recharge that complements more rigorous data-intensive techniques for recharge estimation.