Marion Mittermaier

Intercomparison of Spatial Forecast Verification Methods: Identifying Skillful Spatial Scales Using the Fractions Skill Score

Abstract The fractions skill score (FSS) was one of the measures that formed part of the Intercomparison of Spatial Forecast Verification Methods project. The FSS was used to assess a common dataset that consisted of real and perturbed Weather Research and Forecasting (WRF) model precipitation forecasts, as well as geometric cases. These datasets are all based on the NCEP 240 grid, which translates to approximately 4-km resolution over the contiguous United States. The geometric cases showed that the FSS can provide a truthful assessment of displacement errors and forecast skill. In addition, the FSS can be used to determine the scale at which an acceptable level of skill is reached and this usage is perhaps more helpful than interpreting the actual FSS value. This spatial-scale approach is becoming more popular for monitoring operational forecast performance. The study also shows how the FSS responds to forecast bias. A more biased forecast always gives lower FSS values at large scales and usually at sma...

A Strategy for Verifying Near-Convection-Resolving Model Forecasts at Observing Sites

AbstractRoutine verification of deterministic numerical weather prediction (NWP) forecasts from the convection-permitting 4-km (UK4) and near-convection-resolving 1.5-km (UKV) configurations of the Met Office Unified Model (MetUM) has shown that it is hard to consistently demonstrate an improvement in skill from the higher-resolution model, even though subjective comparison suggests that it performs better. In this paper the use of conventional metrics and precise matching (through extracting the nearest grid point to an observing site) of the forecast to conventional synoptic observations in space and time is replaced with the use of inherently probabilistic metrics such as the Brier score, ranked probability, and continuous ranked probability scores applied to neighborhoods of forecast grid points. Three neighborhood sizes were used: ~4, ~12, and ~25 km, which match the sizes of the grid elements currently used operationally. Six surface variables were considered: 2-m temperature, 10-m wind speed, total...

The set-up of the Mesoscale Verification Inter-Comparison over Complex Terrain (MesoVICT) Project

CapsuleThe project focuses on spatial verification methods as applied to high resolution forecasts of deterministic and ensemble precipitation, wind and temperature forecasts over complex terrain, and includes observation uncertainty assessment.

Feature-Based Diagnostic Evaluation of Global NWP Forecasts

AbstractWith the resolution of global numerical weather prediction (NWP) models now typically between 10 and 20 km, forecasts are able to capture the evolution of synoptic features that are important drivers for significant surface weather. The position, timing, and intensity of jet cores, surface highs and lows, and changes in the behavior of these forecast features is explored using the Method for Object-based Diagnostic Evaluation (MODE) at the global scale. Previously this was only possible with a more subjective approach. The spatial aspects of the forecast features (objects) and their intensity can be assessed separately. The evolution of paired forecast–analysis object attributes such as location and orientation differences, as well as area ratios, can be considered. The differences in the paired object attribute distributions from various model configurations were evaluated using the k-sample Anderson–Darling (AD) test. Increases or decreases in hits, false alarms (forecast-not-observed), and miss...