Randy Bullock

Object-Based Verification of Precipitation Forecasts. Part I: Methodology and Application to Mesoscale Rain Areas

Abstract A recently developed method of defining rain areas for the purpose of verifying precipitation produced by numerical weather prediction models is described. Precipitation objects are defined in both forecasts and observations based on a convolution (smoothing) and thresholding procedure. In an application of the new verification approach, the forecasts produced by the Weather Research and Forecasting (WRF) model are evaluated on a 22-km grid covering the continental United States during July–August 2001. Observed rainfall is derived from the stage-IV product from NCEP on a 4-km grid (averaged to a 22-km grid). It is found that the WRF produces too many large rain areas, and the spatial and temporal distribution of the rain areas reveals regional underestimates of the diurnal cycle in rain-area occurrence frequency. Objects in the two datasets are then matched according to the separation distance of their centroids. Overall, WRF rain errors exhibit no large biases in location, but do suffer from a ...

Object-Based Verification of Precipitation Forecasts. Part II: Application to Convective Rain Systems

The authors develop and apply an algorithm to define coherent areas of precipitation, emphasizing mesoscale convection, and compare properties of these areas with observations obtained from NCEP stage-IV precipitation analyses (gauge and radar combined). In Part II, fully explicit 12–36-h forecasts of rainfall from the Weather Research and Forecasting model (WRF) are evaluated. These forecasts are integrated on a 4-km mesh without a cumulus parameterization. Rain areas are defined similarly to Part I, but emphasize more intense, smaller areas. Furthermore, a time-matching algorithm is devised to group spatially and temporally coherent areas into rain systems that approximate mesoscale convective systems. In general, the WRF model produces too many rain areas with length scales of 80 km or greater. Rain systems typically last too long, and are forecast to occur 1–2 h later than observed. The intensity distribution among rain systems in the 4-km forecasts is generally too broad, especially in the late afternoon, in sharp contrast to the intensity distribution obtained on a coarser grid with parameterized convection in Part I. The model exhibits the largest positive size and intensity bias associated with systems over the Midwest and Mississippi Valley regions, but little size bias over the High Plains, Ohio Valley, and the southeast United States. For rain systems lastin g6ho rmore, the critical success index for matching forecast and observed rain systems agrees closely with that obtained in a related study using manually determined rain systems.

The Method for Object-Based Diagnostic Evaluation (MODE) Applied to Numerical Forecasts from the 2005 NSSL/SPC Spring Program

Abstract The authors use a procedure called the method for object-based diagnostic evaluation, commonly referred to as MODE, to compare forecasts made from two models representing separate cores of the Weather Research and Forecasting (WRF) model during the 2005 National Severe Storms Laboratory and Storm Prediction Center Spring Program. Both models, the Advanced Research WRF (ARW) and the Nonhydrostatic Mesoscale Model (NMM), were run without a traditional cumulus parameterization scheme on horizontal grid lengths of 4 km (ARW) and 4.5 km (NMM). MODE was used to evaluate 1-h rainfall accumulation from 24-h forecasts valid at 0000 UTC on 32 days between 24 April and 4 June 2005. The primary variable used for evaluation was a “total interest” derived from a fuzzy-logic algorithm that compared several attributes of forecast and observed rain features such as separation distance and spatial orientation. The maximum value of the total interest obtained by comparing an object in one field with all objects in ...

Intercomparison of In-Flight Icing Algorithms. Part II: Statistical Verification Results

Abstract Recent research to improve forecasts of in-flight icing conditions has involved the development of algorithms to apply to the output of numerical weather prediction models. The abilities of several of these algorithms to predict icing conditions, as verified by pilot reports (PIREPs), are compared for two numerical weather prediction models (Eta and the Mesoscale Analysis and Prediction System) for the Winter Icing and Storms Program 1994 (WISP94) time period (25 January–25 March 1994). Algorithms included in the comparison were developed by the National Aviation Weather Advisory Unit [NAWAU, now the Aviation Weather Center (AWC)], the National Center for Atmospheric Research’s Research Applications Program (RAP), and the U.S. Air Force. Operational icing forecasts (AIRMETs) issued by NAWAU for the same time period are evaluated to provide a standard of comparison. The capabilities of the Eta Model’s explicit cloud liquid water estimates for identifying icing regions are also evaluated and compar...

Application of Object-Based Time-Domain Diagnostics for Tracking Precipitation Systems in Convection-Allowing Models

AbstractMeaningful verification and evaluation of convection-allowing models requires approaches that do not rely on point-to-point matches of forecast and observed fields. In this study, one such approach—a beta version of the Method for Object-Based Diagnostic Evaluation (MODE) that incorporates the time dimension [known as MODE time-domain (MODE-TD)]—was applied to 30-h precipitation forecasts from four 4-km grid-spacing members of the 2010 Storm-Scale Ensemble Forecast system with different microphysics parameterizations. Including time in MODE-TD provides information on rainfall system evolution like lifetime, timing of initiation and dissipation, and translation.The simulations depicted the spatial distribution of time-domain precipitation objects across the United States quite well. However, all simulations overpredicted the number of objects, with the Thompson microphysics scheme overpredicting the most and the Morrison method the least. For the smallest smoothing radius and rainfall threshold use...

Using Satellite Data to Reduce Spatial Extent of Diagnosed Icing

Overprediction of the spatial extent of aircraft icing is a major problem in forecaster products based on numerical model output. Dependence on relative humidity fields, which are inherently broad and smooth, is the cause of this difficulty. Using multispectral satellite analysis based on NOAA Advanced Very High Resolution Radiometer data, this paper shows how the spatial extent of icing potential based on model output can be reduced where there are no subfreezing cloud tops and, therefore, where icing is unlikely. Fifty-one cases were analyzed using two scenarios: 1) model output only and 2) model output screened by a satellite cloud analysis. Average area efficiency, a statistical validation measure of icing potential using coincident pilot reports of icing, improved substantially when satellite screening was applied.

Computationally Efficient Spatial Forecast Verification Using Baddeley's Delta Image Metric

Abstract An important focus of research in the forecast verification community is the development of alternative verification approaches for quantitative precipitation forecasts, as well as for other spatial forecasts. The need for information that is meaningful in an operational context and the importance of capturing the specific sources of forecast error at varying spatial scales are two primary motivating factors. In this paper, features of precipitation as identified by a convolution threshold technique are merged within fields and matched across fields in an automatic and computationally efficient manner using Baddeley’s metric for binary images. The method is carried out on 100 test cases, and 4 representative cases are shown in detail. Results of merging and matching objects are generally positive in that they are consistent with how a subjective observer might merge and match features. The results further suggest that the Baddeley metric may be useful as a computationally efficient summary metric...

Simulating North American mesoscale convective systems with a convection-permitting climate model

Deep convection is a key process in the climate system and the main source of precipitation in the tropics, subtropics, and mid-latitudes during summer. Furthermore, it is related to high impact weather causing floods, hail, tornadoes, landslides, and other hazards. State-of-the-art climate models have to parameterize deep convection due to their coarse grid spacing. These parameterizations are a major source of uncertainty and long-standing model biases. We present a North American scale convection-permitting climate simulation that is able to explicitly simulate deep convection due to its 4-km grid spacing. We apply a feature-tracking algorithm to detect hourly precipitation from Mesoscale Convective Systems (MCSs) in the model and compare it with radar-based precipitation estimates east of the US Continental Divide. The simulation is able to capture the main characteristics of the observed MCSs such as their size, precipitation rate, propagation speed, and lifetime within observational uncertainties. In particular, the model is able to produce realistically propagating MCSs, which was a long-standing challenge in climate modeling. However, the MCS frequency is significantly underestimated in the central US during late summer. We discuss the origin of this frequency biases and suggest strategies for model improvements.

Model-Evaluation Tools for Three-Dimensional Cloud Verification via Spaceborne Active Sensors

AbstractClouds pose many operational hazards to the aviation community in terms of ceilings and visibility, turbulence, and aircraft icing. Realistic descriptions of the three-dimensional (3D) distribution and temporal evolution of clouds in numerical weather prediction models used for flight planning and routing are therefore of central importance. The introduction of satellite-based cloud radar (CloudSat) and Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) sensors to the National Aeronautics and Space Administration A-Train is timely in light of these needs but requires a new paradigm of model-evaluation tools that are capable of exploiting the vertical-profile information. Early results from the National Center for Atmospheric Research Model Evaluation Toolkit (MET), augmented to work with the emergent satellite-based active sensor observations, are presented here. Existing horizontal-plane statistical evaluation techniques have been adapted to operate on observations in th...

Multiyear Droughts and Pluvials over the Upper Colorado River Basin and Associated Circulations

AbstractThis study analyzes spatial and temporal characteristics of multiyear droughts and pluvials over the southwestern United States with a focus on the upper Colorado River basin. The study uses two multiscalar moisture indices: standardized precipitation evapotranspiration index (SPEI) and standardized precipitation index (SPI) on a 36-month scale (SPEI36 and SPI36, respectively). The indices are calculated from monthly average precipitation and maximum and minimum temperatures from the Parameter-Elevation Regressions on Independent Slopes Model dataset for the period 1950–2012. The study examines the relationship between individual climate variables as well as large-scale atmospheric circulation features found in reanalysis output during drought and pluvial periods. The results indicate that SPEI36 and SPI36 show similar temporal and spatial patterns, but that the inclusion of temperatures in SPEI36 leads to more extreme magnitudes in SPEI36 than in SPI36. Analysis of large-scale atmospheric fields ...