Thomas M. Hamill

Distance-Dependent Filtering of Background Error Covariance Estimates in an Ensemble Kalman Filter

The usefulness of a distance-dependent reduction of background error covariance estimates in an ensemble Kalman filter is demonstrated. Covariances are reduced by performing an elementwise multiplication of the background error covariance matrix with a correlation function with local support. This reduces noisiness and results in an improved background error covariance estimate, which generates a reduced-error ensemble of model initial conditions. The benefits of applying the correlation function can be understood in part from examining the characteristics of simple 2 3 2 covariance matrices generated from random sample vectors with known variances and covariance. These show that noisiness in covariance estimates tends to overwhelm the signal when the ensemble size is small and/or the true covariance between the sample elements is small. Since the true covariance of forecast errors is generally related to the distance between grid points, covariance estimates generally have a higher ratio of noise to signal with increasing distance between grid points. This property is also demonstrated using a twolayer hemispheric primitive equation model and comparing covariance estimates generated by small and large ensembles. Covariances from the large ensemble are assumed to be accurate and are used a reference for measuring errors from covariances estimated from a small ensemble. The benefits of including distance-dependent reduction of covariance estimates are demonstrated with an ensemble Kalman filter data assimilation scheme. The optimal correlation length scale of the filter function depends on ensemble size; larger correlation lengths are preferable for larger ensembles. The effects of inflating background error covariance estimates are examined as a way of stabilizing the filter. It was found that more inflation was necessary for smaller ensembles than for larger ensembles.

Ensemble Square Root Filters

Abstract Ensemble data assimilation methods assimilate observations using state-space estimation methods and low-rank representations of forecast and analysis error covariances. A key element of such methods is the transformation of the forecast ensemble into an analysis ensemble with appropriate statistics. This transformation may be performed stochastically by treating observations as random variables, or deterministically by requiring that the updated analysis perturbations satisfy the Kalman filter analysis error covariance equation. Deterministic analysis ensemble updates are implementations of Kalman square root filters. The nonuniqueness of the deterministic transformation used in square root Kalman filters provides a framework to compare three recently proposed ensemble data assimilation methods.

Interpretation of Rank Histograms for Verifying Ensemble Forecasts

Abstract Rank histograms are a tool for evaluating ensemble forecasts. They are useful for determining the reliability of ensemble forecasts and for diagnosing errors in its mean and spread. Rank histograms are generated by repeatedly tallying the rank of the verification (usually an observation) relative to values from an ensemble sorted from lowest to highest. However, an uncritical use of the rank histogram can lead to misinterpretations of the qualities of that ensemble. For example, a flat rank histogram, usually taken as a sign of reliability, can still be generated from unreliable ensembles. Similarly, a U-shaped rank histogram, commonly understood as indicating a lack of variability in the ensemble, can also be a sign of conditional bias. It is also shown that flat rank histograms can be generated for some model variables if the variance of the ensemble is correctly specified, yet if covariances between model grid points are improperly specified, rank histograms for combinations of model variables...

Ensemble Data Assimilation with the NCEP Global Forecast System

Real-data experiments with an ensemble data assimilation system using the NCEP Global Forecast System model were performed and compared with the NCEP Global Data Assimilation System (GDAS). All observations in the operational data stream were assimilated for the period 1 January–10 February 2004, except satellite radiances. Because of computational resource limitations, the comparison was done at lower resolution (triangular truncation at wavenumber 62 with 28 levels) than the GDAS real-time NCEP operational runs (triangular truncation at wavenumber 254 with 64 levels). The ensemble data assimilation system outperformed the reduced-resolution version of the NCEP three-dimensional variational data assimilation system (3DVAR), with the biggest improvement in data-sparse regions. Ensemble data assimilation analyses yielded a 24-h improvement in forecast skill in the Southern Hemisphere extratropics relative to the NCEP 3DVAR system (the 48-h forecast from the ensemble data assimilation system was as accurate as the 24-h forecast from the 3DVAR system). Improvements in the data-rich Northern Hemisphere, while still statistically significant, were more modest. It remains to be seen whether the improvements seen in the Southern Hemisphere will be retained when satellite radiances are assimilated. Three different parameterizations of background errors unaccounted for in the data assimilation system (including model error) were tested. Adding scaled random differences between adjacent 6-hourly analyses from the NCEP–NCAR reanalysis to each ensemble member (additive inflation) performed slightly better than the other two methods (multiplicative inflation and relaxation-to-prior).

The THORPEX Interactive Grand Global Ensemble

Ensemble forecasting is increasingly accepted as a powerful tool to improve early warnings for high-impact weather. Recently, ensembles combining forecasts from different systems have attracted a considerable level of interest. The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Globa l Ensemble (TIGGE) project, a prominent contribution to THORPEX, has been initiated to enable advanced research and demonstration of the multimodel ensemble concept and to pave the way toward operational implementation of such a system at the international level. The objectives of TIGGE are 1) to facilitate closer cooperation between the academic and operational meteorological communities by expanding the availability of operational products for research, and 2) to facilitate exploring the concept and benefits of multimodel probabilistic weather forecasts, with a particular focus on high-impact weather prediction. Ten operational weather forecasting centers producing daily global ensemble ...

Reanalysis without Radiosondes Using Ensemble Data Assimilation

Abstract Studies using idealized ensemble data assimilation systems have shown that flow-dependent background-error covariances are most beneficial when the observing network is sparse. The computational cost of recently proposed ensemble data assimilation algorithms is directly proportional to the number of observations being assimilated. Therefore, ensemble-based data assimilation should both be more computationally feasible and provide the greatest benefit over current operational schemes in situations when observations are sparse. Reanalysis before the radiosonde era (pre-1931) is just such a situation. The feasibility of reanalysis before radiosondes using an ensemble square root filter (EnSRF) is examined. Real surface pressure observations for 2001 are used, subsampled to resemble the density of observations we estimate to be available for 1915. Analysis errors are defined relative to a three-dimensional variational data assimilation (3DVAR) analysis using several orders of magnitude more observati...

Ensemble Reforecasting: Improving Medium-Range Forecast Skill Using Retrospective Forecasts

Abstract The value of the model output statistics (MOS) approach to improving 6–10-day and week 2 probabilistic forecasts of surface temperature and precipitation is demonstrated. Retrospective 2-week ensemble “reforecasts” were computed using a version of the NCEP medium-range forecast model with physics operational during 1998. An NCEP–NCAR reanalysis initial condition and bred modes were used to initialize the 15-member ensemble. Probabilistic forecasts of precipitation and temperature were generated by a logistic regression technique with the ensemble mean (precipitation) or ensemble mean anomaly (temperature) as the only predictor. Forecasts were computed and evaluated during 23 winter seasons from 1979 to 2001. Evaluated over the 23 winters, these MOS-based probabilistic forecasts were skillful and highly reliable. When compared against operational NCEP forecasts for a subset of 100 days from the 2001–2002 winters, the MOS-based forecasts were comparatively much more skillful and reliable. For examp...

REFORECASTS An Important Dataset for Improving Weather Predictions

A “reforecast” (retrospective forecast) dataset has been developed. This dataset is comprised of a 15-member ensemble run out to a 2-week lead. Forecasts have been run every day from 0000 UTC initial conditions from 1979 to the present. The model is a 1998 version of the National Centers for Environmental Predictions (NCEPs) Global Forecast System (GFS) at T62 resolution. The 15 initial conditions consist of a reanalysis and seven pairs of bred modes. This dataset facilitates a number of applications that were heretofore impossible. Model errors can be diagnosed from the past forecasts and corrected, thereby dramatically increasing the forecast skill. For example, calibrated precipitation forecasts over the United States based on the 1998 reforecast model are more skillful than precipitation forecasts from the 2002 higher-resolution version of the NCEP GFS. Other applications are also demonstrated, such as the diagnosis of the bias for model development and an identification of the most predictable patt...

A Hybrid ETKF–3DVAR Data Assimilation Scheme for the WRF Model. Part I: Observing System Simulation Experiment

Abstract A hybrid ensemble transform Kalman filter–three-dimensional variational data assimilation (ETKF–3DVAR) system for the Weather Research and Forecasting (WRF) Model is introduced. The system is based on the existing WRF 3DVAR. Unlike WRF 3DVAR, which utilizes a simple, static covariance model to estimate the forecast-error statistics, the hybrid system combines ensemble covariances with the static covariances to estimate the complex, flow-dependent forecast-error statistics. Ensemble covariances are incorporated by using the extended control variable method during the variational minimization. The ensemble perturbations are maintained by the computationally efficient ETKF. As an initial attempt to test and understand the newly developed system, both an observing system simulation experiment under the perfect model assumption (Part I) and the real observation experiment (Part II) were conducted. In these pilot studies, the WRF was run over the North America domain at a coarse grid spacing (200 km) t...

NOAA's Second-Generation Global Medium-Range Ensemble Reforecast Dataset

A multidecadal ensemble reforecast database is now available that is approximately consistent with the operational 0000 UTC cycle of the 2012 NOAA Global Ensemble Forecast System (GEFS). The reforecast dataset consists of an 11-member ensemble run once each day from 0000 UTC initial conditions. Reforecasts are run to +16 days. As with the operational 2012 GEFS, the reforecast is run at T254L42 resolution (approximately 1/2° grid spacing, 42 levels) for week +1 forecasts and T190L42 (approximately 3/4° grid spacing) for the week +2 forecasts. Reforecasts were initialized with Climate Forecast System Reanalysis initial conditions, and perturbations were generated using the ensemble transform with rescaling technique. Reforecast data are available from 1985 to present. Reforecast datasets were previously demonstrated to be very valuable for detecting and correcting systematic errors in forecasts, especially forecasts of relatively rare events and longer-lead forecasts. What is novel about this reforecast dat...