Yuejian Zhu

A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems

Abstract The present paper summarizes the methodologies used at the European Centre for Medium-Range Weather Forecasts (ECMWF), the Meteorological Service of Canada (MSC), and the National Centers for Environmental Prediction (NCEP) to simulate the effect of initial and model uncertainties in ensemble forecasting. The characteristics of the three systems are compared for a 3-month period between May and July 2002. The main conclusions of the study are the following: the performance of ensemble prediction systems strongly depends on the quality of the data assimilation system used to create the unperturbed (best) initial condition and the numerical model used to generate the forecasts; a successful ensemble prediction system should simulate the effect of both initial and model-related uncertainties on forecast errors; and for all three global systems, the spread of ensemble forecasts is insufficient to systematically capture reality, suggesting that none of them is able to simulate all sources of forecast ...

THE ECONOMIC VALUE OF ENSEMBLE-BASED WEATHER FORECASTS

Abstract The potential economic benefit associated with the use of an ensemble of forecasts versus anequivalent or higher-resolution control forecast is discussed. Neither forecast systems are post-processed,except a simple calibration that is applied to make them reliable. A simple decision-making model is used where all potential users of weather forecasts are characterized by the ratio between the cost of their action to preventweather-related damages, and the loss that they incur in case they do not protect their operations. It isshown that the ensemble forecast system can be used by a much wider range of users. Furthermore,for many, and for beyond 4-day lead time for all users, the ensemble provides greater potential economicbenefit than a control forecast, even if the latter is run at higher horizontal resolution. It is argued that theadded benefits derive from 1) the fact that the ensemble provides a more detailed forecast probabilitydistribution, allowing the users to tailor their weather forecast...

Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system

Since modern data assimilation (DA) involves the repetitive use of dynamical forecasts, errors in analyses share characteristics of those in short-range forecasts. Initial conditions for an ensemble prediction/forecast system (EPS or EFS) are expected to sample uncertainty in the analysis field. Ensemble forecasts with such initial conditions can therefore (a) be fed back to DA to reduce analysis uncertainty, as well as (b) sample forecast uncertainty related to initial conditions. Optimum performance of both DA and EFS requires a careful choice of initial ensemble perturbations. DA can be improved with an EFS that represents the dynamically conditioned part of forecast error covariance as accurately as possible, while an EFS can be improved by initial perturbations reflecting analysis error variance. Initial perturbation generation schemes that dynamically cycle ensemble perturbations reminiscent to how forecast errors are cycled in DA schemes may offer consistency between DA and EFS, and good performance for both. In this paper, we introduce an EFS based on the initial perturbations that are generated by the Ensemble Transform (ET) and ET with rescaling (ETR) methods to achieve this goal. Both ET and ETR are generalizations of the breeding method (BM). The results from ensemble systems based on BM, ET, ETR and the Ensemble Transform Kalman Filter (ETKF) method are experimentally compared in the context of ensemble forecast performance. Initial perturbations are centred around a 3D-VAR analysis, with a variance equal to that of estimated analysis errors. Of the four methods, the ETR method performed best in most probabilistic scores and in terms of the forecast error explained by the perturbations. All methods display very high time consistency between the analysis and forecast perturbations. It is expected that DA performance can be improved by the use of forecast error covariance from a dynamically cycled ensemble either with a variational DA approach (coupled with an ETR generation scheme), or with an ETKF-type DA scheme.

Ensemble Transform Kalman Filter-based ensemble perturbations in an operational global prediction system at NCEP

The initial perturbations used for the operational global ensemble prediction system of the National Centers for Environmental Prediction are generated through the breeding method with a regional rescaling mechanism. Limitations of the system include the use of a climatologically fixed estimate of the analysis error variance and the lack of an orthogonalization in the breeding procedure. The Ensemble Transform Kalman Filter (ETKF) method is a natural extension of the concept of breeding and, as shown byWang and Bishop, can be used to generate ensemble perturbations that can potentially ameliorate these shortcomings. In the present paper, a spherical simplex 10-member ETKF ensemble, using the actual distribution and error characteristics of real-time observations and an innovation-based inflation, is tested and compared with a 5-pair breeding ensemble in an operational environment. The experimental results indicate only minor differences between the performances of the operational breeding and the experimental ETKF ensemble and only minor differences to Wang and Bishop’s earlier comparison studies. As for the ETKF method, the initial perturbation variance is found to respond to temporal changes in the observational network in the North Pacific. In other regions, however, 10 ETKF perturbations do not appear to be enough to distinguish spatial variations in observational network density. As expected, the whitening effect of the ETKF together with the use of the simplex algorithm that centres a set of quasi-orthogonal perturbations around the best analysis field leads to a significantly higher number of degrees of freedom as compared to the use of paired initial perturbations in operations. As a new result, the perturbations generated through the simplex method are also shown to exhibit a very high degree of consistency between initial analysis and short-range forecast perturbations, a feature that can be important in practical applications. Potential additional benefits of the ETKF and Ensemble Transform methods when using more ensemble members and a more appropriate inflation scheme will be explored in follow-up studies.

The Use of Ensembles to Identify Forecasts with Small and Large Uncertainty

Abstract In the past decade ensemble forecasting has developed into an integral part of numerical weather prediction. Flow-dependent forecast probability distributions can be readily generated from an ensemble, allowing for the identification of forecast cases with high and low uncertainty. The ability of the NCEP ensemble to distinguish between high and low uncertainty forecast cases is studied here quantitatively. Ensemble mode forecasts, along with traditional higher-resolution control forecasts, are verified in terms of predicting the probability of the true state being in 1 of 10 climatologically equally likely 500-hPa height intervals. A stratification of the forecast cases by the degree of overall agreement among the ensemble members reveals great differences in forecast performance between the cases identified by the ensemble as the least and most uncertain. A new ensemble-based forecast product, the “relative measure of predictability,” is introduced to identify forecasts with below and above ave...

Bias Correction for Global Ensemble Forecast

The main task of this study is to introduce a statistical postprocessing algorithm to reduce the bias in the National Centers for Environmental Prediction (NCEP) and Meteorological Service of Canada (MSC) ensemble forecasts before they are merged to form a joint ensemble within the North American Ensemble Forecast System (NAEFS). This statistical postprocessing method applies a Kalman filter type algorithm to accumulate the decaying averaging bias and produces bias-corrected ensembles for 35 variables. NCEP implemented this bias-correction technique in 2006. NAEFS is a joint operational multimodel ensemble forecast system that combines NCEP and MSC ensemble forecasts after bias correction. According to operational statistical verification, both the NCEP and MSC bias-corrected ensemble forecast products are enhanced significantly. In addition to the operational calibration technique, three other experiments were designed to assess and mitigate ensemble biases on the model grid: a decaying averaging bias calibration method with short samples, a climate mean bias calibration method, and a bias calibration method using dependent data. Preliminary results show that the decaying averaging method works well for the first few days. After removing the decaying averaging bias, the calibrated NCEP operational ensemble has improved probabilistic performance for all measures until day 5. The reforecast ensembles from the Earth System Research Laboratory’s Physical Sciences Division with and without the climate mean bias correction were also examined. A comparison between the operational and the bias-corrected reforecast ensembles shows that the climate mean bias correction can add value, especially for week-2 probability forecasts.

Climatology-Calibrated Precipitation Analysis at Fine Scales: Statistical Adjustment of Stage IV toward CPC Gauge-Based Analysis

AbstractTwo widely used precipitation analyses are the Climate Prediction Center (CPC) unified global daily gauge analysis and Stage IV analysis based on quantitative precipitation estimate with multisensor observations. The former is based on gauge records with a uniform quality control across the entire domain and thus bears more confidence, but provides only 24-h accumulation at ⅛° resolution. The Stage IV dataset, on the other hand, has higher spatial and temporal resolution, but is subject to different methods of quality control and adjustments by different River Forecasting Centers. This article describes a methodology used to generate a new dataset by adjusting the Stage IV 6-h accumulations based on available joint samples of the two analyses to take advantage of both datasets. A simple linear regression model is applied to the archived historical Stage IV and the CPC datasets after the former is aggregated to the CPC grid and daily accumulation. The aggregated Stage IV analysis is then adjusted b...

An effective configuration of ensemble size and horizontal resolution for the NCEP GEFS

Two important questions are addressed in this paper using the Global Ensemble Forecast System (GEFS) from the National Centers for Environmental Prediction (NCEP): (1) How many ensemble members are needed to better represent forecast uncertainties with limited computational resources? (2) What is the relative impact on forecast skill of increasing model resolution and ensemble size? Two-month experiments at T126L28 resolution were used to test the impact of varying the ensemble size from 5 to 80 members at the 500-hPa geopotential height. Results indicate that increasing the ensemble size leads to significant improvements in the performance for all forecast ranges when measured by probabilistic metrics, but these improvements are not significant beyond 20 members for long forecast ranges when measured by deterministic metrics. An ensemble of 20 to 30 members is the most effective configuration of ensemble sizes by quantifying the tradeoff between ensemble performance and the cost of computational resources. Two representative configurations of the GEFS—the T126L28 model with 70 members and the T190L28 model with 20 members, which have equivalent computing costs—were compared. Results confirm that, for the NCEP GEFS, increasing the model resolution is more (less) beneficial than increasing the ensemble size for a short (long) forecast range.

Tracking and Verification of East Atlantic Tropical Cyclone Genesis in the NCEP Global Ensemble: Case Studies during the NASA African Monsoon Multidisciplinary Analyses

This study evaluates the performance of the NCEP global ensemble forecast system in predicting the genesis and evolution of five named tropical cyclones and two unnamed nondeveloping tropical systems during the NASA African Monsoon Multidisciplinary Analyses (NAMMA) between August and September 2006. The overall probabilities of the ensemble forecasts of tropical cyclone genesis are verified relative to a genesis time defined to be the first designation of the tropical depression from the National Hurricane Center (NHC). Additional comparisons are also made with high-resolution deterministic forecasts from the NCEP Global Forecast System (GFS). It is found that the ensemble forecasts have high probabilities of genesis for the three strong storms that formed from African easterly waves, but failed to accurately predict the pregenesis phase of two weaker storms that formed farther west in the Atlantic Ocean. The overall accuracy for the genesis forecasts is above 50% for the ensemble forecasts initialized in the pregenesis phase. The forecast uncertainty decreases with the reduction of the forecast lead time. The probability of tropical cyclone genesis reaches nearly 90% and 100% for the ensemble forecasts initialized near and in the postgenesis phase, respectively. Significant improvements in the track forecasts are found in the ensemble forecasts initialized in the postgenesis phase, possibly because of the implementation of the NCEP storm relocation scheme, which provides an accurate initial storm position for all ensemble members. Even with coarser resolution (T126L28 for the ensemble versus T384L64 for the GFS), the overall performance of the ensemble in predicting tropical cyclone genesis is compatible with the high-resolution deterministic GFS. In addition, false alarm rates for nondeveloping waves were low in both the GFS and ensemble forecasts.

Analysis differences and error variance estimates from multi-centre analysis data

NWP forecast performance has made great progress during the past decade due to a few important factors. First, the numerical forecast models at major numerical weather prediction (NWP) centres have improved tremendously due to more accurate physics parametrisation schemes and increased computing power, which permits the use of higher resolution forecast models. Second, more observations, more accurate observing systems and improved data assimilation (DA) methods have been developed, such as 4D-Var (Rabier et al. 2000) and ensemble Kalman filters (Whitaker and Hamill 2002; Tippett et al. 2003; Whitaker et al. 2007). More accurate DA systems have played a key role in providing more accurate initial conditions for the NWP models, which have improved weather forecasts, particularly over the short and medium ranges. Analysis differences and error variance estimates from multi-centre analysis data