Craig H. Bishop

Adaptive Sampling with the Ensemble Transform Kalman Filter. Part I: Theoretical Aspects

A suboptimal Kalman filter called the ensemble transform Kalman filter (ET KF) is introduced. Like other Kalman filters, it provides a framework for assimilating observations and also for estimating the effect of observations on forecast error covariance. It differs from other ensemble Kalman filters in that it uses ensemble transformation and a normalization to rapidly obtain the prediction error covariance matrix associated with a particular deployment of observational resources. This rapidity enables it to quickly assess the ability of a large number of future feasible sequences of observational networks to reduce forecast error variance. The ET KF was used by the National Centers for Environmental Prediction in the Winter Storm Reconnaissance missions of 1999 and 2000 to determine where aircraft should deploy dropwindsondes in order to improve 24‐72-h forecasts over the continental United States. The ET KF may be applied to any well-constructed set of ensemble perturbations. The ET KF technique supercedes the ensemble transform (ET) targeting technique of Bishop and Toth. In the ET targeting formulation, the means by which observations reduced forecast error variance was not expressed mathematically. The mathematical representation of this process provided by the ET KF enables such things as the evaluation of the reduction in forecast error variance associated with individual flight tracks and assessments of the value of targeted observations that are distributed over significant time intervals. It also enables a serial targeting methodology whereby one can identify optimal observing sites given the location and error statistics of other observations. This allows the network designer to nonredundantly position targeted observations. Serial targeting can also be used to greatly reduce the computations required to identify optimal target sites. For these theoretical and practical reasons, the ET KF technique is more useful than the ET technique. The methodology is illustrated with observation system simulation experiments involving a barotropic numerical model of tropical cyclonelike vortices. These include preliminary empirical tests of ET KF predictions using ET KF, 3DVAR, and hybrid data assimilation schemes—the results of which look promising. To concisely describe the future feasible sequences of observations considered in adaptive sampling problems, an extension to Ide et al.’s unified notation for data assimilation is suggested.

Ensemble Square Root Filters

Abstract Ensemble data assimilation methods assimilate observations using state-space estimation methods and low-rank representations of forecast and analysis error covariances. A key element of such methods is the transformation of the forecast ensemble into an analysis ensemble with appropriate statistics. This transformation may be performed stochastically by treating observations as random variables, or deterministically by requiring that the updated analysis perturbations satisfy the Kalman filter analysis error covariance equation. Deterministic analysis ensemble updates are implementations of Kalman square root filters. The nonuniqueness of the deterministic transformation used in square root Kalman filters provides a framework to compare three recently proposed ensemble data assimilation methods.

A Comparison of Breeding and Ensemble Transform Kalman Filter Ensemble Forecast Schemes

Abstract The ensemble transform Kalman filter (ETKF) ensemble forecast scheme is introduced and compared with both a simple and a masked breeding scheme. Instead of directly multiplying each forecast perturbation with a constant or regional rescaling factor as in the simple form of breeding and the masked breeding schemes, the ETKF transforms forecast perturbations into analysis perturbations by multiplying by a transformation matrix. This matrix is chosen to ensure that the ensemble-based analysis error covariance matrix would be equal to the true analysis error covariance if the covariance matrix of the raw forecast perturbations were equal to the true forecast error covariance matrix and the data assimilation scheme were optimal. For small ensembles (∼100), the computational expense of the ETKF ensemble generation is only slightly greater than that of the masked breeding scheme. Version 3 of the Community Climate Model (CCM3) developed at National Center for Atmospheric Research (NCAR) is used to test ...

The THORPEX Interactive Grand Global Ensemble

Ensemble forecasting is increasingly accepted as a powerful tool to improve early warnings for high-impact weather. Recently, ensembles combining forecasts from different systems have attracted a considerable level of interest. The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Globa l Ensemble (TIGGE) project, a prominent contribution to THORPEX, has been initiated to enable advanced research and demonstration of the multimodel ensemble concept and to pave the way toward operational implementation of such a system at the international level. The objectives of TIGGE are 1) to facilitate closer cooperation between the academic and operational meteorological communities by expanding the availability of operational products for research, and 2) to facilitate exploring the concept and benefits of multimodel probabilistic weather forecasts, with a particular focus on high-impact weather prediction. Ten operational weather forecasting centers producing daily global ensemble ...

Cloud-Resolving Hurricane Initialization and Prediction through Assimilation of Doppler Radar Observations with an Ensemble Kalman Filter

Abstract This study explores the assimilation of Doppler radar radial velocity observations for cloud-resolving hurricane analysis, initialization, and prediction with an ensemble Kalman filter (EnKF). The case studied is Hurricane Humberto (2007), the first landfalling hurricane in the United States since the end of the 2005 hurricane season and the most rapidly intensifying near-landfall storm in U.S. history. The storm caused extensive damage along the southeast Texas coast but was poorly predicted by operational models and forecasters. It is found that the EnKF analysis, after assimilating radial velocity observations from three Weather Surveillance Radars-1988 Doppler (WSR-88Ds) along the Gulf coast, closely represents the best-track position and intensity of Humberto. Deterministic forecasts initialized from the EnKF analysis, despite displaying considerable variability with different lead times, are also capable of predicting the rapid formation and intensification of the hurricane. These forecasts...

Ensemble Transformation and Adaptive Observations

Abstract Suppose that the geographical and temporal resolution of the observational network could be changed on a daily basis. Of all the possible deployments of observational resources, which particular deployment would minimize expected forecast error? The ensemble transform technique answers such questions by using nonlinear ensemble forecasts to rapidly construct ensemble-based approximations to the prediction error covariance matrices associated with a wide range of different possible deployments of observational resources. From these matrices, estimates of the expected forecast error associated with each distinct deployment of observational resources are obtained. The deployment that minimizes the chosen measure of forecast error is deemed optimal. The technique may also be used to find the perturbation that evolves into the leading eigenvector or singular vector of an ensemble-based prediction error covariance matrix. This time-evolving perturbation “explains” more of the ensemble-based prediction ...

Which Is Better, an Ensemble of Positive–Negative Pairs or a Centered Spherical Simplex Ensemble?

Abstract New methods to center the initial ensemble perturbations on the analysis are introduced and compared with the commonly used centering method of positive–negative paired perturbations. In the new method, one linearly dependent perturbation is added to a set of linearly independent initial perturbations to ensure that the sum of the new initial perturbations equals zero; the covariance calculated from the new initial perturbations is equal to the analysis error covariance estimated by the independent initial perturbations, and all of the new initial perturbations are equally likely. The new method is illustrated by applying it to the ensemble transform Kalman filter (ETKF) ensemble forecast scheme, and the resulting ensemble is called the spherical simplex ETKF ensemble. It is shown from a multidimensional Taylor expansion that the symmetric positive–negative paired centering would yield a more accurate forecast ensemble mean and covariance than the spherical simplex centering if the ensemble were ...

The North Pacific Experiment (NORPEX-98): Targeted Observations for Improved North American Weather Forecasts

Abstract The objectives and preliminary results of an interagency field program, the North Pacific Experiment (NORPEX), which took place between 14 January and 27 February 1998, are described. NORPEX represents an effort to directly address the issue of observational sparsity over the North Pacific basin, which is a major contributing factor in short-range (less than 4 days) forecast failures for land-falling Pacific winter-season storms that affect the United States, Canada, and Mexico. The special observations collected in NORPEX include approximately 700 targeted tropospheric soundings of temperature, wind, and moisture from Global Positioning System (GPS) dropsondes obtained in 38 storm reconnaissance missions using aircraft based primarily in Hawaii and Alaska. In addition, wind data were provided every 6 h over the entire North Pacific during NORPEX, using advanced and experimental techniques to extract information from multispectral geostationary satellite imagery. Preliminary results of NORPEX dat...

The Effect of Targeted Dropsonde Observations during the 1999 Winter Storm Reconnaissance Program

Abstract In this paper, the effects of targeted dropsonde observations on operational global numerical weather analyses and forecasts made at the National Centers for Environmental Prediction (NCEP) are evaluated. The data were collected during the 1999 Winter Storm Reconnaissance field program at locations that were found optimal by the ensemble transform technique for reducing specific forecast errors over the continental United States and Alaska. Two parallel analysis–forecast cycles are compared; one assimilates all operationally available data including those from the targeted dropsondes, whereas the other is identical except that it excludes all dropsonde data collected during the program. It was found that large analysis errors appear in areas of intense baroclinic energy conversion over the northeast Pacific and are strongly associated with errors in the first-guess field. The “signal,” defined by the difference between analysis–forecast cycles with and without the dropsonde data, propagates at an...

A Comparison of Adaptive Observing Guidance for Atlantic Tropical Cyclones

Abstract Airborne adaptive observations have been collected for more than two decades in the neighborhood of tropical cyclones, to attempt to improve short-range forecasts of cyclone track. However, only simple subjective strategies for adaptive observations have been used, and the utility of objective strategies to improve tropical cyclone forecasts remains unexplored. Two objective techniques that have been used extensively for midlatitude adaptive observing programs, and the current strategy based on the ensemble deep-layer mean (DLM) wind variance, are compared quantitatively using two metrics. The ensemble transform Kalman filter (ETKF) uses ensembles from NCEP and the ECMWF. Total-energy singular vectors (TESVs) are computed by the ECMWF and the Naval Research Laboratory, using their respective global models. Comparisons of 78 guidance products for 2-day forecasts during the 2004 Atlantic hurricane season are made, on both continental and localized scales relevant to synoptic surveillance missions. ...