Stephen E. Cohn

Assessing the Effects of Data Selection with the DAO Physical-Space Statistical Analysis System*

Conventional optimal interpolation (OI) analysis systems solve the standard statistical analysis equations approximately, by invoking a local approximation and a data selection procedure. Although solution of the analysis equations is essentially exact in the recent generation of global spectral variational analysis systems, these new systems also include substantial changes in error covariance modeling, making it difficult to discern whether improvements in analysis and forecast quality are due to exact, global solution of the analysis equations, or to changes in error covariance modeling. The formulation and implementation of a new type of global analysis system at the Data Assimilation Office, termed the Physical-space Statistical Analysis System (PSAS), is described in this article. Since this system operates directly in physical space, it is capable of employing error covariance models identical to those of the predecessor OI system, as well as more advanced models. To focus strictly on the effect of global versus local solution of the analysis equations, a comparison between PSAS and OI analyses is carried out with both systems using identical error covariance models and identical data. Spectral decomposition of the analysis increments reveals that, relative to the PSAS increments, the OI increments have too little power at large horizontal scales and excessive power at small horizontal scales. The OI increments also display an unrealistically large ratio of divergence to vorticity. Dynamical imbalances in the OI-analyzed state can therefore be attributed in part to the approximate local method of solution, and are not entirely due to the simple geostrophic constraint built into the forecast error covariance model. Root-mean-square observation minus 6-h forecast errors in the zonal wind component are substantially smaller for the PSAS system than for the OI system.

Assimilation of Stratospheric Chemical Tracer Observations Using a Kalman Filter. Part I: Formulation

Abstract The first part of this two-part article describes the formulation of a Kalman filter system for assimilating limb-sounding observations of stratospheric chemical constituents into a tracer transport model. The system is based on a two-dimensional isentropic approximation, permitting a full Kalman filter implementation and a thorough study of its behavior in a real-data environment. Datasets from two instruments on the Upper Atmosphere Research Satellite with very different viewing geometries are used in the assimilation experiments. A robust chi-squared diagnostic, which compares statistics of the observed-minus-forecast residuals with those calculated by the filter algorithm, is used to help formulate the statistical inputs to the filter, as well as to tune covariance parameters and to validate the assimilation results. Two significant departures from the standard (discrete) Kalman filter formulation were found to be important in this study. First, it was discovered that the standard Kalman filt...

A fixed-lag Kalman smoother for retrospective data assimilation

Abstract Data assimilation has traditionally been employed to provide initial conditions for numerical weather prediction (NWP). A multiyear time sequence of objective analyses produced by data assimilation can also be used as an archival record from which to carry out a variety of atmospheric process studies. For this latter propose, NWP analyses are not as accurate as they could be, for each analysis is based only on current and past observed data, and not on any future data. Analyses incorporating future data, as well as current and past data, are termed retrospective analyses. The problem of retrospective objective analysis has not yet received attention in the meteorological literature. In this paper, the fixed-lag Kalman smoother (FLKS) is proposed as a means of providing retrospective analysis capability in data assimilation. The FLKS is a direct generalization of the Kalman filter. It incorporates all data observed up to and including some fixed amount of time past each analysis time. A computatio...

Suboptimal Schemes for Atmospheric Data Assimilation Based on the Kalman Filter

Abstract This work is directed toward approximating the evolution of forecast error covariances for data assimilation. The performance of different algorithms based on simplification of the standard Kalman filter (KF) is studied. These are suboptimal schemes (SOSs) when compared to the KF, which is optimal for linear problems with known statistics. The SOSs considered here are several versions of optimal interpolation (OI), a scheme for height error variance advection, and a simplified KF in which the full height error covariance is advected. To employ a methodology for exact comparison among these schemes, a linear environment is maintained, in which a beta-plane shallow-water model linearized about a constant zonal flow is chosen for the test-bed dynamics. The results show that constructing dynamically balanced forecast error covariances rather than using conventional geostrophically balanced ones is essential for successful performance of any SOS. A posteriori initialization of SOSs to compensate for m...

Dynamics of short-term univariate forecast error covariances

Abstract The covariance equation based on second-order closure for dynamics governed by a general scalar nonlinear partial differential equation (PDE) is studied. If the governing dynamics involve n space dimensions, then the covariance equation is a PDE in 2n space dimensions. Solving this equation for n = 3 is therefore computationally infeasible. This is a hindrance to stochastic-dynamic prediction as well as to novel methods of data assimilation based on the Kalman filter. It is shown that the covariance equation can be solved approximately, to any desired accuracy, by solving instead an auxiliary system of PDEs in just n dimensions. The first of these is a dynamical equation for the variance field. Successive equations describe, to increasingly high order, the dynamics of the shape of either the covariance function or the correlation function for points separated by small distances. The second-order equation, for instance, describes the evolution of the correlation length (turbulent microscale) field...

Treatment of Observation Error due to Unresolved Scales in Atmospheric Data Assimilation

Abstract Observations of the atmospheric state include scales of motion that are not resolved by numerical models into which the observed data are assimilated. The resulting observation error due to unresolved scales, part of the “representativeness error,” is state dependent and correlated in time. A mathematical formalism and algorithmic approach has been developed for treating this error in the data assimilation process, under an assumption that there is no model error. The approach is based on approximating the continuum Kalman filter in such a way as to maintain terms that account for the observation error due to unresolved scales. The two resulting approximate filters resemble the Schmidt–Kalman filter and the traditional discrete Kalman filter. The approach is tested for the model problem of a passive tracer undergoing advection in a shear flow on the sphere. The state contains infinitely many spherical harmonics, with a nonstationary spectrum, and the problem is to estimate the projection of this ...

Parallel Implementation of a Kalman Filter for Constituent Data Assimilation

Abstract A Kalman filter for the assimilation of long-lived atmospheric chemical constituents was developed for two-dimensional transport models on isentropic surfaces over the globe. Since the Kalman filter calculates the error covariances of the estimated constituent field, there are five dimensions to this problem, x1, x2, and time, where x1 and x2 are the positions of two points on an isentropic surface. Only computers with large memory capacity and high floating point speed can handle problems of this magnitude. This article describes an implementation of the Kalman filter for distributed-memory, message-passing parallel computers. To evolve the forecast error covariance matrix, an operator decomposition and a covariance decomposition were studied. The latter was found to be scalable and has the general property, of considerable practical advantage, that the dynamical model does not need to be parallelized. Tests of the Kalman filter code examined variance transport and observability properties. This...

Suboptimal Schemes for Retrospective Data Assimilation Based on the Fixed-Lag Kalman Smoother

Abstract The fixed-lag Kalman smoother was proposed recently by S. E. Cohn et al. as a framework for providing retrospective data assimilation capability in atmospheric reanalysis projects. Retrospective data assimilation refers to the dynamically consistent incorporation of data observed well past each analysis time into each analysis. Like the Kalman filter, the fixed-lag Kalman smoother requires statistical information that is not available in practice and involves an excessive amount of computation if implemented by brute force, and must therefore be approximated sensibly to become feasible for operational use. In this article the performance of suboptimal retrospective data assimilation systems (RDASs) based on a variety of approximations to the optimal fixed-lag Kalman smoother is evaluated. Since the fixed-lag Kalman smoother formulation employed in this work separates naturally into a (Kalman) filter portion and an optimal retrospective analysis portion, two suboptimal strategies are considered: (...

Conservation of Mass and Preservation of Positivity with Ensemble-Type Kalman Filter Algorithms

AbstractThis paper considers the incorporation of constraints to enforce physically based conservation laws in the ensemble Kalman filter. In particular, constraints are used to ensure that the ensemble members and the ensemble mean conserve mass and remain nonnegative through measurement updates. In certain situations filtering algorithms such as the ensemble Kalman filter (EnKF) and ensemble transform Kalman filter (ETKF) yield updated ensembles that conserve mass but are negative, even though the actual states must be nonnegative. In such situations if negative values are set to zero, or a log transform is introduced, the total mass will not be conserved. In this study, mass and positivity are both preserved by formulating the filter update as a set of quadratic programming problems that incorporate nonnegativity constraints. Simple numerical experiments indicate that this approach can have a significant positive impact on the posterior ensemble distribution, giving results that are more physically pla...

A Fully Implicit Scheme for the Barotropic Primitive Equations

Abstract An efficient implicit finite-difference method is developed and tested for a global barotropic model. The scheme requires at each time step the solution of only one-dimensional block-tridiagonal linear systems. This additional computation is offset by the use of a time step chosen independently of the mesh spacing. The method is second-order accurate in time and fourth-order accurate in space. Our experience indicates that this implicit method is practical for numerical simulation on fine meshes.