Alberto Carrassi

Data assimilation as a nonlinear dynamical systems problem: stability and convergence of the prediction-assimilation system.

We study prediction-assimilation systems, which have become routine in meteorology and oceanography and are rapidly spreading to other areas of the geosciences and of continuum physics. The long-term, nonlinear stability of such a system leads to the uniqueness of its sequentially estimated solutions and is required for the convergence of these solutions to the systems true, chaotic evolution. The key ideas of our approach are illustrated for a linearized Lorenz system. Stability of two nonlinear prediction-assimilation systems from dynamic meteorology is studied next via the complete spectrum of their Lyapunov exponents; these two systems are governed by a large set of ordinary and of partial differential equations, respectively. The degree of data-induced stabilization is crucial for the performance of such a system. This degree, in turn, depends on two key ingredients: (i) the observational network, either fixed or data-adaptive, and (ii) the assimilation method.

Accounting for Model Error in Variational Data Assimilation: A Deterministic Formulation

In data assimilation, observations are combined with the dynamics to get an estimate of the actual state of a natural system. The knowledge of the dynamics, under the form of a model, is unavoidably incomplete and model error affects the prediction accuracy together with the error in the initial condition. The variational assimilation theory provides a framework to deal with model error along with the uncertainties coming from other sources entering the state estimation. Nevertheless, even if the problem is formulated as Gaussian, accounting for model error requires the estimation of its covariances and correlations, which are difficult to estimate in practice, in particular because of the large system dimension and the lack of enough observations. Model error has been therefore either neglected or assumed to be an uncorrelated noise. In the present work, an approach to account for a deterministic model error in the variational assimilation is presented. Equations for its correlations are first derived along with an approximation suitable for practical applications. Based on these considerations, a new four-dimensional variational data assimilation (4DVar) weak-constraint algorithm is formulated and tested in the context of a linear unstable system and of the three-component Lorenz model, which has chaotic dynamics. The results demonstrate that this approach is superior in skill to both the strong-constraint and a weak-constraint variational assimilation that employs the uncorrelated noise model error assumption.

Lyapunov vectors and assimilation in the unstable subspace: theory and applications

Based on a limited number of noisy observations, estimation algorithms provide a complete description of the state of a system at current time. Estimation algorithms that go under the name of assimilation in the unstable subspace (AUS) exploit the nonlinear stability properties of the forecasting model in their formulation. Errors that grow due to sensitivity to initial conditions are efficiently removed by confining the analysis solution in the unstable and neutral subspace of the system, the subspace spanned by Lyapunov vectors with positive and zero exponents, while the observational noise does not disturb the system along the stable directions. The formulation of the AUS approach in the context of four-dimensional variational assimilation (4DVar-AUS) and the extended Kalman filter (EKF-AUS) and its application to chaotic models is reviewed. In both instances, the AUS algorithms are at least as efficient but simpler to implement and computationally less demanding than their original counterparts. As predicted by the theory when error dynamics is linear, the optimal subspace dimension for 4DVar-AUS is given by the number of positive and null Lyapunov exponents, while the EKF-AUS algorithm, using the same unstable and neutral subspace, recovers the solution of the full EKF algorithm, but dealing with error covariance matrices of a much smaller dimension and significantly reducing the computational burden. Examples of the application to a simplified model of the atmospheric circulation and to the optimal velocity model for traffic dynamics are given.This article is part of a special issue of Journal of Physics A: Mathematical and Theoretical devoted to ‘Lyapunov analysis: from dynamical systems theory to applications’.

DADA: data assimilation for the detection and attribution of weather and climate-related events

We describe a new approach that allows for systematic causal attribution of weather and climate-related events, in near-real time. The method is designed so as to facilitate its implementation at meteorological centers by relying on data and methods that are routinely available when numerically forecasting the weather. We thus show that causal attribution can be obtained as a by-product of data assimilation procedures run on a daily basis to update numerical weather prediction (NWP) models with new atmospheric observations; hence, the proposed methodology can take advantage of the powerful computational and observational capacity of weather forecasting centers. We explain the theoretical rationale of this approach and sketch the most prominent features of a “data assimilation–based detection and attribution” (DADA) procedure. The proposal is illustrated in the context of the classical three-variable Lorenz model with additional forcing. The paper concludes by raising several theoretical and practical questions that need to be addressed to make the proposal operational within NWP centers.

Extending the Square Root Method to Account for Additive Forecast Noise in Ensemble Methods

AbstractA square root approach is considered for the problem of accounting for model noise in the forecast step of the ensemble Kalman filter (EnKF) and related algorithms. The primary aim is to replace the method of simulated, pseudo-random additive so as to eliminate the associated sampling errors. The core method is based on the analysis step of ensemble square root filters, and consists in the deterministic computation of a transform matrix. The theoretical advantages regarding dynamical consistency are surveyed, applying equally well to the square root method in the analysis step. A fundamental problem due to the limited size of the ensemble subspace is discussed, and novel solutions that complement the core method are suggested and studied. Benchmarks from twin experiments with simple, low-order dynamics indicate improved performance over standard approaches such as additive, simulated noise, and multiplicative inflation.

Four-dimensional ensemble variational data assimilation and the unstable subspace

Abstract The performance of (ensemble) Kalman filters used for data assimilation in the geosciences critically depends on the dynamical properties of the evolution model. A key aspect is that the error covariance matrix is asymptotically supported by the unstable–neutral subspace only, i.e. it is spanned by the backward Lyapunov vectors with non-negative exponents. The analytic proof of such a property for the Kalman filter error covariance has been recently given, and in particular that of its confinement to the unstable–neutral subspace. In this paper, we first generalize those results to the case of the Kalman smoother in a linear, Gaussian and perfect model scenario. We also provide square-root formulae for the filter and smoother that make the connection with ensemble formulations of the Kalman filter and smoother, where the span of the error covariance is described in terms of the ensemble deviations from the mean. We then discuss how this neat picture is modified when the dynamics are nonlinear and chaotic, and for which analytic results are precluded or difficult to obtain. A numerical investigation is carried out to study the approximate confinement of the anomalies for both a deterministic ensemble Kalman filter (EnKF) and a four-dimensional ensemble variational method, the iterative ensemble Kalman smoother (IEnKS), in a perfect model scenario. The confinement is characterized using geometrical angles that determine the relative position of the anomalies with respect to the unstable–neutral subspace. The alignment of the anomalies and of the unstable–neutral subspace is more pronounced when observation precision or frequency, as well as the data assimilation window length for the IEnKS, are increased. These results also suggest that the IEnKS and the deterministic EnKF realize in practice (albeit implicitly) the paradigm behind the approach of Anna Trevisan and co-authors known as the assimilation in the unstable subspace.

Estimating model evidence using data assimilation

Author(s): Carrassi, A; Bocquet, M; Hannart, A; Ghil, M | Abstract:

TREATMENT OF THE ERROR DUE TO UNRESOLVED SCALES IN SEQUENTIAL DATA ASSIMILATION

In this paper, a method to account for model error due to unresolved scales in sequential data assimilation, is proposed. An equation for the model error covariance required in the extended Kalman filter update is derived along with an approximation suitable for application with large scale dynamics typical in environmental modeling. This approach is tested in the context of a low order chaotic dynamical system. The results show that the filter skill is significantly improved by implementing the proposed scheme for the treatment of the unresolved scales.

Degenerate Kalman Filter Error Covariances and Their Convergence onto the Unstable Subspace

The characteristics of the model dynamics are critical in the performance of (ensemble) Kalman filters. In particular, as emphasized in the seminal work of Anna Trevisan and co-authors, the error covariance matrix is asymptotically supported by the unstable-neutral subspace only, i.e., it is spanned by the backward Lyapunov vectors with non-negative exponents. This behavior is at the core of algorithms known as Assimilation in the Unstable Subspace, although a formal proof was still missing. This paper provides the analytical proof of the convergence of the Kalman filter covariance matrix onto the unstable-neutral subspace when the dynamics and the observation operator are linear and when the dynamical model is error-free, for any, possibly rank-deficient, initial error covariance matrix. The rate of convergence is provided as well. The derivation is based on an expression that explicitly relates the error covariances at an arbitrary time to the initial ones. It is also shown that if the unstable and neutral directions of the model are sufficiently observed and if the column space of the initial covariance matrix has a non-zero projection onto all of the forward Lyapunov vectors associated with the unstable and neutral directions of the dynamics, the covariance matrix of the Kalman filter collapses onto an asymptotic sequence which is independent of the initial covariances. Numerical results are also shown to illustrate and support the theoretical findings.

Rank Deficiency of Kalman Error Covariance Matrices in Linear Time-Varying System With Deterministic Evolution

We prove that for linear, discrete, time-varying, deterministic system (perfect model) with noisy outputs, the Riccati transformation in the Kalman filter asymptotically bounds the rank of the forecast and the analysis error covariance matrices to be less than or equal to the number of non-negative Lyapunov exponents of the system. Further, the support of these error covariance matrices is shown to be confined to the space spanned by the unstable-neutral backward Lyapunov vectors, providing the theoretical justification for the methodology of the algorithms that perform assimilation only in the unstable-neutral subspace. The equivalent property of the autonomous system is investigated as a special case.