Geir Evensen

Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics

A new sequential data assimilation method is discussed. It is based on forecasting the error statistics using Monte Carlo methods, a better alternative than solving the traditional and computationally extremely demanding approximate error covariance equation used in the extended Kalman filter. The unbounded error growth found in the extended Kalman filter, which is caused by an overly simplified closure in the error covariance equation, is completely eliminated. Open boundaries can be handled as long as the ocean model is well posed. Well-known numerical instabilities associated with the error covariance equation are avoided because storage and evolution of the error covariance matrix itself are not needed. The results are also better than what is provided by the extended Kalman filter since there is no closure problem and the quality of the forecast error statistics therefore improves. The method should be feasible also for more sophisticated primitive equation models. The computational load for reasonable accuracy is only a fraction of what is required for the extended Kalman filter and is given by the storage of, say, 100 model states for an ensemble size of 100 and thus CPU requirements of the order of the cost of 100 model integrations. The proposed method can therefore be used with realistic nonlinear ocean models on large domains on existing computers, and it is also well suited for parallel computers and clusters of workstations where each processor integrates a few members of the ensemble.

Analysis Scheme in the Ensemble Kalman Filter

This paper discusses an important issue related to the implementation and interpretation of the analysis scheme in the ensemble Kalman filter. It is shown that the observations must be treated as random variables at the analysis steps. That is, one should add random perturbations with the correct statistics to the observations and generate an ensemble of observations that then is used in updating the ensemble of model states. Traditionally, this has not been done in previous applications of the ensemble Kalman filter and, as will be shown, this has resulted in an updated ensemble with a variance that is too low. This simple modification of the analysis scheme results in a completely consistent approach if the covariance of the ensemble of model states is interpreted as the prediction error covariance, and there are no further requirements on the ensemble Kalman filter method, except for the use of an ensemble of sufficient size. Thus, there is a unique correspondence between the error statistics from the ensemble Kalman filter and the standard Kalman filter approach.

An Ensemble Kalman Smoother for Nonlinear Dynamics

It is formally proved that the general smoother for nonlinear dynamics can be formulated as a sequential method, that is, observations can be assimilated sequentially during a forward integration. The general filter can be derived from the smoother and it is shown that the general smoother and filter solutions at the final time become identical, as is expected from linear theory. Then, a new smoother algorithm based on ensemble statistics is presented and examined in an example with the Lorenz equations. The new smoother can be computed as a sequential algorithm using only forward-in-time model integrations. It bears a strong resemblance with the ensemble Kalman filter. The difference is that every time a new dataset is available during the forward integration, an analysis is computed for all previous times up to this time. Thus, the first guess for the smoother is the ensemble Kalman filter solution, and the smoother estimate provides an improvement of this, as one would expect a smoother to do. The method is demonstrated in this paper in an intercomparison with the ensemble Kalman filter and the ensemble smoother introduced by van Leeuwen and Evensen, and it is shown to be superior in an application with the Lorenz equations. Finally, a discussion is given regarding the properties of the analysis schemes when strongly non-Gaussian distributions are used. It is shown that in these cases more sophisticated analysis schemes based on Bayesian statistics must be used.

Assimilation of Geosat Altimeter Data for the Agulhas Current Using the Ensemble Kalman Filter with a Quasigeostrophic Model

Abstract The ring-shedding process in the Agulhas Current is studied using the ensemble Kalman filter to assimilate Geosat altimeter data into a two-layer quasigeostrophic ocean model. The properties of the ensemble Kalman filter are further explored with focus on the analysis scheme and the use of gridded data. The Geosat data consist of 10 fields of gridded sea surface height anomalies separated 10 days apart that are added to a climatic mean field. This corresponds to a huge number of data values, and a data reduction scheme must be applied to increase the efficiency of the analysis procedure. Further, it is illustrated how one can resolve the rank problem occurring when a too large dataset or a small ensemble is used.

Data Assimilation and Inverse Methods in Terms of a Probabilistic Formulation

Abstract The weak constraint inverse for nonlinear dynamical models is discussed and derived in term of a probabilistic formulation. The well-known result that for Gaussian error statistics the minimum of the weak constraint inverse is equal to the maximum-likelihood estimate is rederived. Then several methods based on ensemble statistics that can be used to find the smoother (as opposed to the filter) solution are introduced and compared to traditional methods. A strong point of the new methods is that they avoid the integration of adjoint equations, which is a complex task for real oceanographic or atmospheric applications. They also avoid iterative searches in a Hilbert space, and error estimates can be obtained without much additional computational effort. The feasibility of the new methods is illustrated in a two-layer quasigeostrophic ocean model.

The ensemble Kalman filter for combined state and parameter estimation

This article provides a fundamental theoretical basis for understanding EnKF and serves as a useful text for future users. Data assimilation and parameter-estimation problems are explained, and the concept of joint parameter and state estimation, which can be solved using ensemble methods, is presented. KF and EKF are briefly discussed before introducing and deriving EnKF. Similarities and differences between KF and EnKF are pointed out. The benefits of using EnKF with high-dimensional and highly nonlinear dynamical models are illustrated by examples. EnKF and EnKS are also derived from Bayes theorem, using a probabilistic approach. The derivation is based on the assumption that measurement errors are independent in time and the model represents a Markov process, which allows for Bayes theorem to be written in a recursive form, where measurements are processed sequentially in time. The practical implementation of the analysis scheme isdiscussed, and it is shown that it can be computed efficiently in the space spanned by the ensemble realizations. The square root scheme is discussed as an alternative method that avoids the perturbation of measurements. However, the square root scheme has other pitfalls, and it is recommended to use the symmetric square root with or without a random rotation. The random rotation introduces a stochastic component to the update, and the quality of the scheme may then not improve compared to the original stochastic EnKF scheme with perturbed measurements.

Using the extended Kalman filter with a multilayer quasi-geostrophic ocean model

The formulation of the extended Kalman filter for a multilayer nonlinear quasi-geostrophic ocean circulation model is discussed. The nonlinearity in the ocean model leads to an approximative equation for error covariance propagation, where the transition matrix is dependent on the state trajectory. This nonlinearity complicates the dynamics of the error covariance propagation, and effects which are nonexistent in linear systems contribute significantly. The transition matrix can be split into two parts, where one part results in pure evolution of error covariances in the model velocity field, and the other part contains a statistical correction term caused by the nonlinearity in the model. This correction term leads to a linear unbounded instability, which is caused by the statistical linearization of the nonlinear error propagation equation. Different ways of handling this instability are discussed. Further, nonlinear small-scale instabilities also develop, since energy is accumulated at wavelengths 2Δx, owing to the numerical discretization. These small-scale oscillations are removed with a Shapiro filter, and the effect they have on the error covariance propagation is discussed. Some data assimilation experiments are performed using the full extended Kalman filter, to examine the properties of the filter. An experiment where only the first part of the transition matrix is used to propagate the error covariances is also performed. This simplified experiment actually performs better than the full extended Kalman filter because the unbounded instability associated with the statistical correction term is avoided.

Advanced Data Assimilation for Strongly Nonlinear Dynamics

Advanced data assimilation methods become extremely complicated and challenging when used with strongly nonlinear models. Several previous works have reported various problems when applying existing popular data assimilation techniques with strongly nonlinear dynamics. Common for these techniques is that they can all be considered as extensions to methods that have proved to work well with linear dynamics. This paper examines the properties of three advanced data assimilation methods when used with the highly nonlinear Lorenz equations. The ensemble Kalman filter is used for sequential data assimilation and the recently proposed ensemble smoother method and a gradient descent method are used to minimize two different weak constraint formulations. The problems associated with the use of an approximate tangent linear model when solving the Euler‐Lagrange equations, or when the extended Kalman filter is used, are eliminated when using these methods. All three methods give reasonable consistent results with the data coverage and quality of measurements that are used here and overcome the traditional problems reported in many of the previous papers involving data assimilation with highly nonlinear dynamics.

Assimilation of ocean colour data into a biochemical model of the North Atlantic: Part 1. Data assimilation experiments

Abstract An advanced multivariate sequential data assimilation method, the ensemble Kalman filter (EnKF), has been investigated with a three-dimensional biochemical model of the North Atlantic, utilizing real chlorophyll data from the from the Sea-viewing Wide Field-of-view Sensor (SeaWiFS). The approach chosen here differs significantly from conventional parameter estimation techniques. We keep the parameters fixed, and instead update the actual model state, allowing for unknown errors in the dynamical formulation. In the ensemble Kalman filter, estimates of the true dynamical error covariances are provided from an ensemble of model states. The physical ocean is described through the Miami Isopycnic Coordinate Ocean Model (MICOM). Its output, e.g. fields of temperature, velocities and layer thicknesses, is used to force the ecosystem model, which contains 11 biochemical components. The system is driven by realistic ECMWF atmospheric forcing fields. A simple demonstration experiment was performed for April and May 1998, that is, the early part of the North Atlantic spring bloom is included. It is shown that the ensemble Kalman filter analysis estimate is consistent with the choice of prior error variances. Furthermore, it is illustrated that the multivariate analysis scheme also affects the unobserved model compartments, and that the variance fields for all the variables decrease during the assimilation. A discussion of the number of ensemble members needed to ensure proper representations of the true error covariances is also given. In a companion paper [J. Mar. Syst. 40/41 (2003)], hereafter referred to as the Part 2 paper, we illustrate some useful approaches to monitor the evolution (i.e., time series) of the ensemble of model states.

Inverse methods and data assimilation in nonlinear ocean models

Abstract An overview is given of the current status of inverse methods and data assimilation for nonlinear ocean models. The inverse theory for time dependent dynamical models is formulated and the most promising solution methods like simulated annealing, the representer method, and sequential methods based on Monte Carlo simulations, are discussed with special focus on applications with nonlinear dynamics. A rather general “model independent” presentation has been used to make the methodology more accessible for different scientific areas dealing with dynamical models and data.