Lars Nerger

A comparison of error subspace Kalman filters

Three advanced filter algorithms based on the Kalman filter are reviewed and presented in a unified notation. They are the well known Ensemble Kalman filter (EnKF), the Singular Evolutive Extended Kalman (SEEK) filter, and the less common Singular Evolutive Interpolated Kalman (SEIK) filter. For comparison, the mathematical formulations of the filters are reviewed in relation to the extended Kalman filter as error subspace Kalman filters. The algorithms are presented in their original form and possible variations are discussed. The comparison of the algorithms shows their theoretical capabilities for efficient data assimilation with large-scale nonlinear systems. In particular, problems of the analysis equations are apparent in the original EnKF algorithm due to the Monte Carlo sampling of ensembles. Theoretically, the SEIK filter appears to be a numerically very efficient algorithm with high potential for use with nonlinear models. The superiority of the SEIK filter is demonstrated on the basis of identical twin experiments using a shallow water model with nonlinear evolution. Identical initial conditions for all three filters allow for a consistent comparison of the data assimilation results. These show how choices of particular state ensembles and assimilation schemes lead to significant variations of the filter performance. This is related to different qualities of the predicted error subspaces as is demonstrated in a examination of the predicted state covariance matrices.

On Domain Localization in Ensemble-Based Kalman Filter Algorithms

Ensemble Kalman filter methods are typically used in combination with one of two localization techniques. One technique is covariance localization, or direct forecast error localization, in which the ensemble-derived forecast error covariance matrix is Schur multiplied with a chosen correlation matrix. The second way of localization is by domain decomposition. Here, the assimilation is split into local domains in which the assimilation update is performed independently. Domain localization is frequently used in combination with filter algorithms that use the analysis error covariance matrix for the calculation of the gain like the ensemble transform Kalman filter (ETKF) and the singular evolutive interpolated Kalman filter (SEIK). However, since the local assimilations are performed independently, smoothness of the analysis fields across the subdomain boundaries becomes an issue of concern. To address the problem of smoothness, an algorithm is introduced that uses domain localization in combination with a Schur product localization of the forecast error covariance matrix for each local subdomain. On a simple example, using the Lorenz-40 system, it is demonstrated that this modification can produce results comparable to those obtained with direct forecast error localization. In addition, these results are compared to the method that uses domain localization in combination with weighting of observations. In the simple example, the method using weighting of observations is less accurate than the new method, particularly if the observation errors are small. Domain localization with weighting of observations is further examined in the case of assimilation of satellite data into the global finite-element ocean circulation model (FEOM) using the local SEIK filter. In this example, the use of observational weighting improves the accuracy of the analysis. In addition, depending on the correlation function used for weighting, the spectral properties of the solution can be improved.

Software for ensemble-based data assimilation systems-Implementation strategies and scalability

Data assimilation algorithms combine a numerical model with observations in a quantitative way. For an optimal combination either variational minimization algorithms or ensemble-based estimation methods are applied. The computations of a data assimilation application are usually far more costly than a pure model integration. To cope with the large computational costs, a good scalability of the assimilation program is required. The ensemble-based methods have been shown to exhibit a particularly good scalability due to the natural parallelism inherent in the integration of an ensemble of model states. However, also the scalability of the estimation method - commonly based on the Kalman filter - is important. This study discusses implementation strategies for ensemble-based filter algorithms. Particularly efficient is a strong coupling between the model and the assimilation algorithm into a single executable program. The coupling can be performed with minimal changes to the numerical model itself and leads to a model with data assimilation extension. The scalability of the data assimilation system is examined using the example of an implementation of an ocean circulation model with the parallel data assimilation framework (PDAF) into which synthetic sea surface height data are assimilated.

A Unification of Ensemble Square Root Kalman Filters

In recent years, several ensemble-based Kalman filter algorithms have been developed that have been classified as ensemble square root Kalman filters. Parallel to this development, the singular ‘‘evolutive’’ interpolated Kalman (SEIK) filter has been introduced and applied in several studies. Some publications note that the SEIK filter is an ensemble Kalman filter or even an ensemble square root Kalman filter. This study examines the relation of the SEIK filter to ensemble square root filters in detail. It shows that the SEIK filter is indeed an ensemble square root Kalman filter. Furthermore, a variant of the SEIK filter, the error subspace transform Kalman filter (ESTKF), is presented that results in identical ensemble transformations to those of the ensemble transform Kalman filter (ETKF), while having a slightly lower computational cost. Numerical experiments are conducted to compare the performance of three filters (SEIK, ETKF, and ESTKF) using deterministic and random ensemble transformations. The results show better performance for the ETKF and ESTKF methods over the SEIK filter as long as this filter is not applied with a symmetric square root. The findings unify the separate developments that have been performed for the SEIK filter and the other ensemble square root Kalman filters.

PDAF - The Parallel Data Assimilation Framework: Experiences with Kalman Filtering

The application of advanced data assimilation algorithms based on theKalman filter with large-scale numerical models is computationallyextremely demanding. In addition, the implementation of an dataassimilation system on the basis of existing numerical models iscomplicated by the fact that these models are typically not preparedto be used with data assimilation algorithms. To facilitate theimplementation of data assimilation systems and to reduce thecomputing time for data assimilation, the parallel data assimilationframework PDAF has been developed. PDAF allows to combine an existingnumerical model with data assimilation algorithms, like statisticalfilters, with minimal changes to the model code. Furthermore, PDAFenables the efficient use of parallel computers by creating a paralleldata assimilation system. This talk presents the structure andabilities of PDAF. In addition, the application of filter algorithmsbased on the Kalman filter is discussed and their parallel performancewithin PDAF is shown.

On the Choice of an Optimal Localization Radius in Ensemble Kalman Filter Methods

AbstractIn data assimilation applications using ensemble Kalman filter methods, localization is necessary to make the method work with high-dimensional geophysical models. For ensemble square root Kalman filters, domain localization (DL) and observation localization (OL) are commonly used. Depending on the localization method, appropriate values have to be chosen for the localization parameters, such as the localization length and the weight function. Although frequently used, the properties of the localization techniques are not fully investigated. Thus, up to now an optimal choice for these parameters is a priori unknown and they are generally found by expensive numerical experiments. In this study, the relationship between the localization length and the ensemble size in DL and OL is studied using twin experiments with the Lorenz-96 model and a two-dimensional shallow-water model. For both models, it is found that the optimal localization length for DL and OL depends linearly on an effective local obse...

Assimilating SMOS sea ice thickness into a coupled ice-ocean model using a local SEIK filter

The impact of assimilating sea ice thickness data derived from ESAs Soil Moisture and Ocean Salinity (SMOS) satellite together with Special Sensor Microwave Imager/Sounder (SSMIS) sea ice concentration data of the National Snow and Ice Data Center (NSIDC) in a coupled sea ice-ocean model is examined. A period of 3 months from 1 November 2011 to 31 January 2012 is selected to assess the forecast skill of the assimilation system. The 24 h forecasts and longer forecasts are based on the Massachusetts Institute of Technology general circulation model (MITgcm), and the assimilation is performed by a localized Singular Evolutive Interpolated Kalman (LSEIK) filter. For comparison, the assimilation is repeated only with the SSMIS sea ice concentrations. By running two different assimilation experiments, and comparing with the unassimilated model, independent satellite-derived data, and in situ observation, it is shown that the SMOS ice thickness assimilation leads to improved thickness forecasts. With SMOS thickness data, the sea ice concentration forecasts also agree better with observations, although this improvement is smaller.

On Serial Observation Processing in Localized Ensemble Kalman Filters

Ensemble square root filters can either assimilate all observations that are available at a given time at once, or assimilate the observations in batches or one at a time. For large-scale models, the filters are typically applied with a localized analysis step. This study demonstrates that the interaction of serial observation processing and localization can destabilize the analysis process, and it examines under which conditions the instability becomes significant. The instability results from a repeated inconsistent update of the state error covariance matrix that is caused by the localization. The inconsistency is present in all ensemble Kalman filters, except for the classical ensemble Kalman filter with perturbed observations. With serial observation processing, itseffect issmallin cases whenthe assimilation changesthe ensembleof modelstatesonlyslightly. However, when the assimilation has a strong effect on the state estimates, the interaction of localization and serial observation processing can significantly deteriorate the filter performance. In realistic large-scale applications,whenthe assimilationchanges the states only slightlyand whenthe distributionof the observations is irregular and changing over time, the instability is likely not significant.

Assimilating summer sea-ice concentration into a coupled ice-ocean model using a LSEIK filter

Abstract The decrease in summer sea-ice extent in the Arctic Ocean opens shipping routes and creates potential for many marine operations. For these activities accurate predictions of sea-ice conditions are required to maintain marine safety. In an attempt at Arctic sea-ice prediction, the summer of 2010 is selected to implement an Arctic sea-ice data assimilation (DA) study. The DA system is based on a regional Arctic configuration of the Massachusetts Institute of Technology general circulation model (MITgcm) and a local singular evolutive interpolated Kalman (LSEIK) filter to assimilate Special Sensor Microwave Imager/Sounder (SSMIS) sea-ice concentration operational products from the US National Snow and Ice Data Center (NSIDC). Based on comparisons with both the assimilated NSIDC SSMIS concentration and concentration data from the Ocean and Sea Ice Satellite Application Facility, the forecasted sea-ice edge and concentration improve upon simulations without data assimilation. By the nature of the assimilation algorithm with multivariate covariance between ice concentration and thickness, sea-ice thickness fields are also updated, and the evaluation with in situ observation shows some improvement compared to the forecast without data assimilation.

Assessment of a Nonlinear Ensemble Transform Filter for High-Dimensional Data Assimilation

AbstractThis work assesses the large-scale applicability of the recently proposed nonlinear ensemble transform filter (NETF) in data assimilation experiments with the NEMO ocean general circulation model. The new filter constitutes a second-order exact approximation to fully nonlinear particle filtering. Thus, it relaxes the Gaussian assumption contained in ensemble Kalman filters. The NETF applies an update step similar to the local ensemble transform Kalman filter (LETKF), which allows for efficient and simple implementation. Here, simulated observations are assimilated into a simplified ocean configuration that exhibits globally high-dimensional dynamics with a chaotic mesoscale flow. The model climatology is used to initialize an ensemble of 120 members. The number of observations in each local filter update is of the same order resulting from the use of a realistic oceanic observation scenario. Here, an importance sampling particle filter (PF) would require at least 106 members. Despite the relativel...