Di Qi

Blended particle filters for large-dimensional chaotic dynamical systems

Significance Combining large uncertain computational models with big noisy datasets is a formidable problem throughout science and engineering. These are especially difficult issues when real-time state estimation and prediction are needed such as, for example, in weather forecasting. Thus, a major challenge in contemporary data science is the development of statistically accurate particle filters to capture non-Gaussian features in large-dimensional chaotic dynamical systems. New blended particle filters are developed in this paper. These algorithms exploit the physical structure of turbulent dynamical systems and capture non-Gaussian features in an adaptively evolving low-dimensional subspace through particles interacting with evolving Gaussian statistics on the remaining portion of the phase space. A major challenge in contemporary data science is the development of statistically accurate particle filters to capture non-Gaussian features in large-dimensional chaotic dynamical systems. Blended particle filters that capture non-Gaussian features in an adaptively evolving low-dimensional subspace through particles interacting with evolving Gaussian statistics on the remaining portion of phase space are introduced here. These blended particle filters are constructed in this paper through a mathematical formalism involving conditional Gaussian mixtures combined with statistically nonlinear forecast models compatible with this structure developed recently with high skill for uncertainty quantification. Stringent test cases for filtering involving the 40-dimensional Lorenz 96 model with a 5-dimensional adaptive subspace for nonlinear blended filtering in various turbulent regimes with at least nine positive Lyapunov exponents are used here. These cases demonstrate the high skill of the blended particle filter algorithms in capturing both highly non-Gaussian dynamical features as well as crucial nonlinear statistics for accurate filtering in extreme filtering regimes with sparse infrequent high-quality observations. The formalism developed here is also useful for multiscale filtering of turbulent systems and a simple application is sketched below.

Improving Prediction Skill of Imperfect Turbulent Models Through Statistical Response and Information Theory

Turbulent dynamical systems with a large phase space and a high degree of instabilities are ubiquitous in climate science and engineering applications. Statistical uncertainty quantification (UQ) to the response to the change in forcing or uncertain initial data in such complex turbulent systems requires the use of imperfect models due to the lack of both physical understanding and the overwhelming computational demands of Monte Carlo simulation with a large-dimensional phase space. Thus, the systematic development of reduced low-order imperfect statistical models for UQ in turbulent dynamical systems is a grand challenge. This paper applies a recent mathematical strategy for calibrating imperfect models in a training phase and accurately predicting the response by combining information theory and linear statistical response theory in a systematic fashion. A systematic hierarchy of simple statistical imperfect closure schemes for UQ for these problems is designed and tested which are built through new local and global statistical energy conservation principles combined with statistical equilibrium fidelity. The forty mode Lorenz 96 (L-96) model which mimics forced baroclinic turbulence is utilized as a test bed for the calibration and predicting phases for the hierarchy of computationally cheap imperfect closure models both in the full phase space and in a reduced three-dimensional subspace containing the most energetic modes. In all of phase spaces, the nonlinear response of the true model is captured accurately for the mean and variance by the systematic closure model, while alternative methods based on the fluctuation-dissipation theorem alone are much less accurate. For reduced-order model for UQ in the three-dimensional subspace for L-96, the systematic low-order imperfect closure models coupled with the training strategy provide the highest predictive skill over other existing methods for general forced response yet have simple design principles based on a statistical global energy equation. The systematic imperfect closure models and the calibration strategies for UQ for the L-96 model serve as a new template for similar strategies for UQ with model error in vastly more complex realistic turbulent dynamical systems.

Low-Dimensional Reduced-Order Models for Statistical Response and Uncertainty Quantification: Two-Layer Baroclinic Turbulence

AbstractAccurate uncertainty quantification for the mean and variance about forced responses to general external perturbations in the climate system is an important subject in understanding Earth’s atmosphere and ocean in climate change science. A low-dimensional reduced-order method is developed for uncertainty quantification and capturing the statistical sensitivity in the principal model directions with largest variability and in various regimes in two-layer quasigeostrophic turbulence. Typical dynamical regimes tested here include the homogeneous flow in the high latitudes and the anisotropic meandering jets in the low latitudes and/or midlatitudes. The idea in the reduced-order method is from a self-consistent mathematical framework for general systems with quadratic nonlinearity, where crucial high-order statistics are approximated by a systematic model calibration procedure. Model efficiency is improved through additional damping and noise corrections to replace the expensive energy-conserving nonl...

Preventing Catastrophic Filter Divergence Using Adaptive Additive Inflation for Baroclinic Turbulence

AbstractEnsemble-based filtering or data assimilation methods have proved to be indispensable tools in atmosphere and ocean science as they allow computationally cheap, low-dimensional ensemble state approximation for extremely high-dimensional turbulent dynamical systems. For sparse, accurate, and infrequent observations, which are typical in data assimilation of geophysical systems, ensemble filtering methods can suffer from catastrophic filter divergence, which frequently drives the filter predictions to machine infinity. A two-layer quasigeostrophic equation, which is a classical idealized model for geophysical turbulence, is used to demonstrate catastrophic filter divergence. The mathematical theory of adaptive covariance inflation by Tong et al. and covariance localization are investigated to stabilize the ensemble methods and prevent catastrophic filter divergence. Two forecast models—a coarse-grained ocean code, which ignores the small-scale parameterization, and stochastic superparameterization (...

A flux-balanced fluid model for collisional plasma edge turbulence: Model derivation and basic physical features

We propose a new reduced fluid model for the study of the drift wave–zonal flow dynamics in magnetically confined plasmas. Our model can be viewed as an extension of the classic Hasegawa-Wakatani (HW) model and is based on an improved treatment of the electron dynamics parallel to the field lines, to guarantee a balanced electron flux on the magnetic surfaces. Our flux-balanced HW (bHW) model contains the same drift-wave instability as previous HW models, but unlike these models, it converges exactly to the modified Hasegawa-Mima model in the collisionless limit. We rely on direct numerical simulations to illustrate some of the key features of the bHW model, such as the enhanced variability in the turbulent fluctuations and the existence of stronger and more turbulent zonal jets than the jets observed in other HW models, especially for high plasma resistivity. Our simulations also highlight the crucial role of the feedback of the third-order statistical moments in achieving a statistical equilibrium with strong zonal structures. Finally, we investigate the changes in the observed dynamics when more general dissipation effects are included and, in particular, when we include the reduced model for ion Landau damping originally proposed by Wakatani and Hasegawa.We propose a new reduced fluid model for the study of the drift wave–zonal flow dynamics in magnetically confined plasmas. Our model can be viewed as an extension of the classic Hasegawa-Wakatani (HW) model and is based on an improved treatment of the electron dynamics parallel to the field lines, to guarantee a balanced electron flux on the magnetic surfaces. Our flux-balanced HW (bHW) model contains the same drift-wave instability as previous HW models, but unlike these models, it converges exactly to the modified Hasegawa-Mima model in the collisionless limit. We rely on direct numerical simulations to illustrate some of the key features of the bHW model, such as the enhanced variability in the turbulent fluctuations and the existence of stronger and more turbulent zonal jets than the jets observed in other HW models, especially for high plasma resistivity. Our simulations also highlight the crucial role of the feedback of the third-order statistical moments in achieving a statistical equilibrium with ...

Rigorous Statistical Bounds in Uncertainty Quantification for One-Layer Turbulent Geophysical Flows

Statistical bounds controlling the total fluctuations in mean and variance about a basic steady-state solution are developed for the truncated barotropic flow over topography. Statistical ensemble prediction is an important topic in weather and climate research. Here, the evolution of an ensemble of trajectories is considered using statistical instability analysis and is compared and contrasted with the classical deterministic instability for the growth of perturbations in one pointwise trajectory. The maximum growth of the total statistics in fluctuations is derived relying on the statistical conservation principle of the pseudo-energy. The saturation bound of the statistical mean fluctuation and variance in the unstable regimes with non-positive-definite pseudo-energy is achieved by linking with a class of stable reference states and minimizing the stable statistical energy. Two cases with dependence on initial statistical uncertainty and on external forcing and dissipation are compared and unified under a consistent statistical stability framework. The flow structures and statistical stability bounds are illustrated and verified by numerical simulations among a wide range of dynamical regimes, where subtle transient statistical instability exists in general with positive short-time exponential growth in the covariance even when the pseudo-energy is positive-definite. Among the various scenarios in this paper, there exist strong forward and backward energy exchanges between different scales which are estimated by the rigorous statistical bounds.

Effective control of complex turbulent dynamical systems through statistical functionals

Significance Turbulent dynamical systems characterized by both a high-dimensional phase space and a large number of instabilities are ubiquitous among complex systems in science and engineering, including climate, material, and neural science. Control of these complex systems is a grand challenge, for example, in mitigating the effects of climate change or safe design of technology with fully developed shear turbulence. In complex turbulent dynamical systems, it is impossible to track and control the large dimension of instabilities, which strongly interact and exchange energy, and new control strategies are needed. The goal here is to propose an effective statistical control strategy for complex turbulent dynamical systems based on a recent statistical energy principle and statistical linear response theory. Turbulent dynamical systems characterized by both a high-dimensional phase space and a large number of instabilities are ubiquitous among complex systems in science and engineering, including climate, material, and neural science. Control of these complex systems is a grand challenge, for example, in mitigating the effects of climate change or safe design of technology with fully developed shear turbulence. Control of flows in the transition to turbulence, where there is a small dimension of instabilities about a basic mean state, is an important and successful discipline. In complex turbulent dynamical systems, it is impossible to track and control the large dimension of instabilities, which strongly interact and exchange energy, and new control strategies are needed. The goal of this paper is to propose an effective statistical control strategy for complex turbulent dynamical systems based on a recent statistical energy principle and statistical linear response theory. We illustrate the potential practical efficiency and verify this effective statistical control strategy on the 40D Lorenz 1996 model in forcing regimes with various types of fully turbulent dynamics with nearly one-half of the phase space unstable.

Stochastic superparameterization and multiscale filtering of turbulent tracers

Data assimilation or filtering combines a numerical forecast model and observations to provide accurate statistical estimation of the state of interest. In this paper we are concerned with accurate data assimilation of a sparsely observed passive tracer advected in turbulent flows using a reduced-order forecast model. The turbulent flows which contain anisotropic and inhomogeneous structures such as jets are typical in geophysical turbulent flows in atmosphere and ocean science, and passive tracers with a mean gradient can exhibit anisotropic transport with intermittent extreme events, as shown below. Stochastic superparameterization, which is a seamless multiscale method developed for large-scale models of atmosphere and ocean models without scale-gap between the resolved and unresolved scales, generates large-scale turbulent velocity fields using a significantly smaller degree of freedom compared to a direct fine resolution numerical simulation. In a large-scale model of the tracer transport, the tracer...