Jakob Runge

Quantifying causal coupling strength: A lag-specific measure for multivariate time series related to transfer entropy

While it is an important problem to identify the existence of causal associations between two components of a multivariate time series, a topic addressed in Runge, Heitzig, Petoukhov, and Kurths [Phys. Rev. Lett. 108, 258701 (2012)], it is even more important to assess the strength of their association in a meaningful way. In the present article we focus on the problem of defining a meaningful coupling strength using information-theoretic measures and demonstrate the shortcomings of the well-known mutual information and transfer entropy. Instead, we propose a certain time-delayed conditional mutual information, the momentary information transfer (MIT), as a lag-specific measure of association that is general, causal, reflects a well interpretable notion of coupling strength, and is practically computable. Rooted in information theory, MIT is general in that it does not assume a certain model class underlying the process that generates the time series. As discussed in a previous paper [Runge, Heitzig, Petoukhov, and Kurths, Phys. Rev. Lett. 108, 258701 (2012)], the general framework of graphical models makes MIT causal in that it gives a nonzero value only to lagged components that are not independent conditional on the remaining process. Further, graphical models admit a low-dimensional formulation of conditions, which is important for a reliable estimation of conditional mutual information and, thus, makes MIT practically computable. MIT is based on the fundamental concept of source entropy, which we utilize to yield a notion of coupling strength that is, compared to mutual information and transfer entropy, well interpretable in that, for many cases, it solely depends on the interaction of the two components at a certain lag. In particular, MIT is, thus, in many cases able to exclude the misleading influence of autodependency within a process in an information-theoretic way. We formalize and prove this idea analytically and numerically for a general class of nonlinear stochastic processes and illustrate the potential of MIT on climatological data.

Quantifying the Strength and Delay of Climatic Interactions: The Ambiguities of Cross Correlation and a Novel Measure Based on Graphical Models

Lagged cross-correlation and regression analysis are commonly used to gain insights into interaction mechanisms between climatological processes, in particular to assess time delays and to quantify the strength of a mechanism. Exemplified on temperature anomalies in Europe and the tropical Pacific and Atlantic, the authors study lagged correlation and regressions analytically for a simple model system. A strong dependence on the influence of serial dependencies or autocorrelation is demonstrated, which can lead to misleading conclusions about time delays and also obscures a quantification of the interaction mechanism. To overcome these possible artifacts, the authors propose a two-step procedure based on the concept of graphical models recently introduced to climate research. In the first step, graphical models are used to detect the existence of (Granger) causal interactions that determine the time delays of a mechanism. In the second step, a certain partial correlation and a regression measure are introduced that allow one to specifically quantify the strength of an interaction mechanism in a well interpretable way that enables the exclusion of misleadingeffects of serial correlation as well as more generaldependencies. The potential of the approachto quantify interactions between two and more processes is demonstrated by investigating teleconnections of ENSO and the mechanism of the Walker circulation. The article is intended to serve as a guideline to interpret lagged correlations and regressions in the presence of autocorrelation and introduces a powerful approach to analyze time delays and the strength of an interaction mechanism.

Disentangling different types of El Nino episodes by evolving climate network analysis

Complex network theory provides a powerful toolbox for studying the structure of statistical interrelationships between multiple time series in various scientific disciplines. In this work, we apply the recently proposed climate network approach for characterizing the evolving correlation structure of the Earths climate system based on reanalysis data for surface air temperatures. We provide a detailed study of the temporal variability of several global climate network characteristics. Based on a simple conceptual view of red climate networks (i.e., networks with a comparably low number of edges), we give a thorough interpretation of our evolving climate network characteristics, which allows a functional discrimination between recently recognized different types of El Niño episodes. Our analysis provides deep insights into the Earths climate system, particularly its global response to strong volcanic eruptions and large-scale impacts of different phases of the El Niño Southern Oscillation.

Identifying causal gateways and mediators in complex spatio-temporal systems.

Identifying regions important for spreading and mediating perturbations is crucial to assess the susceptibilities of spatio-temporal complex systems such as the Earths climate to volcanic eruptions, extreme events or geoengineering. Here a data-driven approach is introduced based on a dimension reduction, causal reconstruction, and novel network measures based on causal effect theory that go beyond standard complex network tools by distinguishing direct from indirect pathways. Applied to a data set of atmospheric dynamics, the method identifies several strongly uplifting regions acting as major gateways of perturbations spreading in the atmosphere. Additionally, the method provides a stricter statistical approach to pathways of atmospheric teleconnections, yielding insights into the Pacific–Indian Ocean interaction relevant for monsoonal dynamics. Also for neuroscience or power grids, the novel causal interaction perspective provides a complementary approach to simulations or experiments for understanding the functioning of complex spatio-temporal systems with potential applications in increasing their resilience to shocks or extreme events.

Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information

Across geosciences, many investigated phenomena relate to specific complex systems consisting of intricately intertwined interacting subsystems. Such dynamical complex systems can be represented by a directed graph, where each link denotes an existence of a causal relation, or information exchange between the nodes. For geophysical systems such as global climate, these relations are commonly not theoretically known but estimated from recorded data using causality analysis methods. These include bivariate nonlinear methods based on information theory and their linear counterpart. The trade-off between the valuable sensitivity of nonlinear methods to more general interactions and the potentially higher numerical reliability of linear methods may affect inference regarding structure and variability of climate networks. We investigate the reliability of directed climate networks detected by selected methods and parameter settings, using a stationarized model of dimensionality-reduced surface air temperature data from reanalysis of 60-year global climate records. Overall, all studied bivariate causality methods provided reproducible estimates of climate causality networks, with the linear approximation showing

Using Causal Effect Networks to Analyze Different Arctic Drivers of Midlatitude Winter Circulation

AbstractIn recent years, the Northern Hemisphere midlatitudes have suffered from severe winters like the extreme 2012/13 winter in the eastern United States. These cold spells were linked to a meandering upper-tropospheric jet stream pattern and a negative Arctic Oscillation index (AO). However, the nature of the drivers behind these circulation patterns remains controversial. Various studies have proposed different mechanisms related to changes in the Arctic, most of them related to a reduction in sea ice concentrations or increasing Eurasian snow cover.Here, a novel type of time series analysis, called causal effect networks (CEN), based on graphical models is introduced to assess causal relationships and their time delays between different processes. The effect of different Arctic actors on winter circulation on weekly to monthly time scales is studied, and robust network patterns are found. Barents and Kara sea ice concentrations are detected to be important external drivers of the midlatitude circula...

Statistical Mechanics and Information-Theoretic Perspectives on Complexity in the Earth System

This review provides a summary of methods originated in (non-equilibrium) statistical mechanics and information theory, which have recently found successful applications to quantitatively studying complexity in various components of the complex system Earth. Specifically, we discuss two classes of methods: (i) entropies of different kinds (e.g., on the one hand classical Shannon and R´enyi entropies, as well as non-extensive Tsallis entropy based on symbolic dynamics techniques and, on the other hand, approximate entropy, sample entropy and fuzzy entropy); and (ii) measures of statistical interdependence and causality (e.g., mutual information and generalizations thereof, transfer entropy, momentary information transfer). We review a number of applications and case studies utilizing the above-mentioned methodological approaches for studying contemporary problems in some exemplary fields of the Earth sciences, highlighting the potentials of different techniques.

Unified functional network and nonlinear time series analysis for complex systems science: The pyunicorn package

We introduce the pyunicorn (Pythonic unified complex network and recurrence analysis toolbox) open source software package for applying and combining modern methods of data analysis and modeling from complex network theory and nonlinear time series analysis. pyunicorn is a fully object-oriented and easily parallelizable package written in the language Python. It allows for the construction of functional networks such as climate networks in climatology or functional brain networks in neuroscience representing the structure of statistical interrelationships in large data sets of time series and, subsequently, investigating this structure using advanced methods of complex network theory such as measures and models for spatial networks, networks of interacting networks, node-weighted statistics, or network surrogates. Additionally, pyunicorn provides insights into the nonlinear dynamics of complex systems as recorded in uni- and multivariate time series from a non-traditional perspective by means of recurrence quantification analysis, recurrence networks, visibility graphs, and construction of surrogate time series. The range of possible applications of the library is outlined, drawing on several examples mainly from the field of climatology.

Quantifying information transfer and mediation along causal pathways in complex systems.

Measures of information transfer have become a popular approach to analyze interactions in complex systems such as the Earth or the human brain from measured time series. Recent work has focused on causal definitions of information transfer aimed at decompositions of predictive information about a target variable, while excluding effects of common drivers and indirect influences. While common drivers clearly constitute a spurious causality, the aim of the present article is to develop measures quantifying different notions of the strength of information transfer along indirect causal paths, based on first reconstructing the multivariate causal network. Another class of novel measures quantifies to what extent different intermediate processes on causal paths contribute to an interaction mechanism to determine pathways of causal information transfer. The proposed framework complements predictive decomposition schemes by focusing more on the interaction mechanism between multiple processes. A rigorous mathematical framework allows for a clear information-theoretic interpretation that can also be related to the underlying dynamics as proven for certain classes of processes. Generally, however, estimates of information transfer remain hard to interpret for nonlinearly intertwined complex systems. But if experiments or mathematical models are not available, then measuring pathways of information transfer within the causal dependency structure allows at least for an abstraction of the dynamics. The measures are illustrated on a climatological example to disentangle pathways of atmospheric flow over Europe.

Optimal model-free prediction from multivariate time series.

Forecasting a time series from multivariate predictors constitutes a challenging problem, especially using model-free approaches. Most techniques, such as nearest-neighbor prediction, quickly suffer from the curse of dimensionality and overfitting for more than a few predictors which has limited their application mostly to the univariate case. Therefore, selection strategies are needed that harness the available information as efficiently as possible. Since often the right combination of predictors matters, ideally all subsets of possible predictors should be tested for their predictive power, but the exponentially growing number of combinations makes such an approach computationally prohibitive. Here a prediction scheme that overcomes this strong limitation is introduced utilizing a causal preselection step which drastically reduces the number of possible predictors to the most predictive set of causal drivers making a globally optimal search scheme tractable. The information-theoretic optimality is derived and practical selection criteria are discussed. As demonstrated for multivariate nonlinear stochastic delay processes, the optimal scheme can even be less computationally expensive than commonly used suboptimal schemes like forward selection. The method suggests a general framework to apply the optimal model-free approach to select variables and subsequently fit a model to further improve a prediction or learn statistical dependencies. The performance of this framework is illustrated on a climatological index of El Niño Southern Oscillation.