Alessandro Montalto

MuTE: a MATLAB toolbox to compare established and novel estimators of the multivariate transfer entropy.

A challenge for physiologists and neuroscientists is to map information transfer between components of the systems that they study at different scales, in order to derive important knowledge on structure and function from the analysis of the recorded dynamics. The components of physiological networks often interact in a nonlinear way and through mechanisms which are in general not completely known. It is then safer that the method of choice for analyzing these interactions does not rely on any model or assumption on the nature of the data and their interactions. Transfer entropy has emerged as a powerful tool to quantify directed dynamical interactions. In this paper we compare different approaches to evaluate transfer entropy, some of them already proposed, some novel, and present their implementation in a freeware MATLAB toolbox. Applications to simulated and real data are presented.

Lag-Specific Transfer Entropy as a Tool to Assess Cardiovascular and Cardiorespiratory Information Transfer

In the study of interacting physiological systems, model-free tools for time series analysis are fundamental to provide a proper description of how the coupling among systems arises from the multiple involved regulatory mechanisms. This study presents an approach which evaluates direction, magnitude, and exact timing of the information transfer between two time series belonging to a multivariate dataset. The approach performs a decomposition of the well-known transfer entropy (TE) which achieves 1) identifying, according to a lag-specific information-theoretic formulation of the concept of Granger causality, the set of time lags associated with significant information transfer, and 2) assigning to these delays an amount of information transfer such that the total contribution yields the aggregate TE. The approach is first validated on realizations of simulated linear and nonlinear multivariate processes interacting at different time lags and with different strength, reporting a high accuracy in the detection of imposed delays, and showing that the estimated lag-specific TE follows the imposed coupling strength. The subsequent application to heart period, systolic arterial pressure and respiration variability series measured from healthy subjects during a tilt test protocol illustrated how the proposed approach quantifies the modifications in the involvement and latency of important mechanisms of short-term physiological regulation, like the baroreflex and the respiratory sinus arrhythmia, induced by the orthostatic stress.

Cardiorespiratory information dynamics during mental arithmetic and sustained attention

An analysis of cardiorespiratory dynamics during mental arithmetic, which induces stress, and sustained attention was conducted using information theory. The information storage and internal information of heart rate variability (HRV) were determined respectively as the self-entropy of the tachogram, and the self-entropy of the tachogram conditioned to the knowledge of respiration. The information transfer and cross information from respiration to HRV were assessed as the transfer and cross-entropy, both measures of cardiorespiratory coupling. These information-theoretic measures identified significant nonlinearities in the cardiorespiratory time series. Additionally, it was shown that, although mental stress is related to a reduction in vagal activity, no difference in cardiorespiratory coupling was found when several mental states (rest, mental stress, sustained attention) are compared. However, the self-entropy of HRV conditioned to respiration was very informative to study the predictability of RR interval series during mental tasks, and showed higher predictability during mental arithmetic compared to sustained attention or rest.

Neural networks with non-uniform embedding and explicit validation phase to assess Granger causality

A challenging problem when studying a dynamical system is to find the interdependencies among its individual components. Several algorithms have been proposed to detect directed dynamical influences between time series. Two of the most used approaches are a model-free one (transfer entropy) and a model-based one (Granger causality). Several pitfalls are related to the presence or absence of assumptions in modeling the relevant features of the data. We tried to overcome those pitfalls using a neural network approach in which a model is built without any a priori assumptions. In this sense this method can be seen as a bridge between model-free and model-based approaches. The experiments performed will show that the method presented in this work can detect the correct dynamical information flows occurring in a system of time series. Additionally we adopt a non-uniform embedding framework according to which only the past states that actually help the prediction are entered into the model, improving the prediction and avoiding the risk of overfitting. This method also leads to a further improvement with respect to traditional Granger causality approaches when redundant variables (i.e. variables sharing the same information about the future of the system) are involved. Neural networks are also able to recognize dynamics in data sets completely different from the ones used during the training phase.

A linear approach for sparse coding by a two-layer neural network

Abstract Many approaches to transform classification problems from non-linear to linear by feature transformation have been recently presented in the literature. These notably include sparse coding methods and deep neural networks. However, many of these approaches require the repeated application of a learning process upon the presentation of unseen data input vectors, or else involve the use of large numbers of parameters and hyper-parameters, which must be chosen through cross-validation, thus increasing running time dramatically. In this paper, we propose and experimentally investigate a new approach for the purpose of overcoming limitations of both kinds. The proposed approach makes use of a linear auto-associative network (called SCNN) with just one hidden layer. The combination of this architecture with a specific error function to be minimized enables one to learn a linear encoder computing a sparse code which turns out to be as similar as possible to the sparse coding that one obtains by re-training the neural network. Importantly, the linearity of SCNN and the choice of the error function allow one to achieve reduced running time in the learning phase. The proposed architecture is evaluated on the basis of two standard machine learning tasks. Its performances are compared with those of recently proposed non-linear auto-associative neural networks. The overall results suggest that linear encoders can be profitably used to obtain sparse data representations in the context of machine learning problems, provided that an appropriate error function is used during the learning phase.

Multiscale analysis of information dynamics for linear multivariate processes

In the study of complex physical and physiological systems represented by multivariate time series, an issue of great interest is the description of the system dynamics over a range of different temporal scales. While information-theoretic approaches to the multiscale analysis of complex dynamics are being increasingly used, the theoretical properties of the applied measures are poorly understood. This study introduces for the first time a framework for the analytical computation of information dynamics for linear multivariate stochastic processes explored at different time scales. After showing that the multiscale processing of a vector autoregressive (VAR) process introduces a moving average (MA) component, we describe how to represent the resulting VARMA process using statespace (SS) models and how to exploit the SS model parameters to compute analytical measures of information storage and information transfer for the original and rescaled processes. The framework is then used to quantify multiscale information dynamics for simulated unidirectionally and bidirectionally coupled VAR processes, showing that rescaling may lead to insightful patterns of information storage and transfer but also to potentially misleading behaviors.

MuTE: A new Matlab toolbox for estimating the multivariate Transfer entropy in physiological variability series

We present a new time series analysis toolbox, developed in Matlab, for the estimation of the Transfer entropy (TE) between time series taken from a multivariate dataset. The main feature of the toolbox is its fully multivariate implementation, that is made possible by the design of an approach for the non-uniform embedding (NUE) of the observed time series. The toolbox is equipped with parametric (linear) and non-parametric (based on binning or nearest neighbors) entropy estimators. All these estimators, implemented using the NUE approach in comparison with the classical approach based on uniform embedding, are tested on RR interval, systolic pressure and respiration variability series measured from healthy subjects during head-up tilt. The results support the necessity of resorting to NUE for obtaining reliable estimates of the multivariate TE in short-term cardiovascular and cardiorespiratory variability.

Interictal cardiorespiratory variability in temporal lobe and absence epilepsy in childhood

It is well known that epilepsy has a profound effect on the autonomic nervous system, especially on the autonomic control of heart rate and respiration. This effect has been widely studied during seizure activity, but less attention has been given to interictal (i.e. seizure-free) activity. The studies that have been done on the latter, showed that heart rate and respiration can be affected individually, even without the occurrence of seizures. In this work, the interactions between these two individual physiological mechanisms are analysed during interictal activity in temporal lobe and absence epilepsy in childhood. These interactions are assessed by means of an entropy decomposition that allows to split the information carried by the heart rate, into two main components, one related to respiration and another related to different mechanisms. It is shown that in absence epilepsy the information shared by respiration and heart rate is significantly lower than for normal subjects.

Information dynamics in cardiorespiratory analyses: application to controlled breathing.

Voluntary adjustment of the breathing pattern is widely used to deal with stress-related conditions. In this study, effects of slow and fast breathing with a low and high inspiratory to expiratory time on heart rate variability (HRV) are evaluated by means of information dynamics. Information transfer is quantified both as the traditional transfer entropy as well as the cross entropy, where the latter does not condition on the past of HRV, thereby taking the highly unidirectional relation between respiration and heart rate into account. The results show that the cross entropy is more suited to quantify cardiorespiratory information transfer as this measure increases during slow breathing, indicating the increased cardiorespiratory coupling and suggesting the shift towards vagal activation during slow breathing. Additionally we found that controlled breathing, either slow or fast, results as well in an increase in cardiorespiratory coupling, compared to spontaneous breathing, which demonstrates the beneficial effects of instructed breathing.

Information dynamics in cardiorespiratory time series during mental stress testing

In this study, we assessed the information dynamics of respiration and heart rate variability during mental stress testing by means of the cross-entropy, a measure of cardiorespiratory coupling, and the self-entropy of the tachogram conditioned to the knowledge of respiration. Although stress is related to a reduction in vagal activity, no difference in cardiorespiratory coupling was found when 5 minutes of rest and stress were compared. The conditional self-entropy, on the other hand, showed significantly higher values during stress, indicating a higher predictability of the tachogram. These results show that entropy analyses of cardiorespiratory data reveal new information that could not be obtained with traditional heart rate variability studies.