Rick Quax

The diminishing role of hubs in dynamical processes on complex networks.

It is notoriously difficult to predict the behaviour of a complex self-organizing system, where the interactions among dynamical units form a heterogeneous topology. Even if the dynamics of each microscopic unit is known, a real understanding of their contributions to the macroscopic system behaviour is still lacking. Here, we develop information-theoretical methods to distinguish the contribution of each individual unit to the collective out-of-equilibrium dynamics. We show that for a system of units connected by a network of interaction potentials with an arbitrary degree distribution, highly connected units have less impact on the system dynamics when compared with intermediately connected units. In an equilibrium setting, the hubs are often found to dictate the long-term behaviour. However, we find both analytically and experimentally that the instantaneous states of these units have a short-lasting effect on the state trajectory of the entire system. We present qualitative evidence of this phenomenon from empirical findings about a social network of product recommendations, a protein–protein interaction network and a neural network, suggesting that it might indeed be a widespread property in nature.

Increasing risk behaviour can outweigh the benefits of antiretroviral drug treatment on the HIV incidence among men-having-sex-with-men in Amsterdam

BackgroundThe transmission through contacts among MSM (men who have sex with men) is one of the dominating contributors to HIV prevalence in industrialized countries. In Amsterdam, the capital of the Netherlands, the MSM risk group has been traced for decades. This has motivated studies which provide detailed information about MSMs risk behavior statistically, psychologically and sociologically. Despite the era of potent antiretroviral therapy, the incidence of HIV among MSM increases. In the long term the contradictory effects of risk behavior and effective therapy are still poorly understood.MethodsUsing a previously presented Complex Agent Network model, we describe steady and casual partnerships to predict the HIV spreading among MSM. Behavior-related parameters and values, inferred from studies on Amsterdam MSM, are fed into the model; we validate the model using historical yearly incidence data. Subsequently, we study scenarios to assess the contradictory effects of risk behavior and effective therapy, by varying corresponding values of parameters. Finally, we conduct quantitative analysis based on the resulting incidence data.ResultsThe simulated incidence reproduces the ACS historical incidence well and helps to predict the HIV epidemic among MSM in Amsterdam. Our results show that in the long run the positive influence of effective therapy can be outweighed by an increase in risk behavior of at least 30% for MSM.ConclusionWe recommend, based on the model predictions, that lowering risk behavior is the prominent control mechanism of HIV incidence even in the presence of effective therapy.

Quantifying Synergistic Information Using Intermediate Stochastic Variables

Quantifying synergy among stochastic variables is an important open problem in information theory. Information synergy occurs when multiple sources together predict an outcome variable better than the sum of single-source predictions. It is an essential phenomenon in biology such as in neuronal networks and cellular regulatory processes, where different information flows integrate to produce a single response, but also in social cooperation processes as well as in statistical inference tasks in machine learning. Here we propose a metric of synergistic entropy and synergistic information from first principles. The proposed measure relies on so-called synergistic random variables (SRVs) which are constructed to have zero mutual information about individual source variables but non-zero mutual information about the complete set of source variables. We prove several basic and desired properties of our measure, including bounds and additivity properties. In addition, we prove several important consequences of our measure, including the fact that different types of synergistic information may co-exist between the same sets of variables. A numerical implementation is provided, which we use to demonstrate that synergy is associated with resilience to noise. Our measure may be a marked step forward in the study of multivariate information theory and its numerous applications.

Nonparametric estimation of Fisher information from real data

The Fisher information matrix (FIM) is a widely used measure for applications including statistical inference, information geometry, experiment design, and the study of criticality in biological systems. The FIM is defined for a parametric family of probability distributions and its estimation from data follows one of two paths: either the distribution is assumed to be known and the parameters are estimated from the data or the parameters are known and the distribution is estimated from the data. We consider the latter case which is applicable, for example, to experiments where the parameters are controlled by the experimenter and a complicated relation exists between the input parameters and the resulting distribution of the data. Since we assume that the distribution is unknown, we use a nonparametric density estimation on the data and then compute the FIM directly from that estimate using a finite-difference approximation to estimate the derivatives in its definition. The accuracy of the estimate depends on both the method of nonparametric estimation and the difference Δθ between the densities used in the finite-difference formula. We develop an approach for choosing the optimal parameter difference Δθ based on large deviations theory and compare two nonparametric density estimation methods, the Gaussian kernel density estimator and a novel density estimation using field theory method. We also compare these two methods to a recently published approach that circumvents the need for density estimation by estimating a nonparametric f divergence and using it to approximate the FIM. We use the Fisher information of the normal distribution to validate our method and as a more involved example we compute the temperature component of the FIM in the two-dimensional Ising model and show that it obeys the expected relation to the heat capacity and therefore peaks at the phase transition at the correct critical temperature.

Applying Information Theory to Neuronal Networks: From Theory to Experiments

Information-theory is being increasingly used to analyze complex, self-organizing processes on networks, predominantly in analytical and numerical studies. Perhaps one of the most paradigmatic complex systems is a network of neurons, in which cognition arises from the information storage, transfer, and processing among individual neurons. In this article we review experimental techniques suitable for validating information-theoretical predictions in simple neural networks, as well as generating new hypotheses. Specifically, we focus on techniques that may be used to measure both network (microcircuit) anatomy as well as neuronal activity simultaneously. This is needed to study the role of the network structure on the emergent collective dynamics, which is one of the reasons to study the characteristics of information processing. We discuss in detail two suitable techniques, namely calcium imaging and the application of multi-electrode arrays to simple neural networks in culture, and discuss their advantages and limitations in an accessible manner for non-experts. In particular, we show that each technique induces a qualitatively different type of error on the measured mutual information. The ultimate goal of this work is to bridge the gap between theorists and experimentalists in their shared goal of understanding the behavior of networks of neurons.

Simulating Individual-Based Models of Epidemics in Hierarchical Networks

Current mathematical modeling methods for the spreading of infectious diseases are too simplified and do not scale well. We present the Simulator of Epidemic Evolution in Complex Networks (SEECN), an efficient simulator of detailed individual-based models by parameterizing separate dynamics operators, which are iteratively applied to the contact network. We reduce the network generators computational complexity, improve cache efficiency and parallelize the simulator. To evaluate its running time we experiment with an HIV epidemic model that incorporates up to one million homosexual men in a scale-free network, including hierarchical community structure, social dynamics and multi-stage intranode progression. We find that the running times are feasible, on the order of minutes, and argue that SEECN can be used to study realistic epidemics and its properties experimentally, in contrast to defining and solving ever more complicated mathematical models as is the current practice.

Topology dependent epidemic spreading velocity in weighted networks

Many diffusive processes occur on structured networks with weighted links, such as disease spread by airplane transport or information diffusion in social networks or blogs. Understanding the impact of weight-connectivity correlations on epidemic spreading in weighted networks is crucial to support decision-making on disease control and other diffusive processes. However, a real understanding of epidemic spreading velocity in weighted networks is still lacking. Here we conduct a numerical study of the velocity of a Reed–Frost epidemic spreading process in various weighted network topologies as a function of the correlations between edge weights and node degrees. We find that a positive weight-connectivity correlation leads to a faster epidemic spreading compared to an unweighted network. In contrast, we find that both uncorrelated and negatively correlated weight distributions lead to slower spreading processes. In the case of positive weight-connectivity correlations, the acceleration of spreading velocity is weak when the heterogeneity of weight distribution increases.

Anomaly Detection in Clinical Data of Patients Undergoing Heart Surgery

Abstract We describe two approaches to detecting anomalies in time series of multi-parameter clinical data: (1) metric and model-based indicators and (2) information surprise. (1) Metric and model-based indicators are commonly used as early warning signals to detect transitions between alternate states based on individual time series. Here we explore the applicability of existing indicators to distinguish critical (anomalies) from non-critical conditions in patients undergoing cardiac surgery, based on a small anonymized clinical trial dataset. We find that a combination of time-varying autoregressive model, kurtosis, and skewness indicators correctly distinguished critical from non-critical patients in 5 out of 36 blood parameters at a window size of 0.3 (average of 37 hours) or higher. (2) Information surprise quantifies how the progression of one patient’s condition differs from that of rest of the population based on the cross-section of time series. With the maximum surprise and slope features we detect all critical patients at the 0.05 significance level. Moreover we show that a naive outlier detection does not work, demonstrating the need for the more sophisticated approaches explored here. Our preliminary results suggest that future developments in early warning systems for patient condition monitoring may predict the onset of critical transition and allow medical intervention preventing patient death. Further method development is needed to avoid overfitting and spurious results, and verification on large clinical datasets.

Information geometric analysis of phase transitions in complex patterns: the case of the Gray-Scott reaction-diffusion model

The Fisher–Rao metric from information geometry is related to phase transition phenomena in classical statistical mechanics. Several studies propose to extend the use of information geometry to study more general phase transitions in complex systems. However, it is unclear whether the Fisher–Rao metric does indeed detect these more general transitions, especially in the absence of a statistical model. In this paper we study the transitions between patterns in the Gray-Scott reaction–diffusion model using Fisher information. We describe the system by a probability density function that represents the size distribution of blobs in the patterns and compute its Fisher information with respect to changing the two rate parameters of the underlying model. We estimate the distribution non-parametrically so that we do not assume any statistical model. The resulting Fisher map can be interpreted as a phase-map of the different patterns. Lines with high Fisher information can be considered as boundaries between regions of parameter space where patterns with similar characteristics appear. These lines of high Fisher information can be interpreted as phase transitions between complex patterns.

A study on the dynamics of temporary HIV treatment to assess the controversial outcomes of clinical trials: An in-silico approach

It is still unclear under which conditions temporary combined antiretroviral therapy (cART) results in a prolonged remission after interruption. Clinical trials have contradicting reposts about the effect of cART during primary HIV infection on the disease progression. Here we propose that the apparent contradiction is due the presence of a window of opportunity for cART treatment observed in the in silico studies. We study non-linear correlations in the HIV dynamics over time using information theory. This approach requires a large dataset of CD4+ T lymphocytes and viral load concentrations over time. Since it is unfeasible to collect the required amount of data in clinical trials we use C-ImmSim, a clinically validated in silico model of the HIV infection, to simulate the HIV infection and temporary cART in 500 virtual patients for a period of 6 years post infection in time steps of 8 hours. We validate the results of our model with two published clinical trials of temporary cART in acute infection and analyse the impact of cART on the immune response. Our quantitative analysis predicts a “window of opportunity” of about ten months after the acute phase during which a temporary cART has significantly longer-lasting beneficial effects on the immune system as compared to treatment during the chronic phase. This window may help to explain the controversial outcomes of clinical trials that differ by the starting time and duration of the short-term course cART and provides a critical insight to develop appropriate protocols for future clinical trials.