Gilles Wainrib

Topological and dynamical complexity of random neural networks.

Random neural networks are dynamical descriptions of randomly interconnected neural units. These show a phase transition to chaos as a disorder parameter is increased. The microscopic mechanisms underlying this phase transition are unknown and, similar to spin glasses, shall be fundamentally related to the behavior of the system. In this Letter, we investigate the explosion of complexity arising near that phase transition. We show that the mean number of equilibria undergoes a sharp transition from one equilibrium to a very large number scaling exponentially with the dimension on the system. Near criticality, we compute the exponential rate of divergence, called topological complexity. Strikingly, we show that it behaves exactly as the maximal Lyapunov exponent, a classical measure of dynamical complexity. This relationship unravels a microscopic mechanism leading to chaos which we further demonstrate on a simpler disordered system, suggesting a deep and underexplored link between topological and dynamical complexity.

A local Echo State Property through the largest Lyapunov exponent

Echo State Networks are efficient time-series predictors, which highly depend on the value of the spectral radius of the reservoir connectivity matrix. Based on recent results on the mean field theory of driven random recurrent neural networks, enabling the computation of the largest Lyapunov exponent of an ESN, we develop a cheap algorithm to establish a local and operational version of the Echo State Property.

A biological gradient descent for prediction through a combination of stdp and homeostatic plasticity

Identifying, formalizing, and combining biological mechanisms that implement known brain functions, such as prediction, is a main aspect of research in theoretical neuroscience. In this letter, the mechanisms of spike-timing-dependent plasticity and homeostatic plasticity, combined in an original mathematical formalism, are shown to shape recurrent neural networks into predictors. Following a rigorous mathematical treatment, we prove that they implement the online gradient descent of a distance between the network activity and its stimuli. The convergence to an equilibrium, where the network can spontaneously reproduce or predict its stimuli, does not suffer from bifurcation issues usually encountered in learning in recurrent neural networks.

A Deep Learning Architecture for Temporal Sleep Stage Classification Using Multivariate and Multimodal Time Series

Sleep stage classification constitutes an important preliminary exam in the diagnosis of sleep disorders. It is traditionally performed by a sleep expert who assigns to each 30 s of the signal of a sleep stage, based on the visual inspection of signals such as electroencephalograms (EEGs), electrooculograms (EOGs), electrocardiograms, and electromyograms (EMGs). We introduce here the first deep learning approach for sleep stage classification that learns end-to-end without computing spectrograms or extracting handcrafted features, that exploits all multivariate and multimodal polysomnography (PSG) signals (EEG, EMG, and EOG), and that can exploit the temporal context of each 30-s window of data. For each modality, the first layer learns linear spatial filters that exploit the array of sensors to increase the signal-to-noise ratio, and the last layer feeds the learnt representation to a softmax classifier. Our model is compared to alternative automatic approaches based on convolutional networks or decisions trees. Results obtained on 61 publicly available PSG records with up to 20 EEG channels demonstrate that our network architecture yields the state-of-the-art performance. Our study reveals a number of insights on the spatiotemporal distribution of the signal of interest: a good tradeoff for optimal classification performance measured with balanced accuracy is to use 6 EEG with 2 EOG (left and right) and 3 EMG chin channels. Also exploiting 1 min of data before and after each data segment offers the strongest improvement when a limited number of channels are available. As sleep experts, our system exploits the multivariate and multimodal nature of PSG signals in order to deliver the state-of-the-art classification performance with a small computational cost.

Regular graphs maximize the variability of random neural networks.

In this work we study the dynamics of systems composed of numerous interacting elements interconnected through a random weighted directed graph, such as models of random neural networks. We develop an original theoretical approach based on a combination of a classical mean-field theory originally developed in the context of dynamical spin-glass models, and the heterogeneous mean-field theory developed to study epidemic propagation on graphs. Our main result is that, surprisingly, increasing the variance of the in-degree distribution does not result in a more variable dynamical behavior, but on the contrary that the most variable behaviors are obtained in the regular graph setting. We further study how the dynamical complexity of the attractors is influenced by the statistical properties of the in-degree distribution.

Distributed synaptic weights in a LIF neural network and learning rules

Leaky integrate-and-fire (LIF) models are mean-field limits, with a large number of neurons, used to describe neural networks. We consider inhomogeneous networks structured by a connec-tivity parameter (strengths of the synaptic weights) with the effect of processing the input current with different intensities. We first study the properties of the network activity depending on the distribution of synaptic weights and in particular its discrimination capacity. Then, we consider simple learning rules and determine the synaptic weight distribution it generates. We outline the role of noise as a selection principle and the capacity to memorized a learned signal.

Mathematical modeling of lymphocytes selection in the germinal center

Lymphocyte selection is a fundamental process of adaptive immunity. In order to produce B-lymphocytes with a target antigenic profile, mutation selection and division occur in the germinal center, a specific part of lymph nodes. We introduce in this article a simplified mathematical model of this phenomenon, taking into account the main mechanisms. This model is written as a non-linear, non-local, inhomogeneous second order partial differential equation, for which we develop a mathematical analysis. We assess, mathematically and numerically, in the case of piecewise-constant coefficients, the performance of the biological function by evaluating the duration of this production process as a function of several parameters such as the mutation rate or the selection profile, in various asymptotic regimes.

Training performance of echo state neural networks

This article proposes a first theoretical performance analysis of the training phase of large dimensional linear echo-state networks. This analysis is based on advanced methods of random matrix theory. The results provide some new insights on the core features of such networks, thereby helping the practitioner when using them.

Perspectives en matrices aléatoires et grands réseaux

Dans cet article, de nouvelles perspectives de recherche en matrices aleatoires appliquees a la theorie des graphes sont introduites. Nous nous attachons en particulier a l’analyse spectrale des matrices d’adjacence et laplaciennes de graphes de grandes dimensions pour la detection de communautes dans les reseaux, des matrices aleatoires a noyaux pour la classification non supervisee en big data, ainsi qu’a des applications en reseaux de neurones.

Network Modeling of Crohn's Disease Incidence.

Background Numerous genetic and environmental risk factors play a role in human complex genetic disorders (CGD). However, their complex interplay remains to be modelled and explained in terms of disease mechanisms. Methods and findings Crohns Disease (CD) was modeled as a modular network of patho-physiological functions, each summarizing multiple gene-gene and gene-environment interactions. The disease resulted from one or few specific combinations of module functional states. Network aging dynamics was able to reproduce age-specific CD incidence curves as well as their variations over the past century in Western countries. Within the model, we translated the odds ratios (OR) associated to at-risk alleles in terms of disease propensities of the functional modules. Finally, the model was successfully applied to other CGD including ulcerative colitis, ankylosing spondylitis, multiple sclerosis and schizophrenia. Conclusion Modeling disease incidence may help to understand disease causative chains, to delineate the potential of personalized medicine, and to monitor epidemiological changes in CGD.