Claude Berrou

Sparse Neural Networks With Large Learning Diversity

Coded recurrent neural networks with three levels of sparsity are introduced. The first level is related to the size of messages that are much smaller than the number of available neurons. The second one is provided by a particular coding rule, acting as a local constraint in the neural activity. The third one is a characteristic of the low final connection density of the network after the learning phase. Though the proposed network is very simple since it is based on binary neurons and binary connections, it is able to learn a large number of messages and recall them, even in presence of strong erasures. The performance of the network is assessed as a classifier and as an associative memory.

EEG source connectivity analysis: from dense array recordings to brain networks.

The recent past years have seen a noticeable increase of interest for electroencephalography (EEG) to analyze functional connectivity through brain sources reconstructed from scalp signals. Although considerable advances have been done both on the recording and analysis of EEG signals, a number of methodological questions are still open regarding the optimal way to process the data in order to identify brain networks. In this paper, we analyze the impact of three factors that intervene in this processing: i) the number of scalp electrodes, ii) the combination between the algorithm used to solve the EEG inverse problem and the algorithm used to measure the functional connectivity and iii) the frequency bands retained to estimate the functional connectivity among neocortical sources. Using High-Resolution (hr) EEG recordings in healthy volunteers, we evaluated these factors on evoked responses during picture recognition and naming task. The main reason for selection this task is that a solid literature background is available about involved brain networks (ground truth). From this a priori information, we propose a performance criterion based on the number of connections identified in the regions of interest (ROI) that belong to potentially activated networks. Our results show that the three studied factors have a dramatic impact on the final result (the identified network in the source space) as strong discrepancies were evidenced depending on the methods used. They also suggest that the combination of weighted Minimum Norm Estimator (wMNE) and the Phase Synchronization (PS) methods applied on High-Resolution EEG in beta/gamma bands provides the best performance in term of topological distance between the identified network and the expected network in the above-mentioned cognitive task.

Adding a Rate-1 Third Dimension to Turbo Codes

Thanks to the message passing principle, turbo decoding is able to provide strong error correction near the theoretical (Shannon) limit. However, the minimum Hamming distance (MHD) of a turbo code may not be sufficient to prevent a detrimental change in the error rate vs. signal to noise ratio curve, the so-called flattening. Increasing the MHD of a turbo code may involve using component encoders with a large number of states, devising more sophisticated internal permutations, or increasing the dimension of the turbo code, i.e. the number of component encoders. This paper addresses the latter option and proposes a modified turbo code, in which some of the parity bits stemming from the classical component encoders are encoded by a rate-1, third encoder. The result is a significantly increased MHD, which improves turbo decoder performance at low error rates, at the expense of a very small increase in complexity. In this paper, we compare the performance of the proposed turbo code with that of the DVB-RCS turbo code and the DVB-S2 LDPC code. Comparisons with more complex 16-state TCs are also reported.

Storing Sparse Messages in Networks of Neural Cliques

An extension to a recently introduced binary neural network is proposed to allow the storage of sparse messages, in large numbers and with high memory efficiency. This new network is justified both in biological and informational terms. The storage and retrieval rules are detailed and illustrated by various simulation results.

Nearly-optimal associative memories based on distributed constant weight codes

A new family of sparse neural networks achieving nearly optimal performance has been recently introduced. In these networks, messages are stored as cliques in clustered graphs. In this paper, we interpret these networks using the formalism of error correcting codes. To achieve this, we introduce two original codes, the thrifty code and the clique code, that are both sub-families of binary constant weight codes. We also provide the networks with an enhanced retrieving rule that enables a property of answer correctness and that improves performance.

A new algorithm for spatiotemporal analysis of brain functional connectivity.

Specific networks of interacting neuronal assemblies distributed within and across distinct brain regions underlie brain functions. In most cognitive tasks, these interactions are dynamic and take place at the millisecond time scale. Among neuroimaging techniques, magneto/electroencephalography - M/EEG - allows for detection of very short-duration events and offers the single opportunity to follow, in time, the dynamic properties of cognitive processes (sub-millisecond temporal resolution). In this paper, we propose a new algorithm to track the functional brain connectivity dynamics. During a picture naming task, this algorithm aims at segmenting high-resolution EEG signals (hr-EEG) into functional connectivity microstates. The proposed algorithm is based on the K-means clustering of the connectivity graphs obtained from the phase locking value (PLV) method applied on hr-EEG. Results show that the analyzed evoked responses can be divided into six clusters representing distinct networks sequentially involved during the cognitive task, from the picture presentation and recognition to the motor response.

Dynamic reorganization of functional brain networks during picture naming.

For efficient information processing during cognitive activity, functional brain networks have to rapidly and dynamically reorganize on a sub-second time scale. Tracking the spatiotemporal dynamics of large scale networks over this short time duration is a very challenging issue. Here, we tackle this problem by using dense electroencephalography (EEG) recorded during a picture naming task. We found that (i) the picture naming task can be divided into six brain network states (BNSs) characterized by significantly high synchronization of gamma (30-45 Hz) oscillations, (ii) fast transitions occur between these BNSs that last from 30 msec to 160 msec, (iii) based on the state of the art of the picture naming task, we consider that the spatial location of their nodes and edges, as well as the timing of transitions, indicate that each network can be associated with one or several specific function (from visual processing to articulation) and (iv) the comparison with previously-used approach aimed at localizing the sources showed that the network-based approach reveals networks that are more specific to the performed task. We speculate that the persistence of several brain regions in successive BNSs participates to fast and efficient information processing in the brain.

A simple and efficient way to store many messages using neural cliques

Associative memories are devices that are able to learn messages and to recall them in presence of errors or erasures. Their mechanics is similar to that of error correcting decoders. However, the role of correlation is opposed in the two devices, used as the essence of the retrieval process in the first one and avoided in the latter. In this paper, original codes are introduced to allow the effective combination of the two domains. The main idea is to associate a clique in a binary neural network with each message to learn. The obtained performance is dramatically better than that given by the state of the art, for instance Hopfield Neural Networks. Moreover, the model proposed is biologically plausible; it uses sparse binary connections between clusters of neurons provided with only two operations: sum and selection of maximum.

Coded Hopfield networks

Error-correcting coding is introduced in associative memories based on Hopfield networks in order to increase the learning diversity as well as the recall robustness in presence of erasures and errors. To achieve this, the graph associated with the classical Hopfield network is transformed into a bipartite graph in which incoming information is linked to orthogonal or quasi-orthogonal codes. Whereas learning is similar to that of classical (i.e. Hebbian) Hopfield networks, memory retrieval relies on error correction decoding which offers strong discrimination properties between the memorized patterns.

Hamming distance spectra of turbo-codes

This paper is intended to predict the performance of turbo-codes by analytical means. After a brief description of turbo-codes, the concept of basic return-to-zero (rtz) sequences is introduced. Then, it is shown how rtz sequences can be used to compute the Hamming distance spectrum (uds) via a modified version of the Fano algorithm. The hds thus obtained is used to compute an upper bound on the bit error rate. Distance spectra of selected turbo-codes with short interleaving are presented and the validity of their performance prediction is confirmed by simulation (only provisional results are given here). Besides, a class of low-performance turbocodes, referred to as weak, is identified.RésuméCet article est consacré à la prévision des performances des turbo-codes par des moyens analytiques. Après une brève description des turbo-codes, le concept de suite de retour à zéro minimale est introduit. On montre comment employer les suites de retour à zéro pour calculer le spectre de distance de Hamming à ľaide ďune version modifiée de ľalgorithme de Fano. Le spectre de distance ainsi obtenu sert à calculer une borne supérieure de la probabilité ďerreur par bit. Les spectres des distances de quelques turbo-codes avec entrelacement de petite taille sont présentés et la validité des prévisions basées sur eux est confirmée par simulation (les résultats donnés ici ne sont que provisoires). Par ailleurs, une sous-famille de turbo-codes dits faibles, dont les performances sont mauvaises, est identifiée.