Pengsheng Zheng

Network Self-Organization Explains the Statistics and Dynamics of Synaptic Connection Strengths in Cortex

The information processing abilities of neural circuits arise from their synaptic connection patterns. Understanding the laws governing these connectivity patterns is essential for understanding brain function. The overall distribution of synaptic strengths of local excitatory connections in cortex and hippocampus is long-tailed, exhibiting a small number of synaptic connections of very large efficacy. At the same time, new synaptic connections are constantly being created and individual synaptic connection strengths show substantial fluctuations across time. It remains unclear through what mechanisms these properties of neural circuits arise and how they contribute to learning and memory. In this study we show that fundamental characteristics of excitatory synaptic connections in cortex and hippocampus can be explained as a consequence of self-organization in a recurrent network combining spike-timing-dependent plasticity (STDP), structural plasticity and different forms of homeostatic plasticity. In the network, associative synaptic plasticity in the form of STDP induces a rich-get-richer dynamics among synapses, while homeostatic mechanisms induce competition. Under distinctly different initial conditions, the ensuing self-organization produces long-tailed synaptic strength distributions matching experimental findings. We show that this self-organization can take place with a purely additive STDP mechanism and that multiplicative weight dynamics emerge as a consequence of network interactions. The observed patterns of fluctuation of synaptic strengths, including elimination and generation of synaptic connections and long-term persistence of strong connections, are consistent with the dynamics of dendritic spines found in rat hippocampus. Beyond this, the model predicts an approximately power-law scaling of the lifetimes of newly established synaptic connection strengths during development. Our results suggest that the combined action of multiple forms of neuronal plasticity plays an essential role in the formation and maintenance of cortical circuits.

Letters: Some novel double-scroll chaotic attractors in Hopfield networks

In this paper, a 3-neuron chaotic Hopfield network is presented. Numerical simulations show that the network displays rich dynamics by changing the self-connection weight. The dynamic of the network is studied by using the Lyapunov exponents spectrum, bifurcation diagram, power spectrum and topological horseshoe theory. It is shown that the proposed network exhibits some novel double-scroll chaotic attractors.

Robust development of synfire chains from multiple plasticity mechanisms.

Biological neural networks are shaped by a large number of plasticity mechanisms operating at different time scales. How these mechanisms work together to sculpt such networks into effective information processing circuits is still poorly understood. Here we study the spontaneous development of synfire chains in a self-organizing recurrent neural network (SORN) model that combines a number of different plasticity mechanisms including spike-timing-dependent plasticity, structural plasticity, as well as homeostatic forms of plasticity. We find that the network develops an abundance of feed-forward motifs giving rise to synfire chains. The chains develop into ring-like structures, which we refer to as “synfire rings.” These rings emerge spontaneously in the SORN network and allow for stable propagation of activity on a fast time scale. A single network can contain multiple non-overlapping rings suppressing each other. On a slower time scale activity switches from one synfire ring to another maintaining firing rate homeostasis. Overall, our results show how the interaction of multiple plasticity mechanisms might give rise to the robust formation of synfire chains in biological neural networks.

Efficient continuous-time asymmetric hopfield networks for memory retrieval

A novel m energy functions method is adopted to analyze the retrieval property of continuous-time asymmetric Hopfield neural networks. Sufficient conditions for the local and global asymptotic stability of the network are proposed. Moreover, an efficient systematic procedure for designing asymmetric networks is proposed, and a given set of states can be assigned as locally asymptotically stable equilibrium points. Simulation examples show that the asymmetric network can act as an efficient associative memory, and it is almost free from spurious memory problem.

Learning Associative Memories by Error Backpropagation

In this paper, a method for the design of Hopfield networks, bidirectional and multidirectional associative memories with asymmetric connections, is proposed. The given patterns can be assigned as locally asymptotically stable equilibria of the network by training a single-layer feedforward network. It is shown that the robustness in respect to acceptable noise in the input of the constructed networks is enhanced as the memory dimension increases and weakened as the number of the stored patterns grows. More important is that the remembered patterns are not necessarily of binary forms. Neural associative memories for storing gray-level images are constructed based on the proposed method. Numerical simulations show that the proposed method is efficient for the design of Hopfield-type recurrent neural networks.

Color image associative memory on a class of Cohen-Grossberg networks

In this paper, neural associative memories for storing gray-scale and true color images are presented based on a class of reduced Cohen-Grossberg neural networks. Some fundamental conditions for endowing the networks with retrieval properties are proposed. Moreover, a system designing procedure is developed by using matrix decomposition. Numerical simulations show that the constructed networks can act as reliable noise-reducing systems for storing and retrieving color images.

A simple method for designing efficient small-world neural networks.

Small-world neural networks, as well as diluted Hopfield networks, are constructed by using matrix decomposition and a connection elimination strategy. It is shown that, to a certain extent, eliminating the unimportant synaptic couplings does not degrade the network performance. Numerical simulations give strong evidence that the small-world and diluted neural networks, by consuming a small fraction of connections, can perform as well as full-connected ones. The proposed method is simple but efficient and also potentially significant for the applications of neural circuits.

Threshold Complex-Valued Neural Associative Memory

In this brief, threshold complex-valued neural associative memory is proposed for information retrieval. The introduction of threshold improves network performance by excluding rotated patterns from spurious memories. A design method for constructing different types of network is developed based on complex matrix decomposition, which is capable of designing nonthreshold, threshold, non-Hermitian, and Hermitian networks. Further, we illustrate the performance of the proposed method by reconstructing noisy 256 grayscale and true color images. The results show that constructed networks can work efficiently, threshold networks have better performance than nonthreshold ones and networks with small asymmetry in weight matrix function as well as Hermitian ones.

Analysis and design of asymmetric Hopfield networks with discrete-time dynamics

The retrieval properties of the asymmetric Hopfield neural networks (AHNNs) with discrete-time dynamics are studied in this paper. It is shown that the asymmetry degree is an important factor influencing the network dynamics. Furthermore, a strategy for designing AHNNs of different sparsities is proposed. Numerical simulations show that AHNNs can perform as well as symmetric ones, and the diluted AHNNs have the virtues of small wiring cost and high pattern recognition quality.

Generalization ability analysis of one-dimensional wavelet neural network by simulations

In this paper, the generalization ability of one-dimensional wavelet neural network (I-DWNN) was discussed from four aspects which were training sample quality, network complexity, resembled over-fitting and extrapolation fitting. Simulations of the same problem with same training time showed that the over-fitting probability of the wavelet network was much bigger than multilayer perceptron (MLP) and radial basis function (RBF) network. Simulations of one problem with different network complexities showed that the network complexity had little impact on the generalization ability. Resembled over-fitting was discovered by simulations which debased the network generalization ability. To improve the network generalization ability, training method for high-noisy samples was discussed, wavelon-elimination algorithm dealing with resembled over-fitting, training method for the extrapolation fitting and other useful suggestions were proposed.