Nicolangelo Iannella

Spike-Based Synaptic Plasticity in Silicon: Design, Implementation, Application, and Challenges

The ability to carry out signal processing, classification, recognition, and computation in artificial spiking neural networks (SNNs) is mediated by their synapses. In particular, through activity-dependent alteration of their efficacies, synapses play a fundamental role in learning. The mathematical prescriptions under which synapses modify their weights are termed synaptic plasticity rules. These learning rules can be based on abstract computational neuroscience models or on detailed biophysical ones. As these rules are being proposed and developed by experimental and computational neuroscientists, engineers strive to design and implement them in silicon and en masse in order to employ them in complex real-world applications. In this paper, we describe analog very large-scale integration (VLSI) circuit implementations of multiple synaptic plasticity rules, ranging from phenomenological ones (e.g., based on spike timing, mean firing rates, or both) to biophysically realistic ones (e.g., calcium-dependent models). We discuss the application domains, weaknesses, and strengths of various representative approaches proposed in the literature, and provide insight into the challenges that engineers face when designing and implementing synaptic plasticity rules in VLSI technology for utilizing them in real-world applications.

A spiking neural network architecture for nonlinear function approximation

Multilayer perceptrons have received much attention in recent years due to their universal approximation capabilities. Normally, such models use real valued continuous signals, although they are loosely based on biological neuronal networks that encode signals using spike trains. Spiking neural networks are of interest both from a biological point of view and in terms of a method of robust signaling in particularly noisy or difficult environments. It is important to consider networks based on spike trains. A basic question that needs to be considered however, is what type of architecture can be used to provide universal function approximation capabilities in spiking networks? In this paper, we propose a spiking neural network architecture using both integrate-and-fire units as well as delays, that is capable of approximating a real valued function mapping to within a specified degree of accuracy.

Synaptic efficacy cluster formation across the dendrite via STDP.

The role of spike-timing-dependent plasticity (STDP) in shaping the strength of a synapse located on the dendritic tree has gained recent interest. Previous theoretical studies using STDP have mostly used simplified integrate-and-fire models to investigate the evolution of synaptic efficacy with time. Such studies usually show that the final weight distribution is unimodal or bimodal resulting from a multiplicative or additive STDP rule, respectively. However, very little is known about how STDP shapes the spatial organization of synaptic efficacies. Here, for the first time, we demonstrate that spatial clustering of synaptic efficacies can occur on the dendrite via STDP, where changes in synaptic efficacy are driven by timing differences between synaptic inputs and the generation of local dendritic spikes. Specifically, when the model neuron is stimulated by two independent groups of correlated afferent inputs, the synaptic efficacies from each group, are not only spatially clustered on the dendrite but also spatially complementary to each other.

Spike Timing-Dependent Plasticity as the Origin of the Formation of Clustered Synaptic Efficacy Engrams

Synapse location, dendritic active properties and synaptic plasticity are all known to play some role in shaping the different input streams impinging onto a neuron. It remains unclear however, how the magnitude and spatial distribution of synaptic efficacies emerge from this interplay. Here, we investigate this interplay using a biophysically detailed neuron model of a reconstructed layer 2/3 pyramidal cell and spike timing-dependent plasticity (STDP). Specifically, we focus on the issue of how the efficacy of synapses contributed by different input streams are spatially represented in dendrites after STDP learning. We construct a simple feed forward network where a detailed model neuron receives synaptic inputs independently from multiple yet equally sized groups of afferent fibers with correlated activity, mimicking the spike activity from different neuronal populations encoding, for example, different sensory modalities. Interestingly, ensuing STDP learning, we observe that for all afferent groups, STDP leads to synaptic efficacies arranged into spatially segregated clusters effectively partitioning the dendritic tree. These segregated clusters possess a characteristic global organization in space, where they form a tessellation in which each group dominates mutually exclusive regions of the dendrite. Put simply, the dendritic imprint from different input streams left after STDP learning effectively forms what we term a “dendritic efficacy mosaic.” Furthermore, we show how variations of the inputs and STDP rule affect such an organization. Our model suggests that STDP may be an important mechanism for creating a clustered plasticity engram, which shapes how different input streams are spatially represented in dendrite.

Revisiting Special Relativity: A Natural Algebraic Alternative to Minkowski Spacetime

Minkowski famously introduced the concept of a space-time continuum in 1908, merging the three dimensions of space with an imaginary time dimension , with the unit imaginary producing the correct spacetime distance , and the results of Einstein’s then recently developed theory of special relativity, thus providing an explanation for Einstein’s theory in terms of the structure of space and time. As an alternative to a planar Minkowski space-time of two space dimensions and one time dimension, we replace the unit imaginary , with the Clifford bivector for the plane that also squares to minus one, but which can be included without the addition of an extra dimension, as it is an integral part of the real Cartesian plane with the orthonormal basis and . We find that with this model of planar spacetime, using a two-dimensional Clifford multivector, the spacetime metric and the Lorentz transformations follow immediately as properties of the algebra. This also leads to momentum and energy being represented as components of a multivector and we give a new efficient derivation of Compton’s scattering formula, and a simple formulation of Dirac’s and Maxwell’s equations. Based on the mathematical structure of the multivector, we produce a semi-classical model of massive particles, which can then be viewed as the origin of the Minkowski spacetime structure and thus a deeper explanation for relativistic effects. We also find a new perspective on the nature of time, which is now given a precise mathematical definition as the bivector of the plane.

Memristor-based synaptic networks and logical operations using in-situ computing

We present new computational building blocks based on memristive devices. These blocks, can be used to implement either supervised or unsupervised learning modules. This is achieved using a crosspoint architecture which is an efficient array implementation for nanoscale two-terminal mem-ristive devices. Based on these blocks and an experimentally verified SPICE macromodel for the memristor, we demonstrate that firstly, the Spike-Timing-Dependent Plasticity (STDP) can be implemented by a single memristor device and secondly, a memristor-based competitive Hebbian learning through STDP using a 1×1000 synaptic network. This is achieved by adjusting the memristors conductance values (weights) as a function of the timing difference between presynaptic and postsynaptic spikes. These implementations have a number of shortcomings due to the memristors characteristics such as memory decay, highly nonlinear switching behaviour as a function of applied voltage/current, and functional uniformity. These shortcomings can be addressed by utilising a mixed gates that can be used in conjunction with the analogue behaviour for biomimetic computation. The digital implementations in this paper use in-situ computational capability of the memristor.

Novel VLSI implementation for triplet-based spike-timing dependent plasticity

Spike Timing-Dependent Plasticity (STDP) is one of several plasticity rules that is believed to play an important role in learning and memory in the brain. In conventional pair-based STDP learning, synaptic weights are altered by utilizing the temporal difference between pairs of pre- and post-synaptic spikes. This learning rule, however, fails to reproduce reported experimental measurements when using stimuli either by patterns consisting of triplet or quadruplet of spikes or increasing the repetition frequency of pairs of spikes. Significantly, a previously described spike triplet-based STDP rule succeeds in reproducing all of these experimental observations. In this paper, we present a new spike triplet-based VLSI implementation, that is based on a previous pair-based STDP circuit [1]. This implementation can reproduce similar results to those observed in various physiological STDP experiments, in contrast to traditional pair-based VLSI implementation. Simulation results using standard 0.35 µm CMOS process of the new circuit are presented and compared to published experimental data [2].

Tunable low energy, compact and high performance neuromorphic circuit for spike-based synaptic plasticity

Cortical circuits in the brain have long been recognised for their information processing capabilities and have been studied both experimentally and theoretically via spiking neural networks. Neuromorphic engineers are primarily concerned with translating the computational capabilities of biological cortical circuits, using the Spiking Neural Network (SNN) paradigm, into in silico applications that can mimic the behaviour and capabilities of real biological circuits/systems. These capabilities include low power consumption, compactness, and relevant dynamics. In this paper, we propose a new accelerated-time circuit that has several advantages over its previous neuromorphic counterparts in terms of compactness, power consumption, and capability to mimic the outcomes of biological experiments. The presented circuit simulation results demonstrate that, in comparing the new circuit to previous published synaptic plasticity circuits, reduced silicon area and lower energy consumption for processing each spike is achieved. In addition, it can be tuned in order to closely mimic the outcomes of various spike timing- and rate-based synaptic plasticity experiments. The proposed circuit is also investigated and compared to other designs in terms of tolerance to mismatch and process variation. Monte Carlo simulation results show that the proposed design is much more stable than its previous counterparts in terms of vulnerability to transistor mismatch, which is a significant challenge in analog neuromorphic design. All these features make the proposed design an ideal circuit for use in large scale SNNs, which aim at implementing neuromorphic systems with an inherent capability that can adapt to a continuously changing environment, thus leading to systems with significant learning and computational abilities.

Efficient design of triplet based Spike-Timing Dependent Plasticity

Spike-Timing Dependent Plasticity (STDP) is believed to play an important role in learning and the formation of computational function in the brain. The classical model of STDP which considers the timing between pairs of pre-synaptic and post-synaptic spikes (p-STDP) is incapable of reproducing synaptic weight changes similar to those seen in biological experiments which investigate the effect of either higher order spike trains (e.g. triplet and quadruplet of spikes) [1]-[3], or, simultaneous effect of the rate and timing of spike pairs [4] on synaptic plasticity. In this paper, we firstly investigate synaptic weight changes using a p-STDP circuit [5] and show how it fails to reproduce the mentioned complex biological experiments. We then present a new STDP VLSI circuit which acts based on the timing among triplets of spikes (t-STDP) that is able to reproduce all the mentioned experimental results. We believe that our new STDP VLSI circuit improves upon previous circuits, whose learning capacity exceeds current designs due to its capability of mimicking the outcomes of biological experiments more closely; thus plays a significant role in future VLSI implementation of neuromorphic systems.

A new compact analog VLSI model for Spike Timing Dependent Plasticity

Spike Timing Dependent Plasticity (STDP) is a time-based synaptic plasticity rule that has generated significant interest in the area of neuromorphic engineering and Very Large Scale Integration (VLSI) circuit design. During the last decade, STDP and STDP-like learning mechanisms have shown promising solutions for various real world applications, ranging from pattern recognition to robotics. This paper presents a novel analog VLSI model for STDP that possesses advantages compared to previously published VLSI STDP designs. The presented STDP circuit is capable of reproducing the outcomes of several well known experiments using various plasticity rules inducing STDP protocols that utilise pairs, triplets, and quadruplets of spike patterns. When the circuit is compared to state-of-the-art VLSI STDP circuits, it shows a compact and symmetric design that makes the proposed circuit a powerful component for use in designing STDP or time-based Hebbian learning experiments and applications.