On the Self-Repair Role of Astrocytes in STDP Enabled Unsupervised SNNs
11 On the Self-Repair Role of Astrocytes in STDPEnabled Unsupervised SNNs
Mehul Rastogi, Sen Lu, Nafiul Islam, Abhronil Sengupta
Abstract —Neuromorphic computing is emerging to be a dis-ruptive computational paradigm that attempts to emulate variousfacets of the underlying structure and functionalities of the brainin the algorithm and hardware design of next-generation machinelearning platforms. This work goes beyond the focus of currentneuromorphic computing architectures on computational modelsfor neuron and synapse to examine other computational unitsof the biological brain that might contribute to cognition andespecially self-repair. We draw inspiration and insights fromcomputational neuroscience regarding functionalities of glial cellsand explore their role in the fault-tolerant capacity of SpikingNeural Networks (SNNs) trained in an unsupervised fashion usingSpike-Timing Dependent Plasticity (STDP). We characterize thedegree of self-repair that can be enabled in such networks withvarying degree of faults ranging from 50% - 90% and evaluateour proposal on the MNIST and Fashion-MNIST datasets.
Index Terms —Spiking Neural Networks, Astrocytes, Spike-Timing Dependent Plasticity, Unsupervised learning
I. I
NTRODUCTION
Neuromorphic computing has made significant strides overthe past few years - both from hardware [1]–[4] and algo-rithmic perspective [5]–[8]. However, the quest to decode theoperation of the brain have mainly focused on spike basedinformation processing in the neurons and plasticity in thesynapses. Over the past few years, there has been increasingevidence that glial cells, and in particular astrocytes, playa crucial role in a multitude of brain functions [9]. As amatter of fact, astrocytes represent a large proportion of thecell population in the human brain [9]. There have beenalso suggestions that complexity of astrocyte functionalitycan significantly contribute to the computational power ofthe human brain. Astrocytes are strategically positioned toensheath tens of thousands of synapses, axons and dendritesamong others, thereby having the capability to serve as a com-munication channel between multiple components and behaveas a sensing medium for ongoing brain activity [10]. This hasled neuroscientists to conclude that astrocytes play a majorrole in higher order brain functions like learning and memory,in addition to neurons and synapses. Over the past fewyears, there have been multiple studies to revise the neuron-circuit model for describing higher order brain functions toincorporate astrocytes as part of the neuron-glia network model[9], [11]. These investigations clearly indicate and quantify
Manuscript dated September, 2020.The authors are with the School of Electrical Engineering and ComputerScience, The Pennsylvania State University, University Park, PA 16802, USA.M. Rastogi is also affiliated with Department of of Computer Science andInformation Systems, Birla Institute of Technology and Science Pilani, GoaCampus, Goa 403726, India. E-mail: [email protected]. that incorporating astrocyte functionality in network modelsinfluence neuron excitability, synaptic strengthening and, inturn, plasticity mechanisms like Short-Term Plasticity andLong-Term Potentiation, which are important learning toolsused by neuromorphic engineers.The key distinguishing factors of our work against priorefforts can be summarized as follows: (i)
While recent literature reports astrocyte computationalmodels and their impact on fault-tolerance and synaptic learn-ing [9], [11]–[14], the studies have been mostly confined tosmall scale networks. This work is a first attempt to explorethe self-repair role of astrocytes at scale in unsupervised SNNsin standard visual recognition tasks. (ii)
In parallel, there is a long history of implementing as-trocyte functionality in analog and digital CMOS implemen-tations [15]–[21]. More recently, emerging physics in post-CMOS technologies like spintronics are also being lever-aged to mimic glia functionalities at a one-to-one level [22].However, the primary focus has been on a brain-emulationperspective, i.e. implementing astrocyte computational modelswith high degree of detail in the underlying hardware. Weexplore the aspects of astrocyte functionality that would berelevant to self-repair in the context of SNN based machinelearning platforms and evaluate the degree of bio-fidelityrequired. (iii)
While Refs. [23], [24] discusses impact of faults inunsupervised STDP enabled SNNs, self-repair functionality insuch networks have not been studied previously.While neuromorphic hardware based on emerging post-CMOS technologies [3], [25]–[28] have made significant ad-vancements to reduce the area and power efficiency gap ofArtificial Intelligence (AI) systems, such emerging hardwareare characterized by a host of non-idealities which has greatlylimited its scalability. Our work provides motivation towardautonomous self-repair of such faulty neuromorphic hardwareplatforms. The efficacy of our proposed astrocyte enabledself-repair process is measured by the following steps: (i)
Training SNNs using unsupervised STDP learning rules innetworks equipped with lateral inhibition and homeostasis, (ii)
Introducing “faults” in the trained weight maps by settinga randomly chosen subset of the weights to zero and (iii) Implementing self-repair by re-training the faulty network withastrocyte functionality augmented STDP learning rules. We Note that “faults” are disjoint from the concept of “dropout” [29] usedin neural network training. In dropout, neurons are randomly deleted (alongwith their connections) only during training to avoid overfitting. In contrast,faults in our work refer to static non-ideal stuck at zero synaptic connectionspresent during both the training and inference stages. a r X i v : . [ c s . N E ] N ov also compare our proposal with sole STDP based re-trainingstrategy and substantiate our results on the MNIST and F-MNIST datasets.II. M ATERIALS AND M ETHODS
A. Astrocyte Preliminaries
In addition to astrocyte mediated meta-plasticity for learningand memory [12], [30]–[32], there has been indication thatretrograde signalling via astrocytes probably underlie self-repair in the brain. Computational models demonstrate thatwhen faults occur in synapses corresponding to a particularneuron, indirect feedback signal (mediated through retrogradesignalling by the astrocyte via endocannabinoids, a type ofretrograde messenger) from other neurons in the networkimplements repair functionality by increasing the transmissionprobability across all healthy synapses for the affected neu-ron, thereby restoring the original operation [12]. Astrocytesmodulate this synaptic transmission probability (PR) throughtwo feedback signalling pathways: direct and indirect, respon-sible for synaptic depression (DSE) and potentiation (e-SP)respectively. Multiple astrocyte computational models [12],[30]–[32] describe the interaction of astrocytes and neuronsvia the tripartite synapse where the astrocyte’s sensitivity to2-arachidonyl glycerol (2-AG), a type of endocannabinoid, isconsidered. Each time a post synaptic neuron fires, 2-AG isreleased from the post synaptic dendrite and can be describedas: d ( AG ) dt = − AG τ AG + r AG δ ( t − t sp ) (1)where, AG is the quantity of 2-AG, τ AG is the decay rate of2-AG, r AG is the 2-AG production rate and t sp is the time ofthe post-synaptic spike.The 2-AG binds to receptors (CB1Rs) on the astrocyte pro-cess and instigates the generation of IP , which subsequentlybinds to IP receptors on the Endoplasmic Reticulum (ER) toopen channels that allow the release of Ca . It is this increasein cystolic Ca that causes the release of gliotransmittersinto the synaptic cleft that is ultimately responsible for the e-SP N1N2 A1 (a) e-SP N1N2 A1 (b)Fig. 1. (a) Network with no faults, (b) Network with fault occurring insynapse associated with neuron N2 [12]. 2-AG is local signal associated witheach synapse while e-SP is a global signal. A1 is the astrocyte. indirect signaling. The Li-Rinzel model [33] uses three chan-nels to describe the Ca dynamics within the astrocyte: J pump models how Ca is pumped into the ER from the cytoplasmvia the Sarco-Endoplasmic-Reticulum Ca -ATPase (SERCA)pumps, J leak describes Ca leakage into the cytoplasm and J chan models the opening of Ca channels by the mutualgating of Ca and IP concentrations. The Ca dynamicsis thus given by: d Ca dt = J chan + J leak − J pump (2)The details of the equations and their derivations can beobtained from Refs. [12] and [34].The intracellular astrocytic calcium dynamics control theglutamate release from the astrocyte which drives e-SP. Thisrelease can be modelled by: d ( Glu ) dt = − Glu τ Glu + r Glu δ ( t − t Ca ) (3)where, Glu is the quantity of glutamate, τ Glu is the glutamatedecay rate, r Glu is the glutamate production rate and t Ca is thetime of the Ca threshold crossing. To model e-SP: τ eSP d ( eSP ) dt = − eSP + m eSP Glu ( t ) (4)where, τ eSP is the decay rate of e-SP and m eSP is a scalingfactor. Eq. (4) substantiates that the level of e-SP is dependenton the quantity of glutamate released by the astrocyte.The released 2-AG also binds directly to pre-synpaticCB1Rs (direct signaling). A linear relationship is assumedbetween DSE and the level of 2-AG released by the post-synaptic neuron as: DSE = − AG × K AG (5)where, AG is the amount of 2-AG released by the post-synapticneuron and is found from Eq. (1) and K AG is a scaling factor.The PR associated with each synapse is given by the followingequation:PR ( t ) = PR ( t ) + PR ( t ) × (cid:18) DSE ( t ) + eSP ( t )100 (cid:19) (6)where, PR( t ) is the initial PR of the synapses, e-SP and DSEare given by Eq. (4) and (5) respectively. In the computationalmodels, the effect of DSE is local to the synapses connectedto a particular neuron whereas all the tripartite synapses con-nected to the same astrocyte receive the same e-SP. Under no-fault condition, the DSE and e-SP reach a dynamic equilibriumwhere the PR is unchanged over time, resulting in a fixedfiring rate for the neurons. When a fault occurs, this balancesubsides and the PR changes according to Eq. (6) to restorethe firing rate to its previous value. To showcase this effectconsider for instance, Fig. 1 where a simple SNN with twopost-synaptic neurons is depicted. Let us assume that eachpost-neuron receives input spikes from 10 pre-neurons. Theinitial PR of the synapses were set to 0.5. Fig. 1(a) is the casewith no faults, while in Fig. 1(b), faults have occurred aftersome time in of the synapses associated with post-neuronN2 (Fig. 2). Note, here “faults” imply that the synapses do nottake part in transmission of the input spikes i.e. have a PR of 7 L P H V H 6 3 7 L P H V ' 6 ( 1 1 7 L P H V 3 5 1 1 I D X O W \ V \ Q D S V H 1 K H D O W K \ V \ Q D S V H 7 L P H V ) U H T + ] 1 1 (a) 7 L P H V H 6 3 7 L P H V ' 6 ( 1 1 7 L P H V 3 5 1 1 I D X O W \ V \ Q D S V H 1 K H D O W K \ V \ Q D S V H 7 L P H V ) U H T + ] 1 1 (b) 7 L P H V H 6 3 7 L P H V ' 6 ( 1 1 7 L P H V 3 5 1 1 I D X O W \ V \ Q D S V H 1 K H D O W K \ V \ Q D S V H 7 L P H V ) U H T + ] 1 1 (c) 7 L P H V H 6 3 7 L P H V ' 6 ( 1 1 7 L P H V 3 5 1 1 I D X O W \ V \ Q D S V H 1 K H D O W K \ V \ Q D S V H 7 L P H V ) U H T + ] 1 1 (d)Fig. 2. Simulation results of the network in Fig. 1 using the computationalmodel of astrocyte mediated self-repair from [12]. Total simulation time is400s. At 200s, faults are introduced in 70% of the synapses connected to N2.All the synapses have PR( t )=0.5. (a) e-SP of N1 and N2. It is the same forboth N1 and N2 since e-SP is a global function, (b) DSE of N1 and N2. It isdifferent for each neuron as it is dependent upon the neuron output. At 200s,after the introduction of the faults in N2, only DSE of N2 changes, (c) PRof different types of synapses connected to N1 and N2, and (d) Firing rate ofneurons N1 and N2.
0. This results in a drop of the firing frequency associatedwith N2 while operation of N1 is not impacted. Thus, theamount of 2-AG released by N2 decreases, which increasesDSE and in turn increases the PR of the associated synapsesof N2 where no faults have occurred. Hence, we observe inFig. 2(d) that the increased PR recovers the firing rate andapproaches the ideal firing frequency. Note that the degree ofself-recovery, i.e. the difference between the recovered andideal frequency is a function of the fault probability. Thesimulation conditions and parameters for the modelling arebased on Ref. [12]. Interested readers are directed to Ref.[12] for an extensive discussion on the astrocyte computationalmodel and the underlying processes governing the retrogradesignalling.A key question that we have attempted to address in thiswork is the computational complexity at which we requireto model the feedback mechanism to implement autonomousrepair in such self-learning networks. Simplifying the feedbackmodelling would enable us to implement such functionalitiesby efficient hardware primitives. For instance, the core func-tionality of astrocyte self-repair occurs in conjunction withSTDP based learning in synapses. Fig. 3 shows a typical STDP learning rule where the change in synaptic weight variesexponentially with the spike time difference between the pre-and post-neuron [35], according to measurements performedin rat glutamatergic synapses [36]. Typically, the height of theSTDP weight update for potentiation/depression is constant( A + / A − ). However, astrocyte mediated self-repair suggeststhat the weight update should be a function of the firing rate ofthe post-neuron [35]. Assuming the fault-less expected firingrate of the post-neuron to be f ideal and the non-ideal firingrate to be f , the synaptic weight update window height shouldbe a function of ∆ f = f ideal − f . The concept has beenexplained further in Fig. 3 and is also in accordance withFig. 2 where the PR increase after fault introduction variesin a non-linear fashion over time and eventually stabilizesonce the self-repaired firing frequency approaches the idealvalue. The functional dependence is assumed to be that ofa sigmoid function – indicating that as the magnitude of thefault, i.e. deviation in the ideal firing frequency of the neuronincreases, the height of the learning window increases inproportion to compensate for the fault [35]. Note that the term“fault” for the machine learning workloads, described herein,refers to synaptic weights (symbolizing PR) stuck at zero.Therefore, with increasing amount of synaptic faults, f < We utilized the Leaky Integrate and Fire (LIF) spikingneuron model in our work. The temporal LIF neuron dynamicsare described as, τ mem ∂v ( t ) ∂t = − v ( t ) + v rest + I ( t ) (7)where, v ( t ) is the membrane potential, τ mem is the membranetime constant, v rest is the resting potential and I ( t ) denotes thetotal input to the neuron at time t . The weighted summationof synaptic inputs is represented by I ( t ) . When the neuron’s 𝐴 + = 𝐾 −∆𝑓 STDP Learning Macro-modelling astrocyte functionality: C h a ng e i n sy n a p se w e i gh t ( % ) -60 Spike Timing (ms) ∆𝑤 = 𝐴 + 𝑒𝑥𝑝 −∆𝑡 𝜏 + , ∆𝑡 > 0−𝐴 − 𝑒𝑥𝑝 −∆𝑡 𝜏 + , ∆𝑡 < 0 Fig. 3. In the above equations, the STDP learning window height is a non-linear increasing function of the deviation ∆ f from the ideal firing frequencyof the post-neuron. membrane potential crosses a threshold value, v th ( t ) , it firesan output spike and the membrane potential is reset to v reset .The neuron’s membrane voltage is fixed at the reset potentialfor a refractory period, δ ref , after it spikes during which itdoes not receive any inputs.In order to ensure that single neurons do not dominate thefiring pattern, homeostasis [6] is also implemented throughan adaptive thresholding scheme. The membrane threshold ofeach neuron is given by the following temporal dynamics, v th ( t ) = θ + θ ( t ) τ theta ∂θ ( t ) ∂t = − θ ( t ) (8)where, θ > v rest , v reset and is a constant. τ theta is theadaptive threshold time constant. The adaptive threshold, θ ( t ) is increased by a constant quantity θ + , each time the neuronfires, and decays exponentially according to the dynamics inEquation 8.A trace [37] based synaptic weight update rule was used forthe online learning process [6], [23]. The pre and post-synaptictraces are given by x pre and x post respectively. Whenever thepre (post) - synaptic neuron fires, the variable x pre ( x post ) isset to 1, otherwise it decays exponentially to 0 with spike tracedecay time constant, τ trace . The STDP weight update rule ischaracterized by the following dynamics, ∆ w = (cid:40) η post ∗ x pre on post-synaptic spike − η pre ∗ x post on pre-synaptic spike (9)where, η pre /η post denote the learning rates for pre-synaptic /post-synaptic updates respectively. The weights of the neuronsare bounded in the range of [0 , w max ] . It is worth mentioninghere that the sum of the weights associated with all post-synaptic neurons is normalized to a constant factor, w norm [23]. C. Network Architecture … Input Layer Output Layer Dense Connection Recurrent Connection … Fig. 4. The single layer SNN architecture with lateral inhibition andhomeostasis used for unsupervised learning. Our SNN based unsupervised machine learning frameworkis based on single layer architectures inspired from corticalmicrocircuits [6]. Fig. 4 shows the network connectivityof spiking neurons utilized for pattern-recognition problems.Such a network topology has been shown to be efficient inseveral pattern-recognition problems, such as digit recognition[6] and sparse encoding [38]. The SNN, under consideration,has an Input Layer with the number of neurons equivalent to the dimensionality of the input data. Input neurons generatespikes by converting each pixel in the input image to a Poissonspike train whose average firing frequency is proportional tothe pixel intensity. This layer connects in an all-to-all fashionto the Output Layer through excitatory synapses. The Outputlayer has n neurons LIF neurons characterized by homeostasisfunctionality. It also has static (constant weights) recurrentinhibitory synapses with weight values, w recurrent , for lateralinhibition to achieve soft Winner-Take-All (WTA) condition.Each neuron in the Output Layer has an inhibitory connectionto all the neurons in that layer except itself. Trace-based STDPmechanism is used to learn the weights of all synapses betweenthe Input and Output Layers. The neurons in the Output Layerare assigned classes based on their highest response (spikefrequency) to input training patterns [6]. D. Challenges and Astrocyte Augmented STDP (A-STDP)Learning Rule Formulation One of the major challenges in extending the astrocyte basedmacro-modelling in such self-learning networks lies in thefact that the ideal neuron firing frequency is a function ofthe specific input class the neuron responds to. This is sub-stantiated by Fig. 5 which depicts the histogram distributionof the ideal firing rate of the wining neuron in the fault-less network. Further, due to sparse neural firing, the totalnumber of output spikes of the winning neurons over theinference window is also small, thereby limiting the amount ofinformation (number of discrete levels) that can be encoded inthe frequency deviation, ∆ f . This leads to the question: Canwe utilize another surrogate signal that gives us informationabout the degree of self-repair occurring in the network overtime while being independent of the class of the input data? , G H D O I L U L Q J U D W H + L V W R J U D P F R X Q W &