Towards Efficient Processing and Learning with Spikes: New Approaches for Multi-Spike Learning
Qiang Yu, Shenglan Li, Huajin Tang, Longbiao Wang, Jianwu Dang, Kay Chen Tan
11 Towards Efficient Processing and Learning withSpikes: New Approaches for Multi-Spike Learning
Qiang Yu, Shenglan Li, Huajin Tang, Longbiao Wang, Jianwu Dang, Kay Chen Tan,
Fellow, IEEE
Abstract —Spikes are the currency in central nervous systemsfor information transmission and processing. They are alsobelieved to play an essential role in low-power consumptionof the biological systems, whose efficiency attracts increasingattentions to the field of neuromorphic computing. However,efficient processing and learning of discrete spikes still remains asa challenging problem. In this paper, we make our contributionstowards this direction. A simplified spiking neuron model is firstlyintroduced with effects of both synaptic input and firing outputon membrane potential being modeled with an impulse function.An event-driven scheme is then presented to further improve theprocessing efficiency. Based on the neuron model, we proposetwo new multi-spike learning rules which demonstrate betterperformance over other baselines on various tasks includingassociation, classification, feature detection. In addition to effi-ciency, our learning rules demonstrate a high robustness againststrong noise of different types. They can also be generalizedto different spike coding schemes for the classification task,and notably single neuron is capable of solving multi-categoryclassifications with our learning rules. In the feature detectiontask, we re-examine the ability of unsupervised STDP with itslimitations being presented, and find a new phenomenon of losingselectivity. In contrast, our proposed learning rules can reliablysolve the task over a wide range of conditions without specificconstraints being applied. Moreover, our rules can not only detectfeatures but also discriminate them. The improved performanceof our methods would contribute to neuromorphic computing asa preferable choice.
Index Terms —Spiking neural networks, multi-spike learning,feature extraction, STDP, robust recognition, neuromorphic com-puting.
I. I
NTRODUCTION H UMAN brain has shown remarkable abilities on variouscognitive tasks such as recognition, decision making,learning and memory, while operates with an extraordinarily
This work was supported in part by the National Natural Science Foundationof China under Grant 61806139 and Grant 61876162, in part by the NaturalScience Foundation of Tianjin under Grant 18JCYBJC41700 and Grant19ZXZNGX00030, in part by the Peiyang Scholar Program of Tianjin Univer-sity under Grant 2020XRG-0050, in part by the Shenzhen Scientific Researchand Development Funding Program under Grant JCYJ20180307123637294,in part by the Research Grants Council of the Hong Kong SAR under GrantCityU11202418 and Grant CityU11209219. (Corresponding author: QiangYu.)
Q. Yu, S. Li, L. Wang and J. Dang are with Tianjin Key Laboratory ofCognitive Computing and Application, College of Intelligence and Comput-ing, Tianjin University, Tianjin, China (e-mail: [email protected]).H. Tang is with the College of Computer Science and Technology, ZhejiangUniversity, China.J. Dang is also with Japan Advenced Institute of Science and Technology,Japan, and Huiyan Technology (Tianjin) Co., Ltd., Tianjin, China.K.C. Tan is with the Department of Computer Science, City Universityof Hong Kong, and the City University of Hong Kong Shenzhen ResearchInstitute. low consumption of power and a fast speed of cognition [1]–[4]. The excellence of the brain has inspired increasing effortsbeing devoted to understanding the principles how it operatesas well as applying those principles to endow artificial systemswith a similar ability on information processing as the brain.The perceptron [5] is one of the earliest brain-inspiredattempt to build artificial neurons to learn for recognition.Starting from the perceptron model, artificial neural networks(ANNs) have drawn a great amount of attentions in thetrend of artificial intelligence (AI). Driven by advances in aclass of techniques called deep learning, ANNs have beenthriving with a great success in tackling problems acrossdiverse fields including image and speech recognition, naturallanguage processing, autonomous driving and bioinformatics[6]. Despite of their popularity, one of the major criticismsfor current deep learning methods comes from the lack ofbiological plausibility. Additionally, deep ANNs are almostalways trained on very fast and power-hungry modern daysupercomputers with Graphic Processing Units (GPUs), lead-ing to a challenge of running these networks on low-powerdevices. Substantial efforts are invested to improve the effi-ciency of ANNs [7]. However, there is still a huge gap in theefficiency as is compared to their biological counterparts, letalone the superior cognitive abilities of the brain. Therefore,it is desired to develop networks which are efficient on onehand and biologically plausible on the other hand to a certainextent.Neurons in ANNs and their biological counterparts differat least in the way how they communicate with each other.Artificial neurons use analog values while biological ones takeadvantage of spikes. It is believed that discrete spikes play anessential role in efficient processing [1], [8], [9]. Inspired byneuroscience, preliminary neuromorphic approaches from bothsoftware and hardware have been introduced to harness the ad-vantages of biological systems [3], [10]–[15]. Efficiency is oneof the key focuses especially considering the fundamental in-efficiency and non-scalability of the von Neumann architecture[3]. However, the learning capabilities of current neuromorphichardware are relatively simple and limited due to the complexdynamics of the spiking agents [8], [16], [17]. On the otherhand, the complexity of the spike processing and learning insoftware restricts their implementations on hardware. Thus, thegap between the two worlds motivates our study in this worktowards efficient processing and learning with spikes, whilebearing in mind the simplicity for implementation and thefeasibility for potential developments in practical applications.The fundamental currency in nervous systems is spikes,where information could be carried by the number of spikes, a r X i v : . [ c s . N E ] M a y their occurrence time or their shapes [1], [18], [19]. In order toemulate abilities of biological neurons on processing spikes,spiking neuron models are developed, and are believed topossibly lead a new generation of ANNs [20], [21]. Popularspiking neuron models, such as Hodgkin-Huxley model [22],[23], Izhikevich model [24], leaky-integrate-and-fire (LIF)model [25] and spike response model (SRM) [20], are intro-duced with certain levels of resembling behaviors of biologicalneurons. These models differ from the degree on describingdetails of neuronal dynamics, and thus their complexities varyfrom one to another. Although LIF and SRM models arerelatively less biologically plausible as compared to the others,their simpler forms and ease of processing make them nearlythe most commonly used spiking neuron models for brain-inspired or neuromorphic computing [26]–[28].Putting the processing units aside, how spikes can beused for information transmission still remains unclear, whichrestricts developments of spiking neural networks (SNNs) fora broad range of applications [29]. The most two popularcoding assumptions are the rate and the temporal codes [1],[2], [18], [30], [31]. A spike train conveys information withits number of spikes (or firing rate) under a rate code, whileindividual spike timing matters for representing informationunder a temporal one. The rate code is simple and robust tointer-spike-interval noise as it ignores the temporal structureof the spike train [32], [33]. Such a rate code enhances thesimilarity between the non-spiking artificial neurons in ANNsand the spiking ones in SNNs, and thus rendering comparableperformance in recognition tasks [14], [34]. On the other hand,the temporal code has a high information-carrying capacity asa result of making full use of the temporal structure [35], [36].This makes the temporal code an appealing one for efficientprocessing [26], [37]. Most spiking frameworks or learningsystems solely rely on a single coding scheme, but cannot begeneralized, limiting their capabilities of utilizing advantagesof different codes, as well as of exploring processing principlesof the biological systems.Unavoidably, the learning capability is an essential charac-teristic that is required for building cognitive artificial neuralsystems. The learning determines how neurons adapt theirsynaptic efficacies in response to the inputs in a way suchthat they could fit the environment to solve certain cognitivetasks. Therefore, we mainly focus on the learning in this workdue to this importance.Inspired by neuroscience, various learning rules have beendiscovered and developed in recent years. Hebbian learningis one of the earliest principles describing how neuronalconnections are modified [38], and it can be simply stated as“neurons that fire together, wire together.” Increasing exper-imental observations demonstrate that synaptic modificationdepends on tight temporal correlations between the spikes ofpre- and post-synaptic neurons, leading to a temporally asym-metric form of Hebbian learning, the spike-timing-dependentplasticity (STDP) [38]–[40]. STDP enables neuron to processinformation in an unsupervised way [41], [42], but its depen-dence on temporal contiguity could limit its applicability [14],[30], [43].Different supervised learning rules have been developed to train spiking neurons. The tempotron is an efficient learningrule that trains neurons to make decisions by binary behaviorof firing or not, being reminiscent of the perceptron but withan additional time dimension being involved [43]. The binaryresponse of the tempotron could constrain neuron’s ability tofully utilize the temporal structure of the output [44], steeringefforts to a family of learning rules which can train neurons tofire at desired times [27], [45]–[49]. However, how to constructan instructor signal with precise timings is challenging for bothartificial and biological systems [29], [50]. Moreover, mostof these supervised spike learning rules are designed for atemporal code, limiting their generalization to other schemessuch as a rate one [44], [51].Recently, a supervised multi-spike tempotron (MST) rule[28] is developed to train a neuron to fire a desired numberof spikes, which empowers it to discover sensory featuresembedded in a complex background activity. This kind ofmulti-spike learning rule provides a new way for processinginformation under a broad range of coding schemes and hasshown good performance on some sound recognition tasks[52]. Improved modifications have been developed in [44],[53], along with detailed evaluations of different propertiesas well as theoretical proofs on convergence and robustness.However, their complexity with respect to both learning andprocessing would limit their applicability to a large-scaleand efficient neuromorphic developments. In this work, wewill continue to contribute towards this supervised multi-spike learning with simplicity, efficiency and capabilities ofinformation processing bearing in mind. We focus more onextending the potential applicability of multi-spike learningrules by providing efficient alternatives. The significance ofour major contributions can be highlighted in the followingaspects. • A simplified LIF neuron model is introduced for efficientprocessing of spikes, making it valuable for both softwareand hardware implementations. This is significantly im-portant especially with considerations of the highly com-plex nonlinear dynamics of a spiking model. Additionally,an event-driven scheme, where computation is driven byspikes, is described in our framework, further benefitingthe efficiency for both processing and learning of spikes. • We propose two new approaches for multi-spike learn-ing, namely efficient multi-spike learning (EML) andan alternative relying only on neuron’s current response(named as EMLC where ‘C’ stands for ‘current’). Theefficient performance of our learning rules together withtheir simplicity and computational capabilities contributeto build large scale neuromorphic systems which couldpotentially drive a paradigm shift on processing towardsmore brain-like. • We evaluate the performance of our learning rules ona broad range of typical tasks including efficient pro-cessing, multi-category classification and robustness, withcomparisons to other baseline methods, demonstrating theadvanced performance of our work. Our results thus canfurther provide useful reference for applied developmentsof neuromorphic systems. • The ability of spike learning rules on feature detectionfrom background activities is evaluated with a specificinterest due to its importance on perception. Notably, were-examine the unsupervised learning with STDP on thistask, observing a new finding about loss of detection aftersufficient learning. On the contrary, our proposed rulesshow better performance on not only detection but alsodiscrimination in a more challenging task. These prefer-able performances make our algorithms a potential toolfor processing temporal information as well as for betterunderstanding computational principles of the brain.The remainder of this paper is structured as follows. Sec-tion II introduces our proposed approaches for spike process-ing and learning. Section III then shows our experimentalresults, followed by discussions in Section IV. Finally, weconclude our work in Section V.II. M
ETHODS
In this section, we will introduce the methods proposedin our work for spike processing and learning. Firstly, wedescribe an efficient neuron model, followed by descriptions ofan event-driven scheme. Then, two new supervised multi-spikelearning rules are proposed. Additionally, we also introducethe STDP rule that is used as a benchmark in our featuredetection task.
A. Neuron Model
The simplicity of LIF and SRM neuron models makes themthe most commonly used ones in neuromorphic computing.An LIF neuron model can be mapped to SRM with certaindefined spike response functions [20]. Therefore, we start witha current-based leaky integrate-and-fire neuron model due toits simplicity and analytical tractability. Following a typicaldescription of an LIF [20], our neuron model is given as dVdt = − τ V ( t ) + I in ( t ) + I out ( t ) (1)where τ represents the time constant of neuron’s membranepotential, V , I in and I out model the inputs from pre-synapticneurons and firing reset dynamics, respectively. The units ofall parameters except τ are set to 1 for a general description.We set the two inputs in a simple form as I in ( t ) = N (cid:88) i =1 w i (cid:88) t ji ≤ t δ ( t − t ji ) (2) I out ( t ) = − ϑ (cid:88) t j s Fig. 1. Demonstration of the evolving dynamics of the spiking neuronmodel. A , an exemplary spike pattern which contains 10 afferent neurons firingcertain numbers of spikes (denoted by dots) across time. B , the correspondingsynaptic weights ( w ) of a postsynaptic neuron receiving spikes from theseafferents. C , the resulting membrane potential dynamics of the neuron inresponse to the pattern in A . The dashed line represents the firing threshold, ϑ . D , demonstration of the post-synaptic potential kernel, κ . Integrating Eq. (1) with substitutions of I in and I out , weget a form of SRM as V ( t ) = N (cid:88) i =1 w i (cid:88) t ji ≤ t κ ( t − t ji ) − ϑ (cid:88) t j s Following the approach in [44], we adopt an event-drivencomputation for efficient processing. This event-driven ap-proach is more efficient than a clock-based one because itdoes not depend on a step size for simulation, thus reducingcomputational operations to be linearly related to the totalnumber of input spikes ( n ). Moreover, exact solutions can beobtained with the event-driven approach without constraintsfrom the time resolution for simulation in a clock-based one.We consider a stream of input spikes t ≤ t . . . ≤ t n with w , w , . . . , w n denoting the corresponding synaptic weightsassociated with each spike. According to Eq. (4), neuron’smembrane potential can be rewritten as V ( t k ) = V ( t k − ) exp ( − ∆ k − /τ ) + w k (6)where ∆ k − = t k − t k − denotes the inter-spike interval beforethe k -th input spike. Notably, Eq. (6) describes the membranepotential without firing reset. If a neuron’s potential is greaterthan its firing threshold, a reset dynamic will be involved toupdate its potential. Algorithm 1 shows the abstract schemefor our event-driven computation.Notably, following a similar routine, the above event-drivenapproach can be easily transformed to a clock-based one where ∆ k is replaced by a fixed time step. In this work, we adoptthe event-driven scheme. Algorithm 1: Event-driven computation function Response ( S ) ; Input : Spike pattern S = { t k | k = 1 , , ..., n } Output: Number of output spikes n o initialization; while there is a new incoming spike t k do update membrane potential V ( t k ) according toEq. (6); while V ( t k ) > ϑ do update V ( t k ) with firing reset dynamicsaccording to Eq. (1) or Eq. (4); n o ← n o + 1 ; end end return n o ; C. Multi-Spike Learning Rules Recently, a multi-spike tempotron (MST) rule is proposedto train neurons with a desired number of spikes [28], leadinga new family of plasticity rules (here referred as multi-spikelearning rules in this paper). Two threshold-driven plasticity(TDP1 and TDP2) rules are developed depending on the linearassumption around threshold crossing [44], and improvedperformance has been demonstrated. Here, we continue tocontribute to this new family of learning rules with efficiencymainly considered for processing and learning. In this work,we propose two new approaches, i.e. EML and EMLC, asfollows. 1) The EML rule: The first rule we proposed is calledefficient multi-spike learning (EML) which is based on thespike-threshold-surface (STS), like MST [28] and TDP [44].STS characterizes the relation between neuron’s actual outputspike number and its firing threshold. A higher threshold valuenormally results in a lower number of output spikes. Accordingto this property, critical thresholds ϑ ∗ k can be highlighted as theposition at which point neuron’s output spike number jumpsfrom k − to k . Therefore, modifications of these criticalthreshold values can result in a desired output spike number,but the challenge is how to change them.Each critical threshold value ϑ ∗ k corresponds to a voltagedescribed by Eq. (4), and thus it is a function of the synapticweights w i and differentiable with respect to them. We definethe maximum of subthreshold voltages for a given ϑ as v max ( ϑ ) . Consider a ϑ ∗ as the threshold, we assume thereexists a t ∗ such that V ( t ∗ ) = v max ( ϑ ∗ ) = ϑ ∗ . There couldexist a number of output spikes that occur before t ∗ , andthus complicates the derivative evaluations [28], [29]. In ourmethod, we find the dependence through previous outputspikes can be neglected. This is because that for any precedingoutput spikes j , ∃ ξ > such that V ( t j s ) − ϑ > ξ . A sufficientlysmall change on w will hardly affect t j s . The derivative of ϑ ∗ with respect to w i is denoted as ϑ ∗ (cid:48) i , and can thus be given as ϑ ∗ (cid:48) i = ∂V ( t ∗ ) ∂w i = (cid:88) t ji ≤ t ∗ κ ( t ∗ − t ji ) (7) V LTD LTP nu m b e r o f s p i k e s LTP LTD Fig. 3. Illustration of long-term potentiation (LTP) and long-term depression(LTD) plasticity rules of different multi-spike algorithms. Top, plasticity relieson critical thresholds, ϑ ∗ k , of the neuron’s spike-threshold-surface (STS).Vertical dashed line denotes the threshold of the neuron. Bottom, plasticityonly depends on the current dynamics of the neuron in response to a pattern.The red circle illustrates the minimum of reset potentials at all output spiketimes, t j s . The blue circle denotes the maximum of sub-threshold potentials. According to Eq. (7), a training method can thus be de-veloped to adapt critical thresholds via changes of synapticefficacies. Among many possible objectives, we choose oneof the simplest that only considers ϑ ∗ n o and ϑ ∗ n o +1 with n o denoting the actual output spike number. The supervised signalis the difference between the number of n o and n d . The targetis to train the neuron to elicit a desired number of spikes, n d .The learning rule can be given as ∆ w = (cid:40) − λ dϑ ∗ no dw if n o > n d λ dϑ ∗ no +1 dw if n o < n d (8)where dϑ ∗ k /dw represents the directive evaluation calculatedaccording to Eq. (7), and λ is a learning rate that controlsthe step size of each adaption. The essential idea of this rule(see Fig. 3) is to decrease (increase) the critical values thatare bigger (smaller) than ϑ with an LTD (LTP) process if aneuron fails to elicit a desired number of spikes. 2) The EMLC rule: The rules of MST, TDP and also as-proposed EML are all based on STS, and thus we refer themas STS-based rules. These methods depend on evaluations ofcritical thresholds as well as their derivatives with respect tosynaptic efficacy, which is complex and could thus slow downthe processing. Our preliminary attempt [53] has demonstratedgreat improvement on efficiency by combining both the tem-potron and PSD rules. Here, we further our study by proposinga new approach where only neuron’s current states of response are considered. We name this rule as EMLC (here ‘C’ standsfor ‘current’).Intuitively, one quick way to change the output spikenumber is to directly modify the voltages that are close tothe neuron’s threshold. Following this idea, we choose thetime point at the maximum subthreshold voltage to performLTP, and denote this time point as t LTP . For the LTD process,we select the time point, t LTD , where the voltage after firingreset is the minimum among all output spikes. An illustrationof the plasticity is shown in Fig. 3. The EMLC rule can thusbe formalized as ∆ w = (cid:40) − λ ∂V ( t LTD ) ∂w if n o > n d λ ∂V ( t LTP ) ∂w if n o < n d (9)According to Eq. (9), a neuron can thus learn to fire adesired number of spikes based on its current states ratherthan STS. D. Unsupervised STDP Rule for Comparison Here, we introduce the unsupervised STDP learning ruleadopted as a baseline in the subsequent task of feature de-tection. As the unsupervised STDP learning rule is widelystudied in spike-based processing [42], [54] and demonstratedcapability of detecting features from background activities, sowe choose it for a clear comparison with our supervised multi-spike method.Different from approaches in [42], [54] where ‘nearestspike’ approximation is applied for the learning, we use a moregeneral STDP learning rule where every pair of pre- and post-synaptic spikes will contribute a synaptic change (see Fig. 4). LTDLTP prepost Fig. 4. Demonstration of spike-timing-dependent plasticity (STDP). Theweight modulation, ∆ w , depends on the time difference between the pre-and post-synaptic spike timings, ∆ t = t pre − t post . The top presents thelearning window, while the bottom shows exemplary spikes from both pre-and post-synaptic neurons. The basic STDP rule can be formalized as ∆ w = (cid:40) A p exp(∆ t/τ p ) if ∆ t ≤ − A n exp( − ∆ t/τ n ) if ∆ t > (10)where ∆ t = t pre − t post denotes the time difference betweena pair of pre- and post-synaptic spikes. A p and A n are themodulation magnitudes for LTP and LTD, respectively. τ p and τ n are the corresponding time constants of the STDP learningwindow. E. Momentum A momentum scheme could accelerate the learning [43], andthus it is applied in our study. The actual performed synapticupdate ∆ w is composed of two parts: the current modification, ∆ w current , that is determined by the corresponding learningrules, and a fraction of the previous applied update ∆ w previous .Therefore, in each error trial, the resulting synaptic update isas ∆ w = ∆ w current + µ ∆ w previous (11)where µ ∈ [0 , is the momentum parameter determining thefraction of the previous update.III. S IMULATION R ESULTS In order to better benchmark performances of our learningrules, we show simulation results in this section includingderivative evaluation, learning efficiency and capabilities forclassification and feature detection, etc. The default setupsfor the number of connected pre-synaptic afferents and thelearning rate are as: N = 500 and λ = 10 − , respectively.Neuron’s synaptic efficacies (weights) are initialized with boththe mean and standard deviation being set to 0.01. Input spikepatterns are generated over a time window of T = 500 ms witheach afferent neuron firing at a Poisson rate of r in = 4 Hz over T . Different setups from the default will be stated otherwise.Our experiments were performed on a platform of Intel [email protected] with two-processor Intel(R) Core CPU and16GB main memory. A. Derivative Evaluation In addition to the rationale of our proposed EML ruleintroduced above, here we show its derivative evaluation witha similarity metric where the evaluation is compared to the the-oretical derivative whose value is approximated numerically.Following the approach in [44], the theoretical derivative iscalculated as ϑ ∗ ( w i + ξ ) − ϑ ∗ ( w i ) ξ (12)where ξ is an infinitesimal change on weight. The similaritymetric between two vectors, e.g., (cid:126)x and (cid:126)y , is calculated as cos( θ ) = (cid:126)x · (cid:126)y | (cid:126)x || (cid:126)y | (13)Here, (cid:126)x and (cid:126)y are two vectors representing the derivativeevaluation of a method and the theoretical one, respectively.In Fig. 5, we present the evaluations of TDP1, TDP2 andEML rules with this metric. As can be seen from the figure,both TDP1 and TDP2 methods will slowly diverge fromthe theoretical evaluation when n ∗ out increases. The vectorangles will stay around ◦ when n ∗ out is large, indicating apositive component contribution towards the same directionas the theoretical one. We will denote TDP1 as TDP inour following experiments to provide detailed benchmarks.Notably, our proposed EML rule provides a perfect match with n *out c o s () EMLTDP1TDP2 Fig. 5. Derivative evaluations of different STS-based rules. Their similarities, cos( θ ) , with the theoretical one are presented against different critical outputnumbers, n ∗ out . Each line and the corresponding shaded area denote the meanand standard deviation over 100 independent evaluations. the theoretical one due to the characteristics of our model.In our method, the impact of synaptic efficacy via precedingoutput spikes on ϑ ∗ can be neglected, resulting in a simplerand yet accurate evaluation of derivatives, and thus benefitingboth the processing and learning. B. Learning Efficiency In this part, we will examine the learning efficiency ofdifferent multi-spike learning rules, including MST [28], TDP[44], EML and EMLC. In the task, these learning rules needto train neurons to elicit a desired number of output spikes,and their training efficiencies are recorded for comparison.In the first experiment, we set a relatively high firing rateof r in = 6 Hz to increase the computational load. A neuronwas trained with an input spike pattern being presented toit one time after another until it fires a desired spike number, n ∗ out . Each pattern presence is denoted as one epoch. The totaltraining epochs and cpu times are recorded until successfullearning. Fig. 6 shows the learning efficiency versus differentchoices of n ∗ out . When n ∗ out is small, the difference of theserules with respect to training epochs is small. When n ∗ out increases, TDP uses the least number of epochs to finish thelearning while MST is relatively slow as compared to others.When we consider the actual execution time on cpu, both ofour proposed rules, i.e. EML and EMLC, are faster than thosebaseline rules. Our EMLC is the most efficient one, with anaverage over × faster than MST.Different initial setups will result in a different number ofoutput spikes, and can thus affect the learning. To examinethe learning reliability over different initial cases, we conductour second experiment where different initial mean weights areused to initialize synapses. In this task, we set r in = 10 Hz and T = 1 . s to further increase the computational load of eachneuron. Every neuron is trained to fire 10 spikes. Similarly,training epochs and cpu times are recorded until successfullearning. e p o c h s MSTTDPEMLEMLC1 25 50 75 n *out c p u t i m e ( s ) MSTTDPEMLEMLC AB Fig. 6. Efficiency of different multi-spike learning rules. A and B showthe convergence epochs and the corresponding cpu running time, respectively.Neurons are trained with different rules to elicit certain output spike numbers, n ∗ out , in response to a spike pattern. Data were collected over 100 independentsimulation runs. As can be seen from Fig. 7, the learning speeds of allmulti-spike rules change with different initial mean weightsdue to the incremental updating characteristics of the learning.Different mean weights will result in different initial outputspike numbers, and thus the closer this value to the desired,the faster the learning. Similar to the findings in Fig. 6, MST isrelatively slow as compared to the others. Both of our proposedmethods outperform the other two in terms of computationalefficiency, i.e. cpu time. EMLC is the fastest one among thesemethods, and it is more than × efficient as is compared toMST on average. C. Learning to Classify Spike Patterns Classification is a typical cognitive capability of most arti-ficial intelligent agents [55]. In this experiment, we study theability of different rules on discriminating spike patterns ofdifferent categories. Here we design a multi-category problemwith 3 classes as an example. The neuron parameters arethe same as previous except that the mean and standarddeviation of initial weights are set as 0 and 0.001, receptively.Additionally, a momentum scheme [43], [44] with µ = 0 . isapplied to accelerate the learning. We perform two differenttasks.In the first task, we consider a spatiotemporal spike patternclassification where each pattern is generated with r in = 2 Hz resulting in an average of one spike per synaptic channelover the time window [47]–[49]. Three templates are randomly e p o c h s MSTTDPEMLEMLC0.00 0.01 0.02 w initmean c p u t i m e ( s ) AB Fig. 7. Learning efficiency against different initial setup conditions, w initmean . A and B show the convergence epochs and cpu execution time of differentlearning rules. Data were averaged over 100 runs. generated and then fixed after that. Spike patterns of eachcategory are instantiated by adding two types of noise tothe corresponding template: spike jitter noise σ jit and spikedeletion noise p del . We use σ jit = 2 ms and p del = 0 . to trainneurons for different noise types. Three neurons are trainedwith the multi-spike rules to elicit more than 20 spikes for theircorresponding target category and keep silent for the others.A strict readout scheme is adopted to highlight the superiorityof the multi-spike learning over the binary one. During theevaluation phase, one can choose different readout schemes toinference the category of the input pattern. Here, we simplyselect a decision spike number to be half as the one used fortraining. That is, only if the corresponding neuron fires morethan 10 spikes, we refer it as a correct action, otherwise aswrong. Note that, other readout schemes could also be appliedto further improve the performance such as a competing one[29], but we use a simple scheme here to solely highlight thelearning capabilities of our methods.As can be seen from Fig. 8, all the multi-spike learningrules outperform the binary tempotron rule in terms of robust-ness, indicating the advantages of exploiting output temporalstructure with multiple spikes. These multi-spike learningrules can tolerate more than 100 ms jitter and 40% randomspike deletion with a high accuracy (100%) being preserved.Notably, our proposed learning rules, i.e. EML and EMLC,are more robust than the other multi-spike learning ones.Moreover, our methods are more efficient (over × ) than theothers with respect to the inference time after learning. There jit (ms) A cc u r a c y ( % ) EMLEMLCTDPMSTBIN0.0 0.2 0.4 0.6 p del A cc u r a c y ( % ) EML EMLC TDP MST050100150 I n f e r e n c e t i m e ( s ) ABC Fig. 8. Learning performance of different rules on spatiotemporal spikepattern classification. A and B show the learning accuracy against spike jitternoise ( σ jit ) and spike deletion noise ( p del ), respectively. In addition to themulti-spike learning rules, the performance of the binary-spike tempotron rule(‘BIN’) is also presented. Neurons in A and B are trained with σ jit = 2 msand p del = 0 . , respectively. C , inference time of neurons in A with both σ jit = 2 ms and p del = 0 . being imposed. Data were collected over 100independent simulations. is no significant difference on the inference time between EMLand EMLC since learning is not involved. This processingefficiency could be beneficial for low-power devices.In the second task, we examine the ability of our EMLrule to train a single neuron to solve the challenging multi-category classification. Following a similar experimental setupin [44], we consider two different scenarios where a time- orrate-based coding scheme is used to generate spike patterns.The generation of time-based spike patterns is similar to theprevious classification task. For the rate-based scenario, werandomly generated 3 firing-rate templates where a randomhalf afferents have a low firing rate of 2 Hz while the other halfhas 10 Hz. Each spike pattern is generated according to thePoisson process every time, and thus information is carried bythe firing rates rather than precise spike timings. We train theneuron with desired numbers of spikes in response to differentcategories as: 5, 10 and 15. Fig. 9 shows that our learning rulecan successfully train a single neuron to perform the multi- C1 C2 C3 n o u t timerate Fig. 9. Capability of a single neuron to perform multi-category classificationsfor both time- and rate-based spike patterns. The neurons is trained to classifythree categories by eliciting a number of spikes, n out , as: 5 (C1), 10 (C2)and 15 (C3). Data were collected over 1000 evaluations. category tasks for both time- and rate-based spike patterns.This highlights that our learning rule can be generalized todifferent coding schemes and might be applied to a broadrange of situations where how spikes are used to code theinformation might be unclear. D. Feature Detection with STDP Previous studies show that neurons with STDP can detectfeatures from background activities in an unsupervised way[41], [42], but the dependence of STDP on temporal contiguitycould limit its capability. In order to provide a good benchmarkfor our methods on the feature detection task, we firstly re-examine the ability of the well-known STDP for unsupervisedfeature detection. The learning performance and its limitationsare highlighted.We set A p = 5 × − , A n = 0 . A p , τ p = 20 ms, and τ n = 40 ms, resulting in a domination of LTD over LTP as thatin [42]. This domination is required for the success of STDP asit suppresses those non-selective spikes. Otherwise, the neuronwill experience an explosion of spikes without it. We use ahigh initial weight setup with the mean and standard deviationbeing set to 0.05 and 0.01 respectively. This is because STDPdepends on the appearance of output spikes and thus highinitial values can result in more spikes to facilitate the learning.We construct a random feature pattern with 4 Hz of afferentfiring rate over a time window of 100 ms. This feature israndomly embedded by replacement in a background activityof 5 s (called a trial pattern here) with the same firing rateas the feature. We set the occurrence rate of the feature to3 Hz. Then, a background firing noise of 1 Hz is added tofinalize the trial pattern. Each training cycle consists of 10 trialpatterns. In the evaluation phase, we use a specific schemeto reliably evaluate the neuron’s response to spike patterns.Specifically, a background pattern ( P φ ) with a time windowof 2 s is firstly generated, and then is fed to the neuron with aresulting output spike number ( R φ ) being recorded. A feature Cycles R e s p o n s e B1 B2 B3 B4 B5 B6 backgroundfeature P a tt e r n B1B2B3B4B5 Time (s) B6 AB Fig. 10. Feature detection with STDP. A , neuron’s response to bothbackground and feature spike patterns along the training cycles. Solid lines andshaded areas denote the mean and standard deviation, respectively. Data werecollected over 1000 evaluations. Six typical phases (‘B1’-‘B6’) were markedto demonstrate the behavior of the neuron along learning. B , the dynamics ofthe neuron, at different phases marked in A , in response to a sample spikepattern trial. Each black dot in the pattern represents a spike and only 4% ofthe afferents are presented for a better visualization. Appearance of feature ishighlighted with the shaded blue bars. ‘B1’-‘B6’ show the voltage traces andred dots represent the output spikes. pattern (or background with the same length of time as thefeature) is inserted in the middle of P φ , and then we recordthe neuron’s response to it as R f (or R b accordingly). Thedifference between R f (or R b ) and R φ is used as a responsemeasurement. In this way, we can eliminate the reset effecton the neuron. Notably, we additionally constrain synapticweights to fall between 0 and 0.1. This constraint is essentialfor the success of STDP, like that in [42], [54].Fig. 10 shows the learning dynamics of STDP for unsu-pervised feature detection. Initially, the neuron fires at a highrate in response to both background and feature patterns (B1).Then, the neuron gradually depresses its response to back-ground toward zero, and selectivity on feature pattern starts toappear (B2). The learning enters a plateau of selectivity (B2-B4). During this phase, neuron learns to find the starts of thefeature pattern (B4), a characteristic of STDP which is similarto [42], [54]. Differently, we find in our study that the neuron will gradually lose this selectivity for further learning (B4-B6). This is due to the domination of LTD which decreasessynaptic efficacies to a level that neuron will rarely fire. Ournew finding suggests that the domination of LTD is beneficialfor the learning at the beginning on one hand, but probably adisaster for further learning on the other hand. Proper balanceand tuning for the success of STDP are just required. E. Feature Detection with Multi-Spike Learning In this part, we examine the ability of our multi-spikelearning rules for feature detection. We perform two differenttasks. A momentum of µ = 0 . is adopted to accelerate thelearning [43], [44]. w initmean R a t e ( H z ) C v g . C y c l e s Fig. 11. Feature detection with the EML rule under different initial condi-tions, w initmean . Both the initial firing rate of the neuron and the convergencecycles of the learning are presented. Data were collected over 100 independentruns. In the first task, our EML rule is applied to solve the sametask as above with STDP. Differently, we do not impose anyconstraints on weights. Additionally, we evaluate our learningperformance over a broad range of initial setups, w initmean toexamine the learning reliability. Fig. 11 shows the learningperformance of our EML rule. Different values of w initmean canresult in a different initial firing rate ranging from 0 to tensof Hz. Our EML rule can successfully learn the task over allthe given setups without any failure, suggesting the reliabilityof our learning rule. On contrast, this broad range of setupswould be a disaster for STDP. For example, when there is nooutput spikes (with a small w initmean ), there will be no learningfor STDP at all. Differently, our learning rule can convergemuch faster than the unsupervised STDP learning.In the second task, we consider a more challenging onewhere multiple activity patterns are embedded in the back-ground. Similar to the task in [44], we set 6 patterns withhalf being selected as features and the rest as distractors.Different from the previous trial generation, we first generatea background pattern of 2 s, and then randomly insert activitypatterns into it. The occurrence number of each activity patternover the trial is randomly generated by a Poisson process withmean of 3. The time window of the resulting trial pattern thushas a mean value of 3.8 s. A background noise of 1 Hz is thenadded to the trial pattern. Each training cycle contains 100random trial patterns. We consider two scenarios where the Time (s) Cycles R e s p o n s e Cycles C v g . C y c l e s EML EMLC TDP MST0204060 c p u t i m e ( s ) ABC Fig. 12. Multiple feature detection and recognition with multi-spike learningrules. A , neuron’s membrane potential dynamics, after training (with the EMLrule), in response to a spike pattern stream where both features (shaded blue,orange and green) and distractors (shaded gray) are embedded in a randombackground activity. The desired output spike number for each target featureis { 1, 1, 1 } (top) and { 1, 2, 3 } (bottom). B , the response of the neuron toboth features (coded with the same color in A ) and background (black) alonglearning. The left and right present the corresponding tasks in A . C , learningefficiency of different multi-spike rules for the tasks of { 1, 1, 1 } (red) and { 1, 2, 3 } (purple). The top panel shows the number of training cycles untilconvergence, and the bottom is the average cpu time of a training cycle. Datawere collected over 100 independent runs. neuron is trained to fire a desired number of spikes in responseto each feature pattern as: { 1, 1, 1 } and { 1, 2, 3 } . The neuronis required to be silent to both distractors and background.The total desired output spike number n ∗ out in response to atrial pattern is n ∗ out = (cid:80) i c i d i where c i is the occurrencenumber of the i -th feature and d i is the corresponding desiredoutput spike number for this feature pattern. We record theconvergence cycle at the point where the difference betweenneuron’s actual and desired output response to all activitypatterns as well as background is less than 0.05 within 10consecutive cycles.Fig. 12 shows the learning performance. The multi-spikelearning can successfully detect multiple feature patterns thatare embedded in a complex background where both noiseand distractors are presented (Fig. 12A). Importantly, ourlearning rule successfully demonstrates the ability of dis-crimination in addition to detection within tens of trainingcycles (Fig. 12A,B). Fig. 12C shows the learning efficiencyof different rules on the feature detection task. The EMLC is relatively weak as compared to the others in terms ofconvergence cycles, but is comparative to that of TDP andMST in terms of cpu execution time. The challenging taskof feature detection involves a long duration of noise anddistractors where it is slightly harder to explore repetition withthe neuron’s current states (EMLC) than that with STS (theothers). As a result, our EMLC takes longer cycles to converge.Our EML is the most efficient one among all these multi-spikelearning rules. It is around × faster in terms of cpu timethan that of TDP and MST for both scenarios. This efficiencymakes our learning a potential candidate to benefit low-powerneuromorphic computing.IV. D ISCUSSIONS Central nervous systems use spikes for both informationtransmission and processing [1], [2]. Spikes are believed toplay an essential role in low-power consumption which wouldbe of great importance to benefit devices such as mobilesand wearables where energy consumption is one of the majorconcerns [3], [11]. The efficiency is now one of the majorbottlenecks of deep learning methods [6], and thus attractsmore attention to neuromorphic computing [10], [14], [30],[56], [57]. In this work, we make our contributions towardsthis direction.We first start with the spiking neuron model as it is thebuilding block for processing spike information. Differentspiking neuron models have been proposed with differentdegrees for describing details of biological systems [20], [22],[24], [25]. Complex models [22], [24] have a relatively highbiological plausibility, but are computationally inefficient as aresult. Simpler models [20], [25] are just more favorable inneuromorphic computing [30], [56], [58] due to their simplic-ity while being capable of processing spikes. In our work, wepresent a further simplified neuron model where unit impulsefunction is used to describe the effects of both synapticinput and firing output on neuron’s membrane potential. Weconvert this model to an SRM form based on which an event-driven scheme is then introduced for processing. Remarkableefficiency for both processing and learning is just obtained asa result of our model (e.g. Fig. 6B, 7B and 8C). Note that,our neuron model and learning can also benefit a step-basedcomputational scheme [12], [15], making it a favorable choicefor efficient processing and learning.The new family of multi-spike learning rules [28], [44] arerecently developed to train neurons with a desired numberof spikes rather than precise timings. These learning rulescould be preferable to others [45], [49], [59] for makingdecision and exploring temporal features from the signals.This is because the multi-spike rules do not require precisetiming to be specified as an instructor, and importantly can begeneralized to different coding schemes [44]. We first developan STS-based learning rule, i.e. EML, following a similarapproach as in [28], [44]. Importantly, our EML rule inheritsthe advantages of both MST and TDP, namely accuracy andefficiency of derivative evaluation, respectively (see Fig. 5).Differently, the simplicity of our EML makes it better thanthe other two for processing and learning. In order to further improve the processing efficiency, we propose the EMLC rulewhich has no dependence on STS, and thus accelerates theprocessing as a result. Our simulation results show that theEMLC is more efficient for association and recognition tasks(Fig. 6, 7 and 8) as a result of STS avoidance, while the EMLoutperforms the EMLC in the feature detection task becausecritical thresholds could help to extract repeating informationin a complex background.Classification is a typical ability of a learning system thatis widely studied in the field of neuromorphic computing[14], [46], [49], [56], [58]–[60]. We examine the ability ofour proposed rules on this task with multi-category beingconsidered. Our learning rules are highly robust against a widerange of different noise types, and importantly they are moreefficient than other learning rules (see Fig. 8). Since a singleneuron can be assigned to have different output spike numbersin response to different categories, our learning rules canthus empower single neurons to solve the multi-category tasks(Fig. 9). Importantly, this classification ability of our learningrules can be generalized to different spike coding schemes.Therefore, the efficiency, robustness and generalization of ourlearning rules make them a priority over others as a potentialspike-based classifier. With a proper encoding scheme, ourlearning rules show a promising performance on some real-world classification tasks [61].Useful information is often embedded in the streams ofsensory activities, making detection of feature informationa challenging task (also called credit-assignment problem)[28], [62], [63]. Numerous experiment data show that hu-man brain leverages Bayesian principles to analyze sensorystimuli and infer the hidden states from the complex envi-ronment. Moreover, the unsupervised STDP rule has beentheoretically proven as an approximation of the Expectation-Maximization(EM) algorithm in machine learning, which isoften used for parameter estimation with hidden states suchas Bayesian inference [64], [65]. Early studies demonstratethat STDP can find the starts of repeating feature patternsfrom a background in an unsupervised way [42], [54]. How-ever, the success of STDP depends on certain experimentalconditions such as domination of depression and high initialfiring response. Additionally, our new results show that thedetection would not be sustained if further learning occurs dueto the domination of depression. Recent study also proves thatunsupervised learning is fundamentally impossible for certaintasks [66]. On the other hand, our supervised learning rulesdemonstrate a reliability in successfully detecting embeddedfeatures over a broad range of different setups (Fig. 11). Im-portantly, the teacher signals are weak ones by only specifyingthe total number of spikes the neuron should elicit in responseto a spike pattern. The neuron can then automatically detect theoccurrence of each feature pattern. Moreover, discriminationof feature patterns can be accomplished in addition to detec-tion with the multi-spike learning rules. Again, our proposedlearning rule still outperforms the others in terms of efficiency(Fig. 12) as expected.It is important to note that our work is a preliminaryattempt to improve the capabilities of multi-spike learningrules by providing efficient alternatives. We provide systematic insight into various learning properties of our methods withsynthesized experiments, while leaving possible extension tolarger, more complex and practical problems unexplored inthis study. This leaves a room for future developments where aproper encoding scheme is required to convert external stimuliinto spikes [18], [26], [28], [35], [57]. Another limitation ofour work is that only single-layer learning is examined. Aslayered structure is inevitable in nervous systems and hasshown great importance in the success of deep learning [6],a potential direction for future research is thus to extend thelearning capability to multi-layer structures. In spite of variousearly efforts [13], [14], [34], it is still valuable and importantto examine the single-layered learning of spikes, in a wayto closely depict its performance boundary where a complexstructure would be unnecessary thanks to the computationaladvantages of spiking neurons [21], [35], [43].V. C ONCLUSION In this work, we proposed several new approaches towardsefficient processing and learning of spikes. Firstly, we in-troduced a simple spiking neuron model and then convertedit to an efficient form for computation where an event-driven scheme was presented. We highlighted the simplicityof our model for implementations as well as its efficiencyfor processing. Based on our neuron model, we proposed twoefficient multi-spike learning rules, namely EML and EMLC.Our results showed that both our rules are more efficient forspike processing than other baselines. In addition to efficiency,our learning rules are highly robust to strong noise of differenttypes. In our feature detection experiment, we re-examined theunsupervised STDP and found a new phenomenon of losingselectivity due to the domination of depression. In contrast, ourproposed rules demonstrated a reliable learning over a widerange of setups without specific constraints being imposed.Moreover, our efficient learning rules can not only detect thefeatures but also discriminate them. In summary, the simplic-ity, efficiency, robustness, generalization and computationalpower of our rules could make them a preferable choice inneuromorphic computing.R EFERENCES[1] E. R. Kandel, J. H. Schwartz, T. M. Jessell, S. A. Siegelbaum, and A. J.Hudspeth, Principles of Neural Science . McGraw-hill New York, 2000,vol. 4.[2] P. Dayan and L. F. Abbott, Theoretical Neuroscience . Cambridge, MA:MIT Press, 2001, vol. 806.[3] P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada,F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura et al. , “Amillion spiking-neuron integrated circuit with a scalable communicationnetwork and interface,” Science , vol. 345, no. 6197, pp. 668–673, 2014.[4] S. Thorpe, D. Fize, and C. Marlot, “Speed of processing in the humanvisual system,” nature , vol. 381, no. 6582, pp. 520–522, 1996.[5] F. Rosenblatt, “The perceptron: a probabilistic model for informationstorage and organization in the brain.” Psychological review , vol. 65,no. 6, p. 386, 1958.[6] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521,no. 7553, pp. 436–444, 2015.[7] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Bi-narized neural networks,” in Advances in neural information processingsystems , 2016, pp. 4107–4115. [8] C. S. Thakur, J. L. Molin, G. Cauwenberghs, G. Indiveri, K. Kumar,N. Qiao, J. Schemmel, R. Wang, E. Chicca, J. Olson Hasler et al. ,“Large-scale neuromorphic spiking array processors: A quest to mimicthe brain,” Frontiers in neuroscience , vol. 12, p. 891, 2018.[9] J. D. Victor and K. P. Purpura, “Metric-space analysis of spike trains:theory, algorithms and application,” Network: computation in neuralsystems , vol. 8, no. 2, pp. 127–164, 1997.[10] E. J. Fuller, S. T. Keene, A. Melianas, Z. Wang, S. Agarwal, Y. Li,Y. Tuchman, C. D. James, M. J. Marinella, J. J. Yang et al. , “Parallelprogramming of an ionic floating-gate memory array for scalable neu-romorphic computing,” Science , vol. 364, no. 6440, pp. 570–574, 2019.[11] Q. Xia and J. J. Yang, “Memristive crossbar arrays for brain-inspiredcomputing,” Nature materials , vol. 18, no. 4, p. 309, 2019.[12] D. F. Goodman and R. Brette, “The Brian Simulator,” Frontiers inNeuroscience , vol. 3, no. 2, p. 192, 2009.[13] D. Liu and S. Yue, “Event-Driven Continuous STDP Learning WithDeep Structure for Visual Pattern Recognition,” IEEE Transactions onCybernetics , vol. 49, no. 4, pp. 1377–1390, APR 2019.[14] N. Zheng and P. Mazumder, “Online supervised learning for hardware-based multilayer spiking neural networks through the modulation ofweight-dependent spike-timing-dependent plasticity,” IEEE transactionson neural networks and learning systems , vol. 29, no. 9, pp. 4287–4302,2017.[15] M.-O. Gewaltig and M. Diesmann, “NEST (NEural Simulation Tool),” Scholarpedia , vol. 2, no. 4, p. 1430, 2007.[16] A. R. Young, M. E. Dean, J. S. Plank, and G. S. Rose, “A review ofspiking neuromorphic hardware communication systems,” IEEE Access ,vol. 7, pp. 135 606–135 620, 2019.[17] A. Balaji, A. Das, Y. Wu, K. Huynh, F. G. DellAnna, G. Indiveri, J. L.Krichmar, N. D. Dutt, S. Schaafsma, and F. Catthoor, “Mapping spikingneural networks to neuromorphic hardware,” IEEE Transactions on VeryLarge Scale Integration (VLSI) Systems , vol. 28, no. 1, pp. 76–86, 2019.[18] S. Panzeri, N. Brunel, N. K. Logothetis, and C. Kayser, “Sensory neuralcodes using multiplexed temporal scales,” Trends in Neurosciences ,vol. 33, no. 3, pp. 111–120, 2010.[19] M. J. Rowan, G. DelCanto, J. Y. Jianqing, N. Kamasawa, and J. M.Christie, “Synapse-level determination of action potential duration byk+ channel clustering in axons,” Neuron , vol. 91, no. 2, pp. 370–383,2016.[20] W. Gerstner and W. M. Kistler, Spiking neuron models: Single neurons,populations, plasticity . Cambridge university press, 2002.[21] W. Maass, “Networks of spiking neurons: the third generation of neuralnetwork models,” Neural Networks , vol. 10, no. 9, pp. 1659–1671, 1997.[22] A. L. Hodgkin and A. F. Huxley, “A quantitative description of mem-brane current and its application to conduction and excitation in nerve,” The Journal of physiology , vol. 117, no. 4, pp. 500–544, 1952.[23] Q. Kang, B. Huang, and M. Zhou, “Dynamic behavior of artificialHodgkin–Huxley neuron model subject to additive noise,” IEEE Trans-actions on Cybernetics , vol. 46, no. 9, pp. 2083–2093, 2016.[24] E. M. Izhikevich, “Simple model of spiking neurons,” IEEE Transactionson neural networks , vol. 14, no. 6, pp. 1569–1572, 2003.[25] A. N. Burkitt, “A review of the integrate-and-fire neuron model: I.homogeneous synaptic input,” Biological cybernetics , vol. 95, no. 1,pp. 1–19, 2006.[26] Q. Yu, H. Tang, K. C. Tan, and H. Li, “Rapid feedforward computationby temporal encoding and learning with spiking neurons,” IEEE trans-actions on neural networks and learning systems , vol. 24, no. 10, pp.1539–1552, 2013.[27] R.-M. Memmesheimer, R. Rubin, B. P. ¨Olveczky, and H. Sompolinsky,“Learning precisely timed spikes,” Neuron , vol. 82, no. 4, pp. 925–938,2014.[28] R. G¨utig, “Spiking neurons can discover predictive features byaggregate-label learning,” Science , vol. 351, no. 6277, p. aab4113, 2016.[29] Q. Yu, R. Yan, H. Tang, K. C. Tan, and H. Li, “A spiking neural networksystem for robust sequence recognition,” IEEE Transactions on NeuralNetworks and Learning Systems , vol. 27, no. 3, pp. 621–635, 2016.[30] Q. Yu, H. Tang, J. Hu, and K. C. Tan, Neuromorphic CognitiveSystems: A Learning and Memory Centered Approach , 1st ed. SpringerPublishing Company, Incorporated, 2017.[31] R. G¨utig, “To spike, or when to spike?” Current Opinion in Neurobiol-ogy , vol. 25, pp. 134–139, 2014.[32] R. B. Stein, E. R. Gossen, and K. E. Jones, “Neuronal variability: Noiseor part of the signal?” Nature Reviews. Neuroscience , vol. 6, no. 5, p.389, 2005.[33] M. London, A. Roth, L. Beeren, M. H¨ausser, and P. E. Latham,“Sensitivity to perturbations in vivo implies high noise and suggestsrate coding in cortex,” Nature , vol. 466, no. 7302, p. 123, 2010. [34] Y. Cao, Y. Chen, and D. Khosla, “Spiking deep convolutional neuralnetworks for energy-efficient object recognition,” International Journalof Computer Vision , vol. 113, no. 1, pp. 54–66, 2015.[35] J. J. Hopfield, “Pattern recognition computation using action potentialtiming for stimulus representation,” Nature , vol. 376, no. 6535, pp. 33–36, 1995.[36] A. Borst and F. E. Theunissen, “Information theory and neural coding,” Nature neuroscience , vol. 2, no. 11, pp. 947–957, 1999.[37] H. Mostafa, “Supervised learning based on temporal coding in spikingneural networks,” IEEE transactions on neural networks and learningsystems , vol. 29, no. 7, pp. 3227–3235, 2017.[38] S. Song, K. D. Miller, and L. F. Abbott, “Competitive hebbian learn-ing through spike-timing-dependent synaptic plasticity,” Nature neuro-science , vol. 3, no. 9, p. 919, 2000.[39] G.-q. Bi and M.-m. Poo, “Synaptic modification by correlated activity:Hebb’s postulate revisited,” Annual review of neuroscience , vol. 24,no. 1, pp. 139–166, 2001.[40] Y. Dan and M.-m. Poo, “Spike timing-dependent plasticity of neuralcircuits,” Neuron , vol. 44, no. 1, pp. 23–30, 2004.[41] T. Masquelier and S. J. Thorpe, “Unsupervised learning of visualfeatures through spike timing dependent plasticity,” PLoS computationalbiology , vol. 3, no. 2, p. e31, 2007.[42] T. Masquelier, R. Guyonneau, and S. J. Thorpe, “Spike timing dependentplasticity finds the start of repeating patterns in continuous spike trains,” PloS one , vol. 3, no. 1, p. e1377, 2008.[43] R. G¨utig and H. Sompolinsky, “The tempotron: a neuron that learnsspike timing-based decisions,” Nature Neuroscience , vol. 9, no. 3, pp.420–428, Feb. 2006.[44] Q. Yu, H. Li, and K. C. Tan, “Spike timing or rate? neurons learnto make decisions for both through threshold-driven plasticity,” IEEEtransactions on cybernetics , vol. 49, no. 6, pp. 2178–2189, 2019.[45] S. M. Bohte, J. N. Kok, and J. A. L. Poutr´e, “Error-backpropagationin temporally encoded networks of spiking neurons,” Neurocomputing ,vol. 48, no. 1-4, pp. 17–37, 2002.[46] F. Ponulak and A. Kasi´nski, “Supervised learning in spiking neuralnetworks with resume: sequence learning, classification, and spikeshifting,” Neural computation , vol. 22, no. 2, pp. 467–510, 2010.[47] R. V. Florian, “The Chronotron: A Neuron that Learns to Fire Tem-porally Precise Spike Patterns,” PLoS One , vol. 7, no. 8, p. e40233,2012.[48] A. Mohemmed, S. Schliebs, S. Matsuda, and N. Kasabov, “SPAN:Spike Pattern Association Neuron for Learning Spatio-Temporal SpikePatterns,” International Journal of Neural Systems , vol. 22, no. 04, 2012.[49] Q. Yu, H. Tang, K. C. Tan, and H. Li, “Precise-spike-driven synapticplasticity: Learning hetero-association of spatiotemporal spike patterns,” PLoS One , vol. 8, no. 11, p. e78318, 2013.[50] R. Hu, S. Chang, H. Wang, J. He, and Q. Huang, “Efficient multispikelearning for spiking neural networks using probability-modulated timingmethod,” IEEE transactions on neural networks and learning systems ,vol. 30, no. 7, pp. 1984–1997, 2018.[51] R. Brette, “Philosophy of the spike: rate-based vs. spike-based theoriesof the brain,” Frontiers in Systems Neuroscience , vol. 9, p. 151, 2015.[52] Q. Yu, Y. Yao, L. Wang, H. Tang, J. Dang, and K. C. Tan, “Robustenvironmental sound recognition with sparse key-point encoding andefficient multi-spike learning,” IEEE Transactions on Neural Networksand Learning Systems , in press, 2020.[53] Q. Yu, L. Wang, and J. Dang, “Efficient multi-spike learning withtempotron-like ltp and psd-like ltd,” in International Conference onNeural Information Processing . Springer International Publishing,2018, pp. 545–554.[54] T. Masquelier, R. Guyonneau, and S. J. Thorpe, “Competitive stdp-basedspike pattern learning,” Neural computation , vol. 21, no. 5, pp. 1259–1276, 2009.[55] A. Sen, M. M. Islam, K. Murase, and X. Yao, “Binarization with boost-ing and oversampling for multiclass classification,” IEEE Transactionson Cybernetics , vol. 46, no. 5, pp. 1078–1091, 2016.[56] J. J. Wade, L. J. McDaid, J. A. Santos, and H. M. Sayers, “Swat: aspiking neural network training algorithm for classification problems,” IEEE Transactions on Neural Networks , vol. 21, no. 11, pp. 1817–1830,2010.[57] W. Woods and C. Teuscher, “Fast and accurate sparse coding ofvisual stimuli with a simple, ultralow-energy spiking architecture,” IEEEtransactions on neural networks and learning systems , vol. 30, no. 7,pp. 2173–2187, 2018.[58] Y. Zheng, S. Li, R. Yan, H. Tang, and K. C. Tan, “Sparse temporalencoding of visual features for robust object recognition by spiking neurons,” IEEE transactions on neural networks and learning systems ,vol. 29, no. 12, pp. 5823–5833, 2018.[59] J. M. Brader, W. Senn, and S. Fusi, “Learning real-world stimuliin a neural network with spike-driven synaptic dynamics,” Neuralcomputation , vol. 19, no. 11, pp. 2881–2912, 2007.[60] A. Jeyasothy, S. Sundaram, and N. Sundararajan, “Sefron: A new spikingneuron model with time-varying synaptic efficacy function for patternclassification,” IEEE transactions on neural networks and learningsystems , vol. 30, no. 4, pp. 1231–1240, 2018.[61] L. Shenglan and Y. Qiang, “New efficient multi-spike learning for fastprocessing and robust learning,” in AAAI , 2020.[62] W. Schultz, P. Dayan, and P. R. Montague, “A neural substrate ofprediction and reward,” Science , vol. 275, no. 5306, pp. 1593–1599,1997.[63] J. Friedrich, R. Urbanczik, and W. Senn, “Spatio-temporal credit assign-ment in neuronal population learning,” PLoS Computational Biology ,vol. 7, no. 6, p. e1002092, 2011.[64] S. Guo, Z. Yu, F. Deng, X. Hu, and F. Chen, “Hierarchical bayesianinference and learning in spiking neural networks,” IEEE transactionson cybernetics , vol. 49, no. 1, pp. 133–145, 2019.[65] B. Nessler, M. Pfeiffer, L. Buesing, and W. Maass, “Bayesian compu-tation emerges in generic cortical microcircuits through spike-timing-dependent plasticity,” PLoS computational biology , vol. 9, no. 4, 2013.[66] F. Locatello, S. Bauer, M. Lucic, S. Gelly, B. Sch¨olkopf, and O. Bachem,“Challenging common assumptions in the unsupervised learning ofdisentangled representations,” arXiv preprint arXiv:1811.12359 , 2018. Qiang Yu (M’12) received the B.Eng. degreein electrical engineering and automation from theHarbin Institute of Technology, Harbin, China, in2010, and the Ph.D. degree in electrical and com-puter engineering from the National University ofSingapore, Singapore, in 2014.He is an Associate Professor with the Collegeof Intelligence and Computing, Tianjin University,Tianjin, China. Before that, he was a Post-DoctoralResearch Fellow with the Max-Planck-Institute forExperimental Medicine, G¨ottingen, Germany, from2014 to 2016, and a Research Scientist in the Institute for Infocomm Research,Agency for Science, Technology and Research, Singapore, from 2016. He isa recipient of the 2016 IEEE Outstanding TNNLS Paper Award. His currentresearch interests include learning algorithms in spiking neural networks,neural coding, cognitive computations and machine learning. Shenglan Li received her B.Sc. degree in ComputerScience and Technology from Beijing University ofChemical Technology. She is currently a graduatestudent now studying in Tianjin University, Tian-jin, China. Her current research interests includelearning algorithms in spiking neural network, neuralencoding and machine learning. Huajin Tang received the B.Eng. degree from Zhe-jiang University, China in 1998, received the M.Eng.degree from Shanghai Jiao Tong University, Chinain 2001, and received the Ph.D. degree from theNational University of Singapore, in 2005.He is currently a professor at Zhejiang University,China. His research work on Brain GPS has beenreported by MIT Technology Review in 2015. Hereceived the 2016 IEEE Outstanding TNNLS PaperAward. His current research interests include neu-romorphic computing, neuromorphic hardware andcognitive systems, robotic cognition, etc. Dr. Tang has served as an AssociateEditor of IEEE Transactions on Neural Networks and Learning Systems,IEEE Transactions on Cognitive and Developmental Systems and Frontiersin Neuromorphic Engineering. He was the Program Chair of the 6th and7th IEEE CIS-RAM, and Chair of 2016 and 2017 IEEE Symposium onNeuromorphic Cognitive Computing. Longbiao Wang received his Dr. Eng. degree fromToyohashi University of Technology, Japan, in 2008.He was an assistant professor in the faculty of en-gineering at Shizuoka University, Japan from April2008 to September 2012. From October 2012 toAugust 2016 he was an associate professor at Na-gaoka University of Technology, Japan. Currently heis a Professor at the Tianjin University, China. Hisresearch interests include robust speech recognitionand speaker recognition. Jianwu Dang (M’12) graduated from TsinghuaUniv., China, in 1982, and got his M.S. degree atthe same university in 1984. He worked for TianjinUniv. as a lecture from 1984 to 1988. He wasawarded the PhD degree from Shizuoka Univ., Japanin 1992. He worked for ATR Human InformationProcessing Labs., Japan, as a senior researcher from1992 to 2001. He joined the University of Waterloo,Canada, as a visiting scholar for one year from 1998.Since 2001, he has worked for Japan AdvancedInstitute of Science and Technology (JAIST) as aprofessor. He joined the Institute of Communication Parlee (ICP), Center ofNational Research Scientific, France, as a research scientist the first class from2002 to 2003. Since 2009, he has joined Tianjin University, Tianjin, China.His research interests are in all the fields of speech science including brainscience, and speech signal processing. He built MRI-based bio-physiologicalmodels for speech and swallowing, and endeavors to apply these models onclinics.