Frequency selectivity of neural circuits with heterogeneous transmission delays
FFrequency selectivity of neural circuits with heterogeneous transmissiondelays
Akke Mats HoubenSeptember 22, 2020
Abstract
Neurons are connected to other neurons by axons and dendrites that conduct signals with finite velocities,resulting in delays between the firing of a neuron and the arrival of the resultant impulse at other neurons. Sincedelays greatly complicate the analytical treatment and interpretation of models, they are usually neglected ortaken to be uniform, leading to a lack in the comprehension of the effects of delays in neural systems. Thispaper shows that heterogeneous transmission delays make small groups of neurons respond selectively to inputswith differing frequency spectra. By studying a single integrate-and-fire neuron receiving correlated time-shiftedinputs, it is shown how the frequency response is linked to both the strengths and delay times of the afferentconnections. The results show that incorporating delays alters the functioning of neural networks, and changesthe effect that neural connections and synaptic strengths have. a r X i v : . [ q - b i o . N C ] S e p ntroduction Eventhough the brain is quick to respond, neurons areconnected through axons and dendrites that propagatesignals with non negligible and replicable delays (Swad-low, 1985, 1994). The transmission delay between twoneurons depends on the conduction velocity, related tothe diameter of the axon or dendrite (Cullheim, 1978;Cullheim and Ulfhake, 1979; Gasser and Grundfest, 1939;Lee et al., 1986; Waxman, 1980) and the properties ofthe axon and dendrites (Harper and Lawson, 1985; Wax-man, 1980), in combination with the lengh of the patha pulse travels from one neuron to the other ( time = distance/speed ). Conduction delays have been shown tobe plastic (Bakkum et al., 2008), indicating that condu-tion delays are tuneable to some extent. On the otherhand, the conduction velocities have been found to beactivity dependent (Swadlow, 1974; Thalhammer et al.,1994; De Col et al., 2008). Still, given that the re-sponse times of neurons to natural stimuli are robust(Mainen and Sejnowski, 1995), the transmission delaytimes should be stable over time (Swadlow, 1985).Because of the difficulties involved in the analyticaltreatment of delays, they are often either neglected ortaken to be uniform. This has lead to a lack in thecomprehension of the effects of (heterogeneous) delays inneural systems and the absence of delays in principal ac-counts of the working of neural networks. Great insightsinto the functioning and dynamics have been gained bystudying networks without (heterogeneous) transmissiondelays, such as the understanding of the on- and offset ofoscillatory activity (e.g. Wang and Buzs´aki, 1996; Wil-son and Cowan, 1972), signal propagation (e.g. Mehringet al., 2003; Reyes, 2003; Vogels and Abbott, 2009), ac-tivity dynamics (e.g. Destexhe, 2009; Renart et al., 2010;Van Vreeswijk and Sompolinsky, 1996), and memory stor-age (e.g. Amit et al., 1985; Anderson, 1972; Brunel, 2016;Hopfield, 1982; Klampfl and Maass, 2013).However, there are cases in which the incorporationof transmission delays can lead to drastically differentrestults. Organisms, and with that their brains, oper-ate in time and as such it is likely there is an impor-tance in the temporal dimension of the activity of thebrain. It has already been advocated that transmissiondelays endow neural networks with much richer dynam-ics, increasing their functional capacity and possible dy-namics (e.g. Chapeau-Blondeau and Chauvet, 1992; Des-texhe, 1994a; Izhikevich, 2006; Ostojic, 2014), enablingneural communication based on synchrony or spike or-dering (e.g. Brette, 2012; Gautrais and Thorpe, 1998;Thorpe, 1990), and allowing oscillations and synchroni-ation (e.g. Buzs´aki and Draguhn, 2004; Destexhe, 1994b;Ernst et al., 1995; Geisler et al., 2010; Maex and De Schut-ter, 2003; Van Vreeswijk et al., 1994). This paper con-cerns the connection between transmission delays and thefrequencies to which a neuron responds. In particular itexposes the concerted effect of synaptic strenghts and synaptic delays on the frequency selectivity of neurons.Synaptic connections between (excitatory) neuronsare principally understood in light of some variant ofa constitutive Hebbian learning process. The resultantview is that for two neurons to be connected by an ex-citatory connection it means that it is likely to observea co-occurence of their spiking activity, and the strongerthis connection, the more likely they will fire together. Asynaptic connection can also be understood through thecausal effect of a pre-synaptic neuron on the activity of apost-synaptic cell, determined by the synaptic strenghtand pre-synaptic neuron type. Excitatory pre-synapticneurons have a depolarising effect and thus promote thepost-synaptic firing, in general leading to higher post-synaptic firing rates. Inhibitory neurons lead to hyper-polarisation and thus inhibit or delay post-synaptic fir-ing, leading to reduced post-synaptic firing rates. Theseeffects are more pronounced for stronger connections.However, by the inclusion of heterogeneous trans-mission delays one can conceptualise a small circuit ofneurons as a cascade of filters with time-delayed andweighted coupling. Seen in this way the properties ofthe afferent connections determine the sub-threshold fre-quency response of a neuron, with the synaptic weightsfunctioning as feed-forward coefficients in a neural filter-ing circuit and thus influencing the frequency selectivityof the post-synaptic neuron. This means that the effectof synaptic strengths differs from the one given above,and that the effect of a single synapse cannot be com-pletely understood in isolation, but gets a significantlydifferent interpretation when considered in an ensembleof synapses conveing correlated signals.This paper explores this concept with a single integrate-and-fire neuron receiving correlated and time-shifted in-puts. It is shown that the sub-threshold frequency re-sponse of a neuron is determined by the strengths andrelative delay times of the correlated afferent connec-tions. The characteristics of the frequency response ofsome cases are solved exactly, and qualitative observa-tions are given for more general cases. Subsequently,numerical simulations demonstrate the functional signif-icance of this frequency selectivity, the frequency speci-ficity, and show that the described effects hold for a rangeof input correlations. Results
To show the basic principle, consider a leaky integrate-and-fire (IF) neuron, with a membrane potential gov-erned by: τ dvdt + v = G (cid:32) I + M (cid:88) m =1 w m I ( m ) (cid:33) , (1)in which τ is the membrane time constant. This equa-tion is complemented with a spike-and-reset rule: once2 -1 -1 -1 ( m s r a d - ) F r e q u e n c y ( r a d s - ) |H| Figure 1:
Sub-thresold frequency spectrum of (1)measured for different time-constants τ (in s · rad − ),showing the low-pass filtering effect intrinsic to the mem-brane equationthe membrane potential v exceeds a threshold v th , themembrane potential is directly reset to a reset value v r ,after which the membrane potential is again directly gov-erned by (1).The input is given by the term G (cid:0) I + (cid:80) m w m I ( m ) (cid:1) ,which denotes the sum of a stochastic input I with corre-lated and time-shifted inputs I ( m ) , each of which is mul-tiplied by a weight w m . The input can be interpreted asthe output of M similar neurons or groups of neuronsthat receive strongly correlated input and so producehighly correlated outputs, arriving through connectionswith different delay times due to the different finite con-duction velocities of the different axons and dendrites.The term G is a gain factor which, since the frequencyresponse does not qualitatively depend on the absolutegain but on the relative strenghts of the individual in-puts, will be set to be unity in the following analysis ofthis system.Equation (1) without the spiking mechanism (or with v th >> (cid:104) I (cid:105) ) is linear and time-invariant, as such the sub-threshold frequency spectrum of v is given by the productof the intrinsic frequency response 1 / (1 + τ s ), with s being a complex frequency of the form s = α + i ω , andthe frequency spectrum of the input (Oppenheim et al.,1997; Smith, 2007): (cid:101) v ( s ) = 11 + τ s (cid:16)(cid:101) I ( s ) + (cid:88) w m (cid:101) I ( m ) ( s ) (cid:17) , (2)from which we see that the membrane equation has apole on the real axis at s = − τ − , corresponding tothe characteristic decaying response of the leaky IF neu-ron (Stein, 1965; Gluss, 1967). Thus the intrinsic fre-quency response of the membrane acts as a low-pass filterwith time constant τ . Figure 1 shows the intrinsic sub-threshold frequency response of the membrane equation(1) for different values of τ .The frequency transfer function H ( s ) of linear time-invariant systems can be found by dividing the output spectrum by the input spectrum, H ( s ) := (cid:101) V / (cid:101) I (Oppen-heim et al., 1997). Requiring that all the input spiketrains are strongly correlated ( ρ = 1) for different posi-tive lag times, I ( m ) ( t ) = I ( t − d m ) for all m , with d m > (cid:101) I leads to the sub-threshold transferfunction of the membrane equation (1): H ( s ) = (cid:0) (cid:80) w m e − d m s (cid:1) τ s . (3) Frequency response
The frequency response of the neuronal circuit can beunderstood by finding the roots of the denominator andthe nominator of the transfer function (3), which willrespectively give the positions of the poles and zerosof the frequency response. Since we are not occupiedwith feedback components other than the intrinsic mem-brane dynamics, (3) has just a single pole which positionis, as shown above, determined by the membrane time-constant and lies at p = − /τ . The rate of decay ofthe membrane potential is proportional to the distanceof this pole to the origin, which can also be seen fromthe homogeneous solution to (1).The zeros of (3) correspond to frequencies at which H ( s ) vanishes, corresponding to the input frequencies towhich the neuron responds minimally, so the roots of thenumerator correspond to frequencies that are attenuatedby the circuit. Since d m ∈ R it is difficult to find exactexpressions for the roots of (3). Exact analysis of theroots in general cases is thus beyond the scope of thispaper, but it is straightforward to plot (3) in order togain a qualitative insight into the frequency response.However it will be instructive to treat some cases inwhich the roots can be obtained analytically. For thefolowing it is assumed that each delay time d m is an in-teger multiplying some basic time unit δ = 1 / (2 πf max )in sec · rad − . The highest frequency to be analysed isdetermined by f max , or conversely f max can be deter-mined by requirement on δ to fit the desired values of d m . In the following a normalised frequency will be used( f max will be normalised to unity) such that δ = 1 / (2 π ) sec · rad − . Two specific cases will be treated analyti-cally: a neuron receiving one additional input ( M = 2)for a delay time d >
0, resulting in a ‘comb filtering’,and the case of a neuron receiving 2 additional inputs( M = 3) with d = 2 d , d > d = nd , for n ∈ N . Afterwards some qualitativeobservations about the zeros of (3) will be made.In the case of M = 2 the nominator of (3) has peri-3dically distributed roots z n = − w d e i (2 n + k ) πd , (4)for n = 1 , , , ... and k = (cid:40) , d even0 , d odd,from which we immediately see that the attenuated fre-quencies (given by the the angles ∠ z n ) are determinedby the delay time d , and that the weight w only influ-ences the amount of attenuation (given by the magni-tudes | z n | ). Figure 2 shows the frequency responses forsome different values for d , also demonstrating the reasonfor the conventionally used name ‘comb filter’.A more interesting case is to analyse the zeros of thesub-threshold transfer function H ( s ) = (cid:0) w e − ds + w e − ds (cid:1) τ s , (5)describing a neuron receiving M = 3 inputs with har-monically related delays (see figure 3a), which can beobtained exactly by defining σ ( s ) = e ds and multiplyingthe transfer function by σ σ . This allows to rewrite thenominator N ( s ) of the transfer function as N ( σ ) = (cid:0) σ + w σ + w (cid:1) , which is quadratic in σ . Now any standard strategy toobtain the roots of N ( σ ) can be employed, leading to: σ = − w ± (cid:112) w − w . Substituting back e ds for σ , we can use a similar formulaas in the case of M = 2, leading to z n = − (cid:32) − w ± (cid:112) w − w (cid:33) d e i (2 n +1 − k ) πd , with n and k the same as in (4), showing that the zerosof (5) repeat with a period 1 /d , and that each of theseintervals contains 2 zeros. In this case the frequenciesat which the zeros are positioned are influenced by boththe delay times and the connection weights. Notice thatsince each w m is required to be real, the complex zerosoccur in conjugate pairs. Figures 3b and 3c show thefrequency responses of for some values for d and w .The periodicity of the zeros extends also to a moregeneral case where d is an integer multiple of d . In thiscase the zeros repeat again with period 1 /d , but noweach interval contains d /d zeros.As said before, in general it will be difficult to de-termine the zeros of (3) exactly. Still some of the aboveobservations can be extended by graphing the magnitudeof H ( s ) with s = i ω with respect to ω . The first obser-vation is that the number of zeros in the [0 , π ) interval is determined by the longest delay time. More specifi-cally: longer delay times lead to a more rapid successionof zeros. Indeed, this can be inferred from the exacttreatment in the last section, in which higher order nu-merators lead to a faster sucession of zeros. The secondobservation is that when the delays have a harmonic re-lationship, with each d m being an integer multiple of d ,the pattern of zeros occurs periodically with a period of1 /d . Thirdly, in case of complex roots, these roots haveto occur in conjugate pairs, since the weights are definedto be real. Finally, the period of repetition of zeros isdetermined by the delay times, but crucially in case ofcomplex roots, the weights determine the exact attenu-ated frequency within each interval. This leads to theimportant observation that synaptic plasticity not onlyalters the susceptibility, and with that the frequency, ofpost-synaptic firing, but alters the frequency selectivityof the neuronal circuit. Discrimination of inputs
The filtering capabilities endowed by heterogeneous trans-mission delays are not purely theoretic, but can be shownto have a definite effect for the functioning of neuralcircuits. Driving two neurons with differing frequencyselectivity (see methods section for the neuron param-eters) with a communal input that consists of a whitenoise during the first 200 ms and subsequently alternatesbetween two filtered white noise signals, each matchingthe frequency response of one of the neurons, shows thatneurons respond selectively to their matched input (seefigure 4). Whereas both neurons respond with low fir-ing rates during the white-noise input interval (on av-erage 0 . σ = 0 .
00) spikes/second for neuron A and0 .
04 ( σ = 0 .
01) spikes/second for neuron B. During thematched input intervals a clear distinction between theoutputs of the two neurons is observed (36 .
75 ( σ = 3 . .
03 ( σ = 0 .
20) spikes/secondfor neuron B during A-matched input ( t (998) = 468 . , p =0 . .
03 ( σ = 1 .
13) spikes/second for neuron A versus53 .
66 ( σ = 5 .
12) spikes/second for neuron B during B-matched input ( t (998) = − . , p = 0 . t (499) = − . , p = 0 .
00 for neuron A, t (499) = − . , p = 6 . e −
125 for neuron B). In order to visu-alise the specificty of neuronal filtering circuits, neuronalcircuits with sub-threshold frequency responses whichonly pass a narrow band of frequencies are stimulatedwith narrow band filtered white noise input with differ-ent center band frequencies (see methods for the neu-ron parameters). Figure 5 shows that each neuron re-sponds primarily to their matched input, examplified bythe maximal response values in the diagonal entries, andthat the responses diminish rapidly with different center4 d IwI (1) (a) -1 b w F r e q u e n c y ( r a d s - ) |H| (b) -1 c w F r e q u e n c y ( r a d s - ) |H| (c) Figure 2: ‘Comb’-filtering : a) schematic drawing indicating neuron wiring receiving 2 inputs with differing delaytimes d < d , due to differing path lengths; b-c) measured sub-threshold frequency spectra for different synapticweight w values, for two different delay times Iw I (1) w I (2) d d d (a) -1 b w ( r a d ) F r e q u e n c y ( r a d s - ) |H| (b) -1 c w ( r a d ) F r e q u e n c y ( r a d s - ) |H| (c) Figure 3: ‘Quadratic’ filters : a) schematic of neuron wiring; b-c) measured sub-threshold frequency spectra forneurons receiving 3 inputs with w = 1 and different values for w , for two different delay times d , and d = 2 d I A I B A B d a d a d a d b d b d b d b d b d b d a d a d a (a) -505 random unmatched matched b m e a n fi r i n g r a t e ( z - s c o r e ) A B (b) AB c -101 0 1s fi r i n g r a t e ( z - s c o r e ) A B (c)
Figure 4:
Frequency selectivity : a) schematic indicating neuron wiring. Both neurons receive input both froma source matched to their own sub-threshold frequency selectivity as well as from a source matched to the otherneuron. One neuron receives, through an identical set of M input-lines, both the matched and unmatched source.In the figure neuron A receives M = 3 inputs (indicated by the solid blue input lines) from source I A with threedifferent delay-times ( d a < d a < d a ) and weights ( w a , w a and w a ), and through the same synapses the inputof source I B (indicated by the solid orange input lines). Neuron B receives input from source I A (dashed blueinput lines) and I B (dashed orange input lines) with the same connection parameters between sources, but differentfrom those of neuron A. b) box-plots indicating difference in firing rates (transformed into z-scores) of each neuronduring white noise (left 2 box-plots), A-matched (middle box-plots) and B-matched (right most plots) input. c)spike raster and averaged spike-rates of neuron A and B in response to a white-noise input (white area), A-matchedinput (blue areas) and B-matched input (orange areas). Top plot shows the spike timings of 500 repetitions of thesame trial. Bottom plot indicates the average firing rate (transformed to z-scores) of the 500 repetitions per neuron.5 input center frequency0 s y n a p t i c c e n t e r f r e q u e n c y r i n g r a t e ( s p i k e s / t ) Figure 5:
Specificity of neuron response : output fir-ing rate of neurons with narrow band-pass frequency se-lectivity ( M = 13), with different center-frequencies (x-axis), in response to band-pass filtered white noise inputwith different center-frequencies (y-axis). Diagonal en-tries correspond to matched input and synaptic-filteringcenter-frequency. Off-diagonal entries correspond to in-creasing disparity between the synaptic-filter and inputcenter-frequenciesfrequencies, as shown by a rapid diminishing of activ-ity in the off-diagonal entries. Neuronal circuits withsepecific wiring are thus capable to respond strongly tomatching frequency inputs, while supressing their inputto non-frequency matched inputs. In order to suppressmost of the frequency spectrum and only pass an increas-ingly narrow band, a neuron needs to receive more andmore inputs. However the neurons simulated for figure 5received only 13 inputs, much less than the estimates ofthe average number of inputs received by neurons. Robustness for ρ (cid:54) = 1 Throughout this paper it was assumed that the inputsthe post-synaptic neuron receives through the differentsynapses were perfectly correlated ρ = 1. Eventhoughthe intrinsic noise levels of neurons are low (Mainen andSejnowski, 1995), the great number and diversity of synap-tic inputs still likely leads to each neuron being subjectedto a ‘unique’ noise source. Indeed correlations in the out-put of any two neurons are generally weak (Cohen andKohn, 2011). Perfect correlation is thus, in general, un-likely.In order to investigate the tolerance to non-perfectlycorrelated inputs, a neuron is simulated receiving 2 addi-tional inputs M = 3, with different correlation betweenthe inputs arriving through the different delay lines, thus I ( m ) = ρI + (1 − ρ ) η m , (6)with η m being a similar noise source as I . Fourier trans-forming the sub-threshold responses shows, as visible infigure 6a, that with decreasing correlation the shape of the ‘measured’ sub-threshold frequency spectrum quicklyreduces to that of the intrinsic sub-threshold response ofthe membrane equation (equal to the spectrum on theleft for ρ = 0). This suggests that even relatively smalldisrelations between the inputs abolishes the frequencyselectivity of a neuron described in this paper.However, again simulating the two neurons adaptedto discriminate between two spectrally different inputs,but this time with each neuron receiving white noise in-put with strength (1 − ρ ) in addition to the spectrallyshaped input (with strength ρ ), leads to a suprising find-ing. Figures 6b and 6c show the results as in figures 4band 4c, but now for different values of input correlation.These results show that functionally the frequency selec-tive effect due to the input parameters is still present forcorrelations lower ( ρ = .
2) than for which the frequencyresponse shaping effect is visually prominent from thesub-threshold frequency spectrum (c.f. figure 6a).Thus importantly, eventhough from the ‘measured’frequency spectra the effect of the inputs on the fre-quency selectivity seemed to be negligible, the frequencyselectivity of the neurons are functionally still signifi-cantly shaped by the input parameters, and this fre-quency selectivity is predictable by the theory presentedin this paper.
Discussion
This paper shows that delays in the transmission of sig-nals between neurons have a determinate effect on thefrequency response of small neural circuits, making itpossible for neural networks to act as finite impulse re-sponse filters. It is shown that the length of the delaytime but importantly also the strengths of the connec-tions determine the frequency selectivity, and that thefrequency response of a neuron due to its afferent connec-tions can be characterised by the analysis of the strengthsand delay times of the incoming connections. Numericalsimulations demonstrated that neural networks can beconstructed in which neurons with differing connectionsfrom overlapping input sources can differentially respondto spectrally different inputs, and can do so with highspecificity, thus opening up the frequency domain for us-age in neural communication. Finally, it is found thatwith diminishing input correlation the measured sub-threshold frequency spectra do not show clear signs ofthe effects of the input parameters, but the frequencyselectivity is still functionally present.The idea of filtering by neurons is not new: a receptivefield is essentially a filter. It is however important to notethat the frequency filtering as treated in this paper is dis-tinct from a filtering of information (as a receptive fielddoes). The filtering described here is a power filtering:the attenuation (and accentuation) of the magnitudes ofcertain frequencies, leading to differential responses toinputs with different frequencies, which does not neces-6
0 0.2 0.4 0.6 0.8 11/4 π π π π π ρ F r e qu e n c y ( r a d ) |H(s)| (a) -1-0.500.51 p=3e-01 p=6e-53 p=8e-109 p=3e-125 p=3e-128 p=2e-124 m e a n fi r i n g r a t e s ( z - s c o r e ) correlationmatchedunmatched (b) i n p u t c o rr e l a t i o n -0.500.5 0 1s fi r i n g r a t e ( z - s c o r e ) time (c) Figure 6:
Robustness to de-correlated inputs : a)measured sub-threshold frequency spectra for decreasinginter-synapse input correlation ρ , showing fast degrada-tion of frequency selectivity with decreasing correlation.b) boxplot indicating z-scores of firing rate in responseto matched (solid boxes) and unmatched (dashed boxes)with different inter-input correlations (x-axis). c) Spikeraster and averaged firing rates of 1000 neurons with ran-dom synapse parameters (number and position of con-jugate zeros), in response to input alternating betweenmatched and unmatched input, repeated for differentinter-synapse input correlations (indicated with differentcolors, corresponding to the colors of panel b). Top plotindicating the spike timings of the neurons for 6 differentcorrelation values (indicated with different colours). Bot-tom plot indicates the average of the firing rates, trans-formed into z-scores, of all the neurons for each inter-synapse correlation value. sarily alter the signal-to-noise ratio (Lindner, 2014), butalters the dynamics of signal transmission. Thus, unlessspectral content directly conveys information, the effectof delays as presented in this paper does not constitute a‘neural code’ as such. Rather, it plays an indirect role bymaking it possible to dynamically route signals (for ex-ample by amplifying certain frequency bands; excitationor inhibition of other neurons by the frequency selectiveneuron or; resetting the phase of oscillatory populations).Through these effects neural codes can be transmitted,so this type of filtering constitutes a medium, rather thanthe message. Reinterpretation of synaptic strengths
The presented results show that with the inclusion oftransmission delays the effect of synaptic connectivity isdifferent from the generally accepted interpretation: thestrenght of a synaptic connection determines not merelythe frequency of post-synaptic firing, but co-determinesthe frequencies to which the post-synaptic neuron re-sponds. This sub-threshold frequency response is deter-mined by the full ensemble of synapses transmitting cor-related inputs to a neuron, thus the effect of the strengthof a single synapse on the activity of the post-synapticneuron cannot be understood in isolation, but only inrelation to the other synapses. In the light of this ob-servation, the interpretation of the meaning of synapticstrength and the role of synaptic plasticity might needto be reconsidered. It will be crucial, of course, to testthe predictions of the theory presented in this paper ex-perimentally.
Interpretation of the inputs
During the theoretical treatment of the frequency se-lectivity due to synaptic inputs the time-shifted inputswere taken to be perfectly correlated. Eventhough it wasshown that perfect correlation was not needed for neuralcircuits to retain functionally their frequency selectivity,the correlation levels for which the frequency filtering wasqualitatively prominent are higher than reported correla-tions between pairs of neurons (Cohen and Kohn, 2011).This raises the question how to interpret the inputs usedin this study.A first posibility, which is also the most simple ex-planation, is to let the different delayed inputs emergefrom the same, or largely overlapping sources, throughdifferent transmission lines. In this way the unique noisesources are reduced to that due to synaptic transmissionand the propagation of signals along axons and dendrites.It is, by the knowledge of the author, not known whethersuch connection patterns exist in the brain, but in anycase this solution would require a very specific wiring ofneural circuits.Another option arises by observing that the results ofthe numerical situations show (section ) that switching7he input from a unmatched to a matched input elicits arapid response in the matched neuron, as visible from theswitches between the different inputs in figure ?? . Fromthe viewpoint of a neuron this is equivalent to its in-put switching from uncorrelated (with arbitrary spectralcontent) signals, to correlated (and frequency matched)signals. Thus neurons need only transient pre-synapticsynchronisation to detect their matched spectral input.As a third explaination, eventhough single cell to sin-gle cell correlations are low, and seem to actively be keptlow even when driven by the same input (Renart et al.,2010; Graupner and Reyes, 2013), collectively coherentactivity often co-occurs with irregular firing in singleneurons (Buzs´aki and Wang, 2012). Network level os-cillatory activity can arise from sparsely interconnectedand weakly correlated neurons (Brunel and Hakim, 1999;Brunel, 2000), showing that the correlation between pairsof neurons can be low contemporary with the pooled ac-tivation of sub-sets of a population showing stronger cor-relations. Thus the different inputs can be correlated toa high degree if we interpret the input lines of (1) aseach receiving the pooled activity from a population of(sparsely) interconnected and (weakly) correlated neu-rons. AcknowledgementsReferences
Amit, D. J., Gutfreund, H., and Sompolinsky, H. (1985).Spin-glass models of neural networks.
Physical ReviewA , 32(2):1007.Anderson, J. A. (1972). A simple neural network generat-ing an interactive memory.
Mathematical biosciences ,14(3-4):197–220.Bakkum, D. J., Chao, Z. C., and Potter, S. M. (2008).Long-term activity-dependent plasticity of action po-tential propagation delay and amplitude in corticalnetworks.
PLOS one , 3(5):e2088.Brette, R. (2012). Computing with neural synchrony.
PLoS Computational Biology , 8(6).Brunel, N. (2000). Dynamics of sparsely connected net-works of excitatory and inhibitory spiking neurons.
Journal of Computational Neuroscience , 8(3):183–208.Brunel, N. (2016). Is cortical connectivity optimized forstoring information?
Nature neuroscience , 19(5):749–755.Brunel, N. and Hakim, V. (1999). Fast global oscillationsin networks of integrate-and-fire neurons with low fir-ing rates.
Neural Computation , 11(7):1621–1671. Buzs´aki, G. and Draguhn, A. (2004). Neuronal oscilla-tions in cortical networks.
Science , 304(5679):1926–1929.Buzs´aki, G. and Wang, X.-J. (2012). Mechanisms ofgamma oscillations.
Annual review of neuroscience ,35:203–225.Chapeau-Blondeau, F. and Chauvet, G. (1992). Sta-ble, oscillatory, and chaotic regimes in the dynamicsof small neural networks with delay.
Neural Networks ,5(5):735–743.Cohen, M. R. and Kohn, A. (2011). Measuring and in-terpreting neuronal correlations.
Nature Neuroscience ,14(7):811.Cullheim, S. (1978). Relations between cell body size,axon diameter and axon conduction velocity of catsciatic α -motoneurons stained with horseradish per-oxidase. Neuroscience letters , 8(1):17–20.Cullheim, S. and Ulfhake, B. (1979). Relations betweencell body size, axon diameter and axon conduction ve-locity of triceps surae alpha motoneurons during thepostnatal development in the cat.
Journal of Compar-ative Neurology , 188(4):679–686.De Col, R., Messlinger, K., and Carr, R. W. (2008). Con-duction velocity is regulated by sodium channel inacti-vation in unmyelinated axons innervating the rat cra-nial meninges.
The Journal of physiology , 586(4):1089–1103.Destexhe, A. (1994a). Oscillations, complex spatiotem-poral behavior, and information transport in networksof excitatory and inhibitory neurons.
Physical ReviewE , 50(2):1594.Destexhe, A. (1994b). Stability of periodic oscillations ina network of neurons with time delay.
Physics LettersA , 187(4):309–316.Destexhe, A. (2009). Self-sustained asynchronous irregu-lar states and up–down states in thalamic, cortical andthalamocortical networks of nonlinear integrate-and-fire neurons.
Journal of computational neuroscience ,27(3):493.Ernst, U., Pawelzik, K., and Geisel, T. (1995). Synchro-nization induced by temporal delays in pulse-coupledoscillators.
Physical review letters , 74(9):1570.Gasser, H. S. and Grundfest, H. (1939). Axon diametersin relation to the spike dimensions and the conductionvelocity in mammalian a fibers.
American Journal ofPhysiology-Legacy Content , 127(2):393–414.Gautrais, J. and Thorpe, S. (1998). Rate coding versustemporal order coding: a theoretical approach.
Biosys-tems , 48(1-3):57–65.8eisler, C., Diba, K., Pastalkova, E., Mizuseki, K.,Royer, S., and Buzs´aki, G. (2010). Temporal delaysamong place cells determine the frequency of popula-tion theta oscillations in the hippocampus.
Proceedingsof the National Academy of Sciences , 107(17):7957–7962.Gluss, B. (1967). A model for neuron firing with expo-nential decay of potential resulting in diffusion equa-tions for probability density.
The Bulletin of Mathe-matical Biophysics , 29(2):233–243.Graupner, M. and Reyes, A. D. (2013). Synaptic inputcorrelations leading to membrane potential decorrela-tion of spontaneous activity in cortex.
Journal of Neu-roscience , 33(38):15075–15085.Harper, A. and Lawson, S. (1985). Conduction veloc-ity is related to morphological cell type in rat dorsalroot ganglion neurones.
The Journal of physiology ,359(1):31–46.Hopfield, J. J. (1982). Neural networks and physicalsystems with emergent collective computational abili-ties.
Proceedings of the national academy of sciences ,79(8):2554–2558.Izhikevich, E. M. (2006). Polychronization: computationwith spikes.
Neural Computation , 18(2):245–282.Klampfl, S. and Maass, W. (2013). Emergence of dy-namic memory traces in cortical microcircuit modelsthrough stdp.
Journal of Neuroscience , 33(28):11515–11529.Lapicque, L. (1907). Recherches quantitatives surl’excitation electrique des nerfs traitee comme une po-larization.
Journal de Physiologie et de PathologieGeneralej , 9:620–635.Lee, K. H., Chung, K., Chung, J. M., and Coggeshall,R. E. (1986). Correlation of cell body size, axon size,and signal conduction velocity for individually labelleddorsal root ganglion cells in the cat.
Journal of Com-parative Neurology , 243(3):335–346.Lindner, B. (2014). Low-pass filtering of information inthe leaky integrate-and-fire neuron driven by whitenoise. In
International Conference on Theory andApplication in Nonlinear Dynamics (ICAND 2012) ,pages 249–258. Springer.Maex, R. and De Schutter, E. (2003). Resonant synchro-nization in heterogeneous networks of inhibitory neu-rons.
Journal of Neuroscience , 23(33):10503–10514.Mainen, Z. F. and Sejnowski, T. J. (1995). Reliabil-ity of spike timing in neocortical neurons.
Science ,268(5216):1503–1506. Mehring, C., Hehl, U., Kubo, M., Diesmann, M., andAertsen, A. (2003). Activity dynamics and propaga-tion of synchronous spiking in locally connected ran-dom networks.
Biological cybernetics , 88(5):395–408.Oppenheim, A. V., Willsky, A. S., and Hamid, S. (1997).Signals and systems.Ostojic, S. (2014). Two types of asynchronous activity innetworks of excitatory and inhibitory spiking neurons.
Nature neuroscience , 17(4):594–600.Renart, A., De La Rocha, J., Bartho, P., Hollender,L., Parga, N., Reyes, A., and Harris, K. D. (2010).The asynchronous state in cortical circuits.
Science ,327(5965):587–590.Reyes, A. D. (2003). Synchrony-dependent propagationof firing rate in iteratively constructed networks invitro.
Nature neuroscience , 6(6):593–599.Smith, J. O. (2007).
Introduction to digital filters: withaudio applications , volume 2. Julius Smith.Stein, R. B. (1965). A theoretical analysis of neuronalvariability.
Biophysical Journal , 5(2):173–194.Swadlow, H. A. (1974). Systematic variations in theconduction velocity of slowly conducting axons inthe rabbit corpus callosum.
Experimental neurology ,43(2):445–451.Swadlow, H. A. (1985). Physiological properties of indi-vidual cerebral axons studied in vivo for as long as oneyear.
Journal of Neurophysiology , 54(5):1346–1362.Swadlow, H. A. (1994). Efferent neurons and suspectedinterneurons in motor cortex of the awake rabbit:axonal properties, sensory receptive fields, and sub-threshold synaptic inputs.
Journal of neurophysiology ,71(2):437–453.Thalhammer, J., Raymond, S., Popitz-Bergez, F., andStrichartz, G. (1994). Modality-dependent modula-tion of conduction by impulse activity in functionallycharacterized single cutaneous afferents in the rat.
So-matosensory & motor research , 11(3):243–257.Thorpe, S. J. (1990). Spike arrival times: A highly effi-cient coding scheme for neural networks.
Parallel Pro-cessing in Neural Systems , pages 91–94.Van Vreeswijk, C., Abbott, L., and Ermentrout, G. B.(1994). When inhibition not excitation synchronizesneural firing.
Journal of computational neuroscience ,1(4):313–321.Van Vreeswijk, C. and Sompolinsky, H. (1996). Chaosin neuronal networks with balanced excitatory and in-hibitory activity.
Science , 274(5293):1724–1726.9ogels, T. P. and Abbott, L. (2009). Gating multi-ple signals through detailed balance of excitation andinhibition in spiking networks.
Nature neuroscience ,12(4):483.Wang, X.-J. and Buzs´aki, G. (1996). Gamma oscillationby synaptic inhibition in a hippocampal interneuronalnetwork model.
Journal of neuroscience , 16(20):6402–6413.Waxman, S. G. (1980). Determinants of conduction ve-locity in myelinated nerve fibers.
Muscle & Nerve:Official Journal of the American Association of Elec-trodiagnostic Medicine , 3(2):141–150.Wilson, H. R. and Cowan, J. D. (1972). Excitatoryand inhibitory interactions in localized populations ofmodel neurons.
Biophysical Journal , 12(1):1–24.
Methods
All simulations carried out in this paper are done withleaky intergrate-and-fire neurons (Lapicque, 1907), re-ceiving M correlated, weighted and time-shifted inputs w m I ( m ) ( t − d m ). The membrane potential is governedby the equation τ dvdt + v = G (cid:32) I ( t − d ) + M (cid:88) m =1 w m I ( m ) ( t − d m ) (cid:33) , which is supplemented with a spike-and-reset mechanism:each time v surpasses a threshold value v c , it is said to firea spike and is directly reset to a reset value v r . Through-out a normalising gain factor of G = (1 + (cid:80) Mm =1 | w m | ) − is used. In the simulations for the results and figures ofthis article the inputs I ( m ) to the neuron are either purewhite-noise or filtered white-noise, depending on the par-ticular simulation (see the following method sections forthe specifics per simulation). In general, for simplicity,the first delay time is set to zero (i.e. d = 0). Measured sub-threshold frequency responsespectra
The sub-threshold frequency response spectra are mea-sured by driving the neuron with inputs which are time-shifted versions of a single white-noise source with mean µ = 0 and standard deviation σ = 1. To ignore thefast membrane fluctuation caused by spiking, the spikingthreshold was set to infinity ( v c = ∞ ). The simulationswere carried out with 1e timesteps per spectrum. Eachspectrum is the average over the spectra of 1e simulatedneurons. Frequency selectivity: discrimination of in-puts
The ‘base’ input both neurons received alternates be-tween two different noise signals with differing spectralcontent, each of these signals matching the frequency se-lectivity of either one of the two neurons. In the first4000 timesteps of each realisation the administered in-put is an unfiltered white noise signal, in order to observethe baseline firing of each neuron. For these simulationsthe spike-and-reset mechanism v > v th = ⇒ v ← v r isreintroduced, so it is be possible to observe the spikingbehavior of the two neurons in response to the differentinputs. Simulations were carried out with 2 e Frequency selectivity: specificity of responses
Each neurons receives 13 inputs with weights leading tozeros evenly spaced around the unit-circle, with the ex-eption of one conjugate pair. In this way the neuron ismainly responsive to a small band of frequencies aroundthe angle of the missing pair of zeros. These neurons aresubsequently exposed to bandpass filtered white noisewith differing center frequencies. The spike count of eachneuron in response to each bandpass filtered noise stim-ulus is recorded and normalised per neuron over differ-ent inputs such that the maximal count of each neuronequals one. Each pixel correspons to the average spikecount of 500 neurons over 2 e Robustness to non perfect correlation
Each input I ( m ) to the synapses of the neurons in thesesimulations is driven by an input consisting of a sourceinput I , which is a specifically filtered white-noise signalmatched to the prefferred spectrum of a neuron. Thisinput is shared by all the synapses. In addition eachsynapse receives a unique noise eta m , which is a ran-domly permutated version of the matched input. Thisinput is unique to each synapse. Thus: I ( m ) = ρI + (1 − ρ ) η m . (7)The synapse parameters ( M , w m and d m ) are contructedfrom ( M − / m = 0. This resulted in 1000 neuronswith different frequency selectivity. These neurons arethen driven in 6 trials with inputs with differing inter-synapse correlations from perfect correlation ρ = 1 tocompletely uncorrelated ρρ