A Generalized Strong-Inversion CMOS Circuitry for Neuromorphic Applications
AA Generalized Strong–Inversion CMOS Circuitry for NeuromorphicApplications
Hamid Soleimani and Emmanuel. M. Drakakis
Abstract
It has always been a challenge in neuromorphic field to systematically translate biological models intoanalog electronic circuitry. In this paper, a generalized circuit design platform is introduced where biologicalmodels can be conveniently implemented using CMOS circuitry operating in strong–inversion. The applica-tion of the method is demonstrated by synthesizing a relatively complex two–dimensional (2–D) nonlinearneuron model. The validity of our approach is verified by nominal simulated results with realistic processparameters from the commercially available AMS 0.35 µm technology. The circuit simulation results exhibitregular spiking response in good agreement with their mathematical counterpart. Researchers in the neuromorphic community intend to mimic the neuro-biological structures in the nervoussystem using electronic circuitry. To do so different approaches have been developed so far:1. Special purpose computing architectures have been developed to simulate complex biological networksvia special software tools [1–5]. Even though these systems are biologically plausible and flexible withremarkably high performance thanks to their massively parallel architecture, they run on bulky andpower-hungry workstations with relatively high cost and long development time.2. Digital platforms are good candidates nowadays for implementing such biological and bio-inspired sys-tems. Most digital approaches [6–15], use digital computational units to implement the mathematicalequations codifying the behavior of biological intra/extracellular dynamics. Such an approach can beeither implemented on FPGAs or custom ICs, with FPGAs providing lower development time and moreconfigurability. Generally, a digital platform benefits from high reconfigurability, short development time,notable reliability and immunity to device mismatch. Although, the digital platform’s silicon area andpower consumption is comparatively high compared to its analog counterpart.3. Analog CMOS platforms are considered to be the main choice for direct implementation of intra– and ex-tracellular biological dynamics [16–24]. This approach is very power efficient, however, model developmentand adjustment is generally challenging. Moreover, since the non–linear functions in the target models aredirectly synthesized by exploiting the inherent non–linearity of the circuit components, very good layout isimperative in order for the resulting topology not to suffer from the variability and mismatch particularlyCMOS circuits operating in subthreshold.To address the challenges explained in
In this section, a novel current–input current–output circuit is proposed that supports a systematic realiza-tion procedure of strong–inversion circuits capable of computing bilateral dynamical systems at higher speedcompared to the previously proposed log–domain circuit. The validity of our approach is verified by nominalsimulated results with realistic process parameters from the commercially available AMS 0.35 µm technology.1 a r X i v : . [ c s . N E ] J u l I A I B I Cin V b VDDM M M M V C V initial V pulse Figure 1: The “main core” including the initialization circuit highlighted with red color.The current relationship of an NMOS and PMOS transistor operating in strong–inversion saturation when | V DS | > | V GS | − | V th | can be expressed as follows: I D n = 12 µ n C ox ( WL ) n ( V GS − V th ) (1) I D p = 12 µ p C ox ( WL ) p ( V SG − V th ) (2)where µ n and µ p are the charge–carrier effective mobility for NMOS and PMOS transistors, respectively; W is the gate width, L is the gate length, C ox is the gate oxide capacitance per unit area and V th is the thresholdvoltage of the device.Setting k n = µ n C ox ( WL ) n and k p = µ p C ox ( WL ) p in (1) and (2) and differentiating with respect to time,the current expression for I A (see Figure 1) yields:˙ I A = √ k n I A (cid:122) (cid:125)(cid:124) (cid:123) k n ( V GS − V th ) ˙ V GS (3)˙ I A = √ k p I A (cid:122) (cid:125)(cid:124) (cid:123) k p ( V SG − V th ) ˙ V SG (4)(3) and (4) are equal, therefore: ˙ V SG = (cid:115) k n k p ˙ V GS = β ˙ V GS (5)where β = (cid:113) k n k p . Similarly, we can derive the following equation for transistors M and M :˙ V SG = (cid:115) k n k p ˙ V GS = β ˙ V GS . (6)The application of Kirchhoff’s Voltage Law (KVL) and applying the derivative function show the followingrelations: ˙ V C = − ( ˙ V GS + ˙ V SG ) (7)˙ V C = +( ˙ V GS + ˙ V SG ) (8)where V C is the capacitor voltage and V b the bias voltage which is constant (see Figure 1). Substituting (5) and(6) into (7) and (8) respectively yields: ˙ V C = − ˙ V GS · (1 + β ) (9)˙ V C = + ˙ V GS · (1 + β ) . (10)2 ROOT
Main Core I B I A I A F 𝐈 𝐂 𝐢 𝐧 = 𝐅 𝐈 𝐨 𝐮 𝐭 , 𝐈 𝐞 𝐱 𝐭 𝐈 𝐝 𝐜 ( 𝑰 𝑨 + 𝑰 𝑩 ) NROOT I B 𝑰 𝑨 𝑰 𝑩 MULTI dc Main Block I B I A ... ... ...... I d c F I Bo1 I Bon I Bo2 ... I B i I B i n I B i .... I A o1 I A o n I A o2 .... I Ai1 I Ain I Ai2 ... I B I A Figure 2: (a) The “main block” including the main core and two current–mode root square blocks and a bilateralmultiplier. (b) The final high speed circuit including the “main block” with several copied currents (the currentmirrors are represented with double circle symbols)Setting the current I out = I B − I A in Figure 1 as the state variable of our system and using (3) and thecorresponding equation for I B , the following relation is derived:˙ I out = ˙ I B − ˙ I A = 2 (cid:112) k n I B ˙ V GS − (cid:112) k p I A ˙ V GS (11)by substituting (9) and (10) in (11):˙ I out = ( (cid:112) I A + (cid:112) I B ) · √ k n ˙ V C β . (12)Bearing in mind that the capacitor current I Cin can be expressed as C ˙ V C , relation (12) yields:˙ I out = ( (cid:112) I A + (cid:112) I B ) · √ k n I Cin (2 + β ) C . (13)One can show that: (2 + β ) C √ k n · I dc ˙ I out = ( √ I A + √ I B ) I dc · I Cin . (14)Equation (14) is the main core’s relation. In order for a high speed mathematical dynamical system withthe following general form to be mapped to (14): τ ˙ I out = F ( I out , I ext ) (15)where I ext and I out are the external and state variable currents, the quantities CI dc and I Cin must be respectivelyequal to τ √ k n (2+ β ) and F ( I out ,I ext ) I dc ( √ I A + √ I B ) . Note that the ratio value CI dc can be satisfied with different individual valuesfor C and I dc . These values should be chosen appropriately according to practical considerations (see SectionV.G). Since F is a bilateral function, in general, it will hold: I Cin = I + Cin (cid:122) (cid:125)(cid:124) (cid:123) F + ( I A , I B , I + ext , I − ext ) I dc ( √ I A + √ I B ) − I − Cin (cid:122) (cid:125)(cid:124) (cid:123) F − ( I A , I B , I + ext , I − ext ) I dc ( √ I A + √ I B ) (16)where I + Cin and I − Cin are calculated respectively by a root square block (see Figure 2(a) and I ext is separatedto + and – signals by means of splitter blocks. Note that I dc is a scaling dc current and τ has dimensions of second ( s ). Since I Cin can be a complicated nonlinear function in dynamical systems, we need to provide copiesof I out or equivalently of I A and I B to simplify the systematic computation at the circuit level. Therefore, thehigher hierarchical block shown in Figure 2(b) is defined as the NBDS (Nonlinear Bilateral Dynamical System)circuit [16] (see Figure Figure 2(b)) including the main block and associated current mirrors. The form of (15)is extracted for a 1–D dynamical system and can be extended to N dimensions in a straightforward manner asfollows: τ N ˙ I out N = F N ( ¯ I out , ¯ I ext ) (17)where C N I dcN = τ N √ k n (2+ β ) and I Cin N = F N (¯ I out , ¯ I ext ) I dcN ( √ I AN + √ I BN ) . 3 B M VDD I out M M M I in M I out M M M I out I out I b I b I in I b I b out out I b Figure 3: (a) Transistor level representation of the basic Root Square block. The current mirrors are representedwith double circle symbols. (b) Transistor level representation of the MULT core block. The current mirrorsare represented with double circle symbols.)
This block performs current mode root square function on single–sided input signals. By setting ( WL ) =4 × ( WL ) , considering I , I , I and I as the currents flowing respectively into M , M , M and M and alltransistors operate in strong–inversion saturation, the governing TL principle for this block becomes (highlightedwith dotted blue arrow): 12 ( (cid:112) I + (cid:112) I ) = (cid:112) I + (cid:112) I (18)By pushing specific currents (copied by current mirrors) according to Figure 3 (a) into the TL’s transistorswe have: (cid:40) I = I = I in + I out + I b I = I b , I = I in (19)Substituting (19) into (18) yields:12 × ( (cid:112) I in + I out + I b + (cid:112) I in + I out + I b = (cid:112) I in + (cid:112) I b (20)By squaring both sides of (20): I in + I out + I b = (cid:112) I in + (cid:112) I in · I b (21)and finally: I out = 2 (cid:112) I in · I b (22) This block is the main core forming the final bilateral multiplier which is introduced in the next subsection.The block contains six transistors as well as two current mirrors. By assuming I , I , I and I as the currentsflowing respectively into M , M , M and M and the same WL aspect ratio for all transistors operating instrong–inversion saturation, the KVL at the highlighted TL with dotted blue arrow yields: (cid:112) I + (cid:112) I = (cid:112) I + (cid:112) I (23)By forcing specific currents (copied by current mirrors) according to Figure 3 (b) into the TL’s transistorswe have: (cid:40) I = I b , I = I out I = I = ( I in + I out + I b ) (24)4 ULT Core I b I in I out E A
MULT Core I b I in I out F B
MULT Core I b I in I out G C
MULT Core I b I in I out H D I b E F G H A C B D A D B C 𝐗 + 𝐘 + 𝐗 − 𝐘 − I NEG I POS
Figure 4: Schematic and symbolic representation of the bilateral MULT block comprising current mirrors andMULT Core block.Substituting (24) into (23) yields: (cid:112) I out + (cid:112) I b = 2 (cid:114)
12 ( I in + I out I b ) (25)By squaring both sides of (25): (cid:112) I out · I b = I in + 12 I b (26)and: I out = ( I in + I b ) I b (27) This block is able to perform current mode multiplication operation on bilateral input signals. If inputs aresplit to positive and negative sides we have: (cid:40) X = X + − X − Y = Y + − Y − . (28)The multiplication result can be expressed as XY = X + Y + + X − Y − − ( X − Y + + Y − X + ). By extendingequation (27) to I in I b + I b + I in for every basic MULT core block, the output signal constructed by a positiveand negative side can be written as: I out = ( X + + Y + ) I b + ( X + + Y + ) + I b X − + Y − ) I b + ( X − + Y − ) + I b − ( X − + Y + ) I b − ( X − + Y + ) − I b − ( X + + Y − ) I b − ( X + + Y − ) − I b I out = I + out (cid:122) (cid:125)(cid:124) (cid:123) X + Y + + X − Y − ) I b − I − out (cid:122) (cid:125)(cid:124) (cid:123) X − Y + + X + Y − ) I b = 2 XYI b (30) The systematic synthesis procedure provides the flexibility and convenience required for the realization ofnonlinear dynamical systems by computing their time-dependent dynamical behavior. In this subsection, we5
BCD FCDKNBDS
Circuit (v)
VDD B A I dc ALNBDS
Circuit (w)
VDD
VDDB . u A b I out I b-split MULT
VDD
20 uA
VDDA I e x t v out I v out I v out I v out I w out I w out I w out I Source Current
Sink Current I dc F w A F v Sink/Source Current I b-split I in1 I in2 D b I out I b-split MULT
VDD
20 uA I in1 I in2 H 1 uA v out I v out I F GG H0.3 v out I v out I K H0.3 A v out I B v out I L MM B0.8B N0.8 w out I N Figure 5: A block representation of the total circuit implementing the 2–D FHN neuron model.showcase the methodology through which we systematically map the mathematical dynamical models ontothe proposed electrical circuit. Here, the application of the method is demonstrated by synthesizing the 2–Dnonlinear FitzHughNagumo neuron model. In the FHN neuron model [25] with the following representation:˙ v = v − v − w + I ext and ˙ w = 0 . v +0 . − . w ) describing the membrane potential’s and the recovery variable’svelocity, the state variables in the absence of input stimulation remain at ( v, w ) ≈ ( − . , − . v, w ) ≈ (2 , .
7) in the presence of input stimulation. According to this biological dynamicalsystem, we can start forming the electrical equivalent using (17): (cid:40) (2+ β ) C √ k n · I dcv ˙ I out v = F v ( I out v , I out w , I ext ) (2+ β ) C √ k n · I dcw ˙ I out w = F w ( I out v , I out w ) (31)where I dc v = 80 nA , I dc w = a · I dc v = 6 . nA , F v and F w are functions given by: (cid:40) F v ( I out v , I out w , I ext ) = I out v − I outv I b I x − I out w + I ext F w ( I out v , I out w ) = ( I out v + I c − I d I outw I x ) (32)where I b = 3 uA , I c = 0 . uA , I d = 0 . uA and I x = 1 uA .Schematic diagrams for the FHN neuron model is seen in Figure 5, including the symbolic representation ofthe basic TL blocks introduced in the previous sections. According to these diagrams, it is observed how themathematical model is mapped onto the proposed electrical circuit. The schematic contains two NBDS circuitsimplementing the two dynamical variables, followed by two MULT and current mirrors realizing the dynamicalfunctions. As shown in the figure, according to the neuron model, proper bias currents are selected and thecorrespondence between the biological voltage and electrical current is V ⇐⇒ uA . Here, we demonstrate the simulation–based results of the high speed circuit realization of the FHN neuronmodel. The hardware results simulated by the Cadence Design Framework (CDF) using the process parameters6 v & w ( V ) v & w ( u A ) Time (Sec) Time (uSec)(a) (b)
Figure 6: Time-domain representations of regular spiking for (a) for MATLAB and (b) Cadence respectively.Table 1: Electrical Parameter Values for the Simulated FHN Neuron Model operating in strong–inversion.Specifications ValuePower Supply (Volts) 3.3Bias Voltage (Volts) 3.3Capacitances (pF) 800 WL ratio of PMOS and NMOS Devices ( µmµm ) and Static Power Consumption ( mW ) 8.94of the commercially available AMS 0.35 µm CMOS technology are validated by means of MATLAB simulationsas shown in Figure 6. For the sake of frequency comparison, a regular spiking mode is chosen. Generally, resultsconfirm an acceptable compliance between the MATLAB and Cadence simulations while the hardware modeloperates at higher speed (almost 1 million times faster than real–time). Table 1 summarizes the specificationsof the proposed circuit applied to this case study. As shown in the table, the circuit uses a higher V b comparedto the subthreshold version to force the circuit to operate in strong–inversion region. This comes at the expenseof higher power consumption (95000 times higher than the subthreshold version). References [1] S B Furber, F Galluppi, S Temple, and L A Plana. The spinnaker project.
Proceedings of the IEEE ,102(5):652–665, 2014.[2] H Soleimani and A Ahmadi. A gpu based simulation of multilayer spiking neural networks. In , pages 1–5. IEEE, 2011.[3] Tesla k80 gpu accelerator.
Board Specification https://images.nvidia.com/content/pdf/kepler/Tesla-K80-BoardSpec-07317-001-v05.pdf , 2015.[4] Intel xeon processor e5–4669 v3. http://ark.intel.com/products/85766/Intel-Xeon-Processor-E5-4669-v3-45M-Cache-2 10-GHz , 2016.[5] J M Nageswaran, N Dutt, J L Krichmar, A Nicolau, and A V Veidenbaum. A configurable simulationenvironment for the efficient simulation of large-scale spiking neural networks on graphics processors.
Neuralnetworks , 22(5):791–800, 2009.[6] H Soleimani and M Drakakis, E. A low-power digital ic emulating intracellular calcium dynamics.
Inter-national Journal of Circuit Theory and Applications , 46(11):1929–1939, 2018.[7] H Soleimani and M Drakakis, E. An efficient and reconfigurable synchronous neuron model.
IEEE Trans-actions on Circuits and Systems II: Express Briefs , 65(1):91–95, 2017.[8] T Matsubara and Torikai. Asynchronous cellular automaton-based neuron: theoretical analysis and on-fpgalearning.
IEEE transactions on neural networks and learning systems , 24(5):736–748, 2013.[9] A Cassidy and A G Andreou. Dynamical digital silicon neurons.
IEEE Biomedical Circuits and SystemsConference , pages 289–292, 2008. 710] T Hishiki and H Torikai. A novel rotate-and-fire digital spiking neuron and its neuron-like bifurcations andresponses.
Neural Networks and Learning Systems, IEEE Transactions on , 22(5):752–767, 2011.[11] H Soleimani, A Ahmadi, and M Bavandpour. Biologically inspired spiking neurons: Piecewise linear modelsand digital implementation.
Circuit and System I Regular Paper, IEEE Transactions on , 59(12):2991–3004,2012.[12] H Soleimani, M Bavandpour, A Ahmadi, and D Abbott. Digital implementation of a biological astrocytemodel and its application.
IEEE Trans. Neural Netw. Learn. Syst. , 26(1):127–139, 2015.[13] H Soleimani and E M Drakakis. A compact synchronous cellular model of nonlinear calcium dynamics:simulation and fpga synthesis results.
IEEE Trans. Biomedical Circuit and Systems , 17(3):703–713, 2017.[14] E Jokar and H Soleimani. Digital multiplierless realization of a calcium-based plasticity model.
IEEETransactions on Circuits and Systems II: Express Briefs , 64(7):832–836, 2016.[15] A Makhlooghpour, H Soleimani, A Ahmadi, M Zwolinski, and M Saif. High accuracy implementation ofadaptive exponential integrated and fire neuron model. In , pages 192–197. IEEE, 2016.[16] E Jokar, H Soleimani, and M Drakakis, E. Systematic computation of nonlinear bilateral dynamical systemswith a novel low-power log-domain circuit.
IEEE Transactions on Circuits and Systems I: Regular Papers ,64(8):2013 – 2025, 2017.[17] S S Woo, J Kim, and R Sarpeshkar. A cytomorphic chip for quantitative modeling of fundamental bio-molecular circuits.
Biomedical Circuits and Systems, IEEE Transactions on , 9(4):527–542, 2015.[18] G Indiveri, E Chicca, and R Douglas. A vlsi array of low-power spiking neurons and bistable synapses withspike-timing dependent plasticity.
Neural Networks, IEEE Transactions on , 17(1):211–221, 2006.[19] A Houssein, K I Papadimitriou, and E M Drakakis. A 1.26 µ w cytomimetic ic emulating complex nonlinearmammalian cell cycle dynamics: Synthesis, simulation and proof-of-concept measured results. BiomedicalCircuits and Systems, IEEE Transactions on , 9(4):543–554, 2015.[20] K I Papadimitriou, G B V Stan, and E M Drakakis. Systematic computation of nonlinear cellular andmolecular dynamics with low-power cytomimetic circuits: a simulation study.
PloS one , 8(2):e53591, 2013.[21] S Moradi and G Indiveri. An event-based neural network architecture with an asynchronous programmablesynaptic memory.
Biomedical Circuits and Systems, IEEE Transactions on , 8(1):98–107, 2014.[22] M Bavandpour, H Soleimani, S Bagheri-Shouraki, A Ahmadi, D Abbott, and L O Chua. Cellular memristivedynamical systems (cmds).
International Journal of Bifurcation and Chaos , 24(5):1430016–1–1430016–22,2014.[23] M Bavandpour, H Soleimani, B Linares-Barranco, D Abbott, and L O Chua. Generalized reconfigurablememristive dynamical system (mds) for neuromorphic applications.
Frontiers in neuroscience , 9(409):1–19,2015.[24] H Soleimani, A Ahmadi, M Bavandpour, and O Sharifipoor. A generalized analog implementation ofpiecewise linear neuron models using ccii building blocks.
INeural networks , 51:26–38, 2014.[25] C Toumazou, J Georgiou, and E M Drakakis. Impulses and physiological states in theoretical models ofnerve membrane.