Multi-state MRAM cells for hardware neuromorphic computing
Piotr Rzeszut, Jakub Ch?ci?ski, Ireneusz Brzozowski, S?awomir Zi?tek, Witold Skowro?ski, Tomasz Stobiecki
MMulti-state MRAM cells for hardware neuromorphic computing
Multi-state MRAM cells for hardware neuromorphic computing
Piotr Rzeszut, a) Jakub Chęciński, Ireneusz Brzozowski, Sławomir Ziętek, Witold Skowroński, and TomaszStobiecki
1, 2 AGH University of Science and Technology, Institute of Electronics,Al. Mickiewicza 30, 30-059 Kraków, Poland AGH University of Science and Technology, Faculty of Physics and Applied Computer Science,Al. Mickiewicza 30, 30-059 Kraków, Poland (Dated: 9 February 2021)
Magnetic tunnel junctions (MTJ) have been successfully applied in various sensing application and digital informationstorage technologies. Currently, a number of new potential applications of MTJs are being actively studied, includinghigh-frequency electronics, energy harvesting or random number generators. Recently, MTJs have been also proposedin designs of a new platforms for unconventional or bio-inspired computing. In the present work, it is shown thatserially connected MTJs forming a multi-state memory cell can be used in a hardware implementation of a neuralcomputing device. A behavioral model of the multi-cell is proposed based on the experimentally determined MTJparameters. The main purpose of the mutli-cell is the formation of the quantized weights of the network, which canbe programmed using the proposed electronic circuit. Mutli-cells are connected to CMOS-based summing amplifierand sigmoid function generator, forming an artificial neuron. The operation of the designed network is tested using arecognition of the hand-written digits in 20 ×
20 pixel matrix and shows detection ratio comparable to the softwarealgorithm, using the weight stored in a multi-cell consisting of four MTJs or more.
I. INTRODUCTION
Unconventional computing architectures such as artificialneural networks (ANN) have superior properties over conven-tional CMOS-based circuits in solving a number of compu-tational problems, e.g., image or voice recognition, naviga-tion, optimization and prediction . As a concept, neuralnetworks have been proved to be fast, flexible and energy-efficient. However, their digital implementation uses largeamount of resources , which leads to high area needed to im-plement them. An alternative solution, opposite to the digitalimplementation, is to use analog-based circuits, where signalsare represented as continuous voltage values rather than quan-tized bits . In such implementations, a key element is a pro-grammable resistive element, such as memristor , which canact as a weight in an artificial neuron. Using a solely digitalimplementation of a neural network may lead to high resourceand energy consumption. On the contrary, using mixed digitaland analog electronic circuits may enable more compact andenergy efficient solution. In a number of the proposed analogANN implementations, neuron behavior was mimicked by aresistive RAM (RRAM) element , whose resistance changeoriginated from the conductor/insulator transition . However,cells based on resistive or phase-change technology sufferfrom limited durability and may degrade over time and subse-quent programming cycles . On the contrary, spintronic ele-ments such as memristors, nano-oscillators or probabilisticbit , based on magnetic tunnel junctions (MTJ), which relyon the magnetisation switching or dynamics, do not have suchendurance issues, are compatible with the CMOS technologyand have been already shown to exhibit superior biomimeticproperties . In addition, recent theoretical works have pre-dicted that neural networks are able to work efficiently not a) Electronic mail: [email protected] only with weights represented by real numbers but also withbinary or quantized values .Recently, we have proposed a design of multi-state spintransfer torque magnetic random access memory (STT-MRAM) cells , which may be used in neuromorphic com-puting schemes as synapses or as a standard multi-statememory unit. In this paper, we present a fully functional hard-ware implementation design of a neural network, which needsno additional components for operation, except for input andoutput devices. The design of a single synapse is based onmulti-bit STT-MRAM cells, interconnected with a minimalset of transistors forming amplifiers in conventional CMOStechnology. The entire network is made of neurons arrangedin four layers. The operation principle of the proposed neuralnetwork is validated using handwritten digits recognition taskutilizing MNIST database. II. EXPERIMENTAL AND CIRCUIT LEVEL SIMULATIONS
In order to perform simulations of the proposed neural net-work prior to the fabrication process, critical parameters ofserially connected MTJs forming a programmable memristordevice are needed. In theory, such parameters could ultimatelybe derived from a fully-developed simulation of an MTJ dy-namics, based on the Landau-Lifshitz-Gilbert equation. How-ever, this would require an introduction of multiple assump-tions on physical parameters such as magnetic anisotropy,spin-current polarization or temperature dependence, makingthe obtained result less general. On the other hand, the onlything that an MRAM cell needs in order to operate as a neuralnetwork weight is it’s transfer function - or, equivalently, theresistance vs. voltage ( R ( V ) ) loop. Since our aim is to modelthe behavior of a realistic ANN consisting of thousands of in-dividual MTJs, we find it less computationally expensive andmore universal to model R ( V ) loops directly, following theapproach described below. We will show that this approach is a r X i v : . [ c s . ET ] F e b ulti-state MRAM cells for hardware neuromorphic computing 2corroborated by experimental data and that it provides a robustfoundation for building a functional ANN.
1. Modeling of single MTJ R ( V ) curve A typical R ( V ) loop of an MTJ may be approximated usingfour linear functions (resistance vs. bias voltage dependencein each MTJ state) and two threshold points (switching volt-ages) as presented in Fig. 1. In addition, in the case of areal MTJ the following parameters are related to each other: a n = − a p = a b n = b p = b a n = − a p = a b n = b p = b
0. Moreover, a current resistance state (highor low resistance) has to be included. Using such a model ofthe R ( V ) curve allows also to calculate other transport curves,including V ( I ) . The proposed model corresponds to all MTJsthat were investigated during the study. FIG. 1. (a) Experimental R ( V ) dependence (solid points) and themodel consisting of four lines and two critical points (stars) present-ing a single MTJ behaviour. (b) A representative simulation resultof three serially-connected MTJs (solid line) together with a seriesof exemplary measurements (gray-scale lines) of a two-bit mutlicell.Parameters of a single MTJ were used to model the multi-cell char-acteristics. Due to the bias dependence of a tunneling magnetoresis-tance effect , the resistance of an MTJ varies with voltage(thus V ( I ) is not linear) and there is no direct way of calculat-ing R ( V ) transport curve for serially connected MTJs. Instead, V ( I ) may be calculated using Kirchhoff’s law, due to the factthat in serially connected MTJs the current is constant, whiletotal voltage is a sum of the voltage drops on each element .In order to determine a resistance for a given voltage V , thefollowing steps are performed :1. Start with a given state of all MTJs and current I =
02. Increase current by a small step I = I + ∆ I
3. For each MTJ:(a) Check if critical current for the given state wasreached:(b) If yes , set I to 0, change state of the element and goto 2. If no , continue. TABLE I. Expected values and standard deviation of parameters usedfor simulations Param. Unit µ σ a1 A − -310 3b1 Ω
665 12a0 A − -30 3b0 Ω
360 12cN A -3.1e-4 1.5e-5cP A 8.0e-4 1.5e-5 (c) Calculate voltage V i on the MTJ for given I
4. Calculate total voltage V t = ∑ Ni = V i
5. If total voltage V t (cid:62) V , the solution was found and thealgorithm ends. Otherwise, return to step 2.Depending on the polarity of the applied voltage, the sign of ∆ I may be the same or opposite to the sign of applied V . Thevalue of ∆ I should be as small as possible for a given compu-tation time limit. By repeating the process for each requested V , a full I ( V ) and R ( V ) curve can be calculated.Our approach makes it very straightforward to account forthe fact that real-life MTJs are never fully identical, but in-stead have slightly different dynamical behavior due to fab-rication imperfections and partially stochastic nature of thecurrent induced magnetisation switching process . Thanksto the fact that the operational principle of each multi-MTJcell is described by it’s R ( V ) curve, it is sufficient to representthis variation by choosing slightly different R ( V ) parametersfor each cell at the beginning of the simulation.To verify the proposed model, a series of measurementsof R ( V ) curve on three different elements were repeated sev-eral tens of times for each element, and the data was usedto obtain parameters of the model. Then, a series of mea-surements of these MTJs connected in series were performed.The comparison of experimental data and simulation resultsof the exact same MTJs is presented in Fig. 1 (b). One cansee that the simulation results follow the experimental depen-dence closely, with even multistep-switching from high- tolow-resistance state being reproduced.In order to predict behavior of a mutli-cell, measurementsof R ( V ) loops for a few tens of fabricated MTJs were con-ducted and analysed. Next, for each measurement (Fig. 1(a),the model was fitted, resulting in estimation of fabrication pa-rameters and process yield. Across the sample these param-eters represented a normal distribution and, therefore, the ex-pected values µ and their standard deviation σ may be usedto model each parameter.The parameters presented in Tab. I were used to simulate amutli-cell composed of seven serially connected MTJs, whereeight stable resistance states may be observed. After repeatingthe simulation 300 times, distribution of the switching (write)voltages and resistances were calculated (Fig. 2). A clearseparation between stable resistance levels (readout values) aswell as between write voltages were observed.ulti-state MRAM cells for hardware neuromorphic computing 3 FIG. 2. Simulation results for seven serially connected MTJs withgiven parameter spread. (a) Spread of readout resistances for thesimulation. (b) Representative write-read-erase curves. Red linerepresents full write-read-erease cycle, while black ones representwrite-read cycles while programming subsequent values. (c) Spreadof positive write voltages for the simulation. (d) Spread of all writevoltages for the simulation.
A. Electronic neuron
After the analysis of the multi-cell, which may be used asprogrammable resistor for performing weighted sum opera-tion for many input voltages, we turn to the artificial neurondesign. A schematic diagram of the proposed neuron is pre-sented in Fig. 3. Inputs and output ( V INm , V OUT ) are providedas bipolar analog signals. To enable positive and negativeweights, each of the signal inputs uses a pair of programmableMTJ multi-cells ( M mP and M mN ). In the case when the multi-cell resistances meet the condition M mP < M mN , a positiveweight is achieved, whereas for the case of M mP > M mN anegative weight value is realised. An alternative design withmultiple MTJs connected in series with a separate select tran-sistors has been proposed recently in Ref. 32. For an equalmutli-cells’ resistances, a zero weight is provided, which isequivalent to the situation when an input is disconnected fromthe synapse. The summing amplifier architecture is being usedin order to realise addition operation while reducing footprintof the synapse. A differential amplifier converts differentialvoltage to a single bipolar signal, which is transformed usinga non-linear (sigmoid) function. This voltage may be used asthe input of the next synapse, or as the output of the network.Additionally, to provide a constant bias, a standard input withconstant voltage may be used, where the level of this constantbias is determined in the same way as weights for other func-tional inputs. B. Neural network circuit
The electrical circuit realizing proposed neural network wasdesigned in a standard CMOS technology – UMC 180 nm. To
FIG. 3. The proposed neuron design with multi-cells. The circuitconsist of (a) a set of memristors for quantized weight purpose, (b)differential amplifier with summing amplifiers (d) at input and (c)sigmoid function block. program the demanded resistance of seven serially connectedMTJs, a voltage of about 3 .
25 V is needed, so input/output(I/O) 3 . . V d generated by the divider network(Fig. 3(a)) connected to a pair of summing amplifiers (Fig.3(d)) can be expressed as: V d = − R f ( m ∑ i = V INi ( G iP − G iN )) , where: G iX = M iX The operational amplifier, presented in Fig. 4 was designedas two stage circuit consisting of a differential pair M1, M2with a current mirror load M3, M4 biased by M5 with a cur-rent of 1 µA. The output stage M6, M7 provide appropriateamplification and output current. The total current consumedby the operational amplifier is about 12 µA and amplificationwith an open loop of around 74 dB. Dimensions of transistorswere chosen in such a way to obtain the smallest area possiblewhile meeting the required electrical parameters.The final stage of the neuron is a circuit, which performsactivation functions and has sigmoidal transfer characteristic,presented in Fig. 5(b). It is designed as a modified inverter .Transistors M2 and M3 work as resistors, moving operatingpoint of transistors M0 and M1 to the linear region. Finally,ulti-state MRAM cells for hardware neuromorphic computing 4 FIG. 4. A proposed differential amplifier circuit.FIG. 5. (a) A transfer function of a sigmoid-generating inverter real-ized by (b) the proposed inverter circuit. the circuit realizes the transfer characteristic shown in Fig.5(a). Minimum length of channels were used (180 nm, exceptfor M3, which uses 750 nm), while their width was chosen toobtain required characteristics and output current necessary todrive the next stage. Therefore, the width of M0 and M3 is1 . .
56 µm, and M2 is 1 .
12 µm.
III. RESULTS AND DISCUSSION
To evaluate the performance of the multi-bit MTJ cell-based ANN a set of classification tasks using the MNISTdataset of handwritten digits (Fig.6(a)) was prepared. Theconceptual architecture used for the network is shown in Fig.6(b) and consists of the input layer, two hidden layers con-taining N neurons each and the output layer. The network was FIG. 6. Simulated neural network based on multi-bit MRAM cells.Handwritten digits from MNIST database (a) are recognized by astandard neural network with architecture shown in (b), where blacklines represent network weights and yellow circles represent indi-vidual neurons. After training, weights calculated by software arereplaced by discretized values corresponding to 1-7 serial MTJsMRAM cells, which affects the network performance (c). trained using the standard scaled conjugate gradient methodand cross-entropy error metrics, with tanh activation functionfor every layer except the last one, where the softmax func-tion was used. Then, its performance was evaluated on a test-ing subset that has been drawn randomly from the input dataand has not participated in training. This procedure was re-peated 50 times in total, with training and testing subsets be-ing reshuffled each time, leading to an average error estimatefor each network size.Having established the benchmark network, the evaluation ofour MTJ-based design was performed. The original float-accuracy weights between different neurons were replaced bya discrete version corresponding to multi-state MTJ synapsesand characteristics of designed amplifier and activation func-tion circuit were applied to the model. The new weights werecalculated using simulated conductance data (as described inSec. II) and rescaled by tuning amplifiers’ gains to match thedesired value range for the neurons. Then, the performance ofthe network was re-evaluated on the testing data subset. Theresults are presented in Fig. 6(c). It can be seen that, as longas the number of MTJs used per multi-state cell exceeds three,the performance of the MTJ-based solution is comparable tothe original software version, with differences being only in-cremental in character.ulti-state MRAM cells for hardware neuromorphic computing 5
IV. SUMMARY
The presented architecture of full hardware artificial neuralnetwork proves to be an effective way of performing neuro-morphic computing. Compared to other solutions, it utilizesstandard MTJs that are compatible with STT-MRAM tech-nology, which has been recently developed for mass produc-tion. Additionally, MTJs in such application are very stableover time and they exhibit high endurance in terms of re-programming, comparing to low-energy barrier MTJs usedin probabilistic computing. Moreoever, the presented solu-tion enables more efficient computing, as devices may benefitfrom a multi-state memristor. To validate this, the artificialCMOS-based neuron was designed, consisting of multi-cellbased synapses, differential amplifiers and sigmoid functiongenerator. It was shown that the quantized-weight approachenables the developement of a functional artificial neural net-work, capable of solving recognition problems with accuracylevel similar to the benchmark software model.
ACKNOWLEDGEMENT
We would like to thank PhD J. Wrona from Singulus Tech-nologies AG for MTJ multilayer deposition. Scientific workfunded from budgetary funds for science in 2017-2018, as aresearch project under the "Diamond Grant" program (GrantNo. 0048/DIA/2017/46). W.S. acknowledges support by thePolish National Center for Research and Development grantNo.LIDER/467/L-6/14/NCBR/2015. T.S. acknowledges theSPINORBITRONICS grant No. 2016/23/B/ST3/01430. Thenano-fabrication process was performed at Academic Centrefor Materials and Nanotechnology (ACMiN) of AGH Univer-sity of Science and Technology. J. Fu, H. Zheng, and T. Mei, “Look closer to see better: Recurrent atten-tion convolutional neural network for fine-grained image recognition,” in
Proceedings of the IEEE conference on computer vision and pattern recog-nition (2017) pp. 4438–4446. G. K. Venayagamoorthy, V. Moonasar, and K. Sandrasegaran, “Voice recog-nition using neural networks,” in
Proceedings of the 1998 South AfricanSymposium on Communications and Signal Processing-COMSIG’98 (Cat.No. 98EX214) (IEEE, 1998) pp. 29–32. Y. Zhang, S. Li, and H. Guo, “A type of biased consensus-based distributedneural network for path planning,” Nonlinear Dynamics , 1803–1815(2017). K. Muralitharan, R. Sakthivel, and R. Vishnuvarthan, “Neural networkbased optimization approach for energy demand prediction in smart grid,”Neurocomputing , 199–208 (2018). K. Abhishek, M. Singh, S. Ghosh, and A. Anand, “Weather forecastingmodel using artificial neural network,” Procedia Technology , 311–318(2012). E. Nurvitadhi, D. Sheffield, J. Sim, A. Mishra, G. Venkatesh, andD. Marr, “Accelerating binarized neural networks: Comparison of FPGA,CPU, GPU, and ASIC,” in (IEEE, 2016) pp. 77–84. P. Yao, H. Wu, B. Gao, J. Tang, Q. Zhang, W. Zhang, J. J. Yang, andH. Qian, “Fully hardware-implemented memristor convolutional neural net-work,” Nature , 641–646 (2020). P. Yao, H. Wu, B. Gao, S. B. Eryilmaz, X. Huang, W. Zhang, Q. Zhang,N. Deng, L. Shi, H.-S. P. Wong, et al. , “Face classification using electronicsynapses,” Nature communications , 1–8 (2017). S. Yu, “Neuro-inspired computing with emerging nonvolatile memorys,”Proceedings of the IEEE , 260–285 (2018). S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. di Nolfo,S. Sidler, M. Giordano, M. Bodini, N. C. Farinha, et al. , “Equivalent-accuracy accelerated neural-network training using analogue memory,” Na-ture , 60–67 (2018). D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, “The missingmemristor found,” nature , 80–83 (2008). G. W. Burr, R. M. Shelby, A. Sebastian, S. Kim, S. Kim, S. Sidler, K. Vir-wani, M. Ishii, P. Narayanan, A. Fumarola, L. L. Sanches, I. Boybat, M. L.Gallo, K. Moon, J. Woo, H. Hwang, and Y. Leblebici, “Neuromorphic com-puting using non-volatile memory,” Advances in Physics: X , 89–124(2017). Q. Wu, W. Banerjee, J. Cao, Z. Ji, L. Li, and M. Liu, “Improvement ofdurability and switching speed by incorporating nanocrystals in the HfOxbased resistive random access memory devices,” Applied Physics Letters , 023105 (2018). J. Grollier, D. Querlioz, and M. D. Stiles, “Spintronic nanodevices forbioinspired computing,” Proceedings of the IEEE , 2024–2039 (2016). W. A. Borders, A. Z. Pervaiz, S. Fukami, K. Y. Camsari, H. Ohno, andS. Datta, “Integer factorization using stochastic magnetic tunnel junctions,”Nature , 390–393 (2019). M. Romera, P. Talatchian, S. Tsunegi, F. Abreu Araujo, V. Cros, P. Bor-tolotti, J. Trastoy, K. Yakushiji, A. Fukushima, H. Kubota, S. Yuasa,M. Ernoult, D. Vodenicarevic, T. Hirtzlin, N. Locatelli, D. Querlioz,and J. Grollier, “Vowel recognition with four coupled spin-torque nano-oscillators,” Nature , 230–234 (2018). B. Moons, K. Goetschalckx, N. Van Berckelaer, and M. Verhelst, “Mini-mum energy quantized neural networks,” in (IEEE, 2017) pp. 1921–1925. I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Quan-tized neural networks: Training neural networks with low precision weightsand activations,” The Journal of Machine Learning Research , 6869–6898(2017). T. G. Toledo, B. Perach, D. Soudry, and S. Kvatinsky, “MTJ-Based Hard-ware Synapse Design for Quantized Deep Neural Networks,” arXiv preprintarXiv:1912.12636 (2019). P. Rzeszut, W. Skowro´nski, S. Zi˛etek, J. Wrona, and T. Stobiecki, “Multi-bitMRAM storage cells utilizing serially connected perpendicular magnetictunnel junctions,” Journal of Applied Physics , 223907 (2019). D. Zhang, L. Zeng, K. Cao, M. Wang, S. Peng, Y. Zhang, Y. Zhang, J.-O.Klein, Y. Wang, and W. Zhao, “All spin artificial neural networks based oncompound spintronic synapse and neuron,” IEEE transactions on biomedi-cal circuits and systems , 828–836 (2016). J. Torrejon, M. Riou, F. A. Araujo, S. Tsunegi, G. Khalsa, D. Querlioz,P. Bortolotti, V. Cros, K. Yakushiji, A. Fukushima, et al. , “Neuromorphiccomputing with nanoscale spintronic oscillators,” Nature , 428 (2017). S. Lequeux, J. Sampaio, V. Cros, K. Yakushiji, A. Fukushima, R. Mat-sumoto, H. Kubota, S. Yuasa, and J. Grollier, “A magnetic synapse: mul-tilevel spin-torque memristor with perpendicular anisotropy,” Scientific re-ports , 31510 (2016). C. Sung, H. Hwang, and I. K. Yoo, “Perspective: A review on memristivehardware for neuromorphic computation,” Journal of Applied Physics ,151903 (2018). O. Sulymenko, O. Prokopenko, I. Lisenkov, J. Åkerman, V. Tyberkevych,A. N. Slavin, and R. Khymyn, “Ultra-fast logic devices using artificial“neurons” based on antiferromagnetic pulse generators,” Journal of AppliedPhysics , 152115 (2018). S. Fukami and H. Ohno, “Perspective: Spintronic synapse for artificial neu-ral network,” Journal of Applied Physics , 151904 (2018). L. Deng, “The mnist database of handwritten digit images for machinelearning research [best of the web],” IEEE Signal Processing Magazine ,141–142 (2012). J. Sun and D. Ralph, “Magnetoresistance and spin-transfer torque in mag-netic tunnel junctions,” Journal of Magnetism and Magnetic Materials ,1227 – 1237 (2008). S. Fiorentini, R. L. de Orio, S. Selberherr, J. Ender, W. Goes, andV. Sverdlov, “Analysis of switching under fixed voltage and fixed current inperpendicular stt-mram,” IEEE Journal of the Electron Devices Society ,1249–1256 (2020). ulti-state MRAM cells for hardware neuromorphic computing 6 P. Rzeszut, W. Skowro´nski, S. Zi˛etek, J. Ch˛eci´nski, J. Wrona, and T. Sto-biecki, “Serspin simulation source code in matlab,” https://gitlab.com/spin-electronics-agh/serspin-simulation , accessed: 2020-01-30. A. Fukushima, T. Seki, K. Yakushiji, H. Kubota, H. Imamura, S. Yuasa, andK. Ando, “Spin dice: A scalable truly random number generator based onspintronics,” Applied Physics Express , 083001 (2014). A. Amirany, M. H. Moaiyeri, and K. Jafari, “Nonvolatile associative mem-ory design based on spintronic synapses and cntfet neurons,” IEEE Trans-actions on Emerging Topics in Computing , 1–1 (2020). G. Khodabandehloo, M. Mirhassani, and M. Ahmadi, “Analog implemen-tation of a novel resistive-type sigmoidal neuron,” IEEE Transactions onVery Large Scale Integration (VLSI) Systems20