Probabilistic Resistive Switching Device modeling based on Markov Jump processes
11 Probabilistic Resistive Switching Device modelingbased on Markov Jump processes
Vasileios Ntinas, Antonio Rubio, Georgios Ch. Sirakoulis
Abstract
In this work, a versatile mathematical framework for multi-state probabilistic modeling of Resistive Switching(RS) devices is proposed for the first time. The mathematical formulation of memristor and Markov jump processesare combined and, by using the notion of master equations for finite-states, the inherent probabilistic time-evolutionof RS devices is sufficiently modeled. In particular, the methodology is generic enough and can be applied for N states; as a proof of concept, the proposed framework is further stressed for both two-state RS paradigm,namely N = 2 , and multi-state devices, namely N = 4 . The presented I-V results demonstrate in a qualitative andquantitative manner, adequate matching with other modeling approaches. Index Terms
ReRAM devices, Probabilistic Modeling, Markov Processes, Cycle-to-Cycle variability.
I. I
NTRODUCTION
Novel nano-devices with resistive switching (RS) properties have attracted a lot of attention, mainlydue to their unprecedented electrical characteristics suitable for novel memory and processing systems[1]. At the same time, they flourish in the area of artificial neural networks and neuromorphic computingbecause of their adaptive conductivity enabling them as the electronic analog to the biological synapses.Fast switching, non-volatility and low energy consumption are just a few of their evident advantages.Devices like resistive RAM [2], phase-change memories [3], and ferroelectric RAM [4] can modify theirresistivity (resistive switching) under the application of external electrical stimulus, while they maintainit unchanged in the lack of stimulus.Since 2008, when HP Labs [5] managed to bridge RS devices with the memristor conception [6], awide range of mathematical models have been developed to describe the behavior of the aforementionedfabricated devices [7]–[12]. From atomistic-level precision up to more abstract-level compact models, allmodels aim to provide the necessary tools for the researchers as well as circuit and system designers (incase of compact models) to understand and utilize the novel characteristics of RS devices [13] targetingon novel RS related applications like memories and in memory computing. More specifically, the existingcompact models are able to either describe the underlining physical mechanisms or, phenomenologically,the behavior of the devices. However, RS is governed by stochastic phenomena at microscopic level[14] that most of the compact models lack to capture, thus, they insert random perturbations to the statevariables or parameters to incorporate such behavior [15]. On the other hand, another category of inherentlystochastic models is the kinetic Monte Carlo (kMC) based modeling approach [16], [17], which describesat microscopic level the field-enabled stochastic dispositions of ions within the switching layer. However,kMC is not a compact modeling approach, while there are only a few compact models developed onstochastic approaches of RS limited to the two-state (binary) switching of the device [18].In this work, for the first time, a versatile mathematical framework for the probabilistic modeling of RSdevices with multiple finite states is presented. The proposed framework utilizes the finite-state MasterEquations for Markov jump processes [19] for the development of compact model of the stochastic RS in
V. Ntinas, and G. Ch. Sirakoulis are with the Department of Electrical and Computer Engineering, Democritus University of Thrace,Xanthi, Greece (e-mail: { vntinas, gsirak } @ee.duth.gr).A. Rubio and V. Ntinas, also, are with the Department of Electronics Engineering, Universitat Polit´ecnica de Catalunya, Barcelona, Spain(e-mail: { antonio.rubio, vasileios.ntinas } @upc.edu). a r X i v : . [ c s . ET ] S e p such devices, encapsulating local irregularities of the stochastic state evolution and multilevel program-ming, which are essential for the design of real applications such as high-radix arithmetic operationsand neuromorphic computing. As a generic framework, the proposed approach is not focused to anyparticular type of RS devices, but aims to formulate a probabilistic basis for RS devices that sufferfrom stochastic switching. In this work, we expose the proposed theoretical principles to deliver differentmodeling paradigms, one for two-state and one for four-state (2-bit), respectively adopting characteristicsof existing RS devices. II. M ARKOV J UMP P ROCESS M ASTER E QUATIONS
The notion of master equations is widely used in various scientific fields, from classical mechanicsand chemistry to quantum mechanics, where stochasticity is inherent property of the system, like particlekinetics, chemical reactions and quantum state vectors [20]–[22], as they provide simplified mathematicalformulation of complex stochastic systems. The master equations are sets of low-order differential equa-tions that describe the evolution of the probability over time of a discrete-state system and can be solvedwithout the use of nontrivial numerical methods alike the stochastic differential equations.Let S denote a system which, at any time t , can be in one of the states of X = x , x , ..., x N such S ( t ) = x i , where i = 1 , , ..., N and N ∈ N . We define the probability of S ( t ) to be in state x i as P i ( t ) .Assuming a Markov jump process with transition rate w i,j ≥ from x i to x j with { j = 1 , , ..., N | j (cid:54) = i } ,the master equations that describe the time evolution of probability P i ( t ) read: dP i ( t ) dt = N (cid:88) j =1 ,i (cid:54) = j (cid:104) w j,i · P j ( t ) − w i,j · P i ( t ) (cid:105) . (1)In (1), right side, the first term derives from the finite Kolmogorov forward equations that correspond tothe escape rates between states, whereas the second term derives from the finite Kolmogorov backwardequations where the probability to stay in x i is expressed as the negative sum of the transition rates fromall x j to x i . Moreover, the conservation of total probability holds, i.e.: ddt N (cid:88) i =1 P i ( t ) = 0 (2)resulting in (cid:80) Ni =1 P i ( t ) = 1 , ∀ t , if the initial state of the systems is (cid:80) Ni =1 P i ( t ) = 1 . Thus, equation (1)stands for any value of w i,j , which are natural entities not bounded by the limitation of probabilities inthe interval [0 , . III. RS D EVICE M ASTER E QUATIONS
By definition, RS devices are elements with memory, mathematically described by a state variable x and with switching behaviour attributed to various physical entities, e.g. the length of the conductivefilament. Even if the direct correlation of such devices and memristor notion is still arguable [23], themathematical description of memristor is often used to develop RS devices’ models. Thus, along with thestate equation(s) that describe the state evolution over time, the conductance equation of the RS deviceis formulated by the state-dependent Ohm’s law. A general form of a voltage-controlled memristor thatmodels RS devices reads: i ( t ) = g ( x ( t ) , v ( t )) (3) dx ( t ) dt = f ( x ( t ) , v ( t )) (4)where i ( t ) , v ( t ) , and x ( t ) are the time-dependent current through, voltage across and state of the device,respectively. Moreover, g ( x ( t ) , v ( t )) is a function of the device’s conductance and f ( x ( t ) , v ( t )) is the function that governs the state evolution of the device, both dependent on the state and voltage acrossthe device, which represents the external stimuli in this voltage-dependent form. Accordingly, the current-dependent form of such systems can be extracted by interchanging i ( t ) and v ( t ) in (3)-(4) and replacing g ( x ( t ) , v ( t )) with a state- and current-dependent function of device’s resistance r ( x ( t ) , i ( t )) . Without lossof generality, the voltage-dependent form is considered for the rest of the manuscript, where the externallyapplied voltage represents the system’s input.Focusing on the state evolution function f ( x ( t ) , v ( t )) , it is obvious that the state at any time τ = t + ∆ t ,where t, ∆ t > stands and [ t, t + ∆ t ] is an infinitesimal interval, is calculated as a function of the stateand the external stimuli at t . In particular, considering a discrete time evolution, which applies for anynumerical integration method, the state at τ is x τ = x t + ∆ t · f ( x t , v t ) = F ( x t , v t , ∆ t ) , where F is anydiscrete-time numerical integration method. Thus, it is clear that x τ is independent of any previous state x s , where s < t , and complies with the Markov memoryless property that is required for a system to bedescribed as a Markov process.Assuming now a finite-state RS device with instantaneous switching between the states, the stateevolution can be described as a Markov jumping process. This assumption stands for high transitionspeeds, which is valid for RS devices under high amplitude and/or low frequency stimuli; however,the proposed multilevel approach extends the validity of this assumption as it provides finer granularitybetween states.Taking into account the above, the modeling of stochastic RS device can be performed by the combi-nation of memristor’s mathematical formulation and the master equations of Markov jump processes. Therequired step to model a RS device with the proposed mathematical framework is to estimate the transitionrates w i,j . Since the RS devices are dependent to the external stimuli and they are non-volatile, w i,j canbe estimated by the combination of the voltage-dependent ( w v t i,j | t ) and the state-dependent rates ( w x t i,j | t )that are also time-varying functions. The former one can be approximated by the switching frequenciesbetween the states, which corresponds to the inverse of the switching times of the device, such as: w v t i,j | t = (cid:0) t SWi,j ( v t ) (cid:1) − , (5)where t SWi,j ( v t ) is the time needed by memristor to switch from state x i to x j . These rates can be useddirectly in (1), denoting the voltage-dependent probability evolution equation, i.e. dP v t i /dt . On the otherhand, the state-conservation rates for state-dependent probability evolution, due to the non-volatility, aredefined as: dP x t i,t dt = N (cid:88) j =1 i (cid:54) = j (cid:104) − k ( x j , x t ) · P i,t + k ( x i , x t ) · P j,t (cid:105) , (6) k ( x i , x t ) = (cid:40) η i , x i = x t , x i (cid:54) = x t , (7)where η i is the conservation rate of state x i and (6) satisfies (2). Thus, a finite-state RS device withprobabilistic switching in discrete time is formulated as: i t = g ( x t , v t ) , (8) d P t dt = d P v t t dt + d P x t t dt = W v t t · P t + W x t t · P t = W t · P t , (9) The memory of RS devices and the Markov property need not to be confused, as the first attributes the physical capability of the deviceto maintain its state under no external excitation. where in bold are the vector-form of the probabilities with size of N × , whereas, W t is the transitionmatrix of the system composed of the voltage-dependent transition matrix W v t t and the state-dependentconservation matrix W x t t , with size N × N . In addition, the calculation of the state becomes implicitas it depends on the probability to be in any state and a random variable U t , which follows a uniformdistribution bounded in the interval [0, 1] and acts as the selector of the state, and it reads: x t = x i , U t − ∈ (cid:104) (cid:80) i − j =1 P j,t − , (cid:80) ij =1 P j,t − (cid:17) (10)with the exception that P N,t − includes the upper bound of the interval, i.e. the value . Moreover, whena jump from x i to x j occurs with regard to (10), the probability vector P t is redefined as P j , t = 1 and P l = 0 , where { l = 1 , , ..., N | l (cid:54) = j } , because the probability of the system to be at x j in the nexttime-step is the maximum. IV. V ALIDATION
The verification of versatility and expected functionality of the proposed stochastic RS device modelis provided through different paradigms. More specifically, the most common case, namely a two-state(binary) RS device model, is formulated to show in details how the presented framework is applied.Furthermore, the multilevel capabilities of the framework are adequately delivered with a four-state (2-bit) probabilistic RS device example.The proposed generic mathematical framework model covers the macro-modeling of the RS devicesstate evolution, so the conductance function g ( x t , v t ) can be indifferent in regard to the conductancemechanisms that take place in the structure of each fabricated device, like for example quantum tunneling,Schottky emission, Poole-Frenkel effect, etc. In particular, for the two selected examples, we will adoptthe conductance mechanisms of previous works, and more specifically, ohmic conductance for the binarycase and the combination of Schottky emission and ohmic conductance for the 2-bit case.For the application of the aforementioned modeling framework, the transition matrix needs to be defined.In case of a simplified pulse programming scheme, where the amplitude of pulses is fixed, the transitionrates can be pre-defined as a set of coefficients. However, for the general case of continuous range ofinput voltages, the voltage-dependent functions of transition rates require to be estimated.So, starting from the binary example case ( N = 2 ), measurements on fabricated RS devices with abruptbinary switching have shown that their switching time can be estimated by voltage-dependent log-normalor Poisson distributions [24], [25], according to the type of the RS device. Apparently, in this example,the Poisson-like switching from [25] is adopted, i.e. t SWi,j | t = α/ exp ( v t /β ) , where α and β are fittingparameters. Thus, by using (5) and assuming: P t = (cid:34) P on,t P off,t (cid:35) (11)the transition matrix reads: W t = (cid:34) − / ( t SWrst | t ) 1 / ( t SWset | t )1 / ( t SWrst | t ) − / ( t SWset | t ) (cid:35) + (cid:34) − k ( R off ) k ( R on ) k ( R off ) − k ( R on ) (cid:35) (12)where t SWrst | t ( t SWset | t ) is the voltage-dependent switching time for the transition R on -¿ R off ( R off -¿ R on ), whichis assumed infinite when v t ≥ V ( v t ≤ V), and k is calculated from (6) with η on and η off representingthe fitting parameters of the conservation rates. Utilizing (8), (9), (10) with the probability vector (11) and transition matrix (12), the time evolution ofthe binary switching paradigm under a sinusoidal excitation is presented in Fig. 1. In particular, for theduration of two input signal periods, the probability and the state evolution are shown and for this specificrealization, the jump R off -¿ R on never occurred for the second period, while the stochastic switching isalso evident in the first period, where the two jumps occurred randomly in different time, even thoughthe switching parameters are selected to be equal. A more general preview of the stochastic behavior ofthe binary model is illustrated through the I-V characteristic under sinusoidal excitation. Firstly, Fig. 2(a)shows a set of 1000 sweeps of fixed frequency and voltage amplitude { Hz , . V } and the ensembleaverage of the current (orange line), while Figs. 2(b, c) illustrate the over 1000sweeps for different voltage amplitude and frequency values, respectively. The frequency dependency isone of the key features of memristors.Furthermore, another, multi-state this time paradigm, with a 2-bit ( N = 4 ) stochastic RS device model ispresented. In this case, the transition rates are estimated according to the switching time of a deterministicmultistate model [26] that describes the behavior of fabricated multilevel RS devices such as in [27]. Underthese circumstances, the probability vector P t is a × vector, while the size of the transition matrix W t is × and its values are estimated from the energy-dependent switching in [26]. In particular, theswitching time between the states is expressed as function: t SWi,j | t = γ i,j / ( v t · I i | t ) , (13)where I i | t = ζ i · exp ( (cid:112) | v t | ) for the states with Schottky emission ( i = 1 , , ) and I i | t = ζ i · v t for theohmic conductance state ( i = 4 ). The value of ζ i derives from an amount of parameters in [26] and canbe embodied in the parameter γ i,j , so the voltage-dependent switching time reads: t SWi,j | t = (cid:40) γ i,j / (cid:0) v t · exp ( (cid:112) | v t | ) (cid:1) , i = 1 , , γ i,j /v t , i = 4 . (14)Equations (14) and (5) are used to construct W v t t for the N = 4 paradigm, whereas (6) and (7) withstate conservation rates η i are used for W x t t . Similarly to the binary paradigm, the switching time t SWi,j | t for i > j ( j < i ) is assumed infinite when v t ≥ V ( v t ≤ V), as well as when | i − j | > . The inputvoltage, the probability of the 4 states and the state evolution over time for two randomly-selected periodsof the applied voltage are presented respectively in Fig. 3. Under the influence of the applied voltage, theprobability of being in each state is varying, while the stochastic switching is clearly depicted in Fig. 4(c)through the randomly occurrence of transitions between the states. In addition, Fig. 4 presents the I-Vcharacteristic of the 2-bit stochastic model for a sinusoidal excitation with frequency 10 Hz during 100periods, in comparison to the deterministic 2-bit RS device model from [26], along with the ensemblemean switching time between states of the stochastic one (
In this work, we postulate a novel probabilistic framework to model the stochastic resistive switchingof nano-devices using multiple finite states, surpassing the limitations of the existing binary RS devicestochastic models. Such a model has a direct application in multi-bit storage, multi-value logic operationsand neuromorphic computing when limited number of resistive states is used due to the constrains of thestate retention of fabricated RS devices. The mathematical background of the proposed framework is based on the formulation of memristor and the Master Equations of the Markov Jump processes, which can beboth integrated to conventional circuit simulators through either SPICE or compact Verilog-A models.Next steps are the mitigation to any of these circuit simulation platforms, along with the introductionof higher-level features, such as the resistance range degradation due to aging or over-tuning in order toestablish a complete probabilistic platform to evaluate the reliability of the RS device-based systems.VI. A
CKNOLEDGEMENT
This work was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under theFirst Call for H.F.R.I. Research Projects to support Faculty members and Researchers and the procurementof high-cost research equipment grant (Project Number: 3830).R
EFERENCES [1] D. Ielmini and H. P. Wong, “In-memory computing with resistive switching devices,”
Nature Electronics , vol. 1, no. 6, pp. 333–343,2018.[2] H. P. Wong, H.-Y. Lee, S. Yu, Y.-S. Chen, Y. Wu, P.-S. Chen, B. Lee, F. T. Chen, and M.-J. Tsai, “Metal–oxide RRAM,”
Proceedingsof the IEEE , vol. 100, no. 6, pp. 1951–1970, 2012.[3] S. W. Fong, C. M. Neumann, and H.-S. P. Wong, “Phase-change memorytowards a storage-class memory,”
IEEE Transactions onElectron Devices , vol. 64, no. 11, pp. 4374–4385, 2017.[4] A. Chanthbouala, V. Garcia, R. O. Cherifi, K. Bouzehouane, S. Fusil, X. Moya, S. Xavier, H. Yamada, C. Deranlot, N. D. Mathur et al. , “A ferroelectric memristor,”
Nature Materials , vol. 11, no. 10, pp. 860–864, 2012.[5] D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, “The missing memristor found,”
Nature , vol. 453, no. 7191, pp. 80–83,2008.[6] L. Chua, “Memristor-the missing circuit element,”
IEEE Transactions on Circuit Theory , vol. 18, no. 5, pp. 507–519, 1971.[7] Z. Biolek, D. Biolek, and V. Biolkova, “Spice model of memristor with nonlinear dopant drift.”
Radioengineering , vol. 18, no. 2, 2009.[8] S. Kvatinsky, E. G. Friedman, A. Kolodny, and U. C. Weiser, “TEAM: Threshold adaptive memristor model,”
IEEE Transactions onCircuits and Systems I: Regular Papers , vol. 60, no. 1, pp. 211–221, 2012.[9] A. Ascoli, F. Corinto, V. Senger, and R. Tetzlaff, “Memristor model comparison,”
IEEE Circuits and Systems Magazine , vol. 13, no. 2,pp. 89–105, 2013.[10] Z. Jiang, Y. Wu, S. Yu, L. Yang, K. Song, Z. Karim, and H.-S. P. Wong, “A compact model for metal–oxide resistive random accessmemory with experiment verification,”
IEEE Transactions on Electron Devices , vol. 63, no. 5, pp. 1884–1892, 2016.[11] V. Ntinas, A. Ascoli, R. Tetzlaff, and G. C. Sirakoulis, “A complete analytical solution for the on and off dynamic equations of a TaOmemristor,”
IEEE Transactions on Circuits and Systems II: Express Briefs , vol. 66, no. 4, pp. 682–686, 2018.[12] I. Messaris, A. Serb, S. Stathopoulos, A. Khiat, S. Nikolaidis, and T. Prodromakis, “A data-driven Verilog-A ReRAM model,”
IEEETransactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 37, no. 12, pp. 3151–3162, 2018.[13] D. Ielmini and V. Milo, “Physics-based modeling approaches of resistive switching devices for memory and in-memory computingapplications,”
Journal of Computational Electronics , vol. 16, no. 4, pp. 1121–1143, 2017.[14] S. H. Jo, K.-H. Kim, and W. Lu, “Programmable resistance switching in nanoscale two-terminal devices,”
Nano letters , vol. 9, no. 1,pp. 496–500, 2009.[15] H. Li, Z. Jiang, P. Huang, Y. Wu, H. Chen, B. Gao, X. Liu, J. Kang, and H.-S. Wong, “Variation-aware, reliability-emphasized designand optimization of RRAM using SPICE model,” in .IEEE, 2015, pp. 1425–1430.[16] S. Yu, X. Guan, and H.-S. P. Wong, “On the stochastic nature of resistive switching in metal oxide rram: Physical modeling, montecarlo simulation, and experimental characterization,” in . IEEE, 2011, pp. 17–3.[17] S. Menzel, P. Kaupmann, and R. Waser, “Understanding filamentary growth in electrochemical metallization memory cells using kineticmonte carlo simulations,”
Nanoscale , vol. 7, no. 29, pp. 12 673–12 681, 2015.[18] V. Dowling, V. Slipko, and Y. Pershin, “Probabilistic memristive networks–part i: Application of a master equation to networks ofbinary reram cells,” arXiv preprint arXiv:2003.11011 , 2020.[19] R. Toral and P. Colet,
Stochastic numerical methods: an introduction for students and scientists . John Wiley & Sons, 2014.[20] G. Kaniadakis and P. Quarati, “Kinetic equation for classical particles obeying an exclusion principle,”
Physical Review E , vol. 48,no. 6, p. 4263, 1993.[21] M. J. Pilling and S. H. Robertson, “Master equation models for chemical reactions of importance in combustion,”
Annual review ofphysical chemistry , vol. 54, no. 1, pp. 245–275, 2003.[22] A. Rivas, A. D. K. Plato, S. F. Huelga, and M. B. Plenio, “Markovian master equations: a critical study,”
New Journal of Physics ,vol. 12, no. 11, p. 113032, 2010.[23] J. Kim, Y. Pershin, M. Yin, T. Datta, and M. Di Ventra, “An experimental proof that resistance-switching memories are not memristors,” arXiv preprint arXiv:1909.07238 , 2019.[24] G. Medeiros-Ribeiro, F. Perner, R. Carter, H. Abdalla, M. D. Pickett, and R. S. Williams, “Lognormal switching times for titaniumdioxide bipolar memristors: origin and resolution,”
Nanotechnology , vol. 22, no. 9, p. 095702, 2011.[25] R. Naous, M. Al-Shedivat, and K. N. Salama, “Stochasticity modeling in memristors,”
IEEE Transactions on Nanotechnology , vol. 15,no. 1, pp. 15–28, 2015. [26] F. Garc´ıa-Redondo, R. P. Gowers, A. Crespo-Yepes, M. L´opez-Vallejo, and L. Jiang, “Spice compact modeling of bipolar/unipolarmemristor switching governed by electrical thresholds,”
IEEE Transactions on Circuits and Systems I: Regular Papers , vol. 63, no. 8,pp. 1255–1264, 2016.[27] W. Chen, W. Lu, B. Long, Y. Li, D. Gilmer, G. Bersuker, S. Bhunia, and R. Jha, “Switching characteristics of W/Zr/HfO /TiN ReRAMdevices for multi-level cell non-volatile memory applications,” Semiconductor Science and Technology , vol. 30, no. 7, p. 075002, 2015.
Time (s) -1.301.3 A pp li ed V o l t age ( V ) Time (s) -12 -9 -6 -3 P r obab ili t y P on,t P off,t Time (s)
ONOFF S t a t e (a)(b)(c) Fig. 1. Binary stochastic switching. (a) Input voltage. (b) Probability and (c) State evolution over time. The simulation results obtained for α set = α rst = 3 × , β set = β rst = 0 . , and η on = η off = 1 . × . Fig. 2. (a) I-V curve for binary stochastic switching and . (b, c) -V curve for various amplitudes and frequencies. Time (s) -303 A pp li ed V o l t age ( V ) Time (s) -12 -9 -6 -3 P r obab ili t y P P P P Time (s) P P P P S t a t e (a)(b)(c)(a)(b)(c)(a)(b)(c)(a)(b)(c)(a)(b)(c) Fig. 3. 2-bit ( N = 4 ) stochastic switching. (a) Input voltage, (b) Probability and (c) State evolution over time. The simulation results wereobtained for { γ , , γ , , γ , , γ , , γ , , γ , } = { . , . , . , . × − , . × − , . } and η = η = η = η = 3 × . Fig. 4. Logarithmic I-V curve of the 2-bit ( N = 4= 4