Metabolic constraints on synaptic learning and memory
aa r X i v : . [ q - b i o . N C ] O c t Published in:
Journal of Neurophysiology : 1473-1490 (2019).
Metabolic constraints on synaptic learning and memory
Jan Karbowski a,b ( a ) Institute of Applied Mathematics and Mechanics,University of Warsaw, ul. Banacha 2, 02-097 Warsaw, Poland; ( b ) Nalecz Institute of Biocybernetics and Biomedical Engineering,Polish Academy of Sciences, 02-109 Warsaw, Poland
Abstract
Dendritic spines, the carriers of long-term memory, occupy a smallfraction of cortical space, and yet they are the major consumers ofbrain metabolic energy. What fraction of this energy goes for synap-tic plasticity, correlated with learning and memory? It is estimatedhere based on neurophysiological and proteomic data for rat brainthat, depending on the level of protein phosphorylation, the energycost of synaptic plasticity constitutes a small fraction of the energyused for fast excitatory synaptic transmission, typically 4 . − . Keywords : energy cost of learning and memory; molecular mechanisms of synapticplasticity; ATP-driven reactions; phosphorylation; cascade models; memory lifetime;metabolism; non-equilibrium steady state; entropy production.
Author Summary:
Learning and memory involve a sequence of molecular events in dendritic spines, calledsynaptic plasticity. These events are physical in nature and require energy, which hasto be supplied by ATP molecules. However, our knowledge of the energetics of theseprocesses is very poor. This study estimates the empirical energy cost of synapticplasticity, and considers theoretically a metabolic rate of learning and its memory tracein a class of cascade models of synaptic plasticity.2
NTRODUCTION
Brain is one of the most expensive organs in the body (Aiello and Wheeler 1995;Attwell and Laughlin 2001; Clarke and Sokoloff 1994), and larger brains consume cor-respondingly more energy (Karbowski 2007, 2011). Data indicate that much of thisenergy is used by synapses (Harris et al 2012; Karbowski 2014). For example, it wasestimated that fast excitatory synaptic transmission requires 1 . · ATP moleculesper presynaptic stimulation to pump out an influx of Na + ions (Attwell and Laughlin2001), which assuming 1 Hz for such stimulation in the rat cortex (see below) yields ametabolic rate of 8 . · ATP/min per spine.High metabolic requirement of synapses is also visible during mammalian brain de-velopment when cerebral metabolic rate changes in proportion to changes in synapticdensity (Karbowski 2012). On the other hand, imaging experiments show that brainstimulation increases cerebral metabolic rate only weakly by ∼
10% from its resting(baseline) value (Shulman and Rothman 1998; Shulman et al 1999), which may suggestthat housekeeping processes in the brain are more energy demanding than acquiringand processing of a new information (Shulman et al 2004; Raichle and Mintun 2006).Interestingly, recent data on cortical stimulation and energetics of synaptic transmis-sion in rodents reveal that the small increase in the cortical metabolic rate is sharedproportionally between neurons and astrocytes (Sonnay et al 2016, 2018), implying thatboth neuronal and glial compartments are important for synaptic function.Learning and memory in the brain is strictly associated with plasticity mechanismsin synapses (Lisman et al 2012; Kandel et al 2014; Takeuchi et al 2014; Poo et al 2016).There exist a huge literature on modeling of synaptic plasticity and memory (for reviews3ee e.g. Bhalla 2014; Chaudhuri and Fiete 2016), but virtually all of it neglects theenergetic aspect. This omission is surprising, because most of what we know about realtime brain workings is based on imaging techniques, which rely on brain metabolism(Shulman et al 2004; Raichle and Mintun 2006). Moreover, molecular processes insynapses are physical in nature and require a permanent energy influx to counteractdissipation, which is related to ATP (and/or GTP) hydrolysis (Engl and Attwell 2015;Rolfe and Brown 1997; Phillips et al 2012). This means that there should exist somecost to learning and memory in the brain, and an open fundamental questions is howbig is it, and to what extent it can constrain the strength and duration of a memorytrace?Synaptic plasticity mechanisms related to long term potentiation (LTP) and de-pression (LTD) are induced by calcium influx to a dendritic spine through NMDAand voltage-gated receptors, and subsequent activation of various enzymes, such asCAMKII, PSD-95, protein kinase A and C, MAPK, etc (Lisman et al 2012; Kandelet al 2014; Bhalla and Iyengar 1999). These enzymes serve as upstream initiators ofcomplex molecular signaling pathways from spine membrane to postsynaptic density(PSD), within PSD, and beyond in the form of protein activation cascades (Sheng andHoogenraad 2007; Zhu et al 2016). The most common mechanism of protein activa-tion is by phosphorylation (adding of a phosphate group), which is powered by ATPhydrolysis (Qian 2007), and it was found that many PSD proteins are phosphorylatedduring initiation of LTP and LTD (Coba et al 2009; Li et al 2016). Thus, protein phos-phorylation cascades provide a basic biochemical mechanism of signal transduction inplastic synapses. The end product of the phosphorylation cascades is protein synthe-4is in PSD, actin polymerization, and AMPA receptor trafficking on spine membrane(Cingolani and Goda 2008; Lisman et al 2012; Bosch et al 2014). All of these pro-cesses influence synaptic conductance/weight (Kasai et al 2003; Meyer et al 2014), andthey cause some energy drain (consumption of ATP), which magnitude is essentiallyunknown. This paper provides an estimate of the metabolic cost of these reactions.Phenomenological models of cascade synaptic plasticity mimic the richness of bio-chemical pathways in dendritic spines and provide simple means to study theoreticallysynaptic memory maintenance (Fusi et al 2005; Leibold and Kempter 2008; Barrett etal 2009; Benna and Fusi 2016). Recent developments within these models shed light onthe importance of internal synaptic complexity, i.e. bidirectionality of synaptic transi-tions and multiple time scales, for producing long memory lifetimes (Benna and Fusi2016). The aim of this study is to consider the energy requirement for learning andmaintaining of a new information in relation to the baseline cost of synaptic plasticityin a class of cascade models. It is argued that these models, in order to be a minimallyphysically realistic, must contain bidirectional transitions and cyclic motifs (modeledhere as phosphorylation-dephosphorylation biochemical reactions). These two featuresare necessary to generate nonzero and finite synaptic metabolic rate at a steady state,corresponding to a baseline synaptic activity, which presumably stores the memoriesof all previous plasticity events. Keeping these memories is physically associated withmaintaining synaptic structure and processes, and this requires an energy influx tocounteract dissipation. This implies that synapses in the steady state or during base-line activity must operate out of thermal equilibrium to freely exchange energy andmaterial with their environment (neurons, glia). This is similar to the behavior of all5iological systems that have to be in nonequilibrium state to avoid “thermal death”(Hill 1989; Nicolis and Prigogine 1977; Qian 2006).6
ESULTSEstimates of energy requirements of various molecular processesinvolved in synaptic plasticity.
Synaptic plasticity of excitatory synapses, i.e. change in synaptic conductance(weight) and size, involves a sequence of molecular events and can be broadly dividedinto three categories: extra-synaptic (outside dendritic spine), intra-synaptic (inside thespine), and modulatory. Below we estimate ATP rates associated with each of thesecontributions to the plasticity based on empirical data for rat brain.
Extra-synaptic cost.
Extra-synaptic input is necessary for the initiation of synaptic plasticity. In particu-lar, the induction of synaptic plasticity requires the influx of Ca +2 ions to the dendriticspine (Miller et al 2005; Lisman et al 2012). Calcium signal serves as a trigger of variousdownstream molecular pathways necessary to induce LTP and/or LTD, with the endresult of changing the spine conductance weight and size (Lisman et al 2012; Poo et al2016). We consider two types of the extra-synaptic costs: plasticity related glutamaterecycling via astrocytes (Sonnay et al 2016, 2018), and plasticity related ATP releasefrom astrocytes that binds to the spine membrane (Khan and North 2012), as the en-ergy associated with these two processes is directly linked to the energy cost of synapticplasticity initiation, i.e. calcium entering the spine.Plasticity related glutamate recycling.The number of glutamate molecules released by one vesicle of a presynaptic terminalis large ( ∼ influx. There are roughly 10NMDA receptors on the spine (Nimchinsky et al 2004), each binding 1 glutamate, witha stimulation rate equal to the presynaptic firing rate times the neurotransmitter releaseprobability. The average firing rate in rat cortex is about 4-5 Hz (Schoenbaum et al1999; Fanselow and Nicolelis 1999; Attwell and Laughlin 2001; Karbowski 2009), whilethe release probability at these frequencies is about 0.25 (Volgushev et al 2004), whichyields about 1 Hz for the rate of NMDA stimulation. When glutamate unbinds fromNMDA, it is recycled for the next use. Approximately 2.67 ATP molecules have to behydrolyzed for each recycled glutamate (Attwell and Laughlin 2001), which happensmostly through astrocytes and involves several steps, such as glutamate uptake, itsmetabolic processing and conversion to glutamine, glutamine transporting to neurons,and glutamate packing into vesicles (Sonnay et al 2016, 2018). Thus in total, theplasticity related glutamate recycling ATP rate is 10 · .
67 ATP/s or 1602 ATP/min.Plasticity related ATP release from astrocytes and binding to spines.Another molecule that binds to the spine receptors is ATP, which in this case playsthe role of a transmitter (Khan and North 2012). ATP released from astrocytes canactivate purinergic P2X receptors (by transducing its energy), which are known tomodulate synaptic plasticity (Pankratov et al 2009), allowing calcium to enter thespine (see also below the section “plasticity modulation”). It was measured that thepostsynaptic current through P2X receptors constitutes only about 10% of the currentpassing through glutamate receptors, mostly AMPA (Khan and North 2012). Since the8umber of AMPA on the spine is about 100 (Matsuzaki et al 2001), this suggests that thenumber of P2X is about 10 (P2X and AMPA channels have comparable conductances;Khakh and North 2012). Each P2X binds 3 ATP molecules for its activation (Bean etal 1990). The rate of ATP release from astrocytes is not known (it is likely activitydependent), but we can assume that the timing between two consecutive releases shouldbe at least as long as the ATP binding time constant, which for most of P2X receptorsis long and more than 20 sec. Assuming 30 sec, this gives us 30 released ATP moleculesrelated to plasticity per 30 sec, or the consumption rate of 60 ATP/min.
Intra-synaptic cost.
Below we estimate the energy requirements for the basic molecular processes in-side a dendritic spine, which are related to plasticity induction and maintenance.These include: Ca intra-spine trafficking, protein phosphorylation, protein synthe-sis (turnover), actin treadmilling, and receptor (AMPA and NMDA) trafficking.Ca removal.Following a presynaptic action potential or postsynaptic backpropagation calcium flowsinto the spine (either through NMDA channels or voltage sensitive Ca channels) andtrigger a cascade of molecular processes, initiating the synaptic plasticity. For a spinevolume of 0.1 µ m (Honkura et al 2008), it was found that about 2000 Ca ions enter,of which about 95% bind to endogenous buffers (Sabatini et al 2002). The remaining100 Ca are free and can activate CaMKII (and possibly other) proteins, but theyhave to be pumped out after the activation to recover baseline physiological conditionsof resting calcium concentration in the spine. The rate of the extrusion is given bypresynaptic stimulation (1 Hz), and is conducted by Ca -ATP pumps or Na + -Ca ion, we find the ATP hydrolysis rate of calcium extrusion from the spine 100 ATP/s or6000 ATP/min.Protein phosphorylation.Protein phosphorylation is the main activation mechanism of downstream proteins,actin, AMPA receptors, and other spine molecules, and thus it is of prime importance(Zhu et al 2016). One cycle of protein phosphorylation requires the hydrolysis of 1ATP molecule (Hill 1989; Qian 2007). First, we estimate the ATP consumption ratefor resting non-LTP related phosphorylation, i.e. for unstimulated spine with restingCa concentration, and then for stimulated spine undergoing LTP.In general, for the resting spine the levels of protein phosphorylation rates seem tobe uniformly distributed and vary by two orders of magnitude, from 0.001 min − to0.34 min − (Molden et al 2014), which yields an average value of 0.15 min − . There areabout 10 proteins (including their copies) in the spine PSD (Sheng and Kim 2011), withan average of 4-6 phosphorylation sites per protein (Collins et al 2005; Trinidad et al2012). This gives the resting rate of ATP-driven protein phosphorylation as (6 − · ATP/min, with an average of 7500 ATP/min. Because hydrolysis of one ATP requires20 kT, where k is the Boltzmann constant and T is the absolute temperature (Phillipset al 2012), we obtain equivalently the resting energy rate of protein phosphorylationas 15 · kT/min in a single spine. This ATP rate is however unrelated to learningand memory, since the resting conditions (unstimulated spine) do not drive plasticityevents, i.e., changes in AMPA receptor number.More relevant rates for LTP-related ATP consumption can be obtained by noting10hat during the plasticity induction, stimulated by Ca influx to the spine, the phos-phorylation rates of about 10% PSD proteins are strongly enhanced as the proteinsinteract more frequently (about 130 proteins out of roughly 1500; Coba et al 2009; Liet al 2016; Bayes et al 2012). The important point is that this fraction is dependenton the frequency of Ca stimulation (De Koninck and Schulman 1998; Gaertner et al2004). For example, the rate of CaMKII autophosphorylation jumps 3 − − (Colbran 1993; Miller et al 2005) to about60 −
600 min − (Bradshaw et al 2002; Miller et al 2005; Michalski 2013), but this am-plified phosphorylation takes place only for a few percent of CaMKII at about 1 Hz ofCa influx (De Koninck and Schulman 1998; Gaertner et al 2004). We can expect thatduring continuing calcium stimulation, as for regular in vivo cortical conditions, somebalance between proteins highly phosphorylated and those at resting phosphorylationis achieved. In such a stationary state, both of these protein groups contribute to theATP consumption rate, which can be written as:˙ AT P phos = N [(1 − x ) r r M s + xr a M ∗ s ],where N (= 10 ) is the total number of proteins (including their copies) in PSD, x (= 0 .
1) is the fraction of proteins with amplified phosphorylation, M s (= 5) is theaverage number of phosphorylation sites per protein, M ∗ s is the average number of suchsites per protein that become highly phosphorylated upon stimulation ( M ∗ s can be anyinteger between 1 and 5). M ∗ s depends on the frequency of Ca stimulation, and itlikely assumes its lowest values ≈ − r a and r r denote the average rates for active and resting phosphorylation(population averaged resting r r = 0 .
15 min − ). There are virtually no data on r a in11SD proteins other than CaMKII. For this reason, in our estimate we take for r a amedium value of the numbers reported for CaMKII, i.e., r a ≈
300 min − . Using theabove parameters, we obtain the steady-state ATP consumption rate during LTP phaseas ˙ AT P phos = 3 . · ATP/min for M ∗ s = 1, and ˙ AT P phos = 9 . · ATP/min for M ∗ s = 3. These values correspond respectively to 20% and 60% of the protein sites withenhanced phosphorylation.As a curiosity, let us estimate the above ˙ AT P phos for different values of the fractionof highly activated proteins x . First, note that for x = 0 (no proteins with amplifiedphosphorylation), the ˙ AT P phos is exactly equal to the resting non-LTP phosphorylationcost calculated above. Next, since ˙
AT P phos increases with x , we want to find out forwhat value of x the ATP cost of protein phosphorylation is equal to the cost of fastsynaptic transmission (8 . · ATP/min)? For M ∗ s = 1 this never happens, and themaximal value of ˙ AT P phos in this case is 3 · ATP/min, which is 36% of the fastsynaptic transmission cost. For M ∗ s = 3 this happens when x = 0 .
94, i.e., almost allPSD proteins would have to be activated by phosphorylation. For the maximal valueof M ∗ s = 5, this situation occurs for x = 0 .
56, i.e. half of the proteins must be highlyactivated. Finally, the maximal possible value of ˙
AT P phos is 15 · ATP/min (all PSDproteins are maximally phosphorylated), which is nearly twice the cost of fast synaptictransmission. The latter hypothetical calculation is useful, because it sets the upperbound on the possible error in estimating the overall cost of synaptic plasticity.It is interesting to estimate what fraction of the resting global phosphorylation costis taken by two the most abundant proteins: CaMKII ( α and β subunits) and PSD-95.There are 5600 copies of CaMKII α and CaMKII β in a spine (Sheng and Kim 2011), and12oth subunits have similar number of 10 phosphorylation sites (Trinidad et al 2006).The number of copies of the PSD-95 protein in a spine is 300 (Sheng and Kim 2011),and it has between 8 and 12 phosphorylation sites (Trinidad et al 2006; Zhang et al2011). Assuming that both proteins have the same resting phosphorylation rates, i.e.,0.03 min − , corresponding to CaMKII autophosphorylation, we get the energy rate1700 ATP/min for CaMKII, and 60 −
100 ATP/min for PSD-95. If we combine thesetwo numbers, we get that these two proteins consume 20 −
30% of the global restingphosphorylation energy rate of the whole postsynaptic density PSD. This indicatesthat these two plentiful proteins, comprising about 60% of the PSD content, consume asubstantial part of the whole energy devoted to protein phosphorylation during restingnon-LTP spine conditions.Protein synthesis/turnover.The cost of protein turnover is estimated as follows. The total molecular mass of atypical PSD has been calculated as 1 . · Da (Chen et al 2005), and this correspondsto a total number of 10 amino acids, which are bound by the same number of peptidebonds that require 4 ATP molecules/bond to form (Engl and Attwell 2015). The averagehalf-lifetime of PSD proteins is 3.67 days (Cohen et al 2013), which means that afterthat time a half of all peptide bonds are broken. This means that the ATP consumptionrate for protein synthesis as 3 . · ATP/min (or equivalently 7 . · kT/min) for asingle spine.Similarly, we also estimate the fraction of global protein synthesis cost taken byCaMKII and PSD-95. Two subunits CaMKII α and CaMKII β have similar molecularmasses with an average length of 528 amino acids, while PSD-95 has 779 amino acids13Yoshimura et al 2004). The half-times for CAMKII decay is 3.4 days (average of 3.0 forCaMKII α and 3.8 for CaMKII α ) and for PSD-95 decay is 3.67 days (Cohen et al 2013).Given that there are 5600 copies of CaMKII and 300 copies of PSD-95 (Sheng and Kim2011), this yields a synthesis rate 0.57 copies/min of CaMKII, and 0.03 copies/min ofPSD-95. This requires 4 · · .
57 = 1200 ATP/min for CaMKII and 4 · · .
03 = 93ATP/min for PSD-95. From this, it follows that these two the most frequent proteinsconsume for their synthesis about 1/3 of the total energy used for protein synthesis inthe whole PSD.Actin treadmilling.Actin treadmilling energetics in spines can be estimated as follows. Each cycle of actintreadmilling involves 1 ATP. Actin concentration in the mammalian brain is 100 µ M(Devineni et al 1999), which yields 6000 actin molecules per spine (average spine volumeassumed: 0.1 µ m (Honkura et al 2008)). About 90 % of actin in spines degrades fastwith a characteristic lifetime ∼
40 sec, and the remaining 10 % with much longer time ∼
17 min (Honkura et al 2008; Star et al 2002), which can be neglected. This meansthat the metabolic cost of actin turnover is 8100 ATP/min (or 1 . · kT/min).AMPA and NMDA trafficking.The energy cost of AMPA and NMDA receptor trafficking within a spine is composedof receptor insertion/removal to/from the spine membrane (Huganir and Nicoll 2013)and receptor movement along the membrane (Choquet and Triller 2013). First, weconsider the receptor insertion (exocytosis) and removal (endocytosis) contributions.Both of these processes can be envisioned as molecular crossing of energy barriers,because both of them lead to deformations in the membrane structure. The interesting14oint is that the rates of endo- and exocytosis of AMPA receptors are very similar atsteady state and about 0.1 min − (Ehlers 2000, Lin et al 2000), which is much fasterthan the turnover (degradation) rates for AMPA (half-life about 2 days; Cohen et al2013). The approximate equality of the insertion and removal rates indicates that theenergy barriers for these two opposing processes are similar (invoking the Arrheniuslaw). A typical energy barrier for AMPA insertion is 4 −
30 kT, with an average of17 kT, which is the energy of protein insertion into a lipid membrane by a mechanismof membrane fusion (Grafmuller et al 2009; Gumbart et al 2011; Francois-Martin et al2017). Since an average spine contains about 100 AMPA (Matsuzaki et al 2001), weobtain the energy rate for AMPA endocytosis/exocytosis as 2 · kT ) · . − (the prefactor 2 comes from including both endo- and exocytosis transitions), whichyields 340 kT/min, or equivalently 17 ATP/min. Inclusion of NMDA receptors affectsthis figure by only 10%, since the number of NMDA receptors on spine membrane ismuch lower, about 10 (Nimchinsky et al 2004). Thus, the total cost of AMPA andNMDA insertion/internalization is 18.7 ATP/min.Energy requirement of receptor movement is estimated assuming that most of it isdone along spine membrane, which might be an underestimate given that some traffick-ing might also take place internally along actin filaments (Choquet and Triller 2013).In general, the motion of molecules along some substrate is powered by ATP hydrolysis(Bustamante et al 2005), with a generation of a propulsion force F of the order of 6 pN(Visscher et al 1999). Additionally, it was found that AMPA motion along spine mem-brane was alternating between the periods of diffusion and quietness (Borgdorff andChoquet 2002) with comparable durations, and a diffusion coefficient D = 0 . − . m /s, or its average value D = 0 . µ m /s. The power P dissipated by one re-ceptor is proportional to the product of the force F and an average velocity v , whichis v ∼ L/τ , where L is the average spine length and τ is the typical time to traveldistance L . Since for diffusive processes τ ∼ L /D (Phillips et al 2012), we get thatvelocity v ∼ D/L . Thus, we can write that the power dissipated by a single receptortrafficking along spine membrane is P ≈ . F D/L , where the prefactor 0.5 comes fromthe assumption that the time intervals of diffusion and quietness are roughly similar(Borgdorff and Choquet 2002), which reduces the effective velocity by half. For a typ-ical spine length of L = 1 µm , we obtain P ≈ . · − J/s, which using the factthat kT is 4 . · − J at 36 o C , yields P ≈ . . . · kT/min). This figure is 437 times larger than the one for receptorinsertion and internalization, and hence it is much more important. Plasticity modulation cost.
There are many routes for modulation of synaptic strength. One of the betterstudied pathways is through purinergic P2X receptors (Pankratov et al 2009; Khakhand North 2012). Activated by ATP molecules P2X receptors, which are voltage gatednonselective channels, enable Ca ions to flow inside the spine. This calcium influxmodulates various internal molecular pathways, with the end result of affecting thenumber of AMPA receptors on the spine membrane, likely by modulating their endo-and exocytosis rates (Gordon et al 2005; Pougnet et al 2014). Both LTP and LTDhad been observed, depending on which pathway is influenced, but the relative changes16n the synaptic strength did not exceed 30 −
40% (Gordon et al 2005; Pougnet et al2014). This means that the rates of AMPA insertion and removal should be also affectedby this percentage, which suggests that the overall energy rate for AMPA traffickingcan increase by about 35% due to the actions of P2X receptors. This translates toan additional 2871 ATP/min. Because of other possible pathways modulating synapticplasticity, e.g., by adenosinergic receptors, this figure should be considered as a minimalamount of ATP rate due to plasticity modulation.
Summary of the molecular costs.
From these consideration it follows that the intra-synaptic processes are the mostenergy demanding for the plasticity. Within them, the cost of protein phosphorylationdominates over the rest by a factor 10 −
30. The cost of the remaining intra-synapticprocesses (Ca removal, actin treadmilling, receptor trafficking) is very similar, withthe cost of protein synthesis about two times lower (Table 1). Together, all the synapticplasticity processes considered here use (3 . − . · ATP/min and account for onlyabout 4 −
11% of the metabolic cost related to the fast excitatory synaptic transmissionfor rat brain (Table 2). Overall, the small values of these percentages indicate thatsynaptic plasticity is metabolically inexpensive.17 odeling the metabolic cost of learning and memory.
The next major goal is to study theoretically the energy rate associated with aparticular form of learning and memory, and relate it to the lifetime of a new memory.
Motivation for the approach.
All the processes considered above and associated with synaptic plasticity consti-tute a physical backbone of synaptic learning and memory. We want to quantify themetabolic cost of a new learning and its memory trace, using a modeling approach. Todo this, one should in principle include all the above processes in a model. However,combining all of them in a single model is extremely difficult and perhaps even infea-sible, because it is not clear how all these plasticity related mechanisms are mutuallycoupled and with what strength. Instead, we focus in the modeling part on the initialphase of synaptic plasticity, called induction of plasticity (or early LTP), during whichsynapses start to grow and increase their strength, which is associated with learningof a new input and its subsequent decay. The induction phase is accompanied by en-hanced phosphorylation of PSD proteins, including AMPA receptors (while traffickinginto the spine membrane), which is evident in Table 1. This indicates that proteinphosphorylation is the most energetically demanding and dominating process duringthe plasticity initiation. For this reason, and because protein phosphorylation is themost commonly studied cellular microcircuit (Krebs 1981; Hill 1989; Qian 2007), it isthe main component of the modeling part. We neglect in the model the late phase ofplasticity (plasticity maintenance or late LTP) during which most of the protein syn-thesis takes place, because it uses much less energy (Table 1). We also neglect in the18odel the other processes from Table 1, which means that the actual metabolic cost oflearning and memory could be slightly greater than theoretically calculated below.The model used for quantifying the energetics of a memory trace and its durationis based on the so-called cascade models of synaptic plasticity (Fusi et al 2005; Leiboldand Kempter 2008; Barrett et al 2009; Benna and Fusi 2016). In short, these modelsassume that synaptic molecular machinery can be described in terms of discrete proba-bilistic states, with transitions between the states depicting molecular transformations.However, the classic cascade models can not describe properly protein phosphoryla-tion due to their simplistic topology, i.e., too simple pattern of molecular transitions.Below, the cascade model of plasticity is extended to include protein interactions viaphosphorylation-dephosphorylation mechanism, by adding transition motifs with closedloops (Qian 2007).The expenditure of energy in the model with phosphorylation-dephosphorylation cy-cles is quantified using the concept of entropy production rate EPR (Hill 1989; Nicolisand Prigogine 1977). In our case, EPR measures the rate of dissipated energy (metabolicrate) in the spine due to transitions between different molecular states (Rolfe and Brown1997; Qian 2006). When a synapse is in thermodynamic equilibrium with its environ-ment, no energy influx enters the synapse, all internal molecular reactions are balanced(forward and backward reaction rates are equal), and the synapse does not dissipateany energy, implying a vanishing metabolic rate and EPR = 0. Such a condition corre-sponds to a “thermal death” when no biological function can be performed, and thusit cannot represent a baseline or resting synaptic state, during which the synapse keepstrack of its prior plasticity events, i.e. keeps their memory. The functional baseline19ynaptic state is certainly a driven state that requires energy (and material) exchangewith the surrounding, and this implies that the synapse must be an open system in ther-modynamic nonequilibrium even during its steady state baseline activity. This steadystate or baseline must obviously dissipate energy, as the incoming energy flux breaksthe balance in the forward and backward rates of the internal molecular reactions (inour case phosphorylation rates), which leads to nonzero EPR and dissipation, as well asto finite synaptic metabolic rate. We will call this baseline synaptic state, the nonequi-librium steady state (NESS), in analogy to the systems considered in nonequilibriumthermodynamics (Lebowitz and Spohn 1999; Mehta and Schwab 2012). Our theoreti-cal NESS state corresponds to the empirical active steady state phosphorylation phaseanalyzed above as the intra-synaptic cost of plasticity.When a synapse is stimulated (by Ca influx) and the plasticity event initiated,the molecular reactions/transitions are amplified, which leads to perturbations in thedistribution of synaptic states and the emergence of the time dependent memory traceassociated with this perturbation. The synaptic perturbation also causes an increase inthe rate of energy dissipation (EPR), which declines after some time to its baseline levelwhen the stimulation is turned off. The key relationship that we want to explore, is theone between the total energy expanded on memory trace and the memory duration. Thermodynamically realistic model of cascade synaptic plasticity mustcontain cyclic reactions.
We consider synaptic plasticity as transitions between multiple discrete synapticstates (Montgomery and Madison 2004) (Fig. 1). These states represent internal degreesof freedom of the molecular processes in a dendritic spine. It is assumed that many20tates correspond to one synaptic weight, either weak or strong, denoted as up and down(with the symbols + and − in Fig. 1C), and this reflects neurophysiological data on asingle synapse level (Petersen et al 1998; O’Connor et al 2005). The vertical transitionsbetween the states in Fig. 1C correspond to conventional plasticity, while the horizontaltransitions are related to the so-called metaplasticity (Abraham and Bear 1996). Thelatter transitions do not lead to changes in the synaptic weight. It turns out that notevery configuration of these transitions produces nonequilibrium steady state (NESS),required for nonzero metabolic rate. For instance, a “ladder” structure of synapticmolecular reactions (Fig. 1A) yields a zero EPR (and metabolic rate) at steady state(see Eq. 15 and below in the Methods), which is not realistic. The basic requirement forNESS and nonzero EPR is the presence of cyclic and bidirectional molecular reactions,which generate nonzero probability flux that mimics the exchange of energy with anenvironment (Fig. 1B; Qian 2006).This requirement can be verified for a simple and minimally realistic model ofonly 3 states linked by a closed loop of reactions, which represents a phosphorylation-dephosphorylation cycle for one protein interaction (Fig. 1B). Such a loop is the basicmetabolic motif for modeling synaptic plasticity with multiple states, i.e., with manyphosphorylation events (Fig. 1C). In this simple 3 state case, p − denotes the probabilityof a protein in a ground state and p + is the probability of the activated protein (Fig.1B). The protein can go from the state p − to p + either directly, but it is rare and occurswith a small transition rate ǫ α (where ǫ ≪ p + indirectly, intwo steps, through an intermediate state q (by binding an enzymatic substrate). Theinteresting point is that this second step, from q to p + can be very fast, with a large21ransition rate a , depending on the level of protein phosphorylation that is poweredby ATP hydrolysis. This simple 3 state protein is not in thermal equilibrium with itsenvironment, because it continually transfer between the 3 states ( p − , q , p + ), drivenby energy provided by ATP. Therefore, this configuration dissipates energy even in thebaseline, which can be found explicitly as entropy production rate EPR at NESS (Fig.1B) EPR = kT αβZ ( ǫ a − ǫ ǫ b ) ln (cid:18) ǫ aǫ ǫ b (cid:19) , (1)where Z is a function of different transition rates a, b, α, β , with ǫ i unitless small co-efficients ( ǫ i ≪
1) (see Eq. 11 in Methods). The value of a is controlled directly byprotein phosphorylation rate driven by ATP hydrolysis, and b is the dephosphorylationrate. The term ǫ a − ǫ ǫ b represents the bias or deviation from thermal equilibrium,and corresponds to the probability flux (Eq. 9) that circulates between the three states.Eq. (1) implies that if the transitions do not form a loop, i.e. when a = b = 0, thenEPR = 0 (since x ln( x ) x one needsbidirectional transitions. For unidirectional transitions, we obtain EPR
7→ ∞ , as canbe seen e.g. by setting a > b = 0. This obviously is unrealistic.The steady state EPR for the 3 state system (Fig. 1B) is also 0, when the conditionof the so-called detailed balance is met, i.e., when the bias ǫ a − ǫ ǫ b = 0, which corre-sponds to thermodynamic equilibrium. The detailed balance can be broken (leading tononequilibrium) if, e.g., ǫ is much smaller than the rest of the parameters in Eq. (1).22his situation happens in a living cell, where there is a big asymmetry between phos-phorylation and dephosphorylation, i.e. a ≫ ǫ b (Qian 2006). Generally, the higherthat asymmetry the larger EPR .In the remaining of this study we investigate the metabolic constraints on learningan input and on its subsequent memory trace, in the model with multiple states (Fig.1C). Metabolic cost of baseline synaptic plasticity is insensitive on the number ofsynaptic states but it is affected by the transition rates between the states.
When synapses are not stimulated, their dynamics converge into baseline activity,which is the nonequilibrium steady state NESS with distributed occupancies of differentstates and multiple transition loops (Fig. 1C; Methods). All the transitions betweenthese states are appropriately rescaled by a prefactor e − zk to progressively slow down thedownstream dynamics of the loops with the higher index k , where z is the slowing downfactor. This prefactor is introduced to provide multiple time scales in the dynamics ofintrasynaptic molecular interactions. In the NESS state in Fig. 1C, energy is constantlydissipated due to many phosphorylation-dephosphorylation loops, and this leads tononzero basal synaptic metabolic rate. Entropy production rate EPR associated withthis state can be viewed as a metabolic cost of maintaining all the prior plasticityevents, i.e., the memory of the past, and it is always greater than zero, and could besubstantial even for one molecular pathway ∼ . is essentially independent of the number of synaptic states n , but itdepends on the magnitude of the transition rates between synaptic states (Fig. 2).23pecifically, increasing the rate of synaptic slowing down z makes EPR monotonicallysmaller with a saturation for large z . On the other hand, EPR as a function of ATPdriven transitions with an amplitude a exhibits more complicated shapes: either itincreases up to a saturation with increasing a for z = 0, or it has maxima for z > z ) and, simultaneously, by keeping the amplitude of ATPdriven transitions ( a ) either very small or very large. Stronger and longer synaptic stimulations lead to longer memory traces withhigher energy expenditures, but with low relative costs.
Next, we study the energetics and the memory trace related to a single transientplasticity event. The plasticity event corresponds to synaptic stimulation by affecting allATP-driven phosphorylation rates a ± k associated with the transitions q ± k p ± k +1 , whichis a microscopic representation of some macroscopic learning (Fig. 1C). The stimulationis induced by a brief jump in the ATP-driven transition rates a ± k , by a relative fraction A ± ( t ) = A ± exp( − t/τ ), which relaxes exponentially to zero with a time constant τ thatcan be also thought as a learning time. This perturbation leads to a redistributionof the occupancy probabilities of the synaptic states, and the recovery to the baselineprobabilities takes some time, called memory lifetime T m , which in general is muchlonger than the stimulation time τ .The memory trace associated with the single stimulation is defined as the deviationof the average synaptic weight from its baseline value, relative to a noise in the weights.24his essentially corresponds to a signal-to-noise ratio SNR, which is given by Eqs. (12-14) in the Methods. Temporal dependences of memory trace SNR and synaptic energyrate EPR following a single stimulation are presented in Fig. 3. Upon stimulation fromthe NESS state, SNR initially builds up to some maximal value and then it slowly decayswith a characteristic long tail, given by SNR ∼ t − δ , where δ ≈ / T m is defined as the time interval from the stimulation up to themoment when SNR drops below the threshold. Energy rate EPR associated with thisstimulation has a qualitatively different time course than SNR, as it follows closely thetemporal dependence of short-term stimulation A ± ( t ), without exhibiting a long tail(Fig. 3). This means that EPR starts from a high level and quickly decays to itsresting value EPR when the stimulation ends (Fig. 3). Thus, most of the time whenmemory trace is still detectable, the energy rate is essentially at its resting value, whichimplies that the two variables decouple for longer times (Fig. 3). A direct consequenceof this important observation is that the total energy E expanded on a new memorytrace, defined as an area under EPR, differs in most cases only by a small percentagefrom a baseline energy E ( E = EPR T m ) required for supporting the baseline synapticstate related to all prior memories during time T m (Fig. 4; see also below). This effectis much more pronounced for larger slowing down z , when the speed of downstreammolecular reactions is severely reduced (Fig. 4). The relatively low energy cost is asign of metabolic efficiency of a new memory storing in the cascade model of synapticplasticity. 25 hen longer memories require proportionally larger energy expenditure? How the lifetime T m of the memory trace and its energy cost E depend on theamplitude A + and duration τ of synaptic stimulation? Memory lifetime T m generallyincreases with increasing A + and τ , but both dependences are roughly logarithmic (Fig.5). The corresponding energy expenditure E on keeping the new memory trace SNRabove the threshold also grows with A + and τ in a similar manner as T m (Fig. 5).Interestingly, the energy cost E of a new memory increases in proportion to its lifetime T m , as is evident by an approximate constancy of the ratio T m /E over variability in thestimulus amplitude and its duration (Fig. 5). Moreover, the ratio T m /E is larger forlarger z , which implies that the gain in memory duration per invested energy is biggerif the rates of downstream molecular processes are reduced.How general is the finding that energy cost of a memory trace increases proportion-ally with the memory lifetime? What happens if we keep the level of synaptic stim-ulation constant, and instead, change the intrinsic parameters characterizing synapticplasticity? In Figs. (6-7) we show the dependence of T m and an associated energy cost E on two internal synaptic parameters: basic number of synaptic states n , and thefraction of potentiated synapses f .Increasing synaptic states n leads to an increase in T m , but only for small n (Fig.6). For larger n , the memory lifetime T m saturates. The related energy cost E behavessimilarly, such that E is proportional to T m , which again follows from the observationthat the ratio T m /E does not change much over the whole range of n . A qualitativelysimilar pattern, with approximate constancy of T m /E , is observed when the fraction ofpotentiated synapses f is changed (Fig. 7).26aken together, in all these three cases a longer memory trace requires a propor-tionally more energy to sustain. When longer memories do not require more energy?
A different picture emerges, regarding the relationship between E and T m , whentwo other internal parameters are changed that are related to the speed of moleculartransitions. One is the molecular slowing-down factor z , and another is the globalamplitude of the phosphorylation rate a .Growing the parameter z leads to bimodal shapes of memory trace duration T m and its energy cost E with the appearance of maxima (Fig. 8). In this case, however,the ratio T m /E increases significantly with growing z , which indicates a substantialgain in memory duration per expanded energy (Fig. 8). This means that memorylifetime grows faster than its energy cost, i.e., longer memories are relatively cheaper.Interestingly, the ratio T m /E is insensitive to the fraction of potentiated synapses f , asall dependencies collapse on a single line (Fig. 8).A slightly more complex scenario appears when the ATP-driven phosphorylationamplitude a is varied (Fig. 9). For z = 0, the memory lifetime T m and energy E bothincrease monotonically with a such that the ratio T m /E initially decreases and thenstabilizes at some level. For a more interesting case z >
0, the memory duration T m and its cost E are not easily correlated: T m initially grows and then saturates for larger a , while the associated energy cost E always exhibits a maximum at some a . Moreimportantly, for z >
0, the ratio T m /E displays a minimum, which is very close to thepoint where energy E has a maximum (Fig. 9). This means that there are two different27egimes: for small a a relative cost of increasing memory lifetime strongly grows (sharpdecrease in the ratio T m /E ), whereas for large a the opposite happens and the relativecost of memory duration decreases ( T m /E increases). Thus, the latter regime is muchmore energy efficient, which is also visible in the high values of both T m and T m /E forlarge a (Fig. 9).Taken together, these two results suggest that storing longer memories is not alwaysassociated with a higher metabolic burden. In fact, there can be regimes in the internalsynaptic parameters, here z and a , for which longer memories can be relatively cheap. Metabolic cost of a new memory relative to the cost of prior memories.
How expensive is to invoke a new plasticity event and keep its memory in themolecular interactions, relative to the cost of prior plasticity events “encoded” in thebaseline spine metabolic rate? Figure 4 as well as lower panels of Figs. (6-9) provide ananswer to this question. The relative cost of a new memory trace with respect to thebaseline energy cost E during T m , i.e. the ratio ( E − E ) /E , is almost always smalleror much smaller than 1. Thus, the cost of keeping the new memory trace detectable isgenerally marginal to the cost of storing memories of all previous events.28 ISCUSSIONEmpirical metabolic efficiency of long-term synaptic plasticity.
Dendritic spines of excitatory synapses occupy only about 10% of neocortical volumeof adult mammalian brain (Karbowski 2015), but their short-term signaling associatedwith fast synaptic transmission can cause a high metabolic burden to the whole cortex.Its estimated cost in the rat cortex is 8 . · ATP/min per spine (Attwell and Laughlin2001), and the interesting question is how does it relate to the synaptic plasticity cost?Proteins underlying molecular level of synaptic learning and memory constantlyinteract, which is associated with synaptic plasticity, and these processes require energyinflux and thus some metabolic cost (Table 1). What is this overall cost, and to whatextent it can constrain the memory trace? The empirical estimates conducted heresuggest that the energy related to synaptic plasticity in rat cortex comprises only 4 − −
80% of the total cortical metabolic rate (Attwell and Laughlin 2001; Harris et al2012; Karbowski 2009 and 2012), we get that metabolic cost of learning and memoryis only about 1 −
9% of the total metabolic rate. This is the empirical evidence thatthe processes of learning and information storing in synapses are energetically rathercheap.What about the energetic constraints on memory trace? The cascade model consid-ered here suggests that, although a longer memory trace requires proportionally largeramounts of energy (in most cases; Figs. 5-7), these amounts are relatively small ( < ∼ + ions) operating on a time scale of a few seconds (Attwell andLaughlin 2001; Karbowski 2009). On the other hand, faster molecular processes as-sociated with spine plasticity and related to activated protein phosphorylation, actintreadmilling and receptor trafficking, operating on a time scale from 0 . − + ionsthat have to be extruded following synaptic transmission. This high energy efficiencyof memory on a molecular level is reminiscent of energy efficient sparse neural codes ona cellular level (Levy and Baxter 1996; Laughlin et al 1998). Limitations of the empirical estimates.
Every estimation based on incomplete data is only approximate, and this is alsothe case for the above results based on molecular data. It seems that the greatestuncertainties are associated with the protein phosphorylation and plasticity modulation.The rates of protein phosphorylation can vary by a factor of 10 for the active LTPphase. The precise values for each PSD protein are not known; at best we have somedata for CaMKII, which fortunately is the most abundant protein in PSD (Sheng andKim 2011). Additionally, the metabolic cost of phosphorylation depends on the fractionof proteins with elevated phosphorylation rates, which in turn depends on the presynap-tic stimulation by Ca influx. Based on limited data, we assumed that this fraction isabout 10%. However, if it were 30% (under some conditions), then this would increasethe overall cost of synaptic plasticity by a factor 3 to about 12 −
33% of the synaptic31ransmission cost. The maximal possible value for the cost of synaptic plasticity wasestimated above as 15 · ATP/min, which means that in a hypothetical situationwhen all PSD proteins are maximally phosphorylated then the metabolic rate of synap-tic plasticity is about twice of that for synaptic transmission. This value provides anupper bound on the theoretically possible cost of synaptic plasticity.The cost of plasticity modulation is hard to compute exactly, as we are uncertainwhich pathways are affected and to what extent. We took only one of the possible waysof modulation (through P2X receptors), and even for this case we do not have numericalvalues of the transition rates modulation. The modulation cost was deduced based onthe “end product”, i.e., the extent to which the synaptic weight can change.
Cascade model with phosphorylation cycles reveals metabolicefficiency of synaptic memory.
The main theoretical findings of this study are as follows. (i) There is a nonzerocost of storing all previous plasticity events, i.e. prior memory, which is kept in baselineentropy production rate EPR (Figs. 2 and 3). (ii) In most cases, the metabolic cost ofa new learning and its memory trace increases proportionally with memory duration,such that their ratio is essentially constant (Figs. 5-7). (iii) A different trend is observedfor the two key parameters, slowing down factor z and ATP driven transition amplitude a , for which a memory lifetime per expanded energy grows for longer memories (Figs.8 and 9). (iv) Memory trace decays with time as a power law, while the associated withit metabolic rate decays to its baseline much faster, exponentially, which leads to thedynamic decoupling of memory trace and its metabolism (Fig. 3). The likely reason for32he fast decay of the metabolic rate is that EPR depends nonlinearly on the probabilitiesand transition rates. (v) The direct consequence of (iv) is that maintenance of a newmemory takes only a small fraction of the energy required for the baseline synapticstate, i.e. the ratio ∆ E/E ≪ z is large, which indicates that the diversity oftime scales associated with biochemical synaptic cascades increases not only the mem-ory lifetime (Fusi et al 2005; Benna and Fusi 2016), but also enhances its metabolicefficiency. Even bigger metabolic efficiency is obtained if the ATP driven phosphory-lation amplitude a is made larger (Fig. 9). Surprisingly, in this case it is possible tohave longer memories for less energy if a and z are sufficiently large.How can we explain the effects of a and z on the energetic efficiency of memory?They can be explained by noting that increasing z causes decreasing the speed of allmolecular processes that in turn consume less energy per time unit (Fig. 2, middlepanel). Additionally, the resting metabolic rate EPR depends non-monotonically on a for z >
0, and generally EPR decreases with increasing a if a is large (Fig. 2,bottom panel). In this case the ratio T m /E has a minimum for some a if z > T m /E generally steady increases for large a , suggesting an enhancedmetabolic efficiency of synaptic memory in this regime.It should be noted that the energy cost of a new learning and its memory tracefound here is a theoretical minimum, because the cascade model considers only proteinphosphorylation. Inclusion of the other energy consuming molecular processes wouldbe much more complicated and would require a much more complex model that goesfar beyond simple Markov models analyzed within a Master equation approach. Model of synaptic plasticity as phosphorylation cascades.
The cascade model of synaptic plasticity presented in this study differs from theprevious cascade models (Fusi et al 2005; Leibold and Kempter 2008; Barrett et al 2009;Benna and Fusi 2016). First, the transitions between synaptic states are bidirectionalhere, as opposed to the majority of the previous models (see however, Benna and Fusi2016, as an exception). Second, the present model contains many local cyclic motifs,corresponding to ATP driven protein phosphorylation, which are absent in the previousmodels. Strictly speaking, there are some loops in the topology of the previous modelsbut they are unidirectional and non local, and hence it is difficult to interpret theirphysical meaning. Third, and most importantly, the current study ask a fundamentallydifferent question, namely the energy cost and efficiency of memory on a spine level.It should be stressed that, without bidirectional cyclic motifs, most of the previouscascade models are inappropriate for studying metabolic constraints on memory, sincethey are thermodynamically inconsistent and produce singular entropy production rate(metabolic rate), as explained in this study (Qian 2006, 2007).34hese differences generate also qualitative differences in some results. For instance,the signal to noise memory trace SNR decays here for long times as ∼ t − / (Fig.3), which is much faster than in Benna and Fusi (2016), where it decays as ∼ t − / .The primary reason for this is that the Benna and Fusi (2016) model is optimized,while the current model is not. Another difference is that here the memory lifetimealways saturates (Figs. 5-9), also as a function of the number of basic states n (Fig.6). In contrast, in the Benna and Fusi (2016) model the memory duration increasesexponentially with synaptic complexity, which may be equivalent to n . The likelyexplanation for this difference is the presence of many local cycles in the structure ofthe current model, which provide additional pathways for faster relaxation to baselineconditions. Limitations of the present model.
The model presented in this study is an obvious simplification of a much morecomplicated web of molecular interactions in a typical spine. However, the goal herewas not to model the spine in its full complexity, but rather to identify key parametersrelated to energy consumption during memory storage that are most sensitive in termsof metabolic efficiency, and this could be best done for simple models. Nevertheless,the current model can be modified and extended by including specific molecular details,such as those considered in several previous studies (Miller et al 2005; Hayer and Bhalla2005; Graupner and Brunel 2007; Bhalla 2011; Antunes and Schutter 2012; Smolen etal 2012; Kim et al 2013). 35 mplications of energy efficient memory storage.
The empirical estimates as well as the theoretical analysis performed here imply thatmolecular storing of memory can be relatively cheap under some conditions ( ≤
11% ofthe synaptic transmission cost). This result can have important implications for build-ing energy efficient artificial silicon systems that mimic brain function by storing andprocessing information (Esser et al 2016; Sun et al 2018). The key here is to have bidi-rectional loops in the topology of subsynaptic biochemical pathways with appropriatelytuned transition rates and multiple time constants, not only to prevent catastrophicforgetting in neural networks (Kirkpatrick et al 2017) but also to spend small amountsof energy on long-term synaptic computations.Another potential implication of the current results is for biomedical research relatedto neurodevelopmental disorders. There are experimental data showing a close relation-ship between phosphorylation signaling in PSD and diseases such as schizophrenia andautism (Li et al 2016). Interestingly, many genes encoding PSD overlap with mutatedgenes responsible for schizophrenia and for autistic phenotype (De Rubeis et al 2014;Iossifov et al 2014; Fromer et al 2014; Kaizuka and Takumi 2018). In the light of theresults obtained in this study, it is not difficult to understand this close relationshipgiven the easiness with which one can alter memory lifetime and its energetic efficiencyby manipulating transition rates related to ATP driven protein phosphorylation. More-over, there are many empirical studies showing that increased glucose metabolism canenhance memory in mammals, which is known as “glucose memory facilitation effect”,in a dose-dependent fashion (Gold 2005; Smith et al 2011). The latter means that someglucose levels can facilitate memory, while others can be neutral or even detrimental36or memory. Using our model, this phenomenon can be understood by noting thatATP that drives protein phosphorylation is generated directly by glucose (Rolfe andBrown 1997). This means that glucose metabolic rates can affect ATP rates used forpowering spine PSD proteins and downstream molecular processes related to synapticinformation storing, e.g. as shown in Fig. (9).37
ETHODSCascade model of synaptic plasticity with bidirectional cycles.
We model biochemical processes underlying synaptic plasticity in a dendritic spineas transitions between discrete synaptic states (Montgomery and Madison 2004), whichdescribe various levels of protein activation in spine PSD (Fig. 1). The model ofplasticity considered here is a generalization of previous models of cascade synapticplasticity (Fusi et al 2005) and is treated as a Master Equation system (Schnakenberg1976). The main modification in the current approach is the addition of closed loops orcycles with bidirectional transitions between different states, corresponding to proteinphosphorylation (Qian 2006).A synapse can be in one of many “down” and “up” states corresponding to biochem-ical process associated with LTD and LTP, respectively (Fig. 1). There are n basic upstates and n basic down states, whose probabilities of occupancy are given by p + k and p − k , respectively for k = 1 , ..., n . There are two types of transitions between neighboring p ± k states: direct spontaneous transitions driven by thermal fluctuations (with transitionrates α ± k and ǫ α ± k ), and indirect transitions that contain ATP driven reactions (withtransition rates a ± k and ǫ b ) and non-ATP reactions (with transition rates β ± k and ǫ β ± k ).The direct transitions are rare, while the indirect ones can be massive if the synapseis electrically stimulated in a way that induces plasticity mechanisms (LTP and LTD),and elevation of local ATP concentration. Additionally, the ATP related transitions re-quire intermediate states with probabilities of occupancy q ± k , which describe metastablestates with proteins ready for phosphorylation (for details see, e.g., Qian 2007). Each38oop or cyclic motif represented by transitions p ± k q ± k p ± k +1 p ± k , can be thoughtas a cascading phosphorylation of various downstream proteins in spine PSD.It is also assumed that the dynamics of downstream cascades are progressivelyslower, similar to the previous models (Fusi et al 2005; Benna and Fusi 2016). Math-ematically, this is implemented by a prefactor g k = exp( − zk ), which rescales all thetransition rates and gets smaller for deeper states with higher index k , where z is theslowing down factor.The dynamics of state probabilities are given by˙ p ± k = g k − ( ǫ α ± k − p ± k − + a ± k − q ± k − ) + g k ( α ± k p ± k +1 + β ± k q ± k ) (2) − h g k − ( ǫ b + α ± k − ) + g k ( ǫ α ± k + ǫ β ± k ) i p ± k for 2 ≤ k ≤ n −
1, where the dot denotes the time derivative. For k = n we have˙ p ± n = g n − (cid:16) a ± n − q ± n − + ǫ α ± n − p ± n − − ( ǫ b + α ± n − ) p ± n (cid:17) , (3)and for k = 1 we have˙ p +1 = ǫ α p − + a q + g ( α +1 p +2 + β +1 q +1 ) (4) − h g ( ǫ α +1 + ǫ β +1 ) + α + ǫ b i p +1 p − = α p +1 + β q + g ( α − p − + β − q − ) (5) − h g ( ǫ α − + ǫ β − ) + ǫ α + ǫ β i p − . The dynamics of the intermediate states involved in protein phosphorylation aregiven by ˙ q ± k = g k (cid:16) ǫ β ± k p ± k + ǫ b p ± k +1 − ( a ± k + β ± k ) q ± k (cid:17) , (6)for 1 ≤ k ≤ n , and ˙ q = ǫ β p − + ǫ b p +1 − ( a + β ) q . (7)The transition rates α ± k , β ± k , a ± k are heterogeneous across different cycles and given by α ± k = α (1 + ση ± k ), β ± k = β (1 + ση ± k ), and a ± k = a (1 + ση ± k ), where η ± k are randomvariables uniformly distributed between − σ is a measure of heterogeneity(0 ≤ σ < α , β , and a are the amplitudes of the above transitionrates. When the synapses are stimulated, only the ATP-driven rates a ± k , a are timedependent (see below).Let us consider two examples of 3 state models: one with a ladder structure (Fig.40A), and another with a loop structure (Fig. 1B). For the ladder structure (Fig. 1A)we have the dynamics of state probabilities p , p , p as˙ p = J , ˙ p = J − J , ˙ p = J , (8)where the probability fluxes are defined as: J = w p − w p (flux from state 2 tostate 1), J = w p − w p (flux from state 2 to state 3), J = − J (flux from state1 to state 2). At the steady-state ( ˙ p = ˙ p = ˙ p = 0), we obtain w p = w p , and w p = w p . From this it follows that all steady-state fluxes J = J = 0, which isknown as the condition of the detailed balance (Lebowitz and Spohn 1999; Mehta andSchwab 2012).For the second example with a loop (Fig. 1B), the dynamics of state probabilities p + , p − , q read: ˙ p + = J +0 + J + − , ˙ p − = J − − J + − , ˙ q = − J +0 − J − , (9)where the probability fluxes are defined as: J +0 = aq − ǫ bp + (flux from state 0 to +),41 + − = ǫ αp − − αp + (flux from state − to +), J − = βq − ǫ βp − (flux from state 0 to − ), and the opposite fluxes are J = − J +0 , J − + = − J + − , and J − = − J − . For thiscyclic motif, we can find steady-state values of the probabilities as: p + = Z − [ ǫ α ( a + β ) + ǫ aβ ] ,p − = Z − [ α ( a + β ) + ǫ bβ ] ,q = Z − [ ǫ αβ + ǫ ( ǫ α + ǫ β )] , (10)where Z is given by Z = α [(1 + ǫ + ǫ ) β + (1 + ǫ ) a + ǫ ǫ b ] + β [ ǫ a + ǫ (1 + ǫ ) b ] . (11)At the steady-state the above fluxes must balance each other, i.e., J +0 = − J + − = − J − ≡ J , where the emerging flux J = αβZ − ( ǫ ǫ b − ǫ a ). Note that the flux J is generally nonzero, which is a signature of a nonequilibrium steady-state, denotedas NESS (Lebowitz and Spohn 1999; Bustamante et al 2005; Van den Broeck andEsposito 2015). It vanishes only if the phosphorylation and dephosphorylation rates( a and ǫ b ) are both zero (cyclic motif is destroyed), or for the special case of theso-called detailed balance when ǫ ǫ b = ǫ a . The latter two situations correspond tothermodynamic equilibrium when neither energy nor material is exchanged with theenvironment (“thermodynamic death”, e.g., Nicolis and Prigogine 1977).42 emory trace and signal to noise ratio. We consider N s independent synapses for which we first determine their non-equilibriumsteady-state NESS. This is done by starting from uniform initial conditions for thestate probabilities p ± k , q ± k and allowing them to relax to the baseline state denoted as p ± k, ∞ , q ± k, ∞ . A next phase is a brief stimulation of synapses from their baseline and ob-servation of the associated memory lifetime of this event. During the stimulation andsubsequent memory decay, the synapses are divided into two populations: the fraction f of synapses undergoes LTP process, and the remaining 1 − f synapses perform LTDprocess. The LTP synapses are stimulated by a pulse in the ATP-driven transitions a , a + k , while the LTD synapses are activated by a pulse in the ATP-driven transitions a − k , with the explicit time dependences given by a ± k ( t ) = a ± k [1 + A ± exp( − t/τ )] for k >
1, and a ( t ) = a [1 + A + exp( − t/τ )], where τ is the characteristic time of stimula-tion, and t is the time counted from the onset of stimulation. We assume that A + > A − ,which reflects an experimental fact that LTP (LTD) is induced by high (low) frequencystimulation.We assume that all down states (including q ) have the same synaptic efficacy(weight) w , and all up states have the same efficacy 2 w , where w is the synaptic con-ductance. (Its value is irrelevant for the results of this study.) This binary choiceis consistent with neurophysiological data (Petersen et al 1998; O’Connor et al 2005).Thus, the probability that a randomly chosen synapse undergoes LTD and has weight w is (1 − f )( p − LT D + q − LT D ), and the probability that it has weight 2 w is (1 − f )( p + LT D + q + LT D ),where p ± LT D = P k p ± k,LT D , and q ± LT D = P k q ± k,LT D . Similarly, the probability that a ran-domly selected synapse undergoes LTP and has weight w is f ( p − LT P + q − LT P ), and that it43as weight 2 w is f ( p + LT P + q + LT P ), where p ± LT P = P k p ± k,LT P , and q ± LT P = P k q ± k,LT P + q .From this it follows that the average synaptic weight h V i for the whole synaptic popu-lation is given by h V i = w h (1 − f )( p − LT D + q − LT D ) + f ( p − LT P + q − LT P ) i +2 w h (1 − f )( p + LT D + q + LT D ) + f ( p + LT P + q + LT P ) i . (12)Consequently, the variance in a population synaptic weight, i.e. h V i − h V i , is h V i − h V i = w h (1 − f )( p − LT D + q − LT D ) + f ( p − LT P + q − LT P ) i × h (1 − f )( p + LT D + q + LT D ) + f ( p + LT P + q + LT P ) i . (13)We define a synaptic memory trace as a deviation of the average synaptic weight h V i from its baseline value h V i b , and normalized by a standard deviation in V (similarto Fusi et al 2005). This is equivalent to the definition of signal to noise ratio SNR attime t : SNR( t ) = q N s ( h V i − h V i b ) q h V i − h V i , (14)where the prefactor √ N s comes from summing contributions from all the synapses in a44mall cortical region. Note that after a synaptic stimulation both, the signal h V i and thevariance in the denominator, are time dependent. Note also that SNR does not dependon the value of synaptic weight w (it cancels out). It is assumed that when SNR dropsbelow a value of 1, the memory trace becomes undetectable and this time determinesthe memory lifetime T m . The general results and conclusions are independent of theprecise choice of this threshold. Entropy production as a synaptic metabolic rate.
Metabolic rate (or the rate of dissipated energy) associated with cascading biochem-ical processes in a synapse is associated with entropy production rate EPR, which isdefined as (Schnakenberg 1976; Lebowitz and Spohn 1999; Van den Broeck and Esposito2015) EPR = 12 kT X i,j ( w ij P j − w ji P i ) ln w ij P j w ji P i (15)where k is the Boltzmann constant and T the is brain temperature, w ij are the tran-sition rates between states j and i , and P i is the probability of the state i occupancy.The above EPR should be understood as an energy rate per a single biochemical cas-cade; in the case of many cascades, the result should be multiplied by the number ofpathways. Note that when the transition between two given states is unidirectional,then one of transition rates in a pair (either w ij or w ji ) must vanish. This means thata corresponding logarithm in the sum must diverge to infinity, which implies a diver-gent metabolic rate. This is clearly not a realistic description of the energetics of any45iological system, which suggests that one must always keep all the transition rates asbidirectional (regardless of how small they are). Unfortunately, this important fact wasoverlooked in early models of cascade plasticity, where many transitions were chosenas unidirectional (Fusi et al 2005; Leibold and Kempter 2008; Barrett et al 2009), andonly recently it was realized that bidirectionality is important for a memory lifetimeduration (Benna and Fusi 2016).In a particular case of a simple 3 state model with a ladder structure (Fig. 1A),the entropy production rate at steady-state is zero. This is due to the detailed balancecondition w p = w p and w p = w p , which implies that all the terms in Eq.(15) are zero. This situation corresponds to a vanishing flux, which means that our 3state system at steady state is in thermal equilibrium with its environment.On the contrary, for the 3 state model with a loop (Fig. 1B), the detailed balanceis broken, and there is a circulating flux even at steady-state, which leads to a non zeroEPR given by Eq. (1).For our plasticity model with 2 n − − pp ) andwithin up states (EPR + pp ), two contributions caused by indirect transitions involving p and q states either for down (EPR − pq ) or for up states (EPR + pq ), and one contributionrelated to transitions within the basic loop p − p +1 q p − (EPR ppq ). Thus, theEPR associated with synaptic plasticity takes the formEPR = EPR − pp + EPR + pp + EPR − pq + EPR + pq + EPR ppq , (16)46here the appropriate contributions readEPR ± pp = n − X k =1 g k α ± k ( ǫ p ± k − p ± k +1 ) ln ǫ p ± k p ± k +1 ! , (17)EPR ± pq = n − X k =1 g k " β ± k ( ǫ p ± k − q ± k ) ln ǫ p ± k q ± k ! + ( ǫ b p ± k +1 − a ± k q ± k ) ln ǫ b p ± k +1 a ± k q ± k ! , (18)and the basic loop contribution isEPR ppq = α ( p +1 − ǫ p − ) ln p +1 ǫ p − ! + β ( q − ǫ p − ) ln q ǫ p − ! (19)+( a q − ǫ b p +1 ) ln a q ǫ b p +1 ! . Energy used for synaptic stimulation and subsequent recovery to NESS state, whichis the energy needed to keep a memory trace above the threshold is defined as E = Z T m dt EPR( t ) , (20)47here t = 0 relates to the moment of stimulation. The relative energy used for main-taining memory is defined as the ratio ( E − E ) /E , where E = EPR T m is the baselineenergy used during the time interval of duration T m , and EPR is the baseline entropyproduction rate. Parameters used in the model.
The following default values of the parameters were used. For the transition rates: a = 0 . − , b = 0 . − (Molden et al 2014), α = 0 .
05 min − , β = 20 . − (Miller et al 2005), ǫ = 0 . ǫ = 0 . ǫ = 0 . σ = 0 .
25. Note that the direct spontaneous transitions for protein activationinduced by thermal fluctuations are much weaker than intermediate transitions associ-ated with ATP hydrolysis. Default values for synaptic stimulation: A + = 50, A − = 10,and τ = 10 min. Other values: number of synapses N s = 10 (typical number in acortical column with 10 neurons), number of states for up and down configurations n = 5, fraction of potentiated synapses f = 0 .
5, and the slowing-down rate z = 0 .
8. Allthe figures are made for these default values unless indicated otherwise.
Acknowledgments
The work was supported by the Polish National Science Centre (NCN) grant no.2015/17/B/NZ4/02600. 48 eclaration of interests
The author declares no competing interests.49 eferences
Abraham WC, Bear MF (1996) Metaplasticity: the plasticity of synaptic plasticity.
Trends Neurosci. : 126-130.Aiello LC, Wheeler P (1995) The expensive-tissue hypothesis: The brain and thedigestive-system in human and primate evolution. Curr. Anthropology : 199-221.Antunes G, Schutter ED (2012) A stochastic signaling network mediates the probabilis-tic induction of cerebellar long-term depression. J. Neurosci. : 9288-9300.Attwell D, Laughlin SB (2001) An energy budget for signaling in the gray matter of thebrain. J. Cereb. Blood Flow Metabol. : 1133-1145.Barrett AB, Billings GO, Morris RGM, van Rossum MCW (2009) State based modelof long-term potentiation and synaptic tagging and capture. PLoS Comput. Biol. :e1000259.Bayes A, Collins MO, Croning MD, van de Lagemaat LN, Choudhary JS, Grant SG(2012) Comparative study of human and mouse postsynaptic proteomes finds high com-positional conservation and abundance differences for key synaptic proteins. PLoS ONE : e46683.Bean BP, Williams CA, Ceelen PW (1990) ATP-activated channels in rat and bullfrogsensory neurons: current-voltage relation and single-channel behavior. J. Neurosci. :11-19.Benna MK, Fusi S (2016) Computational principles of synaptic memory consolidation. Nature Neurosci. : 1697-1706.Bhalla US (2011) Trafficking motifs as the basis for two-compartment signaling systemsto form multiple stable states. Biophys. J. : 21-32.50halla US, Iyengar R (1999) Emergent properties of networks of biological signalingpathways.
Science : 381-387.Bhalla US (2014) Molecular computation in neurons: a modeling perspective.
Curr.Opin. Neurobiol. : 31-37.Bosch M, Castro J, Saneyoshi T, Matsuno H, Sur M, Hayashi Y (2014) Structural andmolecular remodeling of dendritic spine substructures during long-term potentiation. Neuron : 444-459.Borgdorff AJ, Choquet D (2002) Regulation of AMPA receptor lateral movements. Na-ture : 649-653.Bradshaw JM, Hudmon A, Schulman H (2002) Chemical quenched flow kinetic stud-ies indicate an intraholoenzyme autophosphorylation mechanism for Ca /Calmodulin-dependent protein kinase II. J. Biol. Chem. : 20991-20998.Bustamante C, Liphardt J, Ritort F (2005) The nonequilibrium thermodynamics ofsmall systems.
Phys. Today : 43.Chaudhuri R, Fiete I (2016) Computational principles of memory. Nature Neurosci. : 394-403.Chen X, Vinade L, Leapman RD, Petersen JD, Nakagawa T, Phillips TM, ShengM, Reese TS (2005) Mass of the postsynaptic density and enumeration of three keymolecules. Proc. Natl. Acad. Sci. USA : 11551-11556.Choquet D, Triller A (2013) The dynamic synapse.
Neuron : 691-703.Cingolani LA, Goda Y (2008) Actin in action: the interplay between the actin cy-toskeleton and synaptic efficacy. Nat. Rev. Neurosci. : 344-356.Clarke DD, Sokoloff L (1994) In Siegel GJ et al (eds), Basic Neurochemistry , pp. 645-5180. Raven: New York, NY.Coba MP, Pocklington AJ, Collins MO, Kopanitsa MV, Uren RT, Swamy S, CroningMD, Choudhary JS, Grant SG (2009) Neurotransmitters drive combinatorial multistatepostsynaptic density networks.
Sci. Signal. : ra19.Cohen LD, Zuchman R, Sorokina O, Muller A, Dieterich DC, Armstrong JD, et al.(2013) Metabolic turnover of synaptic proteins: kinetics, interdependencies and impli-cations for synaptic maintenance. PLoS ONE : e63191.Colbran RJ (1993) Inactivation of Ca /Calmodulin-dependent protein kinase II bybasal autophosphorylation. J. Biol. Chem. : 7163-7170.Collins MO, Yu L, Coba MP, Husi H, Campuzano I, Blackstock WP, Choudhary JS,Grant SCN (2005) Proteomic analysis of in vivo phosphorylated synaptic proteins.
J.Biol. Chem. : 5972-5982.De Koninck P, Schulman H (1998) Sensitivity of CaM kinase II to the frequency ofCa oscillations. Science : 227-230.De Rubeis S, He X, Goldberg AP, Poultney CS, Samocha K, et al (2014) Synaptic,transcriptional and chromatin genes disrupted in autism.
Nature : 209-215.Devineni N, Minamide LS, Niu M, Safer D, Verma R, Bamburg JR, Nachmias VT(1999) A quantitative analysis of G-actin binding proteins and the G-actin pool in de-veloping chick brain.
Brain Res. : 129-140.Ehlers MD (2000) Reinsertion or degradation of AMPA receptors determined by activity-dependent endocytic sorting.
Neuron : 511-525.Engl E, Attwell D (2015) Non-signalling energy use in the brain. J. Physiol. :3417-3429. 52ngl E, Jolivet R, Hall CN, Attwell D (2017) Non-signalling energy use in the develop-ing rat brain.
J. Cereb. Blood Flow Metab. : 951-966.Esser SK, Merolla PA, Arthur JV, Cassidy AS, Appuswamy R et al (2016) Convolu-tional networks for fast, energy-efficient neuromorphic computing. Proc. Natl. Acad.Sci. USA : 11441-11446.Fanselow EE, Nicolelis MAL (1999) Behavioral modulation of tactile responses in therat somatosensory system.
J. Neurosci. : 7603-7616.Francois-Martin C, Rothman JE, Pincet F (2017) Low energy cost for optimal speedand control of membrane fusion. Proc. Natl. Acad. Sci. USA : 1238-1241.Fromer M, Pocklington AJ, Kavanagh DH, Williams HJ, Dwyer S, et al (2014) De novomutations in schizophrenia implicate synaptic networks.
Nature : 179-184.Fusi S, Drew PJ, Abbott LF (2005) Cascade models of synaptically stored memories.
Neuron : 599-611.Gaertner TR, Kolodziej SJ, Wang D, Kobayashi R, et al (2004) Comparative analysesof the three-dimensional structures and enzymatic properties of α, β, γ and δ isoformsof Ca -Calmodulin-dependent protein kinase II. J. Biol. Chem. : 12484-12494.Gold PE (2005) Glucose and age-related changes in memory.
Neurobiol. Aging :S60-S64.Gordon GRJ, Baimoukhametova DV, Hewitt SA, Kosala WRA, Rajapaksha JS, FisherTE, Bains J (2005) Norepinephrine triggers release of glial ATP to increase postsynap-tic efficacy.
Nat. Neurosci. : 1078-1086.Grafmuller A, Shillcock J, Lipowsky R (2009) The fusion of membranes and vesicles:pathway and energy barriers from dissipative particle dynamics. Biophys. J : 2658-53675.Graupner M, Brunel N (2007) STDP in a bistable synapse model based on CaMKIIswitch: dependence on the number of enzyme molecules and protein turnover. PLoSComput. Biol. : e221.Gumbart J, Chipot C, Schulten K (2011) Free-energy cost for translocon-assisted inser-tion of membrane proteins. Proc. Natl. Acad. Sci. USA : 3596-3601. Harris JJ,Jolivet R, Attwell D (2012) Synaptic energy use and supply.
Neuron : 762-777.Hayer A, Bhalla US (2005) Molecular switches at the synapse emerge from receptor andkinase traffic. PLoS Comput. Biol. : e20.Hill TL (1989) Free Energy Transduction and Biochemical Cycle Kinetics . New York:Springer-Verlag.Honkura N, Matsuzaki M, Noguchi J, Ellis-Davies GCR, Kasai H (2008) The subspineorganization of actin fibers regulates the structure and plasticity of dendritic spines.
Neuron : 719-729.Huganir RL, Nicoll RA (2013) AMPARs and synaptic plasticity: the last 25 years. Neuron : 704-717.Iossifov I, O’Roak BJ, Sanders SJ, Ronemus M, Krumm N, et al (2014) The contribu-tion of de novo coding mutations to autism spectrum disorder. Nature : 216-221.Jourdain P, Bergersen LH, Bhaukaurally K, et al (2007) Glutamate exocytosis fromastrocytes controls synaptic strength.
Nat. Neurosci. : 331-339.Kaizuka T, Takumi T (2018) Postsynaptic density proteins and their involvement inneurodevelopmental disorders. J. Biochem. : 447-455.Kandel ER, Dudai Y, Mayford MR (2014) The molecular and systems biology of mem-54ry.
Cell : 163-186.Karbowski J (2007) Global and regional brain metabolic scaling and its functional con-sequences.
BMC Biol. : 18.Karbowski J (2009) Thermodynamic constraints on neural dimensions, firing rates,brain temperature and size. J. Comput. Neurosci. : 415-436.Karbowski J (2011) Scaling of brain metabolism and blood flow in relation to capillaryand neural scaling. PLoS ONE : e26709.Karbowski J (2012) Approximate invariance of metabolic energy per synapse duringdevelopment in mammalian brains. PLoS ONE : e33425.Karbowski J (2014) Constancy and trade-offs in the neuroanatomical and metabolicdesign of the cerebral cortex. Front. Neural Circuits : 9.Karbowski J (2015) Cortical composition hierarchy driven by spine proportion econom-ical maximization or wire volume minimization. PloS Comput. Biol. : e1004532.Kasai H, Matsuzaki M, Noguchi J, Yasumatsu N, Nakahara H (2003) Structure-stability-function relationships of dendritic spines. Trends Neurosci. : 360-368.Khakh BS, North RA (2012) Neuromodulation by extracellular ATP and P2X receptorsin the CNS. Neuron : 51-69.Kim B, Hawes SL, Gillani F, Wallace LJ, Blackwell KT (2013) Signaling pathwaysinvolved in striatal synaptic plasticity are sensitive to temporal pattern and exhibitspatial specificity. PloS Comput. Biol. : e1002953.Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, et al (2017)Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. USA : 3521-3526. 55rebs EG (1981) Phosphorylation and dephosphorylation of glycogen phosphorylase:a prototype for reversible covalent enzyme modification.
Curr. Top. Cell. Regul. :401-419.Laughlin SB, de Ruyter van Steveninck RR, Anderson JC (1998) The metabolic costof neural information. Nature Neurosci. : 36-40.Lebowitz JL, Spohn H (1999) A Gallavotti-Cohen-type symmetry in the large deviationfunctional for stochastic dynamics. J. Stat. Phys. : 333-365.Leibold C, Kempter R (2008) Sparseness constrains the prolongation of memory life-time via synaptic metaplasticity. Cereb. Cortex : 67-77.Levy WB, Baxter RA (1996) Energy efficient neural codes. Neural Comput. : 531-543.Li J, Wilkinson B, Clementel VA, Hou J, O’Dell TJ, Coba MP (2016) Long-term po-tentiation modulates synaptic phosphorylation networks and reshapes the structure ofthe postsynaptic interactome. Sci. Signal. : rs8.Lin JW, Ju W, Foster K, Lee SH, Ahmadian G, Wyszynski M, Wang YT, Sheng M(2000) Distinct molecular mechanisms and divergent endocytotic pathways of AMPAreceptor internalization. Nat. Neurosci. : 1282-1290.Lisman J, Yasuda R, Raghavachari S (2012) Mechanisms of CaMKII action in long-termpotentiation. Nat. Rev. Neurosci. : 169-182.Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature :869-878.Matsuzaki M, Ellis-Davies GCR, Nemoto T, Miyashita Y, Iino M, Kasai H (2001) Den-dritic spine geometry is critical for AMPA receptor expression in hippocampal CA1pyramidal neurons.
Nat. Neurosci. : 1086-1092.56ehta P, Schwab DJ (2012) Energetic costs of cellular computation. Proc. Natl. Acad.Sci. USA : 17978-17982.Meyer D, Bonhoeffer T, Scheuss V (2014) Balance and stability of synaptic structuresduring synaptic plasticity.
Neuron : 430-443.Michalski PJ (2013) The delicate bistability of CaMKII. Biophys. J. : 794-806.Miller P, Zhabotinsky AM, Lisman JE, Wang X-J (2005) The stability of a stochasticCaMKII switch: Dependence on the number of enzyme molecules and protein turnover.
PLoS Biol. : e107.Molden RC, Goya J, Khan Z, Garcia BA (2014) Stable isotope labeling of phospho-proteins for large-scale phosphorylation rate determination. Molecular & Cellular Pro-teomics : 1106-1118.Montgomery JM, Madison DV (2004) Discrete synaptic states define a major mecha-nism of synapse plasticity. Trends Neurosci. : 744-750.Nicolis G, Prigogine I (1977) Self-Organization in Nonequilibrium Systems . Wiley: NewYork, NY.Nimchinsky EA, Yasuda R, Oertner TG, Svoboda K (2004) The number of glutamatereceptors opened by synaptic stimulation in single hippocampal spines.
J. Neurosci. : 2054-2064.O’Connor DH, Wittenberg GM, Wang SSH (2005) Graded bidirectional synaptic plas-ticity is composed of switch-like unitary events. Proc. Natl. Acad. Sci. USA :9679-9684.Pankratov Y, Lalo U, Krishtal OA, Verkhratsky A (2009) P2X receptors and synapticplasticity.
Neuroscience : 137-148.57etersen CC, Malenka RC, Nicoll RA, Hopfield JJ (1998) All-or-none potentiation atCA3-CA1 synapses.
Proc. Natl. Acad. Sci. USA : 4732-4737.Phillips R, Kondev J, Theriot J, Garcia H (2012) Physical Biology of the Cell . GarlandScience: London.Poo M-m, Pignatelli M, Ryan TJ, Tonegawa S, Bonhoeffer T, Martin KC, Rudenko A,Tsai L-H, Tsien RW, Fishell G, et al (2016) What is memory? The present state of theengram.
BMC Biol. : 40.Pougnet JT, Toulme E, Martinez A, Choquet D, Hosy E, Boue-Grabot E (2014) ATPP2X receptors downregulate AMPA receptor trafficking and postsynaptic efficacy inhippocampal neurons. Neuron : 417-430.Qian H (2007) Phosphorylation energy hypothesis: Open chemical systems and theirbiological function. Annu. Rev. Phys. Chem. : 113-142.Qian H (2006) Open-system nonequilibrium steady state: Statistical thermodynamics,fluctuations, and chemical oscillations. J. Phys. Chem. B : 15063-15074.Raichle ME, Mintun MA (2006) Brain work and brain imaging.
Annu. Rev. Neurosci. : 449-476.Rolfe DFS, Brown GC (1997) Cellular energy utilization and molecular origin of stan-dard metabolic rate in mammals. Physiol. Revs. : 731-758.Sabatini BL, Oertner TG, Svoboda K (2002) The life cycle of Ca ions in dendriticspines. Neuron : 439-452.Schnakenberg J (1976) Network theory of microscopic and macroscopic behavior ofmaster equation systems. Reviews of Modern Physics : 571-585.Sheng M, Hoogenraad CC (2007) The postsynaptic architecture of excitatory synapses:58 more quantitative view. Annu. Rev. Biochem. : 823-847.Sheng M, Kim E (2011) The postsynaptic organization of synapses. Cold Spring HarbPerspect Biol : a005678.Shoenbaum G, Chiba AA, Gallagher M (1999) Neural encoding in orbitofrontal cortexand basolateral amygdala during olfactory discrimination learning. J. Neurosci. :1876-1884.Shulman RG, Rothman DL (1998) Interpreting functional imaging studies in terms ofneurotransmitter cycling. Proc. Natl. Acad. Sci. USA : 11993-11998.Shulman RG, Rothman DL, Hyder F (1999) Stimulated changes in localized cerebralenergy consumption under anesthesia. Proc. Natl. Acad. Sci. USA : 3245-3250.Shulman RG, Rothman DL, Behar KL, Hyder F (2004) Energetic basis of brain activ-ity: implications for neuroimaging. Trends Neurosci. : 489-495.Smith MA, Riby LM, van Eekelen JAM, Foster JK (2011) Glucose enhancement ofhuman memory: A comprehensive research review of the glucose memory facilitationeffect. Neurosci. Biobehavior. Revs. : 770-783.Smolen P, Baxter DA, Byrne JH (2012) Molecular constraints on synaptic tagging andmaintenance of long-term potentiation: a predictive model. PLoS Comput. Biol. :e1002620.Sonnay S, Duarte JM, Just N, Gruetter R (2016) Compartmentalised energy metabolismsupporting glutamatergic neurotransmission in response to increased activity in the ratcerebral cortex: A 13C MRS study in vivo at 14.1 T. J. Cereb. Blood Flow Metab. :928.Sonnay S, Poirot J, Just N, Clerc AC, Gruetter R, Rainer G, Duarte JMN (2018) Astro-59ytic and neuronal oxidative metabolism are coupled to the rate of glutamate-glutaminecycle in the tree shrew visual cortex. Glia : 477-491.Star EN, Kwiatkowski DJ, Murthy VN (2002) Rapid turnover of actin in dendriticspines and its regulation by activity. Nature Neurosci. : 239-246.Sun L, Zhang Y, Hwang G, Jiang J, Kim D, et al (2018) Synaptic computation enabledby Joule heating of single-layered semiconductors for sound localization. Nano Lett. : 3229-3234.Takeuchi T, Duszkiewicz AJ, Morris RGM (2014) The synaptic plasticity and memoryhypothesis: encoding, storage and persistence. Phil. Trans. R. Soc. B : 20130288.Trinidad JC, Specht CG, Thaalhammer A, Schoepfer R, Burlingame AL (2006) Com-prehensive identification of phosphorylation sites in postsynaptic density preparations.
Molecular & Cellular Proteomics : 914-922.Trinidad JC, Barkan DT, Gulledge BF, Thaalhammer A, Sali A, Schoepfer R, BurlingameAL (2012) Global identification and characterization of both O-GlcNAcylation andphosphorylation at murine synapse. Molecular & Cellular Proteomics : 215-229.Van den Broeck C, Esposito M (2015) Ensemble and trajectory thermodynamics: Abrief introduction. Physica A : 6-16.Visscher K, Schnitzer MJ, Block SM (1999) Single kinesin molecules studied with amolecular force clamp.
Nature : 184-189.Volgushev M, et al (2004) Probability of transmitter release at neocortical synapses atdifferent temperatures.
J. Neurophysiol. : 212-220.Yoshimura Y, Yamauchi Y, Shinkawa T, Taoka M, et al (2004) Molecular constituentsof the postsynaptic density fraction revealed by proteomic analysis using multidimen-60ional liquid chromatography-tandem mass spectrometry. J. Neurochem. : 759-768.Zhang J, Petit CM, King DS, Lee AL (2011) Phosphorylation of a PDZ domain exten-sion modulates binding affinity and interdomain interactions in postsynaptic density-95(PSD-95) protein, a membrane-associated guanylate kinase (MAGUK). J. Biol. Chem. : 41776-41785.Zhu J, Shang Y, Zhang M (2016) Mechanistic basis of MAGUK-organized complexesin synaptic development and signalling.
Nat. Rev. Neurosci. : 209-223.Zhu F, Cizeron M, Qiu Z, Benavides-Piccione R, Kopanitsa MV, Skene NG, KoniarisB, DeFelipe J, Fransen E, Komiyama NH, Grant SGN (2018) Architecture of the mousebrain synaptome. Neuron : 781-799.61 igure Captions Fig. 1Cascade model of synaptic plasticity with cyclic and noncyclic reactions. (A) An example of the 3 state model with noncyclic reactions. For this model with a“ladder” structure EPR at steady state is zero (see the Methods), and hence this config-uration is at thermal equilibrium with the environment and has a vanishing metabolicrate. This is not a realistic situation.(B) Synaptic plasticity model with cyclic reactions. This model yields nonzero EPRand metabolic rate at steady state due to “reverberating” loop that creates a nonzero(cyclic) probability flux (see the Methods). State with the probability p − describes aprotein in a ground state, state with the probability q corresponds to the protein +substrate complex, and state with the probability p + is the activated state of the pro-tein + substrate complex that can signal (activate) to other downstream molecules. Toinitiate some function a synapse must move from the ground state p − to state p + . How-ever, direct transitions between p − and p + are rare (although possible due to thermalfluctuations), and the protein has to use an alternative pathway to reach the activatedstate p + . This involves a sequence of two transitions from p − to q (binding a proteinwith a substrate; relatively fast), and from q to p + (activation of the protein + sub-strate complex). The latter transition is powered by ATP hydrolysis (ATP ↔ ADP+ P), which provides a necessary energy for speeding up this transition. When localconcentration of ATP increases due to calcium influx, the transition rate q → p + in-creases accordingly. The process q → p + can be thought as protein phosphorylation,e.g., phosphorylation of CaMKII and/or PSD-95, which are the most abundant and one62f the main signaling molecules in dendritic spines (Sheng and Kim 2011). This cyclicmolecular motif is known as a phosphorylation-dephosphorylation cycle and serves as abuilding block for constructing more complex signaling molecular networks (Hill 1989;Qian 2007).(C) Cascade synaptic model with many basic cyclic motifs. This is an expansion of themodel in panel B, and contains two chains of “up” and “down” synaptic states corre-sponding to binary synaptic weight or a number of AMPA receptors expressed in thespine membrane (Petersen et al 1998; O’Connor et al 2005). Occupancy probabilities inthe up and down states are respectively p + k , q + k and p − k , q − k . The transitions p ± k ↔ p ± k +1 are spontaneous and correspond to the direct rare transitions p − ↔ p + in panel B.The pathway p ± k ↔ q ± k ↔ p ± k +1 is indirect, since it involves an intermediate state q ± k (corresponding to state q in panel B). The reactions q ± k → p ± k +1 correspond to proteinphosphorylation driven by ATP hydrolysis with transition rates a ± k , while the reversereactions p ± k +1 → q ± k describe much slower dephosphorylations with transition rate ǫ b that is a few orders of magnitude smaller than a ± k (Qian 2006, 2007). All the tran-sition rates a ± k are proportional to the amplitude a . The transitions in the direction p + k → p + k +1 (plus p − → p +1 ) of the up states are associated with LTP process, whereasthe transitions in the direction p − k → p − k +1 of the down states are related to LTD. Thedynamics of downstream cycles (higher index k ) both for the up and down states arescaled down by a factor e − kz with a characteristic slowing down factor z , which meansthat the transitions within deeper states are progressively slower. When the synapseis stimulated and a plasticity process initiated, only the ATP-driven rates from q ± k to p ± k +1 , denoted as a ± k are time dependent. Specifically, for synapses undergoing LTP the63ates a + k are time dependent, while for synapses undergoing LTD the rates a − k changein time. The state with the occupancy p − is the ground state (depressed). Fig. 2Dependence of baseline synaptic plasticity energy rate EPR on internalsynaptic parameters. Baseline EPR is almost independent on the number of basicsynaptic states n (upper panel; blue diamonds for z = 0, red squares for z = 0 .
8, yellowcircles for z = 1 . z (middle panel). Dependence of EPR on the ATP-driven transition rate a ismore complex (bottom panel): when z = 0 (solid line) EPR increases monotonicallywith a up to a saturation, while for z > z = 0 .
8, dashed-dotted linefor z = 1 .
5) EPR exhibits a bimodal shape and decays to 0 for large a . Fig. 3Time course of memory trace SNR and associated entropy production rateEPR after synaptic stimulation . Synaptic stimulation A ± ( t ) affects the ATP driventransitions a ± k in the following way: a ± k a ± k ( t ) = a ± k [1 + A ± ( t )], where A ± ( t ) = A ± exp( − t/τ ). A) A + ( t ), SNR, and EPR as functions of time for different stimulusamplitudes A + (solid blue line for A + = 50, dashed red line for A + = 10). B) A + ( t ),SNR, and EPR as functions of time for different stimulus durations τ (solid blue linefor τ = 20 min, dashed red line for τ = 2 min). Note that for both cases A) and B)the SNR decays to zero as approximately a power law ∼ t − δ with δ ≈ .
3. This ismuch slower decay than EPR relaxation to its baseline, which essentially follows thestimulation A + ( t ). Moreover, EPR stabilizes at a nonzero value, which is a signature64f a non-equilibrium steady state with a corresponding nonzero metabolic rate. For allplots z = 1 . Fig. 4Relative cost of maintaining a memory trace can be small for progressivelyslower downstream synaptic transitions.
Relative energy associated with a mem-ory trace above baseline ∆
E/E (where ∆ E = E − E , and E = EPR T m ) as a functionof amplitude A + and duration τ of the stimulation. Note that ∆ E/E generally getssmaller for increasing the rate of synaptic slowing down z . Fig. 5Effect of synaptic stimulation magnitude on memory lifetime and its energycost. (A) Memory lifetime T m , its energy cost E and their ratio T m /E as functions ofstimulation amplitude A + . (B) The same variables as functions of stimulus duration τ . Note that in both cases the ratio T m /E is essentially constant, which indicatesthat longer memories need proportionally more energy. In all panels, solid blue linecorrespond to z = 0, dashed red line to z = 0 .
8, and dotted yellow line to z = 1 . Fig. 6Memory lifetime and its energy cost as functions of the number of synapticstates n . Memory lifetime per energy is essentially constant as a function of n . Notethat the relative energy above baseline for maintaining memory trace is a tiny percentagefor z > n . Blue diamonds correspond to z = 0, red squares to z = 0 .
8, andyellow circles to z = 1 .
5. 65 ig. 7Dependence of memory lifetime and its energy cost on the fraction of po-tentiated synapses f . For sufficiently large f ( f > . T m , itsenergy cost E , as well as T m /E and ∆ E/E are all essentially constant. Blue solid linecorresponds to z = 0, red dashed line to z = 0 .
8, and yellow dotted line to z = 1 . Fig. 8Memory lifetime and its energy cost as functions of slowing-down rate z .The ratio T m /E increases monotonically but weakly with z , and ∆ E/E decreases with z . Blue solid line corresponds to f = 0 .
2, red dashed line to f = 0 .
5, and yellow dottedline to f = 0 . Fig. 9Memory lifetime and its energy cost as functions of the phosphorylationrate amplitude a . Surprisingly, for sufficiently large z longer memories can requireless energy (two upper panels). The ratio T m /E exhibit a minimum for some a , butincreases relatively fast for larger a , especially for larger z . Blue solid line correspondsto z = 0, red dashed line to z = 0 .
8, and yellow dotted line to z = 1 . − ) Extra-synaptic
Glutamate recycling 1602ATP binding to spine 60
Intra-synaptic Ca removal 6000Protein phosphorylation:active LTP ∗ − Modulatory
P2X receptors 2871 ∗ Values depend on Ca stimulation rate.67able 2: Comparison of the metabolic costs of synaptic plasticity and fastexcitatory synaptic transmission for rat brain.Synaptic plasticity Postsynaptic current Plasticity/Transmissiontotal cost (ATP/min) cost (ATP/min) relative cost (%)(3 . − . · . · . − . B p_p+ q w w w w AC p1+ p2+ p3+ p4+ p5+p1−p2−p3−p4−p5− q0q1−q2−q3−q4− q1+ q2+ q3+ q4+ αε1α ε2β β ATP a ε3 b Figure 1 n EP R ( k T / m i n ) z EP R ( k T / m i n ) a (min -1 ) EP R ( k T / m i n ) Figure 2 -2 A + ( t ) -2 S NR -2 Time (hr) -1 EP R ( k T / m i n ) -2 A + ( t ) -2 S NR -2 Time (hr) -1 EP R ( k T / m i n ) A B
Figure 3
100 200 300 400 500 A + E / E (min) E / E z=0 z=1.5z=0.8z=0 z=0.8 z=1.5 Figure 4
100 200 30010 T m ( h r) T m ( h r) E ( k T ) E ( k T ) A + T m / E ( h r / k T ) (min) T m / E ( h r / k T ) BA Figure 5 E ( k T ) T m / E ( h r / k T ) n E / E T m ( h r) Figure 6 T m ( h r) E ( k T ) T m / E ( h r / k T ) f E / E Figure 7 T m ( h r) E ( k T ) T m / E ( h r / k T ) z E / E Figure 8 T m ( h r) E ( k T ) T m / E ( h r / k T ) a (min -1 ) E / E0