Bifurcation From Networks of Unstable Attractors to Heteroclinic Switching
aa r X i v : . [ n li n . C D ] D ec From Networks of Unstable Attra tors to Hetero lini Swit hingChristoph Kirst − and Mar Timme , Network Dynami s Group, Max Plan k Institute for Dynami s and Self-Organization (MPIDS) and Bernstein Center for Computational Neuros ien e (BCCN) Göttingen, 37073 Göttingen, Germany Fakultät für Physik, Georg-August-Universität Göttingen, Germany DAMTP, Centre for Mathemati al S ien es, Cambridge University, Cambridge CB3 0WA, UKWe present a dynami al system that naturally exhibits two unstable attra tors that are ompletelyen losed by ea h others basin volume. This ounter-intuitive phenomenon o urs in networks ofpulse- oupled os illators with delayed intera tions. We analyti ally show that upon ontinuouslyremoving a lo al non-invertibility of the system, the two unstable attra tors be ome a set of twonon-attra ting saddle states that are hetero lini ally onne ted. This transition equally o ursfrom larger networks of unstable attra tors to hetero lini stru tures and onstitutes a new type ofsingular bifur ation in dynami al systems.PACS numbers: 05.45.Xt, 02.30.Oz,The on epts of attra tor and stability are at the oreof dynami al systems theory [1℄ be ause attra tivity andstability determine the long term behavior and often thetypi al properties of a system. Attra tion and stabilitywhi h may hange via bifur ations are thus fundamen-tal to modeling in all of s ien e and engineering. Forsystems with smooth and invertible (cid:29)ows these on eptshave long been studied and are well understood, allow-ing lassi(cid:28) ations of dynami al systems and their bifur- ations, for example by using topologi al equivalen e andnormal forms.Dynami al systems with non-smooth or non-invertible(cid:29)ows, su h as hybrid or Fillipov systems [2℄, are far lessunderstood although they model a variety of natural phe-nomena, ranging from the me hani s of sti k-slip motionand the swit hing dynami s of ele tri al ir uits, to thegeneration of earthquakes and the spiking a tivity of neu-ral networks [3, 4, 5℄. For instan e, spiking neurons in-tera t by sending and re eiving ele tri al pulses at dis- rete instan es of time that interrupt the intermediatesmooth intera tion-free dynami s. This neural dynami sand similarly that of, e.g., ardia pa emaker ells, platete toni s in earthquakes, hirping ri kets and (cid:29)ashing(cid:28)re(cid:29)ies are often modeled as pulse- oupled os illators.Su h hybrid systems display dynami s very di(cid:27)erentfrom that of temporally ontinuous or temporally dis- rete systems. Networks of os illators with global ho-mogeneous delayed pulse- oupling may robustly exhibitunstable attra tors [5℄ (invariant periodi orbits that areMilnor attra tors [1℄ but lo ally unstable). In the pres-en e of noise, these systems exhibit a dynami s akinto hetero lini swit hing [6℄, a feature that is fun tion-ally relevant in many natural systems su h as in neural,weather and population dynami s [6, 7℄. Rigorous anal-ysis [8℄ shows that invertible systems in general annothave unstable attra tors and that a saddle state an inprin iple be onverted to an unstable attra tor by lo- ally adding a non-invertible dynami s onto the stablemanifold. However, the potential relation of unstable at- tra tors to hetero lini y les is not well understood andit is unknown whether and how unstable attra tors maybe reated or destroyed via bifur ations.In a network of pulse- oupled os illators we heredemonstrate the existen e of two unstable attra tors thatare en losed by the basin of attra tion of ea h other. Weexplain this ounter-intuitive phenomenon: Continuouslylifting the lo al non-invertibility of the system with twounstable attra tors reates a standard hetero lini two- y le. This transition equally o urs from large networksof unstable attra tors to hetero lini stru tures and on-stitutes a new type of singular bifur ation in hybrid dy-nami al systems.We onsider a network of N os illatory units with astate de(cid:28)ned by a phase-like variable φ i ( t ) ∈ R , i ∈{ , , . . . , N } , that in reases uniformly in time tddt φ i = 1 . (1)Upon rossing a threshold at time t s , φ i ( t s ) ≥ , unit i is instantaneously reset, φ i (cid:0) t + s (cid:1) := lim r ց φ i ( t s + r ) = K ( φ i ( t s )) . (2)Here K ( φ ) = U − ( R ( U ( φ ) − is determined by asmooth, unbounded, stri tly monotoni in reasing risefun tion U ( φ ) normalized to U (0) = 0 and U (1) = 1 and a smooth non-negative reset fun tion R satisfying R (0) = 0 . In addition to the reset (2) a pulse is sentwhi h is re eived by all units j after a delay time τ > ,indu ing a phase jump φ j ( t s + τ ) = H ε ji (cid:16) φ j (cid:16) ( t s + τ ) − (cid:17)(cid:17) (3)with intera tion fun tion H ε ( φ ) = U − ( U ( φ ) + ε ) and oupling strength ε ji from unit i to unit j . We set J ε ( φ ) = K ◦ H ε ( φ ) and denote a phase shift by S η ( φ ) = φ + η .Figure 1: ( olor online) Two unstable attra tors en losed bythe basins of ea h other ( c = 0 ). (a) Phases φ i ( t s ) (dots)of all units at times t s just after the s -th reset of a referen eunit i = 1 . Lines indi ate the phases on the invariant orbit A ( α , solid) and A (dashed). Arrows mark times of smallphase perturbations whi h indu e swit hes from A to A or vi e versa. The shaded area highlights a swit h from A to A that is shown in detail in Fig. 2. (b) Fra tion ν i of5000 traje tories rea hing the periodi orbit A i ( • : i = 1 , × : i = 2 ) starting from random phases distributed uniformlyin a box of side width δ entered around φ = a on the orbit A . For < δ < δ crit ≈ . all traje tories rea h the orbit A ( ν = 1 ), indi ating that A is en losed by the basin volumeof A and in parti ular that A is an unstable attra tor.This system represents, for instan e, an abstra tmodel of neuronal os illators with a membrane poten-tial u i ( t ) = U ( φ i ( t )) . The neurons' responses to in-puts are des ribed by in reasing the potentials instanta-neously by an amount ε ij , that represents the transferred harge from the pulse sending (presynapti ) neuron j tothe pulse re eiving (postsynapti ) one i . If this inputis supra-threshold, u i ( t ) = u i ( t − ) + ε ij > , unit i ispartially reset to u i (cid:0) t + (cid:1) = R ( u i ( t ) − ≥ . (4)This a ounts for remaining synapti input harges whi hare not used to rea h the threshold and whi h ontributeto the potential after reset [9℄. For R ( ζ ) ≡ we re overthe model analyzed in previous studies [5℄ whi h has alo al non-invertibility sin e the original phase of a unit annot be re overed after it re eived supra-threshold in-put and was reset to J ε ( φ ) ≡ . For an invertible R the(cid:29)ow be omes lo ally time invertible.Here we fo us on a homogeneous network of all-to-all oupled ex itatory units without self-intera tion, i.e. ε ij = (1 − δ ij ) ε , ε > . The permutation symme-try implies invariant subspa es of two or more syn hro-nized units and thus the possibility of robust hetero- lini y les, f. [10℄. For the numeri al simulations pre-sented below we (cid:28)x ε = 0 . , τ = 0 . , a rise fun tion U ( φ ) = b log (1 + (exp( b ) − φ ) with b = 4 . and par-tial reset R ( ζ ) = cζ with parameter c ∈ [0 , whi h isinvertible for all c > . For these parameters the modelexhibits short swit hing times between periodi orbitswhi h simpli(cid:28)es the presentation of the analysis below;however, the studied phenomena is robust against stru -tural perturbations in τ , ε and the fun tion U . For lo ally non-invertible dynami s ( c = 0 ) the abovesystem exhibits unstable attra tors in a large fra tionof parameter spa e and for di(cid:27)erent network sizes N [5,8℄. For the above parameters, the smallest system inwhi h we observed unstable attra tors has N = 4 units.Curiously, numeri al simulations, e.g. Fig. 1a, indi atethat su h a system exhibits two unstable attra tors ea hof whi h is fully en losed by the basin volume of the otherattra tor, Fig. 1b.We on(cid:28)rm these numeri al (cid:28)ndings analyti ally.Given a periodi orbit A , de(cid:28)ne the basin of attra tion B ( A ) as the set of points in state spa e that onvergeto A in the long time limit. Below we show that in thesystem (1)-(3) with R ( ζ ) ≡ there is a pair of periodi orbits A and A su h that a full measure set of pointsof an open neighborhood of A is ontained in the basin B ( A ) and vi e versa.To study the dynami s in detail we use an event basedanalysis, f. e.g. [4℄. The event when a unit i sends apulse is denoted by s i , the re eption of a pulse from unit j by r j and simultaneous events are en losed in paren-theses. For given parameter c ∈ [0 , , a simple saddleperiodi orbit A ( f. Figs. 1a and 3a) is uniquely deter-mined by the y li event sequen e E ( A ) = ( s , s ) ( r , r , s , s ) ( r , r ) (5)By ex hanging the indi es (1 , ↔ (3 , in (5) we obtainthe event sequen e of a permutation equivalent periodi orbit A . Both orbits lie in the interse tion of the twoinvariant subspa es { φ = φ } and { φ = φ } with syn- hronized units (1 , and (3 , , respe tively, allowing arobust hetero lini onne tion between them. As it turnsout below the lo al stability and non-lo al attra tivityproperties of the A i depend on the parameter c .We now (cid:28)rst lo ally redu e the in(cid:28)nite dimensionalstate spa e of the hybrid dynami al system with de-layed oupling to three dimensions: Lo al to A and A the state spa e redu es in (cid:28)nite time [8℄ to an eight-dimensional state spa e spanned by the four phases φ = ( φ , φ , φ , φ ) , and the four times σ i ≥ , i ∈{ , . . . , } elapsed sin e the most re ent pulse gener-ation of os illator i . We onsider the subset M = { ( φ, σ ) | σ i > τ, i ∈ { , . . . , }} of the state spa e whereall pulses have been re eived: Then the state spa e ise(cid:27)e tively four dimensional, sin e the exa t values of the σ i > τ do not in(cid:29)uen e the dynami s. Due to the uni-form phase shift (1), A is a straight line in M afterthe last and before the (cid:28)rst event in the sequen e (5).We denote the point in the enter of this line by a and onsider states with phases φ = a + ( δ , δ , δ , δ ) ina neighborhood. Be ause of shift invarian e we may fur-ther (cid:28)x δ = 0 , being left with a lo ally three-dimensionalrepresentation P ⊂ R of the original state spa e withstates ( δ , δ , δ ) ∈ P . Similarly, we have a lo al three-dimensional representation ( δ , δ , δ ) ∈ P of the statespa e around a ∈ M ∩ A , onstru ted analogously toFigure 2: ( olor online) Stru ture of the three-dimensional re-du ed state spa e for c = 0 showing that the A i are unstableattra tors en losed by the basins of ea h other. (a) Repre-sentations P i of the state spa e in a neighborhood of A ∈ P ( ross) and A ∈ P (ball). All traje tories starting in the set C ( lose to A ) lead to a swit h to A . The line with arrowsshows a sample traje tory of the marked swit h from A to A in Fig. 1a. (b) Proje tion of P onto the δ - δ plane and( ) of P onto the δ - δ plane, illustrating that, ex ept forthe lower dimensional subset S i , the attra tor A i is en losedby C i , i.e. the A i are unstable attra tors. a , this time (cid:28)xing δ = 0 . There is an open neigh-borhood of A i in the full eight-dimensional state spa efrom whi h every orbit rosses P i after at most eightevents (one y le). In this sense P i is a three-dimensionalPoin aré se tion in a neighborhood of A i .For arbitrary c ∈ [0 , there are regions in P and P from whi h all traje tories evolve ba k to points in ei-ther P or P . Between these regions we derive returnmaps and their domains whi h follow dire tly from thede(cid:28)nition of the lo al state spa e and the event sequen e(Fig. 2 visualizes domains of the key maps and a sampletraje tory for c = 0 ). For instan e, the orbit A is en- losed by the three-dimensional domain C ⊂ P of themap F : C → P , F ( δ , δ , δ ) = (cid:0) sign ( δ ) (cid:2) H ε ◦ S τ ◦ H ε ( τ + | δ | ) − H ε ◦ S τ + | δ | ◦ H ε ( τ − | δ | ) (cid:3) , δ ′ , δ ′ (cid:1) (6)whi h is determined by the event sequen e E ( C ) = ( s ) ( s ) ( r ) ( r , s , s ) ( r , r ) (7)or its equivalent with permuted indi es ↔ . Here δ ′ i = H ε ◦ S τ ◦ J ε ( H ε ( α + τ + δ i ) + | δ | )+1 − H ε ◦ S τ ◦ H ε ( τ + | δ | ) − α (8)for i ∈ { , } , where the phase di(cid:27)eren e α between thetwo syn hronized lusters at a is determined by α = H ε ◦ S τ ◦ J ε ( α + τ ) + 1 − H ε ◦ S τ ◦ H ε ( τ ) . (9) Figure 3: ( olor online) Hetero lini swit hing ( c > ). (a)Phases φ i ( t s ) (dots) as in Fig. 1a for c = 0 . . The invariantperiodi orbits A i , being unstable attra tors at c = 0 , stillexist for c > (solid and dashed line). Starting in a statenear A leads to repeated swit hing between the two states.Inset: Phase di(cid:27)eren e α (9) (solid) and side width w of theset D ′ i (11) (dashed) hange ontinuously upon in reasing c from zero. (b) Swit hing times T k to the k -th swit h ( × : c = 0 . , ◦ : c = 0 . ) in rease exponentially with k , indi atingthat the dynami s evolve near a hetero lini y le betweenthe invariant states. Inset: Fitting T k = γe κk to the swit hingtimes for several values of c we (cid:28)nd a divergen e of κ as c → .For | δ | > , F is expanding in the δ -dire tion sin e | ( F ( δ )) | > k | δ | (10)with k = min φ ∈ [0 , H ′ ε ( φ ) > . For δ = 0 we obtain amap with the same expli it form as in (6) but with eventsequen e as in (5) whose domain is a subset of the two-dimensional invariant set S = { δ ∈ P | δ = 0 } whereunits and are syn hronized. States in S onvergeto A in the long time limit. Similarly, points in S = { δ ∈ P | δ = 0 } rea h A asymptoti ally.If the system is lo ally non-invertible ( c = 0 ) the dy-nami s is as follows (see Fig. 2): Sin e J ε ( φ ) ≡ , δ ′ = δ ′ a ording to (8) and hen e F maps C into two one-dimensional lines D = F ( C ) . Sin e F is expandingin δ (10), all points in D ∩ C are mapped after a (cid:28)-nite number of intera tions into D \ C . The set D \ C is mapped to E ⊂ S and from there to the attra tor A . In Fig. 2 we have plotted a sample traje tory for theswit h marked in Fig. 1a. For the positive measure set C ∪ S , that en loses A , we thus have C ⊂ B ( A ) andonly the zero measure subset S onverges to A . Thus A is an unstable attra tor. Permutation symmetry im-plies analogous dynami s near A . Taken together, for c = 0 the periodi orbits A and A are unstable attra -tors en losed by the basins of ea h other.If we remove the lo al non-invertibility ( c > ), thedynami s hanges qualitatively as shown in Fig. 3: Thetwo periodi orbits A i with event sequen e (5) still exist,only the phase di(cid:27)eren e α hanges ontinuously with c (Fig. 3a). Starting in a state near one of the A i leads totraje tories with swit hing between both. The swit hingtime in reases exponentially with the number of swit hes(Fig. 3b) indi ating that these dynami s originate froman orbit near a hetero lini two- y le. Furthermore theswit hing times diverge as c → ( f. Fig. 3b), suggestingthe transition to a network of unstable attra tors at c =0 . Indeed, the stru ture of the domains of all returnmaps does not hange qualitatively when c in reases fromzero. However, sin e J ε be omes invertible for c > ,a ording to (8) a phase di(cid:27)eren e | δ − δ | shrinks underthe return map F , but does not ollapse to zero as for c =0 ; hen e the image D ′ = F ( C ) stays three-dimensional.It onsists of tubes (around the original lines D ) with asquare ross-se tion of side width w ( c ) = H ε ◦ S τ ◦ U − ( cε ) − H ε ◦ S τ ◦ U − (0) (11)that ontinuously in reases with c from w (0) = 0 (Fig. 3a). This re(cid:29)e ts the lo al c -dependent ontra tionof the state spa e a ording to (6) and (8). All mapswith domains that have a non-empty interse tion with D ′ map D ′ to a three-dimensional state spa e volumearound A that is a subset of C ∪ S . Taken together,states in the three-dimensional set C evolve to states ina positive measure subset of C ∪ S that en loses A .Using symmetry again, C is analogously mapped to asubset of C ∪ S . This explains the observed swit hing.The unstable attra tors are onverted to non-attra ting saddles by lo ally removing the non-invertibility of the dynami s, whi h is re(cid:29)e ted in the ex-pansion of D i , i ∈ { , } to positive measure sets D ′ i whenin reasing c from zero. Moreover, states in the subset of C with syn hronized units and , i.e. states in the set { δ ∈ C | δ = δ } are mapped to S and thus rea h theorbit A asymptoti ally. Hen e, this set together with allits image points in P and P form a hetero lini onne -tion from A to A . Thus, by symmetry, the network oftwo unstable attra tors ( c = 0 ) ontinuously bifur atesto a hetero lini two- y le ( c > ).The underlying me hanism relies on the interplay ofthe lo al instability (10) and the parameter dependent ontra tion indu ed by the reset (4), implying the sametransition in larger systems (not shown). For lo ally non-invertible dynami s these display larger networks of un-stable attra tors [5℄ with a link between two attra tors A i → A j if every neighborhood of A i ontains a pos-itive basin volume of A j . Upon lifting the lo al non-invertibility ea h link in this network is repla ed by ahetero lini onne tion.In summary, we have presented and analyzed the ounter-intuitive phenomenon of two unstable attra torsthat are en losed by ea h other's basin volume. We ex-plained this phenomenon by showing that there is a on-tinuous transition from two unstable attra tors to a het-ero lini two- y le. Larger networks of unstable attra -tors equally show this transition to more omplex hete-ro lini stru tures. It onstitutes a new type of singu-lar bifur ation in dynami al systems and establishes the(cid:28)rst known bifur ation of unstable attra tors. Moreover,our results show that this bifur ation o urs upon on-tinuously removing the non-invertibility of the system,whereas both the non-invertible ( c = 0 ) and the lo ally invertible ( c >0