Spontaneous organizations of diverse network structures in coupled logistic maps with a delayed connection change
Spontaneous organizations of diverse network structures in coupled logistic maps with a delayed connection change
Amika Ohara , Masashi Fujii , Akinori Awazu Graduate School of Integrated Sciences for Life Hiroshima University. Research Center for the Mathematics on Chromatin Live Dynamics, Hiroshima University. *Corresponding author: Akinori Awazu, [email protected] Abstract In this study, we performed comprehensive morphological investigations of the spontaneous formations of effective network structures among elements in coupled logistic maps, specifically with a delayed connection change. Our proposed model showed ten states with different structural and dynamic features of the network topologies. Based on the parameter values, various stable networks, such as hierarchal networks with pacemakers or multiple layers, and a loop-shaped network were found. We also found various dynamic networks with temporal changes in the connections, which involved hidden network structures. Furthermore, we found that the shapes of the formed network structures were highly correlated to the dynamic features of the constituent elements. The present results provide diverse insights into the dynamics of neural networks and various other biological and social networks. I. INTRODUCTION In biological and social systems, various phenomena are regulated by their self-organized network structures. Neural networks, metabolic networks, food webs, and human communities are typical well-known systems that exhibit self-regulation of network structures through learning, cell differentiation and adaptation, evolution, and communication [1β8]. Such systems have been studied using models that consist of dynamic elements involving changes in the mutual relationships among the elements. For example, to consider the various properties of neural networks, each element that imitates a nerve cell had to be described as an excitable or chaotic oscillator, and the dynamics of element coupling were assumed to follow a rule inspired by Hebb's law or spike-timing-dependent plasticity (STDP) [9-20]. Recently, it has been suggested that coupled chaotic map systems with Hebbian-like rules may result in systems that could exhibit spontaneous hierarchical network structures with asymmetric couplings, as well as the emergence of pacemakers [21,22]. Such hierarchical structure formations, with various types of pacemaker-like elements, have also been found in various other social systems inspired models [23-25]. However, the universal features and diversity of such self-organized network structures have not been sufficiently investigated. In this study, we focus on the behaviors of a simple coupled chaotic map system that involves temporal coupling changes inspired by Hebbβs rule [26], with a time delay proposed by Ito and Ohira [21]. This model was proposed as one of the simplest models of dynamical network systems that exhibits an obvious hierarchical network structure with a pacemaker element. Furthermore, we found that the simple extension of this model involved the potential to spontaneously form a wider variety of network structures, as mentioned below. In the following arguments, we consider the morphology of self-organized network structures and reveal the relationships between the formed network structures and dynamic features of each element in this model. II. MODEL We consider a simple extension of the Ito-Ohira model [21], which is a coupled logistic map system with a temporal change in the connections (couplings), among elements described by π₯ ! ! ! ! = π ( β π π₯ ! ! + π π€ ! !" π₯ ! ! ! ! ! ! ! ! ) (1) π€ ! ! ! !" = ! ! ! !" ! ! ! ! ! ! ! ! ! ! ! ! !" ! ! ! !" ! ! ! ! ! ! ! ! ! ! ! ! !" ! ! ! ! ! ! (2) π π₯ = ππ₯ ( β π₯ ) , (3) where π₯ ! ! β€ π₯ ! ! β€ and π€ ! !" ( β€ π€ ! !" β€ ) are the state of the π -th element and the connection strength from the π -th to π -th element at time π , respectively. π indicates the number of elements ( π , π = , β¦ , π β ), and c β€ π β€ indicates the influence strength from the other elements on the dynamics of the π -th element. We also consider πΏ ( β€ πΏ β€ ) , which is the extension of the model in this study and indicates the connection plasticity; a larger πΏ exhibits more sensitive changes in the connection strength. Note that the present dynamical system fully involves the original Ito-Ohira model, and corresponds when πΏ = [21]. In this model, π€ !! = (no self-connecting) was assumed, and π€ ! !" = ! ! always satisfied for π . For the dynamics of π€ ! !" , the present model employs a simple extension of Hebbβs rule [26] where the state of the π -th element influences the connection from the π -th to π -th elements with a one-step time delay. Furthermore, π€ ! ! ! !" tends to be large when π₯ ! ! and π₯ ! ! ! ! are within close proximity to each other. Moreover, the βnormalizationβ of connections strengths over all elements, π€ ! ! ! !" = ! ! , can be regarded as a simple representation of the global competition among the connection strength. In the following simulations, we consider the case π = ( π = , , , β¦ , ) as in the recent study by Ito and Ohira. For the initial condition of π₯ ! ! and π₯ ! ! ! ! , we set π₯ ! ! = π₯ ! ! ! ! = π₯ !"! ! when π = , where π₯ !"! ! is chosen randomly from π₯ !"! ! β ( , ) with uniform probability (we reorder the indices of all elements π to satisfy π₯ !"! ! < π₯ !"! ! < β― < π₯ !"! !" < π₯ !"! !" .). We also consider π€ ! !" = ! ! ! ! ( π β π ), assuming that π€ ! !" is uniformly connected to all the elements, except itself. We performed simulations of the model for each set of π , π and πΏ from 20 different initial conditions and focused on the most frequently obtained type of network structures at π = ! βΌ ! as the typical network structure. Thus, we confirmed that π = ! is sufficiently large for the relaxation to the state with its typical network structure for each parameter set. This is supported by the autocorrelation functions of the connection strength among the elements, as discussed below and in Appendix. We classified the typical network structures obtained by various parameter sets according to their connection profiles and the temporal changes in these profiles through the following procedures. In the present model, the set π€ ! !" represents the directional network structures of the elements at time π . Here, the connection from the π -th to π -th element is considered strong (weak) when π€ ! !" is large (small). Now, similar to the recent studies by Ito and Ohira, etc. [9, 10, 21,22], we define the connection from π -th to π -th elements as existing at time π if π€ ! !" > π€ ! = ! ! ! ! , where π€ ! indicates the average of π€ ! !" over the entire set of π and π ( π β π ). Therefore, we classified the connection profiles of typical networks obtained via the proposed model according to this definition. The dynamic features of connection profiles in typical networks were estimated by the autocorrelation functions of π π β the dimensional vector π π = ( π€ ! !" ! ! ! ) ( π β₯ ! ) defined as πΆ π = ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ( ! ! ! ! ! ! ! ! (4) where < π > was assumed as < π > = ! ! ! ! , β¦ , ! ! ! ! ( π = ) because the average of π€ ! !" was estimated to be ! ! ! ! if π€ ! ! ! changed ergodically in π . III. RESULTS A. Phase diagram and typical dynamical properties of network structures. Figure 1 shows the phase diagrams of the typical network structures among the elements for , π = . , . , . , β¦ , . , and π = . , . , β¦ . with (a) πΏ = , (b) πΏ = . , and (c) πΏ = . . Based on the abovementioned classification, we obtained ten types of qualitatively different states with different structural and dynamic connection network features depending on π , π and πΏ . Each symbol in FIG. 1 indicates the different states, as mentioned in Table 1. Notably, in the regions of π , π and πΏ , as shown by the black circles in FIG. 1, we obtained { π₯ ! ! } ! , which showed chaotic and synchronized motions, as seen in FIG. 2 (a). In this case, π€ ! !" β ! ! ! ! with π β β indicated that the connections among the elements became uniform, and no specific network structures were formed. In the regions of π , π and πΏ , as shown by the blue squares in FIG. 1, πΆ π drops to zero for finite π as seen in FIG. 2 (b) and Appendix, indicating that all π€ ! !" changes occurred ergodically in π and no stable network structures appeared. On the other hand, in the regions of π , π and πΏ , as shown by the orange triangles, yellow circles, yellow standing triangles, red diamonds, or red stars in FIG. 1, πΆ π ~ for all π (Appendix), indicated that the network structures were stable and unchanged once they were formed. We call these stable networks. We also found that, in other cases, πΆ π decreased but for a relaxed finite value for π β β (Appendix). This indicated that the networks of these observed states are not time stable, but hidden basic network structures do exist and π€ ! !" changes temporally around such basic structures. We call these dynamic networks. Based on these facts, we next focused on the structural and dynamical features of the typical stable and dynamic network structures, where πΆ π > for π β β , as described by π€ ! !" ! ! ! , and the dynamic properties of elements, as described by { π₯ ! ! } ! . B. Morphology of typical structures of stable networks We considered the morphology of typical states with stable network structures that πΆ π ~ are maintained for π β β . To characterize the structural properties of observed networks, we defined two values, the indegree and outdegree of the π β th element at time π , πΌ π· ! ! and π π· ! ! , defined by the number of connections to and from the π - th element, as calculated by πΌ π· ! ! = π ( π€ ! !" β ! ! ! ! ) ! ! and π π· ! ! = π ( π€ ! !" β ! ! ! ! ) ! ! , respectively. Here, π π₯ is the Heviside function that yields π π₯ = for π₯ > and π π₯ = for π₯ β€ . Notably, this is because the typical networks obtained below are stable; πΌ π· ! ! and π π· ! ! converge to the respective constant values β πΌ π· ! and β π π· ! for sufficiently large π . In the region marked by orange triangles in FIG. 1 ( π = . , . , π = . , πΏ = . , . and π = . , π = . , πΏ = . ), we found the formations of network structures named βPacemaker networkβ at sufficiently large π values, as shown in FIG. 3. In these states, one or a few elements exhibit, π π· ! = π β but most elements exhibit π π· ! = . On the other hand, the πΌ π· ! of all the elements were equal to the number of elements with π π· ! = π β . We called these few elements βpacemaker(s)β because π₯ ! ! they influence the π₯ ! ! ! ! of the other elements effectively, but those of most other elements do not. These states, moreover, { π₯ ! ! } ! show the chaotic motions and π₯ ! ! of some of the elements that are synchronized with pacemakers, but the π₯ ! ! of the elements that are desynchronized with pacemakers also synchronized with each other, as shown in FIG. 3 (c-d). The typical number of pacemakers depends on the parameter set: i) only one pacemaker appears when π = . , π = . , πΏ = . or . ii) two pacemakers appear when π = . , π = . , πΏ = . iii) two to five pacemakers appear depending on the initial conditions when π = . , π = . , πΏ = . , π = . , π = . , πΏ = . , and a = . , c = . , Ξ΄ = . as shown in FIG. 4. Note that in case i), Ξ΄ = is the same as a result reported by Ito and Ohira [21]. In the region marked by yellow circles in FIG. 1 ( π = . βΌ . , π = . , πΏ = . , . , . and π = . , π = . , πΏ = . , . ), we found the formations of the network structures named βpaired layers networkβ were sufficiently large, and π , in which the elements are divided into paired layers, and each element is connected to and from other elements that belong to another layer, is shown in FIG. 5 (a-c). Here, { π₯ ! ! } ! shows the periodic motions, which are divided into two clusters, where π₯ ! ! represents the elements in the same synchronized layer. This is shown in FIG. 5 (c-d). In these states, the indegree and outdegree of each element differ according to πΌ π· ! < π π· ! for some elements but πΌ π· ! β« π π· ! is often seen for others, as shown in FIG. 5 (e-f). In the region highlighted by red diamonds in FIG. 1 ( π = . , π = . , πΏ = . , . ), we found the formations of network structures called the βloop networkβ at sufficiently larger π values, in which the elements were divided into four layers, as shown in FIG. 6 (aβc). Here, the { π₯ ! ! } ! of the elements showed periodic motions, and the π₯ ! ! of the elements in the same layer were synchronized, as shown in FIG. 6 (c-d). In these network structures, the elements in the same layer were commonly connected from all the elements belonging to one of the three other layers, and were commonly connected to all elements belonging to another of two other layers, and then four layers to form a loop. In the region marked by yellow standing triangles in FIG. 1 ( π = . , . , π = . , πΏ = . ), we found network structure formations named βmulti-layers networkβ at a sufficiently large π , in which the elements were divided into more than three layers, and each element was connected to and from other elements that belong to other layers, as shown in FIG. 7 (a-c). Here, { π₯ ! ! } ! the periodic motions and π₯ ! ! of the elements in the same layer were synchronized, as shown in FIG. 7 (c-d). In these cases, the observed networks contained various types of layers such as those consisting of elements with πΌ π· ! < π π· ! , those consisting of elements with πΌ π· ! > π π· ! , and elements with few or zero π π· ! , where four layers form a similar network structure to the loop network. Moreover, such networks also contain multiple hierarchies with the elements in the upper four layers forming a loop and lower layers exhibiting asymmetric influence flows among the elements. In the region highlighted by red stars in FIG. 1 ( π = . , . , π = . , . , πΏ = . , . , . ), we found that distinct small hierarchical networks with upstream and downstream elements had formed at sufficiently large π values, as shown in FIG. 8 (a-c). Here, the { π₯ ! ! } ! of the elements showed chaotic motions, and the π₯ ! ! of the elements were not synchronized even if they belonged to the same networks, as shown in Fig. 8 (d). In these networks, each distinct network resembled a small βpacemaker network,β but the connections of each network were sparser than the abovementioned pacemaker networks. C. Morphology of typical structures of dynamic networks with hidden structures Next, we considered the morphology of the typical states where πΆ π decreases in π but πΆ π > were maintained for Ο β β . In these cases, as mentioned below, we found various temporal changes in the shapes of the connected network. Still, these changes seemed to be restricted to various parameter-dependent hidden structures. To evaluate the structural and dynamic features of the typical networks for each parameter set, we focused on the temporal changes in indegrees and outdegrees, πΌ π· ! ! and π π· ! ! , and long-term indegrees and outdegrees, πΏπΌ π· ! and πΏπ π· ! . The latter was estimated by πΏπΌ π· ! = π ( β¨ π€ ! !" β© β ! ! ! ! ) ! ! and, πΏπ π· ! = π ( β¨ π€ ! !" β© β ! ! ! ! ) ! ! where β¨ w ! !" β© indicates the average π€ ! !" over n = Γ ! βΌ ! . In the region marked by white squares in FIG. 1, ( π = . , . , π = . , πΏ = , . , . , π = . , . , . , . , π = . , πΏ = , . , π = . , π = . , πΏ = and π = . , π = . , πΏ = ), we found that the states exhibited a dynamic network with hidden structures called the βhidden paired layers networkβ at sufficiently large π values, as shown in FIG. 9. Here, a group of elements involving greater π π· ! ! compared to the others appears for the upstream elements in hierarchical networks or pacemaker networks, as mentioned previously, as shown in FIG. 9 (aβc). However, such a network is not stable but transits temporally. In this state, the { π₯ ! ! } ! of the elements exhibited chaotic motions, as shown in FIG. 9 (d). After a certain amount of time, the elements that played upstream element roles with large π π· ! ! replaced the role with other elements, as shown in FIG. 9 (aβc). Here, the lifetime order of the tentative networks was expected to be π to πΆ π as shown by the relaxed constant in FIG. 9 (f). As a result of such temporal connection changes among elements, in the cases of = . , . , π = . , πΏ = , . , . , π = . , . , . , . , π = . , πΏ = , . and π = . , π = . , πΏ = , the long averaged time of the network structures in such cases involved two groups of elements with higher and lower πΏπ π· ! , as shown in FIG. 9 (g-h). Thus, this seems similar to the βpaired layers networkβ introduced previously. On the other hand, in the cases of π = . , π = . , πΏ = , the long averaged time of the network structures in such case exhibited similar to the βpacemaker networkβ also introduced previously, as shown in FIG. 10. In the region marked by the yellow laying triangle in FIG. 1 ( π = . , π = . , πΏ = . ), we found that the states exhibited a dynamic network with hidden structures called the βhidden modular networkβ at sufficiently larger π values, as shown in FIG. 11. Here, most elements were divided into modules consisting of four elements. In each π , the four element forming module sets were strongly connected with each other, and some modules were connected to the elements that did not belong to any module. In contrast, others existed individually, as shown in FIG. 11 (a-c). Here, the π₯ ! ! of the elements exhibited desynchronized chaotic motions even for the same modules, as shown in Fig. 11 (d). However, the construction of each module was unchanged. Notably, elements that did not belong to any modules were connected from all elements on average, as shown in FIG. 11 (g-h), while such connections change temporally. Moreover, as shown in FIG. 12, the order of the standard deviation of each connection was comparable to its average value, and the connections among elements also changed temporally even in the same modules. Still, each set of four elements in each module was unchanged. In the region marked by the white square in FIG. 1 ( π = . , π = . , πΏ = . , . , . ), we found that the states exhibited a dynamic network with hidden structures called the βhidden randomly connected networkβ at sufficiently large π values, as shown in FIG. 13. Here, the { π₯ ! ! } ! of the elements showed chaotic motions, and various complex networks where elements seem to be connected randomly appeared tentatively. The average order of the lifetime of each tentative network structure was estimated by π , to which πΆ π relaxes the constant in FIG. 13 (f). However, some β¨ w ! !" β© remained larger than / ( π β ) . Each element involved five to ten of πΏπΌ π· ! and πΏπ π· ! with, πΏπΌ π· ! ~ πΏπ π· ! as shown in FIG. 13 (g-j), by which the system exhibited hidden network structures with (not sparse but also not dense) random connections among the elements without any pacemaker-like specific elements or significant directionalities. D. Relationships between formed network structures and the dynamics of the elements Then, we focused on the relationship between the formed network structures and dynamics of π₯ ! ! ! . To estimate such relations, we calculated the split exponent (tangential Lyapunov exponent) of the π -th element [27] as π ! = ln β π + ! ! lim ! β β ππ | π β² ( π₯ ! ! ) | ! ! ! ! (5) π π₯ = ππ₯ ( β π₯ ) . (6) We focused on the maximum split exponent π !" = max π ! π = , , β¦ π β } in the following arguments: _ _ !" were measured for π = . , . , . , _ _ _ , and π = , . , . , _ _ _ , . to compare them to the phase diagram of the dynamics of π₯ ! ! ! , as reported in a recent study [21]. Notably, we evaluated the π !" ! using π₯ ! ! ! for n = Γ ! βΌ ! from five different initial conditions, but the results were almost the same, independent of the initial conditions. In the case of Ξ΄ = . the phase diagram of the π and π dependent dynamics of π₯ ! ! ! were reported, where their dynamics were classified as the coherent phase, ordered phase, partially ordered phase, and desynchronized phase [21]. As shown in Fig. 14 (a), the boundary curves among these phases in the phase diagram were closely related to the π !" of the landscape, as follows. The boundary between the coherent phase and the partially ordered phase was obtained as a curve located at π !" = β . βΌ β . for π larger than βΌ . . Furthermore, the boundary between the partially ordered phase and the ordered phase was obtained as a curve, which grew along some contours, and was from π !" βΌ π to between βΌ . and βΌ . . Finally, the boundary between the ordered phase and the desynchronized phase was obtained as a curve, which grew along some contours from π !" βΌ at π < . . FIG. 14 (b) shows the superposition of π !" and the phase diagram of the network structure in Fig. 1, for the case of Ξ΄ = . . By comparing Fig. 13 (a) and (b), we found the following correlations between the formed typical network structures and the dynamics of the elements: most regions that exhibited uniform π€ ! !" (black circles) matched the coherent phase. There were no typical network structures (blue squares) when the dynamics of the elements were partially ordered with larger π than ~ . , while the pacemaker networks (orange triangles) appeared for smaller π . The paired layer networks (yellow circles) and multi-layer networks (yellow standing triangles) were found in cases where the elements showed ordered motions. In the cases where the elements exhibited desynchronized motions, states with distinct hierarchical networks (red stars) appeared in the region near the boundary between the coherent phase and partially ordered phase, and dynamic networks with hidden paired layers network (white squares) were formed. Furthermore, in the region near the boundary between the desynchronized phase and the ordered motions, dynamic networks with hidden randomly connected networks (purple squares) were formed. Similar correlations were formed between typical network structures, and the dynamics of elements are also found in the cases of Ξ΄ = . and . . Figure 15 (a-b) shows π !" for Ξ΄ = . and . where three curves are drawn according to the same criteria as those in FIG. 14 (a), and FIG. 15 (c-d) show the superposition of π !" and the phase diagram of the network structure in FIG. 1 in these cases. We found almost the same relationships between the formed typical network structures and the dynamic properties of the elements, as shown in FIG. 14 (b), except for the following cases. In the region near the boundary between the two regions corresponding to the desynchronized phase and the ordered phase in Fig. 15 (a) and (b), the loop network structure (red diamonds) was observed differently from the case in Ξ΄ = . Furthermore, in the region near the boundary between the two regions corresponding to the partially ordered motions and the ordered phase in Fig. 15 (a), a dynamic network with a hidden modular network (yellow laying triangle) appeared in the case of Ξ΄ = . . IV. DISCUSSION In this study, we considered the behaviors of a simple coupled map system involving temporal changes, which had connections among elements with a time delay proposed by Ito and Ohira [21]. We found that the present system has the potential to exhibit a variety of network structures of connections among elements, either as stable structures or hidden structures. A recently proposed model inspired by human communication employed the rule of connection among elements to emerge as the leader elements (almost the same meaning as the pacemakers in this study), and it exhibited various types of hierarchical network structures [23]. We also found similar hierarchical networks in the present model, although our discussed model did not contain any specific rules to emerge pacemakers explicitly. Since there are no explicit rules for the emergence of pacemakers, the present model exhibited other types of networks, such as loop networks and dynamics networks with hidden modular networks. The present model employs the extension of Hebbβs rule, where the connection from an element to the other at the time tends to be strengthened if the state of the latter at a particular time and that of the former at one time step before are similar. We could regard this rule containing the time delay as one of the simple descriptions of the effects of spike-timing-dependent-plasticity that is observed in neural network systems [28]. Still, we need more detailed qualitative and quantitative considerations. Recently, various dynamical system models with a change in the connections to the simple Hebbβs like rules were studied, and various hierarchical networks were observed. Still, the formed network structures were dependent on the properties of the models [24, 25, 29]. On the other hand, our discussed model in the present arguments showed not only various hierarchical networks but also other types of networks that were not model-dependent but also parameter-dependent. We expect that the time delay in the rule of connection changes plays a key role in creating such various network structure formations. A more detailed parameter-dependent study is still needed that observes the behaviors of the present model in the future since this model is expected to involve further potential, specifically to emerge a richer variety of network structures. We also need to conduct additional studies on the dynamics and functions of neural networks and other biological networks based on the obtained results and modifications of the present model. ACKNOWLEDGMENTS We thank J. Ito for fruitful information provision and H. Nishimori, M. Shiraishi, and H. Takagi for fruitful discussions. This work was supported by JSPS KAKENHI Grant Number 19K20382 (M. F.). TABLE 1. Summary of classification of typical network structures and dynamics of elements. Symbol in FIG. 1 Network structure Stability of network Dynamics of π₯ ! ! Details in Blue β None Unstable Chaotic None Black β Uniform Stable Chaotic None Orange β½ Pacemaker(s) Stable Chaotic FIG. 3, 4 Yellow β Pared layers Stable Periodic FIG. 5 Red β Loop Stable Periodic FIG. 6 Yellow β³ Multi-layers Stable Periodic FIG. 7 Red β Distinct Hierarchical Stable Chaotic FIG. 8 White β Hidden Pared layers Dynamic Chaotic FIG. 9, 10 Yellow β· Hidden Modular Dynamic Chaotic FIG. 11, 12 Purple β Hidden random Dynamic Chaotic FIG. 13
FIG. 1. Phase diagram of the network structures for (a) Ξ΄ = , (b) Ξ΄ = . , and (c) Ξ΄ = . for π = . ~ . , and π = . ~ . . Each symbol represents the mean typical network structures observed in the case of the given parameter set. Details of each symbol are shown in TABLE 1. FIG. 2. (a) Typical temporal evolutions of π₯ ! ! of each element in the present model with a = . , c = . , Ξ΄ = . for π = ~ . (b) Typical autocorrelation function of { π€ ! !" } ! ! ! of each element in the present model with π = . , π = . , πΏ = . . FIG. 3. (a-c) Example of typical network structure, (a) πΌ π· ! , (b) π π· ! , and (c) π₯ ! ! at π = ! of each element in the present model with a = . , c = . , Ξ΄ = . . The network structure named βPacemaker networkβ was observed in this state. Here, the circle with index π indicates the symbol of π -th element, and the color of each indicates the values of (a) πΌ π· ! , (b) π π· ! , and (c) π₯ ! ! . When π€ ! !" > ! ! ! ! at π = ! , the π -th element is connected from π -th element by an arrow. Notably, π€ ! !" > ! ! ! ! for n β₯ ! since the presented network is stable. (d) Typical temporal evolutions of π₯ ! ! for π = ~ , and histograms of (e) πΌ π· ! and (f) π π· ! in the presented state. FIG. 4. Example of typical network structure and π π· ! of Pacemaker networks with (a) two pacemakers for a = . , c = . , Ξ΄ = . and (b) four pacemakers for a = . , c = . , Ξ΄ = . . The meanings of the circles and arrows are the same as those in FIG. 2. FIG. 5. (a-c) Example of a typical network structure, (a) πΌ π· ! , (b) π π· ! , and (c) π₯ ! ! at π = ! of each element in the present model with a = . , c = . , Ξ΄ = . . The network structure called βpared layers networkβ was observed in this state. The meanings of the circles and arrows are the same as those in FIG. 2. (d) Typical temporal evolutions of π₯ ! ! for π = ~ . (e-h) histograms of (e) πΌ π· ! and (f) π π· ! , and (g) π€ ! !" , in the order from small to large π , π , and (h) π€ ! !" for the reordered π , π in the presented state. FIG. 6. (a-c) Example of a typical network structure, (a) πΌ π· ! , (b) π π· ! , and (c) π₯ ! ! at π = ! of each element in the present model with a = . , c = . , Ξ΄ = . . The network structure called the βloop networkβ was observed in this state. The meanings of the circles and arrows are the same as those in FIG. 2. (d) Typical temporal evolutions of π₯ ! ! for π = ~ . (e-h) histograms of (e) πΌ π· ! and (f) π π· ! , and (g) π€ ! !" , in the order from small to large π , π , and (h) π€ ! !" for the reordered π , π in the presented state. FIG. 7. (a-c) Example of typical network structure, (a) πΌ π· ! , (b) π π· ! , and (c) π₯ ! ! at π = ! of each element in the present model with π = . , π = . , πΏ = . . The network structure called βmulti-layers networkβ was observed in this state. The meanings of circles and arrows are the same as those in FIG. 2. (d) Typical temporal evolutions of π₯ ! ! for π = ~ . (e-h) histograms of (e) πΌ π· ! and (f) π π· ! , and (g) π€ ! !" , in the order from small to large π , π , and (h) π€ ! !" for the reordered π , π in the presented state. Left upper region in (h) means that a loop structure exists in the upper stream of this network structure. FIG. 8. (a-c) Example of typical network structure, (a) πΌ π· ! , (b) π π· ! , and (c) π₯ ! ! at π = ! of each element in the present model with a = . , c = . , Ξ΄ = . . In this case, some distinct hierarchical networks with upstream and downstream elements were observed. The meanings of circles and arrows are the same as those in FIG. 2. (d) Typical temporal evolutions of π₯ ! ! for π = ~ and histograms of (e) πΌπ· ! and (f) π π· ! are in the presented state. FIG. 9. (a-c) Example of a typical network structure, (a) πΌπ· ! ! , (b) ππ· ! ! and (c) π₯ ! ! at n = Γ ! and π = ! of each element in the present model with a = . , c = . , Ξ΄ = . Here, the color of each circle indicates the values of (a) πΌπ· ! ! , (b) ππ· ! ! and (c) π₯ ! ! , and π -th element is connected from the π -th element by an arrow when π€ ! !" > ! ! ! ! . (d) Typical temporal evolutions of π₯ ! ! for π = ~ , (e) typical temporal evolutions of ππ· ! ! for π = ~ , and (f) autocorrelation functions of connection strengths, πΆ π , of the presented state. (g-h) Long time-averaged network structure, (g) πΏπΌ π· ! , and (f) πΏπ π· ! of the presented state. Here, the color of each circle indicates the values of (g) πΏπΌ π· ! and (f) πΏπ π· ! . When β¨ w ! !" β© > ! ! -Ββ ! at π = ! , the π -th element is connected from π -th element by an arrow. In this case, a dynamic network with βhidden paired layers networkβ was observed. FIG. 10. Typical long time-averaged network structure, (a) πΏπΌ π· ! , and (b) πΏπ π· ! in the present model with a = . , c = . , Ξ΄ = . . The meanings of the circles and arrows are the same as those in FIG. 9 (g-h). In this case, a dynamic network with βhidden paired layers networkβ involving the similar shape to the pacemaker network was observed. FIG. 11. (a-c) Example of a typical network structure, (a) πΌπ· ! ! , (b) ππ· ! ! , and (c) π₯ ! ! at π = ! of each element in the present model with a = . , c = . , Ξ΄ = . Here, the meanings of circles and arrows are the same as those in FIG. 9 (a-c). (d) Typical temporal evolutions of π₯ ! ! for π = ~ , (e) typical temporal evolutions of ππ· ! ! for π = ~ , and (f) autocorrelation functions of connection strengths πΆ π , of the presented state. (g-h) Long time-averaged network structure, (g) πΏπΌ π· ! , and (f) πΏπ π· ! of the presented state. Here, the meanings of the circles and arrows are the same as those in FIG. 9 (g-h). In this case, a dynamic network with βhidden modular networkβ was observed. FIG. 12. (a) Average and (b) standard deviation of π€ ! !" over n = Γ ! βΌ ! in the case that was introduced in FIG. 10, where the left indicates π€ ! !" , in the order from small to large π , π , and right indicates π€ ! !" for the reordered π , π , which shows the connections among the four elements in each module clearly. FIG. 13. (a-c) Example of typical network structure, (a) πΌπ· ! ! , (b) ππ· ! ! , and (c) π₯ ! ! at π = ! of each element in the present model with a = . , c = . , Ξ΄ = . Here, the meanings of the circles and arrows are the same as those in FIG. 9 (a-c). (d) Typical temporal evolutions of π₯ ! ! for π = ~ , (e) typical temporal evolutions of ππ· ! ! for π = ~ , and (f) autocorrelation functions of connection strengths πΆ π , of the presented state. (g-h) Long time-averaged network structure, (g) πΏπΌ π· ! , (f) πΏπ π· ! , and histograms of (i) πΏπΌ π· ! and (j) πΏπ π· ! of the presented state. Here, the meanings of the circles and arrows are the same as those in FIG. 9 (g-h). In this case, a dynamical network with a βhidden randomly connected networkβ was observed. FIG. 14. (a) Maximum split exponents π !" of π₯ ! ! ! for π and π at πΏ = . . Red curves indicate the boundary of phases of phase diagram of dynamics of π₯ ! ! ! named coherent phase (C), ordered phase (O), partially ordered phase (P), and desynchronized phase (D) reported by recent research [21]. (b) The superposition of π !" and the phase diagram of the network structure was obtained in FIG. 1(a). FIG. 15. (a-b) Maximum split exponents π !" of π₯ ! ! ! for π and π at (a) πΏ = . and (b) πΏ = . . Each red curve indicates the expected one corresponding to that in FIG. 14 (a), respectively. (c-d) Superposition of π !" and the phase diagram of the network structure for (c) πΏ = . and (d) πΏ = . as obtained in FIG. 1(b) and (c), respectively. Appendix (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) ( a ) Ξ΄ = (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) ( b ) Ξ΄ = . (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) FIG. A1. Autocorrelation functions of connection strengths πΆ π , for various π , and π with (a) Ξ΄ = .0, (b) Ξ΄ = . , and (c) Ξ΄ = . . ( c ) Ξ΄ = . (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) - (cid:1) - (cid:1) l og Ο (cid:1) l og C (cid:1) [1] M. Anghel, Z. Toroczkai, K. E. Bassler, and G. Korniss, Phys. Rev. Lett. 92, 4 (2004). [2] N. Guttenberg and N. Goldenfeld, Phys. Rev. E 81, 046111 (2010). [3] C. Furusawa and K. Kaneko, Bull. Math. Biol. 60, 659 (1998). [4] A. Awazu and K. Kaneko, Phys. Rev. E 76, 041915 (2007). [5] A. Awazu and K. Kaneko, Phys. Rev. E 80, 010902 (2009). [6] A. Awazu and K. Kaneko, Phys. Rev. E 80, 041931 (2009). [7] K. Tokita and A. Yasutomi, Theor. Popul. Biol. 63, 131 (2003). [8] T. Shimada, S. Yukawa, and N. Ito, Artif. Life Robot. 6, 78 (2002). [9] J. Ito and K. Kaneko, Neural Networks 13, 275 (2000). [10] J. Ito and K. Kaneko, Phys. Rev. Lett. 88, 287011, (2002). [11] M. Funabashi, Int. J. Bifurc. Chaos 25, 1550054. (2015). [12] W. J. Yuan and C. Zhou, Phys. Rev. E 84, 016116 (2011). [13] J. Jost and K. M. Kolwankar, Phys. A Stat. Mech. Its Appl. 388, 1959 (2009). [14] O. V. Maslennikov and V. I. Nekorkin, Uspekhi Fiz. Nauk 187, 745 (2017). [15] S. Bornholdt and T. RΓΆhl, Phys. Rev. E 67, 066118 (2003). [16] M. Girardi-Schappo, M. H. R. Tragtenberg, and O. Kinouchi, J. Neurosci. Methods 220, 116, (2013). [17] T. Mo, W. Kejun, Z. Jianmin and Z. Liying, Handb. Res. Artif. Immune Syst. Nat. Comput. Appl. Complex Adapt. Technol. 144, 520, (2009). [18] L. Chen and K. Aihara, Neural Networks 8, 915 (1995). [19] M. Adachi and K. Aihara, Neural Networks 10, 83 (1997). [20] Y. Hayakawa and Y. Sawada, Phys. Rev. E 61, 5091 (2000). [21] J. Ito and T. Ohira, Phys. Rev. E 64, 066205 (2001). [22] J. Ito and K. Kaneko, Underst. Complex Syst. 2009, 137 (2009). [23] A. E. Allahverdyan and A. Galstyan, PLoS One 11, 0159301 (2016). [24] V. Botella-Soler and P. Glendinning, Phys. Rev. E 89, 062809 (2014). [25] V. M. EguΓluz, M. G. Zimmermann, C. J. Cela-Conde, and M. San Miguel, Am. J. Sociol. 110, 977 (2005). [26] A. Herz, B. Sulzer, R. KΓΌhn, and J. L. van Hemmen, Biol. Cybern. 60, 457 (1989). [27] K. Kaneko. Physica D 41, 137 (1990) [28] Y. Dan and M. Poo, Neuron 44, 23 (2004). [29] V. Botella-Soler and P. Glendinning, Euro. Phys. Lett. 97, 50004, (2012).[1] M. Anghel, Z. Toroczkai, K. E. Bassler, and G. Korniss, Phys. Rev. Lett. 92, 4 (2004). [2] N. Guttenberg and N. Goldenfeld, Phys. Rev. E 81, 046111 (2010). [3] C. Furusawa and K. Kaneko, Bull. Math. Biol. 60, 659 (1998). [4] A. Awazu and K. Kaneko, Phys. Rev. E 76, 041915 (2007). [5] A. Awazu and K. Kaneko, Phys. Rev. E 80, 010902 (2009). [6] A. Awazu and K. Kaneko, Phys. Rev. E 80, 041931 (2009). [7] K. Tokita and A. Yasutomi, Theor. Popul. Biol. 63, 131 (2003). [8] T. Shimada, S. Yukawa, and N. Ito, Artif. Life Robot. 6, 78 (2002). [9] J. Ito and K. Kaneko, Neural Networks 13, 275 (2000). [10] J. Ito and K. Kaneko, Phys. Rev. Lett. 88, 287011, (2002). [11] M. Funabashi, Int. J. Bifurc. Chaos 25, 1550054. (2015). [12] W. J. Yuan and C. Zhou, Phys. Rev. E 84, 016116 (2011). [13] J. Jost and K. M. Kolwankar, Phys. A Stat. Mech. Its Appl. 388, 1959 (2009). [14] O. V. Maslennikov and V. I. Nekorkin, Uspekhi Fiz. Nauk 187, 745 (2017). [15] S. Bornholdt and T. RΓΆhl, Phys. Rev. E 67, 066118 (2003). [16] M. Girardi-Schappo, M. H. R. Tragtenberg, and O. Kinouchi, J. Neurosci. Methods 220, 116, (2013). [17] T. Mo, W. Kejun, Z. Jianmin and Z. Liying, Handb. Res. Artif. Immune Syst. Nat. Comput. Appl. Complex Adapt. Technol. 144, 520, (2009). [18] L. Chen and K. Aihara, Neural Networks 8, 915 (1995). [19] M. Adachi and K. Aihara, Neural Networks 10, 83 (1997). [20] Y. Hayakawa and Y. Sawada, Phys. Rev. E 61, 5091 (2000). [21] J. Ito and T. Ohira, Phys. Rev. E 64, 066205 (2001). [22] J. Ito and K. Kaneko, Underst. Complex Syst. 2009, 137 (2009). [23] A. E. Allahverdyan and A. Galstyan, PLoS One 11, 0159301 (2016). [24] V. Botella-Soler and P. Glendinning, Phys. Rev. E 89, 062809 (2014). [25] V. M. EguΓluz, M. G. Zimmermann, C. J. Cela-Conde, and M. San Miguel, Am. J. Sociol. 110, 977 (2005). [26] A. Herz, B. Sulzer, R. KΓΌhn, and J. L. van Hemmen, Biol. Cybern. 60, 457 (1989). [27] K. Kaneko. Physica D 41, 137 (1990) [28] Y. Dan and M. Poo, Neuron 44, 23 (2004). [29] V. Botella-Soler and P. Glendinning, Euro. Phys. Lett. 97, 50004, (2012).