Nerve theorems for fixed points of neural networks
Felicia Burtscher, Carina Curto, Stefania Ebli, Daniela Egas Santander, Katherine Morrison, Alice Patania, Nicole Sanderson
NNerve theorems for fixed points of neural networks
Felicia Burtscher, Carina Curto, Stefania Ebli, Daniela Egas Santander, Katherine Morrison, Alice Patania,Nicole Sanderson
Abstract
Nonlinear network dynamics are notoriously difficult to understand. Here we study a class ofrecurrent neural networks called combinatorial threshold-linear networks (CTLNs) whose dynamics are de-termined by the structure of a directed graph. They are a special case of TLNs, a popular framework formodeling neural activity in computational neuroscience. In prior work, CTLNs were found to be surprisinglytractable mathematically. For small networks, the fixed points of the network dynamics can often be com-pletely determined via a series of graph rules that can be applied directly to the underlying graph. For largernetworks, it remains a challenge to understand how the global structure of the network interacts with localproperties. In this work, we propose a method of covering graphs of CTLNs with a set of smaller directionalgraphs that reflect the local flow of activity. While directional graphs may or may not have a feedforwardarchitecture, their fixed point structure is indicative of feedforward dynamics. The combinatorial structureof the graph cover is captured by the nerve of the cover. The nerve is a smaller, simpler graph that ismore amenable to graphical analysis. We present three nerve theorems that provide strong constraints onthe fixed points of the underlying network from the structure of the nerve. We then illustrate the power ofthese theorems with some examples. Remarkably, we find that the nerve not only constrains the fixed pointsof CTLNs, but also gives insight into the transient and asymptotic dynamics. This is because the flow ofactivity in the network tends to follow the edges of the nerve.
Felicia BurtscherUniversit´e du Luxembourg, Belvaux, Luxembourg, e-mail: [email protected]
Carina CurtoPennsylvania State University, University Park, PA, USA, e-mail: [email protected]
Stefania Ebli´Ecole Polytechnique F´ed´erale de Lausanne, Lausanne, Switzerland, e-mail: [email protected]
Daniela Egas Santander´Ecole Polytechnique F´ed´erale de Lausanne, Lausanne, Switzerland, e-mail: [email protected]
Katherine MorrisonUniversity of Northern Colorado, Greeley, CO, USA, e-mail: [email protected]
Alice PataniaIndiana University Network Science Institute (IUNI), Bloomington, IN, USA, e-mail: [email protected]
Nicole SandersonLawrence Berkeley National Lab, Berkeley, CA, USA, e-mail: [email protected] a r X i v : . [ q - b i o . N C ] F e b F. Burtscher, C. Curto, S. Ebli, D. Egas Santander, K. Morrison, A. Patania, N. Sanderson
Combinatorial threshold-linear networks (CTLNs) are a special class of threshold-linear networks (TLNs)whose dynamics are determined by the structure of a directed graph. The firing rates x ( t ) , . . . , x n ( t ) of n recurrently-connected neurons evolve in time according to the standard TLN equations: dx i dt = − x i + n (cid:88) j =1 W ij x j + θ i + , i = 1 , . . . , n. (1)These networks derive their name from the nonlinear transfer function, [ · ] + = max { , ·} , which is threshold-linear. A given TLN is specified by the choice of a connection strength matrix W and a vector of externalinputs θ . TLNs have been widely used in computational neuroscience as a framework for modeling recurrentneural networks, including associative memory networks [16, 10, 11, 17, 5, 9, 3].What makes CTLNs special is that the matrix W = W ( G, ε, δ ) is determined by a simple directed graph,as follows: W ij = i = j, − ε if j → i in G, − − δ if j (cid:54)→ i in G. (2)Additionally, we fix θ i = θ > ε, δ parameters to satisfy δ >
0, and 0 < ε < δδ +1 . CTLNs were first defined in [15], where the ε < δδ +1 condition was motivatedby the desired property that subgraphs consisting of a single directed edge i → j should not be allowed tosupport stable fixed points. Note that the upper bound on ε implies ε <
1, rendering the W matrix effectivelyinhibitory. We think of the graph edges as excitatory connections in a sea of inhibition (Figure 1A). Figure 1Cshows an example solution for a CTLN whose graph is a 3-cycle. f i r i n g r a t e Fig. 1 (A) A neural network with excitatory pyramidal neurons (triangles) and a background network of inhibitory interneurons(gray circles) that produces a global inhibition. The corresponding graph (right) retains only the excitatory neurons and theirconnections. (B) TLN dynamics. (C) A graph that is a 3-cycle (left), and a solution for the corresponding CTLN showingthat network activity follows the arrows in the graph (right). Peak activity occurs sequentially in the cyclic order 123. Unlessotherwise noted, all simulations have parameters ε = 0 . , δ = 0 . , and θ = 1. TLNs are high-dimensional nonlinear systems whose dynamics are still poorly understood. However, inthe special case of CTLNs, there appears to be a strong connection between the attractors of the network andthe pattern of stable and unstable fixed points [14]. Moreover, these fixed points can often be completelydetermined by the structure of the underlying graph. In prior work, a series of graph rules were proven thatcan be used to determine fixed points of the CTLN by analyzing G , irrespective of the choice of parameters ε, δ, and θ [7, 6]. A key observation is that for a given network, there can be at most one fixed point persupport, σ ⊆ [ n ], where the support of a fixed point is the subset of active neurons (i.e., supp x = { i | x i > } ). A graph is simple if it does not have loops or multiple edges between a pair of vertices. A fixed point, x ∗ , of a TLN is a solution that satisfies dx i /dt | x ∗ = 0 for each i ∈ [ n ].erve theorems for neural networks 3 For a given choice of parameters, we use the notationFP( G ) def = { σ ⊆ [ n ] | σ is a fixed point support of W ( G, ε, δ ) } , where [ n ] def = { , . . . , n } . For many graphs, we find that the fixed point supports in FP( G ) are confined toa subset of the neurons. In other words, there is a partition { ω, τ } of the vertices of G such that, for every σ ∈ FP( G ), we have σ ⊆ τ . In these cases, we observe that solutions of the network activity x ( t ) tend toconverge to a region of the state space where the most active neurons are in τ , and those in ω are eithersilent or have very low firing. In other words, the attractors live where the fixed points live.This motivates us to define directional graphs . A directional graph G is a graph with a proper subset ofneurons τ such that FP( G ) ⊆ FP( G | τ ), where G | τ is the induced subgraph obtained by restricting to thevertices of τ . We also require an additional technical condition that allows us to prove that certain naturalcompositions, like chaining directional graphs together, produce a new directional graph (see Definition 3.1for the full definition). In simulations, we have seen that directional graphs display feedforward dynamics,even if their architecture does not follow a feedforward structure. Activity that is initially concentrated on ω flows towards τ , giving the dynamics an ω → τ directionality. Thus, from a bird’s eye view, directionalgraphs behave like a single directed edge, where the activity flows from the source to the sink. This promptedus to ask the following question: if we cover a graph G with a collection of directional graphs, what can wesay about FP( G ) from the combinatorial structure of the cover?In this paper we develop tools to answer this question, inspired by the construction of the nerve of acover of a topological space. We define a directional cover of G as a set of directional subgraphs that cover G and have well-behaved intersections. Effectively, such a cover is entirely determined by a partition ofthe vertices of G , denoted { ν i } , that satisfies special properties. We define the nerve of a directional coveras a new graph N that has a directed edge for each directional graph in the cover, and a vertex for eachcomponent ν i of the partition. Such a partition always has a canonical quotient map, π : V G → V N := { ν i } , that simply identifies all the vertices in each component ν i . The edges of the nerve reflect the local dynamicsof G . Moreover, the nerve encodes the combinatorics of the intersection pattern of the cover: the directionalgraphs overlap precisely at vertices of the nerve where their corresponding edges meet. (See Definition 4.3for a precise definition.)As an illustration of directional covers and nerves, consider the graph in Figure 2A. This graph is a chainof ten 5-cliques where the edges between adjacent cliques all follow the pattern shown in panel C: thereare edges forward from every node in the first clique to every node in the second clique; every node in thesecond clique (except for the top node) sends edges back to every node in the first clique. Most edges arethus bidirectional arrows (in black), and the edges that only go forward from clique i to clique i+1 are incolor. The induced subgraphs G | ν i ∪ ν i +1 are all directional with direction ν i → ν i +1 , despite all the back edgesfrom right to left. This means that FP( G | ν i ∪ ν i +1 ) ⊆ FP( G | ν i +1 ), and we expect the activity of the neuronsto flow from ν i to ν i +1 (left to right). Figure 2B depicts the nerve of G . Figure 2D shows the solution to aCTLN defined by G , where we have initialized all the activity on the nodes in the first clique ν . We seethat the activity eventually converges to the final component, shown in purple, where the fixed points ofthe network are concentrated. The transient dynamics, however, are rather slow, with each clique activatedin a sequence that follows the path-like structure of the nerve. Note that this network behaves similarly toa synfire chain [1, 2, 12], despite numerous backward edges between components that completely destroythe feedforward architecture (Figure 2C). The sequential dynamics are maintained because these backwardedges do not disrupt the directionality of the graphs in the cover.The main goal of this paper is to prove nerve theorems for CTLNs. Broadly speaking, such a nerve theorem is a result that gives information about the dynamics of a network from properties of the nerve. Specifically,we are interested in results that allow us to constrain the fixed points of G by analyzing structural propertiesof N . Ideally, we’d like to prove the following kind of result: If G has a directional cover with nerve N , then σ ∈ FP( G ) ⇒ π ( σ ) ∈ FP( N ) , (3) This is analogous to the definition of a “good” cover of a topological space, which also requires well-behaved intersections.Nerves of good covers reflect the topology of the underlying space [4, 13]. F. Burtscher, C. Curto, S. Ebli, D. Egas Santander, K. Morrison, A. Patania, N. Sanderson
Fig. 2 Example graph with a directional cover, its nerve, and network activity that flows along the nerve. (A)A chain of ten 5-cliques where the edges between adjacent cliques all follow the pattern shown in panel C. (B) The nerve of thegraph in A induced by the partition of the vertices as ν , . . . , ν . (C) The graph G restricted to a pair of adjacent cliques. Alledges in black are bidirectional, while those in green are unidirectional from the green clique to the pink clique. This restrictedgraph is directional, and all the graphs in the directional cover of G have this form. (D) A solution to the CTLN defined by G (with ε = 0 .
25 and δ = 0 . ν . The transient dynamics slowlyactivate each clique in sequence, following the path of the nerve, until the solution converges to the stable fixed point supportedon the nodes in ν . where π is the canonical quotient map from V G to V N . This can be quite powerful in cases where N is amuch smaller and simpler graph. Unfortunately, the statement (3) is not in general true. However, we do findthat this holds whenever the nerve N is a directed acyclic graph (DAG) or N is a cycle (see Theorems 4.9and 4.10). More generally, whenever N admits a DAG decomposition (see Definition 2.16), Theorem 4.8gives a result similar in spirit to (3) and allows us to greatly constrain FP( G ).Our nerve theorems can be used to simplify a complex network by finding a nontrivial directional cover andstudying its nerve. Finding such covers is an art, however, and we do not yet have a systematic way of doing it.On the other hand, nerve theorems can also be used to engineer complex networks with prescribed dynamicproperties. This is how we constructed the example in Figure 2. We explore both kinds of applications inthe last section of the paper.The organization of this paper is as follows. In Section 2, we review some graph theory terminologyand basic background and notation for CTLNs. We also introduce the DAG decomposition of a graph.In Section 3, we define directional graphs, prove that certain graph structures are always directional, andprovide some other families of examples. In Section 4, we introduce directional covers and their associatednerves. Here we also state and prove our main results, Theorems 4.8, 4.9 and 4.10. Finally, in Section 5, weillustrate the power of our theorems with some applications. In this section we review some useful terminology from graph theory and summarize essential backgroundand prior results about fixed points of CTLNs. We also introduce the DAG decomposition of a graph, anotion that will appear in our main nerve theorems. erve theorems for neural networks 5
Definition 2.1 A directed graph G can be described as a tuple G = ( V G , E G ), where V G is a finite set calledthe set of vertices and E G ⊆ V G × V G is the the set of (directed) edges , where ( i, j ) ∈ E G means there is adirected edge i → j from i to j in G . If ( i, j ) / ∈ E G , we write i (cid:54)→ j . A directed graph is simple if it has noself-loops, so that ( i, i ) / ∈ E G for all i ∈ V G . A directed graph is oriented if has no bidirectional edges.In this paper, we restrict ourselves to simple directed graphs. Unless otherwise noted, we will use the word graph to refer to simple directed graphs. Notation 2.2
Let G be a graph with vertex set V G and edge set E G . For any subset of vertices σ ⊆ V G ,denote by G | σ the induced subgraph obtained by restricting to the vertices σ . More precisely, G | σ = ( σ, E | σ )where E | σ = { ( i, j ) ∈ E G | i, j ∈ σ } .Let σ , σ ⊆ V G be two subsets of the vertices of G . We denote by E G ( σ , σ ) ⊆ E G the set of directededges from vertices in σ to vertices in σ in G .Next, we define some basic notions relevant to graphs. Definition 2.3
Let G be a graph and v ∈ V G be a vertex in G . The in-degree of v is the number of incomingedges to v . The out-degree of v is the number of outgoing edges from v . We say v is a source if v has noincoming edges, and we say v is a proper source if it is a source that has at least one outgoing edge. We say v is a sink if v has no outgoing edges. Note that a source that is not a proper source is an isolated vertex ,and thus it is also a sink. Definition 2.4
We say that a graph G has uniform in-degree if every vertex v ∈ V G has the same in-degree d . Note that an independent set is a graph with uniform in-degree d = 0. A k -clique is an all-to-allbidirectionally connected graph with uniform in-degree d = k −
1. And an n -cycle is a graph with n edges,1 → → · · · → n →
1, which has uniform in-degree d = 1. In this subsection we recall the results from [7] that are relevant for this work and include simple proofs tosome of these to provide intuition to the reader.A fixed point of a CTLN is simply a fixed point of the network equations (1). In other words, it is a vector x ∗ ∈ R n ≥ such that dx i dt | x = x ∗ = 0 for all i ∈ [ n ]. As explained in [7], fixed points of CTLNs can be labelledby their supports (i.e. the subset of active neurons), and for a given G the set of all fixed point supports isdenoted FP( G ). Lemma 2.5 ([7])
Let G be a graph on n vertices, and suppose G has uniform in-degree. Then G has afull-support fixed point, σ = [ n ] ∈ FP( G ).In particular, this lemma says that cliques, cycles, and independent sets all have a full-support fixed point.In fact, this fixed point is symmetric, with x ∗ i = x ∗ j for all i, j ∈ [ n ]. This is true even for uniform in-degreegraphs that are not symmetric.More generally, fixed points can have very different values across neurons. However, there is some level of“graphical balance” that is required of G | σ for any fixed point support σ . For example, if σ contains a pairof neurons j, k that have the property that all neurons mapping to j are also mapping to k , and j → k but k (cid:54)→ j , then σ cannot be a fixed point support. This is because k is receiving strictly more inputs than j , andthis imbalance rules out their ability to coexist in the same fixed point support. To see this more rigorously,we have the following lemma. Lemma 2.6
Let G be a CTLN and σ ⊆ V G . Suppose there exist vertices j, k ∈ σ such that for each i ∈ σ \ { j, k } , if i → j then i → k . Furthermore, suppose j → k but k (cid:54)→ j . Then σ / ∈ FP( G ). F. Burtscher, C. Curto, S. Ebli, D. Egas Santander, K. Morrison, A. Patania, N. Sanderson
Proof
To obtain a contradiction, assume σ ∈ FP( G ). The corresponding fixed point x satisfies x i > i ∈ σ , and dx i /dt = 0. In particular, setting dx j /dt = 0 and dx k /dt = 0 (and recalling W jj = W kk = 0)we obtain: x j = (cid:88) i ∈ σ \{ j,k } W ji x i + W jk x k + θ,x k = (cid:88) i ∈ σ \{ j,k } W ki x i + W kj x j + θ. Now observe that for each i ∈ σ \ { j, k } , the fact that i → j implies i → k tells us that W ji ≤ W ki . Thismeans the summation term in the x j equation above is less than or equal to the analogous term in the x k equation. Using this fact, we see that x j − W jk x k ≤ x k − W kj x j , which can be rearranged as,(1 + W kj ) x j ≤ (1 + W jk ) x k . Now recall that j → k but k (cid:54)→ j , so W kj = − ε and W jk = − − δ . The above inequality thus says that εx j ≤ − δx k . But this is a contradiction, because εx j > − δx k <
0. And so no fixed point supportedon σ can exist. (cid:3) The conditions on j, k ∈ σ used in the above lemma is an example of so-called graphical domination . Thisnotion was first defined in [7], and provides a useful tool for ruling in and ruling out fixed points of CTLNspurely based on the graph structure, and independently of the ε, δ and θ parameters. Definition 2.7 (graphical domination)
Let G be a graph, σ ⊆ V G a subset of the vertices, and j, k ∈ V G such that { j, k } ∩ σ (cid:54) = ∅ . We say that k graphically dominates j with respect to σ , and write k > σ j , if thefollowing three conditions hold:1. For all i ∈ σ \ { j, k } if i → j , then i → k .2. If j ∈ σ , then j → k .3. If k ∈ σ , then k (cid:57) j .This definition of domination covers more cases than what we saw in Lemma 2.6. This greater generalityis reflected in the main theorem about domination, which appeared as Theorem 4 in [7]. We cite a specialcase of this theorem below. Theorem 2.8 (graphical domination [7])
Let σ ⊆ V G be a subset of the vertices of a graph G . If thereis a j ∈ σ and a k ∈ V G such that k > σ j ( k graphically dominates j with respect to σ ), then σ / ∈ FP( G ). Fig. 3 The two cases of graphical domination in Theorem 2.8.
In each panel, k graphically dominates j with respect to σ (the outermost shaded region). The inner shaded regions illustrate the subsets of nodes that send edges to j and k . Note thatthe vertices sending edges to j are a subset of those sending edges to k , but this containment need not be strict. The dashedarrow indicates an optional edge between j and k . We will furthermore use the following useful equivalence, which states that σ can only be a fixed pointsupport if σ ∈ FP( G | σ ) and the fixed point survives the addition of each individual k / ∈ σ. erve theorems for neural networks 7 Lemma 2.9 ([7, Corollary 2])
Consider a CTLN determined by a graph G on a set of neurons V G , andlet σ ⊆ V G . Then σ ∈ FP( G ) ⇔ σ ∈ FP( G | σ ∪{ k } ) for all k ∈ V G . In particular, σ ∈ FP( G ) ⇒ σ ∈ FP( G | σ ). Moreover, σ ∈ FP( G ) ⇒ σ ∈ FP( G | τ ) for any τ with σ ⊆ τ .One simple case where graphical domination can be used to rule out a fixed point support is whenever agraph contains a proper source. This is Rule 6 in [7]. Lemma 2.10 (sources [7])
Let G be a graph and σ ⊆ V G . If there exists a j ∈ σ such that j is a propersource in G | σ or j is a proper source in G | σ ∪{ (cid:96) } for some (cid:96) ∈ V G , then σ / ∈ FP( G ). Proof If j is a proper source in G | σ , then there exists k ∈ σ such that j → k . Since j has no other inputs in σ , clearly k > σ j. If j is not a proper source in G | σ but is a proper source in G | σ ∪ (cid:96) , then j → (cid:96) , and hence (cid:96) > σ j . In either case, by Theorem 2.8 we have that σ / ∈ FP( G ). (cid:3) The lemma above allows us to rule out fixed points of cycles that are not full support.
Lemma 2.11 (cycles) If G is a cycle on n vertices, then G has a unique fixed point, which has full support.In other words, FP( G ) = { [ n ] } . Proof
First observe that [ n ] ∈ FP( G ) by Lemma 2.5 because a cycle has uniform in-degree 1. To see thatthis is the only fixed point support of G , consider any proper subset σ (cid:40) V G . Since G is a cycle, G | σ eithercontains a path or is an independent set. If it contains a path, then the source of that path is a proper sourcein G | σ . If it is an independent set, then for any i ∈ σ , we have i → (cid:96) in G for (cid:96) = i + 1. Then i is a propersource in G | σ ∪ (cid:96) . Thus by Lemma 2.10, σ / ∈ FP( G ). Thus, FP( G ) = { [ n ] } . (cid:3) Another simple case when graphical domination can be used to rule out fixed points is whenever σ has atarget in G . For k ∈ V G , we say that k is a target of σ if i → k for every i ∈ σ \ { k } . Lemma 2.12 (targets [7])
Let G be a graph and σ ⊆ V G . Suppose k ∈ V G is a target of σ .1. If k ∈ V G \ σ , then σ / ∈ FP( G ).2. If k ∈ σ and there exists a j ∈ σ such that k (cid:54)→ j , then σ / ∈ FP( G ). Proof
In case 1, it is straightforward to see that k > σ j for any j ∈ σ . In case 2, we see that for theparticular j such that k (cid:54)→ j , we have k > σ j . In either case, by Theorem 2.8 we have that σ / ∈ FP( G ). (cid:3) The target lemma allows us to rule out fixed points of cliques that are not full support.
Lemma 2.13 (cliques) If G is a clique on n vertices, then G has a unique fixed point, which has fullsupport. In other words, FP( G ) = { [ n ] } . Proof
First observe that [ n ] ∈ FP( G ) by Lemma 2.5 because a clique has uniform in-degree n −
1. To seethat this is the only fixed point support of G , consider any proper subset σ (cid:40) V G and let k ∈ V G \ σ . Then k is a target of σ , and so by Lemma 2.12, σ / ∈ FP( G ). Thus, FP( G ) = { [ n ] } . (cid:3) Finally, using a more general form of domination defined in [7], we obtain the following survival rule tellingus precisely when a uniform in-degree fixed point survives as a fixed point of a larger network (Theorem 5of [7]):
Theorem 2.14 (uniform in-degree [7])
Let G be a graph and σ ⊆ V G such that G | σ has uniform in-degree d . For k ∈ V G \ σ , let d k def = |{ i ∈ σ | i → k }| be the number of edges k receives from σ . Then σ ∈ FP( G ) ⇔ d k ≤ d for every k ∈ V G \ σ. Other than this theorem we will not use the more general form of domination. Therefore, in the remainderof this work when we say domination we mean graphical domination.
F. Burtscher, C. Curto, S. Ebli, D. Egas Santander, K. Morrison, A. Patania, N. Sanderson
Two of our main results are nerve lemmas involving directed acyclic graphs (DAGs). Recall that a DAG is agraph that has no directed cycles. There is a well known characterization of DAGs in terms of a topologicalordering of their vertices. In particular, G is a DAG if and only if there exists an ordering of the verticessuch that edges in G only go from lower numbered to higher numbered vertices. In other words, if i → j then i < j , and if i > j then i (cid:54)→ j . Lemma 2.15 (DAGs)
Let G be a DAG and let τ = { sinks of G } . Then the fixed point supports of G areall the nonempty subsets of τ , i.e. FP( G ) = P ( τ ) , where P ( τ ) denotes the power set of τ . Proof
First to see that P ( τ ) ⊆ FP( G ), notice that any subset of τ is an independent set of sinks. Anindependent set has uniform in-degree 0, and thus by Theorem 2.14, an independent set produces a fixedpoint when it has no outgoing edges. Since all the nodes in τ are sinks, every subset of τ has no outgoingedges, and so every subset produces a fixed point support in FP( G ).Next, to see that no other sets can produce fixed points of G , consider σ ⊆ V G such that σ (cid:54)⊆ τ . Let j bethe lowest number vertex in σ \ τ according to some topological ordering of G . Then j has no incoming edgesfrom other nodes in σ since edges in a DAG can only go from lower numbered vertices to higher numbervertices. Moreover, there exists some (cid:96) ∈ V G such that j → (cid:96) since otherwise j would be a sink, but j / ∈ τ by design, which contains all the sinks of G . Thus j is a proper source in G | σ ∪ (cid:96) , and so σ / ∈ FP( G ) byLemma 2.10. (cid:3) Many graphs that are not DAGs nevertheless have a DAG-like structure on a subgraph. This will also bea useful concept for our nerve theorems.
Definition 2.16 (DAG decomposition)
Let G be a graph. For ω, τ ⊆ V G , we say that ( ω, τ ) is a DAGdecomposition of G if ω ˙ ∪ τ is a partition of the vertices V G such that:1. G | ω is a DAG,2. G | τ contains all sinks of G ,3. there are no edges from τ back to ω , i.e., E G ( τ, ω ) = ∅ .We say a DAG decomposition is non-trivial if ω (cid:54) = ∅ . We say a DAG decomposition is maximal if ω is as largeas possible. More precisely, ( ω, τ ) is a maximal DAG decomposition if there is no other DAG decomposition( ω (cid:48) , τ (cid:48) ) with ω (cid:40) ω (cid:48) .Every graph G that has at least one proper source j has a DAG decomposition with ω = { j } and τ = V G \ { j } . But DAG decompositions are most valuable when τ is as small as possible. To minimizethe size of τ , we’d like to “grow” ω as much as possible, as in a maximal DAG decomposition. It turnsout that there is straight-forward procedure for generating a maximal DAG decomposition of a graph, andmoreover, the maximal DAG decomposition is in fact unique. Specifically, one can iteratively refine a DAGdecomposition by moving any nodes that are proper sources in G | τ to ω (see Figure 4). This process willmaintain the property that G | ω is a DAG (each node that is moved to ω will be at the end of the “topologicalordering” of the DAG) while also guaranteeing that there are no edges from nodes in τ back to nodes in ω .Finally, the process terminates when there are no nodes in τ that are proper sources in G | τ . It turns out thatthe τ satisfying G | τ has no proper sources is both minimal , in the sense that | τ | is smallest and τ ⊆ τ (cid:48) forany other DAG decomposition ( ω (cid:48) , τ (cid:48) ), and unique . As a result, this process yields the unique maximal DAGdecomposition. Note in particular that in any maximal DAG decomposition τ can not have proper sources,because if it did one could move such a vertex to ω , contradicting maximality. Lemma 2.17
Suppose that G contains a proper source. Then the DAG decomposition ( ω, τ ) of G satisfying G | τ has no proper sources is a maximal DAG decomposition. In particular, G has a unique maximal DAGdecomposition. erve theorems for neural networks 9 source
25 4 A B C
12 6
Fig. 4
Iterative construction of DAG decompositions. (A) A DAG decomposition of a graph where ω contains only a singlesource. The gray highlighted node 2 is a proper source in G | τ , but not a source in the full graph. (B) A second DAG decompositionis obtained by moving node 2 to ω . Now node 5 has become a proper source in the new G | τ . (C) A third DAG decompositionis obtained by moving 5 to ω . In this decomposition, G | τ has no proper sources. Notice that node 7 is a source, but becauseit has no outgoing edges, it is not a proper source so will not be moved to ω . In fact, node 7 is a sink, and thus is required tobe in τ by condition 2 of DAG decompositions. We have thus arrived at the unique DAG decomposition with minimal τ andmaximal DAG ω . Proof
Let ( ω, τ ) be a DAG decomposition of G satisfying G | τ has no proper sources, and let ( ω (cid:48) , τ (cid:48) ) be anyother DAG decomposition of G . Suppose ω (cid:48) (cid:54)⊆ ω . Then since each DAG decomposition is a partition of thevertices, this condition on ω implies that τ (cid:54)⊆ τ (cid:48) . Then there exists a node i ∈ τ \ τ (cid:48) . Since i ∈ τ and G | τ has no proper sources, there exists some i ∈ τ such that i → i . Since i also is not a proper source in G | τ ,there exists some i ∈ τ such that i → i → i . Again i is not a proper source, and so there exists i ∈ τ such that i → i → i → i .Note that in the other DAG decomposition ( ω (cid:48) , τ (cid:48) ), since i / ∈ τ (cid:48) , we must have i ∈ ω (cid:48) . Moreover, by thedefinition of DAG decomposition, there are no edges from nodes in τ (cid:48) to nodes in ω (cid:48) , and so all nodes inthe path i → i → i → i must also be in ω (cid:48) . We can continue to trace the path backwards in this waythrough G | τ since it has no proper sources, but since τ is finite, at some point some node must appear twicein this path. Thus this sequence of nodes must contain a bidirectional edges and/or a directed cycle. But allthe nodes in this sequence must be in ω (cid:48) , and by definition, G | ω (cid:48) must be a DAG, thus it cannot contain anybidirectional edges or directed cycles. Thus we have a contradiction, and so τ ⊆ τ (cid:48) , and thus ω (cid:48) ⊆ ω .Since any DAG decomposition ( ω, τ ) of G satisfying G | τ has no proper sources must have maximal ω , itfollows that there must be a unique decomposition satisfying this property. Finally, since any maximal DAGdecomposition must satisfy this property, it follows there is a unique one and it is this one. (cid:3) In this section, we focus on a special class of graphs known as directional graphs , first defined in [8]. Themotivating heuristic behind directional graphs is that they are graphs whose vertices can be partitioned intotwo sets ω and τ such that when the neural activity is initialized on nodes in ω , it flows to the nodes in τ .Computationally, we have seen that this flow of activity occurs whenever the fixed points of G are confinedto live in τ , and thus FP( G ) ⊆ FP( G | τ ) by Lemma 2.9. In order to guarantee nice properties when we uniontogether directional graphs, we require something slightly stronger in our definition of directional graphs,namely that the collapse of the fixed points onto the subnetwork G | τ be the result of graphical domination. Definition 3.1 (directional graph)
We say that a graph G is directional, with direction ω → τ , if ω ˙ ∪ τ = V G is a nontrivial partition of the vertices ( ω, τ (cid:54) = ∅ , ω ∩ τ = ∅ ) such that FP( G ) ⊆ FP( G | τ ) by way ofgraphical domination. Specifically, we require the following property: for every σ (cid:54)⊆ τ , there exists some j ∈ σ ∩ ω and k ∈ V G such that k graphically dominates j with respect to σ , i.e. k > σ j . When this is thecase we say σ dies by (graphical) domination. As mentioned above, we predict that directional graphs will have feedforward dynamics , so that activitythat is initially concentrated on G | ω should flow towards G | τ , giving the dynamics an ω → τ directionality.The most natural examples of directional graphs are those where G has an explicit feedforward architecturein G | ω , for example when G | ω is a DAG, and there are no edges from τ back to ω . In this case, it seemsintuitive that the dynamics will flow along this feedforward structure in ω and end up concentrated in τ . It turns out that any DAG decomposition of a graph G immediately yields a directional partition asintuitively predicted. Lemma 3.2
If ( ω, τ ) is a DAG decomposition of G , then G is directional with direction ω → τ .The key to the proof of Lemma 3.2 is the well known characterization of DAGs in terms of a topologicalordering of their vertices. Recall that G is a DAG if and only if there exists an ordering of the vertices suchthat edges in G only go from lower numbered to higher numbered vertices, i.e., if i > j , then i (cid:54)→ j . Proof
To show that G is directional, we must show that any σ ⊆ V G that intersects ω dies by graphicaldomination. Suppose σ ∩ ω (cid:54) = ∅ , and let j be the lowest numbered vertex in σ ∩ ω with respect to sometopological ordering of the DAG G | ω . Since all the sinks in G are contained in τ , j ∈ ω must have at leastone outgoing edge in G , so j → k for some k ∈ V G . Then j is a proper source in G | σ ∪ k and by Lemma 2.10, σ / ∈ FP( G ) because k > σ j . Thus every σ with σ ∩ ω (cid:54) = ∅ dies by graphical domination, and so G is directionalwith direction ω → τ . (cid:3) Fig. 5 Three types of directional graphs. (A) A nontrivial DAG decomposition ( ω, τ ) is a directional graph with direction ω → τ . (B) If τ contains a target node of ω , and there are no back edges τ → ω , then G is directional irrespective of thestructure of G | ω . (C) A more general directional graph can have a variety of forward and backward edges. DAG decompositions are a very special case of directional graphs ω → τ where there are no back edgesfrom τ to ω , and the ω component of the graph is a DAG. Neither condition needs to hold for more generaldirectional graphs. Figure 5 shows several types of directional graphs. In panel A, there are only edges from ω → τ as in a DAG decomposition. In panel B, the existence of a target in τ that receives edges from allnodes in ω guarantees that G is directional irrespective of the structure of G | ω . Finally, in panel C we see aschematic of a directional graph with both forward edges from ω to τ and backward edges from τ to ω .In fact, directional graphs can have a surprisingly large number of back edges while still preserving their“forward” directionality. All the graphs in Figure 6A are directional with ω → τ , and each of the graphsin A3 – A6 actually have as many back edges from τ to ω as they do forward edges. The dynamics for A3and A6 are shown on the right, and we see that even if we initialize the activity purely on nodes in ω , theactivity flows ω → τ as predicted by the directionality. Note that none of the graphs in panel A has a propersource, and thus none has a nontrivial DAG decomposition.Panel B in Figure 6 shows some example graphs that are not directional for any partition of the vertices.This is because every vertex is involved in at least one fixed point support, so there cannot be a collapseFP( G ) ⊆ FP( G | τ ) for any τ (cid:40) V G . The graph in B2 is particularly surprising since it only has edges forwardfrom the 2-clique 12 to 34, so we might expect this to yield a directional decomposition. But the forwardedges are not sufficient to kill the 2-clique 12, and so we see from the dynamics on the right, that we arenot guaranteed a directionality of flow. Instead, 12 supports a stable fixed point, and thus when we initializeactivity on those nodes, it remains there, and never flows to the other stable fixed point 34. Remark 3.3 If G is a directional graph, its directional decomposition is not unique. For example, as longas ω has more than one vertex, then vertices can always be moved from it to τ and maintain directionality.However, directional decompositions are most useful when the τ component is as small as possible, since thisgives the strongest restrictions on the possible fixed point supports of the whole network. One candidate τ forsuch a decomposition is τ := ∪ σ ∈ FP( G ) σ . However, this set does not guarantee a directional decompositionsince we have not guaranteed that all subsets of V G that intersect ω := V G \ τ die by graphical domination.In order to satisfy this property, it may be necessary to add some additional vertices to τ , and doing this ina minimal way may not be unique. It is an open question if every directional graph has a unique directionaldecomposition with minimal τ . erve theorems for neural networks 11
31 24 31 24 42 31 1 23 431 24 5 1 2 53 431 24 1 23 4 1 2 53 4
Fig. 6 Directional graphs: examples and non-examples.
Note that we refer to fixed point supports { i i , i , . . . , i k } simplyas i i · · · i k . For example, 234 denotes { , , } . In this section, we aim to characterize the fixed points of more complex graphs by covering the graph withdirectional graphs, and then analyzing a simpler associated object known as the nerve of the cover. Theintuition is as follows. As described in Section 3, if G is a directional graph with direction ω → τ , theactivity of the network flows from ω to τ . Thus, from a bird’s eye view, the flow of activity of such a graphcan be represented by the flow of activity along a single directed edge from source to sink. Moreover, thisflow of activity reflects restrictions imposed on the fixed point supports as well. With this in mind, we willtake any graph G and aim to cover it with directional graphs that have appropriate pairwise intersections.From this cover, we construct a nerve, which is a simplified graph where subsets of vertices are collapsed tosingle points, and each directional graph of the cover is now represented by a single directed edge. Theseedges are glued to one another in a way representative of the intersection pattern of the cover. We will seethat, with this construction, we are able to deduce certain restrictions on the fixed point supports of theoriginal graph G by studying the fixed points of the nerve of the cover, which is in general a simpler graph. We begin by making the notion of directional cover and its nerve precise.
Definition 4.1 (graph cover)
Let G be a graph. A graph cover of G is a collection of induced subgraphs U = { G i := G | V i | for some V i ⊆ V G } such that G is the union of the G i . In other words, V G = ∪ i ∈ I V i and E G = ∪ i ∈ I E G i . Remark 4.2
Note that every vertex and every edge of G must live in at least one G i , but often they live inmultiple G i within the cover. In particular, since the covering graphs are induced subgraphs of G , if u, v ∈ V i and u, v ∈ V j , then any edges between u and v will be in both G i and G j . Next we turn to a special type of graph cover which we call rigid directional cover . In a rigid directionalcover, we require that all the graphs of the cover are directional and that they overlap in prescribed waysthat will facilitate associating a nerve to the cover and ensure that this nerve captures constraints on FP( G ).The rigid condition can be informally described as follows. Consider a graph cover U of G , where allthe covering graphs are directional. Let G , G ∈ U be a pair of graphs in the cover, with directionaldecompositions ω → τ and ω → τ , respectively. The graph cover U is rigid if for any pair G and G thathave nontrivial intersection, their overlap is of one of the following three types:1. The τ component of the first graph acts as the ω component of the second, i.e., V G ∩ V G = τ = ω . Inthis case we say the graphs have a chaining overlap . (See Figure 7A.)2. The two covering graphs intersect exactly at their τ component, i.e., V G ∩ V G = τ = τ . In this casewe say the graphs have merging overlap . (See Figure 7B.)3. The two covering graphs intersect exactly at their ω component, i.e., V G ∩ V G = ω = ω , and have theadditional property that there are no back edges from vertices in τ to vertices in ω in either graph, i.e., E G ( τ , ω ) = E G ( τ , ω ) = ∅ . In this case we say the graphs have a splitting overlap . (See Figure 7C.) Fig. 7
A pair of graphs G and G that have (A) a chaining overlap (B) a merging overlap and (C) a splitting overlap. Effectively, a rigid directional cover is always induced by a partition of the vertices of the underlying graphand this partition encodes all the information of the cover itself. Therefore, we formally define a rigid graphcover as follows.
Definition 4.3 (rigid directional cover and its nerve)
Let G be a graph. Given a partition of thevertices, ν = { ν , . . . , ν n } , let E = E ( G, ν ) := { ( i, j ) ∈ [ n ] × [ n ] | G | ν i ∪ ν j is directional with direction ν i → ν j } . We say that the partition { ν , . . . , ν n } induces a rigid directional cover of G if1. U = { G ij := G | ν i ∪ ν j | ( i, j ) ∈ E } ∪ { G | ν i | i ∈ [ n ] } is a graph cover of G , and2. whenever G ij , G ik ∈ U , they have “splitting overlap”, meaning there are no edges from ν j to ν i and noedges from ν k to ν i , i.e., E G ( ν j , ν i ) = E G ( ν k , ν i ) = ∅ .We define the nerve of the cover, denoted by N = N ( G, U ), to be the graph with vertex set V N := [ n ] andedge set E N := E . Note that the partition ν induces a canonical quotient map π : V G → V N that identifiesall the vertices of a component ν i , so that π ( ν i ) = { i } for each i ∈ V N .In this paper we will only work with rigid directional covers. Therefore, in the remainder of this work wewill use the term directional cover to refer to a rigid directional cover.Figure 8 gives an illustration of a graph G with vertex partition ν = { ν i } that induces a collectionof covering graphs { G ij } that are all directional. In the nerve, N = N ( G, U ), we see a vertex i for eachcomponent ν i from G , and an edge i → j corresponding to each covering graph G ij , which has direction ν i → ν j . erve theorems for neural networks 13 Fig. 8 A graph with a directional cover and the corresponding nerve. (Left) A graph G . (Middle) A directional cover U of G . The sets ν i , ν j , ν k , ν l , ν m partition the vertices of G . All the edges of G are contained within some of the coveringgraphs { G ij } shown, each of which is directional with direction ν i → ν j . (Right) The nerve N = N ( G, U ) with a vertex foreach component of the partition and a directed edge for each directional graph of the cover. Remark 4.4
Suppose U is a directional cover of G induced by the partition { ν , . . . , ν n } . The underlyinggraph G can only have edges between vertices in components ν i and ν j if there is an edge between i and j in the nerve N ( G, U ). This follows because every edge of G must be contained in some covering graph G ij ,and every covering graph corresponds to an edge in the nerve. Lemma 4.5
Let G be a graph with nerve N = N ( G, U ) for some directional cover U . Then the nerve N isa simple directed graph that is oriented. Proof
Recall that given a partition ν = { ν , . . . , ν n } of the vertices of G , the nerve N is defined as a graphon n vertices with edge set E given in Definition 4.3. The edge set E is straightforward to determine fromthe partition ν . Specifically, if there are no edges between ν i and ν j in G , then neither ( i, j ) nor ( j, i ) are in E , since G | ν i ∪ ν j is a disjoint union, which can never be directional. If there are any edges between ν i and ν j , we must have either ( i, j ) ∈ E or ( j, i ) ∈ E in order for the { G ij } to cover G . Moreover, we can onlyhave one of ( i, j ) or ( j, i ) in E since G | ν i ∪ ν j can never be directional with both ν i → ν j and ν j → ν i . Thus, N is oriented. Finally, to see that N is simple, notice that ( i, i ) / ∈ E since G ii can never be directional with ω = τ = ν i . (cid:3) Given the complexity of the requirements of a directional cover, specifically that every covering graphmust be directional, it is natural to ask when a graph actually has such a cover. Of course, every graph hasa trivial directional cover induced by the trivial partition ν = V G ; in this case, the nerve of the cover is justa single point. At the other extreme, whenever G is an oriented graph, the partition of singletons ν i = { i } will induce a directional cover, whose nerve is precisely the original graph G . While these two trivial coversexist, they clearly do not provide any insight into the structure or expected dynamics of G . There is an artto finding a partition of V G from which a cover with an informative nerve can be obtained.It is important to note that not every graph has a directional cover induced by a nontrivial partition. Forexample, if G is a clique, then there is no nontrivial partition of V G that can admit a directional cover sinceevery G ij will be a clique, and thus not directional. On the other extreme, there are graphs with multiplenontrivial partitions of V G which induce a directional cover. See for example, Figure 9C-D, which shows twodifferent covers of the same graph, and where the nerve of the first one is directional while the nerve of thesecond is a cycle.It is an open question which graphs have at least one nontrivial partition that admits a directional cover.And unfortunately, there is currently no efficient way to find all the partitions of a graph that do inducedirectional covers. However, when we have a directional cover of G , obtained either by brute force search orfrom intuition into the original construction of the graph, the nerve of the cover can give significant insightinto the collection of fixed points and consequently into the predicted dynamics of the underlying network.In particular, nerve theorems ensure that there is a provable connection between FP( G ) and the structureof the nerve under certain conditions on G and/or N ( G, U ). Ideally, we would hope for a nerve theorem that provides a strong connection between the fixed point supportsof the original graph and those of the nerve. For example, we might hope that for any graph G that admitsa directional cover with nerve N that we can guarantee a condition such as σ ∈ FP( G ) ⇒ π ( σ ) ∈ FP( N ) . (4)Unfortunately, though, this strong restriction on FP( G ) does not hold for all graphs and all directionalcovers. Fig. 9 Counterexamples to general nerve theorems. (A) and (B) give two different graphs with partitions that inducedirectional covers. The nerve of each cover is shown to the right. (C) and (D) give two different partitions for the same graph.In C, the partition induces a nerve that is directional, while in D, the partition induces a nerve that is a cycle.
Example 4.6
For the graph in Figure 9A, we see that the partition { ν , . . . , ν } shown induces a directionalcover of G . The nerve N = N ( G, U ) of this cover is shown on the right. Recall that a cycle is uniformin-degree 1 and thus it supports a fixed point precisely when no external vertex receives more than one edgefrom it (see Theorem 2.14 in Section 2 ). Thus we have 123 ∈ FP( G ) since the external vertices 4 and 5each receive only one edge from the cycle. However, π (123) = 123 / ∈ FP( N ) since in the nerve, the cycle 123has two outgoing edges to vertex 4. Thus, σ ∈ FP( G ) (cid:54)⇒ σ ∈ FP( N ).An alternative style of nerve theorem would enable us to at least restrict the candidate fixed point supportsof G based on the structure of the nerve. For example, we might hope that whenever the nerve N ( G, U )is directional with direction W → T that the directionality would pullback to guarantee G is directionalwith direction ω → τ for ω = π − ( W ) and τ = π − ( T ). If this held, then we could guarantee that the fixedpoint supports of G were confined to τ = π − ( T ), and thus FP( G ) ⊆ FP( G | τ ) by Lemma 2.9. Such a resultwould be somewhat weaker than (4) in that the restrictions on σ ∈ FP( G ) would not be as strong. On otherhand, it would also be somewhat strong in a different direction, since a result like this would guarantee thepresence of graphical domination relationships for ruling out fixed point supports, which is not somethingguaranteed by (4). Unfortunately, this alternative nerve theorem does not hold in general either, and thesame graph as above provides a counterexample, as do the other graphs in Figure 9. Recall that we write 123 to denote the fixed point support { , , } .erve theorems for neural networks 15 Example 4.7
It is straightforward to see that the nerve N in Figure 9A is directional for W := { , , } and T := { } : every subset S ⊆ V N that intersects W either has a proper source in G | S , and thus dies fromdomination by Lemma 2.10, or contains 123, in which case we have 4 > S
1. However, G is not directionalsince 12345 ∈ FP( G ), so there is no collapse of the fixed point supports of G onto a proper subset τ . Figure 9Cgives another counterexample where G has a full support fixed point but a directional cover whose nerve isdirectional.We have seen that in general the existence of a directional relationship W → T of the nerve does notguarantee directionality of G . But are there certain conditions under which this holds? It turns out thatwe can pullback such a directionality relationship in the special case when the nerve has a nontrivial DAGdecomposition (see Definition 2.16). Theorem 4.8 (DAG decomposition of the nerve)
Let G be a graph with nerve N = N ( G, U ) where U is a directional cover induced by a partition { ν , . . . , ν n } , and let π : V G → V N = [ n ] be the canonicalquotient map of the partition. Then for any DAG decomposition ( W , T ) of the nerve N , we have that G isdirectional with direction ω → τ for ω = π − ( W ) and τ = π − ( T ). In particular,FP( G ) ⊆ FP( G | τ ) , and so for all σ ∈ FP( G ), we have π ( σ ) ⊆ T .Notice that in Theorem 4.8, we can only conclude that σ ∈ FP( G ) ⇒ π ( σ ) ⊆ T , and so we do not quitehave the ideal nerve theorem result that π ( σ ) ∈ FP( N ) as in (4). The conclusion in Theorem 4.8 is weakerthan (4) because although the directionality of N guarantees that every element of FP( N ) is contained in T as is each π ( σ ), we cannot guarantee that π ( σ ) is actually a fixed point support of N .Next we consider when the nerve N is itself a DAG so that, in the maximal DAG decomposition, T isprecisely the sinks of N . We can immediately apply Theorem 4.8 to see that the directionality of N pullsback to G , but in fact we can say something stronger: the ideal nerve theorem conditions of (4) hold in thiscase. Moreover, it turns out that there are further restrictions on the fixed point supports of G in terms ofthe fixed points of the component subgraphs G | ν i , which are prescribed by the partition. Theorem 4.9 (DAG nerve)
Let G be a graph with nerve N = N ( G, U ) where U is a directional coverinduced by a partition { ν , . . . , ν n } , and let π : V G → V N = [ n ] be the canonical quotient map of thepartition. Suppose that N is a DAG, and let T = { sinks of N } and W = V N \ T . Then G is directional withdirection ω → τ for ω = π − ( W ) and τ = π − ( T ).Moreover,1. σ ∈ FP( G ) ⇒ π ( σ ) ∈ FP( N ) = P ( T ), where P ( T ) denotes the power set of T .2. σ ∈ FP( G ) ⇒ σ ∩ ν i ∈ FP( G | ν i ) ∪ {∅} for all i ∈ T and σ ∩ ν j = ∅ for all j ∈ W .Figure 10, illustrates three special cases of directional covers U of a graph G with their corresponding nervesshown below. These directional covers have pairwise overlaps that are either: only chainings (Figure 10A),only mergings (Figure 10B), or only splittings (Figure 10C). We refer to these types of overlaps as n -chaining, n -merging and n -splitting, respectively. We see that their correspoding nerves are DAGs where the set of sinks T is either a single sink, T = { n } , as in panels A2 and B2, or an independent set of sinks, T = { , , . . . , n } ,as in panel C2. Theorem 4.8 tells us that in each case the underlying graph G is directional with direction ω → τ for τ = π − ( T ). Additionally, in the case of n -splitting, Theorem 4.9 gives a stronger result. Namely, σ ∈ FP( G ) ⇒ π ( σ ) ∈ FP( N ) and σ ∩ ν i ∈ FP( G | ν i ) ∪ {∅} for i ∈ { , , . . . , n } . That is: any fixed point support σ of G gets pushed forward to a fixed point support of the nerve N , andfor any 2 ≤ i ≤ n we have that if σ intersects ν i then this intersection is also a fixed point support of theinduced subgraph on ν i .Thus far, we have only seen nerve theorems in the case when N has a nontrivial DAG component, but itturns out that a similar nerve result holds in the case when N is a cycle (and thus has no DAG component). Fig. 10 Example directional covers and nerves. (A1-C1) Graphs with simple directional covers in which every pair ofcovering graphs have the same type of overlap (chaining overlap in A, merging overlap in B, and splitting overlap in C). (A2-C2)Nerves for the simple directional covers above.
Theorem 4.10 (cycle nerve)
Let G be a graph with nerve N = N ( G, U ) where U is a directional coverinduced by a partition { ν , . . . , ν n } , and let π : V G → V N = [ n ] be the canonical quotient map.Suppose that N is a cycle on n vertices. Then1. σ ∈ FP( G ) ⇒ π ( σ ) ∈ FP( N ) = { [ n ] }
2. If { ν , . . . , ν n } is a simply-added partition of G , then σ ∈ FP( G ) ⇒ σ ∩ ν i ∈ FP( G | ν i ) for all i ∈ [ n ] . Theorem 4.10 is a repackaging of results from [8], which explores graphs known as directional cycles ; inthe terminology of this paper, these are precisely graphs with a directional cover whose nerve is a cycle. In[8], it was shown that for this family of graphs, every fixed point support must nontrivially intersect every ν i , and so for every σ ∈ FP( G ), we have π ( σ ) = [ n ]. Moreover, when N is a cycle, FP( N ) = { [ n ] } byLemma 2.11, and thus π ( σ ) ∈ FP( N ). Additionally, it was shown that when the partition { ν , . . . , ν n } has aspecial property, known as simply-added , then every fixed point support must restrict to yield a fixed pointof the component subgraphs G | ν i .It is worth noting that another special family of graphs with directional covers was previously studied in[7, Section 5]. That work focused on composite graphs , which are graphs where all the vertices in a componentbehave identically with respect to the rest of the graph. Consequently, the only directional covering graphs G ij used in the cover are those that have all possible edges forward from ν i to ν j and no backward edges. Inthis context, the components of a composite graph correspond to the partition { ν , . . . , ν n } that induces thedirectional cover, and the skeleton of the composite graph is its nerve. With this perspective, many of theresults of [7, Section 5] can be reinterpreted as nerve theorems for the special family of composite graphs. Throughout this subsection we fix the following notation: G is a graph with a partition { ν , . . . , ν n } thatinduces a directional cover U . The nerve N ( G, U ) is denoted by N , and π : V G → V N = [ n ] is the canonicalquotient map induced by the partition.Before proving the nerve theorems we give an overview of the structure of the proofs. To prove Theorem 4.8(DAG decomposition of the nerve), we first show that whenever N has a proper source s , we can guaranteethat G is directional for ω = π − ( s ) and τ = V G \ ω (see Lemma 4.12). This gives a rather coarse directionaldecomposition of G . We will then consider the general case when we have a DAG decomposition ( W , T )of the nerve N . We will use the previous result and show inductively that G is directional with direction π − ( W ) → π − ( T ). For this proof, we will use three ingredients we briefly describe now. We say that { ν , . . . , ν n } is a simply-added partition if every vertex in ν i is treated identically by the rest of the graph. Inother words, for every j ∈ V G \ ν i if j → k for some k ∈ ν i , then j → (cid:96) for all (cid:96) ∈ ν i .erve theorems for neural networks 17 First, we will use a topological ordering on W , which guarantees that the only possible edges in N | W arefrom lower numbered vertices to higher number vertices. With respect to this topological ordering, vertex 1in N is a proper source; vertex 2 is a proper source in N | V N \{ } ; vertex 3 is a proper source in N | V N \{ , } ,and so on. This ordering will allow us to induct on |W| . The second ingredient is Lemma 4.13. This resultwill allow us to refine a directional decomposition of a graph in order to grow ω , and consequently shrink τ by looking at directional decompositions of the subgraph induced on the vertices in τ . The third and finalingredient is Lemma 4.14. This result will show that the construction of a directional cover and its nerve iscompatible with taking subgraphs of the underlying graph corresponding to only some of the components ofthe partition inducing the cover.At the core of the proofs is the process of “extending domination”. To see why this idea is central, considera directional cover of G and σ ⊆ V G a subset of the vertices of G . Let µ be the intersection of σ with thevertices of one of the covering graphs such that µ is not contained in the τ component of that covering graph.Since µ is completely contained within a directional graph then it must die by domination. Under certainconditions, one can show that σ dies by domination by extending the domination relationship on µ to all of σ . Therefore, all the proofs hinge on the following technical result that determines when domination can beextended to a superset. Fig. 11 Figure for Lemma 4.11.
A graph G with subsets of the vertices α ⊂ µ ⊆ σ ⊂ V G satysfing the conditions ofLemma 4.11. Specifically, the vertex j ∈ µ \ α , the vertex k ∈ V G , and there are no edges from vertices in σ to those in µ \ α . Lemma 4.11 (restricting and extending domination)
Let G be a graph, and let α ⊂ µ ⊂ σ ⊆ V G besubsets of vertices with α possibly empty (see Figure 11). Then the following hold:(a) Restriction:
Suppose k > σ j where j ∈ µ and k ∈ V G . Then k > µ j. (b) Extension:
Suppose k > µ j where j ∈ µ \ α and k ∈ V G . If there are no edges from σ \ µ to µ \ α (i.e., E G ( σ \ µ, µ \ α ) = ∅ ), then k > σ j. Proof
Recall from Definition 2.7 that k > σ j if the following three conditions hold:1. For all i ∈ σ \ { j, k } if i → j , then i → k .2. If j ∈ σ , then j → k .3. If k ∈ σ , then k (cid:57) j .For (a), Restriction , we see that k > σ j immediately implies that k > µ j since if condition 1 holds for allof σ , then it holds for the subset µ as well, and conditions 2 and 3 go through directly.For (b), Extension , we see condition 2 goes through immediately since j ∈ µ ⊂ σ . For condition 3, observethat if k ∈ σ then either k ∈ µ or k ∈ σ \ µ , and in both cases k (cid:57) j as required. Finally, for condition 1,notice that for i ∈ µ this condition holds because k > µ j , while for i ∈ σ \ µ , this condition holds triviallybecause j ∈ µ \ α and there are no edges from σ \ µ to µ \ α by hypothesis. Thus condition 1 holds as well,and so k > σ j . (cid:3) We can now prove that it is possible to pull back directionality from the nerve N to G whenever N hasa proper source. Lemma 4.12 If s is a proper source in N , then G is directional with ω = π − ( s ) and τ = V G \ ω . Proof
Let s be a proper source in N , ω := π − ( s ), and τ := V G \ ω . To show that G is directional withdirection ω → τ , consider σ ⊆ V G such that σ ∩ ω (cid:54) = ∅ . We need to show that σ dies by domination, i.e.,that there exists a j ∈ σ ∩ ω and a k ∈ V G such that k > σ j .The organization of the proof is as follows. We first find a covering graph G s ∈ U such that ω ⊆ V G s .Then we set µ ⊆ σ to be the restriction of σ to G s . Since G s is directional, µ dies by domination. Wewill show that by setting α ⊂ µ to be the intersection of µ with the τ component of G s , the conditions ofLemma 4.11 hold (see Figure 12). Thus, we can extend the domination relationship to all of σ . Fig. 12 Figure for proof of Lemma 4.12.
Vertex s is a proper source in N (right) with outgoing edges to vertices 1,2, and3. In G (left), we see the corresponding components ν s and { ν , ν , ν } . We consider a subset σ ⊆ V G (outlined in red), andlet µ := σ ∩ ( ν s ∪ ν ) (shaded in purple). Arrows with an x through them indicate that no edges are allowed in the specifieddirection between the relevant components. Specifically, there are no edges from { ν , ν , ν } into ν s because the graphs G s , G s , and G s must have “splitting overlap” by the definition of directional cover. There are no edges from any other verticesin G into ν s because there are no edges in the nerve between s and any of the other vertices besides 1, 2, and 3. To find G s , notice that since s is a proper source in N there exists at least one vertex in N that s sendsan edge to; without loss of generality, label the vertices of N that s sends edges to as 1 , . . . , m . Since s → N , the covering graph G s := G | ν s ∪ ν must be directional with direction ν s → ν . Let µ := σ ∩ ( ν s ∪ ν )(see Figure 12). Since G s is directional, there exists a j ∈ µ ∩ ν s and k ∈ V G s such that k > µ j . Followingthe notation of Lemma 4.11, let α := µ \ ν s = µ ∩ ν . We will show that there are no edges in G from verticesin σ \ µ to vertices in µ \ α = µ ∩ ν s , enabling us to extend the domination relationship from µ to all of σ .Note that by definition of directional cover, there can only be edges between ν s and ν (cid:96) in G if there isan edge between s and (cid:96) in N (see Remark 4.4). Thus the only candidate vertices in G that could sendedges into µ \ α = µ ∩ ν s are those in { ν , . . . , ν m } since the only edges in N that involve s are those from s to 1 , . . . , m . But since ( s, , . . . , ( s, m ) ∈ E N , condition (2) of the definition of directional cover requiresthat the covering graphs G s , . . . , G sm have splitting overlap, so there are no edges from ν (cid:96) to ν s for any1 ≤ (cid:96) ≤ m . Thus, there are no edges from σ \ µ into µ ∩ ν s = µ \ α , and so by Lemma 4.11, the dominationrelationship k > µ j extends to give k > σ j . Hence G is directional with direction ω → τ . (cid:3) We now give the two lemmas that allow us to inductively use the result above. First, we show that onecan refine a directional decomposition of a graph by looking at possible directional decompositions of thesubgraph induced on the τ component. Lemma 4.13
Suppose that G is a directional graph with direction ω → τ and that G | τ is also directionalwith direction ω → τ (see Figure 13). Then G is directional with direction ω ∪ ω → τ . Proof
Let ω = ω ∪ ω and τ = τ . To show that G is directional with direction ω → τ , consider σ ⊆ V G such that σ ∩ ω (cid:54) = ∅ . We will show that σ dies by domination. erve theorems for neural networks 19 Fig. 13 Figure for Lemma 4.13.
A directional graph G with an initial directional partition ω → τ . Additionally, G | τ isdirectional with direction ω → τ . These two partitions can then be combined to show that G is directional for a larger set ω = ω ∪ ω (outlined in gray) and smaller set τ = τ . Case 1: σ ∩ ω (cid:54) = ∅ . Then since G is directional with ω → τ , σ dies by domination.Case 2: σ ∩ ω = ∅ . Then σ ⊆ τ . Since σ ∩ ω (cid:54) = ∅ , we have σ ∩ ω (cid:54) = ∅ . Then since G | τ is directional with ω → τ , there exists j ∈ σ ∩ ω and k ∈ V G | τ = τ such that k > σ j in G | τ . Since σ ∩ ω = ∅ , there areno vertices in σ outside of G | τ that could potentially subvert the domination relationship between k and j .Thus, k > σ j in all of G , and so G is directional for ω = ω ∪ ω and τ = τ . (cid:3) We now show that the construction of a directional cover and its nerve behaves nicely for induced subgraphsof G, restricting to a subset of the partition components. Lemma 4.14
Let ν I := { ν i | i ∈ I } be a subset of the components of the partition ν of G , for I ⊆ V N .Let G I := G | ∪ i ∈ I ν i denote the induced subgraph of G on the components ν I . Then the partition ν I of thevertices of G I induces a directional cover U I whose nerve is N ( G I , U I ) = N | I , the restriction of the originalnerve N = N ( G, U ) to the vertices I . Proof
Observe that the partition ν I yields the edge set E ( G I , ν I ) := { ( i, j ) ∈ I × I | G | ν i ∪ ν j is directional with direction ν i → ν j } . This edge set is clearly a subset of the edge set E ( G, ν ) for the cover U of G ; specifically, E ( G I , ν I ) = { ( i, j ) ∈ E ( G, ν ) | i, j ∈ I } . Moreover, the set of graphs { G ij | ( i, j ) ∈ E ( G I , ν I ) } form a graph cover of G I because for any i, j ∈ I if there are any edges between ν i and ν j in G , then G ij or G ji must have beenin the cover U of G , so one of these graphs is directional, and thus ( i, j ) or ( j, i ) is in E ( G I , ν I ). Thus, ν I induces a directional cover U I of G I with vertex set I and edge set E ( G I , ν I ). Since E ( G I , ν I ) is preciselythe edges of E ( G, ν ) among the vertices in I , we see that the nerve of the cover N ( G I , U I ) is precisely N | I ,the restriction of the nerve of G to the vertices I . (cid:3) We are now prepared to prove Theorem 4.8 (reprinted below for convenience).
Theorem 4.8 (DAG decomposition of the nerve)
For any DAG decomposition ( W , T ) of the nerve N ,we have that G is directional with direction ω → τ for ω = π − ( W ) and τ = π − ( T ). In particular,FP( G ) ⊆ FP( G | τ ) , and so for all σ ∈ FP( G ), we have π ( σ ) ⊆ T . Proof
Let ( W , T ) be a DAG decomposition of the nerve N . We will show that G is directional with direction π − ( W ) → π − ( T ) by inducting on |W| in the DAG decomposition of N .The base case of |W| = 1 follows immediately from Lemma 4.12 since the first element of W must be aproper source in N . For the inductive step, assume the inductive hypothesis holds whenever |W| < m andconsider a DAG decomposition ( W , T ) of N where |W| = m . Since N | W is a DAG, there is a topologicalordering of the vertices such that the only edges in N | W are from lower numbered to higher numbered nodes; WLOG relabel the vertices of W as 1 , . . . , m according to this ordering. Let W = W \ { m } and T = T ∪ { m } . It is straightforward to check that ( W , T ) is also a DAG decomposition of N . Since |W | < m , by the inductive hypothesis, G is directional with ω = π − ( W ) and τ = π − ( T ) = π − ( m ) ∪ π − ( T ) . We will show that G | τ is also directional, so that we may apply Lemma 4.13 and further refine the directionaldecomposition of G . Specifically, we will show that G | τ has direction ω → τ for ω = π − ( m ) and τ = π − ( T \ { m } ) = π − ( T ).By Lemma 4.14, the original directional cover of G restricts to a directional cover of G | τ and its nerve is N | { m }∪T . Since m ∈ W in the original DAG decomposition of N , m is not a sink in N , so it has at leastone outgoing edge. Moreover there are no edges from T back to m in a DAG decomposition. Thus m is aproper source in N | { m }∪T . Therefore, by Lemma 4.12, G | τ is directional with direction ω → τ for ω = π − ( m ) and τ = π − ( T ) . Finally, since G is directional with ω = π − ( W \{ m } ), τ = π − ( m ) ∪ π − ( T ) and G | τ is directional with ω = π − ( m ) and τ = π − ( T ), we see from Lemma 4.13, that G is directional with direction ω ∪ ω → τ .Since π − ( W ) = ω ∪ ω and π − ( T ) = τ , we see G is directional with direction π − ( W ) → π − ( T ) asdesired. (cid:3) Next we consider when the nerve N is itself a DAG so that in the maximal DAG decomposition T isprecisely the sinks of N . Theorem 4.8 guarantees that the directionality of N pulls back to G , but to provethe rest of the nerve theorem conditions, we must appeal to a result characterizing the fixed point supportsof disjoint unions , proven [7]. The disjoint union of component subgraphs is the graph consisting of thosesubgraphs with no edges between the components. Theorem 4.15 ([7], Theorem 11)
Let G be the disjoint union of component subgraphs G , . . . , G N . Forany nonempty σ ⊆ V G , σ ∈ FP( G ) ⇔ σ ∩ V G i ∈ FP( G i ) ∪ {∅} for all i ∈ [ N ] . We can now prove Theorem 4.9 (reprinted below).
Theorem 4.9 (DAG nerve)
Suppose that N is a DAG, and let T = { sinks of N } and W = V N \ T . Then G is directional with direction ω → τ for ω = π − ( W ) and τ = π − ( T ).Moreover,1. σ ∈ FP( G ) ⇒ π ( σ ) ∈ FP( N ) = P ( T ), where P ( T ) denotes the power set of T .2. σ ∈ FP( G ) ⇒ σ ∩ ν i ∈ FP( G | ν i ) ∪ {∅} for all i ∈ T and σ ∩ ν j = ∅ for all j ∈ W . Proof
The fact that G is directional with direction ω → τ for ω = π − ( W ) and τ = π − ( T ) follows fromTheorem 4.8 since the given choice of ( W , T ) is the maximal DAG decomposition of N . As a consequenceof this, we have FP( G ) ⊆ FP( G | π − ( T ) ), and so we turn our attention to G | π − ( T ) to understand FP( G ).Observe that since T = { sinks of N } , there are no edges between the vertices in T , and so N | T is anindependent set. Thus, there are no edges in G between the components ν i for i ∈ T , and so G | π − ( T ) isa disjoint union of the component subgraphs G | ν i . Applying Theorem 4.15, we see that σ ∈ FP( G | π − ( T ) )precisely when σ ∩ ν i ∈ FP( G | ν i ) ∪ {∅} for all i ∈ T . And since FP( G ) ⊆ FP( G | π − ( T ) ) by the directionalityof G , part (2) of the theorem statement follows immediately.For part (1), observe that since G is directional, σ ∈ FP( G ) implies that π ( σ ) ⊆ T . Since N is a DAG,by Lemma 2.15, FP( N ) = P ( T ), and so every subset of T is an element of FP( N ). Thus, σ ∈ FP( G ) ⇒ π ( σ ) ∈ FP( N ) as desired. (cid:3) erve theorems for neural networks 21 We now turn our attention to some examples that illustrate the power of our nerve theorems. Going back toFigure 2A,B of the Introduction, we see that this graph and its nerve satisfy the hypotheses of Theorem 4.8and Theorem 4.9. The nerve (panel B) is a simple path that has a maximal DAG decomposition with T = { } (10 is the unique sink node). Theorem 4.8 thus predicts that FP( G ) ⊆ FP( G | ν ), since ν = π − ( { } ) . In fact, FP( G ) = { ν } , so the network has a unique fixed point supported on ν . Moreover,this fixed point is stable. As seen in panel D, the dynamics do indeed converge to this stable fixed point.Moreover, for solutions with initial conditions supported on the first clique ( G | ν ), we see that the transientdynamics activate all cliques in the chain, in sequence, following the path of the nerve.In the remainder of this section, we will discuss additional examples of networks whose graphs and nervessatisfy the hypotheses of one or more of our nerve theorems: Theorem 4.8, Theorem 4.9, and Theorem 4.10.Just as in Figure 2, we will see that the nerve not only predicts the fixed points and asymptotic dynamics,but also provides insight into the transient dynamics of the network. Figure 14A depicts the graph of a complex network whose nodes are grouped according to a partition with12 components (8 in gray, 4 in color). In panel B we see the nerve of the induced directional cover, with thecolor of each node matching those in the original graph. This nerve has a DAG decomposition with the eightgray nodes in the DAG part, W , and the four colored in the non-DAG part, T . Theorem 4.8 thus tells usthat all fixed points of the original graph G must be contained in τ = π − ( T ), which is the set of colorednodes in panel A. Indeed, FP( G ) ⊆ FP( G | τ ) for this graph.Note, however, that we can also apply another nerve theorem to G | τ . Because G | τ has a nerve that is acycle, Theorem 4.10 tells us that any fixed point of G must intersect each of the components π − ( i ) for i in the cycle. That is, fixed points of G must contain at least one vertex from each of the four colors (red,blue, green, orange) shown in panel A. This is in fact the case, as the CTLN for G has FP( G ) = { } .Moreover, we see that even if we choose initial conditions supported only on the gray vertices, the dynamicswill converge to a part of the state space where the gray neurons are off and at least one neuron of eachcolor is active (see Figure 14C). time f i r i n g r a t e Fig. 14 (A) A graph G for a CTLN. (B) A nerve N of G with 12 vertices. Each of the gray vertices in N corresponds toa cluster of gray nodes in (A), while the colored vertices in N correspond to the vertices with matching colors in G . (C) Asolution to a CTLN with graph G and initial conditions supported on the top-most gray nodes. The activity flows down thenetwork and converges to a limit cycle involving only the colored nodes.2 F. Burtscher, C. Curto, S. Ebli, D. Egas Santander, K. Morrison, A. Patania, N. Sanderson Figure 15 provides another example where Theorem 4.8 and Theorem 4.10 can be used in concert topredict both the fixed points and the dynamics of a large complex network. In this case, the network hasa grid-like structure, with repeating motifs reminiscent of a cortical column. The nerve is a finite latticewith directed edges moving down and to the right across the grid (panel A). Each vertex corresponds to acomponent ν i of G . We will consider two versions of the nerve: the first version, N , includes the 15 → N , does not (see dotted line in panel A). Fig. 15
A grid-like nerve shapes global sequential dynamics irrespective of the component graphs. (A) Two versions of thenerve, with and without the 15 →
20 edge. (B-C) Solutions for CTLNs where the graph is nerve 1 or nerve 2, respectively. (D)5-clique component graphs and a corresponding directional graph for an edge in the cover. (E) 5-star component graphs andtheir directional graph. (F-I) Solutions for associated CTLNs for all four combinations of nerves and component graphs.
For the first nerve, N , the maximal DAG decomposition ( W , T ) has nodes 1-15 in W and the bottomrow of nodes 16-20 in T . Theorem 4.8 thus predicts that all fixed points of the underlying graph G aresupported in τ = π − ( T ). Moreover, since N | T forms a cycle, applying Theorem 4.10 we expect all fixedpoints to intersect each component of T . In the second nerve, N , 15 (cid:54)→
20 and thus node 15 is a sink. AnyDAG decomposition for N must therefore include node 15 in T . Applying Theorem 4.8, we expect fixedpoint(s) corresponding to the cycle, as before, as well as fixed point(s) supported in ν = π − (15). We mayalso have fixed points whose supports are unions of those from the cycle and ν . erve theorems for neural networks 23 The above observations are all independent of the choice of components ν i of G , so long as the nervecorresponds to a directional cover. In particular, we can chose ν i = { i } as single vertices, in which case G isthe same graph as N . Figure 15B shows a solution for a CTLN with G = N , where the activity is initializedwith x (0) = 0 . x i (0) = 0 for i > → → → → →
18, and once it reaches the bottom row T the dynamics converge to a limit cycle that follows the cycle 18 → → → → →
18 in the graph.If we choose the same initial condition for G = N , we obtain exactly the same result. This is because theactivity never reaches node 15, and so the cut edge is not “seen.” Figure 15C shows the solution when weinitialize with x (0) = 0 .
5, and x i (0) = 0 for i (cid:54) = 3 (initial condition 2). In this case, the activity follows azig-zag trajectory 3 → → → →
15 that ends in a stable fixed point at 15. This is precisely what weexpect since node 15 is a sink.Next, we consider two more complex choices for the components ν i and the directional graphs in the cover.In the first version (Figure 15D), the components are 5-cliques and the directional graphs are the same asin Figure 2. In the second version (Figure 15E), each component is a cyclically symmetric oriented graph onfive vertices, called the 5-star. In this case, the directional graphs G ij = G | ν i ∪ ν j are chosen to have forwardedges from ν i onto three of the nodes in ν j , and backwards edges from the remaining two nodes in ν j to allfive nodes in ν i .Figure 15F-I display the dynamics of the network for the four combinations: 5-clique components withnerve N (panel F), 5-clique components with nerve N (panel G), 5-star components with nerve N (panelH), and 5-star components with nerve N (panel I). In each case, the nerve dictates the global structure ofthe dynamics as activity flows from one component to another.Figure 15F shows the solution for a CTLN with 5-clique components and initial conditions supported in ν .As expected, the asymptotic behavior is of a limit cycle following the cycle 16 → → → → → G ) = { σ } , where σ = ν ∪ · · · ∪ ν isthe union of all vertices in G corresponding to the bottom row of the nerve. Note, however, that the nervepredicts dynamics well beyond the fixed point structure given by our nerve theorems. In particular, we see thatthe transient activity follows the same sequential zig-zag pattern 1 → → → → →
18 that we saw inFigure 15B. The same is true for solutions of the network having 5-star components, depicted in Figure 15H.The structure of the covering graphs, however, do affect the local dynamics within each component of thenerve. In Figure 15F the transient dynamics are considerably more regular than in Figure 15B, and theneurons within each component fire synchronously. In contrast, in Figure 15H the transient dynamics areirregular like in Figure 15B, and the neurons within each component do not fire synchronously.We see similar differences for networks with the second nerve, N . As in the case where the componentswere singletons, we find that for 5-clique and 5-star components the initial conditions supported on ν yieldthe same solutions as with nerve N . On the other hand, for initial conditions supported on ν , we see thetransient dynamics follow the same zig-zag sequence 3 → → → →
15 as in Figure 15C, and thedynamics converge to the neurons in ν . Here again we see global aspects of the dynamics being dictated bythe nerve, while local dynamics are affected by differences in the component graphs G | ν i . In Figure 15G, theactivity converges to a stable fixed point corresponding to the 5-clique supported on ν ; but in Figure 15I,the network does not converge to a fixed point because the 5-star does not support a stable fixed point.Instead, the activity settles into a limit cycle typical of 5-star CTLNs with activity confined almost entirelyto the neurons in ν . Recall that for any partition { ν , . . . , ν n } of the vertices of G , there is an associated quotient map π : V G → V N = [ n ] defined by π ( ν i ) = { i } for each i ∈ [ n ]. We saw in Section 4 that if such a partitioninduces a directional cover, and the nerve N has a DAG decomposition ( W , T ), then the fixed points of thecorresponding CTLN are confined to the non-DAG part T . In other words, Theorem 4.8 tells us that Each vertex k in the 5-star has two outgoing edges: k → k + 1 and k → k + 2 (indexing mod 5). Here we are using the fact that ν is a simply-added partition of G and FP( G | ν i ) = { ν i } for all i to obtain the strongestconclusion from Theorem 4.10.4 F. Burtscher, C. Curto, S. Ebli, D. Egas Santander, K. Morrison, A. Patania, N. Sanderson FP( G ) ⊆ FP( G | τ ) , where τ = π − ( T ). Now if we consider the restricted graph, G (cid:48) = G | τ , we can potentially iterate this processby finding a new directional cover, with nerve N (cid:48) , with DAG decomposition ( W (cid:48) , T (cid:48) ) and quotient map π (cid:48) .This would enable us to further restrict the fixed points of the original network toFP( G ) ⊆ FP( G (cid:48) ) ⊆ FP( G (cid:48)(cid:48) ) , where G (cid:48)(cid:48) = G | τ (cid:48) , and τ (cid:48) = π (cid:48)− ( T (cid:48) ) ⊂ τ . Note that G (cid:48)(cid:48) is the original graph restricted to an even smallersubset of vertices. This kind of iteration may enable us to get more power from our nerve theorems, byfurther constraining FP( G ). f i r i n g r a t e time f i r i n g r a t e time Fig. 16
Iterative application of nerve theorems.
Figure 16A shows an example of a graph (A1) whose nerve (A2) is a DAG. Therefore, we can applyTheorem 4.9 to conclude that the fixed point supports of FP( G ) must be unions of fixed points of G restricted to the green, red, or orange components. Indeed, for a given initial condition supported on the topgray node in the nerve, we see the solution converge to a limit cycle supported only on green nodes (A3).The green component, however, is itself a complex graph. In fact, we may consider the subgraph G (cid:48) of G corresponding to the non-DAG part of N . Figure 16B shows G (cid:48) with nodes colored according to a DAGdecomposition of its own nerve N (cid:48) (B2). (Note that G (cid:48) is a disjoint union of three graphs, and the nerve N (cid:48) is the disjoint union of nerves for each connected component of G (cid:48) .) This allows us to restrict the fixedpoints of G even further, to the vertices that map to the colored nodes in B2. We see this reflected in thedynamics as well. Panel B3 shows the same limit cycle as before, only now it’s clear that the green curves inA3 correspond only to the yellow, purple, and blue neurons in B1. Acknowledgements
This research is a product of one of the working groups at Workshop for Women in ComputationalTopology (WinCompTop) Workshop in Canberra (1-5 July 2019). CC and KM acknowledge funding from NIH R01 EB022862,NSF DMS-1951165, and NSF DMS-1951599.
References
1. M. Abeles.
Local cortical circuits: An electrophysiological study . Springer, Berlin, 1982.2. Y. Aviel, E. Pavlov, M. Abeles, and D. Horn. Synfire chain in a balanced network.
Neurocomput. , 44:285–292, 2002.3. T. Biswas and J. E. Fitzgerald. A geometric framework to predict structure from function in neural networks. Availableat https://arxiv.org/abs/2010.09660
4. Karol Borsuk. On the imbedding of systems of compacta in simplicial complexes.
Fundamenta Mathematicae , 35(1):217–234, 1948.5. C. Curto, A. Degeratu, and V. Itskov. Encoding binary neural codes in networks of threshold-linear neurons.
NeuralComput. , 25:2858–2903, 2013.6. C. Curto, J. Geneson, and K. Morrison. Stable fixed points of combinatorial threshold-linear networks. Available at https://arxiv.org/abs/1909.02947 erve theorems for neural networks 257. C. Curto, J. Geneson, and K. Morrison. Fixed points of competitive threshold-linear networks.
Neural Comput. , 31(1):94–155, 2019.8. C. Curto, J. Londono-Alvarez, K. Morrison, and C. Parmelee. TLN graph rules, network building blocks, and core motifs.
In preparation.
9. C. Curto and K. Morrison. Pattern completion in symmetric threshold-linear networks.
Neural Computation , 28:2825–2852,2016.10. R. H. Hahnloser, R. Sarpeshkar, M.A. Mahowald, R.J. Douglas, and H.S. Seung. Digital selection and analogue amplificationcoexist in a cortex-inspired silicon circuit.
Nature , 405:947–951, 2000.11. R. H. Hahnloser, H.S. Seung, and J.J. Slotine. Permitted and forbidden sets in symmetric threshold-linear networks.
NeuralComput. , 15(3):621–638, 2003.12. G. Hayon, M. Abeles, and D. Lehmann. A model for representing the dynamics of a system of synfire chains.
J. Comput.Neurosci. , 18(41-53), 2005.13. Jean Leray. Sur la forme des espaces topologiques et sur les points fixes des repr´esentations.
J. Math. Pures Appl. , 9:95–248,1945.14. K. Morrison and C. Curto.
Predicting neural network dynamics via graphical analysis , pages 241–277. Book chapter inAlgebraic and Combinatorial Computational Biology, edited by R. Robeva and M. Macaulay. Elsevier, 2018.15. K. Morrison, A. Degeratu, V. Itskov, and C. Curto. Diversity of emergent dynamics in competitive threshold-linear networks:a preliminary report. Available at https://arxiv.org/abs/1605.04463
16. H.S. Seung and R. Yuste.
Principles of Neural Science , chapter Appendix E: Neural networks, pages 1581–1600. McGraw-Hill Education/Medical, 5th edition, 2012.17. X. Xie, R. H. Hahnloser, and H.S. Seung. Selectively grouping neurons in recurrent networks of lateral inhibition.