MMotifs for processes on networks
Alice C. Schwarze ∗ Department of Biology, University of Washington, Seattle, WA, USA andeScience Institute, University of Washington, Seattle, WA, USA
Mason A. Porter † Department of Mathematics, University of California Los Angeles, Los Angeles, CA, USA
The study of motifs in networks can help researchers uncover links between structure and functionof networks in biology, the sociology, economics, and many other areas. Empirical studies of networkshave identified feedback loops, feedforward loops, and several other small structures as “motifs” thatoccur frequently in real-world networks and may contribute by various mechanisms to importantfunctions these systems. However, the mechanisms are unknown for many of these motifs. Wepropose to distinguish between “structure motifs” (i.e., graphlets) in networks and “process motifs”(which we define as structured sets of walks) on networks and consider process motifs as buildingblocks of processes on networks. Using the covariances and correlations in a multivariate Ornstein–Uhlenbeck process on a network as examples, we demonstrate that the distinction between structuremotifs and process motifs makes it possible to gain quantitative insights into mechanisms thatcontribute to important functions of dynamical systems on networks.
I. INTRODUCTION
The study of motifs in networks has advanced theunderstanding of various systems in biology [1–5], eco-nomics [6, 7], social science [8, 9], and other areas. Wheninterpreting motifs as small building blocks that can con-tribute to a network’s functionality, it can be importantto identify motifs that are necessary, beneficial, or dis-advantageous to a network’s function to help uncoverthe relationship between network structure and networkfunction.Traditionally, scientists have considered graphlets (i.e.,small graphs of typically three to five nodes) as buildingblocks of a network’s structure and identified them as“motifs” when empirical data [1, 6, 7, 9–12] or mathemat-ical models [13–16] indicate their importance to systemfunction. In many studies of “real-world” networks fromempirical data, researchers have compared graphlet fre-quencies in a network to graphlet frequencies in an appro-priate random-graph null model [6, 7, 9–12]. They subse-quently have concluded that graphlets that are overrepre-sented in the network are likely to be relevant for impor-tant functions of the system that is associated with thatnetwork. However, the results of such studies dependvery sensitively on the choice of a appropriate random-graph null model [17–19], and this approach to motifidentification does not uncover the mechanisms by whichthe identified graphlets contribute to important systemfunctions.Other studies have aimed to provide mechanistic in-sights by modeling dynamical systems on graphlets inisolation [13–16]. The design of such studies requires an a priori choice of a graphlet, a dynamical system or a ∗ Corresponding author: [email protected] † Corresponding author: [email protected] class of dynamical systems, and a candidate mechanismby which the graphlet facilitates an important systemfunction. It is therefore difficult for such studies to dis-cover new and/or unexpected mechanisms or to providea systematic comparison of the importances of differentgraphlets and different mechanisms for a system function.In the present paper, we propose a framework for con-necting the study of dynamics on networks with thestudy of motifs in networks. We propose to distin-guish between “structure motifs” (i.e., graphlets) in net-works and “process motifs” (which we define in the formof structured sets of walks) on networks, and we con-sider process motifs as building blocks for processes onnetworks[20]. We demonstrate how to use process mo-tifs to connect network structure to dynamics on net-works and to dynamics-based notions of system func-tions. These connections lead to mechanistic and quanti-tative insights into the contribution of all possible struc-ture motifs to a given system function. We give concreteexamples in Section IV.We define a process motif on a graph ( V, E ) with nodeset V and edge set E to be a walk graph , which we defineto be a directed and weighted multigraph in which eachedge corresponds to a walk in ( V, E ) and each edge’sweight corresponds to the length of the associated walk.We show examples of walk graphs in Fig. 1.In Fig. 2, we give an overview and an example of ourprocess-based approach to studying motifs in networks,and we indicate how our results can inform future stud-ies of motifs in network structure. We model a systemfunction as a real-valued function Y of the state of a dy-namical system. One can identify the process motifs thatare relevant for a given mathematical function Y and as-sociate each process motif with a numerical value b thatindicates its contribution to Y . From process motifs andtheir contributions, one can derive structure motifs thatare relevant to the function Y and their contributions c to Y . Process motifs thus offer a framework for identifying a r X i v : . [ phy s i c s . s o c - ph ] A ug (a) (b) (c) (d)(e) (f) (g) Figure 1. Examples of walk graphs. In (a), we show a smallnetwork. In (b)–(g), we show examples of associated walkgraphs. The walk graphs in (b) and (c) are examples of walkgraphs that use each edge in the edge set E at most once andin which each walk-graph node corresponds to a node in V .The walk graphs in (d) and (e) are examples of walk graphsin which two walk-graph nodes correspond to the same nodein V . The walk graphs in (f) and (g) are examples of walkgraphs that use edges in E more than once. of functionally important graphlets (i.e., structure mo-tifs) from mathematical models. This approach can leaddetailed insights into the mechanisms by which structuremotifs can affect a system function (see Section IV) [21].One can use structure-motif contributions to rank mech-anisms based on their efficiency and thereby rank struc-ture motifs based on their importance in contributing toa system function.As an example system, we use the multivariateOrnstein–Uhlenbeck process (mOUP), which is a pop-ular model for noisy coupled systems [22]. It has beenapplied to neuronal dynamics [23], stock prices [24], thestudy of gene expression [25], and other systems. Proper-ties of the mOUP are related to properties of coupled ex-citable systems. For example, one can derive the mOUPas a linear-response approximation of an integrate-and-fire model for excitable neurons [26, 27].As example system properties, we examine the covari-ances and the correlations in the mOUP at steady state.Covariances and correlations between pairs of nodes ina network are relevant for a wide variety of research.In particular, researchers have used correlations betweenvariables to construct networks for various applications[28–30]. For example, in networks of functional con-nectivity, an edge may indicate a large positive corre-lation between two neurons or two brain regions [28].In networks of gene co-expression, an edge may indi-cate a strong correlation between the expression of twogenes [29]. Additionally, existing intuitive results on sim-ple network structures that induce covariance and corre-lation (see, e.g., Reichenbach’s common-cause principle[31]) make covariance and correlation interesting exam-ples for our study. We demonstrate that our approachconfirms known results about covariation between vari-ables and that they yield additional, quantitative insightsinto the mechanisms by which network structure can en-hance or reduce covariance or correlation between nodes.Our process-based approach to the study of motifs on networks yields a list of relevant process motifs (withtheir respective contributions to a system function) anda list of relevant structure motifs (with their respectivecontributions to the same system function). As we indi-cate in Fig. 2, these results depend both on the choice ofa dynamical system and on the choice of system function.However, they do not depend on the choice of a networkor a random-graph model. In Fig. 2, the arrow from thecenter panel to the left panel indicates how our resultscan inform future studies of graphlets in networks andcan lead to quantitative insights into the importance ofgraphlets for a system function on a given network (fromdata or from a random-graph model) or for a random-graph model.Our paper proceeds as follows. In Section II, we re-view some graph-theoretical concepts and define the no-tion of walk graphs, which make it possible to distinguishbetween structure motifs and process motifs. We alsoprovide an overview of the use of motifs in prior stud-ies of networks. In Section III, we show how to deriveprocess motifs, structure motifs, and their contributionsto a given property (such as a correlation) of a dynam-ical system. In Section IV, we give a brief introductionto the mOUP and derive process motifs and structuremotifs for steady-state covariances and correlations fornode pairs in the mOUP. We discuss similarities and dif-ferences between the mechanisms for covariance and cor-relation between nodes in the mOUP. In Section V, weconclude and discuss possible applications of our process-based approach to the study of motifs in networks. Wealso explain why the distinction between process motifsand structure motifs is important for many (but not all)dynamical systems on networks. II. PROCESS MOTIFS AND STRUCTUREMOTIFS
In this section, we define process motifs and struc-ture motifs. In Section II A, we give a brief introductionto relevant graph-theoretical concepts. In Section II B,we introduce walk graphs. We then define process mo-tifs and structure motifs as walk graphs and connectedsubgraphs, respectively. To illustrate the conceptual dif-ference between process motifs and structure motifs, wecompare methods for counting walk graphs to methodsfor counting subgraphs in Section II C. In Section II D,we introduce the concepts of matching process motifs andmatching structure motifs. (These concepts are usefulfor our calculations in Section IV.) In Section II E, weoverview prior uses of process motifs and structure mo-tifs in the study of networks.
A. Some graph-theoretical concepts
We now give definitions for walks and trails on net-works and paths in networks. These words and other ter-
Structure-based approach Process-based approach
Choose a dynamical systemObtain an expression of Y using the initial condition x and powers of the adjacency matrix A Choose a system property Y Interpret terms of this expression as contributions of process motifs (i.e., walk graphs)Calculate specific contributions c of structure motifs from contri-butions of process motifs that can occur on the structure motifUse structure motifs with largest c as indicators of efficient network mechanisms for increasing Y Process-based approach (example)
Ornstein–UhlenbeckprocessCount structure motifs (i.e., graphlets)Choose a networkInfer functional importance from frequency and mechanistic modelInfer functional importance from frequency Steady-state covariances (cid:7) (cid:8)(cid:9) (cid:7) (cid:8)(cid:9) (cid:10) 2(cid:12) (cid:13) (cid:16)(cid:17) (cid:15) (cid:18)(cid:19) (cid:18)(cid:19) (cid:20) (cid:17)(cid:16) (cid:17) (cid:21)(cid:22)(cid:23)(cid:17)(cid:21)(cid:22) (cid:8)(cid:9) l l l
Figure 2. Comparison of a structure-based approach to the study of motifs in networks and the process-based approach thatwe introduce in this paper. In the right panel, we give an overview of the results of applying our approach to the study ofprocess motifs and structure motifs that are relevant for steady-state covariances σ ij (see Eq. (7)) for node pairs ( i, j ) in amultivariate Ornstein–Uhlenbeck process with parameters θ , ς , and (cid:15) and adjacency matrix A (see Eq. (6)). The parameters L and (cid:96) characterize a process motif for σ ij with walk lengths (cid:96) and L − (cid:96) . minology for graph-theoretical concepts are often usedambiguously, and we will need to distinguish these con-cepts clearly for our work in the present paper.We consider a network (i.e., graph ) to be an orderedtuple ( V, E ) that consists of a set V of nodes and a set E ⊆ V × V of edges [32]. If the network is directed , itsedges e ∈ E are ordered pairs of nodes. If the network isundirected, its edges e ∈ E are unordered pairs of nodes.A weighted network is an ordered tuple ( V, E, W ) with anode set V and edge set E as before and a map W thatassigns a weight to each edge in E . For the remainder ofthe present paper, we exclude W from our notation for networks. However, our definitions and results hold forboth weighted and unweighted networks, and we gener-ally assume that edges in a network can have weights.A subgraph ( V (cid:48) , E (cid:48) ) of a network ( V, E ) is a networkthat consists of a node set V (cid:48) ⊆ V and an edge set E (cid:48) ⊆ E [32]. A supergraph ( V (cid:48)(cid:48) , E (cid:48)(cid:48) ) of a network ( V, E ) is anetwork with node set V (cid:48)(cid:48) ⊇ V and an edge set E (cid:48)(cid:48) ⊇ E [33].We distinguish between walks and trails on networksand paths in networks. Consider a directed or undirectednetwork ( V, E ) . A walk in this network is a sequence w = ( v i , e i ,i , v i , e i ,i , . . . ,e i (cid:96) − ,i (cid:96) , v i (cid:96) , e i (cid:96) ,i (cid:96) +1 , v i (cid:96) +1 ) of nodes v i , v i , . . . , v i (cid:96) , v i (cid:96) +1 ∈ V and edges e i ,i , e i ,i , . . . , e i (cid:96) − ,i (cid:96) , e i (cid:96) ,i (cid:96) +1 ∈ E such that each edge e i,j starts at node v i and ends at node v j [32]. The num-ber (cid:96) indicates the number of edges in a walk. We call (cid:96) the length of a walk. If no edge in E appears more thanonce in w , the walk w is also a trail [32]. If no node in V and no edge in E appear more than once in w , onecan use the set of nodes in w and the set of edges in w to construct a path . A path is a subgraph ( V (cid:48) , E (cid:48) ) thatconsists of a node set V (cid:48) ⊆ V and an edge set E (cid:48) ⊆ E that one can combine to construct a sequence ( v i , e i ,i , v i , e i ,i , . . . ,e i (cid:96) − ,i (cid:96) , v i (cid:96) , e i (cid:96) ,i (cid:96) +1 , v i (cid:96) +1 ) of nodes and edges [32]. The number (cid:96) is the length ofthe path.A path is a subgraph of a network. By contrast, a walkis a combination (with repetition allowed) of a network’snodes and edges [34]. One can use walks to describemany processes on networks [35–38]. Additionally, onecan consider the sequence of nodes and edges in a walkto be the temporal sequence of nodes and edges that asignal, a person, or some other entity traverses.A closed walk of length (cid:96) is a sequence w = ( v i , e i i , v i , e i i , . . . , e i (cid:96) − i (cid:96) , v i (cid:96) , e i (cid:96) i , v i ) of nodes and edges [32]. A cycle of length (cid:96) is a subgraph ( V (cid:48) , E (cid:48) ) that consists of a node set V (cid:48) ⊆ V and an edgeset E (cid:48) ⊆ E that one can combine to construct a sequence[32] ( v i , e i ,i , v i , e i ,i , . . . , e i (cid:96) − ,i (cid:96) , v i (cid:96) , e i (cid:96) ,i ) . One can think of a cycle as a closed path.We say that a graph is cyclic if it is a cycle. It is acyclic if it is not a cycle and none of its subgraphs is a cycle.An undirected network ( V, E ) is connected if for ev-ery unordered pair ( i, j ) of nodes in V , there exists apath from i to j . A directed network ( V, E ) is stronglyconnected if for every ordered pair ( i, j ) ∈ V × V , thereexists a path from i to j . It is weakly connected if itscorresponding undirected network is connected.A network has an associated adjacency matrix A =( a ij ) . If the network is unweighted, a ij ∈ { , } and a ij = 1 indicates an edge from node j to node i . Ifthe network is weighted, the non-zero elements of A are a ij = w ( e ) , where w ( e ) is the weight of the edge e fromnode j to node i . B. Walk graphs and process motifs
Consider a directed or undirected network ( V, E ) . Wedefine a walk graph on this network to be a weighted and directed multigraph ( ˜ V , ˜ E, (cid:96) ) , where ˜ V is a combi-nation (with repetition allowed) of nodes in V and eachedge in ˜ E corresponds to a walk on the associated net-work ( V, E ) . The length (cid:96) ( e ) of an edge e ∈ ˜ E is thelength of the corresponding walk (i.e., the number ofedges that the walk on ( V, E ) traverses). When charac-terizing walk graphs, a useful property is the walk graph’s spatial length L := (cid:88) e ∈ ˜ E (cid:96) ( e ) . A walk graph’s duration (i.e., temporal length ) is T :=max e ∈ ˜ E (cid:96) ( e ) . For our paper, the spatial length of walkgraphs is an important quantity. For the rest of our pa-per, we use the term “length” for a walk graph’s spatiallength.To give some examples of walk graphs, we recall thewalk graphs in Fig. 1. A walk graph that consists of asingle edge corresponds to a single walk on the associatednetwork (see, e.g., Figs. 1 (b), (d), and (g)). If a walkgraph consists of a single self-edge, then the walk graphcorresponds to a closed walk in the associated network(see, e.g., Fig. 1 (c)).We noted in Section II A that one can interpret a walkto describe a type of process. One can thus use a walkgraph to describe a composite process that consists ofseveral walks. We can now formally consider a processmotif to be a small, weakly connected walk graph thataffects a given function of the dynamics on a network.We consider a structure motif to be a small, weakly con-nected graph that affects a given function of the dynamicson a network. C. Counts of process motifs and structure motifs
It is common for studies of motifs to associate motifswith a “count”, “number”, or “frequency” to indicate theprevalence of a given motif in a given system [10, 39,40]. The count (i.e., number) of a structure motif s in anetwork ( V, E ) is the number of subgraphs of ( V, E ) thatare isomorphic to s . We define the count of a processmotif p in an unweighted network ( V, E ) to be the numberof sets of walks in ( V, E ) that form a walk graph that isisomorphic to p .For a weighted network ( V, E, W ) , it is useful to weighteach occurrence of a process motif by the product π := (cid:81) e w ( e ) , where one takes the product of the weights ofedges e ∈ E that the walks in p traverse. (If the walks in p traverse an edge k times, the corresponding edge weight w ( e ) appears in π with multiplicity k .) When ( V, E, W ) is a weighted network, we define the count of p to be tothe sum of edge-weight products π for each walk graphon ( V, E, W ) that is isomorphic to p .The counts of structure motifs in a network and thecounts of process motifs are related to each other. Eachstructure motif s has an associated set P s of process mo-tifs that can occur on it. Consequently, a change in the L = 1 L = 2 L = 3 | ˜ V | = | ˜ V | = | ˜ V | = | ˜ V | = N/A N/A N/AN/AN/A N/A 3-node feedforward loop1 2 11 2111 11 11 12 21 11 111 1 11 1 1 11 1 1 1111 111 111
Figure 3. Walk graphs that can occur on a 3-node feedforward loop. We sort walk graphs according to their length L and theirnumber | ˜ V | of nodes. The numerical edge labels on walk graphs indicate the length of an edge. Some walk graphs occur onthe 3-node feedforward loop but not on the 3-node feedback loop. We use orange edges and boxes with dashed boundaries todistinguish these walk graphs from others. The pink inset in the top-right corner shows the structure of a 3-node feedforwardloop. number of motifs of type s (i.e., the count of s ) leadsto a change in the counts for each p ∈ P s . To illustratethis relationship between counts of structure motifs andcounts of process motifs, we consider two small exam-ple networks: a 3-node feedforward loop and a 3-nodefeedback loop [10]. In Fig. 3, we show all length- L walkgraphs that can occur on a 3-node feedforward loop for L ≤ . In Fig. 4, we show all length- L walk graphs thatcan occur on a 3-node feedback loop for L ≤ .From comparing Figs. 3 and 4, we observe that somewalk graphs can occur on the 3-node feedforward loopbut not on the 3-node feedback loop, and vice versa. Thedifferences between the walk graphs for the 3-node feed-forward loop and the walk graphs for the 3-node feedbackloop illustrate that the structure of a network constrainsthe structures of walk graphs that can occur on it. The 3-node feedforward loop is an acyclic network with a max-imum trail length of 2. Because the feedforward loop is adirected acyclic graph (DAG), its associated walk graphsare also acyclic. Walk graphs on a DAG with a maximumtrail length of 2 cannot have edges of length (cid:96) ( e ) > .The structure of the 3-node feedback loop leads toother constraints on the structures of associated walkgraphs. For example, a walk graph that can occur onthe 3-node feedforward loop but not on the 3-node feed-back loop is the circular walk graph with | ˜ V | = 2 and L = 3 in Fig. 3. This walk graph consists of a length-1 edge and a length-2 edge that share both their startingnode and their ending node. D. Matching process motifs and matchingstructure motifs
Consider the set P s of process motifs that can occuron a structure motif s and the set S p of structure motifson which a process motifs p can occur. If one does notspecify a number | ˜ V | of nodes and a length L for a processmotif, the set P s for any s with one or more edges includesinfinitely many process motifs because a process motifcan use each edge of the structure motif infinitely manytimes. Conversely, for a given process motif p , the set S p includes infinitely many structure motifs because onecan add nodes or edges to any s ∈ S p to obtain anotherelement of S p .Most elements in P s are very long process motifs, andmost elements of S p are very large structure motifs. Tra-ditionally, studies of motifs on networks have focused onsmall motifs: process motifs with length L ≤ [41–43]and structure motifs with up to five nodes [10, 39]. Toassociate small process motifs with small structure mo-tifs and vice versa, we define matching process motifs and matching structure motifs . For a given process motif p ,a matching structure motif s ∗ p is a structure motif on L = 1 L = 2 L = 3 | ˜ V | = | ˜ V | = | ˜ V | = | ˜ V | = N/A N/AN/AN/A N/A 3-node feedback loop31 2 11 3 2111 11 11 21 12 12 21 1 1111 1 11 1 1 11 1 1 1111 111 111 111
Figure 4. Walk graphs that can occur on a 3-node feedback loop. Some walk graphs occur on the 3-node feedback loop butnot on the 3-node feedforward loop. We use orange edges and boxes with dashed boundaries to distinguish these walk graphsfrom others.. The inset in the top-right corner shows the structure of a 3-node feedback loop. which p can occur while using each edge in s ∗ p exactlyonce. Conversely, for a given structure motif s , a match-ing process motif is a process motif that can occur on s while using each edge in s exactly once.For a structure motif s with a finite number of edges,the set P ∗ s of matching process motifs has a finite numberof elements. For a process motif p with a finite length L ,the set S ∗ p of matching structure motifs also has a finitenumber of elements.In Fig. 5, we show sets of matching process motifs andsets of matching structure motifs for several structuremotifs and process motifs, respectively. Structure motifsthat do not include cycles have only acyclic matchingprocess motifs. Therefore, for a given number of edges,structure motifs that include cycles (e.g., the structuremotifs in the second and fourth row in the left table ofFig. 5) have more matching process motifs than acyclicstructure motifs (e.g., the structure motifs in the first andthird row in the left table of Fig. 5). Accordingly, acyclicprocess motifs have more matching structure motifs thancyclic process motifs.In general, a structure motif can have many matchingprocess motifs and a process motif can have many match-ing structure motifs. Motif-based research that aims tolink network structure to dynamics on networks requirescareful consideration of these matching motifs. In Sec-tion IV, we demonstrate the importance of these consid-erations using steady-state covariance and steady-state correlation of a multivariate Ornstein–Uhlenbeck process(mOUP) as an example. E. Previous work on process motifs and structuremotifs
To the best of our knowledge, previous research on net-work motifs has not distinguished explicitly between pro-cess motifs and structure motifs. Instead, studies havebeen concerned either with process motifs or with struc-ture motifs, and they have refer to either as “networkmotifs”. In this section, we give an overview of researchon “network motifs” and explain which of the reviewedstudies concern process motifs and which concern struc-ture motifs.Many reviews of network motifs have credited Milo etal. [10] for the idea of characterizing networks by con-nected subgraphs that are more frequent in a networkthan one would expect [2, 44, 45]. (The expectation isusually based on the frequency of connected subgraphsin a configuration model [3, 10, 19].) Other researchershave indicated that the search for frequent patterns innetworks was already a topic of interest in, for example,ecology in the 1970s [3].Milo et al. [10] compared several gene-regulatory net-works, a neural network for the worm
C. elegans , sev-eral food webs, several electronic circuits, and the World
Figure 5. Matching sets of structure motifs and matching sets of process motifs. On the left, we show four structure motifs andtheir sets of matching process motifs. On the right, we show four process motifs and their sets of matching structure motifs.
Wide Web. They viewed gene-regulatory networks andneural networks as systems “that perform informationprocessing” and reported that these networks have sim-ilar overrepresented connected subgraphs. They alsoreported that other networks, such as food webs andthe World Wide Web, did not have similar overrepre-sented subgraphs as the considered gene-regulatory net-works and the
C. elegans neural network. Subsequently,many researchers have studied various networks by iden-tifying overrepresented connected subgraphs (e.g., see[6, 7, 9, 11, 12]). In the corresponding publications, re-searchers used “motif” or “network motif” to refer to anoverrepresented connected subgraph, which are structuremotifs.Closely related to the idea of characterizing networksby examining overrepresented subgraphs is the idea ofcharacterizing networks based on the numbers or frequen-cies of one or several specified subgraphs [46–58]. For ex-ample, several researchers have used the number of trian-gles in an undirected network’s structure to characterizenetworks [46, 47] or to explain aspects of dynamics onthese networks [48, 49]. Others have used the numbersof different structure motifs with 3 or 4 nodes to com-pare networks [50–55] or to explain aspects of dynamicson them [56–58]. In some of these studies, researchershave considered “network motifs” to be small connectedsubgraphs without the requirement of overrepresentationwith respect to a null model [48, 55]. The “network mo-tifs” in these studies are also structure motifs.Estrada and Rodríguez-Velázquez [59] proposed a mea-sure of centrality that exploits the relationship betweenstructure motifs and process motifs in a network. Theircentrality measure, called “subgraph centrality”, is aweighted sum of closed walks that start and end ata node. Noting that “each closed walk is associated with a connected subgraph” [59], Estrada and Rodríguez-Velázquez concluded that one can use a weighted sum ofclosed walks that start and end at a node as a measureof the count of cyclic graphlets that include that node.Their rationale for proposing subgraph centrality thusmakes implicit use of the fact that each process motifthat consists of a single closed walk has a correspondingmatching structure motif that is a cycle.In several theoretical studies of dynamical systems onnetworks, researchers have used process motifs when in-terpreting the results of their derivations [23, 27, 41–43, 60, 61]. In theoretical neuroscience, a common ap-proach to connect network structure with system func-tions is to linearize nonlinear dynamical systems aboutan equilibrium point and consider the effect of small per-turbations on the dynamics. When deriving an approx-imation to the system state or a function of the systemstate for perturbations with a small prefactor (cid:15) , one cansometimes associate the terms of the approximation withprocess motifs. Researchers have used this approach tofind process motifs for “neural complexity” [23, 41, 62],information content [42], transfer entropy [43], cross-correlations [60], and other properties of stochastic dy-namical systems on networks [27, 61]. In these studies,the order of (cid:15) in the approximation indicates the lengthor duration of the corresponding process motif.Barnett et al. [23, 41] considered the mOUP on a net-work and derived an approximation for neural complexityup to third order in (cid:15) . They associated the terms of theirapproximation with “graph motifs” with up to 3 edges[41]. These graph motifs are process motifs of length (cid:96) ≤ . Lizier et al. [42] derived an approximation forthe information content of a multivariate Gaussian au-toregressive process on a network up to fourth order in (cid:15) and associated terms of the approximation with processmotifs with up to four edges. For the same dynamicalsystem, Novelli et al. [43] recently derived process mo-tifs for pairwise transfer entropy up to fourth order in (cid:15) . Trousdale et al. [60] derived an approximation forcross-correlations of a system of coupled integrate-and-fire neurons [63] to arbitrary order in (cid:15) . They associatedeach order of their approximation with a “submotif” thatincluded time-ordered edges. These submotifs are unionsof process motifs. Hu et al. [27] approximated a mea-sure of “global coherence” for the mOUP on a networkto arbitrary order in (cid:15) . They associated each order oftheir approximation with a normalized count (which theycalled a “motif cumulant”) of a so-called “ ( n, m ) motif”.The “ ( n, m ) motifs” are equivalent to the process motifsthat we derive for covariances of the mOUP in SectionIV. Jovanovic and Rotter [61] derived approximationsfor covariance and the third joint cumulant, which is ameasure of dependence between three variables, for a net-work of coupled Hawkes processes. They associated theirapproximation for covariance with 2-edge process motifsand their approximation of the third joint cumulant withprocess motifs with three or more edges.Other applications of dynamics on networks that arerelevant to the perspective of this paper include thespread of opinions [64] and the spread of infectious dis-eases [65]. In probabilistic compartment models on net-works, which are the most common type of model forstudying infectious diseases on networks, the probabilitythat a node is infected can depend on the infection prob-ability of other nodes [66]. For a subset of the nodes, it iscommon to approximate joint moments of infection prob-abilities by products of moments (if there is only a singlenode in the subset) or joint moments (if there are twoor more nodes in the subset) of the node(s) [66, 67]. Indoing so, one selects the joint moments of node-infectionprobabilities on some motifs — typically, connected pairsor connected triples of nodes — to be relevant for aspreading process and other joint moments to be negligi-ble [68, 69]. The motifs in these models can be processmotifs or structure motifs. Researchers have used DAGsto describe the spread of behavior, norms, and ideas [70]and the spread of infectious diseases [71, 72] on networks.One can view subgraphs of these so-called “disseminationtrees” [70] and “epidemic trees” [71, 72] as process motifs.For many studies of the spread of infectious dis-eases, either the choice of compartment model (e.g.,susceptible–infected–recovered [73, 74]) or the choice ofnetwork structure (e.g., if it is locally tree-like [75, 76])constrains the number of relevant process motifs suchthat, for structure motif with five or fewer edges, thereis only one relevant process motif that can occur on it.Because of this one-to-one correspondence between pro-cess motifs and structure motifs, the distinction betweenthem is irrelevant for these models of disease spread, pro-vided that one considers only small motifs (of five edgesor fewer). In Section V C, we discuss when the distinctionbetween process motifs and structure motifs is relevantand when it is not. III. USING PROCESS AND STRUCTUREMOTIFS TO STUDY FUNCTIONS OFDYNAMICS ON NETWORKS
In this section, we motivate the use of process motifsfor the study of dynamical systems on networks. We for-mally define contributions of process motifs and structuremotifs to real-valued functions of the state of a dynami-cal system on a network. We focus on linear dynamicalsystems. In general, one cannot use the same approachto directly study nonlinear dynamical systems, althoughone can apply our approach to linearizations of nonlineardynamical systems.
A. Linking process motifs to properties ofdynamics on networks
Consider a linear dynamical system d x t dt = F ( A ) x t , (1)where x t is a column vector that describes the currentsystem state, the system has the initial state x = x t =0 ,and F is a matrix-valued function of the adjacency ma-trix A of a network. Observables of the linear dynamicalsystem in Eq. (1) are functions of x t and F , and theyare thus functions of A and x . (For systems at steadystate or a system with uniformly distributed initial val-ues x = x = x = . . . , one can often remove thedependence on x and describe functions of the dynam-ical system as functions of only A .) One can thus viewa function of the linear dynamical system (1) as a super-position of walks or a superposition of walk graphs on anetwork.This view motivates the approach that we take in thepresent paper. We study how a function of a linear dy-namical system emerges via the superposition of processmotifs, which are structured sets of walks in an associatednetwork. After identifying relevant process motifs for agiven property of a dynamical system on a network, onecan establish links between dynamics on networks andnetwork structure by identifying the structure motifs onwhich the relevant process motifs can occur. This ap-proach results in a set of structure motifs that contributeto the desired system function along with explanationsof the mechanisms by which these structure motifs con-tribute to this function. When it is possible to quantifythe contribution of process motifs to the function of in-terest, one can also quantify the contribution of structuremotifs. In Section IV, we demonstrate our approach us-ing the covariances and the correlations in the mOUP atsteady state as examples. In the remainder of SectionIII, we explain how we formalize links between processmotifs and structure motifs. B. Contributions of process motifs and structuremotifs
Consider a scalar property Y = f ( A ) of a lineardynamical system on a network with adjacency matrix A . Assume that we have identified the relevant processmotifs p , p , . . . , p k and their real-valued contributions b p , b p , . . . , b p k to Y , so that, in a network on which p i has the count n p i , we can compute Y from Y = k (cid:88) i =1 b p i n p i , (2)which is a weighted sum of counts of process motifs.There are several ways that one can define a structuremotif’s contribution to Y . For example, one can definethe contribution c of a structure motif s to be the real-valued sum of all contributions b p of all process motifs p that can occur on s . This association is intuitive andtends to be computationally easy to obtain. For a lineardynamical system, one can compute c directly from c = f ( A (cid:48) ) , where A (cid:48) is the adjacency matrix of the structuremotif s . We call c the total contribution of a structuremotif s to Y .There are some disadvantages of using c to characterizea structure motif’s contribution to Y . For example, thereis no straightforward way of expressing Y as a weightedsum of counts of structure motifs. Moreover, if all b p i > , large structure motifs tend to contribute much moreto Y than small structure motifs, because large structuremotifs have much larger sets of process motifs that canoccur on them than small structure motifs.To address these two issues, we propose a different defi-nition for the contribution of a structure motif to Y . Thecontribution ˆ c of a structure motif s is the sum of contri-butions b p of process motifs p that can occur on s but noton any subgraph of s . We call ˆ c the specific contribution of a structure motif. One can express Y as the sum Y = (cid:88) i ˆ c s i n s i (3)of weighted counts n s i of structure motifs s i . A contribu-tion ˆ c s of a structure motif s is not necessarily larger thanthe contribution ˆ c t of a subgraph t of s . As we demon-strate in Section IV, the specific contribution ˆ c s tends tobe smaller than the specific contributions ˆ c t of subgraphs.A drawback of using ˆ c to characterize the contribution ofstructure motifs to Y is that specific contributions aremuch harder to compute than total contributions. Ingeneral, the computation of a specific contribution ˆ c s re-quires the computation of ˆ c t for all subgraphs t of s . Onecan compute the specific contribution of a structure motif s recursively via ˆ c s = c s − (cid:88) t ⊂ s ˆ c t , (4) where we use t ⊂ s to denote that t is a proper subgraphof s . Alternatively, one can use the mean total contribu-tions (cid:104) c (cid:105) m (cid:48) of subgraphs of s with m (cid:48) edges to compute ˆ c s . That is, ˆ c s = m (cid:88) m (cid:48) =1 (cid:18) mm (cid:48) (cid:19) (cid:88) q ∈Q ( m − m (cid:48) ) ( − | q | q ! (cid:104) c (cid:105) m (cid:48) , (5)where m is the number of edges in s , the set Q ( m − m (cid:48) ) is the set of integer compositions [77] of m − m (cid:48) , and thesequence q = ( q , q , . . . , q k ) is an integer compositionof m − m (cid:48) with k elements. In Eq. (5), we denote thenumber of elements in a sequence q by | q | and the multi-nomial coefficient of a sequence q of integers by “ q ! ”. Wederive Eq. (5) in Appendix A. For structure motifs with m > edges, it is computationally easier to calculation ˆ c from Eq. (5) than from Eq. (4). IV. COVARIANCE AND CORRELATION FORTHE MULTIVARIATE ORNSTEIN–UHLENBECKPROCESS
In this section, we demonstrate our process-based ap-proach for studying motifs in networks. As an exam-ple, we examine steady-state covariances and steady-state correlations in the mOUP. We derive contributionsof process motifs and total and specific contributions ofstructure motifs to covariances and correlations in themOUP at steady state. We then discuss the relation-ship between specific contributions of structure motifsand network mechanisms that contribute to steady-statecovariances and steady-state correlations in the mOUP.
A. The Ornstein–Uhlenbeck process
Uhlenbeck and Ornstein [78] proposed a stochasticprocess to describe Brownian motion under the influenceof friction. Among the many extensions of their model,the mOUP is a popular model for coupled noisy systems,including neuronal dynamics [23], stock prices [24], andgene expression [25]. In these studies, the mOUP with n variables describes the dynamics on a network with n nodes, where the state of each node represents a neuron,stock, or gene-expression level.One can describe the mOUP via d x t + dt = θ ( (cid:15) A − I ) x t dt + ς dW t , (6)where we use a column vector x t ∈ R N to describe thestate of the process. The process has an adjacency matrix A , which can be directed and/or weighted, and a mul-tivariate Wiener process W t . The reversion rate θ > ,the noise amplitude ς , and the coupling parameter (cid:15) > are parameters of the mOUP.We consider a signal to be a (temporary) deviation ofa node’s state from its mean. The coupling parameter0sets the rate at which a signal’s amplitude increases ordecreases when it is transmitted from one node to an-other. The parameter θ is the rate at which a signal’samplitude increases or decreases over time. It thus deter-mines the expected speed at which a node’s state revertsto its mean. Because of this connection between θ andthe speed of signal decay in the mOUP, many researchersrefer to θ as the reversion rate [79–81].If all eigenvalues of (cid:15) A − I have negative real parts, themOUP has a single stationary distribution. We then saythat the mOUP is a process with signal decay because,in this process, a signal’s amplitude decreases with time.Denoting the spectral radius of a matrix by ρ ( · ) , a suffi-cient condition for signal decay is ρ ( (cid:15) A ) < .The mOUP with signal decay is a Markov process. Itsstationary distribution is a multivariate normal distribu-tion N (0 , Σ ) centered at (cid:104) x (cid:105) = 0 with covariance matrix Σ := (cid:104) x t x Tt (cid:105) [23]. The mOUP with signal decay has thesteady-state covariance matrix Σ = ς θ ∞ (cid:88) L =0 ∞ (cid:88) (cid:96) =0 − L (cid:18) L(cid:96) (cid:19) ( (cid:15) A ) (cid:96) ( (cid:15) A T ) L − (cid:96) . (7)Barnett et al. [41] derived Eq. (7) for the mOUP with θ = ς = 1 . In Appendix B, we show that Eq. (7) alsoholds for arbitrary choices of θ > and ς > .In the remainder of this section, we derive and com-pare process motifs and structure motifs for covariance,variance, and correlation for the mOUP at steady state. B. Process motifs for covariance and correlation
We now derive process motifs and process-motif con-tributions for steady-state covariances and steady-statecorrelations in the mOUP.
1. Process motifs for covariance
We introduce the shorthand notation b L,(cid:96) := 2 − ( L +1) (cid:18) L(cid:96) (cid:19) ς θ and N L,(cid:96) := ( (cid:15) A ) (cid:96) ( (cid:15) A T ) ( L − (cid:96) ) to write Σ = ∞ (cid:88) L =0 L (cid:88) (cid:96) =0 (cid:15) L b L,(cid:96) N L,(cid:96) . (8)The ( i, j ) -th element of N L,(cid:96) corresponds to a count n p of process motifs p for the covariance between nodes i and j . The matrix N L,(cid:96) is not necessarily symmetric.However, the ( i, j ) -th element of N L,(cid:96) is equal to the ( i, j ) -th element of N L,L − (cid:96) .Equation (8) indicates that one can compute the co-variances of the mOUP as a weighted sum of counts ofprocess motifs. A process motif that contributes to the ji L − ‘‘ (a) i‘ L − ‘ (b) L i, − ‘ i, ‘ i, L j, − ‘ j, ‘ j, ‘ j, L j, − ‘ j, ji L − ‘ ‘ (c) Figure 6. Process motifs for (a) covariance, (b) variance, and(c) correlation. covariance between nodes i and j is a walk graph withthree nodes and two edges. Two of the walk-graph nodescorrespond to nodes i and j in the network. We callthese walk-graph nodes the focal nodes of this processmotif. All process motifs for covariance also include athird walk-graph node, which we call the “source node”and which can correspond to any node in a network. Eachedge in this process motif corresponds to a walk from thesource node to one of the two focal nodes. We show adiagram of a process motif that contributes to covari-ance in Fig. 6(a). One can characterize a process motifof this form by the two parameters L ∈ { , , , . . . } and (cid:96) ∈ { , . . . , L } . The parameter L is the length of a pro-cess motif; the parameter (cid:96) is the length of the walk fromthe source node to node i . The contribution of each pro-cess motif to the covariance is b L,(cid:96) .The process motifs for covariance are consistent withproperties of covariation in a system of coupled randomvariables. A covariance σ ij measures the joint “variabil-ity” of two random variables x i and x j [82], where onetakes variability to indicate a variable’s deviation fromits mean. This joint variability of x i and x j can arisefrom several causes [31]:1. Variability in x i induces variability in x j if there is a path from node i tonode j .2. Variability in x j induces variability in x i if there isa path from node j to node i .3. Variability in a third variable x k induces variabilityin both x i and x j if there are paths from k to i andfrom k to j .We now compare the contributions of different process1 ‘ L − ‘ N/A − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − l og ( b L , ‘ ) Figure 7. Contributions b L,(cid:96) of process motifs for covariancewith parameters ( L, (cid:96) ) for θ = 1 , ς = 1 , and (cid:15) = 0 . . Thelength L increases along the diagonal from the bottom left tothe top right. We indicate the parameter pairs for the largestcontributions for each value of L by bold labels and delineatethem with yellow line segments. motifs to covariance. In Fig. 7, we show the contribu-tions b L,(cid:96) of process motifs to covariance. The length L increases along the diagonal from the bottom left tothe top right. We indicate the parameter pairs with thelargest contribution for each value of L by bold labelsand delineate them with yellow line segments. For even L , contributions are maximal when (cid:96) = L/ . For odd L ,contributions are maximal when (cid:96) = ( L ± / . Com-paring the contributions of process motifs of differentlengths, we find that short process motifs (bottom left)tend to contribute more to covariances than long pro-cess motifs. These results are consistent with the notionthat covariances and correlations should decay with thedistance that a signal travels [83]. The result that a pro-cess motif with (cid:96) = L/ contributes more to covariancethan any other process motif with the same length L isconsistent with the notion that a signal that reaches twonodes i and j at the same time contributes more to thecovariance or correlation between i and j than signalsthat reach i and j at different times.
2. Process motifs for variance
A diagonal element of Σ indicates the variance ofa node in the mOUP. By merging the focal nodes inFig. 6(a), one obtains the process motifs that contributeto the variance of a node i (see Fig. 6(b)). Such a pro-cess motif includes two nodes and two edges. It includesa source node and a single focal node i . Its two edgescorrespond to two walks from the source node to node i . We write Σ = ς θ I + Σ (1+) , (9)where Σ (1+) := ∞ (cid:88) L =1 L (cid:88) (cid:96) =0 (cid:15) L b L,(cid:96) N L,(cid:96) , (10)to separate the variance contribution ς θ I (which is in-dependent of a network’s structure) from structure-dependent variance contributions Σ (1+) (which includesall terms of Eq. (8) that are O ( (cid:15) k ) , with k ≥ ). Weinterpret the two terms in Eq. (9) as indicators of twomechanisms by which variance arises in the mOUP:1. Gaussian white noise in each node induces the -th order contribution to variance. This effect con-tributes a value of ς / (2 θ ) to the variance of thestate variable x i at each node i . This contributionis determined by the noise strength ς and the meanreversion rate θ . It is independent of a network’sstructure.2. The variance of a state variable x i exceeds its noise-induced base value of ς / (2 θ ) when it receives inputfrom other nodes via in-edges or from itself via aself-edge. For a node i , these network-dependentcontributions are large when there are many pro-cess motifs for variance that have node i as theirfocal node. This is the case when node i is part ofmany cycles in a network or when many redundantpaths or trails in a network connect other nodes tonode i . Intuitively, cycles can reinforce variance ofa node i . Redundant paths or trails that lead tonode i can amplify input that i receives from othernodes.
3. Process motifs for correlation
One obtains the elements r ij of the correlation matrix R via r ij := σ ij / √ σ ii σ jj . (11)To replace the square root in the denominator of Eq. (11),we use the Taylor-series expansion √ x = ∞ (cid:88) k =0 ( − k · k (cid:18) kk (cid:19) x − k +12 ( x − x ) k , (12)which we obtain from expanding about the point x > .The radius of convergence of the expansion (11) is equalto x . We set x = ς / (2 θ ) and substitute Eq. (12) for / √ σ ii and / √ σ jj to obtain2 r ij = 2 θς σ ij ∞ (cid:88) k =0 ∞ (cid:88) k =0 (cid:18) − θ ς (cid:19) k + k (cid:18) k k (cid:19)(cid:18) k k (cid:19) (cid:18) σ ii − ς θ (cid:19) k (cid:18) σ jj − ς θ (cid:19) k = 2 θς σ ij ∞ (cid:88) k =0 ∞ (cid:88) k =0 (cid:18) − θ ς (cid:19) k + k (cid:18) k k (cid:19)(cid:18) k k (cid:19)(cid:16) σ (1+) ii (cid:17) k (cid:16) σ (1+) jj (cid:17) k , (13)where σ (1+) ii and σ (1+) jj are elements of Σ (1+) (see Eq. (10)). Equation (13) is a valid expression for r ij whenever thesums in Eq. (13) converge. Whenever Eq. (13) converges, we say that the mOUP has short-range signal decay. Wederive a sufficient condition for short-range signal decay in Appendix C.From Eq. (10), we see that one can express σ (1+) ii as a sum over the two indices L and (cid:96) . Consequently, one canexpress the k -th power of σ (1+) ii as a sum over the k indices L , (cid:96) , L , (cid:96) , . . . , L k , (cid:96) k . We use the multisets φ i := { ( L i, , (cid:96) i, ) , ( L i, , (cid:96) i, ) , . . . , ( L i,k , (cid:96) i,k ) } ,φ j := { ( L j, , (cid:96) j, ) , ( L j, , (cid:96) j, ) , . . . , ( L i,k , (cid:96) i,k ) } of pairs of indices to write r ij = 2 θς σ ij (cid:88) φ i ,φ j (cid:18) − θ ς (cid:19) | φ i | + | φ j | (cid:18) | φ i || φ i | (cid:19)(cid:18) | φ j || φ j | (cid:19) | φ i | (cid:89) k (cid:48) =1 b L i,k (cid:48) ,(cid:96) i,k (cid:48) ( N L i,k (cid:48) ,(cid:96) i,k (cid:48) ) ii | φ j | (cid:89) k (cid:48) =1 b L j,k (cid:48) ,(cid:96) j,k (cid:48) ( N L j,k (cid:48) ,(cid:96) j,k (cid:48) ) jj , where | φ i | denotes the number of pairs in φ i . We use (cid:80) φ i ,φ j to denote the double summation over all possiblemultisets of pairs ( L, (cid:96) ) of non-negative integers with (cid:96) ≤ L . We can thus express correlation as a weighted sum ofcounts of process motifs: r ij = (cid:88) L ,(cid:96) ,φ i ,φ j b L ,(cid:96) ,φ i ,φ j N L ,(cid:96) ,φ i ,φ j , where b L ,(cid:96) ,φ i ,φ j := 2 θς (cid:88) φ i ,φ j (cid:18) − θ ς (cid:19) | φ i | + | φ j | (cid:18) | φ i || φ i | (cid:19)(cid:18) | φ j || φ j | (cid:19) b L ,(cid:96) (cid:89) ( L,(cid:96) ) ∈ φ i b L,(cid:96) (cid:89) ( L,(cid:96) ) ∈ φ j b L,(cid:96) and N L ,(cid:96) ,φ i ,φ j := ( N L ,(cid:96) ) ij (cid:89) ( L,(cid:96) ) ∈ φ i ( N L,(cid:96) ) jj (cid:89) ( L,(cid:96) ) ∈ φ j ( N L,(cid:96) ) jj . The parameters L and (cid:96) and the parameter sets φ i and φ j characterize a process motif for correlation. Aprocess motif for correlation consists of a process motiffor covariance with focal nodes i and j , a number | φ i | ≥ of process motifs for variance with positive length andfocal node i , and a number | φ j | ≥ of process motifsfor variance with positive length and focal node j . Weshow a diagram of a process motif that contributes tocorrelation in Fig. 6(c). Both | φ i | and | φ j | can be equalto , so process motifs for covariance are also processmotifs for correlation.A contribution b L ,(cid:96) ,φ i ,φ j has a real non-zero value.All process motifs for correlation affect correlations, butnot all process motifs for correlation contribute positivelyto correlations. The magnitude of b L ,(cid:96) ,φ i ,φ j is propor- tional to the contributions b L,(cid:96) of the included processmotifs for variance and covariance. One can constructprocess motifs for correlation with large contributions b L ,(cid:96) ,φ i ,φ j from process motifs for variance and covari-ance with large contributions to b L,(cid:96) .To illustrate the effect of the number of included vari-ance process motifs on the contribution of a process mo-tifs to correlation, we show contributions of different pro-cess motifs to correlation in Fig. 8. We observe that thesign of b L ,(cid:96) ,φ i ,φ j is positive when the overall number ofincluded process motifs for variance is even and is neg-ative otherwise. The magnitude of b L ,(cid:96) ,φ i ,φ j decreasesas one adds more process motifs for variance at either ofthe two focal nodes ( i and j ). For a given process-motiflength, the process motifs that contribute most to corre-3 k i k j − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − . − . − . − . − . − . l og ( b L , ‘ , φ i , φ j ) − − − l og ( − b L , ‘ , φ i , φ j ) k i k j − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − l og ( b L , ‘ , φ i , φ j ) k i k j − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − l og ( − b L , ‘ , φ i , φ j ) ji } } k i times k j times Figure 8. Contributions b L ,(cid:96) ,φ i ,φ j of process motifs for correlation with parameters ( L , (cid:96) , φ i , φ j ) for θ = 1 , ς = 1 , and (cid:15) = 0 . . We show contributions for correlation process motifs that consist of a covariance process motif between node i andnode j , a number n i of variance process motifs at node i , and n j variance process motifs at node j . All of the process motifsfor variance and covariance have parameters L = 2 and (cid:96) = 1 . lation are process motifs that do not include any processmotifs for variance and are thus identical to process mo-tifs for covariance. The process motifs with the largestnegative contribution to correlation consist of a processmotif for covariance and one process motif for varianceat one of the focal nodes. C. Contributions of structure motifs to covarianceand correlation
We now link the process motifs from Section IV B 1 tonetwork structure. For graphlets of up to six edges, wecompute the total contribution c and their specific contri-butions ˆ c to covariance and correlation in the mOUP atsteady state. We first demonstrate that one can explainmost of the variation in the total contributions of struc-ture motifs using the total contributions of their sub-graphs. We then use the specific contributions of struc-ture motifs to infer mechanisms by which network struc-ture can contribute to covariance and correlation in themOUP, and we compare the efficiency of these mecha-nisms.
1. Total contributions of structure motifs to covariance
In Fig. 9, we show the m -edge structure motifs withthree largest total contributions to covariance for m ∈{ , , . . . , } . (Readers can explore the total and spe-cific contributions of further structure motifs using theJupyter notebook in the Supplementary Materials[84].)There are many aspects of the structure motifs for covari- ance and their total contributions that one can explore.We focus on two results: (1) one can explain almost theentire variation in c for structure motifs with m edgesusing the mean total contribution (cid:104) c (cid:105) m − of subgraphswith m − edges; and (2) process motifs are helpful forexplaining salient properties of the structure motifs inFig. 9. a. Total contributions of subgraphs explain a largeportion of the variation in the total contributions of struc-ture motifs. From Fig. 9, we see that, at least up to m = 6 , the three structure motifs with the largest to-tal contribution c are almost always supergraphs of the ( m − -edge structure motif with the largest total con-tribution c . This observation suggests that the total con-tributions of the subgraphs of a structure motif s have astrong influence on the total contribution of s . To inves-tigate the relationship between the total contributions ofstructure motifs and the the total contributions of theirsubgraphs, we compute the Pearson correlation coeffi-cient between c of structure motifs with m edges and themean total contribution of their subgraphs with m − edges. We show the correlation coefficients in Table I.We observe that one can explain almost all of the varia-tion in c using (cid:104) c (cid:105) m − . All of the correlation coefficientsin Table I are very large, and they increase with the num-ber m of edges and with the mOUP’s coupling parameter (cid:15) . b. Process motifs explain properties of structure mo-tifs with large total contributions. In thirteen of the six-teen structure motifs in Fig. 9, the focal nodes are con-nected bidirectionally. Twelve of the structure motifs inFig. 9 include self-edges. We first explain the high fre-quency of structure motifs with bidirectionally connected4 L a r g e s t c o n t r i bu t i o n ji c = 0.100 (a) One edge S ec o nd - l a r g e s t c o n t r i bu t i o n N/A (b) T h i r d - l a r g e s t c o n t r i bu t i o n N/A (c) ji c = 0.208 (d)
Two edges ji c = 0.139 (e) ji c = 0.111 (f) ji c = 0.263 (g)
Three edges ji c = 0.212 (h) ji c = 0.208 (i) ji c = 0.333 (j)
Four edges ji c = 0.268 (k) ji c = 0.267 (l) j ic = 0.339 (m)
Five edges j ic = 0.333 (n) ji c = 0.297 (o) ji c = 0.375 (p)
Six edges ji c = 0.362 (q) j ic = 0.352 (r)
Figure 9. Structure motifs that have the largest total contribution c to covariance between nodes i and j in the mOUP. Weround displayed values of c to the third decimal place. Each panel with a peach background shows an m -edge structure motifthat is a supergraph of the ( m − -edge structure motif with the largest total contribution c . m Covariance Correlation (cid:15) = 0 . (cid:15) = 0 . (cid:15) = 0 . (cid:15) = 0 . > . > . > . > . > . c of m -edge structure motifs to covariance in themOUP and the mean total contributions (cid:104) c (cid:105) m − of subgraphswith m − edges for different values of the mOUP couplingparameter (cid:15) . For all of the coefficients that we show, thep-values are less than − . focal nodes. Whenever a bidirectional edge between focalnodes is part of a structure motif s , the structure motif’stotal contribution c includes the contributions b L,(cid:96) of allprocess motifs with odd L . All of these process motifscan occur on s because walks can traverse the bidirec-tional edge several times. Process motifs with L = 1 areincluded in the total contribution of s . They contributemuch more to covariance than other process motifs, so s tends to contribute more to covariance than other struc-ture motifs with the same number of edges. For the samereason, a supergraph s (cid:48)(cid:48) of s tends to contribute moreto covariance than other structure motifs with the samenumber of edges as s (cid:48)(cid:48) .To explain the high frequency of self-edges in structure ji (a) ji (b) Figure 10. Effect of including a self-edge in a structure motif.In (a), we show a structure motif s with one edge. The curvedblue edges with numerical labels indicate the only processmotif that can occur on s . In (b), we show s with an additionalself-edge at node i . The curved blue edges with numericallabels indicate one of many process motifs that can occur onthis structure motif. motifs with large total contributions to covariance, weconsider a structure motif s that does not have a self-edgeat either focal node. The inclusion of a self-edge at a focalnode in s yields a structure motif s (cid:48)(cid:48) that is a supergraphof s . We show a simple example of a structure motif s and a corresponding supergraph s (cid:48)(cid:48) in Fig. 10. Thereare at least twice as many process motifs that can occuron s (cid:48)(cid:48) than there are process motifs that can occur on s because for every length- L process motif that can occuron s , there is a length- ( L + 1) process motif that canoccur on s (cid:48)(cid:48) but not on s . This example demonstratesthat the inclusion of self-edges in a structure motif can5greatly increase its total contribution to covariance.The high frequencies of self-edges and edges betweenfocal nodes in structure motifs that contribute the mostto covariance and correlation suggest that signal trans-mission via short paths between focal nodes and signalamplification via short cycles are important for mecha-nisms by which network structure can contribute to co-variances of the mOUP.
2. Specific contributions of structure motifs to covariance
In Section III, we proposed to separate the total con-tribution of a structure motif s into a large part that onecan attribute to subgraphs of s and a small part thatone cannot attribute to subgraphs of s . This small part ˆ c is the specific contribution of s . The specific contri-butions of structure motifs with one edge indicate thecontribution to covariance of a single edge. The specificcontributions of structure motifs with two edges indicatethe contribution to covariance of a pair of edges minusthe specific contributions of each of the two edges alone.Whenever the specific contribution of a structure motif ispositive, the structure motif indicates a mechanism or acombination of mechanisms by which network structurecan enhance covariance. a. Structure motifs with ˆ c > indicate mecha-nisms for structure-based enhancement of covariance. In Fig. 11, we show structure motifs with one or two edgesand their specific contributions ˆ c to covariance. Panels(a)–(g) have blue backgrounds and show structure mo-tifs with a positive ˆ c . These structure motifs indicatemechanisms for enhancing covariance in the mOUP. Inpanel (h), we include a graphlet that is not a structuremotif because it has two components. We include it be-cause it is helpful for discussing the mechanisms by whichnetwork structure can contribute to covariance in themOUP. The positive specific contributions for structuremotifs in panels (a) and (e) indicate that signal trans-mission via short paths from one focal node to the othercan increase covariance. The positive specific contribu-tions for structure motifs in panels (b) and (d) indicatethat signal amplification via a length- cycle can increasecovariance when combined with a path for signal trans-mission between focal nodes. The specific contributionof the graphlet in panel (h) is , which indicates that,without any connection between focal nodes, signal am-plification at a focal node does not increase covariance.In panel (f), the bidirectional edge between focal nodesenables signal transmission from either focal node to theother. It also creates a 2-cycle at each focal node. Thepositive specific contributions for structure motifs in pan-els (c) and (g) indicate that a signal transmission from anon-focal node can contribute to covariance. Comparingpanels (c) and (g) to panel (k), we see that a positive ˆ c requires that there are paths from the non-focal node toboth focal nodes. The contributions of the structuremotifs in panels (i), (j), and (l) indicate that paths from ji ˆ c = 0.100 (a) ji ˆ c = 0.039 (b) ji ˆ c = 0.020 (c) ji ˆ c = 0.011 (d) ji ˆ c = 0.010 (e) ji ˆ c = 0.008 (f) ji ˆ c = 0.003 (g) j i ˆ c = 0.000 (h) ji ˆ c = 0.000 (i) j i ˆ c = 0.000 (j) j i ˆ c = 0.000 (k) ji ˆ c = 0.000 (l) Figure 11. Structure motifs with one or two edges and theirspecific contributions ˆ c to covariance. We round displayedvalues of c to the fourth decimal place. Panels (a)–(g) haveblue backgrounds and show structure motifs with a positive ˆ c . focal nodes to other nodes are not relevant for covariancebetween focal nodes.From these observations, we conclude that two mech-anisms for increasing covariance in the mOUP are (1)signal transmission via paths from one focal node to an-other and (2) signal transmission from non-focal nodesthat are connected to both focal nodes via paths fromthe non-focal node to each focal node. Other mechanismsfor increasing covariance in the mOUP are combinationsof signal transmission via paths between focal nodes andsignal transmission from non-focal nodes. Such mecha-nisms can also be combinations of either or both mecha-nisms with signal amplification via short cycles at focalnodes or other nodes. b. Specific contributions indicate the efficiency ofmechanisms. Thus far, we have used specific contribu-tions to distinguish structure motifs with ˆ c > and thuscontribute to covariance from structure motifs with ˆ c = 0 that thus do not contribute to covariance. We can use thevalue of specific contributions to define a measure of effi-ciency of a mechanism that is associated with a structuremotif. For a structure motif with m edges and specific6contribution ˆ c , we define the efficiency η := ˆ c/m . From Fig. 11, we see that specific contributions andthus η tend to decrease with the number of edges in astructure motif. The mechanisms with large efficiencytend to be associated with small structure motifs. Thestructure motif with the largest ˆ c for covariance (seeFig. 11(a)) indicates direct signal transmission (i.e., sig-nal transmission via a length- path) as a mechanismfor increasing covariance. The associated efficiency is η ≈ . . All other mechanisms have much smaller ef-ficiencies than direct signal transmission. For example,signal transmission via a length- path (see Fig. 11(e))has η ≈ . , and ones through longer paths are evensmaller.When the focal nodes are connected by a single di-rected path, one can think of the focal node with posi-tive out-degree as the “sender” node and the node withpositive in-degree as the “receiver” node. The second-most efficient mechanism is a combination of direct signaltransmission and signal ampflication via a length- cycleat the sender node (see Fig. 11(b)). This mechanism hasan efficiency of η ≈ . . The efficiency of direct signaltransmission with signal amplification via a length- cy-cle at the receiver node (see Fig. 11(d)) has an efficiencyof η ≈ . , which is almost four times smaller thanthe previous mechanisms. Transmission of signals froma third node to both focal nodes via length- paths (seeFig. 11(c)) has an efficiency of η ≈ . . c. Matching motifs give a heuristic way to explainspecific contributions. For the mechanisms that are as-sociated with -edge and -edge structure motifs, one canexplain the ranking of specific contributions using match-ing process motifs. The total contribution of structuremotif s is the sum of contributions of process motifs p that can occur on s . If p contributes to the specific con-tribution ˆ c of s , the process motif p uses each edge in s at least once because otherwise p would contribute to thespecific contribution of a proper subgraph of s . Contribu-tions of process motifs tend to decrease with the lengthof process motifs. Therefore, the largest contributionsof process motifs to ˆ c of s come from matching processmotifs of s . One can use the sum γ := (cid:88) p ∈ P S b ( p ) of contributions of matching process motifs as a heuristicfor estimating ˆ c of s . We show the Pearson correlation co-efficients for ˆ c and γ for different structure-motif lengthsin Table II. For comparison, we also show a second heuris-tic γ (cid:48) = max p ∈ P S b ( p ) that only uses the contribution ofthe process motifs on s that contributes the most to co-variance. We observe a large positive correlation between ˆ c and γ for all considered structure-motif lengths. Theheuristic γ is correlated most strongly with ˆ c when both m and (cid:15) are small. The heuristic γ (cid:48) also has a large pos- itive correlation with ˆ c for m = 2 . However, as we con-sider s with progressively more edges, the Pearson corre-lation coefficient between s and γ (cid:48) decreases much fasterthan the Pearson correlation coefficient between s and γ .This difference between the two heuristics demonstratesthat, for structure motifs with more than two edges, it isimportant to consider all matching process motifs withjust one matching process motif. m Covariance Correlation (cid:15) = 0 . (cid:15) = 0 . (cid:15) = 0 . (cid:15) = 0 . γ and γ (cid:48) with specific contribution ˆ c of m -edge structure motifsto covariances of the mOUP with a coupling parameter of (cid:15) .For all of these Pearson correlation coefficients, the p-valuesare less than . .
3. Specific contributions of structure motifs to correlation
We demonstrated in Section IV C that specific contri-butions of structure motifs indicate covariance-enhancingmechanisms more clearly than total contributions. Inthis section, we focus on specific contributions of struc-ture motifs to correlations in the mOUP. In Fig. 12, weshow the m -edge structure motifs with the three largestspecific contributions to correlation for m ∈ { , , . . . , } .(Readers can explore the total and specific contributionsof further structure motifs using the Jupyter notebook inthe Supplementary Materials[84].) a. Network structure can increase or decrease corre-lations. In Fig. 13, we show structure motifs with one ortwo edges and their specific contributions ˆ c to correlationin the mOUP.Negative specific contributions to correlation in themOUP indicate that there are mechanisms by which net-work structure can decrease the correlation between apair of nodes in the mOUP. The structure motifs withnegative ˆ c for correlation include structure motifs thathave specific contribution to covariance. An exampleis the structure motif in Fig. 13(j). Its specific contribu-tion to covariance is ˆ c = 0 , from which we concluded inSection IV C 2 that signal transmission from a non-focalnode to only one focal node does not increase covariancein the mOUP. The same structure motif has a negativespecific contribution to correlation in the mOUP. There-fore, we conclude that signal transmission from a non-focal node to a single focal node can decrease correlationin the mOUP.The structure motifs with negative ˆ c for correlation7 L a r g e s t c o n t r i bu t i o n ji ˆ c = 0.09806 (a) One edge S ec o nd - l a r g e s t c o n t r i bu t i o n N/A (b) T h i r d - l a r g e s t c o n t r i bu t i o n N/A (c) ji ˆ c = 0.01923 (d) Two edges ji ˆ c = 0.01007 (e) ji ˆ c = 0.00980 (f) ji ˆ c = 0.00708 (g) Three edges ji ˆ c = 0.00288 (h) ji ˆ c = 0.00230 (i) ji ˆ c = 0.00124 (j) Four edges ji ˆ c = 0.00080 (k) ji ˆ c = 0.00058 (l) ji ˆ c = 0.00056 (m) Five edges ji ˆ c = 0.00025 (n) ji ˆ c = 0.00021 (o) ji ˆ c = 0.00012 (p) Six edges ji ˆ c = 0.00011 (q) ji ˆ c = 0.00009 (r) Figure 12. Structure motifs that have the largest specific contribution ˆ c to the correlation between nodes i and j in the mOUP.We round the displayed values of ˆ c to the fifth decimal place. Each panel with a peach background shows an m -edge structuremotif that is a supergraph of the ( m − -edge structure motif with the largest specific contribution ˆ c . also include structure motifs that have a positive ˆ c for co-variance. An example is the structure motif in Fig. 13(l).In Section IV C 2, we concluded that direct signal trans-mission with signal amplification at the receiver node is amechanism by which network structure can increase co-variance in the mOUP. From the structure motif’s nega-tive specific contribution to correlation, we conclude that,by the same mechanism, network structure can decreasecorrelation in the mOUP. b. The ranking of mechanisms by efficiency is dif-ferent for covariance and correlation. Comparing thespecific contributions to covariance and correlation fora given structure motif, we see that the contributions arealmost identical for some structure motifs. For example,the structure motif in Fig. 13(a) has ˆ c ≈ . for co-variance and ˆ c ≈ . for correlation. Another exampleis the structure motif in Fig. 13(b). It has ˆ c ≈ . forcovariance and ˆ c ≈ . for correlation. For other struc-ture motifs, the specific contributions to correlation aremuch smaller than their specific contributions to covari-ance. For example, the structure motif in Fig. 13(c) has ˆ c ≈ . for covariance and ˆ c ≈ . for correlation.This difference between the specific contributions tocovariance and correlation is related to the process mo-tifs for variance in focal nodes. Structure motifs on whichfew long process motifs for focal-node variance node canoccur tend to have very similar specific contributionsto covariance and correlation. Examples of such struc-ture motifs are the ones in Fig. 13(a) and (b). For otherstructure motifs, the specific contributions to covariance and correlation are very different. The structure mo-tif in Fig. 13(c) is an example of this observation. Dueto these differences, ranking structure motifs by specificcontribution to covariance and ranking structure motifsby their specific contribution to correlation leads to dif-ferent rankings. Consequently, the rankings based on ef-ficiency of the associated mechanisms are also different. c. Increasing the in-degree of a receiver node reducescorrelations in locally tree-like networks. The structuremotifs in Figs. 13(e), (g), (h), and (j) include a directededge between focal nodes and an in-edge or out-edge atthe sender node or the receiver node. We use these struc-ture motifs and their specific contributions to correlationto study the effect on increasing the in-degree or out-degree of focal nodes on correlations in a locally tree-likenetwork [85]. For locally tree-like networks, one assumesthat neighbors of a node are not neighbors of each other.We also assume that we can neglect the structure motifswith more than two edges because such structure motifstend to have very small values of ˆ c (see Fig. 12). Underthese assumptions, we note the following:1. an increase of the in-degree of a sender node leadsto an increase of the count of the structure motifin Fig. 13(e) but no others,2. an increase of the out-degree of a sender node leadsto an increase of the count of the structure motifin Fig. 13(g) but no others,3. an increase of the in-degree of a receiver node leadsto an increase of the count of the structure motifs8 ji ˆ c = 0.098 (a) ji ˆ c = 0.019 (b) ji ˆ c = 0.010 (c) ji ˆ c = 0.010 (d) ji ˆ c = 0.001 (e) j i ˆ c = 0.000 (f) ji ˆ c = 0.000 (g) j i ˆ c = 0.000 (h) ji ˆ c = 0.000 (i) j i ˆ c = − (j) ji ˆ c = − (k) ji ˆ c = − (l) Figure 13. Structure motifs with one or two edges and theirspecific contributions ˆ c to correlation. We round displayedvalues of c to the fourth decimal place. Panels (a)–(e) showstructure motifs with a positive ˆ c and have blue backgrounds.Panels (j)–(l) show structure motifs with a negative ˆ c andhave pink backgrounds. in Fig. 13(h) but no others, and4. an increase of the out-degree of a receiver nodeleads to an increase of the count of the structuremotifs in Fig. 13(j) but no others.One can infer the effect of increasing in-degree or out-degree of the sender node or the receiver node from thespecific contributions of these structure motifs. Increas-ing the out-degree of the sender node (see Fig. 13(g)) orthe receiver node (see Fig. 13(f)) does not affect the cor-relation between the sender and the receiver. Increasingthe in-degree of the the sender node (see Fig. 13(e)) hasa small positive effect on correlation. Increasing the in-degree of the receiver node (see Fig. 13(j)) has a negativeeffect on correlation.When two focal nodes are connected bidirectionally,one cannot distinguish between a sender node and a re-ceiver node. Increasing the in-degree of either focal nodeincreases the counts of the structure motif in Fig. 13(e)and the structure motif in Fig. 13(j) by 1 each. The neteffect of increasing the in-degree of a focal node is givenby the sum of the specific contributions of the structuremotifs in Fig. 13(e) and (j) and is negative. Increasing the in-degree of a node in a locally tree-like network thusreduces the correlation between this node and nodes withwhich it is connected bidirectionally. V. CONCLUSIONS AND DISCUSSION
Discovering connections between dynamics on net-works and network structure is an ongoing endeavor ofresearchers in many disciplines. Many researchers haveconsidered it helpful to decompose networks into struc-tural building blocks, which are typically called "motifs".In the present paper, we demonstrated that combiningsuch a decomposition of a network’s structure into struc-ture motifs with a decomposition of processes on a net-work into process motifs can yield both mechanistic andquantitative insights into connections between dynam-ics on networks and network structure. To construct aframework for the combined decomposition of processeson networks and network structure, we introduced walkgraphs as “building blocks” of processes and defined con-tributions of process motifs and total contributions andspecific contributions of structure motifs to observablesof dynamical systems on networks.
A. Mechanisms for enhancing covariance andcorrelation in the Ornstein–Uhlenbeck process
To demonstrate our framework, we performed a com-bined decomposition into process and structure motifs forthe multivariate Ornstein–Uhlenbeck process (mOUP)on a network. We identified the process motifs that con-tribute to variances, covariances, and correlations in themOUP at steady state. We then used the contributions ofthe identified process motifs to variance, covariance, andcorrelation to explain the total contributions and specificcontributions of structure motifs to covariance and cor-relation. The specific contributions of structure motifsindicate several mechanisms by which network structurecan enhance the covariance or the correlation betweentwo focal nodes in the mOUP at steady state. We foundthat edges in structure motifs can positively contributeto covariance and correlation by1. enhancing signal transmission between two focalnodes,2. signal amplification at either focal node, and/or3. signal transmission from other nodes to both focalnodes.Structure motifs contribute positively to covariance orcorrelation by enhancing signal transmission between fo-cal nodes; signal transmission from non-focal nodes; or acombination of signal transmission between focal nodes,signal transmission from non-focal nodes, and signal am-plification at focal nodes or non-focal nodes. The ranking9of structure motifs and associated mechanisms by specificcontributions is different for covariance and correlation,and it depends on the coupling parameter (cid:15) of the mOUP.Some of our results on process motifs and structuremotifs for covariance and correlation for the mOUP maymatch a reader’s intuition for covariance and correlation.For example, the popular phrase “correlation does notimply causation” is consistent with our results that (1)process motifs for covariance or correlation between twonodes i and j do not necessarily include a walk from i to j or from j to i and (2) structure motifs for covarianceor correlation do not necessarily include a path or trailfrom i to j or from j to i . Our results confirm known re-sults about the mechanisms by which network structurecan affect covariances and correlations between variables,and they also offer new quantitative insights into the effi-ciency of these mechanisms and the relationship betweenefficiency and the parameters of the mOUP. B. Applicability to other dynamical systems
In the present paper, we studied covariance and corre-lation in a simple stochastic dynamical system at steadystate. We chose this example for illustrative purposesand to demonstrate that our approach can confirm andextend intuition about the network mechanisms that con-tribute to system function. It is possible to apply ourframework to other system functions, other linear dy-namical systems, and away from a steady state. (Fordynamical systems that are away from a steady state,process-motif decompositions of system functions can de-pend on motif counts and initial conditions.)We considered structure motifs in directed networkswith self-edges. For some systems in biology, chemistry,sociology, and other areas, it can be more appropriate toconsider undirected networks or networks without self-edges than to consider directed networks with self-edges.One can apply our framework to these networks by fo-cusing motif comparison to structure motifs in undirectednetworks or networks without self-edges. Because of theflexibility of our approach, we anticipate that the studyof process motifs can yield insights into many open prob-lems in the study of dynamical systems on networks.
C. When does the distinction between processmotifs and structure motifs matter?
The distinction between process motifs and structuremotifs matters for many but not all dynamical systems.Our core motivation for distinguishing between processmotifs and structure motifs is that a walk on a networkand a path in a network are two fundamentally differentconcepts. A walk can use an edge in a network severaltimes, whereas a path or trail can include each edge onlyonce. When one defines a process on a network such thatit can use each node only once, the distinction between walks and paths becomes unnecessary because every pathcorresponds to a single walk.Examples of relevant dynamical processes includesusceptible–infected (SI) models and susceptible–infected–recovered (SIR) models for the spread of aninfectious disease [66, 86], because infected and recoveredindividuals in these models cannot become infected asecond time; consequently, a disease can spread alongeach edge at most once. One can construct othermodels that allow recurring infections (i.e., an individualcan become infected multiple times). Examples ofsuch models are susceptible–infected–susceptible (SIS)models and susceptible–infected–recovered–susceptible(SIRS) models. For such models, it is important todistinguish between process motifs and structure motifs.One can circumvent the need to make a distinctionby introducing restrictive model assumptions that arepopular in the modeling of infectious diseases [66]. Forexample, one can assume that1. a network is a directed acyclic graph (DAG) or2. a network is directed and locally tree-like and thatthe infection rates are low.On a DAG, there are no process motifs that use an edgein a network more than once. Under assumption (1), thedistinction between process motifs and structure motifsdoes not matter. In networks that are both directed andlocally tree-like, there are no process motifs of length L ≤ that use an edge in a network more than once.A low infection rate ensures that contributions of longprocess motifs are very small. Under assumption (2), thedistinction between process motifs and structure motifsthen has only a small effect on the specific contributionsof structure motifs. We anticipate that distinguishingbetween process motifs and structure motifs can aid re-searchers in the study of diseases on networks with mod-els in which recurring infections are possible.When a network is a DAG, walks on it cannot usean edge more than once. The distinction between pro-cess motifs and structure motifs is thus not relevant forany dynamical system on a DAG. There are numerousapplications of dynamical systems on DAGs in machinelearning and neuroscience [87, 88]. They include feed-forward artificial neural networks and models of naturalneural networks in the visual cortex of several species[89, 90]. Many researchers in machine learning and neu-roscience have highlighted the fundamental differences inthe dynamics of non-recurrent neural networks (i.e., neu-ral networks that are DAGs) and recurrent networks (i.e.,networks that are not DAGs) [87, 91]. We anticipatethat our framework for decomposing processes on net-works into process motifs can help explain some of thesedifferences between non-recurrent and recurrent neuralnetworks.Dynamical systems on temporal networks are anotherexample for which one can sometimes ignore the distinc-tion between process motifs and structure motifs. One0can define many temporal networks such that each edgeis active only at a specified point in time or during a spec-ified time interval [92]. When each edge in a temporalnetwork is active only at very few times points or onlyfor time intervals that are short compared to the tem-poral scales of processes on a network, few or no walkson the temporal network use an edge more than once.For dynamical systems on temporal networks for whichthe edge dynamics are comparably as fast as or fasterthan the dynamics on the network, it is possible thateach structure motif has only a few associated processmotifs. (See [86] for a discussion of the relative tem-poral scales of dynamics on networks and dynamics ofnetworks.) The development of new notions of structuremotifs in temporal networks is an active field of research,and researchers have made several proposals for notionsof structure motifs in temporal networks [93–96]. Thedistinction between process motifs and structure motifsmay be helpful for assessing these proposals and the de-velopment of further notions of motifs on temporal net-works. D. “Unbiased” mechanistic insights from processmotifs and structure motifs
In this paper, we presented an approach for identifyinggraphlets that are relevant to a function of a system. Ourapproach offers several advantages over traditional ap-proaches, in which researchers use overrepresentation ofgraphlets to conclude the relevance of graphlets to a sys-tem function. Those approaches depend strongly on thechoice of a random-graph null model [17–19], and theydo not identify mechanisms by which overrepresentedgraphlets affect a system function. Studies of dynami-cal systems on graphlets in isolation require researchersto choose a graphlet and a candidate mechanism a pri-ori . The reliance on these choices makes such studiesprone to bias towards graphlets or mechanisms that a re-searcher has chosen to study. For example, many studieshave reported the relevance of feedback loops and feed-forward loops for various system functions [2]. However,it is unclear if these two graphlets are generally more im-portant for system functions than other graphlets or ifresearchers have associated them more frequently thanother graphlets with system function because they havestudied them more often.Our approach identifies all structure motifs with a pos-itive (or negative) contribution to a given function of adynamical system. The approach is unbiased in the sensethat its results do not depend on an a priori choice ofa graphlet or a mechanism. Our results for covarianceand correlation in the mOUP demonstrate that therecan be many structure motifs that affect a system func-tion. Had we considered only a single graphlet in ourstudy, it is likely that we would have concluded that thatgraphlet affects covariance and correlation in the mOUPand would then have inferred that that graphlet is impor- tant for these system functions. Our systematic study ofall graphlets with up to six edges enabled us to rankstructure motifs based on their contribution strengthsand made it possible to distinguish between structuremotifs that strongly affect covariance and correlation inthe mOUP and structure motifs that have smaller (oreven negligible) contributions to these system functions.We also demonstrated how one can conduct a com-bined decomposition of dynamics on a network and net-work structure into process motifs and structure motifs.One can use such a decomposition to identify structuremotifs that contribute the most to a given system func-tion and to explain how these structure motifs contributeto the system function. We demonstrated that it can beuseful to consider dynamics on a network (instead of justa network’s structure) as a composite object that one candecompose into many small parts. Our proposed frame-work thereby provides an opportunity to develop insightsinto mechanisms by which dynamics and network struc-ture affect system functions.
ACKNOWLEDGEMENTS
We thank Alex Arenas, Lionel Barnett, Heather ZinnBrooks, Bing Brunton, Michelle Feng, Kameron DeckerHarris, Renaud Lambiotte, Neave O’Cleary, Gesine Rein-ert, and Jonny Wray for helpful discussions and com-ments. A.C.S. was supported by the Engineering andPhysical Sciences Research Council under grant numberEP/L016044/1, the Clarendon Fund, and e-Therapeuticsplc. M.A.P. acknowledges support from the National Sci-ence Foundation (grant number 1922952) through the Al-gorithms for Threat Detection (ATD) program.
Appendix A: Derivation of a non-recursive formulafor the specific contributions of structure motifs
Consider a structure motif s with m edges with specificcontribution ˆ c s . Successive recursions of Eq. (4) lead toan expression that depends only on the total contribu-tions c t of subgraphs t of s . Subgraphs with the samenumber m (cid:48) < m of edges contribute to ˆ c s in the sameway. One can thus write ˆ c s = α (cid:104) c (cid:105) + α (cid:104) c (cid:105) + α (cid:104) c (cid:105) + . . . + α m − (cid:104) c (cid:105) m − + α m (cid:104) c (cid:105) m , (A1)where α m (cid:48) , where m (cid:48) ∈ { , . . . , m } , are integer coeffi-cients and (cid:104) c (cid:105) m (cid:48) is the mean total contribution of sub-graphs with m (cid:48) edges. The structure motif s has ex-actly one subgraph (specifically, the graph itself) with m edges, so (cid:104) c (cid:105) m = c s . From Eq. (4), one can see that α m = 1 in the “0-th” recursion of Eq. (4). Further re-cursions of Eq. (4) do not change α m because subgraphswith m edges are not proper subgraphs of s and thus donot appear in the sum over proper subgraphs t ⊂ s in1Eq. (4). The first recursion of Eq. (4) yields ˆ c s = c s − (cid:88) t ⊂ s (cid:32) c t − (cid:88) t ⊂ t ˆ c t (cid:33) . (A2)From Eq. (A2), one can see that α m − is equal to thenegative of the number of subgraphs of s with m − edgesin the first recursion of Eq. (4). Further recursions ofEq. (4) do not change α m − , because subgraphs with m − edges cannot be proper subgraphs of proper subgraphs t of s and thus do not appear in the sum over propersubgraphs t ⊂ t . It thus follows that α m − = − (cid:18) mm − (cid:19) . Subgraphs with m − edges are proper subgraphs of s . We thus obtain α m − = − (cid:0) mm − (cid:1) in the first recur-sion of Eq. (4). Because subgraphs with m − edges arealso proper subgraphs of proper subgraphs t of s , thesecond recursion of Eq. (4) leads to an additional term (cid:0) mm − (cid:1)(cid:0) m − m − (cid:1) in α m − . Further recursions of Eq. (4) donot change α m − . It thus follows that α m − = − (cid:18) mm − (cid:19) + (cid:18) mm − (cid:19)(cid:18) m − m − (cid:19) Similar considerations lead to α m − = − (cid:18) mm − (cid:19) + (cid:18) mm − (cid:19)(cid:18) m − m − (cid:19) + (cid:18) mm − (cid:19)(cid:18) m − m − (cid:19) + (cid:18) mm − (cid:19)(cid:18) m − m − (cid:19)(cid:18) m − m − (cid:19) . We see that each coefficient α m (cid:48) includes one or severalproducts of binomial coefficients. Each of these productshas the form (cid:18) mm (cid:19)(cid:18) m m (cid:19) . . . (cid:18) m k − m k − (cid:19)(cid:18) m k − m (cid:48) (cid:19) for some k ≤ m − m (cid:48) . Such a product of binomial coef-ficients corresponds to the number of ways that one canpartition the edge set of s into k subsets with sizes m − m , m − m , . . . ,m k − − m k − , m k − − m (cid:48) , m (cid:48) . The coefficient α m (cid:48) includes one such term for each inte-ger composition of m that includes m (cid:48) . It follows that α m (cid:48) = (cid:88) q ∈Q ( m − m (cid:48) ) ( − | q | (cid:18) mm (cid:48) (cid:19) q ! , (A3)where Q ( m − m (cid:48) ) is the set of integer compositions q =( q , q , . . . , q k ) of m − m (cid:48) and q ! := ( q , q , . . . , q k )! is themultinomial coefficient for the sequence ( q , q , . . . , q k ) of k integers. We use | q | := k to denote the number ofintegers in an integer composition q . Substituting the co-efficients α m (cid:48) into Eq. (A1) with Eq. (A3) yields Eq. (5). Appendix B: Derivation of the covariance matrix
At time t + dt , the state vector of the mOUP with ad-jacency matrix A , coupling parameter (cid:15) , noise amplitude ς , and reversion rate θ is x t + dt = Kx t + ς dW t , (B1)where K = I + θ ( (cid:15) A − I ) dt . At steady-state, the mOUPhas the covariance matrix Σ = (cid:104) x t x Tt (cid:105) = (cid:104) x t + dt x Tt + dt (cid:105) , (B2)where (cid:104)·(cid:105) denotes an ensemble average. Using Eq. (B1),we substitute x t + dt into Eq. (B2) to obtain Σ = (cid:104) ( Kx t + ς dW t )( Kx t + ς dW t ) T (cid:105) = (cid:104) Kx t x Tt K T + ς I dt (cid:105) , (B3)where the second equality follows from the fact that dW t is a mean-0, unit-variance stochastic process that is in-dependent of x t . Evaluating the ensemble average inEq. (B3) yields Σ = KΣK T + ς I dt = [ I + θ ( (cid:15) A − I ) ( dt ) ] Σ [ I + θ ( (cid:15) A − I ) ( dt ) ] T + ς I ( dt ) = Σ + θ (cid:20) ( (cid:15) A − I ) Σ + Σ ( (cid:15) A − I ) T + ς θ I (cid:21) dt + O (( dt ) ) . To first order in dt , we thus have (cid:15) A − I ) Σ + Σ ( (cid:15) A − I ) T + ς θ I . (B4)Equation (B4) is a Lyapunov equation [97, 98]. For themOUP with signal decay, the solution of Eq. (B4) is [97] Σ = ς θ (cid:90) ∞ e ( (cid:15) A − I ) t e ( (cid:15) A T − I ) t dt = ς θ Σ , (B5)where Σ := (cid:90) ∞ e ( (cid:15) A − I ) t e ( (cid:15) A T − I ) t dt is the covariance matrix of the mOUP when ς = θ = 1 .For ς = θ = 1 , Barnett et al. [23, 41] derived the covari-ance matrix as a sum of products of A and A T [99]: Σ = ∞ (cid:88) L =0 − ( L +1) L (cid:88) (cid:96) =0 (cid:18) L(cid:96) (cid:19) ( (cid:15) A ) (cid:96) ( (cid:15) A T ) L − (cid:96) . (B6)Therefore, Σ = ς θ ∞ (cid:88) L =0 − ( L +1) L (cid:88) (cid:96) =0 (cid:18) L(cid:96) (cid:19) ( (cid:15) A ) (cid:96) ( (cid:15) A T ) L − (cid:96) . (B7)2 Appendix C: Conditions for short-range signal decay
The sums in Eq. (13) converge if the matrix Σ = θς Σ has eigenvalues ν i =1 ,...,n ∈ (0 , [100, p. 38]. The co-variance matrix Σ is a symmetric positive semidefinitematrix. A sufficient condition for short-range signal de-cay thus only needs to constrain the largest eigenvalue of Σ .First, we show that a sufficient condition for short-range signal decay is (cid:107) (cid:15) A (cid:107) < . (C1)Applying the Hilbert–Schmidt norm to both sides ofEq. (B6) yields (cid:107) Σ (cid:107) = (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) ∞ (cid:88) L =0 − ( L +1) l (cid:88) (cid:96) =0 (cid:18) L(cid:96) (cid:19) ( (cid:15) A T ) (cid:96) ( (cid:15) A ) L − (cid:96) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) ≤ ∞ (cid:88) L =0 − ( L +1) L (cid:88) (cid:96) =0 (cid:18) L(cid:96) (cid:19) (cid:107) (cid:15) A (cid:107) L = 12 ∞ (cid:88) L =0 (cid:107) (cid:15) A (cid:107) L , where we used the identity (cid:107) A T (cid:107) = (cid:107) A (cid:107) and subad-ditivity and submultiplicativity of the Hilbert–Schmidtnorm. When (cid:107) (cid:15) A (cid:107) < / , it follows that (cid:107) Σ (cid:107) < , so ν i =1 ,...,n ∈ (0 , for the positive-semidefinite matrix Σ .It follows that the sums in Eq. (13) converge.For many applications in network analysis, the spec-tral radius ρ ( · ) (which is equal to the largest absoluteeigenvalue of a matrix) is a commonly used matrix norm [101, 102]. We now show that one can relax the conditionin Eq. (C1) for short-range signal decay to ρ ( A ) < (C2)if A is the adjacency matrix of a strongly connectedgraph with non-negative edge weights.The adjacency matrix of a strongly connected graphis irreducible [103]. For an irreducible matrix with non-negative entries, the Perron–Frobenius theorem guaran-tees the existence of a simple, real, positive eigenvalue λ max = ρ ( A ) [103]. The transpose of A is also an ad-jacency matrix of a strongly connected graph with non-negative edge weights, so A T also has a simple positivereal leading eigenvalue λ max . Ortega [104, p. 24] provedthe existence of a submultiplicative matrix norm (cid:107) M (cid:107) ,such that ρ ( M ) = (cid:107) M (cid:107) for all complex square matri-ces M with simple max-modulus eigenvalues [105]. Thematrices A and A T are matrices with a single simplemax-modulus eigenvalue. We thus write ρ ( Σ ) ≤ (cid:107) Σ (cid:107)≤ (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) ∞ (cid:88) L =0 − ( L +1) l (cid:88) (cid:96) =0 (cid:18) L(cid:96) (cid:19) ( (cid:15) A T ) (cid:96) ( (cid:15) A ) L − (cid:96) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) and use the subadditivity and submulitplicativity of (cid:107) · (cid:107) to obtain ρ ( Σ ) ≤ ∞ (cid:88) L =0 (cid:107) (cid:15) A (cid:107) L = 12 ∞ (cid:88) L =0 ( ρ ( (cid:15) A )) L . (C3)When ρ ( (cid:15) A ) < / , it follows from Eq. (C3) that ρ ( Σ ) < . It follows that ν i =1 ,...,n ∈ (0 , , so the sumsin Eq. (13) converge. [1] S. S. Shen-Orr, R. Milo, S. Mangan, and U. Alon, Net-work motifs in the transcriptional regulation network of Escherichia coli , Nature Genetics , 64 (2002).[2] U. Alon, Network motifs: Theory and experimental ap-proaches, Nature Reviews Genetics , 450 (2007).[3] L. Stone, D. Simberloff, and Y. Artzy-Randrup, Net-work motifs and their origins, PLoS Computational Bi-ology , e1006749 (2019).[4] J. M. Rip, K. S. McCann, D. H. Lynn, and S. Fawcett,An experimental test of a fundamental food web motif,Proceedings of the Royal Society of London B: Biologi-cal Sciences , 1743 (2010).[5] K. Ristl, S. J. Plitzko, and B. Drossel, Complex responseof a food-web module to symmetric and asymmetric mi-gration between several patches, Journal of TheoreticalBiology , 54 (2014).[6] T. Ohnishi, H. Takayasu, and M. Takayasu, Networkmotifs in an inter-firm network, Journal of EconomicInteraction and Coordination , 171 (2010).[7] F. W. Takes, W. A. Kosters, B. Witte, and E. M.Heemskerk, Multiplex network motifs as building blocks of corporate networks, Applied Network Science , 39(2018).[8] K. Juszczyszyn, K. Musiał, P. Kazienko, and B. Gabrys,Temporal changes in local topology of an email-basedsocial network, Computing and Informatics , 763(2012).[9] X. Hong-Lin, Y. Han-Bing, G. Cui-Fang, and Z. Ping,Social network analysis based on network motifs, Jour-nal of Applied Mathematics (2014).[10] R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan,D. Chklovskii, and U. Alon, Network motifs: Simplebuilding blocks of complex networks, Science , 824(2002).[11] G. C. Conant and A. Wagner, Convergent evolution ofgene circuits, Nature Genetics , 264 (2003).[12] O. Sporns and R. Kötter, Motifs in brain networks,PLoS Biology , e369 (2004).[13] P. J. Ingram, M. P. Stumpf, and J. Stark, Network mo-tifs: Structure does not determine function, BMC Ge-nomics , 108 (2006).[14] M. Golubitsky, L. Shiau, C. Postlethwaite, and Y. Zhang, The feed-forward chain as a filter-amplifiermotif, in
Coherent Behavior in Neuronal Networks (Springer-Verlag, Heidelberg, Germany, 2009) pp. 95–120.[15] F. Antoneli, M. Golubitsky, and I. Stewart, Homeostasisin a feed forward loop gene regulatory motif, Journal ofTheoretical Biology , 103 (2018).[16] A. Shilnikov, R. Gordon, and I. Belykh, Polyrhythmicsynchronization in bursting networking motifs, Chaos:An Interdisciplinary Journal of Nonlinear Science ,037120 (2008).[17] Y. Artzy-Randrup, S. J. Fleishman, N. Ben-Tal, andL. Stone, Comment on “Network motifs: simple build-ing blocks of complex networks” and “Superfamiliesof evolved and designed networks”, Science , 1107(2004).[18] S. Robin, S. Schbath, and V. Vandewalle, Statisticaltests to compare motif count exceptionalities, BMCBioinformatics , 84 (2007).[19] W. E. Schlauch and K. A. Zweig, Influence of the null-model on motif detection, in (IEEE, 2015) pp. 514–519.[20] In other studies, the term “structural motif” often hasbeen used to refer to structure motifs, but it sometimeshas been used to refer to process motifs. We give anoverview of the use of motifs in the study of networksin Section II E. We use the terms “structure motif” and“process motif” to avoid confusion with conflicting def-initions of “structural motif” in previous work by otherscholars. We use the composite nouns to stress that weconsider structure motifs and process motifs to be twofundamentally different concepts.[21] Note that we distinguish a “system function” (whichmay, for example, be a biological function in a system)from a “mathematical function” like Y .[22] O. O. Aalen and H. K. Gjessing, Survival modelsbased on the Ornstein–Uhlenbeck process, LifetimeData Analysis , 407 (2004).[23] L. Barnett, C. L. Buckley, and S. Bullock, Neural com-plexity and structural connectivity, Physical Review E , 051914 (2009).[24] Z. Liang, K. C. Yuen, and J. Guo, Optimal propor-tional reinsurance and investment in a stock marketwith Ornstein–Uhlenbeck process, Insurance: Mathe-matics and Economics , 207 (2011).[25] R. V. Rohlfs, P. Harrigan, and R. Nielsen, Modelinggene expression evolution with an extended Ornstein–Uhlenbeck process accounting for within-species varia-tion, Molecular Biology and Evolution , 201 (2013).[26] D. Grytskyy, T. Tetzlaff, M. Diesmann, and M. Helias,A unified view on weakly correlated recurrent networks,Frontiers in Computational Neuroscience , 131 (2013).[27] Y. Hu, J. Trousdale, K. Josić, and E. Shea-Brown, Localpaths to global coherence: Cutting networks down tosize, Physical Review E , 032802 (2014).[28] M. D. Fox and M. Greicius, Clinical applications of rest-ing state functional connectivity, Frontiers in SystemsNeuroscience (2010).[29] S. L. Carter, C. M. Brechbühler, M. Griffin, and A. T.Bond, Gene co-expression network topology providesa framework for molecular characterization of cellularstate, Bioinformatics , 2242 (2004).[30] J.-P. Onnela, K. Kaski, and J. Kertész, Clustering and information in correlation based financial networks, TheEuropean Physical Journal B , 353 (2004).[31] H. Reichenbach, The Direction of Time (University ofCalifornia Press, Berkeley and Los Angeles, CA, USA,1956).[32] B. Bollobás,
Modern Graph Theory , Vol. 184 (Springer,Berlin, Germany, 2013).[33] R. J. Trudeau,
Introduction to Graph Theory (DoverPublications, New York, NY, USA, 2013).[34] Other researchers have defined a path to be a combina-tion of nodes and edges without repetitions [106]. Usingthis definition, a path is a special case of a walk. Forour work, it is crucial to distinguish paths and walksas two fundamentally different concepts, where the for-mer is related to processes on networks and the latter isrelated to network structure.[35] D. Spielman, Spectral graph theory, in
CombinatorialScientific Computing (CRC Press, Boca Raton, Florida,USA, 2012).[36] N. Alon, C. Avin, M. Kouck`y, G. Kozma, Z. Lotker,and M. R. Tuttle, Many random walks are faster thanone, Combinatorics, Probability and Computing , 481(2011).[37] C. Godsil and K. Guo, Quantum walks on regulargraphs and eigenvalues, The Electronic Journal of Com-binatorics , P165 (2011).[38] G. H. Nguyen, J. B. Lee, R. A. Rossi, N. K. Ahmed,E. Koh, and S. Kim, Continuous-time dynamic networkembeddings, in Companion Proceedings of the The WebConference 2018 (2018) pp. 969–976.[39] Ö. N. Yaveroğlu, N. Malod-Dognin, D. Davis, Z. Lev-najic, V. Janjic, R. Karapandza, A. Stojmirovic, andN. Pržulj, Revealing the hidden language of complexnetworks, Scientific Reports , 4547 (2014).[40] G. Wu, M. Harrigan, and P. Cunningham, Classifyingwikipedia articles using network motif counts and ra-tios, in Proceedings of the Eighth Annual InternationalSymposium on Wikis and Open Collaboration , 12 (2012)pp. 1–10.[41] L. Barnett, C. L. Buckley, and S. Bullock, Neural com-plexity: A graph theoretic interpretation, Physical Re-view E , 041906 (2011).[42] J. T. Lizier, F. M. Atay, and J. Jost, Information stor-age, loop motifs, and clustered structure in complex net-works, Physical Review E , 026110 (2012).[43] L. Novelli, F. M. Atay, J. Jost, and J. T. Lizier, De-riving pairwise transfer entropy from network structureand motifs, Proceedings of the Royal Society A ,20190779 (2020).[44] G. Ciriello and C. Guerra, A review on models and al-gorithms for motif discovery in protein–protein inter-action networks, Briefings in Functional Genomics andProteomics , 147 (2008).[45] A. Masoudi-Nejad, F. Schreiber, and Z. R. M. Kashani,Building blocks of biological networks: A review on ma-jor network motif discovery algorithms, Iet Systems Bi-ology , 164 (2012).[46] F. Battiston, V. Nicosia, M. Chavez, and V. Latora,Multilayer motif analysis of brain networks, Chaos:An Interdisciplinary Journal of Nonlinear Science ,047404 (2017).[47] J. B. Dechery and J. N. MacLean, Functional tripletmotifs underlie accurate predictions of single-trial re-sponses in populations of tuned and untuned V1 neu- rons, PLoS Computational Biology , e1006153 (2018).[48] G. C. Garcia, A. Lesne, M.-T. Hütt, and C. C. Hilgetag,Building blocks of self-sustained activity in a simple de-terministic model of excitable neural networks, Frontiersin Computational Neuroscience , 50 (2012).[49] V. P. Zhigulin, Dynamical motifs: Building blocks ofcomplex dynamics in sparsely connected random net-works, Physical Review Letters , 238701 (2004).[50] P. M. Gleiss, P. F. Stadler, A. Wagner, and D. A. Fell,Relevant cycles in chemical reaction networks, Advancesin Complex Systems , 207 (2001).[51] A. Vazquez, R. Dobrin, D. Sergi, J.-P. Eckmann, Z. N.Oltvai, and A.-L. Barabási, The topological relation-ship between the large-scale attributes and local interac-tion patterns of complex networks, Proceedings of theNational Academy of Sciences of the United States ofAmerica , 17940 (2004).[52] K. Shen, G. Bezgin, R. M. Hutchison, J. S. Gati, R. S.Menon, S. Everling, and A. R. McIntosh, Informationprocessing architecture of functionally defined clustersin the macaque cortex, Journal of Neuroscience ,17465 (2012).[53] Y. Iturria-Medina, R. C. Sotero, E. J. Canales-Rodríguez, Y. Alemán-Gómez, and L. Melie-García,Studying the human brain anatomical network viadiffusion-weighted MRI and Graph Theory, Neuroimage , 1064 (2008).[54] G. Bianconi, Number of cycles in off-equilibrium scale-free networks and in the internet at the autonomoussystem level, The European Physical Journal B , 223(2004).[55] S. Manrubia and J. Poyatos, Motif selection in a modelof evolving replicators: The role of surfaces and limitedtransport in network topology, Europhysics Letters ,557 (2003).[56] A. Ma’ayan, G. A. Cecchi, J. Wagner, A. R. Rao,R. Iyengar, and G. Stolovitzky, Ordered cyclic motifscontribute to dynamic stability in biological and engi-neered networks, Proceedings of the National Academyof Sciences of the United States of America , 19235(2008).[57] O. Sporns, G. Tononi, and G. M. Edelman, Theoreti-cal neuroanatomy: Relating anatomical and functionalconnectivity in graphs and cortical connection matrices,Cerebral Cortex , 127 (2000).[58] O. Sporns, G. Tononi, and G. M. Edelman, Connec-tivity and complexity: The relationship between neu-roanatomy and brain dynamics, Neural Networks ,909 (2000).[59] E. Estrada and J. A. Rodriguez-Velazquez, Subgraphcentrality in complex networks, Physical Review E ,056103 (2005).[60] J. Trousdale, Y. Hu, E. Shea-Brown, and K. Josić,Impact of network structure and cellular response onspike time correlations, PLoS Computational Biology , e1002408 (2012).[61] S. Jovanović and S. Rotter, Interplay between graphtopology and correlations of third order in spikingneuronal networks, PLoS Computational Biology (2016).[62] G. Tononi, O. Sporns, and G. M. Edelman, A measurefor brain complexity: Relating functional segregationand integration in the nervous system, Proceedings ofthe National Academy of Sciences of the United States of America , 5033 (1994).[63] C. Börgers, An Introduction to Modeling Neuronal Dy-namics (Springer, Cham, Switzerland, 2017).[64] S. Lehmann and Y.-Y. Ahn,
Complex Spreading Phe-nomena in Social Systems (Springer, Cham, Switzer-land, 2018).[65] R. Pastor-Satorras, C. Castellano, P. Van Mieghem, andA. Vespignani, Epidemic processes in complex networks,Reviews of Modern Physics , 925 (2015).[66] I. Z. Kiss, J. C. Miller, and P. L. Simon, Mathematics ofEpidemics on Networks (Springer, Cham, Switzerland,2017).[67] G. Demirel, F. Vazquez, G. Böhme, and T. Gross,Moment-closure approximations for discrete adaptivenetworks, Physica D: Nonlinear Phenomena , 68(2014).[68] T. House, G. Davies, L. Danon, and M. J. Keeling, Amotif-based approach to network epidemics, Bulletin ofMathematical Biology , 1693 (2009).[69] S. Chandra, E. Ott, and M. Girvan, Critical networkcascades with re-excitable nodes: Why treelike approx-imations usually work, when they break down, and howto correct them, Physical Review E , 062304 (2020).[70] S.-W. Oh and M. A. Porter, Complex contagions withtimers, Chaos: An Interdisciplinary Journal of Nonlin-ear Science , 033101 (2018).[71] D. T. Haydon, M. Chase-Topping, D. J. Shaw,L. Matthews, J. K. Friar, J. Wilesmith, and M. E. J.Woolhouse, The construction and analysis of epidemictrees with reference to the 2001 uk foot–and–mouth out-break, Proceedings of the Royal Society of London. Se-ries B: Biological Sciences , 121 (2003).[72] J. S. Juul and S. H. Strogatz, Descendant distributionsfor the impact of mutant contagion on networks, Phys-ical Review Research , 033005 (2020).[73] M. E. J. Newman, Spread of epidemic disease on net-works, Physical Review E , 016128 (2002).[74] F. Radicchi and C. Castellano, Leveraging percolationtheory to single out influential spreaders in networks,Physical Review E , 062314 (2016).[75] D. B. Larremore, M. Y. Carpenter, E. Ott, and J. G. Re-strepo, Statistical properties of avalanches in networks,Physical Review E , 066131 (2012).[76] G. Bianconi, Epidemic spreading and bond percolationon multilayer networks, Journal of Statistical Mechan-ics: Theory and Experiment , 034001 (2017).[77] An “integer composition” of a non-negative number m is a sequence ( q , q , . . . , q k ) of positive integers, where (cid:80) kk (cid:48) =1 q k (cid:48) = m [107].[78] G. E. Uhlenbeck and L. S. Ornstein, On the theory ofthe Brownian motion, Physical Review , 823 (1930).[79] M.-T. Tsai, J.-D. Saphores, and A. Regan, Valuationof freight transportation contracts under uncertainty,Transportation Research Part E: Logistics and Trans-portation Review , 920 (2011).[80] T. Sanderson, G. Hertzler, T. Capon, and P. Hayman, Areal options analysis of australian wheat production un-der climate change, Australian Journal of Agriculturaland Resource Economics , 79 (2016).[81] C. P. Ogbogbo, Modeling crude oil spot price as anOrnstein–Uhlenbeck process, International Journal ofMathematical Analysis and Optimization: Theory andApplications , 261 (2018). [82] J. Rice, Mathematical Statistics and Data Analysis , 3rded. (Thomson Higher Education, Belmont, CA, USA,2006).[83] B. Barzel and O. Biham, Quantifying the connectivityof a network: The network correlation function method,Physical Review E , 046104 (2009).[84] The Jupyter notebook is available as a web ap-plication under https://gitlab.com/aliceschwarze/motifs-for-processes .[85] S. Melnik, A. Hackett, M. A. Porter, P. J. Mucha, andJ. P. Gleeson, The unreasonable effectiveness of tree-based theory for networks with clustering, Physical Re-view E , 036112 (2011).[86] M. A. Porter and J. P. Gleeson, Dynamical Systemson Networks, Frontiers in Applied Dynamical Systems:Reviews and Tutorials (2016).[87] P. Tino, L. Benuskova, and A. Sperduti, Artificial neu-ral network models, in Springer Handbook of Computa-tional Intelligence (Springer, Cham, Switzerland, 2015)pp. 455–471.[88] T. Shinozaki and S. Watanabe, Structure discovery ofdeep neural network based on evolutionary algorithms,in (2015) pp.4979–4983.[89] R. VanRullen and C. Koch, Visual selective behaviorcan be triggered by a feed-forward process, Journal ofCognitive Neuroscience , 209 (2003).[90] B. M. Kampa, J. J. Letzkus, and G. J. Stuart, Corti-cal feed-forward networks for binding different streamsof sensory information, Nature Neuroscience , 1472(2006).[91] G. Manjunath and H. Jaeger, Echo state property linkedto an input: Exploring a fundamental characteristic ofrecurrent neural networks, Neural Computation , 671(2013).[92] P. Holme and J. Saramäki, Temporal Network Theory (Springer, Cham, Switzerland, 2019).[93] A. Paranjape, A. R. Benson, and J. Leskovec, Motifsin temporal networks, in
Proceedings of the Tenth ACMInternational Conference on Web Search and Data Min-ing (2017) pp. 601–610. [94] K. Tu, J. Li, D. Towsley, D. Braines, and L. D. Turner,Network classification in temporal networks using mo-tifs, arXiv:1807.03733 (2018).[95] P. Liu, V. Guarrasi, and A. E. Sarıyüce, Tempo-ral network motifs: Models, limitations, evaluation,arXiv:2005.11817 (2020).[96] J. M. Roldan, V. K. George, G. A. Silva, et al. , Con-struction of edge-ordered multidirected graphlets forcomparing dynamics of spatial temporal neural net-works, arXiv:2006.15971 (2020).[97] F. W. Fairman,
Linear Control Theory: The State SpaceApproach (John Wiley & Sons, New York City, NY,USA, 1998).[98] C.-T. Chen,
Linear System Theory and Design , 4th ed.(Oxford University Press, Inc., Oxford, United King-dom, 2013).[99] In [23, 41], the order of A and A T is reversed, becausethe authors used row vectors instead of column vectorsto describe the state x t .[100] B. Hall, Lie Groups, Lie Algebras, and Representations:An Elementary Introduction , Vol. 222 (Springer-Verlag,Cham, Switzerland, 2015).[101] A. Jamakovic, R. Kooij, P. Van Mieghem, and E. R.van Dam, Robustness of networks against viruses: Therole of the spectral radius, in
IEEE 2006 Symposium onCommunications and Vehicular Technology (2006) pp.35–38.[102] D. Stevanovic,
Spectral Radius of Graphs (AcademicPress, London, UK, 2014).[103] M. Boyle, Notes on the Perron–Frobenius theory of non-negative matrices (2015).[104] J. M. Ortega,
Numerical Analysis: A Second Course (SIAM, Philadelphia, PA, USA, 1990).[105] A max-modulus eigenvalue µ of a matrix M is an eigen-value that satisfies µ = ρ ( M ) .[106] M. E. J. Newman, Networks (Oxford University Press,Oxford, United Kingdom, 2018).[107] S. Eger, Restricted weighted integer compositions andextended binomial coefficients, Journal of Integer Se-quences16