EERGMs are Hard
Michael J. Bannister, William E. Devanny, and David EppsteinDepartment of Computer Science, University of California, Irvine
Abstract
We investigate the computational complexity of the exponential random graph model (ERGM)commonly used in social network analysis. This model represents a probability distribution ongraphs by setting the log-likelihood of generating a graph to be a weighted sum of feature counts.These log-likelihoods must be exponentiated and then normalized to produce probabilities,and the normalizing constant is called the partition function . We show that the problem ofcomputing the partition function is -hard, and inapproximable in polynomial time to withinan exponential ratio, assuming P (cid:54) = NP . Furthermore, there is no randomized polynomial timealgorithm for generating random graphs whose distribution is within total variation distance1 − o (1) of a given ERGM. Our proofs use standard feature types based on the sociologicaltheories of assortative mixing and triadic closure. a r X i v : . [ c s . D S ] D ec Introduction An exponential random graph model (ERGM) is a mathematical description of a probabilitydistribution over the graphs with a given fixed set of vertices, used in social network analysis [11, 15,16, 18]. In this model, every graph is mapped to a feature vector (typically a Boolean or integervector describing the existence or number of local structures such as vertices with given degrees orsmall induced subgraphs of a given type) and the log-likelihood of each graph is determined as theinner product of its feature vector with a weight vector specified as part of the model. ERGMs werefirst developed by Holland and Leinhardt to study graphs arising from social relationships [9]. Incontrast to other probabilistic models for social networks, such as the uniform distribution on graphswith a given degree distribution [3], the features and weights of an ERGM can correspond directlyto sociological theories of network formation. For instance, features that describe the presence orabsence of individual edges may have weights derived from models of assortative mixing , the theorythat people with similar characteristics are more likely to form connections with each other [14],while features describing the presence of triangle subgraphs may have weights derived from modelsof triadic closure , the theory that friends-of-friends are likely to become connected [5]. By usingmachine learning algorithms to fit the weights of an ERGM to real-world data, sociologists mayexperimentally measure the strength of these effects and use them to test their theories.Although there has been some theoretical research on speeding up the computation of featurevectors [6, 7], a low-level step in ERGM computations, as well as research on ERGMs from thegraph limit point of view [13], little has been known to date about the higher level computationalcomplexity of these models. In order to solve key problems on ERGMs, including the problems ofgenerating graphs from a given model, computing the partition function of a model (a normalizingconstant used to transform log-likelihoods into probabilities), and fitting weights to data, researchershave generally resorted to Monte Carlo methods, in which the Metropolis–Hastings algorithm is usedto construct a Markov chain on the set of graphs defined by an ERGM, with its stable distributionequal to the distribution described by the ERGM. Then, this chain is simulated for a number ofsteps with heuristic termination conditions that are intended to detect convergence of the chain toits stable distribution [8, 10]. However, these methods have no guarantees on their running time,accuracy, mixing time, rate of convergence, or correctness of the termination detection method.In this paper, for the first time, we investigate ERGMs from the point of view of their computa-tional complexity. We explain the heuristic nature of previous computations involving these modelsby showing that several key computational problems involving ERGMs are intractable in the worstcase. In particular, we show, for a family of ERGMs parameterized by the number n of verticeswhose description complexity (number of features and weights, and magnitude of the weights) ispolynomial in n , that: • Unless P = NP , there is no polynomial-time approximation to the partition function of a givenERGM that can achieve an approximation ratio exponential in any polynomial of n . • Unless P = , there is no polynomial-time algorithm to compute the partition function of agiven ERGM. • Unless RP = NP , there is no randomized polynomial algorithm for generating random graphswhose output distribution is within total variation distance 1 − o (1) of a given ERGM withvariable weights.Our results can be obtained using an ERGM with features that are very natural in socialnetwork analysis: an independent weight for each potential edge in a graph (representing different1ffinities between different pairs of actors in a network) and a single shared weight for all inducedtriangles (representing triad closure). As we show, an ERGM of this type can be used to describe adistribution that is very close to the uniform distribution on the maximum triangle-free subgraphsof a given graph. Our results follow from the known NP -hardness of finding a maximum triangle-freesubgraph [19] and from a new -hardness proof for counting large triangle-free subgraphs. Wealso show that these problems remain equally hard for an ERGM in which the features are theinduced subgraphs isomorphic to H , for every fixed graph H with three or more vertices, so smalladjustments to the types of features available in the model cannot make these problems easier.Thus, our results destroy all hope of guaranteed-quality polynomial-time computation for ERGMmodels of social networks.From the point of view of theoretical computer science, our methods are mostly standardreductions. However, in this respect, our most innovative result may be the inapproximability of theERGM partition function. There have been past results on -hardness and inapproximability ofpartition functions [2] but these have generally involved systems of colorings or other decorations on afixed underlying graph; instead, the states of an ERGM are themselves all possible graphs on a givenvertex set. Our inapproximability result is much stronger than the polynomial inapproximabilityknown to hold for all problems [12]. The proof of this result avoids PCP theory, which has beenapplied in many recent inapproximability results [1], and instead proceeds by a direct reduction. Exponential random graphs.
An exponential family random graph model, or ERGM for short,is a distribution on random graphs that forms an exponential family. The distributions of exponentialfamilies have the form
P r [ x ] = e θ · w ( x ) Z ( θ )where θ is a vector of weights, sometimes called the model parameters, w ( x ) is a vector valuedfunction that gives features of x , and Z ( θ ), often called the partition function, is a normalizingvalue chosen to make the probabilities sum to one. For an ERGM, x would be a graph on n verticesand the features of x would often include localized structures such as vertices with a given degree,paths with a given length, or cliques with a given number of vertices.We will let F be the set of features we use and w f be the weight corresponding to a feature f ∈ F . It will be convenient for us to linearly transform the weights (by multiplying by log e ),so that we can use 2 rather than e as the base of our exponential functions, allowing us to avoidreal number computations and simplify our proofs. We define the density of a graph G to be d ( G ) = (cid:81) f ∈ F f ( G ) w f where f ( G ) is the value of the feature f in G . The features we use will be0-1 indicators of the presence of particular induced subgraphs and counts of subgraph appearances.The probability of a graph, G , is its density, d ( G ), divided by a normalizing constant to make theprobabilities sum to one. This normalizing constant is Z = (cid:80) G d ( G ), so P r [ G ] = d ( G ) Z ; Z is calledthe partition function of an ERGM. In this paper we are chiefly concerned with two computationalproblems: calculating Z and generating graphs from these distributions.The types of features used in ERGMs are diverse. In general features can be of two types,homogeneous or heterogeneous, according to whether or not all isomorphic graphs G have the samevalue of f ( G ) as each other. An example of a homogeneous subgraph feature would be the number oftriangles in a graph. An example of a heterogeneous subgraph feature would be a 0-1 indicator that isone when three particular vertices form a triangle and zero otherwise. A collection of heterogeneoustriangle features, one for each triple of vertices with all weights equal, are equivalent to a single2omogeneous triangle feature with the same weighting. It is common in social network analysis toweight edge features in a heterogeneous way: in these analyses, vertices typically represent peoplefor whom we might have some prior knowledge or distribution of information that would affecttheir likelihood of being related. As an extreme example, in sexual contact networks, heterosexualcontacts would be expected to be more frequent than homosexual contacts, and individuals whoparticipate in both types of contact might be even less frequent. In some sociological models, thereis a common global constraint on the likelihood of triadic closures, or triangle subgraphs. To alignour proofs with this common model, we will use heterogeneous variable weights on edge features,but use a homogeneous uniform weight on all triangle subgraphs. A menagerie of computational problems.
We now introduce several computational problemsthat we will use throughout the paper:
Problem 1.
ERGM-PARTITIONInput: an ERGM E Output: the partition function for E Problem 2.
TRI-FREEInput: a graph G = ( V, E ) and a positive integer k Output: yes if there is a triangle free subgraph of G with k or more edges, no otherwise Problem 3. a graph G = ( V, E ) and a positive integer k Output: the number of triangle-free subgraphs of G with k or more edges Problem 4. a graph G = ( V, E ) Output: the number of perfect matchings in G Problem 5. a -regular bipartite graph G = ( V, E ) Output: the number of perfect matchings in G and are both known to be -complete [4, 17]. TRI-FREE is NP -complete [19]. We will prove that is -complete. Here we prove an inapproximability result on computing the partition function of an ERGM byreducing to
TRI-FREE and creating a gap in the values of the ERGM’s partition function thatseparates the yes-instances from the no-instances.
Definition 1.
For a given graph G , we define TriFreeERGM(
G, α ) to be an ERGM with thefollowing features and weights. We place heterogeneous weights on edges and a homogeneous weighton triangle subgraphs. If an edge belongs to G we give its indicator feature a weight of α and if itdoes not belong to G we instead give it a weight of β = − (cid:0) n (cid:1) α − (cid:0) n (cid:1) − . Also, we assign the weightfor the count of triangle subgraphs to be β . For two graphs G and H on the same vertex set, let a be the number of edges in H that are alsoin G , b be the number of edges in H that are not in G , and c be the number of triangle subgraphsin H . Then d ( H ) in the distribution defined by TriFreeERGM( G, α ) is 2 aα +( b + c ) β . Also note that a ≤ (cid:0) n (cid:1) . 3 emma 1. Fix an integer α > (cid:0) n (cid:1) and a graph G , and let d i denote the number of triangle freesubgraphs of G with i edges. Then the integer part of the partition function for TriFreeERGM(
G, α ) can be rewritten in base α as d ( n ) . . . d d d .Proof. If H is any graph that has an edge not belonging to G or that has a triangle subgraph, thenits density d ( H ) has a corresponding factor of 2 β , and is thus strictly less than 1 / n ). Thereforethe sum total of the densities of all graphs that contain edges not in G or that contain triangles isstrictly less than 1.The remaining contributions to the partition function come from graphs that are triangle freesubgraphs of G . Let H be such a graph with i edges; then H contributes one unit to d i and d ( H ) = 2 iα . Therefore the integer part of Z is (cid:98) Z (cid:99) = (cid:80) i d i iα . Because each d i is less than 2( n )and α > (cid:0) n (cid:1) , there are no carries in this sum and Z written in base 2 α is d ( n ) . . . d d d . Theorem 1. If P (cid:54) = NP , then ERGM-PARTITION cannot be approximated within a factor of poly( n ) in polynomial time.Proof. We prove this by contrapositive and so suppose we could approximate the partition functionof an ERGM within a factor of f ( n ) in polynomial time where f ( n ) = 2 poly( n ) .Let G be the input to k -TRI-FREE .We use E = TriFreeERGM( G, α ). If there is a triangle free subgraph with at least k edges,then by Lemma 1 Z > kα . If there are no triangle free subgraphs with k edges, then by Lemma 1 Z < n )2 ( k − α . Therefore setting α = (cid:0) n (cid:1) + 2 log ( f ( n )) + 1, we have: f ( n ) n ) +( k − α = 2 f ( n ))+ ( n ) +( k − α = 2 f ( n ))+ k ( n ) +2( k −
1) log( f ( n ))+ k − = 2 k (( n ) +2 log( f ( n ))+1 ) − = 2 kα − < kα So if the computed approximation of Z is greater than 2 kα /f ( n ), then G necessarily has a k -edgetriangle free subgraph. When the approximation of Z is less than f ( n )2( n ) +( k − α , G has no k edgetriangle free subgraph. Therefore if we could approximate Z within a factor of 2 poly( n ) in polynomialtime, then we could solve TRI-FREE in polynomial time implying that P = NP . In this section we prove the -hardness of computing the ERGM partition function. We show thisby reducing from . Unfortunately, the known reduction that shows
TRI-FREE to be NP -hard is not parsimonious [19], meaning that the reduction does not preserve solution countsin a consistent way. So we first need to show that is -complete, which we do byanother reduction from . Definition 2.
We define
Snub( G ) for a -regular bipartite graph, G , to be another graph constructedfrom G as follows. For each vertex v ∈ V ( G ) , create a triangle t v in Snub( G ) . Call the edges ofthese triangles the vertex triangle edges. Set an arbitrary cyclic ordering on the vertices of eachof these triangles. Then for each edge ( i, j ) ∈ E ( G ) , arbitrarily pick one unpicked vertex u in t i and another unpicked vertex w in t j . Add three edges to Snub( G ) : one edge from u to w , one edgefrom u to the vertex after w in t j ’s cyclic ordering, and one edge from w to the vertex after u in t i ’s cyclic ordering. Call the edge from u to w the cross edge for t i and t j and call the other twoedges connecting edges. on a cube to on asnub cubeThe name of this construction comes from the snub cube and snub dodecahedron, two convexpolyhedra whose graphs can be formed by applying this construction to the graphs of the cube anddodecahedron, respectively. Snub( G ) has three edges for every vertex in G and three more edges forevery edge in G . It also has one triangle for every vertex in G and two triangles for every edge in G .If G has n vertices, Snub( G ) has 15 n/ n triangles.Figure 1 illustrates the reduction, applied to the graph of a cube. In the figure, the vertextriangle for each cube vertex v corresponds to the central triangle in one of the patches of fourtriangles with the same color as v . Lemma 2. is -complete.Proof. First we observe that ∈ .We will prove -hardness by reduction from .Given G , a 3-regular bipartite graph on n vertices, we reduce it to Snub( G ). The cross edges andvertex triangle edges of Snub( G ) each participate in two triangles, and the connecting edges eachparticipate in only one triangle. However, the whole graph has 4 n triangles. So, if a triangle-freesubgraph of Snub( G ) can be obtained by deleting 2 n edges from Snub( G ), leaving 11 n/ G are in one-to-one correspondencewith the 11 n/ G ): • Suppose we are given a perfect matching in G . If u and v are not matched, delete the crossedge for t u and t v . If u and v are matched, then for each of t u and t v delete the vertex triangleedge adjacent to both the ( u, v ) cross and connecting edges. After these deletions the graphSnub( G ) has lost 2 n edges and is triangle free. • Suppose we are given a maximum size triangle free subgraph of Snub( G ). Because there is aperfect matching in every regular bipartite graph, this subgraph must have exactly the sizethat a subgraph generated as above from a perfect matching would have: 11 n/ n edges from Snub( G ). Therefore, the only edges that couldhave been deleted are cross and connecting edges. Observe that if one connecting edge hasbeen deleted then so to must its pair in order to destroy the remaining connecting trianglewhile continuing to destroy two triangles per deleted edge. By similar reasoning, at most oneedge from each vertex triangle could have been deleted. Therefore the pairs of connectingedges that have been deleted determine a perfect matching.5hus, this reduction is parsimonious, and is -complete. Theorem 2.
Unless P = , there can be no polynomial time algorithm for ERGM-PARTITION .Proof.
Again we take an instance of and use the TriFreeERGM(
G, α ) with α = (cid:0) n (cid:1) +1this time. By Lemma 1, if d i is the number of triangle free subgraphs with i edges, then the integerpart of Z can be written in base 2 α as d ( n ) . . . d d d . The sum of the digits from d k to d ( n ) is equalto the number of triangle free subgraphs with greater than or equal to k edges. Thus, if we couldcompute Z we could determine this number in polynomial time, implying that P = . Theorem 3.
Unless RP = NP , there is no randomized polynomial algorithm for generating randomgraphs whose output distribution is within total variation distance − o (1) of a given ERGM withvariable weights.Proof. Suppose there was such an algorithm, A . Then for some G and k , an instance of TRI-FREE ,we can construct the corresponding TriFreeERGM(
G, α ) with α = (cid:0) n (cid:1) + 1. Then running A on thisERGM, we can verify whether or not the outputted graph is a triangle free subgraph with at least k edges, output 1 if the verification succeeds, or output 0 if it fails. If G has no k -edge triangle-freesubgraph then this procedure will output one with o (1) probability. On the other hand if there issuch a subgraph, then the valid subgraphs have a 1 − o (1) fraction of the ERGM’s probability andwith high probability A will output one of them. Thus if there is such an algorithm to generaterandom graphs then we can use it to show that RP = NP . In this section we prove a dichotomy theorem that describes the hardness of computing ERGMpartition functions when the set of features only consists of subgraph indicators. Suppose we aregiven a set S of graphs, isomorphic copies of which are to be used as (heterogeneous) features of anERGM. Then, as we show, if all graphs in S have at most two vertices, then the partition function Z of all ERGMs using S as features can be computed in polynomial time. However, if S containsa graph with three or more vertices, then it is -hard to compute Z for ERGMs using featuresfrom S . This demonstrates that our results on the hardness of ERGMs are not specific to trianglefeatures: the hardness results that we have proven using triangles cannot be avoided by replacingthe triangles by a more clever choice of complex subgraph features.We prove this result using a feature replacement strategy, in which we use a combination ofheterogeneous weights on larger subgraphs to simulate weights on smaller subgraphs. In this way,hardness results for all larger features follow from basic hardness results for two three-vertex features:triangles (the main feature for our previous hardness results) or three-vertex paths.For this section we restrict the features for the ERGMs to indicator and count functions forspecific subgraphs from a set S . Without loss of generality we can assume that the ERGMs underconsideration include a feature for each subgraph isomorphic to a graph in S , as missing featurescan be handled by giving them weight 0. Problem 6. S -ERGM-PARTITION for a given set S of graphs Input: an ERGM, E , whose features are subgraph indicators and counts of graphs in S Output: the partition function, Z , for E Lemma 3.
Let S be a set of graphs to be used to define the features for an ERGM. If S containsonly graphs on two or fewer vertices, then S -ERGM-PARTITION can be solved in polynomialtime. roof. The only nontrivial graphs that S can contain are K (a single edge) and its complement K (the empty graph on two vertices). Without loss of generality we assume that S contains bothof these graphs. Given an ERGM E , and a potential edge, ( i, j ), we define the x ( i,j ) to be the sumof two terms: the weight of the indicator function that tests whether i and j induce a K subgraph,and the weight of the count function for K . Symmetrically, we define x ( i,j ) to be the sum of twoterms: the weight of the indicator function that tests whether i and j include a K subgraph, plusthe weight of the count function for K . Then Z = (cid:88) G (cid:89) e ∈ G x e (cid:89) e/ ∈ G x e . To compute this value, first compute Y = (cid:81) ( i,j ) x ( i,j ) and, for each edge e = ( i, j ), define x (cid:48) e = x e − x e .Then Z/Y = (cid:88) G (cid:89) e ∈ G x (cid:48) e = (cid:89) ( i,j ) (cid:16) x (cid:48) ( i,j ) (cid:17) . Thus if S contains only graphs on two vertices, we can compute Z in polynomial time by computing Y and Z/Y . Definition 3.
For a graph H with k vertices, H (cid:48) an induced subgraph of H with k (cid:48) vertices, anERGM E , and a weight γ , we define the feature replacement of H (cid:48) with H in E to be a new ERGM,defined as follows. The vertex set of the new ERGM will include all the vertices of E together withsome new vertices added in the construction. If E includes features that are not isomorphic to H (cid:48) ,these same features continue to exist in the new ERGM. For each indicator feature of a subgraph in E that is isomorphic to H (cid:48) , we perform the following steps:Step 1: Add k additional vertices to the new ERGM and pair k (cid:48) of them up with the k (cid:48) vertices ofthe subgraph of type H (cid:48) .Step 2: Label the k new vertices of the ERGM l , l , . . . , and l k and the k (cid:48) original vertices l (cid:48) , l (cid:48) , . . . , and l (cid:48) k (cid:48) such that the pair of l (cid:48) i is l i . Label the vertices of H in the same way.Step 3: Add a new indicator feature, with weight γ , that matches subgraphs isomorphic to H thatare induced by the set of vertices { l , l , . . . l k } with the numbering of these vertices matchingthe numbering of the isomorphic copy of H . Here γ is a parameter to be specified later.Step 4: For each i from to k (cid:48) , add another indicator feature, with weight γ , matching subgraphsisomorphic to H that are induced by the set of vertices { l , l , . . . , l i − , l (cid:48) i , l i +1 , . . . l k } .Step 5: Add one more indicator feature, with the same weight as indicator feature from E , matchingsubgraphs isomorphic to H that are induced by the set of vertices { l (cid:48) , l (cid:48) , . . . l (cid:48) k (cid:48) , l k (cid:48) +1 , . . . l k } ,i.e. with all the first k (cid:48) vertices swapped out for their pair in the original vertices.To handle features that count the number of subgraphs isomorphic to H (cid:48) , we run the same processfor each of the k (cid:48) ! (cid:0) nk (cid:48) (cid:1) possible induced subgraphs of this type. Figure 2 shows this feature replacement process for replacing K with a wheel graph on 6vertices. Lemma 4.
Given two graphs H and H (cid:48) , where H (cid:48) is an induced subgraph of H , the partitionfunction of an ERGM that has only integer weights and uses H (cid:48) can be computed from the partitionfunction of an ERGM that uses H instead of H (cid:48) . new vertices desired indicator l l l l l l l l l Hl l l l l l l l l Step 1: Step 2:Step 3: Step 4:Step 5:
Figure 2: An example of feature replacement replacing a triangle indicator with six new verticesand five wheel graph indicators.
Proof.
First let w + be the sum of all positive weights in the ERGM that uses H (cid:48) and w − be theabsolute value of the sum of all negative weights. We know the digits of Z for this ERGM are within w − digits of the right of the decimal point and w + digits to the left of the decimal point.Use feature replacement to replace H (cid:48) with H using a weight of γ = ( w + + w − ).The ERGM obtained from feature replacement has polynomially many new vertices and indicatorfeatures. In this ERGM, there exist states in which all of the indicator features with weight 2 γ aretrue; let s be the number of these indicator features. Then for each of these states, the densityof the graph will include a factor of 2 sγ for those features, while all the other states will omit atleast one factor of 2 sγ . Thus, by looking at the binary digits of the partition function extendingfrom the 2 sγ bit upwards (as seen in Figure 3), we can recover the subset of the partition functiongenerated only by the states in which these indicator features are all true. For these states, theremaining terms in the weight of each state coincide with the corresponding terms in the weights ofthe states of the original ERGM. 8 + . w − γ = 2 ( w + + w − ) Z new = γZ old = .a b γ γ γ . . .a b = . γa γb a b a b Figure 3: Z old is found in the leading two digits of Z new in base 2 γ . Lemma 5.
Let S be a set of graphs containing any graph H on three or more vertices. Then S -ERGM-PARTITION is -hard.Proof. H must contain at least one of K , P , P , or K as an induced subgraph; we handle each caseseparately. The cases of P or K can be transformed into the cases of P or K by complementing allof the features used in the ERGM (keeping the weights the same), which produces an ERGM whoseprobability on any graph is the same as the probability of the original ERGM on the complementarygraph. In particular, this transformation leaves the partition function unchanged. Thus, we needonly consider the cases of K and P .If H contains K as an induced subgraph, we proceed in the same manner as Theorem 2.However, we have to simulate the weights on triangle and edge subgraphs using indicator featuresfor copies of H . To do so we observe that because H contains K it also contains K and so twoapplications of Lemma 4 allow us to reduce the instance of to an ERGMusing subgraph indicator features of graphs in S .If H contains P as an induced subgraph, we instead reduce from . Given abipartite graph, G , as input, we create for each edge in G an indicator feature for that edge withweight (cid:0) n (cid:1) . For edges not in G and any P in G , we create an indicator feature for that subgraphwith weight β = − (cid:0) n (cid:1) − (cid:0) n (cid:1) −
1. By an argument similar to Lemma 1, if d i is the number ofmatchings of G with i edges, then the partition function in base 2( n ) is d ( n ) . . . d d d . Thus d n ,the n -th digit of Z , counts the number of perfect matchings. Now using Figure 2 we can reduce thisERGM using P and K to another ERGM using H . Theorem 4.
Given a set of subgraphs, S . If S contains a graph on three or more vertices, S -ERGM-PARTITION is -hard and can be computed in polynomial time otherwise.Proof. The result follows from Lemma 3 and Lemma 5.
We have shown
ERGM-PARTITION to be -hard and inapproximable via reductions that arevery close to the natural usage of ERGMs. Additionally, we showed that the hardness of ERGMpartition function computation can be classified by their subgraph features where ERGMs that usesubgraphs with more than two vertices lead to hard to compute partition functions. All but thesimplest of ERGMs are fundamentally difficult to work with and if the ability to sample from thedistribution is required, then different distributions on graphs are necessary.9 eferences [1] S. Arora and B. Barak.
Computational Complexity: A Modern Approach . CambridgeUniversity Press, 2009.[2] F. Barahona. On the computational complexity of Ising spin glass models.
Journal of PhysicsA: Mathematical and General
Annals of Combinatorics
Theoretical Computer Science
American Sociological Review .[6] D. Eppstein, M. T. Goodrich, D. Strash, and L. Trott. Extended dynamic subgraph statisticsusing h -index parameterized data structures. Theoretical Computer Science h -index of a graph and its application to dynamic subgraphstatistics. Journal of Graph Algortihms and Applications
Journal of the American Statistical Association
Journal of the American Statistical Association
Journal of Computational and Graphical Statistics
Journal of StatisticalSoftware .[12] M. R. Jerrum, L. G. Valiant, and V. V. Vazirani. Random generation of combinatorialstructures from a uniform distribution.
Theoretical Computer Science
Large Networks and Graph Limits ,pp. 259–261. American Mathematical Society, Colloquium Publications 60, 2012.[14] M. E. J. Newman. Mixing patterns in networks.
Phys. Rev. E
British Journal of Mathematical and Statistical Psychology
Psychometrika
Theoretical Computer Science p ∗ . Psychometrika