Markov Equivalences for Subclasses of Loopless Mixed Graphs
aa r X i v : . [ s t a t . O T ] O c t Markov Equivalences for Subclasses of Loopless Mixed Graphs
KAYVAN SADEGHI
Department of Statistics, University of Oxford
ABSTRACT : In this paper we discuss four problems regarding Markov equivalencesfor subclasses of loopless mixed graphs. We classify these four problems as finding con-ditions for internal Markov equivalence, which is Markov equivalence within a subclass,for external Markov equivalence, which is Markov equivalence between subclasses, forrepresentational Markov equivalence, which is the possibility of a graph from a subclassbeing Markov equivalent to a graph from another subclass, and finding algorithms togenerate a graph from a certain subclass that is Markov equivalent to a given graph. Weparticularly focus on the class of maximal ancestral graphs and its subclasses, namelyregression graphs, bidirected graphs, undirected graphs, and directed acyclic graphs, andpresent novel results for representational Markov equivalence and algorithms.
Key words : Bidirected graph, directed acyclic graph, m -separation, Markov equivalence,maximal ancestral graph, regression chain graph, summary graph, undirected graph. Introduction and motivation.
In graphical Markov models several classes of graphshave been used in recent years. A common feature of all these graphs is that their nodescorrespond to random variables, and they represent conditional independence statementsof the node set by specific interpretations of missing edges.These graphs contain up to three different types of edges. Sadeghi & Lauritzen(2011) gathered most classes of graphs defined in the literature under a unifying classof loopless mixed graphs (LMGs). These contain
Summary graphs (SGs) (Wermuth,2011), (maximal) ancestral graphs (MAGs) (Richardson & Spirtes, 2002), acyclic di-rected mixed graphs (ADMGs) (Spirtes et al. , 1997), regression chain graphs (RCGs)(Cox & Wermuth, 1993; Wermuth & Cox, 2004; Wermuth & Sadeghi, 2011), undirected or concentration graphs (UGs) (Darroch et al. , 1980; Lauritzen, 1996), bidirected or co-variance graphs (BGs) (Cox & Wermuth, 1993; Wermuth & Cox, 1998), and directedacyclic graphs (DAGs) (Kiiveri et al. , 1984; Lauritzen, 1996).For the above graphs, in general, two graphs of different types or even two graphs ofthe same type may induce the same independencies. Such graphs are said to be Markovequivalent . It is important to explore the similar characteristics of Markov equivalent raphs, and to find the ways of generating graphs of a certain type with the sameindependence structure from a given graph. Some questions for Markov equivalences.
There are four main questions regardingMarkov equivalence for different types of graphs:
1) Internal Markov equivalence:
The first natural question that arises in this con-text is regarding when two graphs of the same type are Markov equivalent. Thisquestion may be answered for DAGs, MAGs, or other subclasses of LMGs.
2) External Markov equivalence:
In addition to Markov equivalence for graphs ofthe same type, one can discuss Markov equivalence between two graphs of differenttypes.
3) Representational Markov equivalence:
Before checking external Markov equiv-alence, however, it is essential to check whether and under what conditions a graphof a certain type can be Markov equivalent to a graph of another type.
4) Algorithms:
One can also present some algorithms to generate a graph of a certaintype that is Markov equivalent to a given graph of a different type.In this paper we gather and simplify the existing results in the literature for internaland external Markov equivalences, and give novel results for representational Markovequivalence and algorithms.
Some earlier results on Markov equivalence for graphs.
Results concerning Markovequivalence for different classes of graphs have been obtained independently in the statis-tical literature on specifying types of multivariate statistical models, and in the computerscience literature on deciding on special properties of a given graph or on designing fastalgorithms for transforming graphs. In the literature on graphical Markov models two ofthe early results concerning Markov equivalence for DAGs and chain graphs were respec-tively given in Verma & Pearl (1990) and Frydenberg (1990). Two of the later resultsby Zhao et al. (2004) and Ali et al. (2009) respectively provided theoretically neat andcomputationally fast conditions for Markov equivalence for maximal ancestral graphs.Besides these, Pearl & Wermuth (1994) provided conditions for Markov equivalencefor bidirected graphs and DAGs. Spirtes & Richardson (1997) gave some conditions forMarkov equivalence for maximal ancestral graphs, in which the polynomial computa-tional complexity claim was wrong.Efficient algorithms for deciding whether a UG can be oriented into a DAG becameavailable in the computer science literature under the name of perfect elimination orien-tations; see Tarjan & Yannakakis (1984), whose algorithm can be run in O ( | V | + | E | ). nother such linear algorithm can be found in Blair & Peyton (1992). An algorithm forgenerating a Markov equivalent DAG from a bidirected graph is the special case of thealgorithm given in Zhao et al. (2004). Structure of the paper.
In the next section we define the unifying class of LMGs,and provide some basic graph theoretical definitions needed for our results.In Section 3 we present the subclasses of LMGs, and we formally define the sub-classes of interest in this paper. We also define a so-called separation criterion, called m -separation, to provide an interpretation of independencies for the graphs.In Section 4 we formally define Markov equivalence, define maximality and explain itsimportance for Markov equivalences, and motivate why we consider Markov equivalencefor the class of MAGs.In Section 5 we gather the conditions existing in the literature for internal Markovequivalence for the class of MAGs and its subclasses, and give conditions for their externalMarkov equivalence.In Section 6 we discuss the conditions for representational Markov equivalence forMAGs and its subclasses to a specific subclass, and we also provide algorithms to generatea Markov equivalent graph of a specific type to a given graph of another type when theconditions for representational Markov equivalence are satisfied. In each subsection wedeal with different subclasses of MAGs: DAGs in Section 6.1, UGs and BGs in Section6.2, and RCGs in Section 6.3.In Section 7 we summarise the results, presented in the paper. Graphs. A graph G is a triple consisting of a node set or vertex set V , an edge set E , and a relation that with each edge associates two nodes (not necessarily distinct),called its endpoints . A loop is an edge with the same endpoints. When nodes i and j are the endpoints of an edge, these are adjacent and we write i ∼ j . We say the edge is between its two endpoints. We usually refer to a graph as an ordered pair G = ( V, E ).Graphs G = ( V , E ) and G = ( V , E ) are called equal if ( V , E ) = ( V , E ). In thiscase we write G = G .Notice that the graphs that we use in this paper (and in general in the context ofgraphical models) are so-called labeled graphs , i.e. every node is considered a differentobject. Hence, for example, graph i j k is not equal to j i k . Definition of loopless mixed graph.
Sadeghi & Lauritzen (2011) gathered most graphsin the literature of graphical models under the definition of loopless mixed graph , which is graph that contains three types of edges denoted by arrows, arcs (two-headed arrows),and lines (full lines) and does not contain any loops. Basic definitions for LMGs.
We say that i is a neighbour of j if these are endpointsof a line, and i is a parent of j if there is an arrow from i to j . We also define that i isa spouse of j if these are endpoints of an arc. We use the notations ne( j ), pa( j ), andsp( j ) for the set of all neighbours, parents, and spouses of j respectively. In the cases of i ≻ j or i ≺ ≻ j we say that there is an arrowhead pointing to (at) j A subgraph of a graph G is graph G such that V ( G ) ⊆ V ( G ) and E ( G ) ⊆ E ( G )and the assignment of endpoints to edges in G is the same as in G . An inducedsubgraph by nodes A ⊆ V is a subgraph that contains all and only nodes in A and alledges between two nodes in A . A subgraph induced by edges A ⊆ E is a subgraph thatcontains all and only edges in A and all nodes that are endpoints of edges in A . Wedenote the subgraphs induced by arrows, arcs, and lines in a graph H by H [ ≻ ], H [ ≺ ≻ ],and H [ ] respectively.A walk is a list h v , e , v , . . . , e k , v k i of nodes and edges such that for 1 ≤ i ≤ k , theedge e i has endpoints v i − and v i . A path is a walk with no repeated node or edge. Wedenote a path by an ordered sequence of node sets. We say a path is between the firstand the last nodes of the list in G . We call the first and the last nodes endpoints of thepath and all other nodes inner nodes .A cycle in a graph G is a simple subgraph whose nodes can be placed around a circleso that two nodes are adjacent if these appear consecutively along the circle.A path (or cycle) is direction preserving if all its edges are arrows pointing to onedirection. If a direction-preserving path is from a node j to a node i then j is an ancestor of i . We denote the set of ancestors of i by an( i ).A graph is called directed if it only contains arrows. A directed graph is acyclic if ithas no direction-preserving cycle.A chord is an edge between two non-adjacent nodes on the cycle. A cycle is chordless if it has no chords. The notation C n is used for a chordless cycle with n nodes. Noticethat C n can contain different types of edges, so it represents a class of graphs rather thana single graph. We call a graph chordal if it has no C n , n ≥
4, as an induced subgraph.We also use the notation P n for a chordless or minimal path with n nodes, i.e. a paththat has no edge between two non-adjacent nodes on the path.A V-configuration is a path with three nodes and two edges. Notice that originallyand in most texts, e.g. in Kiiveri et al. (1984), the endpoints of a V-configuration is bydefinition not adjacent. In this paper we call such V-configurations unshielded .In a mixed graph the inner node of three V-configurations i ≻ t ≺ j , i ≺ ≻ t ≺ j ,and i ≺ ≻ t ≺ ≻ j is a collider and the inner node of all other V-configurations is a on-collider node on the V-configuration or more generally on a path of which the V-configuration is a subpath. We also call the V-configuration with collider or non-colliderinner node a collider or non-collider V-configuration respectively. We may speak of acollider or non-collider node without mentioning the V-configuration or the path whenthis is apparent from the context. In the case of DAGs the only collider V-configuration i ≻ t ≺ j is called a collision V-configuration. Subclasses of LMGs
The following diagram, presented in Sadeghi & Lauritzen (2011)illustrates the hierarchy regarding inclusions of subclasses of LMGs.
Loopless mixed graphs (cid:15) (cid:15)
Ribbonless graphs (modified MC graphs) (cid:15) (cid:15)
Summary graphs (cid:15) (cid:15) ( ( QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ (Maximal) ancestral graphs (cid:15) (cid:15) w w oooooooooooooooooooooooooooo Regression chain graphs (cid:15) (cid:15) , , XXXXXXXXXXXXXXXXXXXXXXX
Acyclic directed mixed graphs (cid:15) (cid:15) r r fffffffffffffffffffffff Undirected graphs Directed acyclic graphs Bidirected graphs
Figure 1:
The hierarchy of subclasses of LMGs.
The common feature of all these subclasses is that these use the same interpretationfor independencies, known as the m -separation criterion. We will shortly introduce the m -separation criterion for MAGs and its subclasses. Ribbonless graphs , defined and studied in Sadeghi (2011) is a modification of MCgraphs, defined by Koster (2002), to discard the line loops and to use the m -separationcriterion. Definition of ancestral graphs and regression chain graphs. An ancestral graph is amixed graph that, for all nodes i , has (1) i / ∈ an(pa( i ) ∪ sp( i )) and (2) If ne( i ) = ∅ thenpa( i ) ∪ sp( i ) = ∅ . This means that there is no arrowhead pointing to a line and there isno direction-preserving cycle, and there is no arc with one endpoint the ancestor of theother endpoint, in the graph. graph G = ( V, E ) is a regression chain graph if it contains at most the three typesof edge, there is no arrowhead pointing to a line in graph, and it does not contain any arc-direction-preserving cycle, i.e. a cycle that contains arcs and at least one arrow andwhose arrows are all towards one direction. Thus in such graphs the subgraph inducedby lines is so-called at the beginning of graph, and the subgraph induced by the arrowsand arcs is characterised by having a node set that can be partitioned into numberedsubsets forming so-called chains , i.e. V = τ ∪ · · · ∪ τ T such that all edges between nodesin the same subset are arcs and all edges between different subsets are arrows pointingfrom the set with the higher number to the one with the lower number. One can observethat in the subgraph induced by the arrows and arcs if we replace every τ i by a node,we get a DAG. The m -separation criterion Since, as we shall see, we are only interested in the sub-classes of MAGs, we use the simplified version of m -separation criterion, defined inSadeghi & Lauritzen (2011). This is identical to the original definition of m -separation;see Richardson & Spirtes (2002).Let C be a subset of the node set V of a MAG. A path is m -connecting given C ifall its collider nodes are in C ∪ an( C ) and all its non-collider nodes are outside C . Fortwo other disjoint subsets of the node set A and B , we say A ⊥ m B | C if there is no m -connecting path between A and B given C .Notice that the m -separation criterion gives an interpretation of independencies ongraphs, i.e. it induces an independence model. Definition of Markov equivalence.
Now we can formally define Markov equivalence.Two graphs G = ( V, E ) and G = ( V, E ) are Markov equivalent if, for all subsets A , B , and C of V , A ⊥ m B | C in G if and only if A ⊥ m B | C in G . Maximality and Markov equivalence.
A loopless mixed graph G is called maximal ifby adding any edge to G the independence model induced by the m -separation criterionchanges. Alternatively, a graph G = ( V, E ) is maximal if and only if, for every pair ofnon-adjacent nodes i and j of V , there exists a subset C of V \ { i, j } such that i ⊥ m j | C ;see Richardson & Spirtes (2002); Sadeghi & Lauritzen (2011).This implies that two Markov equivalent maximal graphs must have the same skeleton,where the skeleton of a graph results by replacing each edge by a full line. otivations behind using MAGs and its subclasses. In this paper we aim to discussMarkov equivalence for the subclasses presented in Fig. 1. The conditions for internalMarkov equivalence for RGs and SGs are very complex. However, in Sadeghi (2011) itwas demonstrated how RGs can be mapped into a Markov equivalent SG, and how SGscan be mapped into a Markov equivalent AG. Notice that ADMGs are SGs without fulllines, so by the same method one can map ADMGs into Markov equivalent AGs.In addition, since Markov equivalent maximal graphs must have the same skeleton,conditions for Markov equivalence for MAGs are simpler than those for Markov equiv-alence for AGs. In Richardson & Spirtes (2002) it was shown how AGs can be mappedinto a Markov equivalent MAG. Therefore, we map all types of stable independencegraphs into MAGs and discuss the Markov equivalences for MAGs and its subclasses.Notice that all subclasses of MAGs discussed here are maximal by nature. Therefore,for their Markov equivalence they must have the same skeleton.
Thus far, there are two elegant results regarding Markov equivalence for MAGs available(Ali et al. , 2005; Zhao et al. , 2004). These results use different definitions (colliders withorder and minimal collider paths) and arguments. Even though it is not immediatelyobvious from their formulations, it can be shown that these are equivalent.
First result for Markov equivalence for MAGs.
In order to present the first theorem,we quote two definitions from Ali et al. (2005). A path π = h j, q , q , . . . , q m , l, i i , with j not adjacent to i , is a discriminating path for h q m , l, i i in G if and only if, for every node q n , 1 ≤ n ≤ m on π , i.e. excluding j , i , and l , i) q n is a collider on π ; and ii) q n ≻ i , hence forming a non-collider along the path h j, q , . . . , q n , i i .Fig. 2 illustrates what a discriminating path looks like. lq q q m ij Figure 2:
A discriminating path.
Let D n be the set of triples of order n defined recursively as follows: rder : A triple h h, l, i i ∈ D if h and i are not adjacent in G . Order n + 1 : A triple h h, l, i i ∈ D n +1 if1) h h, l, i i / ∈ D p , for some p < n + 1 and2) there exists a discriminating path π = h j, q , . . . , q m = h, l, i i for l in G , and eachof the colliders on the path: h j, q , q , i , . . . , h q m − , q m , l i ∈ S p ≤ n D p .If h h, l, i i ∈ D n then the triple is said to have order n . A discriminating path is said tohave order n if every triple on the path has order at most n and at least one triple hasorder n . If a triple has order n for some n we then say that the triple has order, likewisefor discriminating paths. Theorem 1. (Ali & Richardson, 2004) MAGs H and H are Markov equivalent if andonly if H and H have the same skeleton and colliders with order. In Fig. 3 there are three MAGs with the same skeleton. In H and H since i ≁ k ,the collider h i, j, k i is with order 0, whereas in H this is not a collider. In H and H ,the collider h j, k, h i is with order 1. Therefore, we conclude that H and H are Markovequivalent, but these are not Markov equivalent to H . i j k h i j k h i j k h (a) (b) (c)Figure 3: (a) A maximal ancestral graph H . (b) A maximal ancestral graph H that is Markov equivalent to H . (c) A maximal ancestral graph H that is not Markov equivalent to H or H . Second result for Markov equivalence for MAGs.
In order to present the second the-orem, we quote two definitions from Zhao et al. (2004). A path π is called a colliderpath if all its inner nodes are colliders on π . A collider path π = h i, B, j i is called a minimal collider path if i j and there is no proper subset B ′ ⊂ B such that h i, B ′ , j i is a collider path between i and j . If i ∼ j then we call π a minimal collider cycle . Inthe graph in Fig. 4 the path h i, j, k, h i is a collider path, but it is not minimal collidersince there exists the collider path h i, j, h i , which is minimal collider. i j k h Figure 4:
A non-minimal collider path h i, j, k, h i and a minimal collider path h i, j, h i . heorem 2. (Zhao et al. , 2004) MAGs H and H are Markov equivalent if and onlyif H and H have the same skeleton and minimal collider paths. For the graphs in Fig. 3, by Theorem 2 we can make the same conclusion as before.To observe this it is enough to check that h i, j, k i and h i, j, k, h i are the minimal colliderpaths of H and H , whereas there is no minimal collider path in H . We, therefore,conclude that H and H are Markov equivalent, but these are not Markov equivalentto H . First of all we recall a well-known result regardingMarkov equivalence for DAGs.
Proposition 1. (Verma & Pearl, 1990; Frydenberg, 1990) DAGs G and G are Markovequivalent if and only if they have the same skeleton and unshielded collision V-configurations. In the example in Fig. 5 all three graphs have the same skeleton. In G there aretwo unshielded collision V-configurations h k, i, h i and h j, i, h i . In G there are the sameunshielded collision V-configurations. Therefore, these two graphs are Markov equiva-lent. The only unshielded collision V-configuration in G is, however, h k, i, h i . Hencethis graph is not Markov equivalent to G and G . k j i h k j i h k j i h (a) (b) (c)Figure 5: (a) A directed acyclic graph G . (b) A Markov equivalent directed acyclic graph G to G . (c) Adirected acyclic graph G that is not Markov equivalent to G . Markov equivalence for UGs and BGs.
The following proposition shows when twobidirected or undirected graphs are Markov equivalent.
Proposition 2.
Bidirected or undirected graphs H and H are Markov equivalent if andonly if they are equal.Proof. In the case of the undirected graph, the result follows from Theorem 1 and thefact that there is no collider in undirected graphs.In the case of the bidirected graph, the result follows from Theorem 2 and the factthat every path in bidirected graphs is a collider path. arkov equivalence for RCGs. Since RCGs are a subclass of MAGs, we simplify theconditions for Markov equivalence for MAGs in order to obtain the conditions for Markovequivalence for RCGs.
Proposition 3. (Wermuth & Sadeghi, 2011) RCGs H and H are Markov equivalent ifand only if H and H have the same skeleton and unshielded collider V-configurations.Proof. We apply Theorem 1 to RCGs and simplify its conditions in order to obtain theconditions of this theorem. The first condition of Theorem 1 (having the same skeleton)is the same as the first condition of this theorem. Therefore, it is enough to prove that H and H have the same colliders with order if and only if they have the same unshieldedcollider V-configurations.An unshielded collider V-configuration is by definition a collider with order. Weprove that in RCGs a collider V-configuration that is a collider with order is unshielded.This proves the proposition: Suppose that h h, k, l i is a collider with order and, forcontradiction, is not unshielded. By the definition of collider with order there existsa discriminating path h x, q , . . . , q p = h, k, l i for k . Hence h ∈ sp( k ). In addition,if l ∈ pa( k ) then h ∈ an( k ), a contradiction by the definition of RCGs. Therefore, l ∈ sp( k ), but again this is a contradiction since in RCGs for a collider V-configurationwith two adjacent arcs h h, k, l i , one endpoint ( h ) cannot be the parent of the otherendpoint ( l ).In the example in Fig. 6 all three RCGs have the same skeleton. In H there arethree unshielded collider V-configurations h l, h, k i , h l, j, i i , and h k, i, j i . In H there arethe same unshielded collider V-configurations. Therefore, these two graphs are Markovequivalent. The unshielded collider V-configurations in H are, however, h l, h, k i and h k, i, j i . Hence this graph is not Markov equivalent to H or H . lhk ji lhk ji lhk ji (a) (b) (c)Figure 6: (a) A regression chain graph H . (b) A Markov equivalent regression chain graph H to H . (c) AnRCG (DAG) H that is not Markov equivalent to H . As a corollary of Proposition 3, in order to check the external Markov equivalence forevery two introduced subclasses of MAGs (excluding MAGs), i.e. RCGs, BGs, UGs, nd DAGs, the conditions that are used for Markov equivalence for RCGs can be used.In some cases, if we suppose that the graphs satisfy the conditions for representationalMarkov equivalence, which will be introduced in the next section, then the conditionsfor external Markov equivalence can be simplified. Corollary 1.
Every two of RCG, BG, UG, and DAG are Markov equivalent if and onlyif they have the same skeleton and unshielded collider V-configurations.Proof.
The result follows from the fact that BGs, UGs, and DAGs are subclasses ofRCGs.Notice that for Markov equivalence for a UG and a graph H of another type thecorollary states that there should be no collider V-configurations in H . Structure of the section.
In this section, in each subsection, we deal with represen-tational Markov equivalence for MAGs and its subclasses to a specific subclass. In eachsubsection we first introduce an algorithm for generating a graph of a specific subclasswhich is Markov equivalent to a given MAG. The algorithm is usually trivially simplifiedfor subclasses of MAGs. We then introduce conditions for a MAG under which it isMarkov equivalent to a graph from the given subclass. By simplifying these conditionswe obtain the conditions for subclasses of MAGs under which they are Markov equivalentto a graph of the given subclass. Notice that representational Markov equivalence to theclass of MAGs produces trivial results.
We begin with analgorithm for generating a Markov equivalent DAG to a given MAG that satisfies theconditions of Lemma 1.
Algorithm 1. (Generating a Markov equivalent DAG to a maximal ancestral graph H )Start from H .1. Apply the maximum cardinality search algorithm on H [ ] to order the nodes.2. Orient the edges of H [ ] from a lower number to a higher one.3. Replace unshielded collider V-configurations by unshielded collision V-configurations,i.e. replace i ≺ ≻ ◦ ≺ ≻ j and i ≺ ≻ ◦ ≺ j by i ≻ ◦ ≺ j when i j .4. Order the nodes of the subgraph induced by arrows such that arrows are from highernumbers to lower ones. . Order the nodes of the subgraph induced by arcs arbitrarily by numbers not used inthe previous step if the number for the node does not already exist.6. Replace arcs by arrows from higher numbers to lower ones.Continually apply each step until it is not possible to apply the given step further beforemoving to the next step. Lemma 1.
Let H be a maximal ancestral graph. If H [ ] is chordal and there is nominimal collider path or cycle of length in H then Algorithm 1 generates a Markovequivalent DAG to H .Proof. Denote the generated graph by G . Graph G is directed since by Algorithm 1, alledges are turned into arrows. Since H [ ] is chordal, the graph generated by the perfectelimination order of the maximal cardinality search does not have a direction-preservingcycle; see Section 2.4, Blair & Peyton (1992).In addition, the arrows present in the graph do not change by the algorithm. Weshow that there is no direction-preserving cycle after applying step 3: If, for contradic-tion, there is a shortest direction-preserving cycle after applying step 3 then a colliderV-configuration h j, k, i i (say the jk -edge is an arc) should turn into a transition V-configuration after applying step 3. In this case there is an hj -edge for a node h with anarrowhead pointing to j and h k . Since there is no minimal collider path or cycle oflength 4, h j, k, i i is shielded. Notice that on the ji -edge there is an arrowhead pointingto j since otherwise there is a minimal collider path or cycle of length 4. This impliesthat the ji -edge is an arc, since otherwise a shorter direction-preserving path via thearrow from i to j is generated.Since this edge should turn into an arrow from j to i , i ∼ h and there is a node l pointing to i such that l j . Since h h, j, i, l i and h k, j, i, l i are collider paths (or cycles),on the hi -edge there is an arrowhead pointing to i , and ki is an arc. To turn the ki -arcinto an arrow from i to k there is a kp -edge with its arrowhead pointing to k , and p i .Therefore, h p, k, i, h i is a minimal collider path (or cycle), a contradiction. Therefore,there is no direction-preserving cycle after applying step 3.Therefore, the ordering of step 4 is permissible, and by step 6 there are obviously nodirection-preserving cycles generated. Therefore, G is acyclic.Now we prove that G is Markov equivalent to H : Since there is no minimal colliderpath of length 4 in H , by Theorem 2, H is Markov equivalent to G if and only if they havethe same skeleton and unshielded collider V-configurations. Graph G obviously has thesame skeleton as that of H . In addition, an unshielded collider V-configuration in G isan unshielded collider V-configuration in H . If, for contradiction, an unshielded colliderV-configuration h i, k, j i in H does not exist in G then one of the arrowheads pointing to , say on edge ik , must be removed by step 3. Therefore, there is an unshielded colliderV-configuration h l, i, k i in H . Now h l, i, k, j i is a minimal collider path since l k and i j , a contradiction.It is easy to see that, for UGs, BGs, and RCGs that can be Markov equivalent toDAGs, the algorithm generates a Markov equivalent DAG to a given RCG, steps 1 and2 of the algorithm generate a Markov equivalent DAG to a given UG, and steps 3-6 ofthe algorithm generate a Markov equivalent DAG to a given BG.Fig. 7 illustrates how to apply Algorithm 1 step by step to a MAG. k j i hlm n k=2 j i hl=1m=3 n (a) (b) j i hnklm k j i hlm n (c) (d) k=5 j=3 i=1 h=2lm n=4 k j i hlm n (e) (f)Figure 7: (a) A MAG. (b) The generated graph after applying step 1 of Algorithm 1. (c) The generated graphafter applying step 2. (d) The generated graph after applying step 3. (e) The generated graph after applyingstep 4. (f) The generated DAG after applying step 6. Conditions for representational Markov equivalence for the class of MAGs and its sub-classes to DAGs.
The following proposition shows that sufficient conditions for a givenMAG, presented in Lemma 1, are also necessary. The corollaries of this propositionillustrate the conditions under which RCGs, BGs, and UGs can be Markov equivalentto a DAG.
Theorem 3.
A maximal ancestral graph H is Markov equivalent to a DAG if and onlyif H [ ] is chordal and there is no minimal collider path or cycle of length in H . roof. ( ⇒ ) Suppose that the maximal ancestral graph H is Markov equivalent to adirected acyclic graph G . By Theorem 2 G must have the same skeleton and minimalcollider paths as those of H . Since there is no collider V-configuration in H [ ], thecorresponding induced subgraph of G should have no unshielded collision nodes. If, forcontradiction, this subgraph contains an induced C n , n ≥
4, then there exists a collisionV-configuration on the cycle, otherwise there exists a direction-preserving cycle in G ,which is not permissible. This collision V-configuration is unshielded since n ≥ C n is chordless. This is a contradiction since G and H have the same skeleton. If H contains a minimal collider path or cycle π then edges of π cannot be oriented in G togenerate a collider path.( ⇐ ) The result follows from Lemma 1.We recall the following known statement for BGs as a corollary to the proposition. Corollary 2. (Pearl & Wermuth, 1994) A BG is Markov equivalent to a DAG if andonly if it does not contain any P or C as induced subgraphs.Proof. For BGs, every path is a collider path, and every minimal collider path or cycleis a P or C . Using these, the result follows.We also recall the following known statement for UGs as a corollary to the proposition;see Lauritzen (1996). Corollary 3.
A UG is Markov equivalent to a DAG if and only if it is chordal.Proof.
For UGs, H [ ] = H , and there is no collider path in UGs. Using these, theresult follows.The following corollary shows the conditions under which RCGs can be Markov equiv-alent to DAGs. Corollary 4.
An RCG with chain component node sets τ , . . . , τ T is Markov equivalentto a DAG if and only if, H [ ] is chordal and for ≤ i ≤ T , the induced subgraph by τ i ∪ pa( τ i ) does not contain any collider P or C as an induced subgraph.Proof. For RCGs, every collider path is in one of τ i ∪ pa( τ i ), 1 ≤ i ≤ T . In addition, inRCGs a minimal collider path or cycle is chordless. Using these, the result follows. Necessary conditions for representational Markov equivalence to DAGs.
Here we in-troduce some necessary conditions for Markov equivalence for MAGs, BGs, and RCGsto a DAG. For this purpose we use the following well-known graph theoretical result:
Lemma 2.
If a graph G contains no P or C as an induced subgraph then there is anode that is adjacent to all other nodes. orollary 5. Let H be a MAG and H [ ≺ ≻ ] = ( V, E ) . If H is Markov equivalent to aDAG then there exists a node that is adjacent to all other nodes in V ∪ pa( V ) .Proof. Graphs with no minimal collider paths or cycles of length 4 do not contain acollider P or C . In addition, every collider path in MAGs is in V ∪ pa( V ). Using these,the result follows from Theorem 3 and Lemma 2. Corollary 6.
If a bidirected graph is Markov equivalent to a DAG then there exists anode that is adjacent to all other nodes.Proof.
The result follows from Corollary 2 and Lemma 2.
Corollary 7.
If an RCG is Markov equivalent to a DAG then in each τ i ∪ pa( τ i ) , ≤ i ≤ T , there exists a node that is adjacent to all other nodes in τ i ∪ pa( τ i ) .Proof. The result follows from Corollary 4 and Lemma 2.
By removing all arrow-heads one generates a Markov equivalent UG to a given MAG that satisfies the conditionof Lemma 3.
Lemma 3.
For a maximal ancestral graph H with no unshielded collider V-configuration,removing arrowheads generates a Markov equivalent UG to H .Proof. The generated graph is obviously a UG and is also the only UG that has the sameskeleton as that of H . Neither H nor the generated graph contains any minimal colliderpaths. This completes the proof.One can therefore observe that, for DAGs, BGs, and RCGs that can be Markovequivalent to UGs, removing arrowheads generates a Markov equivalent UG to the givengraph. Conditions for representational Markov equivalence for MAGs and its subclasses toUGs.
The following proposition shows that the sufficient condition for a given MAG,presented in Lemma 3, is also necessary. The corollaries of this proposition illustrate theconditions under which BGs and DAGs can be Markov equivalent to a UG.
Proposition 4.
A maximal ancestral graph H is Markov equivalent to a UG if and onlyif there is no unshielded collider V-configuration in H . roof. ( ⇒ ) Suppose that H is Markov equivalent to an undirected graph G . Graphs H and G have the same skeleton and minimal collider paths, but G has no minimalcollider paths. Since an unshielded collider V-configuration is a minimal collider path, H contains no unshielded collider V-configurations.( ⇐ ) The result follows from Lemma 3.One can also use this result for RCGs. Here we simplify the condition further forDAGs and BGs. Corollary 8.
A directed acyclic graph G is Markov equivalent to a UG if and only ifthere is no unshielded collision V-configuration in H .Proof. The result follows from the fact that the only type of colliders in DAGs is collisions.
Corollary 9.
A BG is Markov equivalent to a UG if and only if it is complete.Proof.
The result follows from the fact that all unshielded V-configurations in BGs arecollider.
Generating a BG which is Markov equivalent to a given MAG.
Replacing all edgesby arcs generates a Markov equivalent BG to a given MAG that satisfies the conditionof Lemma 4.
Lemma 4.
For a maximal ancestral graph H with no unshielded non-collider V-configuration,replacing all edges by arcs generates a Markov equivalent BG to H .Proof. The generated graph is obviously a BG and is also the only BG that has thesame skeleton as that of H . All V-configurations in both H and the generated graph arecolliders. This completes the proof.One can therefore observe that, for DAGs, UGs, and RCGs that can be Markovequivalent to BGs, replacing all edges by arcs generates a Markov equivalent BG to thegiven graph. Conditions for representational Markov equivalence for MAGs and its subclasses toBGs.
The following proposition shows that the sufficient condition for a given MAG,presented in Lemma 4, is also necessary.
Proposition 5.
A maximal ancestral graph H is Markov equivalent to a BG if and onlyif there is no unshielded non-collider V-configuration in H . roof. ( ⇒ ) Suppose that H is Markov equivalent to a bidirected graph G . Graphs H and G have the same skeleton and minimal collider paths, but every minimal path in G is a minimal collider path. Since an unshielded non-collider V-configuration is a minimalbut not a collider path, H contains no unshielded non-collider V-configurations.( ⇐ ) The result follows from Lemma 4.One can also use this result for RCGs. Here we simplify the condition further forDAGs and UGs. The results have been known in the literature for long time, e.g., seePearl & Wermuth (1994). Corollary 10.
A directed acyclic graph G is Markov equivalent to a BG if and only ifthere is no unshielded non-collision V-configuration in H .Proof. The result follows from the fact that the only type of colliders in DAGs is collisions.
Corollary 11.
A UG is Markov equivalent to a BG if and only if it is complete.Proof.
The result follows from the fact that all unshielded V-configurations in UGs arenon-collider.
We begin with analgorithm for generating a Markov equivalent RCG to a given MAG that satisfies theconditions of Lemma 5.
Algorithm 2. (Generating a Markov equivalent RCG to a MAG H )Start from H .1. For a non-collider V-configuration i ≺ ≻ j ≻ k on an arc-direction-preserving cycle,remove the arrowhead pointing to j on the ij -edge when there is no unshieldedcollider V-configuration of form h i, j, l i .Continually apply this step until it is not possible to apply it further. Lemma 5.
For a maximal ancestral graph H with no arc-direction-preserving cycle onwhich every non-collider V-configuration i ≺ ≻ j ≻ k is such that there is an unshieldedcollider V-configuration of form h i, j, l i , Algorithm 2 generates a Markov equivalent RCGto H .Proof. Denote the generated graph by G . To show G is an RCG, it is enough toshow that there is no arc-direction-preserving cycle in G . We know that the only arc-direction-preserving cycles in H are those on which there is a non-collider V-configuration ≺ ≻ j ≻ k such that there is no unshielded collider V-configuration h i, j, l i . In this casestep 3 of the algorithm generates a source V-configuration, and therefore, destroys thearc-direction-preserving cycle.We now prove that G is Markov equivalent to H : First, we prove that minimal colliderpaths in H remain unchanged in G . Suppose, for contradiction, that there is a minimalcollider path π of length n , n ≥
3, containing an ij -arc, and the arrowhead pointing to j is removed by step 3 of the algorithm because of a V-configuration i ≺ ≻ j ≻ k on anarc-direction-preserving cycle. Denote the three consecutive nodes on π by h i, j, l i . Sincethere is an arrowhead pointing to j on the jl -edge, i ∼ l . Since π is minimal collider,there exists another node on π , say h adjacent to l , i is an endpoint of π , and the li -edgeis an arrow from l to i .Now there is an arc-direction-preserving cycle h i, j, l i on which the only non-collider V-configuration j ≺ ≻ l ≻ i is such that there is a collider V-configuration h j, l, h i . There-fore, this collider V-configuration should be shielded, i.e. h ∼ j . This edge is an arrowfrom j to h because π is minimal collider. Therefore, h i, j, l, h i is a primitive inducingpath, and since H is maximal, i ∼ h . This contradicts the fact that π is a minimalcollider path. Therefore, minimal collider paths in H do not change by the algorithm.In addition, a non-minimal collider path in H cannot turn into a minimal colliderpath in G , since we know that in RCGs all collider paths are chordless. This completesthe proof.Fig. 8 illustrates how to apply Algorithm 2 to a MAG. k j il h k j il h (a) (b)Figure 8: (a) A MAG. (b) The generated RCG after applying Algorithm 2. Conditions for representational Markov equivalence for MAGs and its subclasses toRCGs.
Since UGs, BGs, and DAGs are subclasses of RCGs, there are no conditionsneeded under which they are able to be Markov equivalent to an RCG. The follow-ing proposition shows when a given MAG can be Markov equivalent to an RCG. Thecorollary of this proposition shows when UGs can be Markov equivalent to an RCG.
Theorem 4.
A maximal ancestral graph H is Markov equivalent to an RCG if and onlyif there is no arc-direction-preserving cycle on which every non-collider V-configuration i ≺ ≻ j ≻ k is such that there is an unshielded collider V-configuration of form h i, j, l i . roof. ( ⇒ ) Suppose that H is Markov equivalent to a multivariate regression chain graph G . Suppose, for contradiction, that there is an arc-direction-preserving cycle π ′ on whichevery non-collider V-configuration i ≺ ≻ j ≻ k is such that there is an unshielded colliderV-configuration h i, j, l i . If π ′ has a chord qr then there are two shorter cycles includingthe chord and nodes on π ′ that are on one side of q and r . One can observe that at leastone of the two cycles has the same property as the property of π ′ depending on whetherthere is an arrowhead pointing to q or r on ij -path. Hence, consider the minimal cycle π in this sense, which is chordless.On π all collider V-configurations are unshielded. In addition, all collider V-configurationsof form h i, j, l i are also unshielded. Therefore, since H is Markov equivalent to G , in G all these collider V-configurations should be preserved. Hence, all arcs on π remain arcsin G . Moreover, by replacing the arrows on π by arcs or by changing their directionsa new unshielded collider V-configuration is generated. Therefore, arrows on π are alsounchanged in G . Therefore, π exists in G . Since we know that arc-direction-preservingcycles are not permissible in RCGs, this is a contradiction.( ⇐ ) The result follows from Lemma 5. Summary of internal and external Markov equivalence for MAGs and its subclasses.
In Section 5, we showed that for internal and external Markov equivalences for subclassesof MAGs, excluding MAGs themselves, the conditions for Markov equivalences for DAGscan be generalised naturally by using colliders instead of collisions. In other words, twosubclasses of MAGs are Markov equivalent if and only if they have the same skeletonand unshielded collider V-configurations.
Summary of representational Markov equivalence for MAGs and its subclasses.
Thefollowing table represents the summary of the conditions needed for representationalMarkov equivalence for MAGs and its subclasses. In addition, for each non-trivial case oftable, we provided algorithms to generate a graph other types that is Markov equivalentto the graph of a given type.The conditions presented in the table are for the graphs of the type indicated on theleft column, which are to be Markov equivalent to a graph of the type indicated on thefirst row.
Necessary and sufficient conditions on H a graph of a subclass of maximal ancestral graph on the leftcolumn to be able to be Markov equivalent to a graph of the subclass of maximal ancestral graph on the toprow. H \ RCG BG UG DAGNo arc-dir-pr cycle with H [ ] Chordal;MAG every i ≺ ≻ j ≻ k s.t. No unshielded No unshielded no minimal colliderthere is an unshielded non-collider. collider path or cyclecollider h i, j, l i of length 4RCG - No unshielded No unshielded No collider P ornon-collider collider C in τ i ∪ pa( τ i )BG - - Complete No P or C UG - Complete - ChordalDAG - No unshielded No unshielded -non-collision collision
Acknowledgments
The author is grateful to Steffen Lauritzen and Nanny Wermuth for helpful comments.
References
Ali, R. A. & Richardson, T. (2004). Searching across Markov equivalence classes of max-imal ancestral graphs. In
Proceedings of the joint statistical meeting of the Americanstatistical association . Toronto, Canada.Ali, R. A., Richardson, T. & Spirtes, P. (2009). Markov equivalence for ancestral graphs.
Ann. Statist. , 2808–2837.Ali, R. A., Richardson, T., Spirtes, P. & Zhang, J. (2005). Towards characterizing Markovequivalence classes for directed acyclic graphs with latent variables. In Proceedingsof the proceedings of the twenty-first conference annual conference on uncertainty inartificial intelligence (uai-05) . AUAI Press, Arlington, Virginia, pp. 10–17.Blair, J. R. S. & Peyton, B. (1992). An introduction to chordal graphs and clique trees.
Graph Theory and Sparse Matrix Computation (eds. George, A., Gilbert, 25 J. R.,Liu, J. W.) , 1–29.Cox, D. R. & Wermuth, N. (1993). Linear dependencies represented by chain graphs(with discussion).
Stat. Sci. , 204–218; 247–277.Darroch, J. N., Lauritzen, S. L. & Speed, T. P. (1980). Markov fields and log-linearinteraction models for contingency tables. Ann. Statist. , 522–539.Frydenberg, M. (1990). The chain graph Markov property. Scand. J. Stat. , 333–353. iiveri, H., Speed, T. P. & Carlin, J. B. (1984). Recursive causal models. J. Aust. Math.Soc., Ser. A , 30–52.Koster, J. T. A. (2002). Marginalizing and conditioning in graphical models. Bernoulli , 817–840.Lauritzen, S. L. (1996). Graphical models . Clarendon Press, Oxford, United Kingdom.Pearl, J. & Wermuth, N. (1994). When can association graphs admit a causal interpre-tation?
Models and Data, Artificial Intelligence and Statistics , 205–214.Richardson, T. S. & Spirtes, P. (2002). Ancestral graph Markov models. Ann. Statist. , 962–1030.Sadeghi, K. (2011). Stable classes of graphs containing directed acyclic graphs. submitted .Sadeghi, K. & Lauritzen, S. L. (2011). Markov properties for loopless mixed graphs. submitted, http://arxiv.org/abs/1103.2523.Spirtes, P., Richardson, T. & Meek, C. (1997). The dimensionality of mixed ancestralgraphs. Tech. Rep. CMU-PHIL-83, Philosophy Department, CMU.Spirtes, P. & Richardson, T. S. (1997). A polynomial-time algorithm for determiningdag equivalence in the presence of latent variables and selection bias. In Preliminarypapers, sixth international workshop on ai and statistics . Ft. Lauderdale, FL, USA,pp. 489–501.Tarjan, R. E. & Yannakakis, M. (1984). Simple linear-time algorithms to test chordalityof graphs, test acyclicity of hypergraphs, and selectively reduce acyclic hypergraphs.
SIAM J. Comput. , 566–579.Verma, T. & Pearl, J. (1990). On the equivalence of causal models. In Proceedings ofthe proceedings of the sixth conference annual conference on uncertainty in artificialintelligence (uai-90) . Elsevier Science, New York, NY, pp. 220–227.Wermuth, N. (2011). Probability distributions with summary graph structure.
Bernoulli , 845–879.Wermuth, N. & Cox, D. R. (1998). On association models defined over independencegraphs. Bernoulli , 477–495.Wermuth, N. & Cox, D. R. (2004). Joint response graphs and separation induced bytriangular systems. J. R. Stat. Soc. Ser. B Stat. Methodol. , 687–717. ermuth, N. & Sadeghi, K. (2011). Sequences of regressions and their independences. TEST,
To appear as an invited discussion paper, http://arxiv.org/abs/1103.2523.Zhao, H., Zheng, Z. & Liu, B. (2004). On the Markov equivalence of maximal ancestralgraphs.
Sci. China Ser. A , 548–562.Kayvan Sadeghi, Department of Statistics, University of Oxford, 1 South Parks Road,Oxford, OX1 3TG, UK.Email: [email protected], 548–562.Kayvan Sadeghi, Department of Statistics, University of Oxford, 1 South Parks Road,Oxford, OX1 3TG, UK.Email: [email protected]