On Explainability of Graph Neural Networks via Subgraph Explorations
OOn Explainability of Graph Neural Networks via Subgraph Explorations
Hao Yuan Haiyang Yu Jie Wang Kang Li Shuiwang Ji Abstract
We consider the problem of explaining the pre-dictions of graph neural networks (GNNs), whichotherwise are considered as black boxes. Exist-ing methods invariably focus on explaining theimportance of graph nodes or edges but ignorethe substructures of graphs, which are more in-tuitive and human-intelligible. In this work, wepropose a novel method, known as SubgraphX,to explain GNNs by identifying important sub-graphs. Given a trained GNN model and an inputgraph, our SubgraphX explains its predictionsby efficiently exploring different subgraphs withMonte Carlo tree search. To make the tree searchmore effective, we propose to use Shapley valuesas a measure of subgraph importance, which canalso capture the interactions among different sub-graphs. To expedite computations, we proposeefficient approximation schemes to compute Shap-ley values for graph data. Our work represents thefirst attempt to explain GNNs via identifying sub-graphs explicitly. Experimental results show thatour SubgraphX achieves significantly improvedexplanations, while keeping computations at areasonable level.
1. Introduction
Graph neural networks have drawn significant attention re-cently due to their promising performance on various graphtasks, including graph classification, node classification, linkprediction, and graph generation. Different techniques havebeen proposed to improve the performance of deep graphmodels, such as graph convolution (Kipf & Welling, 2016;Gilmer et al., 2017; Gao et al., 2018; Wang et al., 2020),graph attention (Veliˇckovi´c et al., 2018; Wang et al., 2019),and graph pooling (Yuan & Ji, 2020; Gao & Ji, 2019; Zhang Department of Computer Science & Engineering, Texas A&MUniversity, TX, USA Department of Electronic Engineering andInformation Science, University of Science and Technology ofChina, Hefei, China West China Biomedical Big Data Center,West China Hospital, Chengdu, China. Correspondence to: Shui-wang Ji < [email protected] > . et al., 2018). However, these models are still treated asblack boxes, and their predictions lack explanations. With-out understanding and reasoning the relationships behindthe predictions, these models cannot be understood and fullytrusted, which prevents their applications in critical areas.This raises the need of investigating the explainability ofdeep graph models.Recently, extensive efforts have been made to study expla-nation techniques for deep models on images and text (Si-monyan et al., 2013; Yuan et al., 2019; Smilkov et al., 2017;Yuan et al., 2020a; Yang et al., 2019; Du et al., 2018). Thesemethods can explain both general network behaviors andinput-specific predictions via different strategies. However,the explainability of GNNs is still less explored. Unlikeimages and texts, graphs are not grid-like data and containimportant structural information. Thus, methods for imagesand texts cannot be applied directly. While several recentstudies have developed GNN explanation methods, suchas GNNExplainer (Ying et al., 2019), PGExplainer (Luoet al., 2020), and PGM-Explainer (Vu & Thai, 2020), theyinvariably focus on explainability at node, edge, or nodefeature levels. We argue that subgraph-level explanationsare more intuitive and useful, since subgraphs can be simplebuilding blocks of complex graphs and are highly related tothe functionalities of graphs (Alon, 2007; Milo et al., 2002).In this work, we propose the SubgraphX, a novel GNNexplanation method that can identify important subgraphsto explain GNN predictions. Specifically, we propose toemploy the Monte Carlo tree search algorithm (Silver et al.,2017) to efficiently explore different subgraphs for a giveninput graph. Since the information aggregation proceduresin GNNs can be interpreted as interactions among differ-ent graph structures, we propose to employ Shapley val-ues (Kuhn & Tucker, 1953) to measure the importance ofsubgraphs by capturing such interactions. Furthermore, wepropose efficient approximation schemes to Shapley valuesby considering interactions only within the information ag-gregation range. Altogether, our work represents the first at-tempt to explain GNNs via identifying subgraphs explicitly.We conduct both qualitative and quantitative experiments toevaluate the effectiveness and efficiency of our SubgraphX.Experimental results show that our proposed SubgraphXcan provide better explanations for a variety of GNN mod-els. In addition, our method has a reasonable computational a r X i v : . [ c s . L G ] F e b n Explainability of Graph Neural Networks via Subgraph Explorations cost given its superior performance.
2. Related Work
Graph neural networks have demonstrated their effective-ness on different graph tasks. Several approaches are pro-posed to learn representations for nodes and graphs, suchas GCNs (Kipf & Welling, 2016), GATs (Veliˇckovi´c et al.,2018), and GINs (Xu et al., 2019), etc. These methods gen-erally follow an information aggregation scheme that thefeatures of a target node are obtained by aggregating andcombining the features from its neighboring nodes. Herewe use GCNs as an example to illustrate such informationaggregation procedures. Formally, a graph G with m nodescan be represented by an adjacency matrix A ∈ { , } m × m and a feature matrix X ∈ R m × d assuming that each nodeis associated with a d -dimensional feature vector. Then theaggregation operation in GCNs can be mathematically writ-ten as X i +1 = σ ( D − ˆ AD − X i W i ) , where X i denotesthe output feature matrix of i − th GCN layer and X isset to X = X . The node features are transformed from X i ∈ R m × c i to X i +1 ∈ R m × c i +1 . Note that ˆ A = A + I is employed to add self-loops and D is a diagonal nodedegree matrix to perform normalization on ˆ A . In addition, W i ∈ R c i × c i +1 is a learnable weight matrix to perform lin-ear transformations on features and σ ( · ) is the non-linearactivation function. Even though explaining GNNs is crucial to understand andtrust deep graph models, the explainability of GNNs is stillless studied, compared with the image and text domains.Recently, several methods are proposed specifically to ex-plain deep graph models. These methods mainly focus onexplaining GNNs by identifying important nodes, edges,node features. However, none of them can provide input-dependent subgraph-level explanations, which is importantfor understanding graph models. Following a recent surveywork (Yuan et al., 2020c), we categorize these methods intoseveral classes; those are, gradients/features-based meth-ods, decomposition methods, surrogate methods, generation-based methods, and perturbation-based methods.First, several methods employ gradient values or featurevalues to study the importance of input graph nodes, edges,or node features (Baldassarre & Azizpour, 2019; Pope et al.,2019). These methods generally extend existing image ex-planation techniques to the graph domain, such as SA (Zeiler& Fergus, 2014), CAM (Zhou et al., 2016), and GuidedBP (Springenberg et al., 2015). While these methods aresimple and efficient, they cannot incorporate the specialproperties of graph data. Meanwhile, decomposition meth- ods, such as LRP (Schwarzenberg et al., 2019), ExcitationBP (Pope et al., 2019), and GNN-LRP (Schnake et al., 2020),explain GNNs by decomposing the original model predic-tions into several terms and associating these terms withgraph nodes or edges. These methods generally follow abackpropagation manner to decompose predictions layer bylayer until input space. In addition, existing methods (Huanget al., 2020; Vu & Thai, 2020) employ a simple and inter-pretable model as the surrogate method to capture localrelationships of deep graph models around the input data.Then the explanations of the surrogate method are treatedas the explanations of the original predictions. Furthermore,recent work XGNN (Yuan et al., 2020b) proposes to studygeneral and high-level explanations of GNNs by generatinggraph patterns to maximize a certain prediction.In addition, a popular direction to explain GNNs is knownas the perturbation-based method. It monitors the changesin the predictions by perturbing different input features andidentifies the features affecting predictions the most. Forexample, GNNExplainer (Ying et al., 2019) optimizes softmasks for edges and node features to maximize the mutualinformation between the original predictions and new pre-dictions. Then the optimized masks can identify importantedges and features. Meanwhile, PGExplainer (Luo et al.,2020) learns a parameterized model to predict whether anedge is important, which is trained using all edges in thedataset. It employs the reparameterization trick (Jang et al.,2016) to obtain approximated discrete masks instead of softmasks. In addition, GraphMask (Schlichtkrull et al., 2020)follows a similar idea as PGExplainer that train a classifierto predict if an edge can be dropped without affecting modelpredictions. However, it studies the edges in every GNNlayer while PGExplainer only focuses on the input space.
3. The Proposed SubgraphX
While most current methods for GNN explanations are in-variably based on identifying important nodes or edges, weargue that identifying important subgraphs is more naturaland may lead to better explainability. In this work, we pro-pose a novel approach, known as SubgraphX, to explainGNNs by exploring and identifying important subgraphs.
Unlike images and texts, graph data contain important struc-tural information, which is highly related to the propertiesof graphs. For example, network motifs, which can be con-sidered as graph substructures, are simple building blocks ofcomplex networks and may determine the functionalities ofgraphs in many domains, such as biochemistry, ecology, neu-robiology, and engineering (Alon, 2007; Milo et al., 2002;Shen-Orr et al., 2002; Alon, 2019). Hence, investigatinggraph substructures is a crucial step towards the reverse en- n Explainability of Graph Neural Networks via Subgraph Explorations Root
34 5
34 5 Leaves Coalition 1
12 34 5 GNNs GNNs
Coalition 3
12 34 5 GNNs Shapley value0.50 0.450.30 0.55UpdateUpdateUpdateUpdate
Figure 1.
An illustration of our proposed SubgraphX. The bottom shows one selected path from the root to leaves in the search tree, whichcorresponds to one iteration of MCTS. For each node, its subgraph is evaluated by computing the Shapley value via Monte-Carlo sampling.In this example, we show the computation of Shapley value for the middle node (shown in red dashed box) where three coalitions aresampled to compute the marginal contributions. Note that nodes that are not selected are ignored for simplicity. gineering and understanding of the underlying mechanismsof GNNs. In addition, subgraphs are more intuitive andhuman-intelligible (Yuan et al., 2020c).While different methods are proposed to explain GNNs,none of them can directly provide subgraph-level explana-tions for individual input examples. The XGNN can obtaingraph patterns to explain GNNs but its explanations are notinput-dependent and less precise. The other methods, suchas GNNExplainer and PGExplainer, may obtain subgraph-level explanations by combining nodes or edges to formsubgraphs in a post-processing manner. However, the impor-tant nodes or edges in their explanations are not guaranteedto be connected. Meanwhile, since GNNs are very com-plex, node/edge importance cannot be directly converted tosubgraph importance. Furthermore, these methods ignorethe interactions among different nodes and edges, whichmay contain important information. Hence, in this work, wepropose a novel method, known as SubgraphX, to directlystudy the subgraphs to provide explanations. The explana-tions of our SubgraphX are connected subgraphs, which aremore human-intelligible. In addition, by incorporating Shap-ley values, our method can capture the interactions amongdifferent graph structures when providing explanations.
We first present a formal problem formulation. Let f ( · ) denote the trained GNNs to be explained. Without lossof generality, we introduce our proposed SubgraphX byconsidering f ( · ) as a graph classification model. Given aninput graph G , its predicted class is represented as y . Thegoal of our explanation task is to find the most important subgraph for the prediction y . Since disconnected nodes arehard to understand, we only consider connected subgraphsto enable the explanations to be more human-intelligible.Then the set of connected subgraphs of G is denoted as {G , · · · , G i , · · · , G n } where n is the number of differentconnected subgraphs in G . The explanation of prediction y for input graph G can then be defined as G ∗ = argmax |G i |≤ N min Score ( f ( · ) , G , G i ) , (1)where Score ( · , · , · ) is a scoring function for evaluating theimportance of a subgraph given the trained GNNs and theinput graph. We use N min as an upper bound on the sizeof subgraphs so that the obtained explanations are succinctenough. A straightforward way to obtain G ∗ is to enumerateall possible G i and select the most important one as the ex-planation. However, such a brute-force method is intractablewhen the graph is complex and large-scale. Hence, in thiswork, we propose to incorporate search algorithms to ex-plore subgraphs efficiently. Specifically, we propose to em-ploy Monte Carlo Tree Search (MCTS) (Silver et al., 2017;Jin et al., 2020) as the search algorithm. In addition, sincethe information aggregation procedures in GNNs can be un-derstood as interactions between different graph structures,we propose to employ the Shapley value (Kuhn & Tucker,1953) as the scoring function to measure the importance ofdifferent subgraphs by considering such interactions. Weillustrate our proposed SubgraphX in Figure 1. After search-ing, the subgraph with the highest score is considered asthe explanation of the prediction y for input graph G . Notethat our proposed SubgraphX can be easily extended to useother search algorithms and scoring functions. n Explainability of Graph Neural Networks via Subgraph Explorations In our proposed SubgraphX, we employ the MCTS as thesearch algorithm to guide our subgraph explorations. Webuild a search tree in which the root is associated with theinput graph and each of other nodes corresponds to a con-nected subgraph. Each edge in our search tree denotes thatthe graph associated with a child node can be obtained byperforming node-pruning from the graph associated with itsparent node. Formally, we define a node in this search tree as N i , and N denotes the root node. The edges in the searchtree represent the pruning actions a . Note that each nodemay have many pruning actions, and these actions can bedefined based on the dataset at hand or domain knowledge.Then the MCTS algorithm records the statistics of visitingcounts and rewards to guide the exploration and reduce thesearch space. Specifically, for the node and pruning actionpair ( N i , a j ) , we assume that the subgraph G j is obtainedby action a j from G i . Then the MCTS algorithm recordsfour variables for ( N i , a j ) , which are defined as:• C ( N i , a j ) denotes the number of counts for selectingaction a j for node N i .• W ( N i , a j ) is the total reward for all ( N i , a j ) visits.• Q ( N i , a j ) = W ( N i , a j ) /C ( N i , a j ) and denotes the av-eraged reward for multiple visits.• R ( N i , a j ) is the immediate reward for selecting a j on N i ,which is used to measure the importance of subgraph G j .We propose to use R ( N i , a j ) = Score ( f ( · ) , G , G j ) .In each iteration, the MCTS selects a path starting from theroot N to a leaf node N (cid:96) . Note that the leaf nodes can bedefined based on the numbers of nodes in subgraphs suchthat |N (cid:96) | ≤ N min . Formally, the action selection criteria ofnode N i are defined as a ∗ = argmax a j Q ( N i , a j ) + U ( N i , a j ) , (2) U ( N i , a j ) = λR ( N i , a j ) (cid:112)(cid:80) k C ( N i , a k )1 + C ( N i , a j ) , (3)where λ is a hyperparameter to control the trade-off betweenexploration and exploitation. In addition, (cid:80) k C ( N i , a k ) denotes the total visiting counts for all possible actions ofnode N i . Then the subgraph in the leaf node N (cid:96) is evaluatedand the importance score is denoted as Score ( f ( · ) , G , G (cid:96) ) .Finally, all node and action pairs selected in this path areupdated as C ( N i , a j ) = C ( N i , a j ) + 1 , (4) W ( N i , a j ) = W ( N i , a j ) + Score ( f ( · ) , G , G (cid:96) ) . (5)After searching for several iterations, we select the subgraphwith the highest score from the leaves as the explanation.Note that in early iterations, the MCTS tends to select childnodes with low visit counts in order to explore different possible pruning actions. In later iterations, the MCTStends to select child nodes that yield higher rewards, i.e. ,more important subgraphs. In our proposed SubgraphX, both the MCTS rewards and theexplanation selection are highly depending on the scoringfunction Score ( · , · , · ) . It is crucial to properly measure theimportance of different subgraphs. One possible solution isto directly feed the subgraphs to the trained GNNs f ( · ) anduse the predicted scores as the importance scores. However,it cannot capture the interactions between different graphstructures, thus affecting the explanation results. Hence, inthis work, we propose to adopt the Shapley values (Kuhn &Tucker, 1953; Lundberg & Lee, 2017; Chen et al., 2018) asthe scoring function. The Shapley value is a solution conceptfrom the cooperative game theory for fairly assigning a totalgame gain to different game players. To apply it to graphmodel explanation tasks, we use the GNN prediction as thegame gain and different graph structures as players.Formally, given the input graph G with m nodes and thetrained GNN f ( · ) , we study the Shapley value for a targetsubgraph G i with k nodes. Let V = { v , · · · , v i , · · · , v m } denote all nodes in G and we assume that the nodes in G i are { v , · · · , v k } while the other nodes { v k +1 , · · · , v m } belong to G \ G i . Then the set of players is defined as P = {G i , v k +1 , · · · , v m } , where we consider the wholesubgraph G i as one player. Finally, the Shapley value of theplayer G i can be computed as φ ( G i ) = (cid:88) S ⊆ P \{G i } | S | ! ( | P | − | S | − | P | ! m ( S, G i ) , (6) m ( S, G i ) = f ( S ∪ {G i } ) − f ( S ) , (7)where S is the possible coalition set of players. Note that m ( S, G i ) represents the the marginalized contribution ofplayer G i given the coalition set S . It can be computedby the difference of predictions between incorporating G i with and without the coalition set S . The obtained Shapleyvalue φ ( G i ) considers all different coalitions to capture theinteractions among different players. It is the only solutionthat satisfies four desirable axioms, including efficiency,symmetry, linearity, and dummy axiom (Lundberg & Lee,2017), which can guarantee the correctness and fairness ofthe explanations. However, computing Shapley values usingEqs. (6) and (7) is time-consuming as it enumerates allpossible coalitions, especially for large-scale and complexgraphs. Hence, in this work, we propose to incorporate theGNN architecture information f ( · ) to efficiently approxi-mate Shapley values. n Explainability of Graph Neural Networks via Subgraph Explorations Algorithm 1
The algorithm of our proposed SubgraphX.
Input:
GNN model f ( · ) , input graph G , MCTS iterationnumber M , the leaf threshold node number N min , h ( N i ) denotes the associated subgraph of tree node N i . Initialization: for each ( N i , a j ) pair , initialize its C , W , Q , and R variables as 0. The root of search tree is N associated with graph G . The leaf set is set to S (cid:96) = {} . for i = 1 to M do curN ode = N , curP ath = [ N ] while h ( curN ode ) has more node than N min dofor all possible pruning actions of h ( curN ode ) do Obtain child node N j and its subgraph G j .Compute R ( curN ode, a j ) = Score ( f ( · ) , G , G j )) with Algorithm 2. end for Select the child N next following Eq.(2, 3). curN ode = N next , curP ath = curP ath + N next . end while S (cid:96) = S (cid:96) ∪ { curN ode } Update nodes in curP ath following Eq.(4, 5). end for
Select subgraph with the highest score from S (cid:96) . In graph neural networks, the new features of a target nodeare obtained by aggregating information from a limitedneighboring region. Assuming there are L layers of GNNin the graph model f ( · ) , then only the neighboring nodeswithin L -hops are used for information aggregation. Notethat the information aggregation schema can be consideredas interactions between different graph structures. Hence,the subgraph G i mostly interacts with the neighbors within L -hops. Based on such observations, we propose to com-pute the Shapley value of G i by only considering its L -hop neighboring nodes. Specifically, assuming there are r ( r ≤ m − k ) nodes within L -hop neighboring of sub-graph G i , we denote these nodes as { v k +1 , · · · , v r } . Thenthe new set of players we need to consider is represented as P (cid:48) = {G i , v k +1 , · · · , v r } . By incorporating P (cid:48) , the Shapleyvalue of G i can be defined as φ ( G i ) = (cid:88) S ⊆ P (cid:48) \{G i } | S | ! ( | P (cid:48) | − | S | − | P (cid:48) | ! m ( S, G i ) . (8)However, since graph data are complex that different nodeshave variable numbers of neighbors, then P (cid:48) may still con-tain a large number of players, thus affecting the efficiencyof computation. Hence, in our SubgraphX, we further incor-porate the Monte-Carlo sampling ( ˇStrumbelj & Kononenko,2014) to compute φ ( G i ) . Specifically, for sampling step i ,we sample a coalition set S i from the player set P (cid:48) \ {G i } and compute its marginalized contribution m ( S i , G i ) . Then Algorithm 2
The algorithm of subgraph Shapley value.
Input:
GNN model f ( · ) with L layers, input graph G with nodes V = { v , . . . , v m } , subgraph G i with k nodes { v , . . . , v k } , Monte-Carlo sampling steps T . Initialization:
Obtain the L -hop neighboring nodes of G i , denoted as { v k +1 , · · · , v r } . Then the set of players is P (cid:48) = {G i , v k +1 , · · · , v r } . for i = 1 to T do Sampling a coalition set S i from P (cid:48) \ {G i } .Set nodes from V \ ( S i ∪ {G i } ) with zero features andfeed to the GNNs f ( · ) to obtain f ( S i ∪ {G i } ) .Set nodes from V \ S i with zero features and feed tothe GNNs f ( · ) to obtain f ( S i ) .Then m ( S i , G i ) = f ( S i ∪ {G i } ) − f ( S i ) . end forReturn: Score ( f ( · ) , G , G i ) = T (cid:80) Tt =1 m ( S i , G i ) .the averaged contribution score for multiple sampling stepsis regarded as the approximation of φ ( G i ) . Formally, it canbe mathematically written as φ ( G i ) = 1 T T (cid:88) t =1 ( f ( S i ∪ {G i } ) − f ( S i )) , (9)where T is the total sampling steps. In addition, to computethe marginalized contribution, we follow a zero-paddingstrategy. Specifically, to compute f ( S i ∪ {G i } ) , we con-sider the nodes V \ ( S i ∪ {G i } ) which are not belonging tothe coalition or the subgraph and set their node features toall zeros. Then we feed the new graph to the GNNs f ( · ) anduse the predicted probability as f ( S i ∪ {G i } ) . Similarly,we can compute f ( S i ) by setting nodes V \ S i with zerofeatures and feeding to the GNNs. It is noteworthy that weonly perturb the node features instead of removing the nodesfrom the input graph because graphs are very sensitive tostructural changes (Schlichtkrull et al., 2021). Finally, weconclude the computation steps of our proposed SubgraphXin Algorithm 1 and 2. We have described our proposed SubgraphX using graphclassification models as an example. It is noteworthy thatour SubgraphX can be easily generalized to explain graphmodels on other tasks, such as node classification and linkprediction. For node classification models, the explanationtarget is the prediction of a single node v i given the inputgraph G . Assuming there are L layers in the GNN models,the prediction of v i only relies on its L -hop computationgraph, denoted as G c . Then instead of searching from theinput graph G , our SubgraphX sets G c as the correspond-ing graph of the search tree root N . In addition, whencomputing the marginalized contributions, the zero-padding n Explainability of Graph Neural Networks via Subgraph Explorations SubgraphX MCTS_GNN PGExplainer GNNExplainer
Figure 2.
Explanation results on the BA-2Motifs dataset with aGCN graph classifier. The first row shows explanations for acorrect prediction and the second row reports the results for anincorrect prediction. strategy should exclude the target node v i . Meanwhile, forlink prediction tasks, the explanation target is the predictionof a single link ( v i , v j ) . Then the root of the search treecorresponds to the L -hop computation graph of node v i and v j . Similarly, the zero-padding strategy ignores the v i and v j when perturbing node features. Note that our SubgraphXtreats the GNNs as black boxes during the explanation stageand only needs to access the inputs and outputs. Hence, ourproposed SubgraphX can be applied to a general family ofGNN models, including but not limited to GCNs (Kipf &Welling, 2016), GATs (Veliˇckovi´c et al., 2018), GINs (Xuet al., 2019), and Line-Graph NNs (Chen et al., 2017).
4. Experimental Studies
We conduct extensive experiments on different datasetsand GNN models to demonstrate the effectiveness of ourproposed method. We evaluate our SubgraphX with fivedatasets for both graph classification and node classificationtasks, including synthetic data, biological data, and text data.We summarize these datasets as below:• MUTAG (Debnath et al., 1991) and BBBP (Wu et al.,2018) are molecular datasets for graph classification tasks.In these datasets, each graph represents a molecule whilenodes are atoms and edges are bonds. The labels aredetermined by the chemical functionalities of molecules.• Graph-SST2 (Yuan et al., 2020c) is sentiment graphdataset for graph classification. It converts text sentencesto graphs with Biaffine parser (Gardner et al., 2018) thatnodes denote words and edges represent the relationshipsbetween words. Note that node embeddings are initializedas the pre-trained BERT word embeddings (Devlin et al.,2018). Each graph is labeled by its sentiment, which canbe positive or negative.• BA-2Motifs is a synthetic graph classification dataset.Each graph contains a based graph generated by
Barab´asi-Albert (BA) model, which is connected with a house-likemotif or a five-node cycle motif. The graphs are labeled
SubgraphX MCTS_GNN PGExplainer GNNExplainer
Figure 3.
Explanation results on the MUTAG dataset with a GINgraph classifier. We show the explanations for two correct predic-tions. Here Carbon, Oxygen, and Nitrogen are shown in yellow,red, and blue, respectively. based on the type of motifs. All node embeddings areinitialized as vectors containing all 1s.• BA-Shape is a synthetic node classification dataset. Eachgraph contains a base BA graph and several house-likefive-node motifs. The node labels are determined by thememberships and locations of different nodes. All nodeembeddings are initialized as vectors containing all 1s.We explore three variants of GNNs on these datasets, includ-ing GCNs, GATs, and GINs. All GNN models used in ourexperimental studies are trained to obtain reasonable perfor-mance. Then we compare our SubgraphX with several base-lines, including MCTS GNN, GNNExplainer (Ying et al.,2019), PGExplainer (Luo et al., 2020). Here MCTS GNNdenotes the method using MCTS to explore subgraphs butdirectly employing the GNN predictions of these subgraphsas the scoring function. More details regarding the datasets,trained GNN models, and experimental settings can befound in Supplementary Section A. Our code and data willbe released after the anonymous review period.
We first visually compare our SubgraphX with the otherbaselines using graph classification models. The results arereported in Figure 2, 3, and 4 where important substructuresare shown in the bold.The explanation results of the BA-2Motifs dataset are visu-alized in Figure 2. We use the GCNs as the graph classifierand report explanations for both correct and incorrect pre-dictions. Since it is a synthetic dataset, we may consider themotifs as reasonable approximations of explanation groundtruth. In the first row, the model prediction is correct andour SubgraphX can precisely identify the house-like motifas the most important subgraph. In the second row, ourSubgraphX explains the incorrect prediction that the GNNmodel cannot capture the five-node cycle motif as the impor-tant structure, and hence the prediction is wrong. For bothcases, our SubgraphX can provide better visual explanations.In addition, our explanations are connected subgraphs whilePGExplainer and GNNExplainer identify discrete edges. n Explainability of Graph Neural Networks via Subgraph Explorations
SubgraphX diggs lathan , their charm havemakesrapportscreen new . seem and thestory old diggs lathan , their charm havemakesrapportscreen new . seem and thestory old diggs lathan , their charm havemakesrapportscreen new . seem and thestory old diggs lathan , their charm havemakesrapportscreen new . seem and thestory old “ lathan and diggs have considerable personal charm, and their screen rapport makes the old story new. ”“ maybe it is asking too much, but if a movie is truly going to inspire me, I want a little more than this. ” askinggoingbutif wantlittlemovie inspirethanmore ,muchtome asking going but if wantlittlemovie inspirethanmore , muchtome asking going but if wantlittlemovie inspirethanmore , muchtome asking going but if trulywantlittlemovie inspirethanmore , muchtomeis truly truly trulyis is is MCTS_GNN PGExplainer GNNExplainer
Figure 4.
Explanation results on the Graph-SST2 dataset with a GAT graph classifier. The input sentences are shown on the top ofexplanations. Note that some “unimportant” words are ignored for simplicity. The first row shows explanations for a correct predictionand the second row reports the results for an incorrect prediction.
We also show the explanation results of the MUTAG datasetin Figure 3. Note that GINs are employed as the graphclassification model to be explained. Since the MUTAGdataset is a real-world dataset and there is no ground truthfor explanations, we evaluate the explanation results basedon chemical domain knowledge. The graphs in MUTAGare labeled based on the mutagenic effects on a bacterium.It is known that carbon rings and
N O groups tend to bemutagenic (Debnath et al., 1991). In both examples, thepredictions are “mutagenic” and our SubgraphX success-fully and precisely identifies the carbon rings as importantsubgraphs. Meanwhile, the MCTS GNN can capture thekey subgraphs but include several additional edges. Theresults of the PGExplainer and GNNExplainer still containseveral discrete edges.For the dataset Graph-SST2, we employ GATs as the graphmodel and report the results in Figure 4. In the first row,the prediction is correct and the label is positive. Both ourSubgraphX and the MCTS GNN can find word phrases withpositive semantic meaning, such as “makes old story new”,which can reasonably explain the prediction. The expla-nations provided by PGExplainer and GNNExplainer are,however, less semantically related. In the second row, theinput is negative but the prediction is positive. All methodsexcept PGExplainer can explain the decision that the GNNmodel regards positive phrases “truly going to inspire” asimportant, thus yielding a positive but incorrect prediction.It is noteworthy that our method tends to include fewerneural words, such as “the”, “me”, and “screen”, etc.Overall, our SubgraphX can explain both correct and incor-rect predictions for different graph data and GNN models.Our explanations are more human-intelligible than compar-ing methods. More results for graph classification modelsare reported in Supplementary Section B. SubgraphX MCTS_GNN
PGExplainer
GNNExplainer
Figure 5.
Explanation results on the BA-Shape dataset. The targetnode is shown in a larger size. Different colors denote node labels.
We also compare different methods on the node classifica-tion tasks. We use the BA-Shape dataset and train a GCNmodel to perform node classification. The visualization re-sults are reported in Figure 5 where the important substruc-tures are shown in bold. We can verify if the explanationsare consistent with the rules (the motifs) to label differentnodes. For both examples, the target nodes are correctlyclassified. Obviously, our SubgraphX is precisely target-ing the motifs as the explanations, which is reasonable andpromising. For other methods, their explanations only coverpartial motifs and include other structures. More results arereported in Supplementary Section C.
While visualizations are important to evaluate different ex-planation methods, human evaluations may not be accuratedue to the lack of ground truths. Hence, we further conductquantitative studies to compare these methods. Specifically,we employ the metrics Fidelity and Sparsity to evaluateexplanation results (Pope et al., 2019; Yuan et al., 2020c).The Fidelity metric measures whether the explanations arefaithfully important to the model’s predictions. It removesthe important structures from the input graphs and com- n Explainability of Graph Neural Networks via Subgraph Explorations
Figure 6.
The quantitative studies for different explanation methods. Note that since the Sparsity scores cannot be fully controlled, wecompare different methods with Fidelity scores under similar similar levels of Sparsity.
Table 1.
Efficiency studies of different methods.
Method MCTS ∗ MCTS † SubgraphX GNNExplainer PGExplainer T IME >
10 hours . ± . s . ± . s . ± . s . s (Training 362s)F IDELITY
N/A 0.53 0.55 0.19 0.18putes the difference between predictions. In addition, theSparsity metric measures the fraction of structures that areidentified as important by explanation methods. Note thathigh Sparsity scores mean smaller structures are identifiedas important, which can affect the Fidelity scores sincesmaller structures (high Sparsity) tend to be less important(low Fidelity). Hence, for fair comparisons, we comparedifferent methods using Fidelity under similar levels of Spar-sity. The results are reported in Figure 6 where we plot thecurves of Fidelity scores with respect to the Sparsity scores.Obviously, for five out of six experiments, our proposedmethod outperforms the comparing methods significantlyand consistently under different sparsity levels. For the BA-Shape (GCN) experiment, our SubgraphX obtains slightlylower but still competitive Fidelity scores compared withthe PGExplainer. Overall, such results indicate that the ex-planations of our method are more faithful and importantto the GNN models. More details of evaluation metrics areintroduced in Supplementary Section A.
Finally, we study the efficiency of our proposed method. For50 graphs with an average of 24.96 nodes from the BBBPdataset, we show the averaging time cost to obtain explana-tions for each graph. We repeat the experiments 3 times andreport the results Table 1. Here MCTS ∗ denotes the baselinethat follows Eq. (8) to compute Shapley values. Comparedwith our SubgraphX, the difference is the usage of Monte Carlo sampling. In addition, MCTS † indicates the baselinecomputing Shapley values with Monte Carlo sampling butwithout our proposed approximation schemes. Specifically,MCTS † samples coalition sets from the player set P in-stead of the reduced set P (cid:48) . First, the time cost of MCTS ∗ isextremely high since it needs to enumerate all possible coali-tion sets. Next, compared with MCTS † , our SubgraphX is11 times faster while the obtained explanations have similarFidelity scores. It demonstrates our approximation schemesare both effective and efficient. Even though our method isslower than GNNExplainer and PGExplainer, the Fidelityscores of our explanations are 300% higher than theirs. Fur-thermore, the PGExplainer requires to train its model usingthe whole dataset, which introduces the additional and sig-nificant time cost. Considering our explanations are withhigher-quality and more human-intelligible, we believe suchtime complexity is reasonable and acceptable.
5. Conclusions
While considerable efforts have been devoted to study theexplainability of GNNs, none of existing methods can ex-plain GNN predictions with subgraphs. We argue that sub-graphs are building blocks of complex graphs and are morehuman-intelligible. To this end, we propose the SubgraphXto explain GNNs by identifying important subgraphs explic-itly. We employ the Monte Carlo tree search algorithm toefficiently explore different subgraph. For each subgraph,we propose to employ Shapley values to measure its im- n Explainability of Graph Neural Networks via Subgraph Explorations portance by considering the interactions among differentgraph structures. To expedite computations, we proposeefficient approximation schemes to compute Shapley valuesby considering interactions only within the information ag-gregation range. Experimental results show our SubgraphXobtain higher-quality and more human-intelligible explana-tions while keeping time complexity acceptable.
References
Alon, U. Network motifs: theory and experimental ap-proaches.
Nature Reviews Genetics , 8(6):450–461, 2007.Alon, U.
An introduction to systems biology: design princi-ples of biological circuits . CRC press, 2019.Baldassarre, F. and Azizpour, H. Explainability tech-niques for graph convolutional networks. In
InternationalConference on Machine Learning (ICML) Workshops,2019 Workshop on Learning and Reasoning with Graph-Structured Representations , 2019.Chen, J., Song, L., Wainwright, M. J., and Jordan, M. I. L-shapley and c-shapley: Efficient model interpretation forstructured data. arXiv preprint arXiv:1808.02610 , 2018.Chen, Z., Li, X., and Bruna, J. Supervised communitydetection with line graph neural networks. arXiv preprintarXiv:1705.08415 , 2017.Debnath, A. K., Lopez de Compadre, R. L., Debnath, G.,Shusterman, A. J., and Hansch, C. Structure-activity rela-tionship of mutagenic aromatic and heteroaromatic nitrocompounds. correlation with molecular orbital energiesand hydrophobicity.
Journal of medicinal chemistry , 34(2):786–797, 1991.Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert:Pre-training of deep bidirectional transformers for lan-guage understanding. arXiv preprint arXiv:1810.04805 ,2018.Du, M., Liu, N., Song, Q., and Hu, X. Towards explanationof dnn-based prediction with guided feature inversion.In
Proceedings of the 24th ACM SIGKDD InternationalConference on Knowledge Discovery & Data Mining , pp.1358–1367, 2018.Gao, H. and Ji, S. Graph U-Nets. In international conferenceon machine learning , pp. 2083–2092. PMLR, 2019.Gao, H., Wang, Z., and Ji, S. Large-scale learnable graphconvolutional networks. In
Proceedings of the 24th ACMSIGKDD International Conference on Knowledge Dis-covery and Data Mining , pp. 1416–1424, 2018.Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi,P., Liu, N., Peters, M., Schmitz, M., and Zettlemoyer, L. Allennlp: A deep semantic natural language processingplatform. arXiv preprint arXiv:1803.07640 , 2018.Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., andDahl, G. E. Neural message passing for quantum chem-istry. In
International Conference on Machine Learning ,pp. 1263–1272. PMLR, 2017.Huang, Q., Yamada, M., Tian, Y., Singh, D., Yin, D., andChang, Y. Graphlime: Local interpretable model ex-planations for graph neural networks. arXiv preprintarXiv:2001.06216 , 2020.Jang, E., Gu, S., and Poole, B. Categorical reparameteriza-tion with gumbel-softmax. In
International Conferenceon Learning Representations , 2016.Jin, W., Barzilay, R., and Jaakkola, T. Multi-objectivemolecule generation using interpretable substructures. In
International Conference on Machine Learning , pp. 4849–4859. PMLR, 2020.Kipf, T. N. and Welling, M. Semi-supervised classifica-tion with graph convolutional networks. arXiv preprintarXiv:1609.02907 , 2016.Kuhn, H. W. and Tucker, A. W.
Contributions to the Theoryof Games , volume 2. Princeton University Press, 1953.Lundberg, S. M. and Lee, S.-I. A unified approach to inter-preting model predictions. In
Advances in neural infor-mation processing systems , pp. 4765–4774, 2017.Luo, D., Cheng, W., Xu, D., Yu, W., Zong, B., Chen, H.,and Zhang, X. Parameterized explainer for graph neuralnetwork. In
Advances in neural information processingsystems , 2020.Milo, R., Shen-Orr, S., Itzkovitz, S., Kashtan, N.,Chklovskii, D., and Alon, U. Network motifs: simplebuilding blocks of complex networks.
Science , 298(5594):824–827, 2002.Pope, P. E., Kolouri, S., Rostami, M., Martin, C. E., andHoffmann, H. Explainability methods for graph convo-lutional neural networks. In
Proceedings of the IEEEConference on Computer Vision and Pattern Recognition ,pp. 10772–10781, 2019.Schlichtkrull, M. S., De Cao, N., and Titov, I. Interpretinggraph neural networks for nlp with differentiable edgemasking. arXiv preprint arXiv:2010.00577 , 2020.Schlichtkrull, M. S., Cao, N. D., and Titov, I. Interpret-ing graph neural networks for { nlp } with differentiableedge masking. In International Conference on LearningRepresentations , 2021. URL https://openreview.net/forum?id=WznmQa42ZAx . n Explainability of Graph Neural Networks via Subgraph Explorations Schnake, T., Eberle, O., Lederer, J., Nakajima, S., Sch¨utt,K. T., M¨uller, K.-R., and Montavon, G. Xai for graphs:Explaining graph neural network predictions by identi-fying relevant walks. arXiv preprint arXiv:2006.03589 ,2020.Schwarzenberg, R., H¨ubner, M., Harbecke, D., Alt, C.,and Hennig, L. Layerwise relevance visualization inconvolutional text graph classifiers. arXiv preprintarXiv:1909.10911 , 2019.Shen-Orr, S. S., Milo, R., Mangan, S., and Alon, U. Net-work motifs in the transcriptional regulation network ofescherichia coli.
Nature genetics , 31(1):64–68, 2002.Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou,I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M.,Bolton, A., et al. Mastering the game of go withouthuman knowledge. nature , 550(7676):354–359, 2017.Simonyan, K., Vedaldi, A., and Zisserman, A. Deep in-side convolutional networks: Visualising image clas-sification models and saliency maps. arXiv preprintarXiv:1312.6034 , 2013.Smilkov, D., Thorat, N., Kim, B., Vi´egas, F., and Watten-berg, M. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 , 2017.Springenberg, J. T., Dosovitskiy, A., Brox, T., and Ried-miller, M. Striving for simplicity: The all convolutionalnet.
International Conference on Learning Representa-tions , 2015.ˇStrumbelj, E. and Kononenko, I. Explaining predictionmodels and individual predictions with feature contribu-tions.
Knowledge and information systems , 41(3):647–665, 2014.Veliˇckovi´c, P., Cucurull, G., Casanova, A., Romero, A.,Li`o, P., and Bengio, Y. Graph attention networks. In
International Conference on Learning Representations ,2018.Vu, M. N. and Thai, M. T. Pgm-explainer: Probabilisticgraphical model explanations for graph neural networks.In
Advances in neural information processing systems ,2020.Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., andYu, P. S. Heterogeneous graph attention network. In
TheWorld Wide Web Conference , pp. 2022–2032, 2019.Wang, Z., Liu, M., Luo, Y., Xu, Z., Xie, Y., Wang, L.,Cai, L., and Ji, S. Advanced graph and sequence neu-ral networks for molecular property prediction and drugdiscovery. arXiv preprint arXiv:2012.01981 , 2020. Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Ge-niesse, C., Pappu, A. S., Leswing, K., and Pande, V.Moleculenet: a benchmark for molecular machine learn-ing.
Chemical science , 9(2):513–530, 2018.Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerfulare graph neural networks? In
International Conferenceon Learning Representations , 2019. URL https://openreview.net/forum?id=ryGs6iA5Km .Yang, F., Pentyala, S. K., Mohseni, S., Du, M., Yuan, H.,Linder, R., Ragan, E. D., Ji, S., and Hu, X. Xfake: ex-plainable fake news detector with visualizations. In
TheWorld Wide Web Conference , pp. 3600–3604, 2019.Ying, Z., Bourgeois, D., You, J., Zitnik, M., and Leskovec, J.Gnnexplainer: Generating explanations for graph neuralnetworks. In
Advances in neural information processingsystems , pp. 9244–9255, 2019.Yuan, H. and Ji, S. Structpool: Structured graph pooling viaconditional random fields. In
International Conferenceon Learning Representations , 2020.Yuan, H., Chen, Y., Hu, X., and Ji, S. Interpreting deep mod-els for text analysis via optimization and regularizationmethods. In
AAAI-19: Thirty-Third AAAI Conference onArtificial Intelligence . Association for the Advancementof Artificial Intelligence, 2019.Yuan, H., Cai, L., Hu, X., Wang, J., and Ji, S. Interpretingimage classifiers by generating discrete masks.
IEEETransactions on Pattern Analysis and Machine Intelli-gence , 2020a.Yuan, H., Tang, J., Hu, X., and Ji, S. XGNN: Towardsmodel-level explanations of graph neural networks. KDD’20, pp. 430–438, New York, NY, USA, 2020b. Associa-tion for Computing Machinery. ISBN 9781450379984.doi: 10.1145/3394486.3403085. URL https://doi.org/10.1145/3394486.3403085 .Yuan, H., Yu, H., Gui, S., and Ji, S. Explainability in graphneural networks: A taxonomic survey. arXiv preprintarXiv:2012.15445 , 2020c.Zeiler, M. D. and Fergus, R. Visualizing and understand-ing convolutional networks. In
European conference oncomputer vision , pp. 818–833. Springer, 2014.Zhang, M., Cui, Z., Neumann, M., and Chen, Y. An end-to-end deep learning architecture for graph classification. In
AAAI , volume 18, pp. 4438–4445, 2018.Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba,A. Learning deep features for discriminative localization.In
Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition , pp. 2921–2929, 2016. n Explainability of Graph Neural Networks via Subgraph Explorations:Appendix
Table 2.
Statistics and properties of five datasets.
Dataset
MUTAG BBBP G
RAPH -SST2 BA-2M
OTIFS
BA-S
HAPE
A. Datasets and Experimental Settings
A.1. Datasets and GNN Models
We first report the statistics and properties of the datasets in Table 2. We employ different GNN variants to fit these datasetsand explain the trained GNNs. Note that these models are trained to obtain reasonable performance. Specifically, we reportthe architectures and performance of these GNNs as below:•
MUTAG (GCNs) : This GNN model consists of 3 GCN layers. The input feature dimension is 7 and the output dimensionsof different GCN layers are set to 128, 128, 128, respectively. We employ max-pooling as the readout function and ReLUas the activation function. The model is trained for 2000 epochs with a learning rate of 0.005 and the testing accuracy is0.92. We study the explanations for the whole dataset.•
MUTAG (GINs) : This GNN model consists of 3 GIN layers. For each GIN layer, the MLP for feature transformations isa two-layer MLP. The input feature dimension is 7 and the output dimensions of different GIN layers are set to 128, 128,128 respectively. We employ max-pooling as the readout function and ReLU as the activation function. The model istrained for 2000 epochs with a learning rate of 0.005 and the testing accuracy is 1.00. We study the explanations for thewhole dataset.•
BBBP (GCNs) : This GNN model consists of 3 GCN layers. The input feature dimension is 9 and the output dimensionsof different GCN layers are set to 128, 128, 128, respectively. We employ max-pooling as the readout function and ReLUas the activation function. The model is trained for 800 epochs with a learning rate of 0.005 and the testing accuracy is0.863. We randomly split this dataset into the training set (80%), validation set (10%), and testing set (10%). We study theexplanations for the testing set.•
Graph-SST2 (GATs) : This GNN model consists of 3 GAT layers. The input feature dimension is 768 and all GAT layershave 10 heads with 10-dimensional features. We employ max-pooling as the readout function and ReLU as the activationfunction. In addition, we set the dropout rate to 0.6 to avoid overfitting. The model is trained for 800 epochs with alearning rate of 0.005 and the testing accuracy is 0.881. We follow the training, validation, and testing splitting of theoriginal SST2 dataset. We study the explanations for the testing set.•
BA-2Motifs (GCNs) : This GNN model consists of 3 GCN layers. The input feature dimension is 10 and the outputdimensions of different GCN layers are set to 20, 20, 20, respectively. For each GCN layer, we employ L2 normalizationto normalize node features. We employ average pooling as the readout function and ReLU as the activation function. Themodel is trained for 800 epochs with a learning rate of 0.005 and the testing accuracy is 0.99. We randomly split thisdataset into the training set (80%), validation set (10%), and testing set (10%). We study the explanations for the testingset.•
BA-Shape (GCNs) : This GNN model consists of 3 GCN layers. The input feature dimension is 10 and the outputdimensions of different GCN layers are set to 20, 20, 20, respectively. For each GCN layer, we employ L2 normalization n Explainability of Graph Neural Networks via Subgraph Explorations to normalize node features. In addition, we use ReLU as the activation function. The model is trained for 800 epochswith a learning rate of 0.005 and the testing accuracy is 0.957. We randomly split this dataset into the training set (80%),validation set (10%), and testing set (10%). We study the explanations for the testing set.
A.2. Experimental Settings
We conduct our experiments using one Nvidia V100 GPU on an Intel Xeon Gold 6248 CPU. Our implementations arebased on Python 3.7.6, PyTorch 1.6.0, and Torch-geometric 1.6.3. For our proposed SubgraphX and other algorithms withMCTS, the MCTS iteration number M is set to 20. To explore a suitable trade-off between exploration and exploitation,we set the hyperparameter λ in Eq.(3) to 5 for Graph-SST2 (GATs) and BBBP (GCNs) models, and 10 for other models.Since all GNN models contain 3 network layers, we consider 3-hop computational graphs to compute Shapley values forour SubgraphX. For the Monte-Carlo sampling in our SubgraphX, we set the Monte-Carlo sampling steps T to 100 for alldatasets. For MCTS † , we set Monte-Carlo sampling steps to 1000 to obtain good approximations since it samples from allnodes in a graph. A.3. Evaluation Metrics
We further introduce the evaluation metrics in detail. First, given a graph G i , its prediction class y i , and its explanation, weobtain a hard explanation mask M i where each element is 0 or 1 to indicate whether the corresponding node is identified asimportant. For our SubgraphX and MCTS-based baselines, the masks can be directly determined by the obtained subgraphs.For GNNExplainer and PGExplainer, their explanations are edge masks and can be converted to explanation masks byselecting the nodes connected with these important edges. Then by occluding the important nodes in G i based on M i , wecan obtain a new graph ˆ G i . Finally, the Fidelity score can be computed as F idelity = 1 N N (cid:88) i =1 ( f ( G i ) y i − f ( ˆ G i ) y i ) , (10)where N is the total number of testing samples, f ( G i ) y i means the predicted probability of class y i for the original graph G i . Intuitively, Fidelity measures the averaged probability change for the predictions by removing important input features.Since simply removing nodes significantly affect the graph structures, we occlude these nodes with zero features to computethe Fidelity. In addition, we also employ Sparsity to measure the fraction of nodes are selected in the explanations. Then itcan be computed as Sparsity = 1 N N (cid:88) i =1 (1 − | M i || G i | ) , (11)where | M i | denotes the number of important nodes identified in M i and | G i | means the number of nodes in G i . Ideally,good explanations should select fewer nodes (high Sparsity) but lead to significant prediction drops (high Fidelity). B. Explanations for Graph Classification Models
In this section, we report more visualizations of explanations for graph classification models. The results are reported inFigure 7 and 8. In Figure 7, we show the explanations of real-world datasets BBBP and MUTAG. Obviously, our proposedmethod can provide more human-intelligible subgraphs as explanations while PGExplainer and GNNExplainer focus ondiscrete edges. In addition, we also report the results of sentiment dataset Graph-SST2 in Figure 8. The results show thatour SubgraphX can provide reasonable explanations to explain the predictions. For example, in the second row, the inputsentence is “none of this violates the letter of behan‘s book, but missing is its spirit, its ribald, full-throated humor”, whoselabel is negative and the prediction is correct. From the human’s view, “missing” should be the keyword for the semanticmeaning. Our SubgraphX shows that the “missing is its spirit” phrase is important, which successfully captures the keyword.The other methods capture the words and phrases such as “violates”, “none of this”, which are less related to the negativemeaning.
C. Explanations for Node Classification Models
In this section, we report more visualizations of explanations for node classification models. The results are reported inFigure 9 where we show the explanations of node classification dataset BA-Shape. Obviously, our SubgprahX focuses on n Explainability of Graph Neural Networks via Subgraph Explorations
Table 3.
The studies of different pruning strategies.
Method
Time FidelityL OW HIGH
IGH LOW
D. Study of Pruning Actions
Finally, we discuss the pruning actions in our MCTS. For the graph associated with each non-leaf tree search node, weperform node pruning to obtain its children subgraphs. Specifically, when a node is removed, all edges connected withit are also removed. In addition, if multiple disconnected subgraphs are obtained after removing a node, only the largestsubgraph is kept. Instead of exploring all possible node pruning actions, we explore two strategies: Low2high and High2low.First, Low2high arranges the nodes based on their node degrees from low to high and only considers the pruning actionscorresponding to the first k low degree nodes. Meanwhile, High2low arranges the nodes in order from high degree to lowdegree and only considers the first k high degree nodes for pruning. Intuitively, High2low is more efficient but may ignorethe optimal solutions. In this work, we employ the High2low strategy for BA-Shape(GCNs), and Low2high strategy forother models, and set the k to 12 for all the datasets.We conduct experiments to analyze these two pruning strategies for our SubgraphX algorithm and show the average timecost and Fidelity score in Table 3. Specifically, we randomly select 50 graphs from the BBBP datasets with an average nodenumber of 24.96, which is the same in Section 4.5. In addition, we set Monte-Carlo sampling steps T to 100, and select thesubgraphs with the highest Shapley values and contain less than 15 nodes to calculate the Fidelity. Obviously, High2low is 5times faster than Low2high but the Fidelity scores of its explanations are inferior. n Explainability of Graph Neural Networks via Subgraph Explorations SubgraphX MCTS_GNN
PGExplainer
GNNExplainer
Dataset BBBP
Model: GCNs
Label: penetration
Correct predictionDataset BBBP
Model: GCNs
Label: penetration
Correct predictionDataset BBBP
Model: GCNs
Label: penetration
Incorrect prediction
Dataset BBBPModel: GCNs
Label: penetration
Incorrect predictionDataset BBBPModel: GCNsLabel: penetration
Incorrect predictionDataset MUTAG
Model: GCNs
Label: mutagenic
Correct predictionDataset MUTAGModel: GCNs
Label: mutagenic
Incorrect predictionDataset MUTAG
Model: GCNs
Label: mutagenic
Correct predictionDataset MUTAG
Model: GINsLabel: mutagenicCorrect prediction
Figure 7.
Explanation results of the BBBP and MUTAG datasets. Here Carbon, Oxygen, Nitrogen, and Chlorine are shown in yellow, red,and blue, green respectively. n Explainability of Graph Neural Networks via Subgraph Explorations
SubgraphX MCTS_GNN PGExplainer GNNExplainerLabel: positive, correct prediction, input: “ reinforces the talents of screen writer charlie kaufman, creator of adaptation and being john malkovich. ” Label: negative, correct prediction, input: “ none of this violates the letter of behan`s book, but missing is its spirit, its ribald, full-throated humor. ” Label: positive, incorrect prediction, input: “ smart science fiction for grown-ups, with only a few false steps along the way. ” Label: positive, incorrect prediction, input: “ a whole lot foul, freaky and funny. ” Figure 8.
Explanation results of Grpah-SST2 dataset. n Explainability of Graph Neural Networks via Subgraph Explorations
SubgraphX MCTS_GNN PGExplainer GNNExplainerDataset: BA-ShapeModel: GCNs
Target: large blue nodeCorrect prediction
Dataset: BA-ShapeModel: GCNs
Target: large green nodeCorrect prediction
Dataset: BA-ShapeModel: GCNs
Target: large red nodeIncorrect prediction
Dataset: BA-Shape
Model: GCNsTarget: large green node
Correct prediction
Dataset: BA-ShapeModel: GCNs
Target: large red nodeIncorrect prediction