Explaining Deep Graph Networks with Molecular Counterfactuals
EExplaining Deep Graph Networks with MolecularCounterfactuals
Danilo Numeroso
University of Pisa [email protected]
Davide Bacciu
University of Pisa [email protected]
Abstract
We present a novel approach to tackle explainability of deep graph networks in thecontext of molecule property prediction tasks, named MEG (Molecular ExplanationGenerator). We generate informative counterfactual explanations for a specificprediction under the form of (valid) compounds with high structural similarity anddifferent predicted properties. We discuss preliminary results showing how themodel can convey non-ML experts with key insights into the learning model focusin the neighborhood of a molecule.
The prediction of functional and structural properties of molecules by machine learning models forgraphs is a research field with long-standing roots [1]. Much of current research on the topic relieson Deep Graph Networks (DGNs) [2, 3], as they provide a flexible and scalable means to learneffective vectorial representations of the molecules. This has resulted in a trail of works targetingincreasing levels of effectiveness, breadth and performance in the prediction of chemo-physicalproperties [4]. The scarce intelligibility of such models and of the internal representation theydevelop can, however, act as a show-stopper for their consolidation, e.g. to predict safety-criticalmolecule properties, especially when considering well known issues of opacity in DGN assessment[5]. In this respect, attention is building towards the development of interpretability techniquesspecifically tailored to DGNs. While some DGN shows potential for interpretability by-design thanksto its probabilistic formulation [6], the majority of works in literature take a neural-based approachwhich requires the use of an external model explainer. GNNExplainer [7] is the front-runner of themodel-agnostic methods providing local explanations to neural DGNs in terms of the sub-graphand node features of the input structure which maximally contribute to the prediction. RelEx [8]extends GNNExplainer to surpass the need of accessing the model gradient to learn explanations.GraphLIME [9] attempts to create locally interpretable models for node-level predictions, withapplication limited to single network data. This paper fits into this pioneering field of research bytaking a novel angle to the problem, targeting the generation of interpretable insights for the primaryuse of the experts of the molecular domain. We build our approach upon the assumption that a domainexpert would be interested in understanding the model prediction for a specific molecule based ondifferential case-based reasoning against counterfactuals, i.e. similar structures which the modelconsiders radically different with respect to the predicted property. Such counterfactual moleculesshould allow the expert to understand if the structure-to-function mapping learned by the model iscoherent with the consolidated domain knowledge, at least for what pertains a tight neighborhoodaround the molecule under study. We tackle the problem of counterfactual molecule generation byintroducing an explanatory agent based on reinforcement learning (RL) [10]. This explanatory agenthas access to the internal representation of the property-prediction model as well as to its outputand uses this information to guide the exploration of the molecular structure space to seek for thenearest counterfactuals. Our approach is specifically thought for molecular applications and the RL a r X i v : . [ q - b i o . Q M ] N ov gent leverages domain knowledge to constrain the generated explanations to be valid molecules.We test our explainer on DGNs tackling the prediction of toxicity (classification task) and solubility(regression task) of chemical compounds. Figure 1: DGN ϕ is a trained molecule property pre-dictor, whereas the Explainer g is a generative agentproducing counterfactuals, constrained by prior domainknowledge DK . The overall architecture of our explanationframework, named MEG, is depicted in Fig-ure 1. Here we denote with ϕ : I → Y aDGN that is fit to solve a molecular propertyprediction task. I represents the space of (la-belled) molecule structures and Y is the task-dependent output space. The Explainer is anRL agent implementing a generative function g : I → I targeting the generation of counter-factual explanations. Molecular counterfactualsought to satisfy three properties: (i) they needto resemble the molecule under study; (ii) pre-dicted properties on counterfactuals must differsubstantially from those predicted on the inputone; (iii) molecular counterfactuals need to be in compliance with chemical constraints. To this end,the agent g receives information about an input molecule m and its associated prediction score ϕ ( m ) ,and generates a molecular counterfactual m (cid:48) , leveraging prior domain knowledge to ensure validity ofthe generated sample. Counterfactual generation is formalised as a maximisation problem in which,given a target molecule m with prediction ϕ ( m ) , the generator g is trained to optimize: arg max θ L (cid:0) ϕ ( m ) , ( ϕ ◦ g )( · | θ ) (cid:1) + K (cid:2) m, g ( · | θ ) (cid:3) . (1)The composition ( ϕ ◦ g )( · | θ ) formalizes the model ϕ counter-predictions, made over the counter-factuals produced by g . Given the counterfactual m (cid:48) = g ( · | θ ) we rewrite Equation 1 as arg max m (cid:48) L (cid:0) ϕ ( m ) , ϕ ( m (cid:48) ) (cid:1) + K (cid:2) m, m (cid:48) (cid:3) (2)where L is a measure of prediction disagreement between the molecule m and its counterfactual m (cid:48) , while K measures ( m, m (cid:48) ) similarity. In our framework, m is used to bootstrap the generativeprocess in g which operates on the current candidate counterfactual with graph editing operationsunder domain knowledge constraints. Given the non-differentiable nature of the graph alterations, wemodel g through a multi-objective RL problem [11], that takes the form of an MDP( S , A , Q , π , R , γ ). Apart from well known differentiability issues of graph operations, the generator g is modeledas an RL agent for its ease in modelling and handling multi-objective optimization. This allows toeasily steer towards the generation of counterfactuals optimizing several properties at a time [12, 13].Since we are interested in generating counterfactuals that are compliant to chemical knowledge, theaction space A is restricted so as to only retain actions that preserve the chemical validity of themolecule. To this end, we base the implementation of our agent on the MolDQN [14] model, thatis an RL-based approach to molecule graph generation leveraging double Q-learning [15]. At eachstep, the reward function R exploits the prediction from ϕ so as to notify the agent of its currentperformance, emitting a scalar reward. In our design, R binds together a term regulating the changein prediction scores, which is inherently task-dependent, with a second term controlling similaritybetween the original molecule and its counterfactual, as presented in Equation 1. Currently, we haveexplored two formulations for the latter term. The former leverages the Tanimoto similarity overthe Morgan fingerprints [16]. The latter is a ϕ -model dependent metric exploiting the encoding ofmolecules in the DGN internal representation. An advantage of using the latter approach is that ittakes into account the model’s own perception of structural similarity between molecules.The leftmost term L in Equation 2 can be specialized for classification and regression tasks. Asregards classifications, given a set of classes C , a model ϕ emits a probability distribution ϕ ( · ) = y = [ y , ..., y |C| ] over the predicted classes. In this case, given an input-prediction pair (cid:104) m, c =arg max c ∈C ϕ ( m ) (cid:105) , the generator is trained to produce counterfactual explanations m (cid:48) minimisingthe prediction score for class c , as follows arg max m (cid:48) − αy c + (1 − α ) K [ m (cid:48) , m ] (3)2 a) A0 (b) A1 (c) A2 (d) A3Figure 2: Experimental results for the Tox21 sample, reported in Table 1. where α ∈ [0 , is a hyper-parameter weighing the two parts. Hence, the model ϕ returns at eachstep a smooth reward, which is actually the inverse of the probability of m (cid:48) belonging to class c .Differently, for a regression task, the objective function can be defined as arg max m (cid:48) α sgn (cid:0) (cid:107) s m (cid:48) − s (cid:107) − (cid:107) s m − s (cid:107) (cid:1) (cid:107) s m (cid:48) − s m (cid:107) + (1 − α ) K [ m (cid:48) , m ] (4)where sgn is the sign function, s is the regression target, and s m and s m (cid:48) are the predicted valuesfor the original molecule and its counterfactual, respectively. The sign function is needed to preventthe agent from generating molecules whose predicted scores move towards the original target, byproviding negative rewards.The main use of counterfactual explanations is to provide insights into the function learned by themodel ϕ . In this sense, a set of counterfactuals for a molecule may be used to: (i) identify changes tothe molecular structure leading to substantial changes in the properties, enabling domain experts todiscriminate whether the model predictions are well founded; (ii) validate existing interpretability ap-proaches, by running them on both the original input graph and its related counterfactual explanations.The main idea behind this latter point is that a local interpretation method may provide explanationsthat work well within a very narrow range of the input, but do not give a strong suggestion on awider behaviour. To show evidence and usefulness of such a differential analysis, in the followingsection we use our counterfactuals to assess the quality of explanations given by GNNExplainer[7]. Given the undirected nature of the graphs in our molecular application, we restrict the originalGNNExplainer model to discard the effect of edge orientation on the explanation. (a) B0 (b) B1(c) B2 (d) B3Figure 3: ESOL sample alongside its counterfactuals(B1-3). Quantitative results are reported in Table 1. We discuss a preliminary assessment of our ex-planations on two popular molecular propertyprediction benchmarks: Tox21 [17], addressingtoxicity prediction as a binary classification task,and ESOL [18], that is a regressive task on watersolubility of chemical compounds. Preliminar-ily, we scanned both datasets to filter non-validchemical compounds. We considered structuresto be valid molecules if they pass the RDKit [19]sanitization check. In the end, Tox21 comprises1900 samples, equally distributed among the twoclasses, while ESOL includes 1129 compounds.The trained DGN comprises three GraphConv[20] layers with ReLu activations, whose hid-den size is per layer for Tox21, and forESOL. The network builds a layer-wise molec-ular representation via concatenation of max and mean pooling operations, over the set of noderepresentations. The final neural encoding of the molecule is obtained by sum-pooling of the inter-mediate representations. This neural encoding is then feed to a three-layer feed-forward network,with hidden sizes of [128, 64, 32], to perform the final property prediction step. The trained DGNsachieved 87% of accuracy and 0.52 MSE over the Tox21 and ESOL test sets, respectively. All experi-ments have been performed by using the Adam optimiser with a learning rate of · − . Duringgeneration, we employed MEG to find the best counterfactual explanations for each molecule in3est, ranked according to the multi-objective score in Section 2. Ideally, we would like to observecounterfactual molecules that are structurally similar to the original compound while leading to asubstantially different prediction. Due to the stringent page constraints, in the following we reporttwo example explanation cases (one for each dataset). Further examples and results are available inthe appendix. Molecule Target Prediction Similarity RewardA0: Figure 2
NoTox NoTox (0.70) - -A1: Figure 2 -
Tox (0.90) 0.76 0.80A2: Figure 2 -
Tox (0.83) 0.79 0.72A3: Figure 2 -
Tox (0.80) 0.68 0.66B0: Figure 3 -4.28 -4.01 - -B1: Figure 3 - -6.11 0.29 1.14B2: Figure 3 - -5.93 0.31 1.11B3: Figure 3 - -5.07 0.28 0.66Table 1: Summary of preliminary results. A0 and B0 refers tomolecules belonging to Tox21 and ESOL, respectively. Subsequentindexes refers to the related counterfactuals explanations.
We present some quantitative resultin Table 1, listing the best three coun-terfactual explanations collected, forboth tasks. We tested two similar-ity metrics: cosine similarity over theneural encodings, for Tox21, and theTanimoto, for ESOL. Qualitative re-sults are shown in Figure 2 and Fig-ure 3. To ease the interpretation of ourresults, counterfactual modificationshave been highlighted in red, whileblurred edges represent those edgesthat have been masked out by GNNEx-plainer predictions. In other words,GNNExplainer interpretations are the sub-graphs formed by non-blurred edges. As for the Tox21sample, we evaluate MEG against a test molecule (i.e, A0) that has been correctly classified bythe DGN as being non-toxic, outputting the counterfactuals A1-3 (i.e. molecules which the modelconsiders toxic). We can see that the addition of a carbon atom may alter the DGN prediction,as shown by A1 and A2. In fact, while A0 is classified correctly with 70% certainty, A1-2 arepredicted as toxic, with certainty of 90% and 83%, respectively. Differently, A3 breaks the left sidering and achieves the lowest neural encoding similarity score among the three, giving clues aboutpotential substructure-awareness. Furthermore, in Figure 2 we show how counterfactuals may helpto detect inconsistencies in GNNExplainer predictions. In fact, although GNNExplainer identifiesthe substructure CC(N)O as explanation for the original sample A0, MEG counterfactuals prioritizechanges to different molecule fragments. These inconsistencies suggest that the GNNExplainerinterpretation is too much targeted to the input molecule (A0) and does not generalize even for minormodifications of the input graph.We now turn our attention to ESOL results (B0-3) shown in Table 1. B0 is an organic compoundnamed pentachlorophenol, commonly used as a pesticide or a disinfectant, and is characterized bynearly absolute insolubility in water. While the DGN achieved good predictive performance forits aqueous solubility value, the counterfactuals underlined that the ϕ -model predicted solubilitydecreases in case the oxygen atom is removed (e.g, B2), or modified somehow (e.g, B1, B3),highlighting how it is highly relevant for the DGN prediction. As in the Tox21 sample, such relationis not adequately captured by GNNExplainer explanation for B0. It is our hope that, based on ourinterpretability approach, an expert of the molecular domain could be able to gain a better insightinto the whether the properties and patterns captured by the predictive model are meaningful from achemical standpoint. We have presented MEG, a novel interpretability framework that tackles explainability in the chemicaldomain by generation of molecular counterfactual explanations. MEG can work with any DGNmodel as we only exploit input-output properties of such models. As a general comment of thepreliminary results, one can note that while a local approach such as GNNExplainer may give goodapproximations when it comes to explaining the specific prediction, it lacks sufficient breadth tocharacterize the model behaviour already in a near vicinity of the sample under consideration. Onthe other hand, our counterfactual interpretation approach may find new samples which are likelyto highlight the causes of a given model prediction, providing a better approximation to a locallyinterpretable model, e.g. B1-3 in Figure 2. In conclusion, apart for its value in generating explanationsthat are well understood by a domain expert, MEG proposes itself both as a sanity checker for otherlocal model explainers, as well as a sampling method to strengthen the coverage and validity of localinterpretable explanations, such as in the original LIME method for vectorial data [21].4 eferences [1] Alessio Micheli, Alessandro Sperduti, and Antonina Starita. An introduction to recursive neuralnetworks and kernel methods for cheminformatics.
Current Pharmaceutical Design , 13(8),2007.[2] Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, ChangchengLi, and Maosong Sun. Graph neural networks: A review of methods and applications, 2018.[3] Davide Bacciu, Federico Errica, Alessio Micheli, and Marco Podda. A gentle introduction todeep learning for graphs.
Neural Networks , 129:203–221, 2020.[4] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl.Neural message passing for quantum chemistry. volume 70 of
Proceedings of Machine LearningResearch , pages 1263–1272, International Convention Centre, Sydney, Australia, 06–11 Aug2017. PMLR.[5] Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli. A fair comparison ofgraph neural networks for graph classification. In
International Conference on LearningRepresentations , 2020.[6] Davide Bacciu, Federico Errica, and Alessio Micheli. Probabilistic learning on graphs viacontextual architectures.
Journal of Machine Learning Research , 21(134):1–39, 2020.[7] Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer:Generating explanations for graph neural networks, 2019.[8] Yue Zhang, David Defazio, and Arti Ramesh. Relex: A model-agnostic relational modelexplainer, 2020.[9] Qiang Huang, Makoto Yamada, Yuan Tian, Dinesh Singh, Dawei Yin, and Yi Chang. Graphlime:Local interpretable model explanations for graph neural networks, 2020.[10] Richard S Sutton and Andrew G Barto. Reinforcement learning: an introduction cambridge.
MA: MIT Press.[Google Scholar] , 1998.[11] C. Liu, X. Xu, and D. Hu. Multiobjective reinforcement learning: A comprehensive overview.
IEEE Transactions on Systems, Man, and Cybernetics: Systems , 45(3):385–398, 2015.[12] Benjamin Sanchez-Lengeling, Carlos Outeiral, Gabriel L. Guimaraes, and Alan Aspuru-Guzik.Optimizing distributions over molecular space. an objective-reinforced generative adversarialnetwork for inverse-design chemistry (organic), Aug 2017.[13] Mariya Popova, Olexandr Isayev, and Alexander Tropsha. Deep reinforcement learning for denovo drug design.
Science Advances , 4(7):eaap7885, Jul 2018.[14] Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N. Zare, and Patrick Riley. Optimization ofmolecules via deep reinforcement learning, 2018.[15] Hado van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with doubleq-learning, 2015.[16] David Rogers and Mathew Hahn. Extended-connectivity fingerprints.
Journal of ChemicalInformation and Modeling , 50(5):742–754, 2010. PMID: 20426451.[17] Kristian Kersting, Nils M. Kriege, Christopher Morris, Petra Mutzel, and Marion Neumann.Benchmark data sets for graph kernels, 2016.[18] Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S.Pappu, Karl Leswing, and Vijay Pande. Moleculenet: A benchmark for molecular machinelearning, 2017.[19] Greg Landrum et al. Rdkit: Open-source cheminformatics, 2006.[20] Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen,Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neuralnetworks, 2018.[21] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you?": Explainingthe predictions of any classifier. In
Proceedings of the 22nd ACM SIGKDD InternationalConference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17,2016 , pages 1135–1144, 2016. 5
Additional Results
Molecule Target Prediction Similarity RewardC0: Figure 5 -4.755 -4.5195 - 1.57C1: Figure 5 - -2.6488 0.39 1.33C2: Figure 5 - -3.0170 0.65 1.12D0: Figure 6
NoTox NoTox (0.71) - -D1: Figure 6 -
Tox (0.86) 0.90 0.78D2: Figure 6 -
Tox (0.80) 0.91 0.73E0: Figure 4
Tox Tox (0.78) - -E1: Figure 4 -
NoTox (0.94) 0.69 0.86E2: Figure 4 -
NoTox (0.84) 0.89 0.73Table 2: Summary of other preliminary results.
Table 1 provides experimental re-sults for three compounds, one ofwhich belongs to ESOL (C0-2) andtwo to Tox21 (D0-2, E0-2). Vi-sual feedback is shown in Figure 4-5-6. As before, sharpness of graphedges indicates GNNExplainer expla-nations, while counterfactual modifi-cations have been colored in red.We seek for counterfactuals for anESOL test compound, whose pre-dicted solubility is close to the actual target. In this case, the atom of sulphur seems to have anegative impact on the predicted aqueous solubility. In this regard, C1 increases the compundsolubility by removing, indeed, the atom of sulphur. In nature, a molecule of sulphur (i.e, S8 inSMILES encoding) is known to be insoluble. Such an analysis can drop preliminary hints abouthow the trained model may have learned such characteristics. Similarly to C1, C2 added an atom ofoxygen causing the predicted water solubility to increase. (a) E0(b) E1 (c) E2Figure 4: E1 modifies the cyclohexane ring, which was not consid-ered important in the explanation provided by GNNExplainer forthe original molecule E0. E2 breaks the bond highlighted in red.
Another significant examples com-prises D0-2. In fact, D0 is correctlyclassified as a non-toxic compound.However, a simple addition of nitro-gen makes the prediction change com-pletely, resulting in classifying D1 andD2 as toxic with certainty of 86% and80% respectively. Furthermore, san-ity checks on GNNExplainer expla-nation for D0 emphasize that D2 up-dates a blurred explanation fragment(i.e, the atom of carbon attached tothe atom of nitrogen nor its incidentbonds have been considered impor-tant in D0). More interestingly, E0-2present a potentially dangerous situa-tion. In detail, starting from a toxiccompound (E0), E1 achieves to be recognized as non-toxic by simply adding an atom of carbon,and so does E2 by breaking one of the rings, as shown in Figure 4. In this case, the usefulness ofour counterfactuals can be exploited to the fullest, highlighting such difficulties of the model underconsideration which is crucial in real-world applications. (b) C0(c) C1 (d) C2Figure 5: ESOL. C1 removes the atom of sulphur.C2, instead, adds a new atom of oxygen and con-nect it to the molecule through a double bond. (b) D0(c) D1 (d) D2Figure 6: Tox21. The agent adds atoms of nitrogento the rightmost ring.(b) C0(c) C1 (d) C2Figure 5: ESOL. C1 removes the atom of sulphur.C2, instead, adds a new atom of oxygen and con-nect it to the molecule through a double bond. (b) D0(c) D1 (d) D2Figure 6: Tox21. The agent adds atoms of nitrogento the rightmost ring.