xAI-GAN: Enhancing Generative Adversarial Networks via Explainable AI Systems
xxAI-GAN: Enhancing Generative AdversarialNetworks via Explainable AI Systems
Vineel Nagisetty , Laura Graves , Joseph Scott and Vijay Ganesh University of Waterloo { vineel.nagisetty, laura.graves, joseph.scott and vijay.ganesh } @uwaterloo.ca Abstract
Generative Adversarial Networks (GANs) are a revolution-ary class of Deep Neural Networks (DNNs) that have beensuccessfully used to generate realistic images, music, text,and other data. However, GAN training presents many chal-lenges, notably it can be very resource-intensive. A potentialweakness in GANs is that it requires a lot of data for success-ful training and data collection can be an expensive process.Typically, the corrective feedback from discriminator DNNsto generator DNNs (namely, the discriminator’s assessmentof the generated example) is calculated using only one real-numbered value (loss). By contrast, we propose a new classof GAN we refer to as xAI-GAN that leverages recent ad-vances in explainable AI (xAI) systems to provide a “richer”form of corrective feedback from discriminators to genera-tors. Specifically, we modify the gradient descent process us-ing xAI systems that specify the reason as to why the dis-criminator made the classification it did, thus providing the“richer” corrective feedback that helps the generator to betterfool the discriminator. Using our approach, we observe xAI-GANs provide an improvement of up to 23.18% in the qualityof generated images on both MNIST and FMNIST datasetsover standard GANs as measured by Fr´echet Inception Dis-tance (FID). We further compare xAI-GAN trained on 20%of the data with standard GAN trained on 100% of data onthe CIFAR10 dataset and find that xAI-GAN still shows animprovement in FID score. Further, we compare our workwith Differentiable Augmentation - which has been shown tomake GANs data-efficient - and show that xAI-GANs outper-form GANs trained on Differentiable Augmentation. More-over, both techniques can be combined to produce even betterresults. Finally, we argue that xAI-GAN enables users greatercontrol over how models learn than standard GANs.
Generative Adversarial Networks (GANs), introduced onlya few short years ago, already have had a revolutionary im-pact on generating data of varied kinds such as images, text,music, and videos (Goodfellow et al. 2014). The critical in-sight behind a GAN is the idea of corrective feedback loopfrom a deep neural network (DNN) called the discriminator back to a generator . However, a notable weakness of GANsis that they require a lot of data for successful training. For
Figure 1: System Architecture of xAI-GAN.example, in order to learn to write digits, a human may onlyneed a few ( ≤
10) examples before she or he learns to repli-cate them whereas a GAN would often need several ordersof magnitude more data ( ≥ To answer the above-mentioned research questions, we pro-pose a new class of GANs we refer to as xAI-GAN whereinit is possible to provide “richer” corrective feedback (morethan a single value) during training from discriminator togenerators via Explainable AI (xAI) systems. A high-level a r X i v : . [ c s . L G ] O c t ystem architectural overview of our xAI-GAN system isgiven in Figure 1. Consider the problem of training a GANwith the aim of producing images of digits. Initially, the un-trained generator G is given a noise sample z from a priornoise distribution p z and produces an example G ( z ) that isthen given to discriminator D . The loss is calculated, andthen the generated image G ( z ) , the discriminator D , andthe output of the discriminator D ( G ( z )) are fed into an xAIsystem that produces an explanation as to why the image re-sulted in that loss. This explanation is then used to guide thetraining of the generator (refer Section 4 for details).A common analogy for GAN training is that of a coun-terfeiter (the generator) and a detective (the discriminator)playing an adversarial game where the counterfeiter makesa fake and the detective tries to tell if it’s real or not. Over thetraining process, the detective and counterfeiter both get bet-ter at their jobs, with the end goal being that the counterfeiteris so proficient that their fakes can pass for the real thing.To extend this analogy to the xAI-GAN setting, our methodworks by using an expert in the field (the xAI system) tohelp improve the counterfeiter. When the detective recog-nizes a fake, the expert tells the counterfeiter what parts ofthe fake tipped off the detective. The counterfeiter is thusable to learn better why the detective detected a fake, andmake better decisions to avoid pitfalls in future training. Explanation Matrix:
The explanation produced by xAIsystems are converted to the form of an “explanation matrix” M , wherein, for every feature in an example (e.g., an inputimage), a value in the range [0,1] is assigned to the corre-sponding cell of the matrix M . If a feature (more precisely,the corresponding cell in M ) is assigned value 0 (or closeto 0), then it means that pixel had no impact on the classi-fication decision made by the discriminator D . If a featureis assigned a value of 1 (or close to 1), then that means thatfeature is very important. This could be due to the feature be-ing “determinative” in the classification made by D or whenit “hurts” the classification made by D (more precisely, ifthe feature were to be changed, then the confidence of D inits classification would improve). Creating the explanationmatrix in this manner helps us focus the learning processon the most influential features, regardless of whether thosefeatures were beneficial or harmful to the classification. xAI-guided Gradient Descent: The matrix M generated bythe xAI system is then used in a modified gradient descentalgorithm (see Algorithm 1) to update the weights of thegenerator as follows: traditionally, in a GAN the weights ofthe generator are modified by first computing the gradient ofgenerator’s output with respect to the loss and then applyingthe chain rule. We modify this algorithm by first comput-ing the explanation matrix M (via the xAI system) and thencalculating the product (specifically, an element wise or aHadamard product (Horn 1990)) between M and the gradi-ent of the generator output with respect to the loss ∆ G ( z ) .More precisely, the explanation matrix M is used to maskthe gradient function, and consequently the “importance ofthe pixels” that went into the discriminator’s classificationare taken into account in the modification of the generator’sweights during the application of the gradient descent.Using our approach, one can foresee that users might be able to augment such explanation matrices with specifica-tions that spell out relationships (using logical formulas) be-tween the update methods for the various weights of a gener-ator. We would like to emphasize that the standard gradientdescent method simply moves toward the greatest decreasein loss over an n -dimensional space, while by contrast, xAI-guided gradient descent algorithms can give users greatercontrol over the learning process. xAI-guided Gradient Descent Algorithm and xAI-GAN: Our key contribution is an xAI-guided gradient de-scent method (and a resultant GAN we refer to as xAI-GAN) that utilizes xAI systems to focus the gradient de-scent algorithm on weights that are determined to be mostinfluential by the xAI system (refer Section 4.2). We im-plement several different versions of xAI-GAN using 3different state-of-the-art xAI systems (refer Section 4.2),namely saliency map (Simonyan, Vedaldi, and Zisserman2013), shap (Lundberg and Lee 2017a), and lime (Ribeiro,Singh, and Guestrin 2016). In Section 6.1, we discusshow xAI-guided gradient descent methods can give thosetraining models greater control over the learning process.2.
Experimental Evaluation of Quality of Images pro-duced by xAI-GAN vs. Standard GAN:
We performedexperiments to evaluate the quality (as measured byFr´echet Inception Distance, abbreviated as FID (Heuselet al. 2017)) of xAI-GANs relative to standard GANs. Weshow that on MNIST and Fashion MNIST datasets, xAI-GANs achieve an improvement of up to 23.18% in FIDscore compared to standard GANs (refer Section 5.3).3.
Experimental Evaluation of Data Efficiency of xAI-GANs vs. Standard GANs:
We extend our experimentto the CIFAR 10 dataset, using only 20% of the datafor xAI-GAN while letting standard GAN use 100% ofthe data. We show that xAI-GAN outperforms standardGAN in FID score even in this setting. We further com-pare our work with Differentiable Augmentation (Zhaoet al. 2020) technique which has been shown to improvedata-efficiency of GANs. We show that xAI-GAN outper-forms Differentiable Augmentation, resulting in a betterFID score. Finally, we modify our xAI-GAN to incorpo-rate Differentiable Augmentation and show that the re-sulting model has better performance than either version.
Goodfellow et al. were the first to introduce GANs in (2014).Since then, GANs have continued to be a popular researchtopic with many versions of GANs developed (Pan et al.2019). GANs can be broadly classified based on their archi-tecture (Radford, Metz, and Chintala 2015; Mirza and Osin-dero 2014; Chen et al. 2016; Makhzani et al. 2015) and thetype of objective function used (Metz et al. 2016; Arjovsky,Chintala, and Bottou 2017; Mao et al. 2017; Dziugaite, Roy,and Ghahramani 2015; Zhao, Mathieu, and LeCun 2016;Gulrajani et al. 2017). To the best of our knowledge, thereis no GAN that uses xAI feedback for training, thus makingxAI-GAN the first of its kind. We note that the xAI-guidedradient descent algorithm is independent of architecture ortype of objective function used, and therefore can be appliedto make any type of GAN an xAI-GAN.Differentiable Augmentation (Zhao et al. 2020) is a re-cent technique that aims to make GANs more data-efficientby augmenting the training data. The main idea behindthis technique is to increase the data via various types ofaugmentations on both real and fake images during GANtraining. These augmentations are differentiable and so thefeedback from the discriminator can be propagated backto the generator through the augmentation. On the otherhand, while xAI-GAN also aims to make GANs more data-efficient, this is done by passing “richer” information fromthe discriminator to the generator through an xAI system.We compare xAI-GANs with Differentiable Augmentationin section 5.4 and show that they can be combined to pro-vide further improvements in data-efficiency.ADAGRAD (Duchi, Hazan, and Singer 2011) is an op-timization algorithm that maintains separate learning-ratesfor each parameter of a DNN based on how frequently theparameter is updated. On the other hand, xAI-GAN usesxAI feedback to determine how generator parameters are up-dated (refer Section 4.2).
Explainable AI (xAI) Systems:
As AI models becomemore complex, there is an increasing demand for inter-pretability or explainability of these models from decisionmakers, stakeholders, and lay users. In addition, one canmake a strong case for a scientific need for explainable AI.Consequently, there has been considerable interest in xAIsystems aimed at creating interpretable AI models that en-able human understanding of AI systems (Biran and Cotton2017).One way to define xAI systems is as follows: they are al-gorithms that, given a model and a prediction, assigns valuesto each feature of an input that measures how important thatfeature is to the prediction. There have been several differentxAI systems applied to DNNs with the goal of improvingour understanding of how these systems learn. These sys-tems can do so in a variety of ways, and approaches havebeen developed such as ones using formal logic (Ignatiev,Narodytska, and Marques-Silva 2019), game-theoretic ap-proaches (Lundberg and Lee 2017b), or gradient descentmeasures (Shrikumar, Greenside, and Kundaje 2017). Thesesystems output explanations in a variety of forms such asranked lists of features, select subsets of the feature sets, andvalues weighting the importance of features of input dataused to train machine learning models.
We implement and compare several xAI systems. Note thatour xAI-GAN is xAI system agnostic and can be used withany xAI system. However, the efficacy of the training ofxAI-GAN depends on the efficacy of the xAI system andhence selecting the appropriate xAI system is crucial.
Inspired by the processes by which animals focus attention,saliency maps (Simonyan, Vedaldi, and Zisserman 2013) compute the importance of each feature in a given inputto the resulting classification by a DNN model. In order tocompute a saliency map, a DNN model M , image x and tar-get label y are required. The loss of the prediction M ( x ) iscomputed with respect to y and used to perform backprop-agation to the calculate the gradient ∇ x . This is then nor-malized to produce the saliency map. While there are simi-larities between saliency maps and the process by which thegradients passed by the discriminator to the generator is cal-culated, there are some differences. Notably, in the case ofcolor images (such as CIFAR10 dataset), saliency map com-putes the maximum magnitude of ∇ x for each pixel acrossall color channels, while the gradients are computed for eachcolor channel separately. Lime (Ribeiro, Singh, and Guestrin 2016), short for LocalInterpretable Model-Agnostic, is used to explain the predic-tions of a ML classifier by learning an interpretable localmodel. Given a DNN model M and input x , Lime createsa set of new N inputs x , ..., x N by slightly perturbing x .It then queries M on these new inputs to generate labels y , ..., y N . The new inputs and labels are used to train asimple regression model which is expected to approximate M well in the local vicinity of x . The weights of this localmodel are used to determine the feature importance of x . DeepSHAP (Lundberg and Lee 2017b) is a combination ofthe DeepLIFT platform and
Shapley value explanations . In-troduced in 2017, the platform is well-suited for neural net-work applications and is freely available. DeepSHAP is anefficient Shapley value estimation algorithm. It uses linearcomposition rules and backpropagation to calculate a com-positional approximation of feature importance values.
Shapley Value Estimation:
Classic Shapley regression val-ues are intended for linear models, where the values repre-sent feature importance. Values are calculated by retrainingmodels on every subset of features S ⊆ F and valuing eachfeature based on the prediction values on models with thatfeature and without. Unfortunately, this method not only re-quires significant retraining but also requires at least | F | separate models to cover all combinations of included fea-tures. Methods to approximate the Shapley values by iter-ating only over local feature regions, approximating impor-tance using samples from the training dataset, and other ap-proaches have been proposed to reduce computational effort. The DeepLIFT xAI system:
DeepLIFT uses a set of ref-erence inputs and the consequent model outputs to identifythe importance of features (Shrikumar, Greenside, and Kun-daje 2017). The difference between an output and a refer-ence output, denoted by ∆ y , is explained in terms of thedifferences between the corresponding input and a referenceinput, given by ∆ x i . The reference input is chosen by theuser based on domain knowledge to represent a typical un-informed state. It is often a set of images from the originaldataset that the model is trained on. Each feature x i is givena value C ∆ x i ∆ y which measures the effect of the model out-put on that feature being the reference value instead of its lgorithm 1: Generator Training Algorithm. Notethat the code block under the if use xAI block onlyapplies to the xAI-guided generator training. input : generator G input : discriminator D input : boolean Flag use xAI output: trained generator G foreach noise sample z do Loss L =
Loss (1 − D ( G ( z )) computeDiscriminator Gradient ∆ D from L computeGenerated Example Gradient ∆ G ( z ) from ∆ D if use xAI is True then compute Explanation Matrix M using xAIcompute Modified Gradient ∆ (cid:48) G ( z ) = ∆ G ( z ) + α ∗ ∆ G ( z ) ∗ M computeGenerator Gradient ∆ G from ∆ (cid:48) G ( z ) else compute Generator Gradient ∆ G from ∆ G ( z ) end update Generator parameters θ G using ∆ G original value. The system uses a summation property wherethe sum of each feature’s changes sum up to the change inthe model output ∆ o of the original in comparison to thereference model: (cid:80) ni =1 C ∆ x i ∆ y = ∆ o . In this section, we provide a detailed overview of our xAI-GAN system (please refer to Figure 1 for the system archi-tecture of xAI-GAN and Algorithm 1 for the gradient de-scent algorithm for generator training) and contrast it withstandard GAN architectures as well as the way they aretrained. The intuition behind the xAI-guided generator train-ing process is that the xAI system acts as a guide, shapingthe gradient descent in a way that focuses generator trainingon those input features that the discriminator recognizes asmost important.
Briefly, standard GAN architectures consist of a system ofpaired DNNs, namely, a discriminator D and a generator G . The standard training method involves alternate cycles ofdiscriminator and generator training. Initially, the discrimi-nator is trained on a mini batch of examples drawn from bothtraining data from the target distribution, as well as data gen-erated by the untrained generator (which initially is expectedto be just noise). These examples are correctly labeled as“real” (if they were from the training set) or “generated” (ifthey are from the generator).Subsequently, the generator is trained as follows (pleaserefer to Algorithm 1): a selection of noise samples are drawnfrom the noise prior and passed through the generator to geta batch of generated examples (line 1). This batch is labeledas “real” and given to the discriminator, where the loss isfound (line 2), and then used to update the generator pa-rameters (the corrective feedback step). More precisely, the Generator Discriminator
Dense (100, 256) Dense (1024, 1296)Dense (256, 512) Dense (1296, 512)Dense (512, 1296) Dense (512, 256)Dense (1296, (32, 32)) Dense (256, 1)Table 1: Model Architecture of Fully Connected GAN usedin the context of MNIST and Fashion MNIST experiments.
Generator Discriminator
Conv (4 x 4) Conv (16 x 16)Conv (8 x 8) Conv (8 x 8)Conv (16 x 16) Conv (4 x 4)Conv (32 x 32) Conv (1 x 1)Table 2: Model Architecture of DC-GAN which we used inthe context of the CIFAR10 experiments.discriminator’s gradient ∆ D is computed using the parame-ters of the discriminator and its loss (line 3), which is usedto find the gradient of the generated example ∆ G ( z ) (line 4).Further, the gradients of all layers in the generator ∆ G arethen computed using ∆ G ( z ) (line 10). Finally, the parame-ters Θ G of the generator are updated using ∆ G (line 12) -completing one training iteration.In subsequent iterations, the discriminator receives minibatches of real and generated examples from the generatortrained in the previous iterations. The ideal termination con-dition for this process is when both the generated examplesare high-quality and the discriminator is unable to distin-guish between “real” and “generated” examples. We start our description of xAI-guided training by first ob-serving that in the standard GAN setting the discrimina-tor calculates the corrective feedback to the generator us-ing only a single value (loss) per generated image. The en-tire point of xAI-guided training is to augment this feedbackwith the “reason” for the discriminator’s decision, as deter-mined by the xAI system.During our xAI-guided gradient descent generator train-ing process, the backpropagation algorithm is modified tofocus generator training on the most meaningful features forthe discriminator’s prediction (please refer to lines 5-8 of Al-gorithm 1). Following with propagating the loss through thediscriminator to find ∆ G ( z ) , we use an xAI system E to find M = E ( G ( z )) (line 6). M is a set of real values ∈ [0 , ,where greater values represent features that are more impor-tant to the discriminator’s prediction. The Hadamard (ele-ment wise) product of ∆ G ( z ) and M is calculated to get themodified gradient ∆ (cid:48) G ( z ) (line 7). In an intuitive sense, theexplanation M acts as a mask for ∆ G ( z ) , focusing the gradi-ent on the most important features and limiting the gradienton the less important ones. From there, the gradients of thegenerator ∆ G are calculated from ∆ (cid:48) G ( z ) using a small valuefor α (line 8) and the parameters are then updated (line 12).igure 2: FID scores on MNIST DatasetFigure 3: Sample of Images Generated by the xAI-GAN lime System using 35% Data on the MNIST Dataset
We implemented xAI-GAN using Pytorch 1.6 (Paszke et al.2019), an open source machine learning framework popu-lar in deep learning research. For saliency and shap xAIsystems we used Captum 0.2.0 (Kokhlikyan et al. 2019),an open source interpretability framework developed by theteam at Pytorch. For the lime-based xAI-GAN system weuse the implementation from Lime 0.2.0.1 (Ribeiro, Singh,and Guestrin 2016). We process the explanation matrix Mgenerated by each of the xAI systems by taking the absolutevalue and normalizing the matrix to create a mask vectorwith values in range [0 , .Pytorch notably has the autograd (Paszke et al. 2017)package which handles automatic differentiation of all ten-sors. In order to provide xAI-guided feedback to the genera-tor, we overrode the register backward hook func-tion normally used to inspect gradients. We modified thegradients of the output layer of the generator using the re-sultant vector computed by the Hadamard (element wise)product with the computed mask. This modified gradient isback-propagated through the generator via the autograd .After extensive testing, we found that switching on the xAI-guided gradient descent after half the number of trainingepochs gives the best results. This is because the discrimi-nator would have learnt the distribution of the task at handto a certain extent and consequently the xAI system is likelyto produce better explanations. We performed extensive experimental evaluation of our xAI-GAN implementation that used 3 different xAI systems, The code of our implementation can be found at:https://github.com/explainable-gan/XAIGAN
Figure 4: FID scores on Fashion MNIST DatasetFigure 5: Sample of Images Generated by the xAI-GAN shap
System using 35% Data on the Fashion MNIST Datasetcomparing against standard GAN. We performed these ex-periments on three different datasets:1. MNIST (LeCun and Cortes 2010) a collection of 70,00028x28 grayscale images of handwritten digits,2. Fashion MNIST (Xiao, Rasul, and Vollgraf 2017) a col-lection of 70,000 28x28 grayscale images of clothing, and3. CIFAR10 (Krizhevsky, Nair, and Hinton 2010) a collec-tion of 60,000 3x32x32 color images of objects.For the MNIST and Fashion MNIST datasets, we resizedthe images to 32x32 and used fully connected GANs forboth standard and xAI-GANc the architecture for which isshown in Table 1. Leaky relu was the activation used for allbut the last layers in the generator and discriminator. In thelast layer, we used tanh for the generator, and sigmoid for thediscriminator. A dropout rate of 0.3 was used during trainingin discriminator. For CIFAR10 dataset, we use a DC-GANarchitecture for both standard and xAI-GAN as described inTable 2. The generator and discriminator use four convolu-tional layers, each with a stride of 2 and padding of 1. Eachof them also use a batchnorm layer after every convolutionallayer, except for the last one. The activation functions areidentical to the fully-connected GAN architecture.
The batch size was selected to be 128 in our experiments.The Adam optimizer (Kingma and Ba 2014) was used forboth generator and discriminator training. We used a learn-ing rate of 0.0002. We ran experiments using Amazon’s EC2on a p2.xlarge instance which uses 1 Nvidia’s K80 GPU with64GiB RAM.
Based on a thorough literature survey of metrics (Pan et al.2019) for the image domain, we developed the followingataset Standard Shap Lime SaliencyMNIST 1221.19 9925.1 38865.93 2215.73FMNIST 991.11 9694.81 39237.33 2162.24Table 3: Average experiment time taken (in seconds) onMNIST and Fashion MNIST DatasetsDataset Standard Shap Lime SaliencyCIFAR10 536.76 13290.34 12988.32 1214.44+Diff 674.42 14319.87 13352.37 1175.99Table 4: Average experiment time taken (in seconds) by eachGAN on MNIST and Fashion MNIST (FMNIST) datasets.criteria in order to perform a fair comparison of xAI-GANsvs. standard GANs:1.
Fr´echet Inception Distance (FID):
We opted to useFr´echet Inception Distance (FID) to measure quality sinceit has been shown to be consistent with human evalua-tion of quality (Heusel et al. 2017). FID was introducedby Heusel et al., to address the shortcomings of InceptionScore (IS) such as the latter’s inability to detect intra-classmode dropping and vulnerability to noise (2017).At a high level, FID converts a set of images to the featurespace provided by a specific layer in the Inception model.Various statistics, such as the mean and covariance arecomputed on the activation values of that layer to gener-ate a multi-dimension Gaussian distribution. Finally, theFr´echet distance of the two distributions created using thegenerated and the training images is computed and pro-vided as the output. In order to apply FID to MNIST andFashion MNIST, we use the LeNet classifier, consistentwith (Bi´nkowski et al. 2018).2.
Training Time:
We also measure the time required fortraining to identify the overhead added by xAI systems.
We ran the experiments on both MNIST and FashionMNIST datasets using two settings: 100% data and 35% data(to see the performance of xAI-GAN when data is scarce).The results of the experiments on MNIST dataset can befound in Figure 2. For 100% data, standard GAN producedan FID score of 1.31. The xAI-GAN shap , xAI-GAN lime andxAI-GAN saliency systems resulted in scores of 1.36, 1.21and 1.26 respectively. The xAI-GAN lime system had thebest performance and resulted in an improvement of 7.35%in the FID score, as compared to standard GAN. For 35%data, standard GAN produced a score of 1.75 while thexAI-GAN shap , xAI-GAN lime and xAI-GAN saliency sys-tems produced scores of 1.50, 1.41 and 1.55 respectively.All three xAI-GAN systems outperformed standard GAN inthis setting, with the xAI-GAN lime system resulting in animprovement of 19.62%. A sample of the images generatedby the xAI-GAN lime system using 35% data can be seen inFigure 3.The results of the experiments on the Fashion MNISTdataset can be found in Figure 4. For 100% data, stan- Figure 6: FID scores on CIFAR10 DatasetFigure 7: Sample of Images Generated by the xAI-GAN saliency
System using 20% Data on CIFAR10 Datasetdard GAN produced an FID score of 1.16. The xAI-GAN shap , xAI-GAN lime and xAI-GAN saliency systemsproduced scores of 1.06, 0.97 and 1.1 respectively. Again,the xAI-GAN lime system had the best results, with animprovement of 16.67% over standard GAN. For 35%data, standard GAN produced a score of 1.61. The xAI-GAN shap , xAI-GAN lime and xAI-GAN saliency systemsproduced scores of 1.24, 1.35 and 1.34 respectively. Here,the xAI-GAN shap system had the best performance, with animprovement of 23.18%. A sample of the images generatedby the xAI-GAN shap system using 35% data can be seen inFigure 5.The average time taken by each of the GANs on therespective dataset can be found in Table 3. The xAI-GAN saliency system runs in about 2x the time that stan-dard GAN does, while the xAI-GAN shap and xAI-GAN lime systems take around 10x and 35x the time that standardGAN requires respectively. The reason for the discrepancyin times between the xAI-GANs systems is due to the differ-ence between their implementation. Overall, xAI-GAN hasbeen shown to outperform standard GAN in terms of FIDscores, with improvements of up to 23.18%.
We next ran our experiments on the CIFAR10 dataset usingthe parameters described earlier. In order to view the effi-cacy of xAI-GAN in the case where data is scarce, we used100% of the data for the standard GAN while only using20% data for xAI-GAN. Further, to compare with the workin (Zhao et al. 2020), we used the Differential Augmenta-tion implementation code linked in the paper to run anotherset of experiments. In the latter experiments, we added Dif-ferential Augmentation to all GANs - which we will here-after refer to as “+ Diff”. Note that standard GAN+Diffstill uses 100% of the data while all the xAI-GANs+Diffse 20% data. The results of these experiments are foundin Figure 6. In the first run of the experiment (i.e with-out Differential Augmentation), standard GAN resulted in aFID score of 214.81 while the xAI-GAN shap , xAI-GAN lime and xAI-GAN saliency systems resulted in scores of 218.48,210.04 and 211.16 respectively. The xAI-GAN lime systemshowed the best results with around 2.22% improvement inFID score over standard GAN, even with 20% of the data.Adding Differential Augmentation to standard GAN re-sulted in an FID score of 212.81, which is 0.93% im-provement in FID score over standard GAN. As previouslyshown, xAI-GAN (in particular the xAI-GAN lime system)produced better results than Differential Augmentation -even when using 20% of the data. Moreover, both meth-ods can be combined to produce complementary results.Running xAI-GANs with Differential Augmentation pro-duced better scores for all GANs, with xAI-GAN shap +Diff,xAI-GAN lime +Diff and xAI-GAN saliency +Diff resulting inscores of 214.27, 208.45 and 209.70 respectively.The times taken (in seconds) by each of the GANs to runthe experiments can be found in Table 4. xAI-GAN saliency takes around 2x the time and xAI-GAN lime and xAI-GAN shap take around 25x the time that standard GAN re-quires. The time taken by xAI-GAN shap system is simi-lar to the xAI-GAN lime system as the implementation ofshap is expensive in the case of color images. Overall, xAI-GAN lime +Diff shows an improvement of 2.96% in FIDscore over standard GAN while using 20% of the data. Notethat the discrepancy in FID scores (and conversely, the train-ing time) between xAI-GAN and standard GAN would behigher if xAI-GANs used 100% of the dataset for training.
We performed extensive experiments using MNIST, Fash-ion MNIST and CIFAR10 datasets on both fully connectedand DC GANs and showed that xAI-GAN, particularly oneusing the lime xAI system, results in improvements of upto 23.18% in FID scores over standard GAN. We comparedour work with (Zhao et al. 2020) and showed that xAI-GANresulted in improvement over Differential Augmentation inour experiments, and that both techniques are complemen-tary and can be combined to result in even more improve-ment in FID scores. We believe that the important take-away is that xAI-GANs show an improvement over standardGANs in terms of FID score even when using less data.
Regarding the increased training time of xAI-GAN:
While xAI-GAN requires more training time compared tostandard GANs due to the overhead of the xAI system, webelieve this is still an advantageous trade-off. GAN researchfocuses on improving the quality of the images and han-dling data scarcity, and consequently the time required totrain is not as important. In addition, xAI systems scale lin-early with the number of neurons in the DNN model. There-fore, the overhead caused by the xAI system will be linearto the model - allowing most hardware that can train stan-dard GANs to be able to train xAI-GANs. Furthermore, withthe advent of parallel and distributed computing as well aseasier access to powerful computational resources, the timerequired to train xAI-GAN will be further mitigated.
Our results suggest xAI-GAN can be leveraged in settingswhere data efficiency is important - such as where trainingdata is limited or in privacy conscious settings. It can be alsoused in normal settings to produce better quality GANs.
While standard GANs only use one value (loss) to calculatecorrective feedback to the generator, there are many waysthis feedback is used. For instance, several GANs vary thetype of loss function (Metz et al. 2016; Arjovsky, Chintala,and Bottou 2017; Zhao, Mathieu, and LeCun 2016; Dziu-gaite, Roy, and Ghahramani 2015; Gulrajani et al. 2017) andthe selection of the optimizer (such as Stochastic GradientDescent) to control how the model learns. Similarly, we be-lieve that the feedback provided by xAI system using xAI-GAN - which is “richer” compared to only using the lossvalue - can allow for greater control over this learning pro-cess. This control can be applied in various ways, such as inselecting the type of xAI system to use, varying the parame-ters of the chosen xAI system, offsetting the mask M to ad-just the weight given to xAI feedback, alternating betweenxAI-guided and standard generator training, and selectingmethods to combine xAI feedback with loss. We argue thatxAI-GANs are a powerful way for users to gain greater con-trol over the training process of GAN models, and that thereare many avenues, applications, and extensions of this ideaworth exploring in the future. In this paper, we introduce xAI-GANs, a class of genera-tive adversarial network (GAN) that use an explainable AI(xAI) system to provide “richer” feedback from the discrim-inator to the generator to enable more guided training andgreater control. We next overview xAI systems and standardGAN training and then introduce our xAI-guided genera-tor training algorithm, contrasting it’s difference with stan-dard generator training. To the best of our knowledge, xAI-GAN is the first GAN to utilize xAI feedback for training.We perform experiments using MNIST and Fashion MNISTdatasets and show that xAI-GAN has an improvement inFr´echet Inception Distance of up to 23.18% as comparedto standard GANs. In addition, we train xAI-GAN on theCIFAR10 dataset using only 20% of the data and compareit with standard GAN trained on 100% and show that xAI-GAN outperforms standard GAN even in this setting. Wecompare our work to the Differentiable Augmentation tech-nique and show that xAI-GAN trained on 20% of the dataoutperforms standard GAN trained with Differential Aug-mentation. We further combine xAI-GAN with Differen-tial Augmentation to produce even better results. There isa trade-off between data-efficiency, training time and qual-ity of images in GANs and our experiments show that xAI-GAN saliency provides the best value out of the xAI systemscompared. Ultimately, xAI-GAN may enable greater controlover the GAN learning process - allowing for better perfor-mance as well as a better understanding of GAN learning. eferences
Arjovsky, M.; Chintala, S.; and Bottou, L. 2017. Wassersteingan. arXiv preprint arXiv:1701.07875 .Bi´nkowski, M.; Sutherland, D. J.; Arbel, M.; and Gret-ton, A. 2018. Demystifying mmd gans. arXiv preprintarXiv:1801.01401 .Biran, O.; and Cotton, C. 2017. Explanation and justificationin machine learning: A survey. In
IJCAI-17 workshop onexplainable AI (XAI) , volume 8, 1.Chen, X.; Duan, Y.; Houthooft, R.; Schulman, J.; Sutskever,I.; and Abbeel, P. 2016. Infogan: Interpretable representationlearning by information maximizing generative adversarialnets. In
Advances in neural information processing systems ,2172–2180.Duchi, J.; Hazan, E.; and Singer, Y. 2011. Adaptive subgra-dient methods for online learning and stochastic optimiza-tion.
Journal of Machine Learning Research arXiv preprint arXiv:1505.0396 .Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.;Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y.2014. Generative Adversarial Nets. In Ghahramani, Z.;Welling, M.; Cortes, C.; Lawrence, N. D.; and Weinberger,K. Q., eds.,
Advances in Neural Information Processing Sys-tems 27 , 2672–2680. Curran Associates, Inc. URL http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf.Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; andCourville, A. C. 2017. Improved training of wassersteingans. In
Advances in neural information processing systems ,5767–5777.Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; andHochreiter, S. 2017. Gans trained by a two time-scale updaterule converge to a local nash equilibrium. In
Advances inNeural Information Processing Systems , 6626–6637.Horn, R. A. 1990. The hadamard product. In
Proc. Symp.Appl. Math , volume 40, 87–169.Ignatiev, A.; Narodytska, N.; and Marques-Silva, J. 2019.Abduction-based explanations for machine learning models.In
Proceedings of the AAAI Conference on Artificial Intelli-gence , volume 33, 1511–1519.Kingma, D. P.; and Ba, J. 2014. Adam: A method forstochastic optimization. arXiv preprint arXiv:1412.6980 .Kokhlikyan, N.; Miglani, V.; Martin, M.; Wang, E.;Reynolds, J.; Melnikov, A.; Lunova, N.; and Reblitz-Richardson, O. 2019. PyTorch Captum. https://github.com/pytorch/captum.Krizhevsky, A.; Nair, V.; and Hinton, G. 2010. CIFAR-10 (Canadian Institute for Advanced Research).
Unpub-lished Manuscript ∼ kriz/cifar.html. LeCun, Y.; and Cortes, C. 2010. MNIST handwritten digitdatabase. http://yann.lecun.com/exdb/mnist/. URL http://yann.lecun.com/exdb/mnist/.Lundberg, S. M.; and Lee, S.-I. 2017a. A Unified Ap-proach to Interpreting Model Predictions. In Guyon, I.;Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vish-wanathan, S.; and Garnett, R., eds., Advances in Neural In-formation Processing Systems 30 , 4765–4774. Curran Asso-ciates, Inc. URL http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf.Lundberg, S. M.; and Lee, S.-I. 2017b. A unified approachto interpreting model predictions. In
Advances in NeuralInformation Processing Systems , 4765–4774.Makhzani, A.; Shlens, J.; Jaitly, N.; Goodfellow, I.; andFrey, B. 2015. Adversarial autoencoders. arXiv preprintarXiv:1511.05644 .Mao, X.; Li, Q.; Xie, H.; Lau, R. Y.; Wang, Z.; andPaul Smolley, S. 2017. Least squares generative adversarialnetworks. In
Proceedings of the IEEE International Confer-ence on Computer Vision , 2794–2802.Metz, L.; Poole, B.; Pfau, D.; and Sohl-Dickstein, J. 2016.Unrolled Generative Adversarial Networks.Mirza, M.; and Osindero, S. 2014. Conditional generativeadversarial nets. arXiv preprint arXiv:1411.1784 .Pan, Z.; Yu, W.; Yi, X.; Khan, A.; Yuan, F.; and Zheng, Y.2019. Recent progress on generative adversarial networks(GANs): A survey.
IEEE Access
7: 36322–36333.Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.;DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; and Lerer,A. 2017. Automatic differentiation in pytorch.Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.;Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.;et al. 2019. PyTorch: An imperative style, high-performancedeep learning library. In
Advances in Neural InformationProcessing Systems , 8024–8035.Radford, A.; Metz, L.; and Chintala, S. 2015. Unsupervisedrepresentation learning with deep convolutional generativeadversarial networks. arXiv preprint arXiv:1511.06434 .Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2016. ”WhyShould I Trust You?”: Explaining the Predictions of AnyClassifier. In
Proceedings of the 22nd ACM SIGKDD In-ternational Conference on Knowledge Discovery and DataMining, San Francisco, CA, USA, August 13-17, 2016 ,1135–1144.Roh, Y.; Heo, G.; and Whang, S. E. 2019. A survey on datacollection for machine learning: a big data-ai integration per-spective.
IEEE Transactions on Knowledge and Data Engi-neering .Shrikumar, A.; Greenside, P.; and Kundaje, A. 2017. Learn-ing important features through propagating activation differ-ences. In
Proceedings of the 34th International Conferenceon Machine Learning-Volume 70 , 3145–3153. JMLR. org.Simonyan, K.; Vedaldi, A.; and Zisserman, A. 2013.Deep inside convolutional networks: Visualising imagelassification models and saliency maps. arXiv preprintarXiv:1312.6034 .Xiao, H.; Rasul, K.; and Vollgraf, R. 2017. Fashion-mnist:a novel image dataset for benchmarking machine learningalgorithms. arXiv preprint arXiv:1708.07747 .Zhao, J.; Mathieu, M.; and LeCun, Y. 2016. Energy-based generative adversarial network. arXiv preprintarXiv:1609.03126 .Zhao, S.; Liu, Z.; Lin, J.; Zhu, J.-Y.; and Han, S. 2020. Dif-ferentiable Augmentation for Data-Efficient GAN Training. arXiv preprint arXiv:2006.10738arXiv preprint arXiv:2006.10738