Direct Adversarial Training: An Adaptive Method to Penalize Lipschitz Continuity of the Discriminator
DDirect Adversarial Training for GANs
Ziqiang Li , Pengfei Xia , Mingdao Yue , Bin Li CAS Key Laboratory of Technology in Geo-spatial Information Processing and ApplicationSystemsUniversity of Science and Technology of China, Anhui, China College of Mechanical and Electrical Engineering, Suzhou University, Anhui, China. { iceli,xpengfei } @mail.ustc.edu.cn, { ymdustc,binli } @ustc.edu.cn Abstract
There is an interesting discovery that several neural networksare vulnerable to adversarial examples. That is many machinelearning models misclassify the samples with only a littlechange which will not be noticed by human eyes. Genera-tive adversarial networks (GANs) are the most popular mod-els for image generation by jointly optimizing discriminatorand generator. In order to stabilize training, some regulariza-tion and normalization have been used to let the discriminatorsatisfy Lipschitz continuous. In this paper, we have analyzedthat the generator may produce adversarial examples for dis-criminator during the training process, which may cause theunstable training of GANs. For this reason, we propose directadversarial training for GANs. At the same time, we provethat this direct adversarial training can limit the Lipschitzconstant of the discriminator adaptively and accelerate theconvergence of the generator. We have verified the advancedperforms of the method on multiple baseline networks, suchas DCGAN, WGAN, WGAN-GP, and WGAN-LP.
Introduction
Recently, generative adversarial networks (GANs) (Good-fellow et al. 2014) have been used in several generativetasks, such as image impainting (Yu et al. 2018; Yeh et al.2017), attribute editing (Tao et al. 2019; Shen et al. 2020),adversarial examples (Xiao et al. 2018; Song et al. 2018).From the game perspective, GANs is a two-player zero-sum game, in which a discriminator measures the distancebetween generated and real distributions, while a genera-tor tries to fool the discriminator by minimizing the dis-tance between real and generated distributions. Specifically,vanilla GAN (Goodfellow et al. 2014) trains a critic to ap-proximate the JS divergence, f-GAN(Nowozin, Cseke, andTomioka 2016) and WGAN (Arjovsky, Chintala, and Bot-tou 2017) trains a critic to approximate the f divergence andWasserstein divergence respectively. According to the opti-mal transport theory (Bonnotte 2013), Wasserstein distanceis the transportation cost of the optimal transportation map.The solution for Wasserstein divergence can be get by Kan-torovich duality(Kantorovich 2006), which is existence anduniqueness. In order to solve the Kantorovich duality prob-lem of the Wasserstein distance, discriminator must satisfy Copyright c (cid:13) the 1-Lipschitz continuous. This is the first time that Lips-chitz continuous has been used for training of GANs. Exceptfor 1-Lipschitz continuous in WGANs, Lipschitz continuousis important for generalizability and distributional consis-tency for GANs (Qi 2020). Qi et al. (Qi 2020) proved that,for the discriminator with Lipschitz continuous, the gener-ated distributions converge to the real distributions in GANs.Adversarial examples (Szegedy et al. 2013) is a prob-lem that often occurs in neural networks. Many sam-ples with small perturbations has been misclassified withhigh probability for state-of-the-art deep neural networks.These misclassified samples were named as
AdversarialExamples . This misclassification is called adversarial at-tack. At present, the best defense of adversarial attacks isthrough adversarial training.
Adversarial training (Good-fellow, Shlens, and Szegedy 2014) used adversarial exam-ples to train the classifier, that improved the robustness ofthe models. Inspired by adversarial examples in deep neuralnetworks, we think that the instability of the GANs trainingis due to the adversarial examples generated by generatorwhich fool the discriminator. Based on the above assump-tions, we first did a confirmatory experiment. The adversar-ial attack with the discriminator of DCGAN (Radford, Metz,and Chintala 2015) are shown in Fig.1. The success of theattack is defined as: | D (ˆ x f ) − D ( x r ) | ≤ . | D (ˆ x r ) − D ( x f ) | ≤ . (1)Where D represents the discriminator. x f and x r repre-sent the fake images which generated by generator with-out convergence and real images samples from datasets, re-spectively. ˆ x f and ˆ x r are adversarial examples of generatedimages and real images, respectively. From Fig.1, we seethat the average iterations when discriminator without ad-versarial training has been successfully attacked are about3.5, which is too easy. Also, we can not distinguish betweenclean images and adversarial examples, intuitively. Since themodel is assumed to have infinite capacity (Goodfellow et al.2014), without any prior, the generator has a high probabilityto generate adversarial examples of the discriminator, whichmakes the training of GANs unstable. In order to avoid thisissue, this paper proposes a method called direct adversarialtraining for GANs, which accelerates the convergence, re-duces the Wasserstein distance between real and generated a r X i v : . [ ee ss . I V ] A ug ake samples Real samples
Images with Adversarial attackClean images (a) adversarial attack without the adversarial training( ¯ ξ fake = 3 . , ¯ ξ real = 3 . ) Fake samples
Real samples
Images with Adversarial attackClean images (b) adversarial attack with the adversarial training( ¯ ξ fake = 37 . , ¯ ξ real = 7 . ) Figure 1: Adversarial attack with the discriminator. (a) and (b) are the adversarial samples when DCGAN has no adversarialtraining and adversarial training on mnist, respectively. Top two lines are the fake samples which are generated by generatorwithout convergence. Bottom two lines are real samples from mnist dataset. ¯ ξ fake and ¯ ξ real are the average number of iterationswhen fake images and real images are successfully attacked, respectivelydistributions, and improves the quality of the generated im-ages. The main contributions can be summarized as follows: • Different from the adversarial perturbation on the classi-fier, we propose a new perturbation method based on thedistance metric in the discriminator. • We propose to use direct adversarial training in the train-ing of GANs, which has achieved better results on multi-ple popular GANs. • We prove direct adversarial training can adaptively adjustLipschitz continuous of the discriminator, which is differ-ent from the gradient penalty proposed by previous workand accelerate the convergence of the generator.
Background and Related Work
Generative Adversarial Networks
Generative adversarial networks (GANs) is a two-playerzero-sum game, where the generator G ( z ) is a distribu-tion mapping function that transforms samples of a low-dimensional latent distribution z into samples from the targetimages distribution P g ( x ) . The generator is trained with an-other network D ( x ) , which evaluates the distance betweengenerated distribution P g ( x ) and real distribution P r ( x ) .The generator and discriminator minimize and maximize thedistribution distance respectively. This minimax game canbe expressed as: min φ max θ f ( φ, θ ) = E x ∼ p r [ g ( D θ ( x ))]+ E z ∼ p z [ g ( D θ ( G φ ( z )))] (2)Where φ and θ are parameters of the generator G and dis-criminator D , respectively. P r and P z represent the realdistribution and latent distribution respectively. Specifically,vanilla GAN (Goodfellow et al. 2014) can be described by g ( t ) = g ( − t ) = − log(1 + e − t ) , f -GAN (Nowozin,Cseke, and Tomioka 2016) and WGAN (Arjovsky, Chintala,and Bottou 2017) can be written as g ( t ) = − e − t ,g ( t ) = 1 − t and g ( t ) = g ( − t ) = t respectively. Lipschitz Constant and WGAN
Lipschitz constant of the function f : X → Y is defined by: || f || L = sup x,y ∈ X ; x (cid:54) = y || f ( x ) − f ( y ) |||| x − y || (3)Intuitively, the function f is called K-Lipschitz continuous,if there exists a constant K ≥ for which || f ( x ) − f ( y ) || ≤ K || x − y || for any x, y ∈ X . Theoretically, Lipschitz con-stant of the neural network can be approximated by spectralnorm of the weight matrix: (cid:107) W (cid:107) = max x (cid:54) =0 (cid:107) W x (cid:107)(cid:107) x (cid:107) (4)Lipschitz constant represents the continuous of the neuralnetworks. The low Lipschitz constant means that the neuralnetwork is less sensitive to input perturbation and has bet-ter generalization (Yoshida and Miyato 2017; Oberman andCalder 2018; Couellan 2019).For GANs, a lot of work is used to limit the Lipschitzconstant of the discriminator. The main mothods are spec-tral normalization (Miyato et al. 2018a) and WGAN (Ar-jovsky, Chintala, and Bottou 2017). Spectral normalizationnormalize the spectral norm of the discriminator, which lim-its its Lipschitz constant to 1. And the Lipschitz constant inWGAN is derived from Kantorovich duality (Kantorovich2006), the Wasserstein distance corresponding to the opti-mal transmission can be represented as: W ( P , P ) = sup || f || L =1 E x ∼ p r f ( x ) − E x ∼ p g f ( x ) (5)Where f : X → R is called the Kantorovich potential,which can be used as distriminator. In order to make thediscriminator satisfy the lipschitz continuous, WGAN (Ar-jovsky, Chintala, and Bottou 2017) use the weight clippingwhich restrict the maximum value of each weight; WGAN-GP (Gulrajani et al. 2017) use the gradient penalty withthe interpolation of real samples and generated samples: x = tx +(1 − t ) y for t ∼ U [0 , and x ∼ P r , y ∼ P g being areal and generated samples; WGAN-ALP (Terj´ek 2019) in-spired by Virtual Adversarial Training (VAT) (Miyato et al.2018b) restrict the 1-Lipschitz continuous at ˆ x = { x, y } with the direction of adversarial perturbation. Different fromthe above methods which restrict the 1-Lipschitz continuous(Gulrajani et al. 2017; Terj´ek 2019; Kodali et al. 2017; Zhouet al. 2019b), WGAN-LP (Petzka, Fischer, and Lukovnicov2017) restrict the k -Lipschitz continuous ( k ≤ ), whichis derived from the optimal transport with regularization.Also, qi et al. (Qi 2020) are motivated to have lower sam-ple complexity by directly minimizing the Lipschitz con-stant rather than constraining it to one, which can be de-scribed as 0-GP (Zhou et al. 2019a; Mescheder, Geiger, andNowozin 2018; Thanh-Tung, Tran, and Venkatesh 2019). Insummary, there are many methods for restricting the Lips-chitz constant, some restrict the constant to 1, some restrictit to k ( k ≤ , and some minimize the Lipschitz constant. Adversarial Examples and Adversarial Training
Adversarial examples are a common problem in neural net-works. Given a pre-trained model h , an adversarial example x (cid:48) is defined by x (cid:48) = x + δ with h ( x (cid:48) ) (cid:54) = h ( x ) for un-targeted attack or h ( x (cid:48) ) = t for targeted attack. Where x is a clean image and δ is a imperceptible tiny perturbation.There are many methods to generate adversarial examples,such as Fast Gradient Sign Method(FGSM) (Goodfellow,Shlens, and Szegedy 2014), Basic Iterative Methods(BIM)(Kurakin, Goodfellow, and Bengio 2016), and ProjectedGradient Descent(PGD) (Madry et al. 2017). FGSM uses thesingle gradient step to create adversarial examples: (cid:26) x (cid:48) = x + (cid:15) · sign ( ∇ x L ( x, y )) f or untargetedx (cid:48) = x − (cid:15) · sign ( ∇ x L ( x, t )) f or targeted (6)Where L is the loss function. For untargeted attack, y is thetrue label of the clean image, so we use the gradient ascent;and for targeted attack, t is the label of the target image,so we use the gradient descent. PGD is a multi-step methodwhich create adversarial examples by iterative: (cid:40) x k +1 = clip (cid:0) x k + α · sign ( ∇ x L ( x k , y )) (cid:1) f or untargetedx k +1 = clip (cid:0) x k − α · sign ( ∇ x L ( x k , t )) (cid:1) f or targeted (7)Where clip is the clip function, α is the step of gradient, x = x , x (cid:48) = x K and K is the number of iterations.Adversarial training is a good and sample method to avoidadversarial examples, which improves the robustness of theneural networks by: min θ E x,y ∼ D [ max || δ || p ≤ (cid:15) L θ ( x + δ, y )] (8)Where x, y ∼ D is the joint distribution of data (image, la-bel), θ is the parameter of the model. This min-max problemis similar to the GANs, the main difference is the indepen-dent variable of the maximization problem is image sample x in adversarial training, not parameters of the discriminator.For GANs, most of the work attack the generator to pro-duce images we dont expect. There is no paper analyzing the relationship between the Lipschitz continuous of dis-criminator and adversarial training. The work closest to us isRob-GAN(Liu and Hsieh 2019), which added the adversar-ial training for classifier in cGANs. Different from the Rob-GAN, we consider that the discriminator may be affected byadversarial examples during training. Based on this, we pro-pose a direct adversarial training for the discriminator, andprove that this adversarial training can improve the robust-ness of the discriminator, accelerate the convergence of thegenerator, and improve the quality of the generated images. Proposed Approach
We propose the direct adversarial training for GANs. Unlikeprevious adversarial training for classifiers, direct adversar-ial training is for discriminators, which is a distribution met-ric function. The adversarial examples on the discriminatoris defined as Eq1, the unintentional attack for discriminatorcan be seen as a targeted attack which minimize the distance(under the discriminator) between the adversarial examplesof the generated images and the real ones (or that betweenthe examples of the reals images and generated ones). Inthis part, we first introduce a adversarial perturbation basedon distribution metric, and then we propose the direct adver-sarial training to defend against unintentional discriminatorattacks, finally we analyze the relationship between directadversarial training and gradient penalty and its ability toaccelerate generator convergence. G D PGD attack
PGD attack
Discrimination
Loss
Figure 2: Illustration of the training process. This is similarto the GANs training. The difference is that we add the one-step PGD attack for real and generated images, we hope thatthe discriminator can not only identify real or feak, but alsobe robust to adversarial samples
Adversarial Perturbation of the Discriminator
Most of the adversarial training is about classifiers, the goalof the classifier is fixed. For untargeted attacks, the directionof adversarial perturbation is the solution of min L ( x, y ) ;and for targeted attacks, the direction of adversarial pertur-bation is the solution of max L ( x, t ) , where mathcalL isloss function, y and t are true and target label of the image lgorithm 1 Direct Adversarial Training
Input:
The batch size m, the real image distribution P r ( x ) , the random noize z ∼ N (0 , , the maximum number of trainingsteps K, the number of steps to apply to the discriminator N, the loss function d and g , the parameter λ and λ . Output: a fine-tuned generator G and discriminator D for k=1, 2, · · · , K do for n=1,2, · · · , N do Draw m real samples x r = { x (1) r , x (2) r , · · · , x ( m ) r } from the real data discribution p r ( x ) . Draw m latent noise z = { z (1) , z (2) , · · · , z ( m ) } . x f = { x (1) f , x (2) f , · · · , x ( m ) f } = G φ (cid:0) { z (1) , z (2) , · · · , z ( m ) } (cid:1) { δ (1) f , δ (2) f , · · · , δ ( m ) f } = − (cid:15) ∇ x ( i ) f (cid:0) (cid:12)(cid:12)(cid:12) g ( D ( x ( i ) f )) − g ( ¯ D ( x r )) (cid:12)(cid:12)(cid:12) (cid:1) { δ (1) r , δ (2) r , · · · , δ ( m ) r } = − (cid:15) ∇ x ( i ) r (cid:0) (cid:12)(cid:12)(cid:12) g ( D ( x ( i ) r )) − g ( ¯ D ( x f )) (cid:12)(cid:12)(cid:12) (cid:1) x (1) f − adv , x (2) f − adv , · · · , x ( m ) f − adv = (cid:0) { x (1) f + δ (1) f , x (2) f + δ (2) f , · · · , x ( m ) f + δ ( m ) f } (cid:1) x (1) r − adv , x (2) r − adv , · · · , x ( m ) r − adv = (cid:0) { x (1) r + δ (1) r , x (2) r + δ (2) r , · · · , x ( m ) r + δ ( m ) r } (cid:1) Update the discriminator by ascending its stochastic gradient: ∇ θ (cid:26) λ m m (cid:88) i =1 (cid:2) g ( D θ ( x ( i ) r )) + g ( D θ ( x ( i ) f ) (cid:3) + λ m m (cid:88) i =1 (cid:2) g ( D θ ( x ( i ) r − adv )) + g ( D θ ( x ( i ) f − adv ) (cid:3)(cid:27) end for Draw m latent noise { z (1) , z (2) , · · · , z ( m ) } . Update the generator by descending its stochastic gradient: ∇ φ − m m (cid:88) i =1 (cid:2) g ( D θ ( x ( i ) r )) + g ( D θ ( G φ ( z ( i ) ))) (cid:3) end for return x , respectively. But for discriminator, the goal of the out-put is dynamic, such as vanilla GAN, the output of the realimages and fake images is . for optimal discriminator. Sothe target of the output changes dynamically with training.Based on this, we propose a new adversarial perturbation fordiscriminator. Defining the optimization problem: δ ( x r ) = arg min || δ r || p ≤ (cid:15) | g ( D θ ( x r + δ r )) − ¯ g ( D θ ( x f )) | δ ( x f ) = arg min || δ f || p ≤ (cid:15) | g ( D θ ( x f + δ f )) − ¯ g ( D θ ( x r )) | (9)Where x r and x f are real image and generated image, re-spectively. δ ( x r ) and δ ( x f ) are adversarial perturbation ofthe real image x r and generated image x f , respectively. ¯ g ( D θ ( x r )) and ¯ g ( D θ ( x f )) are the average value of cur-rent discriminator output of the real images and generatedimages with a batch, respectively. g and g are function inEq2. For above optimization problem in Eq (9). We use theone-step PGD attack to achieve it: δ ( x r ) = − (cid:15) ∇ x r (cid:0) | g ( D ( x r )) − ¯ g ( D ( x f )) | (cid:1) δ ( x f ) = − (cid:15) ∇ x f (cid:0) | g ( D ( x f )) − ¯ g ( D ( x r )) | (cid:1) (10)Where the goal of the adversarial perturbation is change withdiscriminator. Direct Adversarial Training
Motivated by adversarial examples lead to unstable train-ing of GANs, we propose the direct adversarial training forGANs in Fig.2. According to the loss function of GANs inEq (2), the GANs loss with adversarial training can be wr-ited as: min φ max θ f (cid:48) ( φ, θ ) = λ f o ( φ, θ ) + λ f a ( φ, θ ) = E x ∼ p r [ λ g ( D θ ( x )) + λ g ( D θ ( x + δ r ))]+ E z ∼ p z [ λ g ( D θ ( G φ ( z ))) + λ g ( D θ ( G φ ( z ) + δ f ))] (11)Where δ r and δ f are adversarial perturbation of the discrim-inator. According to the above formula, we can write a com-plete algorithm in Algorithm 1 . Direct Adversarial Training and LipschitzContinuous
In this part, we analyze the relationship between direct ad-versarial training and gradient penalty. The results showsthat our method can adjust the Lipschitz constant adap-tively. The instability of GANs training is mainly causedby the discriminator, and our adversarial training is onlyfor the discriminator, so we do not consider the genera-tor at present. For adversarial perturbation of real images δ ( x r ) = − (cid:15) ∇ x r (cid:0) | g ( D ( x r )) − ¯ g ( D ( x f )) | (cid:1) , update of the a) real samples (b) GAN-2k (c) GAN-3k (d) GAN-10k (e) GAN-adv-2k (f) GAN-adv-3k (g) GAN-adv-10k Figure 3: Qualitative results of 9 2D-Gaussian synthetic data. (a): the real samples from a mixture of nine Gaussians. Wherevariance is 0.1 and means are { -1, 0, 1 } ; (b), (c) and (d): the results of iterating 2k times, 3k times, and 10k times in DCGAN;(e), (f) and (g): the results of iterating 2k times, 3k times, and 10k times in DCGAN with adversarial training respectively. (a) real samples (b) GAN-2k (c) GAN-3k (d) GAN-10k (e) GAN-adv-2k (f) GAN-adv-3k (g) GAN-adv-10k Figure 4: Qualitative results of an imbalanced 9 2D-Gaussian synthetic data. (a): the real samples from a imbalanced mixture ofnine Gaussians. Where variance is 0.1, means are { -1, 0, 1 } , and the probability of mixing Gaussian from left to right is 0.15,0.5, 0.8; (b), (c) and (d): the results of iterating 2k times, 3k times, and 10k times in DCGAN; (e), (f) and (g): the results ofiterating 2k times, 3k times, and 10k times in DCGAN with adversarial training respectively.Figure 5: The results of the DCGAN with different parame-ters in Cifar10. Where ’lr’ is learning rate, ’b1’ is the param-eter of the adam optimizer, and ’It’ is the ratio of the updatetimes of the discriminator to the generator. IS is InceptionScore which represents the quality of the generated images.discriminator can be writed as: max θ E x ∼ p r (cid:20) g (cid:0) D θ ( x + δ ( x )) (cid:1)(cid:21) ≈ max θ E x ∼ p r (cid:20) g (cid:0) D θ ( x ) (cid:1) + ∇ x g (cid:0) D θ ( x ) (cid:1) · δ ( x ) (cid:21) (12)Calculate the gradient of the Eq (12): ∇ θ (cid:20) g (cid:0) D θ ( x r ) (cid:1) + ∇ x r g (cid:0) D θ ( x r ) (cid:1) · δ ( x r ) (cid:21) = ∇ θ (cid:20) g (cid:0) D θ ( x r ) (cid:1) − (cid:15) ∇ x r g (cid:0) D θ ( x r ) (cid:1) · ∇ x r | g ( D θ ( x r )) − ¯ g ( D θ ( x f )) | (cid:21) = ∇ θ (cid:20) g (cid:0) D θ ( x r ) (cid:1) − (cid:15)g (cid:48) ∇ x r D θ ( x r ) · ∇ x r | g ( D θ ( x r )) − ¯ g ( D θ ( x f )) | (cid:21) (13)Because x f is sampled from fake images, which is in-dependent of x r , so when g ( D θ ( x r )) − ¯ g ( D θ ( x f )) ≥ ,which is true in most cases, the Eq (13) is ∇ θ (cid:20) g (cid:0) D θ ( x r ) (cid:1) − (cid:15)g (cid:48) ∇ x r D θ ( x r ) (cid:21) . In this case, max ∇ θ (cid:20) g (cid:0) D θ ( x r ) (cid:1) − (cid:15)g (cid:48) ∇ x r D θ ( x r ) (cid:21) is equivalent to adding ”gradient penalty”to loss, which can be seen as 0-GP mentioned in Section2. 0-GP can be used to limit the Lipsschitz constant andstabilize the training of GANs. Also when g ( D θ ( x r )) − ¯ g ( D θ ( x f )) < , which means that the discriminator isclearly trained incorrectly, the Eq (13) is ∇ θ (cid:20) g (cid:0) D θ ( x r ) (cid:1) + (cid:15)g (cid:48) ∇ x r D θ ( x r ) (cid:21) . In this case, max ∇ θ (cid:20) g (cid:0) D θ ( x r ) (cid:1) + (cid:15)g (cid:48) ∇ x D θ ( x r ) (cid:21) means that we hope the discriminator willhave a large change, so as to jump out of the situation ofwrong discrimination.igure 6: The results of the WGAN with different parame-ters in cifar10. Where ’it’ is the ratio of the update times ofthe discriminator to the generator. ’update step of the genera-tor’ is the update times of the generator and ’IS’ is InceptionScore which represents the quality of the generated images.’Wasserstein distance’ is the wasserstein distance betweenthe true distribution and the generated distribution.For adversarial perturbation of generated images δ ( x f ) = − (cid:15) ∇ x f (cid:0) | g ( D ( x f )) − ¯ g ( D ( x r )) | (cid:1) , where x f = G θ ( z ) .Update of the discriminator can be writed as: max θ E x ∼ p f (cid:20) g (cid:0) D θ ( x + δ ( x )) (cid:1)(cid:21) ≈ max θ E x ∼ p f (cid:20) g (cid:0) D θ ( x ) (cid:1) + ∇ x g (cid:0) D θ ( x ) (cid:1) · δ ( x ) (cid:21) (14)We can get the gradient form similar to the real images: ∇ θ (cid:20) g (cid:0) D θ ( x f ) (cid:1) − (cid:15)g (cid:48) ∇ x f D θ ( x f ) · ∇ x f | g ( D θ ( x f )) − ¯ g ( D θ ( x r )) | (cid:21) (15)When g ( D θ ( x f )) ≥ ¯ g ( D θ ( x r )) , the gradient of thediscriminator in generated images is ∇ θ (cid:20) g (cid:0) D θ ( x f ) (cid:1) − (cid:15)g (cid:48) ∇ x f D θ ( x f ) (cid:21) . For generated images x f , usually g ( D θ ( x f )) is greater than g ( D θ ( x r )) , in this case theadversarial training is equivalent to gradient penalty; and g ( D θ ( x f )) < ¯ g ( D θ ( x r )) indicates that the discriminatorhas an error and may get the local saddle point, so we max-imize the gradient of the discriminator ∇ x f D θ ( x f ) , whichcan make the discriminator jump out of the error point assoon as possible.From the above analysis, it can be seen that our proposeddirect adversarial training can adaptively limit the lipschitzcontinuity, which is equivalent to 0-gp when the discrimina-tor performance is better, and relax the limit on the lipschitz Figure 7: The results of the WGAN-GP with different pa-rameters in Cifar10. Where ’It’ is the ratio of the updatetimes of the discriminator to the generator. ’update step ofthe generator’ is the update times of the generator and ’IS’is Inception Score which represents the quality of the gen-erated images. ’Wasserstein distance’ is the wasserstein dis-tance between the true distribution and the generated distri-bution.constant when the discriminator performance is poor. Accelerate generator convergence by DirectAdversarial Training
We reconsider the direct adversarial training on the gener-ated images from the perspective of generator convergence.We assume that there is a constant C that satisfies: (cid:107) D θ ( G φ ( z ) + δ ) − D θ ( G φ ( z )) (cid:107) ≤ C (16)Where δ is the adversarial perturbation, which represents themaximum perturbation direction for the discriminator, so forany other perturbation δ : C ≥ (cid:107) D θ ( G φ ( z ) + δ ) − D θ ( G φ ( z )) (cid:107)≥ (cid:107) D θ ( G φ ( z ) + δ ) − D θ ( G φ ( z )) (cid:107) (17)Of course, if we consider the generator to be updated twicebefore and after as a small perturbation, there are: C ≥ (cid:107) D θ ( G φ ( z ; w t +1 )) − D θ ( G φ ( z ; w t )) (cid:107)≥ (cid:107) D (cid:48) θ ( G φ ( z ; w t )) (cid:107) · (cid:107) G φ ( z ; w t +1 ) − ( G φ ( z ; w t )) (cid:107)≥ (cid:107) D (cid:48) θ ( G φ ( z ; w t )) (cid:107) · (cid:107) ∂∂w G φ ( z ; w t ) (cid:107) · (cid:107) w t +1 − w t (cid:107) (18)So the update of the generator weight (cid:107) w t +1 − w t (cid:107) ∝ C (cid:107) D (cid:48) θ ( G φ ( z ; w t )) (cid:107)·(cid:107) ∂∂w G φ ( z ; w t ) (cid:107) . If there is no gradient penalty,meaning (cid:107) D (cid:48) θ ( G φ ( z ; w t )) (cid:107) is high, than the update of thegenerator is slow, which shows the direct adversarial train-ing can acclerate the convergence of the generator. GAN-GP WGAN-GP with adversarial training
Figure 8: The generated images of WGAN-GP and WGAN-GP with direct adversarial training. The images are all gener-ated by the last generator in the training phase. Using ’It=1’strategy during training.
Experiments
This section, we will introduce the impact of the di-rect adversarial training on DCGAN, WGAN, WGAN-GPand WGAN-LP. The results show the effectiveness of ourmethod.
Experiments on DCGAN
First, we evaluate our method on a 2D synthetic dataset,which is a mixture of nine Gaussians. The variance of theGaussian distribution is 0.1, and the covariance is 0. Weuse four layers fully-connected MLP with 64 hidden unitsper layer to model the generator and discriminator. Fig 3shows the qualitative results of different iterations. The ad-versarial training can speed up the generation. When iter-ating 2k times, the generated distribution with adversarialtraining is closer to the true distribution. Also, we evaluatethe moethod on a imbalanced mixture of nine Gaussians.The probability of the three Gaussian components on the leftis 0.15, the probability of the middle three Gaussian compo-nents is 0.05, and the right three Gaussian components is 0.8.Fig 4 shows the qualitative results of different iterations. Itcan be seen from the results that GAN will lose some smallprobability distribution, resulting in mode collapse, and thisphenomenon will be significantly improved after adversarialtraining is added.At the last, we used DCGAN to do a comparative exper-iments on cifar10, results are shown in Fig 5. From the re-sults, under different parameters, the methods of direct ad-versarial training we proposed can improve the quality of thegenerated images.
Experiments on WGAN
This part, we used WGAN to do a comparative experimenton cifar10, the results are shown in Fig 6. From the results,The methods we propose can achieve better performance.For inception score, direct adversarial training can signifi-cantly improve the quality of generation. Especially when it=5, the training process falls into the local optimum, whichleads to a long-term decline in inception score, and the timeof the decline process is significantly shortened after usingdirect adversarial training, thereby reducing the difficulty oftraining. For Wasserstein distance, WGAN has obvious mu-tations in the training process. This mutation can be consid-ered as the result of adversarial examples. When using theadversarial training, the Wasserstein distance is smaller andmore stable.
Experiments on WGAN-GP and WGAN-LP
This part, we first used WGAN-GP to do a comparative ex-periment on cifar10, the results are shown in Fig 7. Becauseadversarial training can accelerate convergence, if ’It=5’, thediscriminator will be trained too well, it is not good for train-ing of GANs, the result with adversarial training is worsethan the previous method. And when ’It=1’, although theimprovement is not obvious from the Inception Score, theWasserstein distance is smaller and more stable. Also, wevisualize the generated results in Fig 8. From the result, wecan clearly observe that the direct adversarial training canimprove the quality of generation.Also, we used WGAN-LP to do a comparative experimenton cifar10, the results are shown in Fig 9 and Fig 10. Theresult is similar to the result of WGAN-GP.Figure 9: The results of the WGAN-LP with different pa-rameters in Cifar10. Where ’it’ is the ratio of the updatetimes of the discriminator to the generator. ’update step ofthe generator’ is the update times of the generator and ’IS’is Inception Score which represents the quality of the gen-erated images. ’Wasserstein distance’ is the Wasserstein dis-tance between the true distribution and the generated distri-bution.
Conclusions
Motivited by adversarial examples may affact the stabletraining of the GANs, in this paper, we use the direct ad-versarial training for GANs. Since the discriminator is a dis-tributed metric function, based on this, we propose a new
GAN-LP WGAN-LP with adversarial training
Figure 10: The generated images of WGAN-LP andWGAN-LP with adversarial attack. The images are all gen-erated by the last generator in the training phase. Using’It=1’ strategy during training.adversarial perturbation for the discriminator. Of course, wealso proved that this kind of adversarial training can adap-tively adjust the Lipschitz continuous, which can improvestability of the training, prevent falling into local optimum,and improve the quality of generated images.
References
Arjovsky, M.; Chintala, S.; and Bottou, L. 2017. Wassersteingan. arXiv preprint arXiv:1701.07875 .Bonnotte, N. 2013. From Knothe’s rearrangement to Bre-nier’s optimal transport map.
SIAM Journal on Mathemati-cal Analysis arXiv preprintarXiv:1904.06253 .Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.;Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y.2014. Generative adversarial nets. In
Advances in neuralinformation processing systems , 2672–2680.Goodfellow, I. J.; Shlens, J.; and Szegedy, C. 2014. Explain-ing and harnessing adversarial examples. arXiv preprintarXiv:1412.6572 .Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; andCourville, A. C. 2017. Improved training of wassersteingans. In
Advances in neural information processing systems ,5767–5777.Kantorovich, L. V. 2006. On a problem of Monge.
J. Math.Sci.(NY) arXiv preprintarXiv:1705.07215 .Kurakin, A.; Goodfellow, I.; and Bengio, S. 2016. Ad-versarial machine learning at scale. arXiv preprintarXiv:1611.01236 . Liu, X.; and Hsieh, C.-J. 2019. Rob-gan: Generator, dis-criminator, and adversarial attacker. In
Proceedings of theIEEE conference on computer vision and pattern recogni-tion , 11234–11243.Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.; andVladu, A. 2017. Towards deep learning models resistant toadversarial attacks. arXiv preprint arXiv:1706.06083 .Mescheder, L.; Geiger, A.; and Nowozin, S. 2018. Whichtraining methods for GANs do actually converge? arXivpreprint arXiv:1801.04406 .Miyato, T.; Kataoka, T.; Koyama, M.; and Yoshida, Y.2018a. Spectral normalization for generative adversarial net-works. arXiv preprint arXiv:1802.05957 .Miyato, T.; Maeda, S.-i.; Koyama, M.; and Ishii, S. 2018b.Virtual adversarial training: a regularization method for su-pervised and semi-supervised learning.
IEEE transactionson pattern analysis and machine intelligence
Advances in neural information processingsystems , 271–279.Oberman, A. M.; and Calder, J. 2018. Lipschitz regular-ized deep neural networks converge and generalize. arXivpreprint arXiv:1808.09540 .Petzka, H.; Fischer, A.; and Lukovnicov, D. 2017. Onthe regularization of wasserstein gans. arXiv preprintarXiv:1709.08894 .Qi, G.-J. 2020. Loss-sensitive generative adversarial net-works on lipschitz densities.
International Journal of Com-puter Vision arXiv preprint arXiv:1511.06434 .Shen, Y.; Gu, J.; Tang, X.; and Zhou, B. 2020. Interpretingthe latent space of gans for semantic face editing. In
Pro-ceedings of the IEEE/CVF Conference on Computer Visionand Pattern Recognition , 9243–9252.Song, Y.; Shu, R.; Kushman, N.; and Ermon, S. 2018. Con-structing unrestricted adversarial examples with generativemodels. In
Advances in Neural Information Processing Sys-tems , 8312–8323.Szegedy, C.; Zaremba, W.; Sutskever, I.; Bruna, J.; Erhan,D.; Goodfellow, I.; and Fergus, R. 2013. Intriguing prop-erties of neural networks. arXiv preprint arXiv:1312.6199 .Tao, R.; Li, Z.; Tao, R.; and Li, B. 2019. ResAttr-GAN: Un-paired Deep Residual Attributes Learning for Multi-DomainFace Image Translation.
IEEE Access
7: 132594–132608.Terj´ek, D. 2019. Adversarial Lipschitz Regularization. In
International Conference on Learning Representations .Thanh-Tung, H.; Tran, T.; and Venkatesh, S. 2019. Improv-ing generalization and stability of generative adversarial net-works. arXiv preprint arXiv:1902.03984 .iao, C.; Li, B.; Zhu, J.-Y.; He, W.; Liu, M.; and Song, D.2018. Generating adversarial examples with adversarial net-works. arXiv preprint arXiv:1801.02610 .Yeh, R. A.; Chen, C.; Yian Lim, T.; Schwing, A. G.;Hasegawa-Johnson, M.; and Do, M. N. 2017. Semantic im-age inpainting with deep generative models. In
Proceed-ings of the IEEE conference on computer vision and patternrecognition , 5485–5493.Yoshida, Y.; and Miyato, T. 2017. Spectral norm regular-ization for improving the generalizability of deep learning. arXiv preprint arXiv:1705.10941 .Yu, J.; Lin, Z.; Yang, J.; Shen, X.; Lu, X.; and Huang, T. S.2018. Generative image inpainting with contextual atten-tion. In
Proceedings of the IEEE conference on computervision and pattern recognition , 5505–5514.Zhou, Z.; Liang, J.; Song, Y.; Yu, L.; Wang, H.; Zhang, W.;Yu, Y.; and Zhang, Z. 2019a. Lipschitz generative adversar-ial nets. arXiv preprint arXiv:1902.05687 .Zhou, Z.; Shen, J.; Song, Y.; Zhang, W.; and Yu, Y. 2019b.Towards Efficient and Unbiased Implementation of Lips-chitz Continuity in GANs. arXiv preprint arXiv:1904.01184arXiv preprint arXiv:1904.01184