Generative Creativity: Adversarial Learning for Bionic Design
GGenerative Creativity:Adversarial Learning for Bionic Design
Simiao Yu , Hao Dong , Pan Wang , Chao Wu , and Yike Guo Imperial College London Zhejiang University
Abstract.
Bionic design refers to an approach of generative creativity in which atarget object (e.g. a floor lamp) is designed to contain features of biological sourceobjects (e.g. flowers), resulting in creative biologically-inspired design. In this work,we attempt to model the process of shape-oriented bionic design as follows: given aninput image of a design target object, the model generates images that 1) maintainshape features of the input design target image, 2) contain shape features of imagesfrom the specified biological source domain, 3) are plausible and diverse. We proposeDesignGAN, a novel unsupervised deep generative approach to realising bionic design.Specifically, we employ a conditional Generative Adversarial Networks architecturewith several designated losses (an adversarial loss, a regression loss, a cycle lossand a latent loss) that respectively constrict our model to meet the correspondingaforementioned requirements of bionic design modelling. We perform qualitative andquantitative experiments to evaluate our method, and demonstrate that our proposedapproach successfully generates creative images of bionic design.
Generative creativity refers to the generation process of new and creativeobjects composing features of existing domains. In computer vision, achievinggenerative creativity is a long-term goal, and there exist works that involvegenerative creativity. For example, image style transfer [14, 22, 15] can be seenas a generative creativity process in which the creative images are generated by a r X i v : . [ c s . C V ] M a y Simiao Yu, Hao Dong, Pan Wang, Chao Wu, and Yike Guo composing the features of existing content images and style images in a novelmanner.In this paper, we attempt to automate the process of bionic design [21, 52] byusing deep generative networks. Bionic design refers to a method of productdesign, in which a biologically-inspired object is created by combining thefeatures of a target design object with those of biological source objects. In thiswork, we mainly focus on shape-oriented bionic design, which is the crucial stepin studying the general bionic design problem. More specifically, given an inputimage of the design target, we aim to generate images that 1) maintain theshape features of the input image, 2) contain the shape features of images fromthe biological source domain, 3) remain plausible and diverse. Fig. 1 illustratesexamples of bionic design results generated by our proposed model. Essentially,bionic design is the ideal task to demonstrate generative creativity, becausethis process can be seen as composing the features of design target imagesand biological source images into novel and creative images that never existedbefore.Automating the aforementioned process of bionic design is a challenging taskdue to the following reasons. First, the task is of unsupervised learning, sincethe nature of creative design implies that there is no or very few availableimages of biologically-inspired design. In our case, we only have unpaired dataof design target images and biological source images. Second, there shouldbe multiple ways of integrating features of biological source images into thegiven design target image. In other words, bionic design is a one-to-manygeneration process, and the learned generative model should be able to achievethis variation. Third, the generated biologically-inspired design should preservekey features of input design target image and biological source images, which enerative Creativity: Adversarial Learning for Bionic Design 3 requires the model to be able to select and merge the features of differentsources.We propose DesignGAN, a novel unsupervised deep generative approach forbionic design. Our method is based on the architecture of conditional generativeadversarial networks (cGAN) [17, 39], with various enhancements designed toresolve the challenges mentioned above. First, the generator takes as input bothan image and a latent variable sampled from a prior Gaussian distribution,which enables the model to generate diverse output images. This is implementedby the introduction of an encoder and a latent loss. Second, our approachemploys both cycle loss [62, 30, 56] and regression loss to help maintain thekey features of the design target. Last, an adversarial loss is used to integratethe features of biological source images into the input image.We conduct both qualitative and quantitative experiments on the ”Quick,Draw!” dataset [19], and show that our proposed model is capable of generatingplausible and diverse biologically-inspired design images. Fig. 1 (c) presentsexamples of 3D product modelling designed by a human designer who is inspiredby the generated creative images.
Deep generative networks.
Several deep neural network architectures forimage generation have been proposed recently, such as generative adversar-ial networks (GAN) [17], variational autoencoders (VAE) [32, 47, 18], andautoregressive models [42, 43]. Our proposed approach is based on the GANarchitecture that learns to approximate the data distribution implicitly, bytraining a generator and a discriminator in a competing manner. The generator
Simiao Yu, Hao Dong, Pan Wang, Chao Wu, and Yike Guo
Design target image Biological source images Generated biologically-inspired images ++++++ !!!!!! . . .. . .. . .. . .. . .. . . (a) (b) (c) !! Fig. 1.
Given a design target image and biological source images, our proposed model generatesvaried biologically-inspired images. (a) Wine bottle + pear, generates pear-like bottles. (b) Teapot +whale, generates whale-shaped teapots. (c) Examples of 3D product modelling designed by a humandesigner, inspired by the generated creative images from our proposed model ( top : a pear-like bottle; bottom : a whale-shaped teapot). of the original GAN takes as input a noise vector and can be further condi-tioned by taking as input other conditional information (such as labels [8],texts [46, 58] and images), which forms the conditional GAN architecture(cGAN) [39]. There are also numerous follow-up works proposed to enhancethe image generation performance and training stability of GAN, in terms ofnew architectures [45, 12, 40, 41], objective functions [60, 3, 38] and trainingprocedure [9, 48, 24, 29].
Image-to-image generation.
When conditioned on images, deep genera-tive networks learn to solve image-to-image generation tasks, such as superresolution [33], user-controlled image editing [61, 6], image inpainting [44],colorization [59, 49], etc. Many of these tasks can be considered as a domaintranslation problem where the goal is to find a mapping function between sourcedomain and target domain. This problem can be of both supervised learn- enerative Creativity: Adversarial Learning for Bionic Design 5 ing and unsupervised learning settings. In the supervised domain translationproblem (e.g. [27, 57]), paired samples (sampled from the joint distributionof data from two domains) are observed. In the unsupervised counterpart(e.g. [36, 62, 30, 56, 35, 51, 53, 5]), only unpaired samples (sampled from themarginal distribution of data from each domain) are available. Our bionic de-sign problem can be seen as a related task to the unsupervised image-to-imagetranslation, as images from the design target domain and biological sourcedomain are unpaired, and there is no existing samples of biologically-inspiredimages. However, a significant difference is that the generating function to belearned should be able to merge the features of images from both domainsand generate biologically-inspired images that are of a third “intermediate”domain, rather than finding a mapping function between the two domains.This is detailed in the next section.Image-to-image translation is often multi-modal: an image from source domaincould be translated to multiple reasonable results (i.e. one-to-many mappings).Previous works attempt to learn this multi-modality only for supervised do-main transfer problems [16, 7, 4, 55]. By contrast, our proposed approachto modelling bionic design is capable of generating diverse outputs given asingle input image, in the unsupervised learning setting. Two contemporaryworks (AugCGAN [2] and MUNIT [25]) also attempt to model the multi-modality for unsupervised image-to-image translation by making extensions toCycleGAN [62] and UNIT [35] respectively.
Generative creativity.
Deep generative networks for image-to-image genera-tion have enabled the development of various creative applications in computervision. Here “creative” means the generated results should be a novel com-bination of existing features (e.g. colours, textures, shapes, etc.) and did not
Simiao Yu, Hao Dong, Pan Wang, Chao Wu, and Yike Guo exist in the training dataset (i.e. rather than generating a similar samplefrom the real data distribution). For instance, neural style transfer mod-els [15, 28, 34, 54, 62, 13, 23] generate creative images by combining thesemantic content of a given image with the style of another artwork image.Many image-to-image translation tasks involve generative creativity, such aspainting-to-photo translation and object transfiguration [62]. Another work [11]synthesises novel images based on a given image and a natural language de-scription, such that the generated images correspond to the description whilemaintaining other features of the given image. A recent work [50] createsinnovative designs for fashion. Another recent work [37] achieves generativecreativity by seamlessly copying and pasting an object into a painting. Theseworks mainly focus on colour and texture generation or manipulation (exceptthe work [50] that also involves the generation of shapes of abstract patterns),while the problem of bionic design in this work is mainly (semantically-related)shape-oriented, which is a more challenging task for generative creativity.
The problem of bionic design can be formulated as follows. Given a design targetdomain D containing samples { d k } Mk =1 ∈ D (e.g. floor lamps) and a biologicalsource domain B containing samples { b k } Nk =1 ∈ B (e.g. flowers), we have thecorresponding latent spaces of D and B (respectively Z d and Z b ) that containthe representations of each domain. We denote the data distribution of D and B as p ( d ) and p ( b ). We then make two key assumptions of the bionic designproblem: 1) there exists an “intermediate” domain I containing the generatedobjects of biologically-inspired design { ˆ i k } Ok =1 ∈ I , and 2) the corresponding enerative Creativity: Adversarial Learning for Bionic Design 7 latent space of I (denoted as Z ) contains the merged representations of thosefrom Z d and Z b , as illustrated in Fig. 2. D BIZZ d Z b Fig. 2.
Our assumption of the bionic design problem.
Based on these two assumptions, the objective of bionic design is to learn agenerating function G DB : D × Z → I , such that the generative distributionmatches the distribution of I (denoted as p ( i )). Since in our case we do not haveany existing samples from I , it is impossible to explicitly learn such generativedistribution. Nonetheless, we could still learn it in an implicit fashion via realdata distributions p ( d ) and p ( b ), and the careful design of the model architecture,as discussed in the next section. This is where generative creativity comesfrom. Also note that G DB takes as input the latent variable z ∈ Z sampledfrom the distribution p ( z ), the requirement of variations for bionic design issatisfied directly: multiple samples based on a single d can then be generatedby sampling different z from p ( z ). At first glance, the shape-oriented bionic design problem can be tackled byemploying the CycleGAN architecture [62, 30, 56]. However, we reveal thesignificant limitations of CycleGAN for this problem, which motivates our
Simiao Yu, Hao Dong, Pan Wang, Chao Wu, and Yike Guo development of a new architecture. In this section, we start with a briefdiscussion on the applicability and limitations of CycleGAN model and itsextensions. We then describe in detail our proposed DesignGAN model andthe corresponding objective functions. d G DB G BD D B b d G DB b D D b L BDc L BDd d d G DB G BD D B b d G DB G BD b D D b L DBc L BDc L DBa L BDd d ˆ z b d z b ˆ z d G DB G BD D B E B E D b d ˆ z b ˆ z d G DB G BD E B E D b D D z d b L DBc L BDc L DBl L DBa L BDd L BDl z z L DBc L DBa G BD zz d (a)(a) (b) (c) ˆ i DB ˆ i DB ˆ i DB ˆ i BD ˆ i BD ˆ i BD Fig. 3.
Schema of CycleGAN model and its extensions. We explicitly present the separate componentsof the models to illustrate the dual learning process and the loss functions more clearly. (a)CycleGAN [62, 30, 56]. (b) CycleGAN+N. (c) CycleGan+2E.
Our initial choice is to employ the
CycleGAN architecture directly, wheretwo image-based cGAN models are cascaded and trained jointly (Fig. 3 (a)).We use the cycle loss to maintain the features of the given design target imageand an adversarial loss to integrate the features of biological source images.Since the images only contain representations of shapes, the two losses will beforced to directly compete with each other, which makes it possible to generateimages from the “intermediate” domain that contains shape features of bothdomains. However, this model will only learn a deterministic mapping, whichwill not be able to generate diverse results. enerative Creativity: Adversarial Learning for Bionic Design 9
A straightforward way to make the system learn a one-to-many mapping isto inject noise as the input of the system (Fig. 3 (b)). The limitation of thisapproach, as also discussed in [2, 25], is that the cycle-consistence restrictionwould make the generator ignore the noise input. This is because each generatorwill be under conflicting constraints imposed by each cycle loss (respectively one-to-many and many-to-one mappings), which would eventually be degeneratedinto one-to-one mappings. We denote this model as
CycleGAN+N .We further propose a new architecture to solve the limitation of CycleGAN+Nby integrating two encoders E D , E B into the architecture (Fig. 3 (c)). Eachencoder takes as input a generated image and encodes it back to the correspond-ing latent space. The generated latent code is used to compute a latent loss tomatch the input noise vector, which enforces the generator to generate diverseresults. The system will never ignore the noise input because of this latent loss,thus resolves the problem of CycleGAN+N. However, the generated imagesof this system will heavily depend on the latent variable, without taking intoaccount the input image. More specifically, given an input image d and differentnoise vectors z b , diverse ˆ i DB should be generated by G DB because of the latentloss L DBl . The problem emerges when calculating the cycle loss L DBc . G BD is supposed to map all generated diverse images back to the original designtarget image d . Since d is encoded into a fixed ˆ z d by the encoding function E D , G BD would simply learn a one-to-one mapping from ˆ z d to d . In other words,the generators will tend to ignore the input images. We denote this model as CycleGAN+2E . To address the problem of CycleGAN+2E, we propose a new model, denoted as
DesignGAN , as illustrated in Fig. 4. Specifically, DesignGAN is comprised offive functions parametrized by deep neural networks (two generators G DB and G BD , two discriminators D B and D D , and one encoder E ) and four designatedloss functions that are discussed in detail as follows. Our model is end-to-end,with all component networks trained jointly. d ˆ i DB z G DB G BD D B E b D D b ˆ i BD L DBc L BDc L DBl L DBa L DBr L BDr L BDa L BDl ˆ z ˜ d ˜ bz G DB G BD D B ED D ˆ z d Fig. 4.
Schema of our proposed DesignGAN model that has two key enhancements on CycleGAN+2E.First, our model employs a single encoder E as the encoding function E : B × D → Z to learn thevariation of the bionic design problem. Second, we further propose to use the discriminators D B and D D simultaneously as forward regression functions to preserve the features of the input imagedomain, imposed by the regression loss L DBr and L BDr , without competing with the generators.
Adversarial loss.
We employ two sources of adversarial loss L DBa ( G DB , D B )and L BDa ( G BD , D D ) that respectively enforce the outputs of G DB and G BD to match the empirical data distribution p ( b ) and p ( d ), as an approach tointegrate corresponding features to the generated images. L a ( G DB , G BD , D B , D D ) = L DBa ( G DB , D B ) + L BDa ( G BD , D D ) L DBa ( G DB , D B ) = E b ∼ p ( b ) [log D B ( b )] + E d ∼ p ( d ) ,z ∼ p ( z ) [log(1 − D B ( G DB ( d, z )))] L BDa ( G BD , D D ) = E d ∼ p ( d ) [log D D ( d )] + E b ∼ p ( b ) ,z ∼ p ( z ) [log(1 − D D ( G BD ( b, z )))] (1) where D B and D D are discriminators that distinguish between generated andreal images from B and D . enerative Creativity: Adversarial Learning for Bionic Design 11 Cycle loss.
The problem of bionic design requires the generated images tomaintain the features of the input design target. In other words, the generatedimage should still be recognised as in the class of the design target. For theshape-oriented bionic design problem, it simply implies that the generatedimages should resemble the input images to a large extent. After all, it wouldbe unreasonable to generate biologically-inspired images that in turn share norelationship to the input design target image. We apply cycle loss L DBc and L BDc to constrict the generators G DB and G BD to retain the shape representationsof the input images: L c ( G DB , G BD ) = L DBc ( G DB , G BD ) + L BDc ( G BD , G DB ) L DBc ( G DB , G BD ) = E d ∼ p ( d ) ,z ∼ p ( z ) [ k G BD ( G DB ( d, z ) , E ( G DB ( d, z ) , d )) − d k ] L BDc ( G BD , G DB ) = E b ∼ p ( b ) ,z ∼ p ( z ) [ k G DB ( G BD ( b, z ) , E ( b, G BD ( b, z ))) − b k ] (2) where we employ L2 norm in the loss. The inclusion of cycle loss makes ourmodel optimised in a dual-learning fashion [62, 30, 56]: we introduce an auxiliarygenerator G BD and train all the generators and discriminators jointly. Aftertraining, only G DB will be used for bionic design purpose. Regression loss.
The cycle loss enforces the generated images to maintainthe shape features of the input image only. Another way of maintaining thedesign target features is to simultaneously force the generated images to containkey features of the design target domain, which directly makes the generatedimages recognised as the class of the design target. We therefore introduce theregression loss L DBr and L BDr imposed by the discriminator D D and D B . L DBr and L BDr respectively constricts G DB and G BD to maintain representationsfrom the domain of input images. Note that in such a situation D D and D B areemployed as a regression function only, without competing with the generatorsas the adversarial loss does. This is why in Fig. 4 there is only one input to D D and D B when referring to L r . The regression loss if one of the major extensionsof the CycleGAN architecture. L r ( G DB , G BD ) = L DBr ( G DB ) + L BDr ( G BD ) L DBr ( G DB ) = E d ∼ p ( d ) ,z ∼ p ( z ) [log(1 − D D ( G DB ( d, z )))] L BDr ( G BD ) = E b ∼ p ( b ) ,z ∼ p ( z ) [log(1 − D B ( G BD ( b, z )))] (3) Latent loss.
We employ a unified encoder E and a latent loss to model thevariation of the bionic design problem: L l ( G DB , G BD , E ) = L DBl ( G DB , E ) + L BDl ( G BD , E ) L DBl ( G DB , E ) = E d ∼ p ( d ) ,z ∼ p ( z ) [ k E ( G DB ( d, z ) , d ) − z k ] L BDl ( G BD , E ) = E b ∼ p ( b ) ,z ∼ p ( z ) [ k E ( b, G BD ( b, z )) − z k ] (4) Unlike the encoders of CycleGAN+2E that take as input one image, the encoder E of DesignGAN encodes a pair of images from each domain (either ( ˆ i DB , d )or ( b, ˆ i BD )) into the latent space Z of domain I , which acts as an encodingfunction E : B × D → Z and corresponds to our assumption of the bionicdesign problem. The latent loss is computed by the L1 norm distance betweenthe generated latent variable ˆ z and the input noise vector z , which forces themodel to generate diverse output images. More importantly, this choice ofencoder ensures that neither the generated images nor the generative latentvariable will be ignored under the cycle consistent constraints. This is anothermajor extension to the CycleGAN architecture that addresses the limitation ofboth CycleGAN+N and CycleGAN+2E model. Full objective.
The full objective function of our model is: min { G DB ,G BD ,E } max { D B ,D D } L ( G DB , G BD , E, D B , D D ) = λ a L a ( G DB , G BD , D B , D D )+ λ c L c ( G DB , G BD ) + λ r L r ( G DB , G BD ) + λ l L l ( G DB , G BD , E ) (5) where we employ λ a , λ c , λ r and λ l to control the strength of individual losscomponents. enerative Creativity: Adversarial Learning for Bionic Design 13 Methods.
All models discussed in Section 4, including CycleGAN, Cycle-GAN+N, CycleGAN+2E and DesignGAN, are evaluated. We employ the samenetwork architecture in all models for a fair comparison.
Dataset.
We evaluate our models on ”Quick, Draw!” dataset [19] that containsmillions of simple grayscale drawings of size 28 ×
28 across 345 common objects.It is an ideal dataset for the shape-oriented bionic design problem. We selectseveral pairs of domains of design targets and biological sources as the variedbionic design problems. We randomly choose 4000 images from each domain ofthe domain pairs for training.
Network architecture.
For the generator networks, we adopt the encoder-decoder architecture. The encoder contains three convolutional layers andthe decoder has two transposed convolutional layers. Six residual units [20]are applied after the encoder. The latent vector is spatially replicated andconcatenated to the input image, where applicable. The discriminator networkscontain four convolutional layers. For the encoder network, the two input imagesare concatenated and encoded by three convolutions and six residual units.We employ ReLU activation in the generators and encoder, and leaky-ReLUactivation in the discriminators. Batch normalisation [26] is implemented in allnetworks.
Training details.
The networks are trained for 120 epochs using Adam opti-miser [31] with a learning rate of 0.0001 and a batch size of 64. The learningrate is decayed to zero linearly over the last half number of epochs. Due tothe distinct complexity of images from different domains, the values of λ a , λ c , λ r and λ l and dimension of latent variable z are set independently for each of the domain pairs. We use the objective functions of Least SquaresGAN [38] to stabilise the learning process. The discriminator is updated using ahistory of generated images, as proposed in [51], in order to alleviate the modeloscillation problem [62]. We apply random horizontal flipping and random ± × Qualitative results.
Fig. 5 illustrates the qualitative comparison results ofour investigated and proposed models. We maintain the same value of thelatent variable for the corresponding three generated images for each groupof generation, where possible. Specifically, CycleGAN in some cases is ableto generate images of bionic design, while in other cases it fails to maintainfeatures of the input design target image (e.g. Fig. 5 (d)). Also, since it isa deterministic model, no variation is produced. Similar to CycleGAN, inmost cases CycleGAN+N only generates a single result given one input image,which indicates that the input noise vector is completely ignored. AlthoughCycleGAN+2E can generate diverse results, they are either of low-quality (e.g.Fig. 5 (b) (c)), or failed to maintain any features of the input design targetimage. We observe that the input latent variable dominates and the designtarget image is ignored by CycleGAN+2E, which corresponds to our analysisin Section 4.1. By contrast, DesignGAN is capable of generating plausible anddiverse biologically-inspired images that successfully maintain representationsof both input design target image and biological source images.
Quantitative results.
How to quantitatively evaluate the performance ofgenerative models for creative tasks remains a challenging problem. In thiswork, we leverage human judgement to evaluate our investigated and proposed enerative Creativity: Adversarial Learning for Bionic Design 15 (a)(c)(e) (b)(d)(f) CycleGANCycleGAN+NCycleGAN+2EDesignGANCycleGANCycleGAN+NCycleGAN+2EDesignGANCycleGANCycleGAN+NCycleGAN+2EDesignGAN
Fig. 5.
Qualitative results of our investigated and proposed models for bionic design. (a) Hat +rabbit. (b) Floor Lamp + flower. (c) Vase + pineapple. (d) Suitcase + onion. (e) Wine glass +flower. (f) Hat + octopus. models for bionic design. Despite subjective factors being involved, it is themost dependable measurement of creativity and plausibility of generated results.More specifically, we use 8 domain pairs of design targets and biological sourcesshown in this paper. For each pair of domains, we select 10 images of designtarget as the input to our models. We then generate 3 output biologically-inspired images for every input image. There are 25 subjects recruited, shownall the input and generated images, and required to rank the models (from 1 to4, 1 for the best) based on whether the generated images 1) maintain the keyfeatures of input design target image, 2) contain the key features of biologicalsource domain, 3) are diverse, and 4) are creative and plausible.We then average all the ranking scores and calculate the final overall scoresfor each model, which is presented in Table 1. All models are capable ofintegrating the features of biological source images to the generated images,but our proposed DesignGAN performs best in terms of maintaining the features of design target image. Although DesignGAN ranks second in diversity(CycleGAN+2E ranks first, but many of its generated images are either low-quality or failed to maintain any design target features), it gains the highestscore of creativity and plausibility. Overall, DesignGAN performs best whenall aspects of judging criteria are considered.
Table 1.
Human evaluation results of our investigated and proposed models for bionic design.CycleGAN CycleGAN+N CycleGAN+2E DesignGANMaintain design target features 2.125 2.715 3.620 1.540Integrate biological source features 2.290 2.690 2.780 2.240Diversity 4.000 2.820 1.580 1.600Creativity and plausibility 2.455 2.675 3.545 1.325Overall 2.718 2.725 2.881 1.676
Comparison of regression loss and cycle loss.
We study the effect ofregression loss and cycle loss by setting varied values to λ r and λ c and generatingthe corresponding images, which can be seen in Fig. 6. Both regression loss andcycle loss are able to improve the generated images by forcing them to containthe features of the input image (e.g. see the area pointed by the red arrow).However, if the cycle loss is applied alone, it is only when setting the weight toa relatively large value that the generated images will resemble the input image.In this case, the results tend to lose the details of features of biological sourcedomain. By contrast, applying the regression loss makes the model generatebetter images, though the weight of regression loss λ r needs to be set to areasonable value, in order to prevent the generated images from being exactly enerative Creativity: Adversarial Learning for Bionic Design 17 identical to the input image (i.e. not able to integrate the representations ofthe biological source domain). r = 0 c = 10 c = 10 r = 0 . r = 1 c = 10 c = 20 r = 0 r = 0 c = 50 Input
Fig. 6.
Comparison results of regression loss and cycle loss using an example of Suitcase + onion.
Latent variable interpolation
Fig. 7 shows the generated biologically-inspired design images by linearly interpolating the input latent variable z . Thesmooth semantic transitions of generated results verify that our model learnsa smooth latent manifold as well as the disentangled representations for thebionic design problem. Design target image Generated biologically-inspired images !!!
Fig. 7.
Generated biologically-inspired design images by interpolating the input latent variable.( top ) Suitcase + onion. ( middle ) Floor lamp + flower. ( bottom ) Hat + octopus.
In this paper, we proposed DesignGAN as a novel unsupervised deep generativenetwork with the capacity of shape-oriented bionic design. We presented asystemic design path of this architecture. The research shows how the CycleGAN architecture can be further evolved into an adversarial learning frameworkwith strong generative creativity. We conducted qualitative and quantitativeexperiments on the methods of our design path and demonstrated that ourproposed model achieves superior results of plausible and diverse biologically-inspired design images. The shape-oriented bionic design problem we addressedcan be regarded as an essential prerequisite for tackling more comprehensiveand complicated bionic design problems that may require the manipulation ofcolours, textures, etc. Another direction of future work is to model the bionicdesign problem in a goal-oriented manner as human designers would (ratherthan random generation) where advanced technologies such as reinforcementlearning can be applied. ibliography [1] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin,S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga,S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden,M. Wicke, Y. Yu, and X. Zheng. TensorFlow: a system for large-scalemachine learning. In
OSDI , 2016.[2] A. Almahairi, S. Rajeswar, A. Sordoni, P. Bachman, and A. Courville.Augmented CycleGAN: learning many-to-many mappings from unpaireddata. arXiv preprint arXiv:1802.10151 , 2018.[3] M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarialnetworks. In
ICML , 2017.[4] A. Bansal, Y. Sheikh, and D. Ramanan. PixelNN: example-based imagesynthesis. In
ICLR , 2018.[5] K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan.Unsupervised pixel-level domain adaptation with generative adversarialnetworks. In
CVPR , 2017.[6] A. Brock, T. Lim, J. M. Ritchie, and N. Weston. Neural photo editingwith introspective adversarial networks. In
ICLR , 2017.[7] Q. Chen and V. Koltun. Photographic image synthesis with cascadedrefinement networks. In
ICCV , 2017.[8] X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel.InfoGAN: interpretable representation learning by information maximizinggenerative adversarial nets. In
NIPS , 2016.[9] E. Denton, S. Chintala, A. Szlam, and R. Fergus. Deep generative imagemodels using a laplacian pyramid of adversarial networks. In
NIPS , 2015. [10] H. Dong, A. Supratak, L. Mai, F. Liu, A. Oehmichen, S. Yu, and Y. Guo.TensorLayer: a versatile library for efficient deep learning development. In
ACM-MM , 2017.[11] H. Dong, S. Yu, C. Wu, and Y. Guo. Semantic image synthesis viaadversarial learning. In
ICCV , 2017.[12] A. Dosovitskiy and T. Brox. Generating images with perceptual similaritymetrics based on deep networks. In
NIPS , 2016.[13] V. Dumoulin, J. Shlens, and M. Kudlur. A learned representation forartistic style. In
ICLR , 2017.[14] A. A. Efros and W. T. Freeman. Image quilting for texture synthesis andtransfer. In
SIGGRAPH , 2001.[15] L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer usingconvolutional neural networks. In
CVPR , 2016.[16] A. Ghosh, V. Kulharia, V. Namboodiri, P. H. S. Torr, and P. K. Doka-nia. Multi-agent diverse generative adversarial networks. arXiv preprintarXiv:1704.02906 , 2017.[17] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In
NIPS , 2014.[18] K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, and D. Wierstra.DRAW: a recurrent neural network for image generation. In
ICML , 2015.[19] D. Ha and D. Eck. A neural representation of sketch drawings. In
ICLR ,2018.[20] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for imagerecognition. In
CVPR , 2016.[21] M. Helms, S. S. Vattam, and A. K. Goel. Biologically inspired design:process and products.
Design Studies , 30(5):606 – 622, 2009. enerative Creativity: Adversarial Learning for Bionic Design 21 [22] A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin.Image analogies. In
SIGGRAPH , 2001.[23] X. Huang and S. Belongie. Arbitrary style transfer in real-time withadaptive instance normalization. In
ICCV , 2017.[24] X. Huang, Y. Li, O. Poursaeed, J. Hopcroft, and S. Belongie. Stackedgenerative adversarial networks. In
CVPR , 2017.[25] X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz. Multimodal unsupervisedimage-to-image translation. arXiv preprint arXiv:1804.04732 , 2018.[26] S. Ioffe and C. Szegedy. Batch normalization: accelerating deep networktraining by reducing internal covariate shift. In
ICML , 2015.[27] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros. Image-to-image translationwith conditional adversarial networks. In
CVPR , 2017.[28] J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time styletransfer and super-resolution. In
ECCV , 2016.[29] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing ofGANs for improved quality, stability, and variation. In
ICLR , 2018.[30] T. Kim, M. Cha, H. Kim, J. K. Lee, and J. Kim. Learning to discovercross-domain relations with generative adversarial networks. In
ICML ,2017.[31] D. Kingma and J. Ba. Adam: a method for stochastic optimization. In
ICLR , 2014.[32] D. P. Kingma and M. Welling. Auto-encoding variational bayes. In
ICLR ,2014.[33] C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Aitken, A. Tejani, J. Totz,Z. Wang, and W. Shi. Photo-realistic single image super-resolution usinga generative adversarial network. In
CVPR , 2017. [34] C. Li and M. Wand. Precomputed real-time texture synthesis with Marko-vian generative adversarial networks. In
ECCV , 2016.[35] M.-Y. Liu, T. Breuel, and J. Kautz. Unsupervised image-to-image trans-lation networks. In
NIPS , 2017.[36] M.-Y. Liu and O. Tuzel. Coupled generative adversarial networks. In
NIPS , 2016.[37] F. Luan, S. Paris, E. Shechtman, and K. Bala. Deep painterly harmoniza-tion. arXiv preprint arXiv:1804.03189 , 2018.[38] X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. P. Smolley. Leastsquares generative adversarial networks. In
ICCV , 2017.[39] M. Mirza and S. Osindero. Conditional generative adversarial nets. arXivpreprint arXiv:1411.1784 , 2014.[40] A. Nguyen, J. Clune, Y. Bengio, A. Dosovitskiy, and J. Yosinski. Plug& play generative networks: conditional iterative generation of images inlatent space. In
CVPR , 2017.[41] A. Odena, C. Olah, and J. Shlens. Conditional image synthesis withauxiliary classifier GANs. In
ICML , 2017.[42] A. V. d. Oord, N. Kalchbrenner, and K. Kavukcuoglu. Pixel recurrentneural networks. In
ICML , 2016.[43] A. V. d. Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves, andK. Kavukcuoglu. Conditional image generation with PixelCNN decoders.In
NIPS , 2016.[44] D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros.Context encoders: feature learning by inpainting.
CVPR , 2016.[45] A. Radford, L. Metz, and S. Chintala. Unsupervised representation learningwith deep convolutional generative Adversarial Networks. In
ICLR , 2016. enerative Creativity: Adversarial Learning for Bionic Design 23 [46] S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee. Gener-ative adversarial text to image synthesis. In
ICML , 2016.[47] D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagationand approximate inference in deep generative models. In
ICML , 2014.[48] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, andX. Chen. Improved techniques for training GANs. In
NIPS , 2016.[49] P. Sangkloy, J. Lu, C. Fang, F. Yu, and J. Hays. Scribbler: controllingdeep image synthesis with sketch and color. In
CVPR , 2017.[50] O. Sbai, M. Elhoseiny, A. Bordes, Y. LeCun, and C. Couprie. DesIGN: de-sign inspiration from generative networks. arXiv preprint arXiv:1804.00921 ,2018.[51] A. Shrivastava, T. Pfister, O. Tuzel, J. Susskind, W. Wang, and R. Webb.Learning from simulated and unsupervised images through adversarialtraining. In
CVPR , 2017.[52] L. Shu, K. Ueda, I. Chiu, and H. Cheong. Biologically inspired design.
CIRP Annals , 60(2):673 – 693, 2011.[53] Y. Taigman, A. Polyak, and L. Wolf. Unsupervised cross-domain imagegeneration. In
ICLR , 2017.[54] D. Ulyanov, V. Lebedev, A. Vedaldi, and V. Lempitsky. Texture networks:feed-forward synthesis of textures and stylized images. In
ICML , 2016.[55] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catan-zaro. High-resolution image synthesis and semantic manipulation withconditional GANs. In
CVPR , 2018.[56] Z. Yi, H. Zhang, P. Tan, and M. Gong. DualGAN: unsupervised duallearning for image-to-image translation. In
ICCV , 2017.[57] D. Yoo, N. Kim, S. Park, A. S. Paek, and I. S. Kweon. Pixel-level domaintransfer. In
ECCV , 2016. [58] H. Zhang, T. Xu, H. Li, S. Zhang, X. Huang, X. Wang, and D. Metaxas.StackGAN: text to photo-realistic image synthesis with stacked generativeadversarial networks. In
ICCV , 2017.[59] R. Zhang, P. Isola, and A. A. Efros. Colorful image colorization. In
ECCV ,2016.[60] J. Zhao, M. Mathieu, and Y. LeCun. Energy-based generative adversarialnetwork. In
ICLR , 2017.[61] J.-Y. Zhu, P. Kr¨ahenb¨uhl, E. Shechtman, and A. A. Efros. Generativevisual manipulation on the natural image manifold. In
ECCV , 2016.[62] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-imagetranslation using cycle-consistent adversarial networks. In