Recovery of underdrawings and ghost-paintings via style transfer by deep convolutional neural networks: A digital tool for art scholars
Anthony Bourached, George Cann, Ryan-Rhys Griffiths, David G. Stork
RRecovery of underdrawings and ghost-paintings viastyle transfer by deep convolutional neural networks:
A digital tool for art scholars
Anthony Bourached, a George H. Cann, b Ryan-Rhys Griffiths, c and David G. Stork da Computer Science Department, University College London, London, UK b Department of Space and Climate Physics, University College London, London, UK c Department of Physics, University of Cambridge, Cambridge, UK d Portola Valley, CA 94028 USA
ABSTRACT
We describe the application of convolutional neural network style transfer to the problem of improved visualiza-tion of underdrawings and ghost-paintings in fine art oil paintings. Such underdrawings and hidden paintingsare typically revealed by x-ray or infrared techniques which yield images that are grayscale, and thus devoid ofcolor and full style information. Past methods for inferring color in underdrawings have been based on physicalx-ray fluorescence spectral imaging of pigments in ghost-paintings and are thus expensive, time consuming, andrequire equipment not available in most conservation studios. Our algorithmic methods do not need such expen-sive physical imaging devices. Our proof-of-concept system, applied to works by Pablo Picasso and Leonardo,reveal colors and designs that respect the natural segmentation in the ghost-painting. We believe the computedimages provide insight into the artist and associated oeuvre not available by other means. Our results stronglysuggest that future applications based on larger corpora of paintings for training will display color schemes anddesigns that even more closely resemble works of the artist. For these reasons refinements to our methods shouldfind wide use in art conservation, connoisseurship, and art analysis.
Keywords: ghost-paintings, style transfer, deep neural network, computational art analysis, artificial intelli-gence, computer-assisted connoisseurship
1. INTRODUCTION AND BACKGROUND
Many paintings in the Western canon, particularly realist easel paintings from the Renaissance to the present,bear underdrawings and pentimenti (from Italian, “to repent”)—preliminary versions of the work created as theartist altered and developed the final design. The interpretation of such underdrawings is central in addressingnumerous problems in the history and interpretation of art, including inferring artists’ praxis as well as interpret-ing, attributing, and authenticating such works.
1, 2
For example, copies of some paintings have been exposed asforgeries through an analysis of underdrawings revealed through such x-ray imaging; after all, a forger generallydoes not have access to the underdrawings and so can duplicate only what is visible to the naked eye. X-ray and infrared imaging do not directly reveal colors of underdrawings.
In most studies, this lack ofcolor information does not pose a major impediment because (it is widely acknowledged) the artist generallyuses the same colors throughout the development of the work; the colors in the primary, visible artwork are likelyquite similar to the ones the artist used through its development. In such cases it is simply the geometric designthat is relevant. For instance, in a computational study of the compositional style of Piet Mondrian, x-ray imagesof his Neoplastic geometric paintings revealed the designs of prior designs, which the artist ultimately rejected.Such “near miss” designs, along with the final accepted designs, could be used to train statistical models ofMondrian’s compositional principles. In that study, x-ray images sufficed because color was of no concern.There are, however, some paintings in which the design of the underdrawings are not directly related to thatof the visible painting, and these pose a rather different challenge in art analysis. These are underdrawings in
Send correspondence to [email protected] . a r X i v : . [ c s . C V ] J a n hich the entire compositions and designs were painted over by a second unrelated design, which is the visibleartwork. Such hidden artworks or so-called “ghost-paintings” arise when artists re-used canvases, either becausethey were unsatisfied with their first paintings, or more commonly when they could not afford new canvases.In some cases the orientation of the hidden design is rotated 90 ◦ when the artist preferred landscape formatto portrait format—or vice versa—for the second painting, as in Picasso’s Crouching beggar . In some cases therotation is 180 ◦ with respect to the visible painting, as for instance Rembrandt’s An old man in military costume .Prominent paintings that have of such ghost-paintings include: • Kazimir Malevich’s
Black square (1915) • Rembrandt’s
An old man in military costume (1630–31) • Vincent van Gogh’s
Patch of grass (1887) • Pablo Picasso’s
The blue room (1901),
Mother and child by the sea (1902),
The crouching beggar (1902),
Barcelona rooftops (1903),
Old guitarist (1903–04), and
Woman ironing (1904) • Ren´e Magritte’s
The portrait (1935), and
The human condition (1935) • Edgar Degas’
Portrait of a woman (c. 1876–80) • Francisco Goya’s
Portrait of Do˜na Isabel de Porcel (1805) • Leonardo’s
The Virgin of the rocks (1495–1508) ∗ Indeed, as many as 20 out of 130 paintings paintings by van Gogh examined at the van Gogh Museum,Amsterdam, have at least partially completed ghost-paintings. There is of course scholarly and general interest in “lost” works, as they give a richer understanding of theartist and his or her oeuvre. Art scholars interpret such ghost-paintings for a number of reasons, such as to betterunderstand an artist’s career development and choice of subjects (both general and specific), and to learn what anartist did or did not wish to preserve in his oeuvre.
10, 11
It is clear that such tasks are best served by a high-qualitydigital image of the underdrawing, including features (such as color) not captured by x-radiography and infraredreflectography. To date, the principal non-destructive method for recovering colors in ghost-paintings is throughx-ray fluorescence spectroscopy, which reveals the elemental composition of pigments. Such measured elementalcompositions are then matched to pigment databases so as to infer the likely pigments and their proportions,which in turn indicates the colors in the ghost-painting.
This method requires expensive equipment notavailable in most conservation studios. Moreover, ultraviolet radiation has shallow penetration power, and maynot reveal underdrawings for purely physical reasons.This, then, is the overriding goal of our work: to compute a digital image of the ghost-painting as close tothe original—in color, form, and style—so the general public and art scholars can better study a hidden artwork,all without the need for such expensive physical sensing equipment.In brief, our approach is to transfer the style—specifically the color—from comparable artworks to thegrayscale underdrawing. Related color style transfer in fine art scholarship has been applied to the rejuve-nation of fine art tapestries, whose vegetable pigments are fugitive, and thus faded over centuries. In some casesthe cartoons or source paintings survive, where the oil pigments retain the reference colors. However, tapestryateliers frequently alter the designs of the works—adding or deleting figures, changing their poses, altering back-grounds, and so forth—and thus a simple overlay of the cartoon or painting design atop the image of a fadedtapestry will not lead to a coherent image. Color transfer with modest tolerance for spatial disparities canproduce acceptable “rejuvenated” tapestries, but a full solution will likely require methods based on deep neuralnetworks. ∗ There are two versions of this work: one in the Louvre and the other in the National Gallery London, which is shownin Fig. 3, below, and is the focus of our efforts.
2e begin in Sect. 2 with a brief overview of the problem of separating the design of the underdrawing from thex-ray or infrared reflectogram containing the mixtures of the underdrawing and the primary visible image. Wethen turn in Sect. 3 to the problem of mapping of style and colors (learned from representative artworks) to theghost-painting, an extension and improvement upon earlier efforts, and enhancing the design of a underdrawingby style transfer from an ensemble of drawings. We focus on this second stage, where we use deep neural networkmethods for style transfer.
18, 19
We present in Sect 4 our image results for two reconstructed ghost-paintingsfrom a single style image and one drawing and from an ensemble of artworks. We also discuss how such methodscan empower art scholars to better interpret these hidden works. We conclude in Sect. 5 with a summary of ourresults, current limitations and caveats, and future directions for developing this digital tool for the communityof art scholars.
2. IMAGE SEPARATION
An x-ray of a painting containing a ghost-painting reveals the visible and the ghost compositions overlapped.Thus the first processing stage is to isolate the hidden image from the primary, visible image. This can be a verychallenging computational task, even in simple cases. The leading algorithmic approach is signal separation or blind source separation , which was first developed for separating the sounds of separate sources from a singleor multiple audio recordings, as for instance isolating the sound of a single talker from recordings of a noisycocktail party.
21, 22
The analogous method for images to computationally split the value of each pixel betweentwo candidate images such that each component image is maximally internally correlated and spatially consistentyet maximally uncorrelated and independent from the other image. The optimization problem works best, atleast in theory, when the image-mixing process is simple, for instance that the value of each pixel is the simplesum of the two component images.Such a simple image mixing does not occur in the x-ray images of fine art paintings, however. Of course, anx-ray or infrared reflectogram image of a painting bearing a ghost-painting reveals a single image with the twodesigns overlapped. However the x-ray opacities of the two component layers of paint do not simply add normultiply. Instead, the composition is a nonlinear function that depends upon the particular pigments and bindermixtures (e.g., linseed oil). Moreover, the x-ray attenuation depends upon the thickness of any componentlayer, even if the reflected appearance of that layer does not. For instance a layer of chromium yellow pigmentof thickness 100 µm and one of thickness 300 µm will appear nearly identical in reflection, yet affect an x-rayimage rather differently.Blind source separation is especially difficult with x-ray images of paintings for three reasons: a) as mentioned,the image mixing is highly nonlinear, b) the spatial and chromatic statistics of art vary far more widely thancomparable statistics of photographs of natural scenes so spatial coherence is often low, and c) there are fewerart images available for statistical estimation than numerous easily available corpora of photographs. For thesereasons the image separation stage of our work involved hand-editing the underdrawing extracted from x-raysof the target artworks using the GIMP .
10 dodge and burn tool. Future efforts should lead to progress on thischallenging problem in art analysis.
3. NEURAL STYLE TRANSFER
Style transfer is the task of applying the low-level style (such as color, spatial, and contour statistics) from one ormultiple “style” images to the design of another image, the “content” image. This is an extremely well-studiedproblem for natural photographs, leading to qualitatively excellent results.
The metrics for such successof these algorithms are generally qualitative, however: the modified image should “appear as in the style of”the style images. For example, one can apply the style of an Impressionist painting to a personal photographyielding subjectively convincing results.Style transfer to artworks for scholarly interpretation is a far more difficult problem. Connoisseurs andart interpreters study the subtlest of details in brushstrokes, paint thickness, contours, colors, and more intheir analyses of artworks. A computed work that merely appears “in the style of” does not suffice for mostscholarly analyses beyond coarse, qualitative analyses. What is needed are systems whose resulting works arenearly indistinguishable in style from genuine works by the artist in question. Artifacts and spurious features3n any computed image must be kept to a minimum, and connoisseurs must learn to identify and account forcomputational artifacts so as to avoid misinterpreting works. Our methods, below, are but a first step in thischallenging task.Recently, deep neural networks have been applied to the task of style transfer.
18, 19
In broad overview, suchnetworks consist of alternating layers processing “content” image data and “style” data, with cross connections tointegrate such information at increasingly higher, more abstract levels. We present the first use of such networksto recovering ghost-paintings from x-ray images of fine art paintings.
We now turn to the application of neural style transfer to the ghost-painting behind two works by Pablo Picasso:
The old guitarist (1903–04) and
The crouching beggar (1902). First, we note that Picasso executed these worksearly in his career, in his so-called “Blue period” in Barcelona, when he sold very few paintings and sometimesre-used canvases.
26, 27
It seems quite likely that his re-use of canvases was based on financial, rather than strictlyaesthetic or artistic reasons. Once Picasso’s career became more secure—particularly starting in Paris—he rarely,if ever re-used his canvases. Art scholars will gain a deeper, more complete understanding of Picasso’s earlyartistic development once his early ghost-paintings are revealed.Figure 1 shows an overview of our approach on the ghost-painting in Pablo Picasso’s
Old guitarist . Thex-radiograph reveals the ghost-painting: a full-length portrait of a seated female nude, much in Picasso’s style ofthat time, with strong outlines and solid, weighty forms. Of course this x-ray reveals no color information aboutthis hidden work. Nevertheless, given that it was executed during the artist’s blue period (exemplified by
Oldguitarist itself), it is natural to transfer style from paintings of figures of the same period, such as
La vie (1903),also executed in Barcelona at that time. a) b) c) d) e)Figure 1. Works by Picasso used in the computational estimation of the ghost-paintings in
Old guitarist . a)
Old guitarist (122 . × . cm ), oil on canvas (1903–04), Art Institute of Chicago, b) x-ray of Old guitarist , Art Institute of Chicago,c) the “content image,” i.e., the hand-edited underdrawing of
Old guitarist in grayscale, d) the “style image,”
La vie (196 . × . cm ), oil on canvas (1903), Cleveland Museum of Art, and e) the computed portrait, where the color stylefrom d) has been transferred to the design from c) by means of deep neural networks. The recognition network used was the VGG-Network, a Convolutional Neural Network that rivals humanperformance on common visual object recognition benchmark tasks and was introduced and extensively de-scribed. As in we consider the representations in the feature space of the convolutional layers. We representthe two-dimensional “content” image as a vector, x C , and a single “style” images as x S , where the pixel dimen-sions are the same, here 1420 × l of the network are F l ( x C ) and F l ( x S ),respectively, where F l ( x C ) ∈ R M l ( x C ) × N l and F l ( x S ) ∈ R M l ( x S ) × N l . Here M l represents layers and N l are thenumber of feature maps within a layer l , specifically the number of color channels, N l = 3.
18, 19
We denote the stylized final image as (cid:98) x , which is obtained by computationally minimizing the following totalerror or cost function: 4 ) b) c) d) e)Figure 2. a) Pablo Picasso’s The crouching beggar (101 . × cm ), oil on canvas (1902), rotated 90 ◦ counter-clockwise,Art Gallery of Ontario, Toronto, b) x-ray of The crouching beggar , Art Gallery of Ontario, Toronto, c) the “contentimage,” the hand-edited underdrawing of
The crouching beggar , d) the “style image,” Santiago Rusi˜nol’s
Terraced Gardenin Mallorca (dimensions unknown), oil on canvas (1911), and e) the computed landscape, where the color style from d)has been mapped to the design from c). C tot = α N l c M l c ( x C ) (cid:88) ij ( F l c ( (cid:98) x ) − F l c ( x C )) ij (cid:124) (cid:123)(cid:122) (cid:125) C cont + β (cid:88) l w l E l (cid:124) (cid:123)(cid:122) (cid:125) C style , (1)where i and j are the indices over the component layers and neurons in each component layer, C cont and C style are the content and style cost functions, and w l and E l are component errors at each layer. Training consistsof stochastic gradient descent in these error functions. The relative weights of the two components of the errorwere set by hand to α = 10 and β = 40, which yielded good images.Our compute environment was Google Compute Engine VM Instance, running on NVIDIA
Tensorflow .
14 and
CUDA -10 .
0. All content andnoise images were scaled to 1412 × GIMP α = 10 and style cost coefficient β = 40, set by hand, worked well.
18, 19
Style transfer ran for 10 iterations minimizing the cost in Eq. 1. We used the default learning rate of 1 .
0. The NST code implementedused a previously trained convolutional neural network from
MatConvNet called the
VGG -19 network—a 19-layerneural network that has been trained on imagenet-vgg-verydeep-19 , a large dataset of
ImageNet images. We shall interpret all our results in Sect. 4.
The style transfer in each of the two Picasso works above was based on a complete “content” image and a single “style” artwork. As such, the method is somewhat limited. It is extremely rare that a single work is fullyrepresentative of the style of an artist or period. The rare exceptions of this principle, involving color style,include series in which only color is changed, as in Josef Alber’s
Homage to the square , Karl Gerstner’s
Homageto the homage to the square , Claude Monet’s series on
Haystacks and
Rouen Cathedral , and a few others. Forthe more general case we must expand our techniques, described in Sect. 3.1, to the application of style from an ensemble of representative artworks.Some underdrawings themselves are not complete and do not contain full bounding contours. In such cases,color style transfer is unlikely to produce acceptable images because the colors will not be sufficiently constrainedgeometrically. In order to transfer color from a “style” image or images in such cases, we must first computa-tionally complete boundaries. This processes is, technically speaking, style transfer but is focused on form anddesign, rather than color.Leonardo completed a mere 36 or so oil paintings in his career, and thus any underdrawing or ghost-paintingis correspondingly very informative to art scholars. Consider the underdrawing revealed through x-radiographyof Leonardo’s
Virigin of the rocks in the National Gallery London, one of the great masterpieces of Renaissanceart, shown in Fig. 3. Clearly this underdrawing is preliminary and incomplete, with numerous contours left open.In order to transfer color style to this underdrawing, then, our first step is to compute a more complete design,where the contours are closed and, to the extent possible, preserve the drawing style of Leonardo.5 igure 3. Leonardo’s
Virgin of the rocks (189 . × cm ), oil on canvas (c. 1499), and negative (white-black reversed)underdrawing, National Gallery London. We applied style transfer to the underdrawing revealed by x-radiography of thiswork, shown at the left in Fig. 4. Figure 4 shows an overview of our approach to this painting, which relies on deep neural networks. The x-rayimage of the painting is shown at the left, and then a color-inverted (negative) version—dark lines on whitesupport—which facilitates the incorporation of style learned from Leonardo’s traditional artworks, in which darklines are marked on light supports. We followed the procedure described in Sect. 3.1 with a few modifications.For an ensemble of images, the final recovered image is simply the mean of the individual images: (cid:98) x f = 1 m m (cid:88) k =1 (cid:98) x k . (2)We shall analyze the resulting work in the next section.
4. RESULTS AND INTERPRETATIONS
The right panels in Figs. 1 and 2 show our results on works by Picasso. The blue flesh, characteristic of Picasso’sportraits of this period, is indeed mapped to the recovered work, though a bit less evenly than generally appears6 igure 4. An overview of our method for extraction of the ghost-painting in Leonardo’s
Virgin of the rocks and styletransfer based on an ensemble of Leonardo’s works. At the left is an x-ray of
Virgin of the rocks , and next is a color-inverted version of the x-ray, which facilitates incorporation of Leonardo’s traditional drawings and artworks (dark lineson white support). Next is a set of representative drawings by the artist, each of which leads, through style transfer, toa corresponding contour. These are then integrated to yield the final, contour-completed model of the underdrawing, atthe right. in his works of that time. There are a few passages in which the intensity of the blue conforms to shading, forexample the seated figure’s shins and wrist. The recovered landscape behind
The crouching beggar captures theoverall color scheme of the reference style image and the colors’ general placement (e.g., sky, trees, fields). Aswith the recovered seated portrait, the color application in large passages of relatively uniform color, such asthe sky, are somewhat blocking and uneven. It would be a simple matter to convolve such regions with a broadspatial filter (or use more sophisticated special technique), but a methodologically preferable approach wouldbe to use a large corpora of style images. Such a principled and data-driven approach reduces significantly thechance that a scholar can bias the method toward images that conform to a prior expectation through choice of“style” artwork.
5. SUMMARY, CONCLUSIONS, AND FUTURE DIRECTIONS
We have demonstrated that deep neural networks can be used to transfer style—both color and contour—fromrepresentative artworks to an achromatic underdrawing and ghost-painting. The resulting images of artworksseem, informally at least, to reveal features hard to interpret in achromatic x-ray and infrared reflectogramimages. Clearly this is a proof-of-concept demonstration and should be validated, for instance through ground-truth artworks created in two layers.We note that the key early step of separating overlapping images from x-ray images of underdrawings auto-matically is extremely challenging and remains unsolved.
20, 22
The general approach that works for linear blindsource separation will have to be modified to work in this case because of the physical complexities of x-rayopacity mixing. In particular two overlapping images do not merely add opacities to create a final x-ray image.Presumably methods will build upon blind image separation, but include penalty functions based on the spatialand chromatic statistics of the particular artist in question, and improvements in the nonlinear summation ofsuch component images. Moreover, the regularization penalties that work best for a given artwork will surelydepend upon the artist in question.Our work advances the recent trend of sophisticated image analysis in the study of fine art. Such work hassolved problems for which traditional connoisseurship was insufficient, such as the claim that Renaissance painterssecretly traced optically projected images during the execution of some paintings such as lighting analysis. We conclude that these proof-of-concept results imply that our techniques, suitably extended, refined, andbased on larger corpora of style images, will enable art scholars to fill in missing works in an artist’s oeuvreand hence gain a more complete view of the artists’s development. Computed works should suffice for studies oflarge-scale composition, or number and identities of figures. The grand challenge is to someday recover hiddenartworks with such fidelity to the original artistic intention that the computed works are worthy of the careulconnoisseurial analysis enjoyed by our “visible” cultural patrimony.
ACKNOWLEDGEMENTS
We would like to thank the National Gallery London, Art Institute of Chicago, Art Gallery of Ontario, MorettiFine Art, Ltd., the Italian Cultural Institute in London, the Italian Chamber of Commerce Industry for theUK, Leasys, GRoW@Annenberg, and 59 Productions. The last author would like to thank the Getty ResearchCenter for access to its Research Library, where some of the above research was conducted.
REFERENCES
1. D. Bomford, ed.,
Art in the making: Underdrawings in Renaissance paintings , National Gallery London,London, UK, 2002.2. R. H. Marijnissen,
The masters’ and the forgers’ secrets: X-ray authentication of paintings from earlyNetherlandish till modern , Mercatorfonds, Brussels, Belgium, 2011.3. T. Lenain,
Art forgery: The history of a modern obsession , Reaktion Books, London, UK, 2012.4. S. Legrand, F. Vanmeert, G. Van der Snickt, M. Alfeld, W. De Nolf, J. Dik, and K. Janssens, “Exami-nation of historical paintings by state-of-the-art hyper-spectral imaging methods: From scanning infra-redspectroscopy to computed X-ray laminography,”
Heritage Science (1), pp. 1–11, 2014.5. K. Janssens, G. Van der Snickt, F. Vanmeert, S. Legrand, G. Nuyts, M. Alfeld, L. Monico, W. Anaf,W. D. Nolf, M. Vermeulen, J. Verbeeck, and K. D. Wael, “Non-invasive and non-destructive examinationof artistic pigments, paints, and paintings by means of x-ray methods,” Topics in Current Chemistry:Analytical Chemistry for Cultural Heritage , pp. 77–128, 2016.6. J. R. J. van Asperen de Boer, “Infrared reflectography: A method for the examination of paintings,”
AppliedOptics (9), pp. 1711–1714, 1968.7. D. Andrzejewski, D. G. Stork, X. Zhu, and R. Spronk, “Inferring compositional style in the neo-plasticpaintings of Piet Mondrian by machine learning,” in Computer vision and image analysis of art , D. G.Stork, J. Coddington, and A. Bentkowska-Kafel, eds., , pp. 75310G1–75310G11, (San Jose, CA), 2010.8. S. van Heugten, “Radiographic images of Vincent van Gogh’s paintings in the collection of the van GoghMuseum,”
Van Gogh Museum Journal , pp. 62–85, 1995.9. N. Charney,
The museum of lost art , Phaidon Press, New York, NY, 2018.10. S. Lowry, “Ghosted forms,” in
Erasure: The spectre of cultural memory , B. Buckley and J. Conomos, eds.,ch. 13, pp. 199–218, Libri Publishing, Ltd., Oxfordshire, UK, 2015.11. M. Faries, “Analytical capabilities of infrared reflectography: An art historian’s perspective,” in
Scientificexamination of art: Modern techniques in conservation and analysis , pp. 87–104, National Academies Press,Washington, DC, 2003.12. A. Anitha, A. Brasoveanu, M. Duarte, S. Hughes, I. Daubechies, J. Dik, K. Janssens, and M. Alfeld, “Virtualunderpainting reconstruction from x-ray fluorescence imaging data,” in , pp. 1239–1243, 2011.13. A. Anitha, A. Brasoveanu, M. Duarte, S. Hughes, I. Daubechies, J. Dik, K. Janssens, and M. Alfeld,“Restoration of x-ray fluorescence images of hidden paintings,”
Signal processing (3), pp. 592–604, 2013.14. J. Dik, K. Janssens, G. Van der Snickt, L. van der Loeff, K. Rickers, and M. Cotte, “Visualization of a lostpainting by Vincent van Gogh using synchrotron radiation based x-ray fluorescence elemental mapping,” Analytical chemistry (16), pp. 6436–6442, 2008. 85. P. Kammerer, E. Zolda, and R. Sablatnig, “Computer aided analysis of underdrawings in infrared reflecto-grams,” in Proceedings of the 4th International Symposium on Virtual Reality, Archaeology and IntelligentCultural Heritage , D. Arnold, A. Chalmers, and F. Nicolucci, eds., pp. 19–27, (Brighton, United Kingdom),2003.16. M. Str¨om, E. Johansson, and D. G. Stork, “Mapping colors from paintings to tapestries: Rejuvenating thefaded colors in tapestries based on colors in reference paintings,” in
Electronic Imaging: Human Vision andElectronic Imaging XVII , B. E. Rogowitz, T. N. Pappas, and H. de Ritter, eds., , pp. 82911F1–10,IS&T/SPIE, (Bellingham, WA), 2012.17. A. Bourached and G. H. Cann, “Raiders of the lost art,” 2019. arXiv:1909.05677v1 .18. L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks,” in
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 2414–2423, 2016.19. L. A. Gatys, A. S. Ecker, M. Bethge, A. Hertzmann, and E. Shechtman, “Controlling perceptual factors inneural style transfer,” in
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 3985–3993, 2017.20. J. Miskin and D. J. C. MacKay, “Ensemble learning for blind image separation and deconvolution,” in
Advances in Independent Component Analysis , M. Girolami, ed., pp. 123–141, Springer-Verlag, London,UK, 2000.21. P. Comon and C. Jutten,
Handbook of blind source separation: Independent component analysis and appli-cations , Academic Press, New York, NY, 2010.22. R. O. Duda, P. E. Hart, and D. G. Stork,
Pattern classification , John Wiley and Sons, New York, NY,Second ed., 2001.23. S. Bae and F. Durand, “Statistical analysis and transfer of pictorial styles,” in
MIT Oxygene Workshop ,(Cambridge, MA), 2005.24. X. Xie, F. Tian, and H. S. Seah, “Feature guided texture synthesis (FGTS) for artistic style transfer,” in
Proceedings of the 2nd International Conference on Digital Interactive Media in Entertainment and Arts ,pp. 44–49, Association for Computing Machinery, (New York, NY), 2007.25. K. Zeng, R. Zhang, X. Lan, Y. Pan, and L. Lin, “Color style transfer by constraint locally linear embedding,”in , pp. 1121–1124, (New York, NY), 2011.26. R. Bouvier, ed.,
Picasso: The blue and rose periods , Hatje Cantz Verlag, Berlin, Germany, 2019.27. P. O’Brian,
Picasso: A biography , W. W. Norton and Company, New York, NY, 1994.28. M. McCully,
Picasso: The early years, 1892–1906 , National Gallery Washington/Yale University Press, NewHaven, CT, 1997.29. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXivpreprint arXiv:1409.1556 , 2014.30. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014. arXiv:1409.15565v6 .31. F. Z¨ollner,
Leonardo da Vinci, 1452–1519: The complete paintings , Taschen, Los Angeles, CA, 2017.32. D. M. Stone and D. G. Stork, “Computer assisted connoisseurship: The interdisciplinary science of computervision and image analysis in the study of art,” in
Proceedings of the Image Processing for Art Investigations(IP4AI) , Museum of Modern Art, (Museum of Modern Art, New York), 2011.33. D. G. Stork, “Optics and realism in Renaissance art,”
Scientific American (6), pp. 76–84, 2004.34. D. G. Stork, J. Collins, M. Duarte, Y. Furuichi, D. Kale, A. Kulkarni, M. D. Robinson, C. W. Tyler,S. Schechner, and N. Williams, “Did early Renaissance painters trace optically projected images? Theconclusion of independent scientists, art historians and artists,” in
Digital imaging for cultural heritagepreservation , F. Stanco, S. Battiato, and G. Gallo, eds., ch. 8, pp. 379–407, CRC Press, Boca Raton, FL,2011.35. M. K. Johnson, D. G. Stork, S. Biswas, and Y. Furuichi, “Inferring illumination direction estimated fromdisparate sources in paintings: An investigation into Jan Vermeer’s
Girl with a pearl earring ,” in
Computerimage analysis in the study of art , D. G. Stork and J. Coddington, eds., , pp. 68100I–1–12, SPIE/IS&T,Bellingham, WA, 2008. 96. M. Irfan and D. G. Stork, “Multiple visual features for the computer authentication of Jackson Pollock’sdrip paintings: Beyond box-counting and fractals,” in
SPIE Electronic Imaging: Image processing: Machinevision applications II , K. S. Niel and D. Fofi, eds., , pp. 72510Q1–11, SPIE/IS&T, Bellingham, WA,2009.37. P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarialnetworks,” in
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 1125–1134,(Honolulu, HI), 2017.38. T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, “High-resolution image synthesisand semantic manipulation with conditional GANS,” in
IEEE Conference on Computer Vision and PatternRecognition (CVPR) , pp. 8798–8807, (Salt Lake City, UT), 2018.39. E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,”
IEEE Transactionson Computer Graphics and Applications (5), pp. 34–41, 2001.40. Y. Li, C. Fang, J. Yang, Z. Wang, X. Lu, and M.-H. Yang, “Universal style transfer via feature transforms,”in Advances in Neural Information Processing Systems 30 , I. Guyon, U. von Luxburg, S. Bengio, H. Wallach,R. Fergus, S. Vishwanathan, and R. Garnett, eds., pp. 386–396, Curran Associates, Red Hook, NY, 2017.41. A. Selim, M. A. Elgharib, and L. E. Doyle, “Painting style transfer for head portraits using convolutionalneural networks,”
ACM Transactions on Graphics (4), p. Article 129, 2016.42. W. Zhang, C. Cao, S. Chen, J. Liu, and X. Tang, “Style transfer via image component analysis,” IEEETransactions on Multimedia (7), pp. 1594–1601, 2013.43. J. B. Tenenbaum and W. T. Freeman, “Separating style and content with bilinear models,” Neural compu-tation12