Photorealistic Material Editing Through Direct Image Manipulation
PPhotorealistic Material Editing Through Direct Image Manipulation
K ´AROLY ZSOLNAI-FEH ´ER,
TU Wien
PETER WONKA,
KAUST
MICHAEL WIMMER,
TU Wien
Fig. 1. We propose a hybrid technique to empower novice users and artists without expertise in photorealistic rendering to create sophisticated materialmodels by applying standard image editing operations to a source image. Then, in the next step, our method proceeds to find a photorealistic BSDF that,when rendered, resembles this target image. Our method generates each of the showcased fits within 20-30 seconds of computation time and is able to offerhigh-quality results even in the presence of poorly-executed edits (e.g., the background of the gold target image, the gold-colored pedestal for the watermaterial and the stitched specular highlight above it). Scene: Reynante Martinez.
Creating photorealistic materials for light transport algorithms requirescarefully fine-tuning a set of material properties to achieve a desired artisticeffect. This is typically a lengthy process that involves a trained artist withspecialized knowledge. In this work, we present a technique that aims toempower novice and intermediate-level users to synthesize high-qualityphotorealistic materials by only requiring basic image processing knowledge.In the proposed workflow, the user starts with an input image and applies a
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).© 2016 Copyright held by the owner/author(s). XXXX-XXXX/2016/1-ART1 $15.00DOI: 10.1145/nnnnnnn.nnnnnnn few intuitive transforms (e.g., colorization, image inpainting) within a 2Dimage editor of their choice, and in the next step, our technique producesa photorealistic result that approximates this target image. Our methodcombines the advantages of a neural network-augmented optimizer and anencoder neural network to produce high-quality output results within 30seconds. We also demonstrate that it is resilient against poorly-edited targetimages and propose a simple extension to predict image sequences with astrict time budget of 1-2 seconds per image.CCS Concepts: •
Computing methodologies → Neural networks; Ren-dering; Ray tracing;
Additional Key Words and Phrases: neural networks, photorealistic render-ing, material modeling, neural rendering , Vol. 1, No. 1, Article 1. Publication date: January 2016. a r X i v : . [ c s . G R ] S e p :2 • Zsolnai-Feh´er et al. 2018 ACM Reference format:
K´aroly Zsolnai-Feh´er, Peter Wonka, and Michael Wimmer. 2016. Photoreal-istic Material Editing Through Direct Image Manipulation. 1, 1, Article 1(January 2016), 12 pages.DOI: 10.1145/nnnnnnn.nnnnnnn
The expressiveness of photorealistic rendering systems has seengreat strides as more sophisticated material models became availablefor artists to harness. Most modern rendering systems offer a node-based shader tool where the user can connect different kinds ofmaterial models and perform arbitrary mathematical operationsover them (e.g., addition and mixing), opening up the possibilityof building a richer node graph that combines many of the morerudimentary materials to achieve a remarkably expressive model.These are often referred to as “principled” shaders and are commonlyused within the motion picture industry [Burley and Studios 2012].However, this expressiveness comes with the burden of complexity,i.e., the user has to understand each of the many parameters ofthe shader not only in isolation, but also how they influence eachother, which typically requires years of expertise in photorealisticmaterial modeling. In this work, we intend to provide a tool thatcan be used by a wider target audience, i.e., artists and novicesthat do not have any experience creating material models, but areadept at general-purpose image processing and editing. This ishighly desirable as human thinking is inherently visual and is notbased on physically-based material parameters [R¨oder et al. 2002;White 1989]. We propose a workflow in which the artist starts outwith an image of a reference material and applies classic imageprocessing operations to it. Our key observation is that even thoughthis processed target image is often not physically achievable, inmany cases, a photorealistic material model can be found that isremarkably close to it (Fig. 2). These material models can then beeasily inserted into already existing scenes by the user (Fig. 3).In summary, we present the following contributions: • An optimizer that can rapidly match the target image whengiven an approximate initial guess. • A neural network to solve the adjoint rendering problem,i.e., take the target image as an input and infer a shaderthat produces a material model to approximate it. • A hybrid method that combines the advantages of thesetwo concepts and achieves high-quality results for a varietyof cases within 30 seconds. • A simple extension of our method to enable predictingsequences of images within 1-2 seconds per image.We provide our pre-trained neural network and the source codefor the entirety of this project here: https://users.cg.tuwien.ac.at/zsolnai/gfx/photorealistic-material-editing/
A common workflow for photorealistic material acquisition requiresplacing the subject material within a studio setup and using mea-surement devices to obtain its reflectance properties [Marschneret al. 1999; Miyashita et al. 2016]. To import this measured data into a production renderer, it can be either used as-is, can be com-pressed down into a lower-dimensional representation [Papas et al.2013; Rainer et al. 2019] or approximated through an analytic BSDF model [Papas et al. 2014]. Many recent endeavors improve the costefficiency and convenience of this acquisition step by only requir-ing photographs of the target material [Aittala et al. 2016, 2015;Deschaintre et al. 2018; Li et al. 2017, 2018] while still requiringphysical access to these source material samples, while precom-puted BSDF databases offer an enticing alternative where the usercan choose from a selection of materials [Dupuy and Jakob 2018;Matusik 2003]. We aim to provide a novel way to exert direct artis-tic control over these material models. Our method can be relatedto inverse rendering approaches [Marschner and Greenberg 1998;Ramamoorthi and Hanrahan 2001], where important physical mate-rial properties are inferred from a real photograph with unknownlighting conditions. In our work, the material test scene contains aknown lighting and geometry setup, but in return, enables not onlythe rapid discovery of new materials, but artistic control throughstandard and well-known image-space editing operations. To be able to efficiently use the most common photorealistic render-ing systems, an artist is typically required to have an understandingof physical quantities pertaining to the most commonly modeledphenomena in light transport, e.g., indices of refraction, scatteringand absorption albedos and more [Burley and Studios 2012; Songet al. 2009]. This modeling time can be cut down by techniques thatenable editing BRDF models directly within the scene [Ben-Artziet al. 2006; Cheslack-Postava et al. 2008; Sun et al. 2007], however,with many of these methods, the artist is still required to under-stand the physical properties of light transport, often incurring asignificant amount of trial and error. Instead of editing the ma-terials directly, other techniques enable editing secondary effects,such as caustics and indirect illumination within the output image[Ben-Artzi et al. 2008; Schmidt et al. 2013]. An efficient materialediting workflow also opens up the possibility of rapid relightingpreviously rendered scenes [Ng et al. 2004; Wang et al. 2008, 2004].Reducing the expertise required for material editing workflows hasbeen a subject to a large volume of research works: an intuitiveeditor was proposed by pre-computing many solutions to enablerapid exploration [Haˇsan and Ramamoorthi 2013], carefully craftedmaterial spaces were derived to aid the artist [Lagunas et al. 2019;Serrano et al. 2016; Soler et al. 2018], and learning algorithms havebeen proposed to create a latent space that adapts to the prefer-ences of the user [Zsolnai-Feh´er et al. 2018]. We endeavored tocreate a solution that produces the desired results rapidly by look-ing at a non-physical mockup image, requiring expertise only in2D image editing, which is considered to be common knowledge bynearly all artists in the field. Generally, BRDF relighting methodsare preferable when in-scene editing is a requirement, otherwise,we recommend using our proposed technique in the case of onesought material to moderate-scale problems and Gaussian MaterialSynthesis (GMS) for mass-scale material synthesis. Bidirectional scattering distribution function. Bidirectional reflectance distribution function., Vol. 1, No. 1, Article 1. Publication date: January 2016. hotorealistic Material Editing Through Direct Image Manipulation • 1:3
Fig. 2. Our proposed hybrid technique offers an intuitive workflow where the artist takes a source material ( (cid:182) ) and produces the target image by applyingthe desired edits to it within a 2D raster image editor of their choice ( (cid:183) ). Then, one or more encoder neural networks are used to propose a set of approximateinitial guesses ( (cid:184) ) to be used with our neural network-augmented optimizer ( (cid:185) ), which rapidly finds a photorealistic shader setup that closely matches thetarget image ( (cid:186) ). The artist then finishes the process by assigning this material to a target object and renders the final scene offline.
Optimization is present at the very core of every modern neural net-work: to be able to minimize the prescribed loss function efficiently,the weights of the networks are fine-tuned through gradient de-scent variants [Bottou 2010; Robbins and Monro 1951] or advancedmethods that include the use of lower-order moments [Kingma andBa 2014], while additional measures are often taken to speed upconvergence and avoid poor local minima [Goh 2017; Sutskever et al.2013]. Similar optimization techniques are also used to generate themodel description and architecture of these neural networks [Elskenet al. 2018; Zoph and Le 2016], or the problem statement itself canalso be turned around by using learning-based methods to discovernovel optimization methods [Bello et al. 2017]. In this work, wepropose two combinations of a neural network and an optimizer –first, the two can be combined indirectly by endowing the optimizerwith a reasonable initial guess, and directly by using the optimizerthat invokes a neural renderer at every function evaluation step tospeed up the convergence by several orders of magnitude (steps (cid:184) and (cid:185) in Fig. 2). This results in an efficient two-stage systemthat is able to rapidly match a non-physical target image and doesnot require the user to stay within a prescribed manifold of artisticediting operations [Zhu et al. 2016].
Many trained artists are adept at creating new photorealistic materi-als by engaging in a direct interaction with a principled shader. Thisworkflow includes adjusting the parameters of this shader and wait-ing for a new image to be rendered that showcases the appropriate output material. If at most a handful of materials are sought, this isa reasonably efficient workflow, however, it also incurs a significantamount of rendering time and expertise in material modeling. Ourgoal is to empower novice and intermediate-level users to be ableto reuse their knowledge from image processing and graphic designto create their envisioned photorealistic materials.In this work, we set up a material test scene that contains a knownlighting and geometry setup, and a fixed principled shader witha vector input of x ∈ R m where m =
19. This shader is able torepresent the most commonly used diffuse, glossy, specular andtranslucent materials with varying roughness and volumetric ab-sorption coefficients. Each parameter setup of this shader producesa different material model when rendered. In our workflow, theuser is offered a variety of images, and chooses one desired materialmodel as a starting point. Then, the user is free to apply a variety ofimage processing operations on it, e.g., colorization, image inpaint-ing, blurring a subset of the image and more. Since these imageprocessing steps are not grounded in a physically-based framework,the resulting image is not achievable by adjusting the parametersin the vast majority of cases. However, we show that our proposedmethod is often able a produce a photorealistic material that closelymatches this target image.
Solution by optimization.
When given an input image t ∈ R p ,it undergoes a series of transformations (e.g., colorization, imageinpainting) as the artist produces the target image ˜ t = Ψ ( t ) , where Ψ : R p → R p . Then, an image is created from an initial shaderconfiguration, i.e., ϕ : R m → R p , where m refers to the number ofparameters within the shader and p is the number of variables that , Vol. 1, No. 1, Article 1. Publication date: January 2016. :4 • Zsolnai-Feh´er et al. 2018 Fig. 3. To demonstrate the utility of our system, we synthesized a newmaterial and deployed it into an already existing scene using Blender andCycles. In this scene, we made a material mixture to achieve a richer andfoggier nebula effect inside the glass. Left: theirs, right: 50% theirs, 50%ours. Scene: Reynante Martinez. describe the output image (in our case p = · is used with therange of 0-255 for each individual pixel). This operation is typicallyimplemented by a global illumination renderer. Our goal is to findan appropriate parameter setup of the principled shader x ∈ R m that, when rendered, reproduces ˜ t . Generally, this is not possibleas a typical Ψ leads to images that cannot be perfectly matchedthrough photorealistic rendering. However, surprisingly, we canoften find a configuration x that produces an image that closelyresembles ˜ t through solving the minimization problemargmin x || ϕ ( x ) − ˜ t || , subject to x min ≤ x ≤ x max , (1)where the constraints stipulate that each shader parameter hasto reside within the appropriate boundaries (i.e., 0 ≤ x i ≤ x j ≥ x i , x j ∈ x ). Tobe able to benchmark a large selection of optimizers, we introducean equivalent alternative formulation of this problem where the constraints are reintroduced as a barrier function Γ (·) , i.e.,argmin x (cid:16) || ϕ ( x ) − ˜ t || + Γ ( x ) (cid:17) , where Γ ( x ) = (cid:26) , if x ∈ C , + ∞ , otherwise , C = (cid:110) x | f i ( x ) ≥ , i = , (cid:111) , f ( x ) = x max − x , f ( x ) = x − x min . (2)where C denotes the feasible region chosen by a set of constraintsdescribed by f i (·) (equivalent to the second line in (1)) and thevector comparison operator ( ≥ ) here is considered true only whenall of the vector elements exceed (or equal to) zero. In a practicalimplementation, the infinity can be substituted by a sufficiently largeinteger. This formulation enabled us to compare several optimizers(Table 3 in Appendix B), where we found Nelder and Mead’s simplex-based self-adapting optimizer [1965] to be the overall best choicedue to its ability to avoid many poor local minima through itscontraction operator and used that for each of the reported resultsthroughout this manuscript.Nonetheless, solving this optimization step still takes severalhours as each function evaluation invokes ϕ , i.e., a rendering stepto produce an image, which clearly takes too long for day-to-dayuse in the industry. We introduce two solutions to remedy this limi-tation, followed by a hybrid method that combines their advantages. Neural renderer.
To speed up the function evaluation process,we replace the global illumination engine that implements ϕ with aneural renderer [Zsolnai-Feh´er et al. 2018]. This way, instead of run-ning a photorealistic rendering program at each step, our optimizerinvokes the neural network to predict this image, thus reducing theexecution time of the process by several orders of magnitude, in ourcase, from an average of 50 seconds to 4ms per image at the cost ofrestricting the material editing to a prescribed scene and lightingsetup. Because of the lack of a useful initial guess, this solution stillrequires many function evaluations and is unable to reliably providesatisfactory solutions. Solution by inversion.
One of our key observations is that anapproximate solution can also be produced without an optimiza-tion step by finding an appropriate inverse to ϕ : since ϕ is realizedthrough a decoder neural network (i.e., neural renderer) that pro-duces an image from a shader configuration, ϕ − , its inverse, can beimplemented as an encoder network that takes an image as an inputand predicts the appropriate shader parameter setup that generatesthis image. This adjoint problem has several advantages: first, sucha neural network can be trained on the same dataset as ϕ by onlyswapping the inputs and outputs and retains the advantageous prop-erties of this dataset, e.g., arbitrarily many new training samplescan be generated via rendering, thereby loosening the ever-presentrequirement of preventing overfitting via regularization [Nowlanand Hinton 1992; Srivastava et al. 2014; Zou and Hastie 2005]. Sec-ond, we can use it to find a solution directly through x ≈ ϕ − ( ˜ t ) , Vol. 1, No. 1, Article 1. Publication date: January 2016. hotorealistic Material Editing Through Direct Image Manipulation • 1:5 Fig. 4. Whenever the target image strays too far away from the images contained within their training set (lower right), our 9 inversion networks typically failto provide an adequate solution. However, using our “best of n” scheme and our hybrid method, the best performing prediction of our neural networks can beused to equip our optimizer with an initial guess, substantially improving its results. without performing the optimization step described in (1-2). Asthe output image is not produced through a lengthy optimizationstep, but is inferred by this encoder network, this computes in a fewmilliseconds. We will refer to this solution as the inversion network and note that our implementation of ϕ − only approximately admitsthe mathematical properties of a true inverse function. We alsodiscuss the nature of the differences in more detail in Section 4. Wehave trained 9 different inversion network architectures and foundthat typically, each of them performs well on a disjoint set of inputs.Our other key observation is that because we have an atypical prob-lem where the ground truth image (˜ t ) is available and each of thecandidate images can be inferred inexpensively (typically within5 milliseconds), it is possible to compute a “best of n ” solution bycomparing all of these predictions to the ground truth, i.e., x = ϕ − ( i ) ( ˜ t ) , where i = argmin j || ϕ ( ϕ − ( j ) ( ˜ t )) − ˜ t || , (3)where ϕ − ( i ) denotes the prediction of the i -th inversion network, j = ( , . . . , n ) , and in our case, n = Hybrid solution.
Both of our previous solutions suffer from draw-backs: the optimization approach provides results that resemble ˜ t but is impracticable due to the fact that it requires too many function evaluations and gets stuck in local minima, whereas the inversionnetworks rapidly produce a solution, but offer no guarantees whenthe target image significantly differs from the ones shown in thetraining set. We propose a hybrid solution based on the knowledgethat even though the inverse approach does not provide a perfectsolution, since it can produce results instantaneously that are sig-nificantly closer to the optimum than a random input, it can beused to endow the optimizer with a reasonable initial guess. Thismethod is introduced as a variant of (2) where x init = ϕ − ( ˜ t ) and amore detailed description of this hybrid solution is given below inAlgorithm 1. Additionally, this technique is able to not only providea “headstart” over the standard optimization approach but was alsoable to find higher quality solutions in all of our test cases. Predicting image sequences.
A typical image editing workflowtakes place within a raster graphics editor program where the artistendeavors to find an optimal set of parameters, e.g., the kernel width σ in the case of a Gaussian blur operation to obtain their envisionedartistic effect. This process includes a non-trivial amount of trialand error where the artist decides whether the parameters shouldbe increased or decreased; this is only possible in the presence ofnear-instant visual feedback that reflects the effect of the parameterchanges on the image. We propose a simple extension to our hybridmethod to accommodate these workflows: consider an examplescenario where the k -th target image in a series of target images˜ t ( k ) are produced by subjecting a starting image t to an increasinglywide blurring kernel. This operation is denoted by Ψ σ ( t ) = G σ ∗ t , , Vol. 1, No. 1, Article 1. Publication date: January 2016. :6 • Zsolnai-Feh´er et al. 2018 Fig. 5. Results for three techniques on common global colorization opera-tions including saturation increase and grayscale transform. The “referencematerial” labels showcase materials that can be obtained using our shaderand are used as source images for the materials below them, where thearrows denote the evolution of the target image. where G σ is a zero-centered Gaussian, and for simplicity, the targetimages are produced via ˜ t ( k ) = Ψ k ( t ) , with the initial condition of˜ t ( ) = t . We note that many other transforms can also be substitutedin the place of Ψ without loss of generality. We observe that suchworkflows create a series of images where each neighboring imagepair shows only minute differences, i.e., for any positive non-zero k , Algorithm 1
Photorealistic Material Editing Given t , ϕ (·) , (cid:2) ϕ − ( ) (·) , . . . , ϕ − ( n ) (·) (cid:3) , x min , x max ˜ t ← Ψ ( t ) (cid:46) Obtain target image for i ← n do (cid:46) Predict with n inversion networks Compute each ϕ − ( i ) ( ˜ t ) Find i = argmin j ∈ .. n || ϕ ( ϕ − ( j ) ( ˜ t )) − ˜ t || (cid:46) Find best candidate Define x init ← ϕ − ( i ) ( ˜ t ) Define f ( x ) = x max − x (cid:46) Set up constraints Define f ( x ) = x − x min Define C = (cid:8) x | f i ( x ) ≥ , i = , (cid:9) (cid:46) Construct feasibleregion
Define Γ ( x ) = (cid:26) , if x ∈ C , + ∞ , otherwise (cid:46) Construct barrier
Initialize optimizer with x init Minimize argmin x (cid:0) || ϕ ( x ) − ˜ t || + Γ ( x ) (cid:1) (cid:46) Refine initial guess
Display ϕ ( x ) to user || ˜ t ( k + ) − ˜ t ( k ) || remains small. As in these cases, we are requiredto propose many output images, we can take advantage of thisfavorable mathematical property by extending the pool of initialinversion networks with the optimized result of the previous frameby modifying Steps 3-5 of Algorithm 1 to add ϕ − ( n + ) ( ˜ t k ) = argmin x (cid:16) || ϕ ( x ) − ˜ t k − || + Γ ( x ) (cid:17) . (4)Note that this does not require any extra computation as the resultof Step 12 of the previous run can be stored and reused. Intuitively,this means that both the inversion network predictions and theprediction of the previous image are used as candidates for the op-timization (whichever is better). This way, after the optimizationstep is finished, the improvements can be “carried over” to the nextframe. This method we refer to as reinitialization and in Section4, we show that it consistently improves the quality of our outputimages for such image sequences, even with a strict budget of 1-2seconds per image. In this section, we discuss the properties of our inverse problemformulation (i.e., inferring a shader setup that produces a prescribedinput image), followed by both a quantitative and qualitative eval-uation of our proposed hybrid method against the optimizationand inversion network solutions. We also show that our systemsupports a wide variety of image editing operations and can rapidlypredict image sequences. To ensure clarity, we briefly revisit thethree introduced methods: • The optimization approach relies on minimizing (2) withNelder and Mead’s simplex method using a random initialguess, and implementing ϕ through a neural renderer, • the inversion network refers to the “best of 9” inversionsolution, i.e., x ≈ ϕ − ( i ) ( ˜ t ) as shown in (3), , Vol. 1, No. 1, Article 1. Publication date: January 2016. hotorealistic Material Editing Through Direct Image Manipulation • 1:7 Initial guess 50 fun. evals 300 fun. evals 1500 fun. evalsInput Random NN Optimizer Ours Optimizer Ours Optimizer OursFig. 5, Row 1 41.93 5.94 33.81 . . . Fig. 5, Row 2 78.45 32.72 68.55 . . . Fig. 5, Row 4 35.37 18.68 30.88 . . . Fig. 5, Row 7 41.65 22.42 38.10 . . . Fig. 5, Row 8 29.04 19.82 26.79 . . . Fig. 8, Row 2 23.78 12.79 20.31 . . . Fig. 8, Row 3 21.60 9.09 16.54 . . . Fig. 8, Row 8 29.58 9.74 22.69 . . . Table 1. A comparison of the optimization approach (with random initialization) and our hybrid method (with “best of 9” NN initialization) on a variety ofchallenging global and local image editing operations in Fig. 5 and 8. The numbers indicate the RMSE of the outputs, and for reference, the first row showcasesan input image that is reproducible by the shader.
Image ID in sequence (i.e., k of ˜ t ( k ) )F. evals Technique 0 10 20 30 40 50 60 70 80 90 100 110 120 Σ
100 No reinitialization . .
55 12 . .
93 1 .
34 1 .
88 2 .
54 3 .
34 4 .
30 5 .
30 6 .
38 7 .
50 8 .
69 9 .
93 11 .
55 12 .
99 77 .
300 No reinitialization . .
41 12 . .
64 1 .
30 1 .
80 2 .
42 3 .
25 4 .
25 5 .
25 6 .
33 7 .
45 8 .
64 9 .
88 11 .
41 12 .
82 76 .
600 No reinitialization . .
41 12 . .
57 1 .
29 1 .
80 2 .
49 3 .
33 4 .
20 5 .
18 6 .
27 7 .
38 8 .
58 9 .
81 11 .
41 12 .
80 76 . Table 2. Our proposed reinitialization technique consistently outperforms per-frame computation for the image sequence shown in Fig. 6. The numbersindicate the RMSE of the outputs. • our hybrid method is obtained by combining the twoabove approaches as described in Algorithm 1.Furthermore, in Appendix A, we report the structure of the neuralnetworks used to implement each individual ϕ − ( i ) shown in Fig. 4,and compare our solution to a selection of local and global mini-mizers in Appendix B. At the end of this section, we also comparethe total time taken to synthesize 1, 10, and 100 selected materialsagainst a recent method for mass-scale material synthesis. Inversion accuracy.
Our inversion technique leads to an ap-proximate solution within a few milliseconds, however, becausethe structure of the forward and inverse networks differ, the inver-sion operation remains imperfect, especially when presented witha target image that includes materials that are only approximatelyachievable. To demonstrate this effect, we have trained 9 differentinversion networks to implement ϕ − and show that none of theproposed solutions are satisfactory as a final output for the globalcolorization case (Fig. 4). Our goal with this experiment was todemonstrate that a solution containing only one inversion networkgenerally produces unsatisfactory outputs, regardless of networkstructure. However, these predictions can be used to equip our opti-mizer with an initial guess, substantially improving its results. Aseach neural network consumes between 300MB and 1GB of videomemory, we were able to keep all of them loaded during the entiretyof the work session. Optimizer and hybrid solution accuracy.
In Table 1, we com-pared our hybrid solution against the “best of 9” inversion networkand optimization approaches and recorded the RMS error after 50,300 and 1500 function evaluations (these roughly translate to 1, 6,and 30-second execution times) to showcase the early and late-stageperformance of these methods. The table contains a selection ofscenarios that we consider to be the most challenging and note thatthe outputs showed no meaningful change after 1500 function eval-uations. Our hybrid method produced the lowest errors in each ofour test cases, and surprisingly, the inversion network initializationnot only provides a “headstart” for our method, but also improvesthe final quality of the output, thereby helping the optimizer toavoid local minima. To validate the viability of our solutions, wealso ran a global minimizer [Wales and Doye 1997] with severaldifferent parameter choices and a generous allowance of 30 minutesof computation time for each; our hybrid method was often ableto match (and in some cases, surpass) the quality offered by thissolution (Appendix B, Table 3), further reinforcing how our inver-sion network initialization step helps avoid getting stuck in poorlocal minima. Note that the optimizer was unable to meaningfullyimprove the best prediction of the 9 inversion networks in Fig. 5,Row 7 – in this case, a better solution can be found by using theprediction of only the first neural network and passing it to theoptimizer, improving the reported RMSE from 22.38 to 19.39 byusing 300 function evaluations. , Vol. 1, No. 1, Article 1. Publication date: January 2016. :8 • Zsolnai-Feh´er et al. 2018
Fig. 6. Our image sequence starts with an input that is achievable using ourshader (upper left), where each animation frame slightly increases its blacklevels. The lower right region showcases the 300th frame of the animation.
Supported image editing operations.
A typical workflow us-ing our technique includes the artist choosing a source materialand applying an appropriate image editing operation ( Ψ ) insteadof engaging in a direct interaction with the principled shader. Wecluster the set of possible transforms into global (Fig. 5) and local (Fig. 8) operations: these cases include saturation increase, grayscaletransform, colorization, image mixing, stitching and inpainting, andselective blurring of highlights. Both the optimizer and our hybridmethod were run for 1500 function evaluations to obtain the resultsshowcased in these two figures. As these transformations comefrom a 2D raster editor and are not grounded in a physically-basedframework, a perfect match is often not possible, however, in eachof these cases, our hybrid method proposed a solution of equivalentor better quality compared to the “best of 9” inversion network andthe optimizer solutions. Image sequence prediction.
As our earlier results in Table 1revealed that the global colorization techniques typically prove tobe among the more difficult cases, we have created a challengingimage sequence with an input image that is achievable with ourshader, and subjected it to a slight black level increase over manyframes (Fig. 6). Every image within this sequence is reproducedboth with independent per-frame inference and our reinitializationtechnique with a strict time budget of 2, 6, and 12 seconds per image(100, 300, and 600 function evaluations). In Table 2, we show thatthis simple extension successfully exploits the advantageous math-ematical properties of these workflows and consistently reducesthe output error for the majority of the sequence, i.e., images 1-100.We also report the RMSE of images 101-120 for reference, whichwe refer to as the “converged” regime in which the target imagesstray further and further away from the feasible domain, and theproposed solution remains the same despite these changes. Evenin these cases, our reinitialization technique performs no worse than the “no reinitialization” method, and because of its negligibleadditional cost, we consider it to be a strictly better solution.
Modeling and execution time.
In Fig. 7, we have recordedthe modeling times for 1, 10, and a 100 similar materials using ourmethod and compared them against Gaussian Material Synthesis[Zsolnai-Feh´er et al. 2018] (GMS), a learning-based technique formass-scale material synthesis. We briefly describe the most im-portant parameters of the recorded execution times and refer theinterested reader to this paper for more details – the novice andexpert user timings were taken from the GMS paper and indicatethe amount of time these users took to create the prescribed numberof materials by hand using Disney’s “principled” shader [Burleyand Studios 2012], whereas GMS and our timings contain both themodeling (i.e., scoring a material gallery in GMS and performingimage processing for our technique) and execution times. If onlyone material is desired, our technique outperforms this previouswork and nearly matches the efficiency of an expert user. When 10materials are sought (1 base material and 9 variants), our proposedmethod was adapted to use the re-initialization technique and offersthe best modeling times, outperforming both GMS and expert users.In the case of mass-scale material synthesis, i.e., 100 or more mate-rials, both methods outperform experts, where GMS offers the bestscaling solution. In each case, the timings for our technique includethe fixed cost of loading the 9 neural networks (5.5s). Throughoutthis manuscript, all results were generated using a NVIDIA TITANRTX GPU.
Fig. 7. The recorded modeling times reveal that if at most a handful (i.e.,1-10) of target materials are sought, our technique offers a favorable entrypoint for novice users into the world of photorealistic material synthesis.
As demonstrated in Fig. 4, the results of ϕ − depend greatly on theperformance of the encoder and decoder neural networks. As thesemethods enjoy significant research attention, we encourage furtherexperiments in including these advances to improve them (e.g.,architecture search [Real et al. 2017], capsule networks [Hintonet al. 2018; Sabour et al. 2017] and skip connections [Mao et al. , Vol. 1, No. 1, Article 1. Publication date: January 2016. hotorealistic Material Editing Through Direct Image Manipulation • 1:9 Fig. 8. Results for three techniques on local image editing operations andimage mixing. The “reference material” labels showcase materials that canbe obtained using our shader and are used as source images for the materialsbelow them, where the arrows denote the evolution of the target image. t as an initial guess for ˜ t . In-scene editing still remains the keyadvantage of BRDF relighting techniques.We also note that our learning technique assumes an input shaderof dimensionality m and a renderer that is able to produce images ofthe materials that it encodes. In this work, our principled shader wasmeant to demonstrate the utility of this approach by showcasingintuitive workflows with the most commonly used BSDFs. However,this method needs not to be restricted to a classic principled BSDF,and is also expected to perform well on a rich selection of morespecialized material models including thin-film interference [Dias1991; Ikeda et al. 2015], fluorescence [Wilkie et al. 2001] birefrin-gence [Weidlich and Wilkie 2008], microfacet models [Heitz et al.2016] layered materials [Belcour 2018; Zeltner and Jakob 2018], andmore. We have presented a hybrid technique to empower novice usersand artists without expertise in photorealistic rendering to createsophisticated material models by applying image editing operationsto a source image. The resulting images are typically not achievablethrough photorealistic rendering, however, in many cases, solutionsbe found that are close to the desired output. Our learning-basedtechnique is able to take such an edited image and propose a photo-realistic material setup that produces a similar output, and provideshigh-quality results even in the presence of poorly-edited images.Our proposed method produces a reasonable initial guess and usesa neural network-augmented optimizer to fine-tune the parametersuntil the target image is matched as closely as possible. This hybridmethod is simple, robust, and its computation time is within 30seconds for every test case showcased throughout this paper. Thislow computation time is beneficial especially in the early phases ofthe material design process where a rapid iteration over a variety ofcompeting ideas is an important requirement (Fig 9). Our two keyinsights can be summarized as follows: • Normally, using an input image that was generated by aprincipled shader is not useful given that the user has togenerate this image themselves with a known parametersetup. However, our main idea is that the user can subjectthis image to raster editing operations and “pretend” thatthis input is achievable, and reliably infer a shader setup tomimic it. • Our neural networks can be combined with optimizersboth directly , i.e., by using an optimizer that invokes aneural renderer at every function evaluation step to speedup the convergence and indirectly by using a set of neuralnetworks network to endow the optimizer with a reasonableinitial guess (steps (cid:184) and (cid:185) in Fig. 2). This combinationresults in a two-stage sytem that opens up efficient materialediting workflows for artists without expertise in this area. , Vol. 1, No. 1, Article 1. Publication date: January 2016. :10 • Zsolnai-Feh´er et al. 2018
Furthermore, we proposed a simple extension to support predict-ing image sequences with a strict time budget of 1-2 seconds andbelieve this method will offer an appealing entry point for novicesinto world of photorealistic material modeling.
ACKNOWLEDGMENTS
We would like to thank Reynante Martinez for providing us thegeometry and some of the materials for the Paradigm (Fig. 1) andGenesis scenes (Fig. 3), ianofshields for the Liquify scene that servedas a basis for Fig. 9, Robin Marin for the material test scene, AndrewPrice and G´abor M´esz´aros for their help with geometry modeling,Fel´ıcia Zsolnai-Feh´er for her help improving our figures, ChristianFreude, David Ha, Philipp Erler and Adam Celarek for their usefulcomments. We also thank NVIDIA for providing the hardwareto train our neural networks. This work was partially funded byAustrian Science Fund (FWF), project number P27974.
A NEURAL NETWORK ARCHITECTURES
Below, we describe the neural network architectures we used toimplement ϕ − ( i ) . The Conv2D notation represents a 2D convolutionallayer with the appropriate number of filters , spatial kernel sizes and strides , where FC represents a dense, fully-connected layer with aprescribed number of neurons and dropout probability .(1) 2x { Conv2D(32,3,1), MaxPool(2,2) } –1x { Conv2D(64,3,1), MaxPool(2,2) } –2x { Conv2D(128,3,1), MaxPool(2,2) } –2x { FC(1000, 0.1) } - FC(m, 0.0)(2) 2x { Conv2D(32,3,1), MaxPool(2,2) } –2x { FC(1000, 0.1) } - FC(m, 0.0)(3) 2x { Conv2D(32,3,1), MaxPool(2,2) } –2x { FC(1000, 0.5) } - FC(m, 0.0)(4) 2x { Conv2D(32,3,1), MaxPool(2,2) } –1x { Conv2D(64,3,1), MaxPool(2,2) } –2x { Conv2D(128,3,1), MaxPool(2,2) } –2x { FC(3000, 0.5) } - FC(m, 0.0)(5) 2x { Conv2D(32,3,1), MaxPool(2,2) } –1x { Conv2D(64,3,1), MaxPool(2,2) } –2x { Conv2D(128,3,1), MaxPool(2,2) } –2x { FC(3000, 0.0) } - FC(m, 0.0)(6) 2x { Conv2D(32,3,1), MaxPool(2,2) } –2x { FC(1000, 0.0) } - FC(m, 0.0)(7) 2x { Conv2D(32,3,1), MaxPool(2,2) } –2x { FC(1000, 0.0) } - FC(m, 0.0)(8) 2x { Conv2D(32,3,1), MaxPool(2,2) } –2x { FC(100, 0.0) } - FC(m, 0.0)(9) 2x { Conv2D(32,3,1), MaxPool(2,2) } –2x { FC(1000, 0.0) } - FC(m, 0.0)Neural networks 6,7 and 9 are isomorphic and were run for adifferent number of epochs to test the effect of overfitting later in thetraining process, and therefore offer differing validation losses. Theimplementation of ϕ is equivalent to the one used in Zsolnai-Feh´eret al.’s work [2018]. B COMPARISON OF OPTIMIZERS
In Table 3, we have benchmarked several optimizers, i.e., L-BFGS-B [Byrd et al. 1995], SLSQP [Kraft 1994], the Conjugate Gradientmethod [Hestenes and Stiefel 1952] and found Nelder and Mead’ssimplex-based self-adapting optimizer [1965] to be the overall bestchoice for our global and local image-editing operations. For refer-ence, we also ran Basin-hopping [Wales and Doye 1997], a globalminimizer with a variety of parameter choices and a generous al-lowance of 30 minutes of execution time for each test case. Thismethod is useful for challenging non-linear optimization problemswith high-dimensional search spaces. Note that when being runfor long enough, this technique is less sensitive to initialization dueto the fact that it performs many quick runs from different start-ing points, and hence, we report one result for both initializationtechniques. The cells in the intersection of “Nelder-Mead” and “NN”denote our proposed hybrid method, which was often able to match,and in some cases, outperform this global minimization technique.
REFERENCES
Miika Aittala, Timo Aila, and Jaakko Lehtinen. 2016. Reflectance modeling by neuraltexture synthesis.
ACM Transactions on Graphics
35, 4 (2016), 65.Miika Aittala, Tim Weyrich, Jaakko Lehtinen, et al. 2015. Two-shot SVBRDF capturefor stationary materials.
ACM Transactions on Graphics
34, 4 (2015), 110–1.Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W Pellegrini,Ralf S Klessen, Lena Maier-Hein, Carsten Rother, and Ullrich K¨othe. 2018. Analyzinginverse problems with invertible neural networks. arXiv preprint arXiv:1808.04730 (2018).Laurent Belcour. 2018. Efficient Rendering of Layered Materials using an AtomicDecomposition with Statistical Operators.
ACM Transactions on Graphics
37, 4(2018), 1. https://doi.org/10.1145/3197517.3201289Irwan Bello, Barret Zoph, Vijay Vasudevan, and Quoc V Le. 2017. Neural optimizersearch with reinforcement learning. In
Proceedings of the 34th International Confer-ence on Machine Learning-Volume 70 . JMLR. org, 459–468.Aner Ben-Artzi, Kevin Egan, Fr´edo Durand, and Ravi Ramamoorthi. 2008. A precom-puted polynomial representation for interactive BRDF editing with global illumina-tion.
ACM Transactions on Graphics (TOG)
27, 2 (2008), 13.Aner Ben-Artzi, Ryan Overbeck, and Ravi Ramamoorthi. 2006. Real-time BRDF editingin complex lighting. In
ACM Transactions on Graphics , Vol. 25. ACM, 945–954.L´eon Bottou. 2010. Large-scale machine learning with stochastic gradient descent. In
Proceedings of COMPSTAT’2010 . Springer, 177–186.Brent Burley and Walt Disney Animation Studios. 2012. Physically-based Shading atDisney. In
ACM SIGGRAPH , Vol. 2012. 1–7.Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. 1995. A limited memoryalgorithm for bound constrained optimization.
SIAM Journal on Scientific Computing
16, 5 (1995), 1190–1208.Ewen Cheslack-Postava, Rui Wang, Oskar Akerlund, and Fabio Pellacini. 2008. Fast,realistic lighting and material design using nonlinear cut approximation. In
ACMTransactions on Graphics , Vol. 27. ACM, 128.Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and AdrienBousseau. 2018. Single-image SVBRDF capture with a rendering-aware deep net-work.
ACM Transactions on Graphics (TOG)
37, 4 (2018), 128.Maria Lurdes Dias. 1991. Ray tracing interference color.
IEEE Computer Graphics andApplications
Transactions on Graphics (Proceedings ofSIGGRAPH Asia) (Dec. 2018).Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2018. Neural architecturesearch: A survey. arXiv preprint arXiv:1808.05377 (2018).Gabriel Goh. 2017. Why momentum really works.
Distill
2, 4 (2017), e6.Milovˇs Haˇsan and Ravi Ramamoorthi. 2013. Interactive albedo editing in path-tracedvolumetric materials.
ACM Transactions on Graphics (TOG)
32, 2 (2013), 11.Eric Heitz, Johannes Hanika, Eugene d’Eon, and Carsten Dachsbacher. 2016. Multiple-scattering microfacet BSDFs with the Smith model.
ACM Transactions on Graphics(TOG)
35, 4 (2016), 58.Magnus Rudolph Hestenes and Eduard Stiefel. 1952.
Methods of conjugate gradients forsolving linear systems . Vol. 49. NBS.Geoffrey E Hinton, Sara Sabour, and Nicholas Frosst. 2018. Matrix capsules with EMrouting. (2018)., Vol. 1, No. 1, Article 1. Publication date: January 2016. hotorealistic Material Editing Through Direct Image Manipulation • 1:11
Fig. 9. Our technique is especially helpful early in the material design process where the user seeks to rapidly iterate over a variety of possible artistic effects.Both material types were synthesized using our described method. We demonstrate this workflow in our supplementary video.
Input Init. type Init. RMSE Nelder-Mead L-BFGS-B SLSQP CG Basin-hoppingFig. 5, Row 1 Rand 41.93 5.62 20.47 17.96 5.24
Fig. 5, Row 1 NN 5.94 2.37 5.84 5.94 5.94Fig. 5, Row 2 Rand 78.45 40.21 78.45 78.45 78.45
Fig. 5, Row 2 NN 32.72
Fig. 5, Row 7 NN 22.42
Fig. 8, Row 2 NN 12.79 7.80 12.79 12.79 12.79Fig. 8, Row 3 Rand 21.60 6.19 21.60 21.60 20.83 5.86Fig. 8, Row 3 NN 9.09
Fig. 8, Row 8 NN 9.74 5.36 9.61 9.61 9.68
Table 3. A comparison of a set of classical optimization techniques revealed that when using Nelder and Mead’s simplex-based optimizer with our “best of9” inversion network initialization, we can often match, and in some cases, outperform the results of Basin-hopping, a global minimizer. In the interest ofreadability, we have marked the cases where the optimizers were unable to improve upon the initial guess with red. For reference, the first two rows showcasean input image that is reproducible by the shader. , Vol. 1, No. 1, Article 1. Publication date: January 2016. :12 • Zsolnai-Feh´er et al. 2018
Sho Ikeda, Shin Watanabe, Bisser Raytchev, Toru Tamaki, and Kazufumi Kaneda. 2015.Spectral rendering of interference phenomena caused by multilayer films underglobal illumination environment.
ITE Transactions on Media Technology and Appli-cations
3, 1 (2015), 76–84.Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Dieter Kraft. 1994. Algorithm 733: TOMP–Fortran modules for optimal control calcula-tions.
ACM Transactions on Mathematical Software (TOMS)
20, 3 (1994), 262–281.Manuel Lagunas, Sandra Malpica, Ana Serrano, Elena Garces, Diego Gutierrez, andBelen Masia. 2019. A Similarity Measure for Material Appearance.
ACM Transactionson Graphics (SIGGRAPH 2019)
38, 4 (2019).Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. 2017. Modeling surface appearance froma single photograph using self-augmented convolutional neural networks.
ACMTransactions on Graphics (TOG)
36, 4 (2017), 45.Zhengqin Li, Kalyan Sunkavalli, and Manmohan Chandraker. 2018. Materials formasses: SVBRDF acquisition with a single mobile phone image. In
Proceedings ofthe European Conference on Computer Vision (ECCV) . 72–87.Xiaojiao Mao, Chunhua Shen, and Yu-Bin Yang. 2016. Image restoration using verydeep convolutional encoder-decoder networks with symmetric skip connections. In
Advances in neural information processing systems . 2802–2810.Stephen Robert Marschner and Donald P Greenberg. 1998.
Inverse rendering for computergraphics . Citeseer.Stephen R Marschner, Stephen H Westin, Eric PF Lafortune, Kenneth E Torrance, andDonald P Greenberg. 1999. Image-based BRDF measurement including human skin.In
Rendering Techniquesfi 99 . Springer, 131–144.Gary Mataev, Michael Elad, and Peyman Milanfar. 2019. DeepRED: Deep Image PriorPowered by RED. (2019). arXiv:cs.CV/1903.10176Wojciech Matusik. 2003.
A data-driven reflectance model . Ph.D. Dissertation. Mas-sachusetts Institute of Technology.Leo Miyashita, Kota Ishihara, Yoshihiro Watanabe, and Masatoshi Ishikawa. 2016. Zoe-Matrope: A system for physical material design. In
ACM SIGGRAPH 2016 EmergingTechnologies . ACM, 24.John A Nelder and Roger Mead. 1965. A simplex method for function minimization.
The computer journal
7, 4 (1965), 308–313.Ren Ng, Ravi Ramamoorthi, and Pat Hanrahan. 2004. Triple product wavelet integralsfor all-frequency relighting. In
ACM Transactions on Graphics (TOG) , Vol. 23. ACM,477–487.Steven J Nowlan and Geoffrey E Hinton. 1992. Simplifying neural networks by softweight-sharing.
Neural Computation
4, 4 (1992), 473–493.Marios Papas, Krystle de Mesa, and Henrik Wann Jensen. 2014. A Physically-BasedBSDF for Modeling the Appearance of Paper. In
Computer Graphics Forum , Vol. 33.Wiley Online Library, 133–142.Marios Papas, Christian Regg, Wojciech Jarosz, Bernd Bickel, Philip Jackson, WojciechMatusik, Steve Marschner, and Markus Gross. 2013. Fabricating translucent materialsusing continuous pigment mixtures.
ACM Transactions on Graphics (TOG)
32, 4(2013), 146.Gilles Rainer, Wenzel Jakob, Abhijeet Ghosh, and Tim Weyrich. 2019. Neural BTF Com-pression and Interpolation.
Computer Graphics Forum (Proceedings of Eurographics)
38, 2 (March 2019).Ravi Ramamoorthi and Pat Hanrahan. 2001. A signal-processing framework for inverserendering. In
Proceedings of the 28th annual conference on Computer graphics andinteractive techniques . ACM, 117–128.Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, JieTan, Quoc Le, and Alex Kurakin. 2017. Large-scale evolution of image classifiers. arXiv preprint arXiv:1703.01041 (2017).Herbert Robbins and Sutton Monro. 1951. A stochastic approximation method.
Theannals of mathematical statistics (1951), 400–407.Brigitte R¨oder, Oliver Stock, Siegfried Bien, Helen Neville, and Frank R¨osler. 2002.Speech processing activates visual cortex in congenitally blind humans.
EuropeanJournal of Neuroscience
16, 5 (2002), 930–936.Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing betweencapsules. In
Advances in Neural Information Processing Systems . 3856–3866.Thorsten-Walther Schmidt, Jan Novak, Johannes Meng, Anton S. Kaplanyan, Tim Reiner,Derek Nowrouzezahrai, and Carsten Dachsbacher. 2013. Path-Space Manipulationof Physically-Based Light Transport.
ACM Transactions on Graphics (Proceedings ofACM SIGGRAPH 2013)
32, 4 (Aug. 2013).Ana Serrano, Diego Gutierrez, Karol Myszkowski, Hans-Peter Seidel, and Belen Masia.2016. An intuitive control space for material appearance.
ACM Transactions onGraphics
35, 6 (2016), 186.Cyril Soler, Kartic Subr, and Derek Nowrouzezahrai. 2018. A Versatile Parameterizationfor Measured Material Manifolds. In
Computer Graphics Forum , Vol. 37. Wiley OnlineLibrary, 135–144.Ying Song, Xin Tong, Fabio Pellacini, and Pieter Peers. 2009. SubEdit: a representationfor editing measured heterogeneous subsurface scattering.
ACM Transactions onGraphics (TOG)
28, 3 (2009), 31. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and RuslanSalakhutdinov. 2014. Dropout: a simple way to prevent neural networks fromoverfitting.
Journal of Machine Learning Research
15, 1 (2014), 1929–1958.Xin Sun, Kun Zhou, Yanyun Chen, Stephen Lin, Jiaoying Shi, and Baining Guo. 2007.Interactive relighting with dynamic BRDFs.
ACM Transactions on Graphics
26, 3(2007), 27.Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. 2013. On the impor-tance of initialization and momentum in deep learning. In
International conferenceon machine learning . 1139–1147.David J Wales and Jonathan PK Doye. 1997. Global optimization by basin-hopping andthe lowest energy structures of Lennard-Jones clusters containing up to 110 atoms.
The Journal of Physical Chemistry A
The Visual Computer
24, 7-9 (2008), 565–575.Rui Wang, John Tran, and David P Luebke. 2004. All-Frequency Relighting of Non-Diffuse Objects using Separable BRDF Approximation.. In
Rendering Techniques .345–354.Andrea Weidlich and Alexander Wilkie. 2008. Realistic rendering of birefringency inuniaxial crystals.
ACM Transactions on Graphics (TOG)
27, 1 (2008), 6.Randall White. 1989. Visual thinking in the ice age.
Scientific American
Rendering Techniques 2001 . Springer,197–204.Tizian Zeltner and Wenzel Jakob. 2018.
Transactions on Graphics (Proceedings of SIG-GRAPH)
37, 4 (July 2018), 74:1–74:14. https://doi.org/10.1145/3197517.3201321Jun-Yan Zhu, Philipp Kr¨ahenb¨uhl, Eli Shechtman, and Alexei A Efros. 2016. Generativevisual manipulation on the natural image manifold. In
European Conference onComputer Vision . Springer, 597–613.Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcementlearning. arXiv preprint arXiv:1611.01578 (2016).Hui Zou and Trevor Hastie. 2005. Regularization and variable selection via the elasticnet.
Journal of the Royal Statistical Society: Series B (Statistical Methodology)
67, 2(2005), 301–320.K´aroly Zsolnai-Feh´er, Peter Wonka, and Michael Wimmer. 2018. Gaussian MaterialSynthesis.