Computer Science

This chapter describes how to use intersection and closest-hit shaders to implement real-time visualizations of complex fractals using distance functions. The Mandelbulb and Julia Sets are used as examples.

We address the problem of accelerating thin-shell deformable object simulations by dimension reduction. We present a new algorithm to embed a high-dimensional configuration space of deformable objects in a low-dimensional feature space, where the configurations of objects and feature points have approximate one-to-one mapping. Our key technique is a graph-based convolutional neural network (CNN) defined on meshes with arbitrary topologies and a new mesh embedding approach based on physics-inspired loss term. We have applied our approach to accelerate high-resolution thin shell simulations corresponding to cloth-like materials, where the configuration space has tens of thousands of degrees of freedom. We show that our physics-inspired embedding approach leads to higher accuracy compared with prior mesh embedding methods. Finally, we show that the temporal evolution of the mesh in the feature space can also be learned using a recurrent neural network (RNN) leading to fully learnable physics simulators. After training our learned simulator runs 500−10000× faster and the accuracy is high enough for robot manipulation tasks.

We propose a reduced-order simulation and optimization technique for a type of digital materials which we denote as geometric meta-materials. They are planar cellular structures, which can be fabricated in 2d and folded in 3d space and thus well shaped into sophisticated 3d surfaces. They obtain their elasticity attributes mainly from the geometry of their cellular elements and their connections. While the physical properties of the base material (i.e., the physical substance) of course influence the behavior as well, our goal is to factor them out. However, the simulation of such complex structures still comes with a high computational cost. We propose an approach to reduce this computational cost by abstracting the meso-structures and encoding the properties of their elastic deformation behavior into a different set of material parameters. We can thus obtain an approximation of the deformed pattern by simulating a simplified version of the pattern using the computed material parameters.

Relighting of human images has various applications in image synthesis. For relighting, we must infer albedo, shape, and illumination from a human portrait. Previous techniques rely on human faces for this inference, based on spherical harmonics (SH) lighting. However, because they often ignore light occlusion, inferred shapes are biased and relit images are unnaturally bright particularly at hollowed regions such as armpits, crotches, or garment wrinkles. This paper introduces the first attempt to infer light occlusion in the SH formulation directly. Based on supervised learning using convolutional neural networks (CNNs), we infer not only an albedo map, illumination but also a light transport map that encodes occlusion as nine SH coefficients per pixel. The main difficulty in this inference is the lack of training datasets compared to unlimited variations of human portraits. Surprisingly, geometric information including occlusion can be inferred plausibly even with a small dataset of synthesized human figures, by carefully preparing the dataset so that the CNNs can exploit the data coherency. Our method accomplishes more realistic relighting than the occlusion-ignored formulation.

We consider the scattering of light in participating media composed of sparsely and randomly distributed discrete particles. The particle size is expected to range from the scale of the wavelength to the scale several orders of magnitude greater than the wavelength, and the appearance shows distinct graininess as opposed to the smooth appearance of continuous media. One fundamental issue in physically-based synthesizing this appearance is to determine necessary optical properties in every local region. Since these optical properties vary spatially, we resort to geometrical optics approximation (GOA), a highly efficient alternative to rigorous Lorenz-Mie theory, to quantitatively represent the scattering of a single particle. This enables us to quickly compute bulk optical properties according to any particle size distribution. Then, we propose a practical Monte Carlo rendering solution to solve the transfer of energy in discrete participating media. Results show that for the first time our proposed framework can simulate a wide range of discrete participating media with different levels of graininess and converges to continuous media as the particle concentration increases.

This paper introduces a method of calculating and rendering shapes in a non-Euclidean 2D space. In order to achieve this, we developed a physics and graphics engine that uses hyperbolic trigonometry to calculate and subsequently render the shapes in a 2D space of constant negative or positive curvature in real-time. We have chosen to use polar coordinates to record the parameters of the objects as well as an azimuthal equidistant projection to render the space onto the screen because of the multiple useful properties they have. For example, polar coordinate system works well with trigonometric calculations, due to the distance from the reference point (analogous to origin in Cartesian coordinates) being one of the coordinates by definition. Azimuthal equidistant projection is not a typical projection, used for neither spherical nor hyperbolic space, however one of the main features of our engine relies on it: changing the curvature of the world in real-time without stopping the execution of the application in order to re-calculate the world. This is due to the projection properties that work identically for both spherical and hyperbolic space, as can be seen in the Figure 1 above. We will also be looking at the complexity analysis of this method as well as renderings that the engine produces. Finally we will be discussing the limitations and possible applications of the created engine as well as potential improvements of the described method.

We propose a compute shader based point cloud rasterizer with up to 10 times higher performance than classic point-based rendering with the GL_POINT primitive. In addition to that, our rasterizer offers 5 byte depth-buffer precision with uniform or customizable distribution, and we show that it is possible to implement a high-quality splatting method that blends together overlapping fragments while still maintaining higher frame-rates than the traditional approach.

We propose a method to realistically insert synthetic objects into existing photographs without requiring access to the scene or any additional scene measurements. With a single image and a small amount of annotation, our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene. We demonstrate in a user study that synthetic images produced by our method are confusable with real scenes, even for people who believe they are good at telling the difference. Further, our study shows that our method is competitive with other insertion methods while requiring less scene information. We also collected new illumination and reflectance datasets; renderings produced by our system compare well to ground truth. Our system has applications in the movie and gaming industry, as well as home decorating and user content creation, among others.

We present a technique for rendering highly complex 3D scenes in real-time by generating uniformly distributed points on the scene's visible surfaces. The technique is applicable to a wide range of scene types, like scenes directly based on complex and detailed CAD data consisting of billions of polygons (in contrast to scenes handcrafted solely for visualization). This allows to visualize such scenes smoothly even in VR on a HMD with good image quality, while maintaining the necessary frame-rates. In contrast to other point based rendering methods, we place points in an approximated blue noise distribution only on visible surfaces and store them in a highly GPU efficient data structure, allowing to progressively refine the number of rendered points to maximize the image quality for a given target frame rate. Our evaluation shows that scenes consisting of a high amount of polygons can be rendered with interactive frame rates with good visual quality on standard hardware.

Regions of Interest (ROI) contain morphological features in pathology whole slide images (WSI) are delimited with polygons[1]. These polygons are often represented in either a textual notation (with the array of edges) or in a binary mask form. Textual notations have an advantage of human readability and portability, whereas, binary mask representations are more useful as the input and output of feature-extraction pipelines that employ deep learning methodologies. For any given whole slide image, more than a million cellular features can be segmented generating a corresponding number of polygons. The corpus of these segmentations for all processed whole slide images creates various challenges for filtering specific areas of data for use in interactive real-time and multi-scale displays and analysis. Simple range queries of image locations do not scale and, instead, spatial indexing schemes are required. In this paper we propose using Hilbert Curves simultaneously for spatial indexing and as a polygonal ROI representation. This is achieved by using a series of Hilbert Curves[2] creating an efficient and inherently spatially-indexed machine-usable form. The distinctive property of Hilbert curves that enables both mask and polygon delimitation of ROIs is that the elements of the vector extracted ro describe morphological features maintain their relative positions for different scales of the same image.