Robert M. Toth

A sort-based deferred shading architecture for decoupled sampling

Stochastic sampling in time and over the lens is essential to produce photo-realistic images, and it has the potential to revolutionize real-time graphics. In this paper, we take an architectural view of the problem and propose a novel hardware architecture for efficient shading in the context of stochastic rendering. We replace previous caching mechanisms by a sorting step to extract coherence, thereby ensuring that only non-occluded samples are shaded. The memory bandwidth is kept at a minimum by operating on tiles and using new buffer compression methods. Our architecture has several unique benefits not traditionally associated with deferred shading. First, shading is performed in primitive order, which enables late shading of vertex attributes and avoids the need to generate a G-buffer of pre-interpolated vertex attributes. Second, we support state changes, e.g., change of shaders and resources in the deferred shading pass, avoiding the need for a single über-shader. We perform an extensive architectural simulation to quantify the benefits of our algorithm on real workloads.

Coarse pixel shading

We present a novel architecture for flexible control of shading rates in a GPU pipeline, and demonstrate substantially reduced shading costs for various applications. We decouple shading and visibility by restricting and quantizing shading rates to a finite set of screen-aligned grids, leading to simpler and fewer changes to the GPU pipeline compared to alternative approaches. Our architecture introduces different mechanisms for programmable control of the shading rate, which enables efficient shading in several scenarios, e.g., rendering for high pixel density displays, foveated rendering, and adaptive shading for motion and defocus blur. We also support shading at multiple rates in a single pass, which allows the user to compute different shading terms at rates better matching their frequency content.

Hierarchical stochastic motion blur rasterization

We present a hierarchical traversal algorithm for stochastic rasterization of motion blur, which efficiently reduces the number of inside tests needed to resolve spatio-temporal visibility. Our method is based on novel tile against moving primitive tests that also provide temporal bounds for the overlap. The algorithm works entirely in homogeneous coordinates, supports MSAA, facilitates efficient hierarchical spatio-temporal occlusion culling, and handles typical game workloads with widely varying triangle sizes. Furthermore, we use high-quality sampling patterns based on digital nets, and present a novel reordering that allows efficient procedural generation with good anti-aliasing properties. Finally, we evaluate a set of hierarchical motion blur rasterization algorithms in terms of both depth buffer bandwidth, shading efficiency, and arithmetic complexity.

Image Processing Techniques

Image processing includes those methods that operate on arrays of pixels to produce a modified image in which some of the information is enhanced in visibility and necessarily some other information is reduced. This is done presumably because the user can determine from the intended use of the image which information is currently important. In other situation, different processing might be used to extract other information from the image. There are several different ways to organize image processing methods to aid in understanding and to provide a logical structure for explanation. One is by purpose: to correct defects in images that arise from imperfect cameras, illumination, etc; to enhance the visibility of particular structures within images; to recognize and locate features; etc. In many cases, the same tools can be used for several of these different objectives. Another method is to distinguish between operations that work on single pixels, within small neighborhoods, and globally on the entire image. This article discussed single-pixel operations. Keywords: image processing; single pixel; operations; neighborhood operations; ranking; edge enhancement; texture; combinations; thresholding; frequency domains; convolution; correlation

AMFS: adaptive multi-frequency shading for future graphics processors

We propose a powerful hardware architecture for pixel shading, which enables flexible control of shading rates and automatic shading reuse between triangles in tessellated primitives. The main goal is efficient pixel shading for moderately to finely tessellated geometry, which is not handled well by current GPUs. Our method effectively decouples the cost of pixel shading from the geometric complexity. It thereby enables a wider use of tessellation and fine geometry, even at very limited power budgets. The core idea is to shade over small local grids in parametric patch space, and reuse shading for nearby samples. We also support the decomposition of shaders into multiple parts, which are shaded at different frequencies. Shading rates can be locally and adaptively controlled, in order to direct the computations to visually important areas and to provide performance scaling with a graceful degradation of quality. Another important benefit of shading in patch space is that it allows efficient rendering of distribution effects, which further closes the gap between real-time and offline rendering.

Adaptive image space shading for motion and defocus blur

We present a novel anisotropic sampling algorithm for image space shading which builds upon recent advancements in decoupled sampling for stochastic rasterization pipelines. First, we analyze the frequency content of a pixel in the presence of motion and defocus blur. We use this analysis to derive bounds for the spectrum of a surface defined over a two-dimensional and motion-aligned shading space. Second, we present a simple algorithm that uses the new frequency bounds to reduce the number of shaded quads and the size of decoupling cache respectively by 2X and 16X, while largely preserving image detail and minimizing additional aliasing.

Adaptive texture space shading for stochastic rendering

When rendering effects such as motion blur and defocus blur, shading can become very expensive if done in a naïve way, i.e. shading each visibility sample. To improve performance, previous work often decouple shading from visibility sampling using shader caching algorithms. We present a novel technique for reusing shading in a stochastic rasterizer. Shading is computed hierarchically and sparsely in an object‐space texture, and by selecting an appropriate mipmap level for each triangle, we ensure that the shading rate is sufficiently high so that no noticeable blurring is introduced in the rendered image. Furthermore, with a two‐pass algorithm, we separate shading from reuse and thus avoid GPU thread synchronization. Our method runs at real‐time frame rates and is up to 3 × faster than previous methods. This is an important step forward for stochastic rasterization in real time.

Efficient Depth of Field Rasterization Using a Tile Test Based on Half-Space Culling

For depth of field (DOF) rasterization, it is often desired to have an efficient tile versus triangle test, which can conservatively compute which samples on the lens that need to execute the sample‐in‐triangle test. We present a novel test for this, which is optimal in the sense that the region on the lens cannot be further reduced. Our test is based on removing half‐space regions of the (u, v) ‐space on the lens, from where the triangle definitely cannot be seen through a tile of pixels. We find the intersection of all such regions exactly, and the resulting region can be used to reduce the number of sample‐in‐triangle tests that need to be performed. Our main contribution is that the theory we develop provides a limit for how efficient a practical tile versus defocused triangle test ever can become. To verify our work, we also develop a conceptual implementation for DOF rasterization based on our new theory. We show that the number of arithmetic operations involved in the rasterization process can be reduced. More importantly, with a tile test, multi‐sampling anti‐aliasing can be used which may reduce shader executions and the related memory bandwidth usage substantially. In general, this can be translated to a performance increase and/or power savings.

Efficient bounding of displaced Bézier patches

In this paper, we present a new approach to conservative bounding of displaced Bézier patches. These surfaces are expected to be a common use case for tessellation in interactive and real-time rendering. Our algorithm combines efficient normal bounding techniques, min-max mipmap hierarchies and oriented bounding boxes. This results in substantially faster convergence for the bounding volumes of displaced surfaces, prior to tessellation and displacement shading. Our work can be used for different types of culling, ray tracing, and to sort higher order primitives in tiling architectures. For our hull shader implementation, we report performance benefits even for moderate tessellation rates.

Comparison of projection methods for rendering virtual reality

Virtual reality is rapidly gaining popularity, and may soon become a common way of viewing 3D environments. While stereo rendering has been performed on consumer grade graphics processors for a while now, the new wave of virtual reality display devices have two properties that typical applications have not needed to consider before. Pixels no longer appear on regular grids and the displays subtend a wide field-of-view. In this paper, we evaluate several techniques designed to efficiently render for head-mounted displays with such properties. We show that the amount of rendered pixels can be reduced down to 36% without compromising visual fidelity compared to traditional rendering, by rendering multiple optimized sub-projections.