Gero Müller

Acquisition, Synthesis, and Rendering of Bidirectional Texture Functions

One of the main challenges in computer graphics is still the realistic rendering of complex materials such as fabric or skin. The difficulty arises from the complex meso structure and reflectance behavior defining the unique look‐and‐feel of a material. A wide class of such realistic materials can be described as 2D‐texture under varying light‐ and view direction, namely, the Bidirectional Texture Function (BTF). Since an easy and general method for modeling BTFs is not available, current research concentrates on image‐based methods, which rely on measured BTFs (acquired real‐world data) in combination with appropriate synthesis methods. Recent results have shown that this approach greatly improves the visual quality of rendered surfaces and therefore the quality of applications such as virtual prototyping. This state‐of‐the‐art report (STAR) will present the techniques for the main tasks involved in producing photo‐realistic renderings using measured BTFs in details.

Photo-realistic Rendering of Metallic Car Paint from Image-Based Measurements

State‐of‐the‐art car paint shows not only interesting and subtle angular dependency but also significant spatial variation. Especially in sunlight these variations remain visible even for distances up to a few meters and give the coating a strong impression of depth which cannot be reproduced by a single BRDF model and the kind of procedural noise textures typically used. Instead of explicitly modeling the responsible effect particles we propose to use image‐based reflectance measurements of real paint samples and represent their spatial varying part by Bidirectional Texture Functions (BTF). We use classical BRDF models like Cook‐Torrance to represent the reflection behavior of the base paint and the highly specular finish and demonstrate how the parameters of these models can be derived from the BTF measurements. For rendering, the image‐based spatially varying part is compressed and efficiently synthesized. This paper introduces the first hybrid analytical and image‐based representation for car paint and enables the photo‐realistic rendering of all significant effects of highly complex coatings.

Rapid synchronous acquisition of geometry and appearance of cultural heritage artefacts

In order to produce visually appealing digital models of cultural heritage artefacts, a meticulous reconstruction of the 3D geometry alone is often not sufficient, as colour and reflectance information give essential clues of the objects material. Standard texturing methods are often only able to overcome this fact under strict material and lighting condition limitations. The realistic reconstruction of complex yet frequently encountered materials such as fabric, leather, wood or metal is still a challenge. In this paper, we describe a novel system to acquire the 3Dgeometry of an object using its visual hull, recorded in multiple 2D images with a multi-camera array. At the same time, the material properties of the object are measured into Bidirectional Texture Functions (BTF), that faithfully capture the mesostructure of the surface and reconstruct the look-and-feel of its material. The high rendering fidelity of the acquired BTF texture data with respect to reflectance and self-shadowing also alleviates the limited precision of the visual hull approach for 3D geometry acquisition.

Reflectance field based real-time, high-quality rendering of bidirectional texture functions

Abstract The bidirectional texture function (BTF) is a suitable representation for the appearance of highly detailed surface structures under varying illumination and viewing conditions. Since real-time rendering of the full BTF data is currently not feasible, approximations of the six-dimensional BTF are used such that the amount of data is reduced and current graphics hardware can be exploited. While existing methods work well for materials with low-depth variation, realism is lost if the depth variation grows. In this article we analyze this problem and devise a new real-time rendering paradigm based on linear interpolation of reflection fields, which provides significant improvements with respect to realism for such highly structured materials without sacrificing the general applicability and speed of previous algorithms. We propose real-time rendering algorithms for this new method supporting either point light sources or image-based lighting and demonstrate the capabilities of our new approach with several examples.

Fast environmental lighting for local-PCA encoded BTFs

Rendering geometric models with complex surface materials in arbitrary lighting environments is a challenging problem. In order to relight and render geometries covered with complex, measured BTFs two problems have to be addressed: the memory problem resulting from the large size of the measured BTF data and the light integration problem resulting from summing up the contributions from all measured light-sources. In this paper we describe how highly efficient BTF compression methods like local-PCA and suitable representations of environmental light based on spherical harmonics can be combined leading to fast environmental lighting for efficiently encoded BTFs. As a side effect the method supports precomputed radiance transfer

BTF-CIELab: A Perceptual Difference Measure for Quality Assessment and Compression of BTFs

Driven by the advances in lossy compression of bidirectional texture functions (BTFs), there is a growing need for reliable methods to numerically measure the visual quality of the various compressed representations. Based on the CIE ΔE00 colour difference equation and concepts of its spatio‐temporal extension ST‐CIELab for video quality assessment, this paper presents a numerical quality measure for compressed BTF representations. By analysing the BTF in its full six‐dimensional (6D) space, light and view transition effects are integrated into the measure. In addition to the compressed representation, the method only requires the source BTF images as input and thus aids the objective evaluation of different compression techniques by means of a simple numerical comparison. By separating the spatial and angular components of the difference measure and linearizing each of them, the measure can be incorporated into any linear or multi‐linear compression technique. Using a per‐colour‐channel principal component analysis (PCA), compression rates of about 500:1 can be achieved at excellent visual quality.

Verification of rendering quality from measured BTFs

One of the most important, still unsolved problems in computer graphics is the generation of predictive imagery, i.e., images that represent perfect renditions of reality. Such perfect images are required in application areas like Virtual Prototyping for making reliable decisions in the costly design development of novel products like cars and airplanes. Recently, measured material properties received significant attention since they enable generation of highly accurate images that appear to be predictive at a first glance.In this work we investigate the degree of realism that can be achieved using measured bidirectional texture functions (BTFs) by comparing photographs and rendered images at two scales. To analyze the realism of rendered images at a coarse scale, we compare the light distribution resulting from standard materials to the one from measured BTFs by automatic procedures. At a fine scale, accurate reproduction of material structures is checked by a psychophysical study. Our results show that measured BTFs lead to much more accurate results than standard materials at both scales.

Fractional Fourier Texture Masks: Guiding Near-Regular Texture Synthesis

Recently, the special kind of near-regular texture has drawn significant attention from researchers in the field of texture synthesis. Near-regular textures contain global regular structures that pose significant problems to the popular sample-based approaches, and irregular stochastic structures that can not be reproduced by simple tiling. Existing work tries to overcome this problem by user assisted modeling of the regular structures and then relies on regular tiling. In this paper we use the concept of fractional Fourier analysis to perform a fully automatic separation of the global regular structure from the irregular structure. The actual synthesis is performed by generating a fractional Fourier texture mask from the extracted global regular structure which is used to guide the synthesis of irregular texture details. Our new method allows for automatic and efficient synthesis of a wide range of near-regular textures preserving their regular structures and faithfully reproducing their stochastic elements.

Data-driven Local Coordinate Systems for Image-Based Rendering

Image‐based representations of an object profit from known geometry. The more accurate this geometry is known, the better corresponding pixels in the different images can be aligned, which leads to less artifacts and better compression performance. For opaque objects the per‐pixel data can then be interpreted as a sampling of the BRDF at the respective surface point. In order to parameterize this sampled data a coordinate frame has to be defined. In previous work this coordinate frame was either the global frame or a local frame derived from the base geometry. Both approaches lead to misalignments between sample vectors: Features of basically very similar BRDFs will be shifted to different regions in the sample vector leading to poor compression performance. In order to improve alignment between the sampled BRDFs in image‐based rendering, we propose an optimization algorithm which determines consistent coordinate frames for every sample point on the object surface. This way we efficiently align the features even of anisotropic reflection functions and reconstruct approximate local coordinate frames without performing an explicit 3D‐reconstruction. The optimization is calculated efficiently by exploiting the Fourier‐shift theorem for spherical harmonics. In order to deal with different materials in a scene, the technique is combined with a clustering algorithm. We demonstrate the utility of our method by applying it to BTFs and 6D surface reflectance fields.