Ralf Sarlette

Acquisition, Synthesis, and Rendering of Bidirectional Texture Functions

One of the main challenges in computer graphics is still the realistic rendering of complex materials such as fabric or skin. The difficulty arises from the complex meso structure and reflectance behavior defining the unique look‐and‐feel of a material. A wide class of such realistic materials can be described as 2D‐texture under varying light‐ and view direction, namely, the Bidirectional Texture Function (BTF). Since an easy and general method for modeling BTFs is not available, current research concentrates on image‐based methods, which rely on measured BTFs (acquired real‐world data) in combination with appropriate synthesis methods. Recent results have shown that this approach greatly improves the visual quality of rendered surfaces and therefore the quality of applications such as virtual prototyping. This state‐of‐the‐art report (STAR) will present the techniques for the main tasks involved in producing photo‐realistic renderings using measured BTFs in details.

Photo-realistic Rendering of Metallic Car Paint from Image-Based Measurements

State‐of‐the‐art car paint shows not only interesting and subtle angular dependency but also significant spatial variation. Especially in sunlight these variations remain visible even for distances up to a few meters and give the coating a strong impression of depth which cannot be reproduced by a single BRDF model and the kind of procedural noise textures typically used. Instead of explicitly modeling the responsible effect particles we propose to use image‐based reflectance measurements of real paint samples and represent their spatial varying part by Bidirectional Texture Functions (BTF). We use classical BRDF models like Cook‐Torrance to represent the reflection behavior of the base paint and the highly specular finish and demonstrate how the parameters of these models can be derived from the BTF measurements. For rendering, the image‐based spatially varying part is compressed and efficiently synthesized. This paper introduces the first hybrid analytical and image‐based representation for car paint and enables the photo‐realistic rendering of all significant effects of highly complex coatings.

Design and Implementation of Practical Bidirectional Texture Function Measurement Devices Focusing on the Developments at the University of Bonn

Understanding as well as realistic reproduction of the appearance of materials play an important role in computer graphics, computer vision and industry. They enable applications such as digital material design, virtual prototyping and faithful virtual surrogates for entertainment, marketing, education or cultural heritage documentation. A particularly fruitful way to obtain the digital appearance is the acquisition of reflectance from real-world material samples. Therefore, a great variety of devices to perform this task has been proposed. In this work, we investigate their practical usefulness. We first idey a set of necessary attributes and establish a general categorization of different designs that have been realized. Subsequently, we provide an in-depth discussion of three particular implementations by our work group, demonstrating advantages and disadvantages of different system designs with respect to the previously established attributes. Finally, we survey the existing literature to compare our implementation with related approaches.

Markerless 4 gestures 6 DOF real-time visual tracking of the human hand with automatic initialization

In this paper we present a novel computer vision based hand‐tracking technique, which is capable of robustly tracking 6+4DOF of the human hand in real‐time (at least 25 frames per second) with the help of 3 (or more) off‐the‐shelf consumer cameras. ‘6+4DOF’ means that the system can track the global pose (6 continuous parameters for translation and rotation) of 4 different gestures. A key feature of our system is its fully automatic real‐time initialization procedure, which, along with a sound tracking‐lost detector, makes the system fit for real‐world applications. Because of this, our method acts as an enabling technology for uncumbersome hand‐based 3D Human‐Computer‐Interaction (HCI). Previously, using the hand as an at least 6DOF input device involved the use of either datagloves or markers. Using our tracking we evaluated the use of the hand as an input device for two prevalent Virtual Reality applications: fly‐through exploration of a virtual world and a simple digital assembly simulation.

Exploitation of human shadow perception for fast shadow rendering

In this paper we describe an experiment to obtain information about the perceptual potential of the human visual system regarding shadow perception. Shadows play an important part for communicating spatial structures of objects to the observer. They are also essential for the overal realism of the rendered image. Unfortunately, most algorithms in computer graphics which are capable of producing realistic shadows are computationally expensive. The main idea behind the experiment is to use a simplified version of the shadow caster to generate hard and soft shadows, which would rapidly increase performance and to evaluate up to which degree a simplification is possible, without producing noticeable errors. Therefore, an experiment is performed, in which the test persons should mark the point of the just noticeable difference. First results show, that a mesh simplified to only 1% of its original complexity is capable to cast soft shadows that satisfy 90% of the test persons.

DOME II: a parallelized BTF acquisition system

Bidirectional Texture Functions (BTFs) provide a realistic depiction of the appearance of many real-world materials as they contain the spatially varying light scattering behavior of the material surface. Since editing of existing BTF data is still in its early stages, materials have to be measured from real-world samples. In contrast to the related Spatially Varying BRDFs (SVBRDFs), the reflectance information encoded in a BTF also includes non-local scattering effects and therefore does not obey energy conservation or reciprocity. While this higher degree of freedom also contributes to an increased realism, it inadvertently calls for an extensive measurement of reflectance samples, as many regularization approaches from BRDF measurement do not apply. In this paper, we present an automated, parallelized, robust, fast and transportable setup for the acquisition of BTFs from flat samples as well as 3D objects using camera and light arrays: the DOME II. In contrast to previous camera array approaches, the present setup, which is comprised of high-quality industry grade components, overcomes several issues regarding stability, reliability and precision. It achieves a well balanced state-of-the-art acquisition performance in terms of speed and quality at reasonable costs.

Data-driven Local Coordinate Systems for Image-Based Rendering

Image‐based representations of an object profit from known geometry. The more accurate this geometry is known, the better corresponding pixels in the different images can be aligned, which leads to less artifacts and better compression performance. For opaque objects the per‐pixel data can then be interpreted as a sampling of the BRDF at the respective surface point. In order to parameterize this sampled data a coordinate frame has to be defined. In previous work this coordinate frame was either the global frame or a local frame derived from the base geometry. Both approaches lead to misalignments between sample vectors: Features of basically very similar BRDFs will be shifted to different regions in the sample vector leading to poor compression performance. In order to improve alignment between the sampled BRDFs in image‐based rendering, we propose an optimization algorithm which determines consistent coordinate frames for every sample point on the object surface. This way we efficiently align the features even of anisotropic reflection functions and reconstruct approximate local coordinate frames without performing an explicit 3D‐reconstruction. The optimization is calculated efficiently by exploiting the Fourier‐shift theorem for spherical harmonics. In order to deal with different materials in a scene, the technique is combined with a clustering algorithm. We demonstrate the utility of our method by applying it to BTFs and 6D surface reflectance fields.

Procedural editing of bidirectional texture functions

Measured material representations like Bidirectional Texture Functions or Reflectance Fields offer very realistic appearance but the user is currently not capable of changing this appearance in an effective and intuitive way. Such editing operations would require a low-dimensional but expressive model for appearance that exposes only a small set of intuitively editable parameters (1D-sliders, 2D-maps) to the user but preserves all visually relevant details. In this paper we present a novel editing technique for complex spatially varying materials. It is based on the observation that we are already good in modeling the basic geometric structure of many natural and manmade materials but still have not found effective models for the detailed small-scale geometry and the interaction of light with these materials. Our main idea is to use procedural geometry to define the basic structure of a material and then to enrich this structure with the BTF information captured from real materials. By employing recent algorithms for real-time texture synthesis and BTF compression our technique allows interactive editing.

Realtime Shading of Folded Surfaces

In this paper we present a new, simple, and efficient way to illuminate folded surfaces with extended light sources in realtime including shadows. In a preprocessing step we compute the parts of the surrounding environment that are visible from a set of points on the surface and represent this information in a binary visibility map.

Capturing shape and reflectance of food

Photo-realistic 3D content is crucial for creating convincing digital images. For certain classes of everyday objects, such as food, the human perception is sensitive to even small inconsistencies, making the creation of such content difficult and time-consuming, even for experts. In this sketch, we will explore the use of an automated pipeline for capturing 3D shape and Bidirectional Texture Function of food. The acquired data is used to render photo-realistic images of purely virtual objects under arbitrary lighting and with full global illumination.