Yukio Sakagawa

Shading and shadow casting in image-based rendering without geometric models

representation of complex objects without geometric models. Now the key to IBR future prosperity is how to accomplish functionalities of conventional geometry-based approaches. We have already reported how to interact with image-based objects in the virtual environment 3. The next goal is how to make objects without polygon models react to transition of lighting conditions. This sketch describes a real-time rendering method that changes the shading of image-based objects and casts appropriate shadows according to the motion of viewpoint or objects and the transition of a local lighting. Method A set of multiple pictures taken under a certain lighting condition while moving a camera is called an image-set. Our idea is based on a simple method to select the image-set with the nearest lighting condition to that of virtual environment from the image-sets taken under various lighting conditions. Let I c be the image-set taken under ambient light. An image-set I(l) stores only the luminance part of pictures of real objects taken under a lighting condition l. While changing values of l, the image data is stored into the corresponding image-set. To render an image from a given viewpoint, corresponding pixels are extracted from I(l) and I c. Then the chrominance part of I c and luminance part of I(l) are applied to the pixel in order to get the final shading. Note that as the technology of IBR is used in this process, an image seen from any point of view can be reproduced. The shadow cast by an object is represented by the texture mapping of the image contour, which is viewed from the light position, onto the surface the object resides on as shown in Fig. 1. Implementation A system called CyberMirage 99 is realized in which ray-based data are placed in the geometry space. A light source can be switched on and off, and be moved. Of course, translation and rotation of objects and observers walk-through are implemented. Note that all of these functions are realized in real-time. Fig. 2(a) shows an image of objects lit from a certain light source. It is noticed that the objects are appropriately shaded and cast an appropriate shadow. Fig. 2(b) is the image when an observer comes near to the objects and Fig. 2(c) is the image when an observer comes to another side of the objects. Fig. 2(d) and 2(e) are images when the light source is moved …

A method of shading and shadowing in image-based rendering

This paper presents a method to generate dynamic shading of image-based objects without geometric models. Since conventional rendering techniques cannot be used to render the shading of this type of objects, image processing operation, such as interpolation or combination of image data, is performed. However, the variation of the shading of objects is not linear. Therefore, complicated processing is required to generate the desired shading, and they may not be compatible with real-time systems. Then, to perform the variation of the shading of image-based objects in a real-time system, we adopted a very simple method. From a set of image-based data with several different light conditions we select the image-based data with the light condition that is the closest to that of the rendering time, and use it to reconstruct the image. However, as it is necessary to save a large number of image data in this method, the volume of data is huge. Then we developed a method to reduce the data volume while maintaining real-time rendering. In this paper, we present not only our method, but also its application in the real-time system CyberMirage.

Data compression and hardware implementation of ray-space rendering for interactive augmented virtuality

This article describes approaches to solve two drawbacks of ray-space representation: the large amount of data necessary to represent an object and the massive use of CPU to render an image. Ray-space representation, an image-based rendering technique, is used in our interactive augmented virtuality system. We developed a compression method optimized for ray-space data and a hardware architecture to render images from ray-space data. The compression method uses a hybrid combination of motion-compensated prediction, discrete cosine transform, and vector quantization. The proposed method compresses the data while assuring random and fast access to the decoded data. The dedicated hardware architecture for consumer PCs to interactively render photorealistic images using ray-space representation is also described. This hardware architecture is used to efficiently transfer the processing load from the CPU to the hardware. These two improvements help to use PCs that do not have much memory and CPU resources in applications such as an interactive virtual museum, in which the scenes are generated from both geometric model data and ray-space data.