Hideyuki Tamura

Mixed reality: future dreams seen at the border between real and virtual worlds

Mixed reality (MR) is a kind of virtual reality (VR) but a broader concept than augmented reality (AR), which augments the real world with synthetic electronic data. On the opposite side, there is a term, augmented virtuality (AV), which enhances or augments the virtual environment (VE) with data from the real world. Mixed reality covers a continuum from AR to AV. This concept embraces the definition of MR stated by Milgram and Kishino (1994). We describe some technical achievements we made in the Mixed Reality Project in Japan.

Gaze-directed adaptive rendering for interacting with virtual space

This paper presents a new method of rendering for interaction with 3D virtual space with the use of gaze detection devices. In this method, hierarchical geometric models of graphic objects are constructed prior to the rendering process. The rendering process first calculates the visual acuity, which represents the importance of a graphic object for a human operator, from the gaze position of the operator. Second, the process selects a level from the set of hierachical geometric models depending on the value of visual acuity. That is, a simpler level of detail is selected where the visual acuity is lower, and a more complicated level is used where it is higher. Then, the selected graphic models are rendered on the display. This paper examines three visual characteristics to calculate the visual acuity: the central/peripheral vision, the kinetic vision, and the fusional vision. The actual implementation and our testbed system are described, as well as the details of the visual acuity model.

MR Platform: a basic body on which mixed reality applications are built

This paper describes a platform package, called MR Platform, which we have been implementing for research and development of augmented reality technology and applications. This package includes a parallax-less stereo video see-through HMD and a software development kit (SDK) for a Linux PC environment. The SDK is composed of a C++ class library for making runtime MR applications and related utilities such as a camera calibration tool. By using the SDK, the following functions are available: capturing video, handling a six degree-of-freedom (DOF) sensor, image processing such as color detection, estimating head position and orientation, displaying the real world image, and calibrating sensor placement and camera parameters of two cameras mounted on the HMD.

Viewpoint-dependent stereoscopic display using interpolation of multiviewpoint images

This paper presents a novel approach to autostereoscopic display which can show viewpoint dependent images according to viewers movement. This display system is not supported by a specialized display device but consists of a computer, a head tracking device, and a popular binocular stereoscopic display. The keypoint of our approach is based on the idea that the interpolation and reconstruction of multi-viewpoint images can provide a viewer with unlimited number of images according to his/her smooth movement. After describing the basic concepts, algorithms, and problems of the proposed method, the very successful experimental results, which are achieved for a real scene, are shown.

Robot vision-based registration utilizing bird's-eye view with user's view

This paper describes new vision-based registration methods utilizing not only cameras on a users head-mounted display but also a birds-eye view camera that observes the user from an objective viewpoint. Two new methods, the line constraint method (LCM) and global error minimization method (GEM), are proposed. The former method reduces the number of unknown parameters concerning the users viewpoint by restricting it to be on the line of sight from the birds-eye view. The other method minimizes the sum of errors, which is the sum of the distance between the fiducials on the view and the calculated positions of them based on the current viewing parameters, for both the users view and the birds-eye view. The methods proposed here reduce the number of points that should be observed from the users viewpoint for registration, thus improving the stability. In addition to theoretical discussions, this paper demonstrates the effectiveness of our methods by experiments in comparison with methods that use only a users view camera or a birds-eye view camera.

A mixed reality system with visual and tangible interaction capability: application to evaluating automobile interior design

This paper presents a mixed reality (MR) system with tangible interface as well as visual fusion in an MR space. The sense of touch is given by physical objects on which computer generated imagery is accurately registered and superimposed. The proposed approach is especially useful in industrial design where digital mockups and physical mockups are thoroughly utilized.

The VIEW-Station environment: tools and architecture for a platform-independent image-processing workstation

An image processing workstation called the vision and image engineering workstation, or VIEW-Station, which as developed in order to merge the state-of-the-art software environment of a UNIX workstation and the computing power of the fast image processor, is described. The software system of the VIEW-Station is constructed hierarchically in order to allow device independence in porting to various hardware configurations and to achieve flexibility in developing application systems. An image computing language called V-Sugar, which provides specific data types and a functional notation, allowing image processing algorithms to be expressed simply, is described. The image memory management mechanism, V-server, and the window system extensions, VIEW-Windows, are described. They are integrated to provide an independent basis to hardware configurations.<<ETX>>

Vision and graphics in producing mixed reality worlds

This paper introduces prominent topics of our Mixed Reality (MR) project and states what kind of role computer vision and graphics play in the project. MR is a part of VR in broader sense, but it treats the physical space as well as the virtual space created in computers. Here we use the term mixed reality instead of often used augmented reality (AR). This is because MR is not only the mixture of real world and virtual world but also the mixture of AR and augmented virtuality (AV). In that sense, we first describe that there is no clear distinction between AR and AV, then introduce some research results at the early stage of our MR project.

The delaunay triangulation for accurate three-dimensional graphic model

This paper presents a method to construct the Delaunay triangulation for creating a polygon patch model, which is used as a graphic model for applications such as computer graphics and virtual reality. This method is based on the conventional incremental method to construct the Voronoi diagram, but has two characteristics that are especially suitable for a three-dimensional geometric modeling. One of these characteristics is that a point can be removed from the mesh. This feature is useful for the interactive modeling application in which points are often removed and moved. The other is that a line, which lies between two points, can be constrained to be an edge of the mesh. This is necessary for creating an accurate three-dimensional model. Without this feature, the mesh usually obtained has totally different three-dimensional structure from the real object. A process to apply this method to a radial range image is described as well as the details of this method. Experimental results and evaluation of this method are also shown.

Real 3 communication and aromatic group computing: HCI and CSCW research at Canon Media Technology Lab.

INTRODUCTION AND BACKGROUND The Media Technology Laboratory is one of Canon Inc.ts corporate research labs. Originally called the Information Systems Research Center, the laboratory changed its name when it started research and development in information media, Now, approximately 70 research scientists and engineers, some of them managers, are in charge oi mearch and development mainly in HCI, CSCW, and other fields, Their ‘range of expertise covers AI, natural language understanding, computer vision and graphics, computer architecture, and system software (OS, database, etc.).