Sho Kamuro

Design of TELESAR V for transferring bodily consciousness in telexistence

This paper focuses on design of a dexterous anthropomorphic robot where the operator can perceive the transferring bodily consciousness to the slave robot during a tele-operation. Accordingly, we propose a telexistence surrogate anthropomorphic robot called “TELESAR V”, which was designed and constructed by development of the following: a 52 DOF slave robot with a torso, upper limbs, hands and head to model the operators posture on all parts of the upper body and maintain a 6 DOF accuracy in arm endpoint; a HD Head mounted display with 6 DOF point of view accuracy for wide angle stereovision; and a mechanism for sensing and reproducing fingertip haptic and thermal sensation. This paper describes the development of the TELESAR V system, where the effectiveness has been verified through functional experiments.

GhostGlove: haptic existence of the virtual world

In the real world, we usually do not perceive haptic sensations consciously. However, if the haptic sensations are absent or if they are not of the desired quality, it would result in discomfort, and the existence of the world would be degraded. In order to eliminate the feelings of discomfort and represent genuine feelings of the existence of objects in virtual reality interactions, as shown in figure 2, we show that haptic sensations play a decisive role in inciting cognitive functions in a virtual environment. There are many researches on haptic displays for virtual reality interactions with the hand, such as CyberTouch and CyberGrasp [Immersion Corp.]. However, the represented sensation is still inadequate to provide us with the experience of feeling the definite existence of virtual objects or virtual creatures. Generally, it is considered that a complex and expensive device is required to provide realistic haptic sensations; many attempts to simplify the device would result in poor sensations, for example, simple vibrations. To merge the simplicity of the device and realistic sensation of existence, we propose a novel wearable haptic interface named “GhostGlove.” This device generates natural and realistic haptic sensations over the entire hand—on each finger and the palm—and integrates the perceptions on the entire hand along with the visual sensations to enables us to recognize the existence of the virtual world.

RePro3D: full-parallax 3D display with haptic feedback using retro-reflective projection technology

We propose a novel full-parallax three-dimensional (3D) display system-RePro3D-that is suitable for interactive 3D applications with haptic feedback.Our approach is based on the retro-reflective projection technology in which several images projected from a projector array are displayed on a retro-reflective screen. When viewers view the screen through a half mirror, they see, without the aid of glasses, a 3D image superimposed in real space. RePro3D has a sensor function that recognizes user input; therefore, it can support some interactive features such as manipulation of 3D objects. In addition, a wearable haptic device, which is a part of our system, provides the user with a sensation of having touched the 3D image. In this paper, we describe the optical system of the high-density projector array used in RePro3D. Then, we describe the development of a prototype of RePro3D. The prototype is used to demonstrate that our system displays full-parallax images superimposed in real space from 42 different viewpoints. The proposed system enables a user to physically interact with the 3D image with haptic feedback.

Ungrounded kinesthetic pen for haptic interaction with virtual environments

We proposed a pen-shaped handheld haptic display that provides kinesthetic sensations to the fingers of a user without the use of mechanical linkages. The users movements are not restricted since the device does not have mechanical linkages, and the user can enjoy haptic interactions with virtual environments. In order to downsize the device, we designed our device on the basis of a hypothesis that kinesthetic sensations on the users fingers alone are sufficient to represent the sensations of touch. We implemented a prototype device and performed an experiment to confirm the representational ability of our device. We also developed a prototype haptic augmented reality system, using which the user can see and touch a computer graphics object.

Haptic transmission system to recognize differences in surface textures of objects for telexistence

In this paper, we propose a haptic transmission system for telexistence to improve the ability to sense the presence of remote objects. This system can transmit information about the existence and surface textures of objects in remote locations. The system consists of a conjugated haptic sensor and display. The sensor on the robots finger detects the pressure, vibration, and temperature of a remote object, and the display provides above information on the operators finger. Based on this information, the operator can understand what he/she is touching and whether its surface is hard or soft, cold or hot, and smooth or rough. With the use of our system, the operator can recognize the difference between objects such as silk and denim.

RePro3D: full-parallax 3D display using retro-reflective projection technology

Motion parallax is important to recognize the depth of a 3D image. In recent years, many 3D display methods that enable parallax images to be seen with the naked eye have been developed. In addition, there has been an increase in research to design interfaces that enable humans to intuitively interact with and operate 3D objects using their hands. However, realizing 3D object interaction as if the user is actually touching the object in the real world is quite difficult. One of the reasons for this is that the screen shape in conventional methods is restricted to a flat panel. In addition, it is difficult to achieve a balance between displaying the 3D image and sensing the user input. Therefore, we propose a novel full-parallax 3D display system that is suitable for interactive 3D applications. We call this system RePro3D. Our approach is based on a retro-reflective projection technology[Inami et al. 2000]. A number of images from a projector array are projected onto the retro-reflective screen. When a user looks at the screen through a half mirror, he or she, without the use of glasses, can view a 3D image that has motion parallax. We can choose the screen shape depending on the application. Image correction according to the screen shape is not required. Consequently, we can design a touch-sensitive soft screen, a complexly curved screen, or a screen with an automatically moving surface. RePro3D has a sensor function to recognize the user input. Some interactive features, such as operation of 3D objects, can be achieved by using it.

An ungrounded pen-shaped kinesthetic display: Device construction and applications

In this paper, we implement an ungrounded pen-shaped kinesthetic display and construct a three-dimensional (3D) haptic interaction system. The ungrounded pen-shaped kinesthetic display provides kinesthetic sensations to a users fingers without the use of mechanical linkages. Therefore, the user can move his/her hand freely in air and interact with virtual environments with sensation of touching. We verified the ability of the device to provide forces and to represent a virtual surface for 3D input. Then we constructed a 3D haptic interaction system in that the user can touch virtual objects displayed as 3D images directly. We also applied the device to an interface for 3D modeling and constructed a 3D haptic modeling system in that the user can create touchable 3D models by sketching two-dimensional (2D) figures in air.

HAPMAP: haptic walking navigation system with support by the sense of handrail

In this study, we propose a method of displaying unaware-usage haptic sensation of navigation. People view a map when they visit an unfamiliar place, but when they are taken away their eyes from the map, their attention is diverted and margin of the heart. However, when we are relaxed and do not worry about getting lost, we can discover the intrinsic beauty of the unfamiliar land. Therefore, we have focused on the sense of touch and ensured security by using a support such as a wall or a handrail in the streets, along with the sense of touch. Therefore, we propose a haptic navigation system to release human eyes from the requirement of constantly looking into a map in order to enhance the experience of our daily walk or sightseeing.

Haptic duplicator

We propose a haptic interface for touching 3DCG models based on the tactile copy & paste technology. In case of conventional systems for providing haptic sensation of touching virtual objects, it is challenging to reproduce realistic surface textures of objects. By scanning vibrotactile stimulation as audio signals, we realized recording of textures of real objects and reproducing of them with a wearable haptic glove with force and vibrotactile feedbacks. In our system, the user can appreciate a number of virtual objects with various scanned textures, and also, change their textures by haptic copy & paste from real objects to virtual 3DCG models.

A Palm-Worn Haptic Display for Bimanual Operations in Virtual Environments

We propose a wearable haptic display that indicates the pressure and vibration on the palm for bimanual operations in virtual reality environments. This system aims to provide the touch and stroke sensations of virtual objects or virtual creatures. We constructed a prototype device that can reproduce vertical and shearing forces on the palm and evaluate the capability of the proposed method to recognize the existence of a virtual object in one-handed and two-handed operations.