Charith Lasantha Fernando

Design of TELESAR V for transferring bodily consciousness in telexistence

This paper focuses on design of a dexterous anthropomorphic robot where the operator can perceive the transferring bodily consciousness to the slave robot during a tele-operation. Accordingly, we propose a telexistence surrogate anthropomorphic robot called “TELESAR V”, which was designed and constructed by development of the following: a 52 DOF slave robot with a torso, upper limbs, hands and head to model the operators posture on all parts of the upper body and maintain a 6 DOF accuracy in arm endpoint; a HD Head mounted display with 6 DOF point of view accuracy for wide angle stereovision; and a mechanism for sensing and reproducing fingertip haptic and thermal sensation. This paper describes the development of the TELESAR V system, where the effectiveness has been verified through functional experiments.

An operating method for a bipedal walking robot for entertainment

Tele-existence applications for robotic systems are becoming popular and widespread. Tey enable users to control a remote machine while experiencing a sense of being in the remote location. Initially, tele-existence was used for remote de-mining and mission-critical tasks in space, to avoid risking human life. Recently it has been applied in many entertainment and gaming applications, to enable a community to play together in one virtual environment and share the experience. But existing tele-existence systems require a large-scale interface, a lot of processing power, and a large space for proper operation.

Huggy pajama: a parent and child hugging communication system

Huggy Pajama is a novel wearable system aimed at promoting physical interaction in remote communication between parent and child. This system enables parents and children to hug one another through a novel hugging interface device and a wearable, hug reproducing pajama connected through the Internet. The hugging device is a small, mobile doll with an embedded pressure sensing circuit that is able to accurately sense varying levels of the range of human force produced from natural touch. This device sends hug signals to a haptic jacket that simulates the feeling of being hugged to the wearer. It features air pressure actuation to reproduce hug.

Haptic transmission system to recognize differences in surface textures of objects for telexistence

In this paper, we propose a haptic transmission system for telexistence to improve the ability to sense the presence of remote objects. This system can transmit information about the existence and surface textures of objects in remote locations. The system consists of a conjugated haptic sensor and display. The sensor on the robots finger detects the pressure, vibration, and temperature of a remote object, and the display provides above information on the operators finger. Based on this information, the operator can understand what he/she is touching and whether its surface is hard or soft, cold or hot, and smooth or rough. With the use of our system, the operator can recognize the difference between objects such as silk and denim.

Graphical instruction for a garment folding robot

This project proposes to use interactive graphical editing interface for an end user to give instructions to intelligent robots to complete a real world object manipulation task. Natural language is often considered as an ideal communication method for robots, but it not intuitive at specifying tasks that require visual (geometry) information. Learning from demonstration can be useful, but it is not easy to generalize a provided example into a working program. Our approach is to provide a specialized graphical editor that abstracts the target task and to have the user specify how to complete the task by performing simple editing operations (clicking and dragging). We show the effectiveness of our approach by building and testing an example application based on this concept, which is a graphical editor for teaching garment folding to a robot. This example shows that our approach is particularly effective for an end user to configure the robot behavior to satisfy their own needs, which cannot be covered by a single, pre-programmed solution for general audience.

Enforced telexistence: teleoperating using photorealistic virtual body and haptic feedback

Telexistence [Tachi 2010] systems require physical limbs for remote object manipulation [Fernando et al. 2012]. Having arms and hands synchronized with voluntary movements allows the user to feel robots body as his body through visual, and haptic sensation. In this method, we introduce a novel technique that provides virtual arms for existing telexistence systems that does not have physical arms. Previous works [Mine et al. 1997; Poupyrev et al. 1998; Nedel et al. 2003] involved the study of using virtual representation of user hands in virtual environments for interactions. In this work, the virtual arms serves for several interactions in a physical remote environment, and most importantly they provide the user the sense of existence in that remote environment. These superimposed virtual arms follows the users real-time arm movements and reacts to the dynamic lighting of real environment providing photorealistic rendering adapting to remote place lighting. Thus, it allows the user to have an experience of embodied enforcement towards the remote environment. Furthermore, these virtual arms can be extended to touch and feel unreachable remote objects, and to grab a functional virtual copy of a physical instance where device control is possible. This method does not only allow the user to experience a non-existing arm in telexistence, but also gives the ability to enforce remote environment in various ways.

Telexistence — from 1980 to 2012

Telexistence technology enables a highly realistic sensation of existence in a remote place without any actual travel. The concept was originally proposed by the first author in 1980, and its feasibility has been demonstrated through the construction of alter-ego robot systems such as TELESAR & TELESAR V, which were developed under the national large scale project on “Robots in Hazardous Environments” and the “CREST Haptic Telexistence Project.” A mutual telexistence system, such as TELESAR II & IV, capable of generating the sensation of being in a remote place in local space using a combination of an alter-ego robot and retro-reflective projection technology (RPT), has been developed, and the feasibility of mutual telexistence has been demonstrated. Thirty-two years of telexistence development are historically reviewed in this jubilee video.

Real-time egocentric superimposition of operator's own body on telexistence avatar in virtual environment

During teleoperation manipulation, the synchronization of the user behavior and a remote avatar is important to deliver the sensation of being in that remote place. Current telexistence technologies allow full upper body posture synchronization through multi-DOF humanoid robot structures and allow the operator to control the remote body as his own. However, it does not preserve a consistent feedback, such as the human like skin tones, operators hand shape and the current outfit he is wearing during the operation. Thus in this paper we propose a new method that provides the operators body shape, complexion, and light correction using real-time visuals taken from a see-through camera placed in the HMD and superimposed over robot vision. By using hand and arm trajectory from a kinematics solver, a virtual representation is used to generate masking images that isolate his local body appearance and superimpose it into the virtual environment. Local body appearance is captured via a see-through HMD. This paper describes the design and implementation of the above technique and obtained basic results.

Mutual hand representation for telexistence robots using projected virtual hands

In this paper, a mutual body representation for Telexistence Robots that does not have physical arms were discussed. We propose a method of projecting users hands as a virtual superimposition that not only the user sees through a HMD, but also to the remote participants by projecting virtual hands images into the remote environment with a small projector aligned with robots eyes. These virtual hands are produced by capturing users hands from the first point of view (FPV), and then segmented from the background. This method expands the physical body representation of the user, and allows mutual body communication between the user and remote participants while providing a better understanding users hand motion and intended interactions in the remote place.

Experiencing ones own hand in telexistence manipulation with a 15 DOF anthropomorphic robot hand and a flexible master glove

This paper describes a new type of robust control mechanism for a 15 DOF anthropomorphic robot hand in telexistence manipulations using a flexible fiber based master glove to experience the visual-kinesthetic sensation of ones own hand in remote manipulations. Accordingly, a master-slave telexistence system was constructed with the following: a 14 DOF modified optical fiber based data glove for capturing the complex finger postures of the master operator without any mechanical constraints; a novel finger posture mapping algorithm that is independent from the effects of different finger sizes and digit ratios; and a 15 DOF anthropomorphic slave robot hand for reconstructing the operators finger posture. This paper describes the importance of feeling ones fingers in a telexistence manipulation, control mechanism for accurate finger posture capture/reconstruction where the effectiveness has been verified through a set of experiments and a subjective evaluation.