Yuta Sugiura

SenSkin: adapting skin as a soft interface

We present a sensing technology and input method that uses skin deformation estimated through a thin band-type device attached to the human body, the appearance of which seems socially acceptable in daily life. An input interface usually requires feedback. SenSkin provides tactile feedback that enables users to know which part of the skin they are touching in order to issue commands. The user, having found an acceptable area before beginning the input operation, can continue to input commands without receiving explicit feedback. We developed an experimental device with two armbands to sense three-dimensional pressure applied to the skin. Sensing tangential force on uncovered skin without haptic obstacles has not previously been achieved. SenSkin is also novel in that quantitative tangential force applied to the skin, such as that of the forearm or fingers, is measured. An infrared (IR) reflective sensor is used since its durability and inexpensiveness make it suitable for everyday human sensing purposes. The multiple sensors located on the two armbands allow the tangential and normal force applied to the skin dimension to be sensed. The input command is learned and recognized using a Support Vector Machine (SVM). Finally, we show an application in which this input method is implemented.

iRing: intelligent ring using infrared reflection

We present the iRing, an intelligent input ring device developed for measuring finger gestures and external input. iRing recognizes rotation, finger bending, and external force via an infrared (IR) reflection sensor that leverages skin characteristics such as reflectance and softness. Furthermore, iRing allows using a push and stroke input method, which is popular in touch displays. The ring design has potential to be used as a wearable controller because its accessory shape is socially acceptable, easy to install, and safe, and iRing does not require extra devices. We present examples of iRing applications and discuss its validity as an inexpensive wearable interface and as a human sensing device.

Cooking with robots: designing a household system working in open environments

We propose a cooking system that operates in an open environment. The system cooks a meal by pouring various ingredients into a boiling pot on an induction heating cooker and adjusts the heating strength according to the users instructions. We then describe how the system incorporates robotic- and human-specific elements in a shared workspace so as to achieve a cooperative rudimentary cooking capability. First, we use small mobile robots instead of built-in arms to save space, improve flexibility and increase safety. Second, we use detachable visual markers to allow the user to easily configure the real-world environment. Third, we provide a graphical user interface to display detailed cooking instructions to the user. We hope insights obtained in this experiment will be useful for the design of other household systems in the future.

An actuated physical puppet as an input device for controlling a digital manikin

We present an actuated handheld puppet system for controlling the posture of a virtual character. Physical puppet devices have been used in the past to intuitively control character posture. In our research, an actuator is added to each joint of such an input device to provide physical feedback to the user. This enhancement offers many benefits. First, the user can upload pre-defined postures to the device to save time. Second, the system is capable of dynamically adjusting joint stiffness to counteract gravity, while allowing control to be maintained with relatively little force. Third, the system supports natural human body behaviors, such as whole-body reaching and joint coupling. This paper describes the user interface and implementation of the proposed technique and reports the results of expert evaluation. We also conducted two user studies to evaluate the effectiveness of our method.

PINOKY: a ring that animates your plush toys

PINOKY is a wireless ring-like device that can be externally attached to any plush toy as an accessory that animates the toy, such as by moving its limbs. A user is thus able to instantly convert any plush toy into a soft robot. The user can control the toy remotely or input the movement desired by moving the plush toy and having the data recorded and played back. Unlike other methods for animating plush toys, PINOKY is non-intrusive, so alterations to the toy are not required.

A thin stretchable interface for tangential force measurement

We have developed a simple skin-like user interface that can be easily attached to curved as well as flat surfaces and used to measure tangential force generated by pinching and dragging interactions. The interface consists of several photoreflectors that consist of an IR LED and a phototransistor and elastic fabric such as stocking and rubber membrane. The sensing method used is based on our observation that photoreflectors can be used to measure the ratio of expansion and contraction of a stocking using the changes in transmissivity of IR light passing through the stocking. Since a stocking is thin, stretchable, and nearly transparent, it can be easily attached to various types of objects such as mobile devices, robots, and different parts of the body as well as to various types of conventional pressure sensors without altering the original shape of the object. It can also present natural haptic feedback in accordance with the amount of force exerted. A system using several such sensors can determine the direction of a two-dimensional force. A variety of example applications illustrated the utility of this sensing system.

An operating method for a bipedal walking robot for entertainment

Tele-existence applications for robotic systems are becoming popular and widespread. Tey enable users to control a remote machine while experiencing a sense of being in the remote location. Initially, tele-existence was used for remote de-mining and mission-critical tasks in space, to avoid risking human life. Recently it has been applied in many entertainment and gaming applications, to enable a community to play together in one virtual environment and share the experience. But existing tele-existence systems require a large-scale interface, a lot of processing power, and a large space for proper operation.

Facial Expression Recognition in Daily Life by Embedded Photo Reflective Sensors on Smart Eyewear

This paper presents a novel smart eyewear that uses embedded photo reflective sensors and machine learning to recognize a wearers facial expressions in daily life. We leverage the skin deformation when wearers change their facial expressions. With small photo reflective sensors, we measure the proximity between the skin surface on a face and the eyewear frame where 17 sensors are integrated. A Support Vector Machine (SVM) algorithm was applied for the sensor information. The sensors can cover various facial muscle movements and can be integrated into everyday glasses. The main contributions of our work are as follows. (1) The eyewear recognizes eight facial expressions (92.8% accuracy for one time use and 78.1% for use on 3 different days). (2) It is designed and implemented considering social acceptability. The device looks like normal eyewear, so users can wear it anytime, anywhere. (3) Initial field trials in daily life were undertaken. Our work is one of the first attempts to recognize and evaluate a variety of facial expressions in the form of an unobtrusive wearable device.

Tangential force sensing system on forearm

In this paper, we propose a sensing system that can detect one dimensional tangential force on a forearm. There are some previous tactile sensors that can detect touch conditions when a user touches a human skin surface. Those sensors are usually attached on a fingernail, so therefore a user cannot touch the skin with two fingers or with their palm. In the field of cosmetics, for example, they want to measure contact forces when a customer puts their products onto their skin. In this case, it is preferable that the sensor can detect contact forces in many different contact ways. In this paper, we decided to restrict a target area to a forearm. Since the forearm has a cylindrical shape, its surface deformation propagates to neighboring areas around a wrist and an elbow. The deformation can be used to estimate tangential force on the forearm. Our system does not require any equipment for the active side (i.e. fingers or a palm). Thus a user can touch the forearm in arbitrary ways. We show basic numerical simulation and experimental results which indicate that the proposed system can detect tangential force on the forearm. Also we show some possible applications that use the forearm as a human-computer interface device.

Graffiti fur: turning your carpet into a computer display

Computer displays play an important role in connecting the information world and the real world. In the era of ubiquitous computing, it is essential to be able to access information in a fluid way and non-obstructive integration of displays into our living environment is a basic requirement to achieve it. Here, we propose a display technology that utilizes the phenomenon whereby the shading properties of fur change as the fibers are raised or flattened. One can erase drawings by first flattening the fibers by sweeping the surface by hand in the fibers growth direction and then draw lines by raising the fibers by moving a finger in the opposite direction. These material properties can be found in various items such as carpets and plush toy in our living environment. Our technology can turn these ordinary objects into displays without requiring or creating any non-reversible modifications to the objects. It can be used to make a large-scale display and the drawings it creates have no running costs.