Munehiko Sato

Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects

Touché proposes a novel Swept Frequency Capacitive Sensing technique that can not only detect a touch event, but also recognize complex configurations of the human hands and body. Such contextual information significantly enhances touch interaction in a broad range of applications, from conventional touchscreens to unique contexts and materials. For example, in our explorations we add touch and gesture sensitivity to the human body and liquids. We demonstrate the rich capabilities of Touché with five example setups from different application domains and conduct experimental studies that show gesture classification accuracies of 99% are achievable with our technology.

Capacitive fingerprinting: exploring user differentiation by sensing electrical properties of the human body

At present, touchscreens can differentiate multiple points of contact, but not who is touching the device. In this work, we consider how the electrical properties of humans and their attire can be used to support user differentiation on touchscreens. We propose a novel sensing approach based on Swept Frequency Capacitive Sensing, which measures the impedance of a user to the environment (i.e., ground) across a range of AC frequencies. Different people have different bone densities and muscle mass, wear different footwear, and so on. This, in turn, yields different impedance profiles, which allows for touch events, including multitouch gestures, to be attributed to a particular user. This has many interesting implications for interactive design. We describe and evaluate our sensing approach, demonstrating that the technique has considerable promise. We also discuss limitations, how these might be overcome, and next steps.

Botanicus Interacticus: interactive plants technology

Botanicus Interacticus is a technology for designing highly expressive interactive plants, both living and artificial. We are motivated by the rapid fusion of computing and our dwelling spaces, as well as the increasingly tactile and gestural nature of our interactions with digital devices. Today, however, this interaction happens either on the touch screens of tablet computers and smart phones, or in free air, captured by camera-based devices, such as the Kinect. What if, instead of this limited range of devices, a broad variety of objects in living, social and working spaces become aware and responsive to human presence, touch and gesture?

Evaluating cross-sensory perception of superimposing virtual color onto real drink: toward realization of pseudo-gustatory displays

In this research, we aim to realize a gustatory display that enhances our experience of enjoying food. However, generating a sense of taste is very difficult because the human gustatory system is quite complicated and is not yet fully understood. This is so because gustatory sensation is based on chemical signals whereas visual and auditory sensations are based on physical signals. In addition, the brain perceives flavor by combining the senses of gustation, smell, sight, warmth, memory, etc. The aim of our research is to apply the complexity of the gustatory system in order to realize a pseudo-gustatory display that presents flavors by means of visual feedback. This paper reports on the prototype system of such a display that enables us to experience various tastes without changing their chemical composition through the superimposition of virtual color. The fundamental thrust of our experiment is to evaluate the influence of cross-sensory effects by superimposing virtual color onto actual drinks and recording the responses of subjects who drink them. On the basis of experimental results, we concluded that visual feedback sufficiently affects our perception of flavor to justify the construction of pseudo-gustatory displays.

Particle Display System — Virtually Perceivable Pixels with Randomly Distributed Physical Pixels —

In this study, the authors propose and implement a particle display system (PDS) that consists of hundreds of randomly distributed pixels. The wireless capability of this system enables each node to move freely without distant limitation of the use of wire cables. The authors also propose effective visual presentation techniques for a display system with randomly distributed pixels. One of the optimization techniques involves the extension of a well-known phenomenon where humans can perceive two-dimensional static or moving images from a set of high-frequency flashing one-dimensional pixel arrays, such as LED arrays, as a characteristic of a humans vision system. While this technique can only extend the virtual resolution of a display in a direction perpendicular to the aligned pixels, our technique enables the display of multi-directional scrolling of two-dimensional images with randomly distributed pixels. In addition, the advantages of presenting information on a display with nonuniform pixel distribution and virtual pixels with fast flash of pixels are discussed. The proposed techniques help in reducing the cost of installing a large-scale display and the time taken for the initial preparation of the setup, which involves carrying large pixel arrays and determining the precise size and shape of the display.

Particle display system: a real world display with physically distributable pixels

In this paper, the author designs and implements a new display system called Particle Display System, which can be installed on the non-planar surface of any objects. It consists of hundreds of full-color and wireless Light Emitting Diode (LED) nodes with a PC and video camera. The wireless capability makes the each node freely movable without distant limitation of the use of wire cables. By processing the images from the camera, the system calculates the positioning information of the each node and performs the timing control of the LED in the each node in real time. Therefore, the author is able to design a uniquely arranged pattern in full-color in the real world, by distributing and controlling the smart nodes. This paper describes the design and implementation of the prototype of Particle Display System.

Touché: touch and gesture sensing for the real world

Touché proposes a novel Swept Frequency Capacitive Sensing technique that can not only detect a touch event, but also recognize complex configurations of the human hands and body. Such contextual information significantly enhances touch interaction in a broad range of applications, from conventional touchscreens to unique contexts and materials. For example, in our explorations we add touch and gesture sensitivity to the human body and liquids. We demonstrate the rich capabilities of Touché with five example setups from different application domains and conduct experimental studies that show gesture classification accuracies of 99% are achievable with our technology.

Mental Fatigue Monitoring Using a Wearable Transparent Eye Detection System

We propose mental fatigue measurement using a wearable eye detection system. The system is capable of acquiring movement of the pupil and blinking from the light reflected from the eye. The reflection is detected by dye-sensitized photovoltaic cells. Since these cells are patterned onto the eyeglass and do not require external input power, the system is notable for its lightweight and low power consumption and can be combined with other wearable devices, such as a head mounted display. We performed experiments to correlate information obtained by the eye detection system with the mental fatigue of the user. Since it is quite difficult to evaluate mental fatigue objectively and quantitatively, we assumed that the National Aeronautics and Space Administration Task Load Index (NASA-TLX) had a strong correlation with te mental fatigue. While a subject was requested to conduct calculation tasks, the eye detection system collected his/her information that included position, velocity and total movement of the eye, and amount and frequency of blinking. Multiple regression analyses revealed the correlation between NASA-TLX and the information obtained for 3 out of 5 subjects.

Supporting heterogeneous networks and pervasive storage in mobile content-sharing middleware

Sharing digital content with others is now an important part of human social activities. Despite the increasing need to share, most sharing operations are not simple. Many applications are not interoperable with others, require an Internet connection, or require cumbersome configuration and coordination efforts. Our idea is to simplify digital content sharing on mobile devices by providing support for self-organizing heterogeneous networks and pervasive storage. That is, mobile devices can spontaneously connect to each other over a mixture of different available networks (e.g., 3G/4G, WiFi, Bluetooth, etc.) without requiring an explicit user action of network selection or mandatory Internet access. Moreover, indirect communication can be further augmented by pervasive storage. Mobile devices can store shared content on it, which can later be automatically downloaded by other devices in proximity, thus allowing location-based sharing with minimal coordination even when devices are not in the same location at the same time. This paper shows how these technologies can be incorporated into mobile content-sharing middleware to simplify sharing operations among mobile devices without any modification to commercially available devices or applications. In particular, (i) we provide an implementation of our approach as extension modules for existing content-sharing middleware, (ii) we present two example applications built on top of it, and (iii) we demonstrate our approach through experiments in representative situations.

Particle display system: a large scale display for public space

In this paper, the authors propose a large-scale display for public spaces. The display is based on the particle display system (PDS), which consists of hundreds of randomly distributed pixels. PDSs can be realized with random distributions, unlike traditional displays that require ordered matrices. Effective visual presentation techniques for a display system with randomly distributed pixels are employed to overcome the drawbacks and even realize advantages. The proposed display system can be used in applications in large spaces and public art and architecture facade displays, owing to its features described in this paper. In addition, the authors report the design principle and an implementation of a life-size prototype of a large ceiling display.