Amy Koike

You as a Puppet: Evaluation of Telepresence User Interface for Puppetry

We propose an immersive telepresence system for puppetry that transmits a human performers body and facial movements into a puppet with audiovisual feedback to the performer. The cameras carried in place of puppets eyes stream live video to the HMD worn by the performer, so that performers can see the images from the puppets eyes with their own eyes and have a visual understanding of the puppets ambience. In conventional methods to manipulate a puppet (a hand-puppet, a string-puppet, and a rod-puppet), there is a need to practice manipulating puppets, and there is difficulty carrying out interactions with the audience. Moreover, puppeteers must be positioned exactly where the puppet is. The proposed system addresses these issues by enabling a human performer to manipulate the puppet remotely using his or her body and facial movements. We conducted several user studies with both beginners and professional puppeteers. The results show that, unlike the conventional method, the proposed system facilitates the manipulation of puppets especially for beginners. Moreover, this system allows performers to enjoy puppetry and fascinate audiences.

Cross-field haptics: push-pull haptics combined with magnetic and electrostatic fields

The representation of texture is a major concern during fabrication and manufacturing in many industries. Thus, the approach for fabricating everyday objects and the digital expression of their textures before fabrication process has become a popular research area. Although it is easy to change the texture of objects in the digital world (i.e. just setting texture parameters), it is difficult to achieve this in the real world.

Yadori: mask-type user interface for manipulation of puppets

We propose a system for animatronics storytelling that enables performers to manipulate puppets by wearing a mask-type device on their faces.

Transformed Human Presence for Puppetry

We propose a system for transmitting a human performers body and facial movements to a puppet with audiovisual feedback to the performer. The system consists of a head-mounted display (HMD) that shows the performer the video recording of the puppets view, a microphone for voice capture, and photoreflectors for detecting the mouth movements of the human performer. In conventional puppetry, there is also the need for practice in the manipulation of the puppets to achieve good performance. The proposed telepresence system addresses these issues by enabling the human performer to manipulate the puppet through their own body and facial movements. The proposed system is expected to contribute to the development of new applications of puppetry and expand the interactivity of puppetry and the scope of entertainment.

LeviFab: stabilization and manipulation of digitally fabricated objects for superconductive levitation

Aerial manipulation of material objects is fascinating and is used in many performance situations. Many scientific demonstrations and magic shows employ these levitations. Acoustic, magnetic, electric, and superconductive levitation are used in many situations. Adding controllability and increasing the design space of these levitation methods are often studied for use in entertainment applications in graphics and HCI communities. In this study, we focus on superconductive levitation (Figure 1) because it has not been well explored for entertainment applications.

Sonovortex: rendering multi-resolution aerial haptics by aerodynamic vortex and focused ultrasound

Aerial haptic feedback is a popular topic in research fields on real-world-oriented interaction, augmented reality (AR), and virtual reality (VR). Various methods such as magnetic force, ultrasound, and air vortices have been proposed for this purpose.

Human Coded Orchestra: a System for Extemporary Group Singing Performance

We propose Human Coded Orchestra, a new approach to enable a group of individuals to sing in harmony by using computed directional speakers. The possibilities of musical performance by an untrained group have been explored in the fields of science and art. However, previous work has rarely proceeded beyond simple rhythm-based music and failed to achieve musical complexity. Human Coded Orchestra employs a number of directional speakers, each set at a different pitch, enabling them to deliver different pitches to each participant to sing to, according to their positions. Experiments demonstrated that participants succeeded in singing in harmony extemporaneously, and they reported that they enjoyed both the experience of singing and the feeling that they were able to participate in an activity with others. Our system does not require preparation on the part of singers, which opens up the possibility of practical application in the area of interactive performance.

Redesign of Cartesian Diver for Underwater Expression Combining Dynamic Fabrication with Non-contact Manipulation

In this study, we aim to combine dynamic fabrication with non-contact manipulation system applying the mechanism of Cartesian Diver. To achieve this, we propose the design method for underwater objects and non-contact manipulation technique using water pressure with PID control. We successfully designed and manipulate the object by our method. We discussed the principles and methods to create a digitally designed and fabricated the diver and to stabilize it in the middle of water.

Cross-field haptics: Multiple direction haptics combined with magnetic and electrostatic fields

We present a new method of rendering haptic textures that utilizes electrostatic and magnetic fields. In conventional research, a single physical quantity is used to render haptic textures. By contrast, our method combines multiple fields (electrostatic and magnetic). Although these fields have no direct interference, combining them provides benefits such as the ability to produce multi-resolution haptic images and synergistic effects on haptic perception. We investigate the increase in the variation of texture by comparing each single field method. Furthermore, we conduct user experiments and quantitative measurements.

Digital fabrication and manipulation method for underwater display and entertainment

Underwater expression is attractive. It seems like underwater objects are floating like anti-gravity scape by buoyancy and it is also impressive that bubbles rise while refracting the light. In this work, we aim to combine digital fabrication with interactive technology and expand underwater expression. To achieve this, we focused on a classic science experiment called the Cartesian Diver. Because of growing interest in the materialization of computer graphics, digital fabrication technologies have recently emerged as one of the most important application fields in real-world-oriented computer graphics. In particular, research on digital fabrication that gives dynamics properties is common. Spin-it [Bächer et al. 2014] presents design method for spinning objects by optimizing rotational dynamics properties. Some studies use non-contact manipulation. For example, ZeroN [Lee et al. 2011] controls the magnetic field to manipulate the object and uses it as a floating screen and input user interface(UI). Our work connects digital fabrication and non-contact manipulation that uses the space transmission power (water pressure) around the object (the diver). [Koike et al. 2016] proposes a design and manipulation method for the diver. In this work, we updated the method and investigate stability of PID control. Furthermore, we propose some applications.