Asako Kimura

MAI painting brush: an interactive device that realizes the feeling of real painting

Many digital painting systems have been proposed and their quality is improving. In these systems, graphics tablets are widely used as input devices. However, because of its rigid nib and indirect manipulation, the operational feeling of a graphics tablet is different from that of real paint brush. We solved this problem by developing the MR-based Artistic Interactive (MAI) Painting Brush, which imitates a real paint brush, and constructed a mixed reality (MR) painting system that enables direct painting on physical objects in the real world.

Psychophysical influence of mixed-reality visual stimulation on sense of hardness

In a mixed-reality (MR) environment, the appearance of touchable objects can be changed by superimposing a computer-generated image (CGI) onto them (MR visual stimulation). At the same time, when humans sense the hardness of real objects, it is known that their perception is influenced not only by tactile information but also by visual information. In this paper, we studied the psychophysical influence on the sense of hardness by using a real object that has a CGI superimposed on it. In this experiment, we deform in an extreme way the CGI animation on the real object, while the subject pushes the real object using his/her finger. The results of the experiments found that human subjects sensed different hardnesses by emphasizing the dent deformation of the CGI animation.

Novel interaction methods with mixed reality space

Mixed reality (MR), which merges the real mid virtual worlds in real time, is a form of further extension of the conventional virtual reality (\R). There have been many studies in the field of MR, however, very few studies discuss interaction methods. In this study, we propose two novel methods for interaction with an MR space, RealSound Interaction and ToolDevice.

Enjoying virtual handcrafting with ToolDevice

ToolDevice is a set of devices developed to help users in spatial work such as layout design and three-dimensional (3D) modeling. It consists of three components: TweezersDevice, Knife/HammerDevice, and BrushDevice, which use hand tool metaphors to help users recognize each devices unique functions. We have developed a mixed reality (MR) 3D modeling system that imitates real-life woodworking using the TweezersDevice and the Knife/HammerDevice. In the system, users can pick up and move virtual objects with the TweezersDevice. Users can also cut and join virtual objects using the Knife/HammerDevice. By repeating these operations, users can build virtual wood models.

Psychophysical Influence on Tactual Impression by Mixed-Reality Visual Stimulation

This paper describes the influence of visual stimulation on the tactual sense in a mixed-reality environment; i.e., how a tactual impression of a real object is affected by seeing a superimposed image of a different type of material. If the behavior and the extent of such an influence, a sort of illusion, are investigated in detail, the objects composed of a limited variety of materials can be perceived differently. This would be useful in the field of digital engineering. Therefore, we performed various experiments systematically.

Shape-COG Illusion: Psychophysical influence on center-of-gravity perception by mixed-reality visual stimulation

Mixed reality (MR) is a technology that merges real and virtual worlds in real time. In MR environments, visual appearance of a real object can be changed by superimposing a virtual object on it. In this study, we focus on the center-of-gravity (COG) and verify the influence of MR visual stimulation on the COG in MR environments. This paper describes the systematic experiments performed to study the influence. The results obtained are interesting: (1) the presence of COG can be changed by MR visual stimulation; (2) although COG differs in vision and force, the presence of COG can be represented by MR visual stimulation under certain conditions; (3) COG perception can also be changed by varying the mass of the real object. We named this psychophysical influence the “Shape-COG Illusion.”

Virtual Handcrafting: Building Virtual Wood Models Using ToolDevice

Most 3-D modeling software are difficult for beginners to learn. The operations are often complicated, and the user is required to have prior mathematical knowledge. Therefore, we developed a simple modeling system using ToolDevice to simplify such operations. ToolDevice consists of a set of interaction devices, which use metaphors of real-life hand tools to help users recognize each devices unique functions. Using TweezersDevice, KnifeDevice, and HammerDevice, we developed a mixed reality (MR) 3-D modeling system that imitates real-life woodworking. In the system, TweezersDevice is used to pick up and move objects, while KnifeDevice and HammerDevice are, respectively, used to cut and join virtual objects represented as wood materials. In this study, we describe the motivation for developing the system, the available interactions, and the procedures for creating 3-D models in the system. We also present the results of a user study in which we compare user performance in our system and a common 3-D modeling software. Finally, we discuss the contributions and limitations of this study and future work.

A Two-by-Two Mixed Reality System That Merges Real and Virtual Worlds in Both Audio and Visual Senses

There have been many implementations of virtual reality, using audio and visual senses. However, implementations of mixed reality (MR) have thus far only dealt with the visual sense. We have developed an MR system that merges real and virtual worlds in both the audio and visual senses, wherein the geometric consistency of the audio sense was fully coordinated with the visual sense. We tried two approaches for merging real and virtual worlds in the audio sense, using open-air and closed-air headphones.

Efficient Use of Textured 3D Model for Pre-observation-based Diminished Reality

Diminished reality (DR) deletes or diminishes undesirable objects from the perceived environments. We present a pre-observation-based DR (POB-DR) framework that uses a textured 3D model (T-3DM) of a scene for efficiently deleting undesirable objects. The proposed framework and T-3DM data structure enable geometric and photometric registration that allow the user to move in six degrees-of-freedom (6DoF) under dynamic lighting during the deletion process. To accomplish these tasks, we allow the user to pre-observe backgrounds to be occluded similar to existing POB-DR approaches and preserve hundreds of view-dependent images and triangle fans as a T-3DM. The proposed system effectively uses the T-3DM for all of processes to fill in the target region in the proposed deletion scheme. The results of our experiments demonstrate that the proposed system works in unknown 3D scenes and can handle rapid and drastic 6DoF camera motion and dynamic illumination changes.

Dent-softness illusion in mixed reality space: Further experiments and considerations

When humans sense the hardness of real objects, their perception is known to be influenced by not only tactile information but also visual information. In a mixed-reality (MR) environment, the appearance of touchable objects can be changed by superimposing a computer-generated image onto them (MR visual stimulation). In our previous study, we first superimposed computer-generated images (CGI) on real objects, and then, applied extreme deformation to the superimposed images while the participants pressed the real objects with their fingers. The results showed that humans sense hardness differently while receiving MR visual stimulation; we named this psychophysical effect the dent-softness illusion. As the next step, in this study, we verify whether humans perceive hardness differently when pressing a real object with and without superimposed animation and whether the dent-softness illusion occurs in different parts of the body such as the finger, palm, and arm.