Tsugunosuke Sakai

Development of an Automatic Location-determining Function for Balloon-type Dialogue in a Puppet Show System for the Hearing Impaired.

People with hearing impairments have a tendency to experience difficulties in obtaining audio information. They have difficult to watch puppet shows. In this study, we have undertaken the development of a dialogue presentation function in a puppet show system for the hearing impaired. The dialogue presentation function that was developed is an automatic location-determination system for balloon-type dialogue. The balloontype dialogue turns the lines of dialogue into text and displays it as balloons in the background animation of the puppet show. This function automatically places the balloon-type dialogue in the vicinity of the locations of the puppets. In the evaluation test, 24 college students with a hearing impairment were used as participants, and a comparison was made between a system that implemented the automatic locationdetermination function for balloon-type dialogue and a system that did not implement said function. These results indicate the effectiveness of the location tracking function for balloon-type dialogue as a means for ensuring access to audio information in puppet shows.

Designing a Collaborative Interaction Experience for a Puppet Show System for Hearing-Impaired Children

In this study we have developed a puppet shows system for hearing-impaired children. It is difficult for hearing-impaired children to experience a puppet show. One of the reasons that hearing-impaired children have difficulty experiencing a puppet show is because the performance is a collaborative interaction experience. Collaborative interaction experiences encourage immersive viewing toward an empathetic understanding of the characters. This paper aims to design a collaborative interaction experience function for hearing-impaired children. This function provides collaborative interaction for the audience to work with the characters to resolve issues in the story by using body motion. From the results of the evaluation experiment, we understood that the collaborative interaction experience function generally supported an immersive puppet show experience for the audience.

Participatory Design of UKIYO-E Game for Children to Support Art Appreciation Based on Interacting with Pictures

We are developing an art-appreciation system that will provide learning information about pictures while generating interest in the pictures themselves. This system can provide learning information on pictures by talking to an artist or people portrayed in the pictures using voice recognition. Furthermore, the system operates through movement and location of the user, and it generates sensation such that the user feels he/she is in the pictures, thus actively appreciating the pictures. As the first step toward this system, we have developed a system that provides learning information to the user by actively talking using ukiyo-e. Here, we quantitatively evaluate this system for primary schoolchildren and determine whether the user is interested in the pictures while talking to the pictures using voice recognition measured by electrodermal activity. In this paper, we summarize the current system and describe two evaluation results.

Development of Gesture Recognition Sub-system for BELONG - Increasing the Sense of Immersion for Dinosaurian Environment Learning Support System.

We are developing an immersive learning support system for paleontological environments in museums. The system measures the body movement of the learner using a Kinect sensor, and provides a sense of immersion in the paleontological environment. Conventional systems are only able to recognize simple body movements, which is insufficient to completely immerse learners in the paleontological environment. On the other hand, when they need to perform complicated body movements, learners move their bodies eagerly while thinking. This emphasizes the importance of developing a sub-system capable of recognizing complicated body movements. In this paper, we describe a sub-system developed to recognize the body movement of the most important learner in the immersive learning support system.

SATOYAMA: Time-limited Decision Game for Students to Learn Hundreds Years Forestry Management.

Global environmental problems continue to worsen. In this situation, it is important to understand and experience changes in the natural environment in realistic ways. However, it was difficult to experience these changes in real time because they happen over large time scales. To overcome this problem, the authors developed a game-like learning tool that enables players to learn about vegetation succession. This game is Windows-based and enables players to simulate the conservation of SATOYAMA, rural natural areas. The player selects the actions within a designated time and the vegetation succession of the SATOYAMA changes according to that action. At the end of this game, the score is based on the SATOYAMA conserved by the player. Thirty-seven students participated in this experiment whose result suggested that this game would enable children to develop an interest in vegetation succession and motivate them to learn about it.

Implementation and Evaluation of Accessible Caption System in Universal Puppetry: A Case Study on Hearing Impaired Children at the Elementary School of Deaf

This study tested and evaluated two functions developed as a part of the balloon-type caption display system in a universal puppetry. Two functions are the automatic location-determining function and the emotional expression function. The automatic location-determining function is feature that make captions positioned according to the positions in which the puppets move. And emotional expression function changes the shape of balloons that represents emotions such as joy, anger, sadness, and pleasure. We conducted the evaluation experiment. Participants are fifteen students with hearing impairments from the elementary school of deaf. The result shows the effectiveness of the automatic location-determining function and the emotional expression function as a means to compensate for vocal information in puppetry for hearing impaired children.

COSEY: Computer Supported Enhancement of Young Childrenźs Cooperation

We have developed a full-body interaction game that allows children to cooperate and interact with other children in small groups. The intent of the full-body interaction game developed in this study is to encourage cooperation between children. The game requires two children to jump together with the same timing. We let children experience the game, and observed the children using several strategies to coordinate the timing of their jumps. These included shouting time, watching each other, and jumping in a constant rhythm as if they were skipping rope. In this manner, we observed the children playing the game while cooperating with each other. Therefore, the game enabled children to experience cooperation with others by using body movements.We have developed a full-body interaction game that allows children to cooperate and interact with other children in small groups. The intent of the full-body interaction game developed in this study is to encourage cooperation between children. The game requires two children to jump together with the same timing. We let children experience the game, and observed the children using several strategies to coordinate the timing of their jumps. These included shouting time, watching each other, and jumping in a constant rhythm as if they were skipping rope. In this manner, we observed the children playing the game while cooperating with each other. Therefore, the game enabled children to experience cooperation with others by using body movements.

Multiple-Player Full-Body Interaction Game to Enhance Young Children's Cooperation

We have developed a full-body interaction game that allows children to cooperate and interact with other children in small groups. The aim of the game developed in this study is to encourage cooperation between children. The game requires two children to jump together at the same time. In a series of experiments, we observed children using several strategies to coordinate the timing of their jumps, such as shouting to one another and watching each other. The results of a questionnaire to evaluate the proposed interactive game indicate that children strategize to cooperate with each other while enjoying the game. Therefore, the game enables children to cooperate with others using bodily movements.

BESIDE: Immersive System to Enhance Learning Within a Museum

We are developing an immersive learning support system for a paleontological environment within a museum. The system measures the physical movement of a learner using a Kinect sensor and provides a sense of immersion in the paleontological environment by modifying the surroundings according to these movements. As the first stage of this project, we have developed a prototype system that enables learners to experience paleontological environments. We evaluated the operability of the system, the degree of learning support, and the sense of immersion for primary schoolchildren. This paper summarizes the current system and describes the evaluation results.