Tomohito Yamamoto

Timing control of utterance and gesture in interaction between human and humanoid robot

Towards establishing future-oriented relations between humans and robots, not only verbal, but also nonverbal robot functions have been studied. However, it is not clear how these different functions should be combined and implemented in robots in order to achieve ‘natural’ human-robot communication or interaction. In this paper, we focus on the timing for the coordination between utterance as a verbal function and body motion as a nonverbal function. To begin with, we investigated the temporal order and correlation between utterance and gesture in an interpersonal (human-human) interaction scenario as a preliminary experiment (Experiment 1). Then, we controlled the robots response timing based on findings of the first experiment and evaluated the effectiveness of the robots timing control in human-robot interaction (Experiment 2). It was shown that elderly participants significantly preferred the controlled robots response. In addition, the timing of utterance and gestures affected the impression of the robot and interpretation of the verbal message. These results are important findings for designing socially interactive robots for the aging society.

Analysis of interaction in musical communication and its modeling

Listening to music at a live performance is different from listening to music from passive media (e.g. a CD), and a live performance gives a better impression. This is caused not only by the sound facilities but also by interaction between the player and the listener. In this research, in order to analyze such interactions in music communication, the relation between the listeners respiration and the music was investigated. As a result, forced entrainment was discovered between the period of a musical bar and the period of the listeners respiration in listening to music from passive media, and mutual entrainment was discovered between these periods in listening to music from a live performance. Based on our measurement results, a communication model was constructed with a mutual entrainment model and an experiment was performed. The result was that most of the subjects felt that its music was live.

Timing control of utterance and body motion in human-robot interaction

Towards developing robots that are capable of communicating in dasianaturalpsila ways with humans, we believe that it is important to study, and note any important interrelation between, both verbal and non-verbal information in human communication. The model of a timing control system was developed based on a previous study that analyzed the utterance and body motions in the context of human-human communication. This paper presents the realization of this timing control system for the purpose of human-robot interaction, implemented on the Wakamaru robot platform.

Hierarchical timing structure of utterance in human dialogue

In this research, to clarify the relation between a cognitive process and a timing structure of utterance, the dialogue that consisted of an instruction and a response utterance was analyzed. The results showed that correlation between duration of instruction utterance and that of switching pause was negative and low value, when a change of utterance speed was too small to be aware. However, the correlation between them was positive and high value, when a change of utterance speed was explicitly big. The positive correlation between durations has been reported in some researches, however the negative and low correlation between them have not been reported. These results suggest that there is hierarchical timing structure in human dialog.

A virtual Kanji puzzle game based on 3D graphics and an intuitive inputting device

Nowadays, many people are concerned about their Kanji education level, when a child asks the parents about Kanji, or when they have to use Kanji expression in their works. For this reason, many computer games have been developed for learning and practicing Kanji.

Effect of a listener on a player in a musical live performance

Abmcr-Listening to music at a live concert is different from listening to it with CD player. It is caused from not only sound facilities but aIso interaction between players and listeners. In this study, we investigated the relation between listener’s respiration and player’s I-bar period to analyze interaction in a live performance. As a result, it was suggested that listener’s respiration period was changed by music, and also player’s 1-bar period was changed by a listener, and mutual entrainment occurred between I-bar period and respiration period

Analysis of interaction between players in a cooperative music performance

In this study, to investigate the harmony and synchronization of a cooperative music performance, the process of dynamic players change in a music performance was analyzed. The results suggested that there was an interaction between players 1-bar periods, and between respiration periods, and also suggested that when 1-bar periods were not synchronous, respiration period were synchronous.

Mobile - based streaming system for omnidirectional contents

Many types of display systems have been developed for providing a spatial viewing experience, and surround sound systems, for expressing high levels of presence. However, these types of visual or auditory display systems sometimes require the allocation of large spaces for fixed, specialized equipment, and they tend to be expensive. On the other hand, mobile devices such as smartphones and tablets are now widespread. Thus, it may be possible to build an immersive reality system on mobile devices, which users can experience at any time and in any place.

Temporal relationship between pause and utterance durations in speech of short sentence

In this paper we focused on a pause in speech, and analyzed the factors affecting pause duration. It has been considered that utterance duration just before the pause is the only factor affecting pause duration (preboundary effect), recently effect of utterance duration just after the pause has also been noticed (postboundary effect). However, the relation between two utterance durations and pause duration sandwiched by the durations (pre-post boundary effect) has not been analyzed. Therefore we analyzed these factors inclusively, by using a simple sentence (XY sentence) consisting of two words in speech experiment. Then we used two-way analysis of variance (ANOVA) for analyzing the contribution of factors, which were the utterance duration of these words. As a result, we found two factors affecting a pause. One is utterance duration just before the pause which was already observed, and the other is the ratio of prior and posterior utterance duration. These results suggest that not only a pre or postboundary effect but also a pre-postboundary effect exist in speech.

System for Measuring Teacher–Student Communication in the Classroom Using Smartphone Accelerometer Sensors

The quality of communication between a teacher and students is deeply related to the cultivation of students’ motivation, autonomy, and creativity in a university education. It is important to evaluate such communication and improve it to enhance faculty development. In this study, a system for measuring this communication has been developed. To implement the system, an application for measuring students’ body movements using the acceleration sensor of a smartphone was developed. At the same time, a server-side web system that visualizes the measured data was developed. Using this measurement system, the communication in a seminar of a university laboratory was measured. The results show that the activities of a presenter and audience can be clearly detected by the raw and frequency-analyzed accelerometer data. Moreover, the correlation between the sonograms of the presenter and of the audience members became stronger when they had constructive discussion. These results suggest that the synchronization between a presenter and the audience is related to their level of rapport.