Tomio Watanabe

InterActor: Speech-Driven Embodied Interactive Actor

A speech-driven embodied interactive actor called InterActor, with functions of both speaker and listener, was developed for activating human interaction and communication by generating expressive actions and motions coherently related to speech input. InterActor is the electronic media version of a physical interaction robot called InterRobot for robot-mediated communication support, which is free of the hardware restrictions for human interface of advanced graphical user interface based network communication. By using InterActor, the concept of speech-driven embodied interaction system is proposed for human interaction sharing by the entrainment between human speech and InterActor motions in remote communication. The prototype of the system is developed, and the sensory evaluation and behavioral analysis in human communication through InterActor demonstrates the effectiveness of the system. Actual applications of InterActor to human interface are also demonstrated. The system is a practical communication support system, which activates human interaction and communication on the basis of only speech input.

InterRobot: speech-driven embodied interaction robot

In face-to-face communication, humans communicate using non-verbal behavior such as nodding and gestures as well as verbal messages. In particular, the coherently related synchrony of biorhythms between verbal signals and non-verbal behavior between talkers, which is called entrainment in communication, plays an important role in human interaction. The embodied communication closely related to behavioral entrainment is an essential form of communication, which forms the basis of the interaction between talkers. In this article, a speech-driven embodied interaction robot system based on the entrainment mechanism is developed for supporting human interaction and some applications are demonstrated.

Human-entrained Embodied Interaction and Communication Technology

An embodied communication system for mind connection (E-COSMIC) has been developed by applying the entrainment mechanism of the embodied rhythms of nodding and body movements to physical robots and CG characters in verbal communication. E-COSMIC comprises an embodied virtual communication system for human interaction analysis by synthesis and a speech-driven embodied interaction system for supporting essential human interaction and communication based on the analysis that uses the embodied virtual communication system. A human-entrained embodied interaction and communication technology for an advanced media society is introduced through some applications of E-COSMIC. A generation and control technology of human-entrained embodied media is also introduced.

E-COSMIC: embodied communication system for mind connection

On the basis of the embodied entrainment between speech and body motions such as nodding in face-to-face communication, embodied communication system for mind connection (E-COSMIC) is developed for supporting essential human interaction and communication. E-COSMIC consists of the embodied virtual communication system for human interaction analysis by synthesis and the speech-driven embodied interaction system for embodied interaction and communication support on the basis of the analysis by using the embodied virtual communication system. Some actual applications of E-COSMIC to human interface are demonstrated on robot and human interactive communication. The system would form the foundation of mediated communication technologies as well as the methodology for the analysis and understanding of human interaction and communication.

Visualization of respiration in the embodied virtual communication system and its evaluation

A proposed embodied virtual communication system provides a virtual face-to-face communication environment in which two remote talkers can share embodied interaction by observing their interaction with two types of avatars. One is VirtualActor, a human avatar that represents talker communicative motion and respiratory motion. The other is VirtualWave, an abstract avatar that expresses human behavior and respiration by simplified wave motion. By using the system for the analysis by synthesis of embodied communication, the effectiveness of the visualization of respiration in VirtualActor and VirtualWave is demonstrated by the analysis of the entrainment of interaction and the sensory evaluation in remote communication.

Effects of facial color on virtual facial image synthesis for dynamic facial color and expression under laughing emotion

Human facial color was applied to the affect display of virtual facial image, emotional evaluation, individual identification, and remote health care. We dynamically analyzed facial color under the emotional change of laughing by the simultaneous measurement of facial image and facial skin temperature, and proposed a dynamic facial color model. In this paper, on the basis of the proposed dynamic facial color model under laughing emotion, we analyze the effects of facial color on virtual facial image synthesis for dynamic facial color and expression of smile by using an average facial image of 68 females. Sensory evaluation demonstrates the effectiveness of dynamic facial color for the affect display of virtual facial image under laughing emotion.

Timing control effects of utterance to communicative actions on embodied interaction with a robot

The timing to generate communicative actions and utterance in face-to-face greeting interaction is analyzed by synthesis for applying to an embodied interaction support with a robot. First, the analysis of human greetings clarifies the pause and time delay of utterance to communicative actions. Then, the analysis by synthesis is performed by using an embodied robot system and confirms that the variation of the pause and lags of utterance to communicative actions brings different communicative effects, i.e. about 0.3 sec lag is desirable for familiar greetings, and the longer lag is for polite greetings. These results demonstrate the importance of timing control in robot-human embodied interaction and the applicability in advanced robot-human communication.

Time lag effects of utterance to communicative actions on robot-human greeting interaction

The timing to generate communicative action and utterance in face-to-face greeting interaction is analyzed by synthesis for applying to a robot-human interaction support. The analysis by synthesis is performed by using an embodied robot system and confirms that the variation of the pause and lags of utterance to communicative actions brings different communicative effects, i.e. about 0.3 sec lag is desirable for familiar greetings, and the longer lag is for polite greetings. This result demonstrates the importance of timing in robot-human embodied interaction and the applicability in advanced human-robot communication.

Development of a Handshake Robot System Based on a Handshake Approaching Motion Model

Handshake refers to embodied interaction using physical contact for closeness. In this paper, we analyze the handshake approaching motion between humans in cases with and without voice greeting and propose a handshake approaching motion model based on the analysis. In this model, a robot can generate handshake approaching motion that is emotionally acceptable to humans by using the second-order lag element and the dead time element from trajectory of human hand. Furthermore, a handshake robot system that adopts this proposed model is developed and motion characteristics of handshake approaching motions with and without voice greeting are analyzed using analysis-by-synthesis technique. The effectiveness of this model and the handshake robot system are demonstrated by sensory evaluation.

Human-Entrained Embodied Interaction and Communication Technology for Advanced Media Society

The embodied communication system for mind connection (E-COSMIC) has been developed by applying the entrainment mechanism of the embodied rhythms of nodding and body movements to physical robots and CG characters in verbal communication. E-COSMIC comprises an embodied virtual communication system for human interaction analysis by synthesis and a speech-driven embodied interaction system for supporting essential human interaction and communication based on the analysis that uses the embodied virtual communication system. A human-entrained embodied interaction and communication technology for an advanced media society is introduced through some applications of E-COSMIC. A generation and control technology of human entrained embodied media is also introduced.