Masanobu Abe

WebMessenger: a new framework to produce multimedia content by combining synthesized speech and moving pictures in the WWW environment

To facilitate multimedia content production for Internet distribution, this paper proposes a new framework for generating multimedia content. Advantages of the framework are minimized amount of transmitted data and easy-to-update multimedia content. Key points are (1) to produce speech messages by a text-to-speech (TTS) system and (2) to produce moving pictures by concatenating templates from a set of moving pictures. To support the proposed framework, a speech design tool and a movie assign tool are introduced. Three kinds of contents have been produced using the proposed framework and informally evaluated. Important results are: (1) the transmitted data of WebMessenger is 120 times smaller than the full movie version of the content, (2) quality of speech synthesized by TTS is not sufficient and a speech design tool is necessary, (3) WebMessenger is 50% cheaper than human speech recording, (4) speech synthesis by combining manual and automatic methods is effective.

A new communication paradigm; action-to-speech

We propose a new technology that enhances the communication ability of human beings. We call it action-to-speech (A2S). A2S generates speech from actions or gestures and makes it possible for people who speak different languages or the speaking impaired to communicate. Features of A2S are as follows: because meanings are defined by mapping functions between action and speech, it is easy for the system to handle new languages; because prosodic characteristics of speech can be controlled in real time, smooth communication is possible; and because sensors capture actions, reliable responses are guaranteed. A prototype system was developed. Evaluations indicate that A2S makes it possible for persons speaking different languages to easily and effectively communicate.