Hiroshi Harashima

Communication over the Internet using a 3D agent with real-time facial expression analysis, synthesis and text to speech capabilities

We present a system for Internet communication that enhances traditional text-based chatting with real-time analysis and synthesis of the chat parties facial expressions. It is composed of three main modules: a real-time facial expression analysis component, a 3D agent with facial expression synthesis and text-to-speech capabilities - a talking head, and a communication module. So far we have realized a prototype to find out attractive ways of communication on the Internet and are currently experimenting on how to utilize this new type of modalities in chat communication.