Fumihiro Adachi

Speech-activated text retrieval system for multimodal cellular phones

The paper describes an on-line manual page retrieval system activated by spoken queries for multimodal cellular phones. The system recognizes a users naturally spoken queries by telephone LVCSR and searches an on-line manual with a retrieval module on a server. The user can view the retrieved data on the screen of the phone via Web access. The LVCSR module consists of a telephone acoustic model and an n-gram language model derived from a task query corpus. The adaptation method using the target manual is also presented. The retrieval module utilizes pairs of words with dependency relations and also distinguishes affirmative and negative expressions to improve precision. The proposed system gives 82.6% keyword recognition accuracy and 77.5% task achievement rate. The field trial of the system is now underway.

Multimedia Blog Creation System using Dialogue with Intelligent Robot

A multimedia blog creation system is described that uses Japanese dialogue with an intelligent robot. Although multimedia blogs are increasing in popularity, creating blogs is not easy for users who lack high-level information literacy skills. Even skilled users have to waste time creating and assigning text descriptions to their blogs and searching related multimedia such as images, music, and illustrations. To enable effortless and enjoyable creation of multimedia blogs, we developed the system on a prototype robot called PaPeRo. Video messages are recorded and converted into text descriptions by PaPeRo using continuous speech recognition. PaPeRo then searches for suitable multimedia contents on the internet and databases, and then, based on the search results, chooses appropriate sympathetic comments by using natural language text retrieval. The retrieved contents, PaPeRos comments, and the video recording on the users blog is automatically uploaded and edited. The system was evaluated by 10 users for creating travel blogs and proved to be helpful for both inexperienced and experienced users. The system enabled easy multimedia-rich blog creation and even provided users the pleasure of chatting with PaPeRo.