Shinichi Ando

An automatic speech translation system on PDAs for travel conversation

We present an automatic speech-to-speech translation system for personal digital assistants (PDAs) that helps oral communication between Japanese and English speakers in various situations while traveling. Our original compact large vocabulary continuous speech recognition engine, compact translation engine based on a lexicalized grammar, and compact Japanese speech synthesis engine lead to the development of a Japanese/English bi-directional speech translation system that works with limited computational resources.

Automatic Interpretation System Integrating Free-Style Sentence Translation and Parallel Text Based Translation

This paper proposes an automatic interpretation system that integrates freestyle sentence translation and parallel text based translation. Free-style sentence translation accepts natural language sentences and translates them by machine translation. Parallel text based translation provides a proper translation for a sentence in the parallel text by referring to a corresponding translation of the sentence and supplements free-style sentence translation. We developed a prototype of an automatic interpretation system for Japanese overseas travelers with parallel text based translation using 9206 parallel bilingual sentences prepared in task-oriented manner. Evaluation results show that the parallel text based translation covers 72% of typical utterances for overseas travel and the user can easily find an appropriate sentence from a natural utterance for 64% of typical travelers tasks. This indicates that the user can benefit from reliable translation based on parallel text for fundamental utterances necessary for overseas travel.

Lexicalized Tree Automata-based Grammars for translating conversational texts

We propose a new lexicalized grammar formalism called Lexicalized Tree Automata-based Grammar, which lexicalizes tree acceptors instead of trees themselves. We discuss the properties of the grammar and present a chart parsing algorithm. We have implemented a translation module for conversational texts using this formalism, and applied it to an experimental automatic interpretation system (speech translation system).

Topic detection based on dialogue history

In this paper, we propose a topic detection method using a dialogue history for a speech translator. The method uses a k-nearest neighbor method for the algorithm, automatically clusters target topics into smaller topics grouped by similarity, and incorporates dialogue history weighted in terms of time to detect and track topics on spoken phrases. From the evaluation of detection performance using test data comprised of realistic spoken dialogue, the method has shown to perform better with clustering incorporated, and when combined with dialogue history of three sentences, gives detection accuracy of 72.1%.

NEC: description of the VENIEX system as used for MUC-5

NEC Corporation has had years of experience in natural language processing and machine translation[1, 2, 3, 4, 5], and currently markets commercial natural language processing systems. Utilizing dictionaries and parsing engines we have already had, we have developed the VENIEX System (VENus for Information EXtraction) as used for MUC-5 in only three months. Our method is to apply both domain-specific keyword-based analysis and full sentential parsing with general grammar[6, 7]. The keyword dictionary of VENIEX contains about thirty thousand entries, whose semantic structures are sub_ME_Capability frame, and the parsing and discourse processing are controlled with the information given in this semantic structure of keywords. The resulting scores of VENIEX for formal run texts were from 0.7181 (minimum) to 0.7548 (maximum) in Richness-Normalized Error and 48.33 in F-MEASURES (P&R).

Topic Detection Based on Dialogue History

In this paper, we propose a topic detection method using a dialogue history for selecting a scene in the automatic interpretation system (Ikeda et al., 2002). The method uses a k-nearest neighbor method for the algorithm, automatically clusters target topics into smaller topics grouped by similarity, and incorporates dialogue history weighted in terms of time to detect and track topics on spoken phrases. From the evaluation of detection performance using test corpus comprised of realistic spoken dialogue, the method has shown to perform better with clustering incorporated, and combined with time-weighted dialogue history of three sentences, gives detection accuracy of 77.0%.

Multimedia Blog Creation System using Dialogue with Intelligent Robot

A multimedia blog creation system is described that uses Japanese dialogue with an intelligent robot. Although multimedia blogs are increasing in popularity, creating blogs is not easy for users who lack high-level information literacy skills. Even skilled users have to waste time creating and assigning text descriptions to their blogs and searching related multimedia such as images, music, and illustrations. To enable effortless and enjoyable creation of multimedia blogs, we developed the system on a prototype robot called PaPeRo. Video messages are recorded and converted into text descriptions by PaPeRo using continuous speech recognition. PaPeRo then searches for suitable multimedia contents on the internet and databases, and then, based on the search results, chooses appropriate sympathetic comments by using natural language text retrieval. The retrieved contents, PaPeRos comments, and the video recording on the users blog is automatically uploaded and edited. The system was evaluated by 10 users for creating travel blogs and proved to be helpful for both inexperienced and experienced users. The system enabled easy multimedia-rich blog creation and even provided users the pleasure of chatting with PaPeRo.