Akitoshi Okumura

Speech-activated text retrieval system for multimodal cellular phones

The paper describes an on-line manual page retrieval system activated by spoken queries for multimodal cellular phones. The system recognizes a users naturally spoken queries by telephone LVCSR and searches an on-line manual with a retrieval module on a server. The user can view the retrieved data on the screen of the phone via Web access. The LVCSR module consists of a telephone acoustic model and an n-gram language model derived from a task query corpus. The adaptation method using the target manual is also presented. The retrieval module utilizes pairs of words with dependency relations and also distinguishes affirmative and negative expressions to improve precision. The proposed system gives 82.6% keyword recognition accuracy and 77.5% task achievement rate. The field trial of the system is now underway.

Parallel LVCSR Algorithm for Cellphone-Oriented Multicore Processors

A parallel large vocabulary continuous speech recognition (LVCSR) algorithm for cellphone-oriented multicore processors is proposed. We introduce an acoustic look-ahead and blockwise computation to our compact LVCSR algorithm, in order to distribute its computational load to multiple CPU cores. We implement the proposed LVCSR algorithm on an evaluation board of a cellphone-oriented three CPU core chip, and show real-time processing of stand-alone LCVSR on cellphones can be achieved with recognition vocabulary of about 50,000 words. We also implement a speech-input text retrieval system using the proposed LVCSR on the same evaluation board, and confirm the ability of the proposed LVCSR algorithm to provide comfortable responses to query sentences spoken to a cellphone, without requiring any outside resources

An automatic speech translation system on PDAs for travel conversation

We present an automatic speech-to-speech translation system for personal digital assistants (PDAs) that helps oral communication between Japanese and English speakers in various situations while traveling. Our original compact large vocabulary continuous speech recognition engine, compact translation engine based on a lexicalized grammar, and compact Japanese speech synthesis engine lead to the development of a Japanese/English bi-directional speech translation system that works with limited computational resources.

Automatic Interpretation System Integrating Free-Style Sentence Translation and Parallel Text Based Translation

This paper proposes an automatic interpretation system that integrates freestyle sentence translation and parallel text based translation. Free-style sentence translation accepts natural language sentences and translates them by machine translation. Parallel text based translation provides a proper translation for a sentence in the parallel text by referring to a corresponding translation of the sentence and supplements free-style sentence translation. We developed a prototype of an automatic interpretation system for Japanese overseas travelers with parallel text based translation using 9206 parallel bilingual sentences prepared in task-oriented manner. Evaluation results show that the parallel text based translation covers 72% of typical utterances for overseas travel and the user can easily find an appropriate sentence from a natural utterance for 64% of typical travelers tasks. This indicates that the user can benefit from reliable translation based on parallel text for fundamental utterances necessary for overseas travel.

A Speech Translation System with Mobile Wireless Clients

We developed a client-server speech translation system with mobile wireless clients. The system performs speech translation between English and Japanese of travel conversation and helps foreign language communication in an area where wireless LAN connection is available.

An HMM-based text segmentation method using variational Bayes approach and its application to LVCSR for broadcast news

Recent progress in large vocabulary continuous speech recognition (LVCSR) has raised the possibility of applying information retrieval techniques to the resulting text. This paper presents a novel unsupervised text segmentation method. Assuming a generative model of a text stream as a left-to-right hidden Markov model (HMM), text segmentation can be formulated as model parameter estimation and model selection using the text stream. The formulation is derived based on the variational Bayes framework, which is expected to work well with highly sparse data such as text. The effectiveness of the proposed method is demonstrated through a series of experiments, where broadcast news programs are automatically transcribed and segmented into separate news stories.

Information Classification and Navigation based on 5W1H of the Target Information

This paper proposes a method by which 5W1H (who, when, where, what, why, how, and predicate) information is used to classify and navigate Japanese-language texts. 5W1H information, extracted from text data, has an access platform with three functions: episodic retrieval, multi-dimensional classification, and overall classification. In a six-month trial, the platform was used by 50 people to access 6400 newspaper articles. The three functions proved to be effective for office documentation work and the precision of extraction was approximately 82%.

Topic detection based on dialogue history

In this paper, we propose a topic detection method using a dialogue history for a speech translator. The method uses a k-nearest neighbor method for the algorithm, automatically clusters target topics into smaller topics grouped by similarity, and incorporates dialogue history weighted in terms of time to detect and track topics on spoken phrases. From the evaluation of detection performance using test data comprised of realistic spoken dialogue, the method has shown to perform better with clustering incorporated, and when combined with dialogue history of three sentences, gives detection accuracy of 72.1%.

Documentation know-how sharing by automatic process tracking

Some groupware products support office jobs by providing cooperative functions such as workflow management. However, they cannot support documentation jobs because the jobs need individual creativity which is difficult to share. This paper proposes a ‘Know-how Sharing Agent’ which allows individual creativity used for documentation jobs to be shared. The Know-how Sharing Agent supports document creation by preparing the most exemplary document and its documentation operations. The documentation operations were also saved by the agent automatically when the document was created. This type of agent thus promotes documentation activities by sharing the know-how needed for documentation.

Identity Verification of Ticket Holders at Large-scale Events Using Face Recognition

This paper proposes a system of verifying the identity of ticket holders at large-scale events using face recognition, which is called Ticket ID System. Such a system has been required to prevent illegal resale such as ticket scalping. Since illegal resale is a critical problem for popular events in Japan, strict steps are followed for verifying individuals holding tickets at event venues by human visual inspection with ID cards. This task is time consuming for venue attendants. It is also stressful because ticket holders feel uncomfortable when being kept waiting. The problem in verifying ticket holders is how to simultaneously verify identities efficiently and prevent individuals from impersonating others at a large-scale event in which tens of thousands of people participate. Ticket ID system makes it possible to secure the identity of the purchaser and holder of a ticket by using a face-recognition system developed for tablet terminals. Ticket ID System was proven effective for preventing illegal resale by verifying 50,324 attendees at a large concert of a popular music group. The average accuracy of face recognition was 90%. The average time for identity verification was 7 seconds per person including guidance to ticket holders, which decreased identity-verification time by 30% compared to using only human visual inspection as well as reducing the psychological workload of venue attendants. Survey results obtained from the attendees showed that 94.6% felt it provided more equity in ticket purchasing than methods used before, 83% felt it provided added convenience in verification, and 93.8% felt it would effectively prevent illegal resale.