Ryota Nishimura

Integrating Symbols and Signals Based on Stream Reasoning and ROS

We have developed PRactical INTElligent aPplicationS (PRINTEPS) which is a total intelligent application development platform. This paper introduces an application of PRINTEPS for detecting events by using stream reasoning and Robot Operating System (ROS), and for integrating image sensing with knowledge processing. Based on this platform, we demonstrate that the behaviors of a robot in a robot cafe can be modified by changing the applicable rule sets.

Voice interaction system with 3D-CG virtual agent for stand-alone smartphones

In this paper, we propose a voice interaction system using 3D-CG virtual agents for stand-alone smartphones. Because the proposed system can handle speech recognition and speech synthesis on a stand-alone smartphone differently from the existing mobile voice interaction systems, this system enables us to talk naturally without encountering delays caused by network communications. Moreover, proposed system can be fully customized by dialogue scripts, Java-based plugins, and Android APIs. Therefore, developers can make original voice interaction systems for smartphones easily based on proposed system. We have made a subset of the proposed system available as open-source software. We expect that this system will contribute to studies of human-agent interaction using smartphones.

Development of a dialogue scenario editor on a web browser for a spoken dialogue system

We developed a scenario editor to improve the environment of scenario editing for a spoken dialogue system. The purpose of this study is to improve the editing environment of the spoken dialogue system. We have developed MMDAgent (a fully open-source toolkit for voice interaction systems), which runs on a variety of platforms such as personal computers (running Windows, Mac OS, or Linux) and smartphones (running Android). So that MMDAgent can be used in a similar environment, the scenario editor is also implemented on a Web browser. Experiments were conducted for subjects using the scenario editor of our system. It was found that our system provides better readability of a scenario and allows easier editing.

Response timing generation and response type selection for a spontaneous spoken dialog system

If a dialog system can respond to a user as naturally as a human, the interaction will appear smoother. In this research, we aim to develop a dialog system that emulates human behavior in a chat-like dialog. The proposed system makes use of a decision tree to generate chat-like responses at the appropriate times. These responses include “aizuchi” (back-channel), “repetition”, “collaborative completion”, etc. The system also reacts robustly to the users overlapping utterances (barge-in) and disfluencies. The subjective evaluation shows that there is a high degree of naturalness in the timing of ordinary responses, overlap, and aizuchi, and that the dialog system exhibits user-friendly behavior. The recorded voices system was preferred, and almost all subjects felt familiarity with aizuchi, and the barge-in was also useful.

Development and Evaluation of Spoken Dialog Systems with One or Two Agents through Two Domains

Almost all current spoken dialog systems treat dialog as that where a single user talks to an agent. On the other hand, we set out to investigate a multiparty dialog system that deals with two agents and a single user. We developed a three person (one user and two agents) and a two person (one user and one agent) dialog system to consider the same dialog tasks, that is, “Which do you prefer, udon or ramen (Japanese noodle or Chinese noodle)?” and ‘Which do you want to travel to Hokkaido or Okinawa (snowy region or tropical region)?” and compared them with respect to user behavior and satisfaction. According to the results of the experiments, the three person dialog system performed better in terms of lively conversation, and user can talk with the agents more like chatting.

Development environment of a spoken dialogue system based on PRINTEPS

In this paper, we describe the development of a spoken dialogue system based on PRINTEPS architecture. This spoken dialogue system is composed of five modules (speech recognition, language understanding, dialogue management, response generation, speech synthesis). In PRINTEPS, when calling the spoken dialogue system, system developers specify the small-scale dialogue goal. The system performs a dialogue with the user in order to obtain necessary information from the user. Dialogue processing rules corresponding to the dialogue goal is prepared in advance. The advantage of PRINTEPS-based system development is that system developers can build a spoken dialogue system without the knowledge of spoken dialogue system.

A VoIP-based voice interaction system for a virtual telephone operator using video calls

We have developed a telephone operator system using a spoken dialogue system. The system can be used for video calls, with Microsoft Lync used as the VoIP client. The spoken dialogue system is used in the system to display a 3D character, and to interact with the user over the video call. In this study, the interactive scenario for the telephone operator is automatically generated from a university employee database. Experimental results from subjects using the system showed good system performance.

Proposal of reminiscence therapy system using spoken dialog to suppress dementia

The number of dementia patients has increased in recent years. The burden on caregivers has also increased. Nevertheless, no established treatment for dementia exists. Controlling dementia progression is an important goal of dementia treatment. The reminiscence method is one means of suppressing dementia progression. In the reminiscence method, a caregiver talks with a dementia patient. However, an important difficulty is the necessity of listening to repeated dialog many times, which greatly increases the burden on caregivers. Therefore, for this research, we propose a system to support reminiscence using MMDAgent: a spoken dialog system.

Design of Reminiscence Therapy System for Elderly People with Dementia

In recent years, the number of dementia patients has increased, imposing ever greater burdens on caregivers. However, no treatment has been established for dementia. Controlling its progression is a goal of treatment. Using the reminiscence method, one means of suppressing the progression of dementia, a caregiver talks with a dementia patient. However, a difficulty of the method that it is necessary to conduct a dialogue many times, which also burdens caregivers. For that reason, we propose a method to support reminiscence using a spoken dialogue system called MMDAgent in place of a caregiver. The system apparently decreases burdens on caregivers. As described herein, we specifically present the design of the proposed system.

Proposal of voice control-based interactive design method of agent system

Agent systems have been designed and deployed using recent agent technologies. However, design and deployment of these systems remain as difficult tasks of developers because agents have situational and nondeterministic characteristics and because any useful design and deployment support facilities have not been provided for developers. To raise the efficiency of the agent system design and deployment process, we propose a voice control-based interactive design method of agent system founded on an agent-repository-based multiagent framework.