Yasuhiro Asa

Predicting Evidence of Understanding by Monitoring User's Task Manipulation in Multimodal Conversations

The aim of this paper is to develop animated agents that can control multimodal instruction dialogues by monitoring users behaviors. First, this paper reports on our Wizard-of-Oz experiments, and then, using the collected corpus, proposes a probabilistic model of fine-grained timing dependencies among multimodal communication behaviors: speech, gestures, and mouse manipulations. A preliminary evaluation revealed that our model can predict a instructors grounding judgment and a listeners successful mouse manipulation quite accurately, suggesting that the model is useful in estimating the users understanding, and can be applied to determining the agents next action.

Explore/exploit strategic movement of co-occurrence network based on a conscious/subconscious model

Computation learning from living organisms should be useful for adapting to uncomputable problems in complex social systems. According to an interpretation of Gödels incompleteness theorems, a human is a “more than Turing computation (MTTC)” system consisting of a Turing machine represented as a computable recursive function and an environmental interaction represented as an oracle function. Recent findings from both science and philosophy show that a human is an autopoietic system that reproduces itself by recursion of consciousness and subconsciousness with self-reference and contingency. Using an approach combining MTTC and autopoiesis, we propose a conscious/subconscious model in which subconscious/necessary recursion and conscious/contingent recursion circulate doubly with environmental interaction. Application of this model to a web search with user interaction revealed that the co-occurrence network shows strategic movement in which the exploitation of known information is balanced against the exploration of unknown information. This movement indicates that our conscious/subconscious model is an MTTC/autopoietic system that improves the fitness for the uncomputable problem of ambiguous user expectation.