Tsuneo Kato

A marker-less Augmented Reality based on fast fingertip detection for smart phones

We propose a marker-less Augmented Reality (AR) application on users hand based on a fast fingertip detection technique for smart phones. A conventional hand-based marker-less AR system does not have enough accuracy and speed in human hand detection on a mobile device. This paper presents a fast hand position and posture estimation algorithm based on a fast and robust detection of non-skin regions around the hand. The proposed method realized rendering of virtual 3D models on a hand over 10 frames per second (fps) on a smart phone. Simulation results show that we can archive up to 90% complexity reduction and more accurate than the conventional method.

A camera-based tangible controller for cellular phones

This paper proposes a novel easy-to-use camera-based tangible controller for cellular phone applications. It realizes continuous analog input by tracking a marker at the top end of a controller device attached to the embedded camera. In contrast to the conventional keypad which enables limited operability to four discrete directions, the proposed controller brings not only an unconstrained continuous input to arbitrary directions but also continuous input for depth and for a rotation angle. In order to evaluate operability, we conducted a user experiment of time trial for path tracing, and the results showed that the subjects completed the task with the proposed controller in 27.9% less than with the conventional keypad input.

Typing Tutor: Automatic Error Detection and Instruction in Text Entry for Elderly People

Smartphones offer new opportunities to improve the lives of elderly people. Although many elderly people are interested in smartphones, most of them face difficulties in self-instruction and need support. Text entry, which is essential for various applications, is one of the most difficult operations to master. Therefore, we propose a tutoring system for text entry that can perform the role of a human tutor by detecting errors and providing the instructions to resolve the errors. We constructed the tutoring system based on the collected data in a user study. An evaluation with novice elderly people (60+) showed that the tutoring system increased typing speed by 17.2% and reduced error incidence by 59.1% compared with users initial rate. The improvement rates were almost same as that of human tutors.

Features of learner corpus collected with joining-in type robot assisted language learning system

Computer assisted language learning (CALL) is going to be more attractive, more interactive and more realistic by introducing humanoid robots. We propose a novel joining-in type robot assisted language learning (RALL) system using two humanoid robots, one playing a role of a teacher and the other playing a role of an advanced learner. The system has a seamless scenario of a natural dialogue flow among two robots and a learner to draw the learner into the dialogue. We designed the system to switch two modes of learning: tutoring and peer learning. As a characteristic behavior of the peer learning, we expect an “alignment” phenomenon between the robots sample answer and the learners utterance. That is to say, the learner learns a sentence pattern by mimicking a sample answer presented by the advanced learner robot. We created a learner corpus of over 60 participants, carried out an analysis on how much alignment occurred in the interactions.

Easy Handheld Training: Interactive Self-learning App for Elderly Smartphone Novices

Smartphones have great potential for elderly people to enrich their lives. Elderly people, however, hesitate to use smartphones compared to younger people due to several factors such as anxieties about the difficulties of unfamiliar devices and the lack of daily assistance. Moreover, a conventional research reported that elderly IT novices struggled with basic operations in the first stage. Hence, we tried to find the common issues faced by the elderly with no previous smartphone experience and then created an interactive self-learning application app of the basic smartphone operations. Our demonstration app was designed to give hands-on experience by using integrated real videos. Results of usability testing showed that the subjects easily learned by themselves how to operate the smartphones without background information. The subjective evaluation results showed that the app engaged the interest of the subjects and also gave them confidence about acquiring operational skills by themselves.

Difference in Eye Gaze for Floor Apportionment in Native- and Second-Language Conversations

In face-to-face communication, eye gaze is known to play various roles such as managing the attention of interlocutors, expressing intimacy, exercising social control, highlighting particular speech content, and coordinating floor apportionment. For second language (L2) communication, one’s perception of eye gaze is expected to have more importance than for native language (L1) because eye gaze is assumed to partially compensate for the deficiencies of verbal expressions. This paper examines and clarifies the efficiency of the function of eye gaze in the coordination of floor apportionment through quantitative analyses of eye gaze during three-party conversations in L1 and L2. Specifically, the authors conducted ANOVA tests on the eye-gaze statistics of a speaker and two listeners during utterances while focusing on whether floor-switch occurs subsequent to the utterance. The analysis results show that the listener who is gazed at more by the speaker is more likely to be the next speaker with a higher probability in L2 than in L1 conversations. Meanwhile, the listeners gaze more at the speaker in L2 than in L1 conversation for both the utterances just before a floor switch and cases with no floor switch. These results support the observation that the eye gaze of the speaker is efficient for floor apportionment in L2 conversations and suggest that longer listeners’ eye gazes in L2 conversations also function efficiently in smooth floor apportionment.

The Retention Effect of Learning Grammatical Patterns Implicitly Using Joining-in-Type Robot-Assisted Language-Learning System.

Conducting a multiparty conversation among two robots and a human learner for the purpose of language learning is a novel idea. It can help in conveying grammatical information to the human learner in an implicit manner. The main focus in this paper is the quantification of the level of retention of what was learned implicitly over a period of four weeks. We had evaluated the utterances of the human learners on the level of similarity of n-grams with a reference answer, and on the basis of grammatical correctness of use. The experiments revealed effect of repletion of implicit learning for learning corrective use of grammatical patterns.

BubbleFlick: investigating effective interface for Japanese text entry on smartwatches

We propose BubbleFlick, an effective interface for Japanese text entry on smartwatches. While various ideas have been proposed to provide easy and fast text entry for the Latin alphabet, Japanese text entry has additional challenges such as having more than fifty syllabary characters, or kana, to enter and the subsequent kana-kanji conversion, which translates a sequence of the syllabary characters into a standard expression with a mixture of kanji and kana characters. This paper focuses on interfaces for entry of kana syllabary characters. We designed and prototyped three interfaces: 1) Japanese kana syllabary keyboard, 2) Dial&Flick interface, and 3) DualBelts interface. Through a comparative pilot study of the prototypes, we refined the most promising Dial&Flick interface into BubbleFlick. BubbleFlick provides the widest possible area for easy flick operations while also leaving an area for editing text. We conducted a 30-day consecutive user study on BubbleFlick in comparison with Googles latest Japanese text-entry method based on a numeric keypad. After thirty days, BubbleFlick showed a text-entry speed of over 35 characters per minute, which was comparable to Googles numeric-keypad-based method for novice participants. Through the user study, BubbleFlick showed a lower error rate and gave us informative hints for further improvement.

Floor Apportionment and Mutual Gazes in Native and Second-Language Conversation

Quantitative analysis of gazes between a speaker and listeners was conducted from the viewpoint of mutual activities in floor apportionment, with the assumption that mutual gaze plays an important role in coordinating speech interaction. We conducted correlation analyses of the speakers and listeners gazes in a three-party conversation, comparing native language (L1) and second language (L2) interaction in two types (free-flowing and goal-orient- ed). The analyses showed significant correlations between gazes from the current to the next speaker and those from the next to the current speaker during utterances preceding a speaker change in L1 conversation, suggesting that the participants were coordinating their speech turns with mutual gazes. In L2 conversation, however, such a correlation was found only in the goal-oriented type, suggesting that linguistic proficiency may affect the floor-apportionment function of mutual gazes, possibly because of the cognitive load of understanding/producing utterances.

“Re:ROS”: Prototyping of Reinforcement Learning Environment for Asynchronous Cognitive Architecture

Reinforcement learning (RL), which is a field of machine learning, is effective for behavior acquisition in robots. Asynchronous cognitive architecture, which is a method to model human intelligence, is also effective for behavior acquisition. Accordingly, the combination of RL and asynchronous cognitive architecture is expected to be effective. However, early work on the RL toolkit cannot apply asynchronous cognitive architecture because it cannot solve the difference between the asynchrony, which the asynchronous cognitive architecture has, and the synchrony, which RL modules have. In this study, we propose an RL environment for robots that can apply the asynchronous cognitive architecture by applying asynchronous systems to RL modules. We prototyped the RL environment named “Re:ROS.”