Dai Hasegawa

The effect of metaphoric gestures on schematic understanding of instruction performed by a pedagogical conversational agent

In this paper, we examine the impact of metaphoric gestures performed by Pedagogical Conversational Agent (PCA) on learners’ memorization of technical terms, understanding of relationships between abstract concepts, learning experience, and perception of the PCA. The study employed a one-factor three-level between-participants design where we manipulated gesture factor (speech-gesture match vs. speech-gesture mismatch vs. no-gesture). The data of 97 students were acquired in on-line learning environment. As the results, while there was no effect found on memorization of technical terms, we found that students showed accurate schematic understanding of the relationship between abstract concepts when the PCA used metaphoric gestures matched to speech content than when used gestures mismatched, and no gesture. Contrary to the result, we also found that students judged the PCA useful, helpful, and felt the PCA looked like a teacher when performed mismatched gestures to speech content than when performed matched gesture.

Creating a Gesture-Speech Dataset for Speech-Based Automatic Gesture Generation

In our research, we recorded 298 min (1049 sentences) of speech audio data and the motion capture data of the accompanying gestures from two 25-year-old male participants aiming for future usage in deep learning concerning gesture and speech. The data was recorded in form of an interview, the participant explaining a topic prepared in advance, using a headset microphone and the motion capture software Open image in new window . The speech audio was stored in mp3, and the motion data was stored in bvh, related as data from the same sentence. We aimed to mainly acquire metaphoric gestures and iconic, as categorized by McNiel. For the categories of the recorded gestures, metaphoric gestures appeared the most, 68.41% of all gestures, followed by 23.73% beat gestures, 4.76% iconic gestures, and 3.11% deictic gestures.

Speech-to-Gesture Generation: A Challenge in Deep Learning Approach with Bi-Directional LSTM

In this research, we take a first step in generating motion data for gestures directly from speech features. Such a method can make creating gesture animations for Embodied Conversational Agents much easier. We implemented a model using Bi-Directional LSTM taking phonemic features from speech audio data as input to output time sequence data of rotations of bone joints. We assessed the validity of the predicted gesture motion data by evaluating the final loss value of the network, and evaluating the impressions of the predicted gesture by comparing it with the actual motion data that accompanied the audio data used for input and motion data that accompanied a different audio data. The results showed that the accuracy of the prediction for the LSTM model was better than a simple RNN model. In contrast, the impressions evaluation of the predicted gesture was rated lower than the original and mismatched gestures, although individually some predicted gestures were rated the same degree as the mismatched gestures.

Student interactions with e-learning systems: User and topic analysis

This paper presents student interaction data obtained from an e-learning system to understand student behavior and explore student learning patterns. A graphical programming course is examined in terms of individual user interests and accessed e-learning topics. The experimental results show that students primarily used the e-learning system to access topics that included a certification test exam. In addition, student-learning flow is extracted from the system.

Change in Subjective Evaluation of Weight by the Proteus Effect

In virtual environments, it is known that synchronizing the actions of an avatar with the operator causes a change in the operator’s self-perception, resulting in a change of behavior. This phenomenon is called the Proteus effect. In the research concerning the Proteus effect, researches on the behavior change have been actively conducted. However, the difference between the self-perception changed by the Proteus Effect and the physical sense in real world has not been sufficiently discussed. Therefore, in this research, we prepared two different kinds of avatars with different appearances, and investigated the influence of the change of self-perception changed for each avatar on the evaluation of physical sensation. As a result, a significant difference in the evaluation of weight between the two conditions was found. It was clarified that the Proteus effect affects the evaluation of physical sensation.

P-30 Evaluation of Avatar Mediated Distant-care System by the Elderly

In recent years, long-term, continuous distant-care has been drawing an attention to maintain elderly persons’ healthy life expectancy. And under such circumstances, a novel approach of distant-care system has been proposed, in which the elderly persons are represented by an avatar, protecting their privacy. In this paper, we empirically examine the technology acceptance of an avatar-mediated distant-care system in elderly persons. The study employs one-factor (avatar-mediated distant-care system vs. image based distant-care system) with-in participant design and evaluate by using a questionnaire. We had 34 participants and an average age of participants was 70.4. As the results of experiment, participants rated that the avatar-mediated distant-care system was less discomfort, has less risk of privacy violation, and more likable than the image based distant-care system. We conclude, from above results, that the avatar-mediated distant-care system had advantageous properties in technology acceptance for elderly persons than the image based distant-care system.

Autonomous mobile digital signage: persuasiveness of life-like behaviors on daily zodiac horoscope

We examined the effects of life-like movements on persuasion and attention-drawing in an Autonomous Mobile Digital Signage (AMDS) that offered zodiac horoscopes. The study employed a one-factor three-level between-participants design where we manipulated the life-likeness of movement of the AMDS (life-like vs. monotonic vs. Fixed). We set up the three versions of the AMDS for eight days in rotation, and collected the data of the number of users and the percentage of the users who were agreed with the horoscope results told by the AMDS. As the results, we found that the AMDS with life-like movement had higher number of users than the AMDS with no movement. We also found that the percentage of the users who were agreed with the horoscope results increased when the AMDS performed life-like movement compared with other two conditions.

Development of Web-Based Platform for Privacy Protective Avatar Mediated Distance-Care

We propose a web-based platform for privacy-protected avatar mediated distant-care. The system can avateer an elderly person based on their articular angles acquired by a motion capture system, and render the whole body animation of the avatar on a web-browser. The avatar-mediated architecture design allow caregivers to observe the elderly persons’ behavior all day long, without violating their privacy. In addition, we will show an example implementation integrating multiple sensors in the platform. By integrating multiple sensors into the platform, caregivers can observe the avatar with the elderly person’s health-related status. The implemented system showed the low communication bandwidth dependency, and sufficient frame-rate for the animation to be smoothly seen.

The Effects of Life-Likeness on Persuasion and Attention-Drawing in a Mobile Digital Signage

In this paper, we examined the effects of life-like movements on persuasion and attention-drawing in a Mobile Digital Signage (MDS). The study employed a one-factor three-level between-participants design where we manipulated the life-likeness of movement of the MDS (life-like movement vs. simple movement vs. no-movement). We set up the three versions of the MDS at our department building for eight days in rotation, and collected the data of the number of users and the percentage of the users who answered YES at the end of the interactions. As the results of our analysis on the data of the number of users, we found that there was a main effect in the movements of MDS and the MDS with life-like movement had higher than the MDS with no movement. In addition, the analysis on the percentage of the users who answered Yes showed that there were statistically significant differences between the MDS with life-like movements and the MDS with sim ple movement, and the MDS with life-like movement and the MDS with no movement. The results indicated that the power of persuasion and attention drawing increased when the MDS performed life-like movement.