Sandra Y. Okita

Synchronized gesture and speech production for humanoid robots

We present a model that is capable of synchronizing expressive gestures with speech. The model, implemented on a Honda humanoid robot, can generate a full range of gesture types, such as emblems, iconic and metaphoric gestures, deictic pointing and beat gestures. Arbitrary input text is analyzed with a part-of-speech tagger and a text-to-speech engine for timing information of spoken words. In addition, style tags can be optionally added to specify the level of excitement or topic changes. The text, combined with any tags, is then processed by several grammars, one for each gesture type to produce several candidate gestures for each word of the text. The model then selects probabilistically amongst the gesture types based on the desired degree of expressivity. Once a gesture type is selected, it coincides with a particular gesture template, consisting of trajectory curves that define the gesture. Speech timing patterns and style parameters are used to modulate the shape of the curve before it sent to the whole body control system on the robot. Evaluation of the models parameters were performed, demonstrating the ability of observers to differentiate varying levels of expressiveness, excitement and speech synchronization. Modification of gesture speed for trajectory tracking found that positive associations like happiness and excitement accompanied faster speeds, with negative associations like sadness or tiredness occurred at slower speeds.

Learning together: ASIMO developing an interactive learning partnership with children

Humanoid robots consist of biologically inspired features, human-like appearance, and intelligent behavior that naturally elicit social responses. Complex interactions are now possible, where children interact and learn from robots. A pilot study attempted to determine which features in robots led to changes in learning and behavior. Three common learning styles, lecture, cooperative, and self-directed, were implemented into ASIMO to see if children can learn from robots. General features such as monotone robot-like voice and human-like voice were compared. Thirty-seven children between the ages 4-to 10- years participated in the study. Each child engaged in a table-setting task with ASIMO that exhibited different learning styles and general features. Children answered questions in relation to a table-setting task with a learning measure. Promissory evidence shows that learning styles and general features matter especially for younger children.

Learning by Teaching Human Pupils and Teachable Agents: The Importance of Recursive Feedback

Feedback is important for learning. However, there are different types of feedback, and not all feedback is effective. Here we introduce recursive feedback (RF), which occurs when tutors observe their pupils use what they have been taught. Two experiments examined the value of RF during learning by teaching. In the first study adults taught another adult face to face about human biology. Those participants who observed their pupil interact with an examiner exhibited superior learning relative to individuals in several control conditions that included elements of learning by teaching but not RF. The second study examined whether RF benefits extend to teaching computerized teachable agents in regular classrooms. High school students played games in which they induced logical rules. Students taught their agent the governing rules. They received RF when they observed their teachable agent play a prediction game against a second competitor agent. On a posttest, these students exhibited greater abilities to use logic to solve novel problems compared to students in control conditions who received direct feedback by playing against the competitor agent themselves. RF may further generalize to nonteaching situations that also involve a production–appropriation cycle, such as do-it-yourself projects in which people have a chance to learn from how other people take up their handiwork.

Multimodal approach to affective human-robot interaction design with children

Two studies examined the different features of humanoid robots and the influence on childrens affective behavior. The first study looked at interaction styles and general features of robots. The second study looked at how the robots attention influences childrens behavior and engagement. Through activities familiar to young children (e.g., table setting, story telling), the first study found that cooperative interaction style elicited more oculesic behavior and social engagement. The second study found that quality of attention, type of attention, and length of interaction influences affective behavior and engagement. In the quality of attention, Wizard-of-Oz (woz) elicited the most affective behavior, but automatic attention worked as well as woz when the interaction was short. The type of attention going from nonverbal to verbal attention increased childrens oculesic behavior, utterance, and physiological response. Affective interactions did not seem to depend on a single mechanism, but a well-chosen confluence of technical features.

Augmenting pediatric constraint-induced movement therapy and bimanual training with video gaming technology

Cerebral palsy (CP) is the most common cause of physical disability in childhood. Hemiplegia is among the most common forms of CP and the resulting impaired hand function is one of the most disabling symptoms, affecting self-care activities such as feeding, dressing, and grooming. To date, evidence-based treatments are limited. Recent approaches, however, have capitalized on findings that show children with hemiplegia h ave residual motor capabilities and neuroplastic changes in nervous system function that emerge and improve with practice. Here the etiology and neural basis of hemiplegic CP is first briefly reviewed, followed by a description of the residual motor capabilities in the involved upper extremity and the potentia l role of intensive practice. Two promising approaches that target residual motor function, constraint-induced movement therapy (CIMT) and bimanual training, are then described. Recent evidence suggests that such task-oriented training approaches to re habilitation are enhanced when the tasks are meaningful to the performer. Increasingly, this means use of current technology, specifi cally video gaming, to maintain salience and motivation and target specific motor impairments. Thus a method for using commerc ially available video gaming, including the Nintendo Wii to augment such intensive treatment approaches is described. It is s uggested that with such intensive treatment programs, gaming can be an important compliment to, but not a replacement for, salient task-oriented activities in the real world and that video ga ming and virtual reality training will be an important part o f future rehabilitation efforts.

Learning by teaching with virtual peers and the effects of technological design choices on learning

Advancements in technology have brought about new forms of learning and online instruction that allow communication through virtual representations without physically meeting in person. This study builds on previous work involving recursive feedback that tests the hypothesis that an important facet of learning-by-teaching is the opportunity to watch ones pupil perform. Sixty graduate students examined the value of recursive feedback that occurred when tutors observed their pupil subsequently apply what they had been taught. The study took place in the virtual environment Second Life where adults tutored another adult about human biology through their virtual representations. The tutors who observed their pupil avatar interact with an examiner exhibited superior learning relative to several control conditions that included learning-by-teaching elements but not recursive feedback. The second study examined the effect of popular design choices on recursive feedback during learning-by-teaching (e.g., customization, look-alike features). The customization condition involved tutoring a pupil avatar that the participant customized prior to the study and observing the pupil avatar answer questions. The doppelganger look-alike condition involved tutoring a pupil avatar that looked like the participant and observing the pupil avatar answer questions. Results showed that conscious awareness of look-alike features and the extent to which one customizes the pupil avatar influences learning.

The relative merits of transparency: Investigating situations that support the use of robotics in developing student learning adaptability across virtual and physical computing platforms

This study examined whether developing earlier forms of knowledge in specific learning environments prepares students better for future learning when they are placed in an unfamiliar learning environment. Forty-one students in the fifth and sixth grades learned to program robot movements using abstract concepts of speed, distance and direction. Students in high-transparency environments learned visual programming to control robots (eg, organizing visual icons), and students in low-transparency environments learned syntactic programming to control robots (eg, text-based coding). Both groups received feedback and models of solutions during the learning phase. The assessment midway showed students in both conditions learned equally well when solving problems using familiar materials. However, a difference emerged when students were asked to solve new problems, using unfamiliar materials. The low-transparency group was more successful in adapting and repurposing their knowledge to solve novel problems that required the use of unfamiliar high-transparency materials. Students in the high-transparency group were less successful in adapting their knowledge when solving new problems using unfamiliar low-transparency materials. Both groups then proceeded to learn in the opposing transparency environments. The posttest revealed the benefits of initial learning in low-transparency environments as students performed better on repeated and new inferential problems across virtual and physical platforms. [ABSTRACT FROM AUTHOR]

Captain may i?: proxemics study examining factors that influence distance between humanoid robots, children, and adults, during human-robot interaction

This proxemics study examines whether the physical distance between robots and humans differ based on the following factors: 1) age: children vs. adults, 2) who initiates the approach: humans approaching the robot vs. robot approaching humans, 3) prompting: verbal invitation vs. non-verbal gesture (e.g., beckoning), and 4) informing: announcement vs. permission vs. nothing. Results showed that both verbal and non-verbal prompting had significant influence on physical distance. Physiological data is also used to detect the appropriate timing of approach for a more natural and comfortable interaction.

Effects of age on associating virtual and embodied toys.

Technologies such as videos, toys, and video games are used as tools in delivering education to young children. Do children spontaneously transfer between virtual and real-world mediums as they learn? Fifty-six children learned facts about a toy dog presented through varying levels of technology and interactivity (e.g., video game, stuffed animal, picture books). They then met a similar dog character in a new embodiment (e.g., as a stuffed animal if first met the dog as video character). Would children spontaneously generalize the facts they learned about the dog character across mediums (dynamic and static environments)? Results indicate that younger children were more likely to generalize facts across mediums. Specific aspects of the level of technology and interactivity had little effect.

Extended duration human-robot interaction: Tools and analysis

Extended human-robot interactions possess unique aspects which are not exhibited in short-term interactions spanning a few minutes or extremely long-term spanning days. In order to comprehensively monitor such interactions, we need special recording mechanisms which ensure the interaction is captured at multiple spatio-temporal scales, viewpoints and modalities(audio, video, physio). To minimize cognitive burden, we need tools which can automate the process of annotating and analyzing the resulting data. In addition, we also require these tools to be able to provide a unified, multi-scale view of the data and help discover patterns in the interaction process. In this paper, we describe recording and analysis tools which are helping us analyze extended human-robot interactions with children as subjects. We also provide some experimental results which highlight the utility of such tools.