Wing-Chee So

The development of co-speech gesture and its semantic integration with speech in 6- to 12-year-old children with autism spectrum disorders:

Previous work leaves open the question of whether children with autism spectrum disorders aged 6–12 years have delay in producing gestures compared to their typically developing peers. This study examined gestural production among school-aged children in a naturalistic context and how their gestures are semantically related to the accompanying speech. Delay in gestural production was found in children with autism spectrum disorders through their middle to late childhood. Compared to their typically developing counterparts, children with autism spectrum disorders gestured less often and used fewer types of gestures, in particular markers, which carry culture-specific meaning. Typically developing children’s gestural production was related to language and cognitive skills, but among children with autism spectrum disorders, gestural production was more strongly related to the severity of socio-communicative impairment. Gesture impairment also included the failure to integrate speech with gesture: in particular, supplementary gestures are absent in children with autism spectrum disorders. The findings extend our understanding of gestural production in school-aged children with autism spectrum disorders during spontaneous interaction. The results can help guide new therapies for gestural production for children with autism spectrum disorders in middle and late childhood.

Gesture is more effective than spatial language in encoding spatial information

The present research investigates whether producing gestures with and without speech facilitates route learning at different levels of route complexity and in learners with different levels of spatial skills. It also examines whether the facilitation effect of gesture is stronger than that of spatial language. Adults studied routes with 10, 13, and 16 steps and reconstructed them with sticks, either without rehearsal or after rehearsal by producing gestures with speech, gestures alone, or speech only. For all levels of route complexity and spatial skills, participants who were encouraged to gesture (with or without speech) during rehearsal had the best recall. Additionally, we found that number of steps rehearsed in gesture, but not that rehearsed in speech, predicted the recall accuracy. Thus, gesture is more effective than spatial language in encoding spatial information, and thereby enhancing spatial recall. These results further corroborate the beneficial nature of gesture in processing spatial information.

The Use of Hand Gestures to Communicate About Nonpresent Objects in Mind Among Children With Autism Spectrum Disorder

PURPOSE The current study examined whether children with autism spectrum disorder (ASD), in comparison with typically developing children, perceive and produce gestures to identify nonpresent objects (i.e., referent-identifying gestures), which is crucial for communicating ideas in a discourse. METHOD An experimenter described the uses of daily-life objects to 6- to 12-year-old children both orally and with gestures. The children were then asked to describe how they performed daily activities using those objects. RESULTS All children gestured. A gesture identified a nonpresent referent if it was produced in the same location that had previously been established by the experimenter. Children with ASD gestured at the specific locations less often than typically developing children. Verbal and spatial memory were positively correlated with the ability to produce referent-identifying gestures for all children. However, the positive correlation between Ravens Children Progressive Matrices score and the production of referent-identifying gestures was found only in children with ASD. CONCLUSIONS Children with ASD might be less able to perceive and produce referent-identifying gestures and may rely more heavily on visual-spatial skills in producing referent-identifying gestures. The results have clinical implications for designing an intervention program to enhance the ability of children with ASD to communicate about nonpresent objects with gestures.

Iconic gestures prime words: comparison of priming effects when gestures are presented alone and when they are accompanying speech

Previous studies have shown that iconic gestures presented in an isolated manner prime visually presented semantically related words. Since gestures and speech are almost always produced together, this study examined whether iconic gestures accompanying speech would prime words and compared the priming effect of iconic gestures with speech to that of iconic gestures presented alone. Adult participants (N = 180) were randomly assigned to one of three conditions in a lexical decision task: Gestures-Only (the primes were iconic gestures presented alone); Speech-Only (the primes were auditory tokens conveying the same meaning as the iconic gestures); Gestures-Accompanying-Speech (the primes were the simultaneous coupling of iconic gestures and their corresponding auditory tokens). Our findings revealed significant priming effects in all three conditions. However, the priming effect in the Gestures-Accompanying-Speech condition was comparable to that in the Speech-Only condition and was significantly weaker than that in the Gestures-Only condition, suggesting that the facilitatory effect of iconic gestures accompanying speech may be constrained by the level of language processing required in the lexical decision task where linguistic processing of words forms is more dominant than semantic processing. Hence, the priming effect afforded by the co-speech iconic gestures was weakened.

Absence of delay in spontaneous use of gestures in spoken narratives among children with Autism Spectrum Disorders

BACKGROUND Gestures are spontaneous hand movements produced when speaking. Despite gestures being of communicative significance, little is known about the gestural production in spoken narratives in six- to 12-year-old children with Autism Spectrum Disorders (ASD). AIMS The present study examined whether six- to 12-year-old children with ASD have a delay in gestural production in a spoken narrative task, in comparison to their typically-developing (TD) peers. METHODS AND PROCEDURES Six- to-12-year-old children with ASD (N=14) and their age- and IQ-matched TD peers (N=12) narrated a story, which could elicit spontaneous speech and gestures. Their speech and gestures were then transcribed and coded. OUTCOMES AND RESULTS Both groups of children had comparable expressive language skills. Children with ASD produced a similar number of pointing and marker gestures to TD children and significantly more iconic gestures in their spoken narratives. While children with ASD produced more reinforcing gestures than their TD counterparts, both groups of children produced comparable numbers of disambiguating and supplementary gestures. CONCLUSIONS Our findings indicate that children with ASD may be as capable as TD children in gestural production when they engage in spoken narratives, which gives them spontaneity in producing gestures.

Using a social robot to teach gestural recognition and production in children with autism spectrum disorders

Abstract While it has been argued that children with autism spectrum disorders are responsive to robot-like toys, very little research has examined the impact of robot-based intervention on gesture use. These children have delayed gestural development. We used a social robot in two phases to teach them to recognize and produce eight pantomime gestures that expressed feelings and needs. Compared to the children in the wait-list control group (N = 6), those in the intervention group (N = 7) were more likely to recognize gestures and to gesture accurately in trained and untrained scenarios. They also generalized the acquired recognition (but not production) skills to human-to-human interaction. The benefits and limitations of robot-based intervention for gestural learning were highlighted. Implications for Rehabilitation Compared to typically-developing children, children with autism spectrum disorders have delayed development of gesture comprehension and production. Robot-based intervention program was developed to teach children with autism spectrum disorders recognition (Phase I) and production (Phase II) of eight pantomime gestures that expressed feelings and needs. Children in the intervention group (but not in the wait-list control group) were able to recognize more gestures in both trained and untrained scenarios and generalize the acquired gestural recognition skills to human-to-human interaction. Similar findings were reported for gestural production except that there was no strong evidence showing children in the intervention group could produce gestures accurately in human-to-human interaction.

Social and Communication Skills Predict Imitation Abilities in Children with Autism

This study examined whether social and communicative skills would predict the ability to imitate conventional-interactive gestures (e.g., HAND WAVING for saying goodbye) in children with autism spectrum disorders (ASD). Twenty 6- to 11-year-old Chinese-speaking children with ASD participated. Their social and communicative skills were assessed based on their caregivers’ ratings on their social and communication impairments and the observation of their gesture production in spontaneous interactions with the caregivers three months prior to the administration of the gesture imitation task. After controlling for gesture recognition and visual-motor coordination skills, the results of the partial correlation analyses have shown that there was significant association among the number of conventional-interactive gestures imitated accurately, the caregivers’ ratings, and the number of conventional-interactive gestures produced during interactions. Four regression analyses were thus conducted. We found the severity of social and communication impairments predicted the imitation of communicative gestures, with such relation mediated by the production of conventional-interactive gestures during the interactions. Overall, the results suggest that social and communication skills may play a causal role in imitation abilities in children with autism.

Neural evidence for reduced automaticity in processing emotional prosody among men with high levels of autistic traits

This study aimed to examine individual differences in the integration of emotional prosody when processing semantic meaning in speech among men with high and low levels of autistic traits, as measured by the Autism Spectrum Quotient (AQ). The behavioral and neural responses of high- and low-AQ men during semantic valence judgment were compared. The stimuli were positive or negative words spoken with either happy or sad prosody; in other words, the prosody was either congruous or incongruous to the valence of meaning. Participants were required to judge the (positive vs. negative) valence of word meaning as accurately and as quickly as possible while ignoring emotional prosody. Behavioral results showed that high-AQ men responded significantly more slowly than low-AQ men in all stimulus conditions, indicating lower automaticity in processing emotional speech. Neural data revealed that low-AQ men (but not high-AQ men) had significantly increased N200 and N400 amplitudes for incongruous (compared to congruous) stimuli spoken with happy prosody. Our findings supported our hypotheses that high levels of autistic traits are associated with reduced behavioral automaticity and less differential neural resources allocated to processing emotional speech stimuli with different cognitive demands.

Robot-based intervention may reduce delay in the production of intransitive gestures in Chinese-speaking preschoolers with autism spectrum disorder

BackgroundPast studies have shown that robot-based intervention was effective in improving gestural use in children with autism spectrum disorders (ASD). The present study examined whether children with ASD could catch up to the level of gestural production found in age-matched children with typical development and whether they showed an increase in verbal imitation after the completion of robot-based training. We also explored the cognitive and motor skills associated with gestural learning.MethodsChildren with ASD were randomly assigned to two groups. Four- to 6-year-old children with ASD in the intervention group (N = 15) received four 30-min robot-based gestural training sessions. In each session, a social robot, NAO, narrated five stories and gestured (e.g., both hands clapping for an awesome expression). Children with ASD were told to imitate the gestures during training. Age-matched children with ASD in the wait-list control group (N = 15) and age-matched children with typical development (N = 15) received the gestural training after the completion of research. Standardized pretests and posttests (both immediate and delayed) were administered to assess the accuracy and appropriateness of gestural production in both training and novel stories. Children’s language and communication abilities, gestural recognition skills, fine motor proficiencies, and attention skills were also examined.ResultsChildren with ASD in the intervention condition were more likely to produce accurate or appropriate intransitive gestures in training and novel stories than those in the wait-list control. The positive learning outcomes were maintained in the delayed posttests. The level of gestural production accuracy in children with ASD in the delayed posttest of novel stories was comparable to that in children with typical development, suggesting that children with ASD could catch up to the level of gestural production found in children with typical development. Children with ASD in the intervention condition were also more likely to produce verbal markers while gesturing than those in the wait-list control. Gestural recognition skills were found to significantly predict the learning of gestural production accuracy in the children with ASD, with such relation partially mediated via spontaneous imitation.ConclusionsRobot-based intervention may reduce the gestural delay in children with ASD in their early childhood.

How We Think about Temporal Words: A Gestural Priming Study in English and Chinese

Spatial metaphors are used to represent and reason about time. Such metaphors are typically arranged along the sagittal axis in most languages. For example, in English, “The future lies ahead of us” and “We look back on our past.” This is less straightforward for Chinese. Specifically, both the past and future can either be behind or ahead. The present study aims to explore these cross-linguistic differences by priming auditory targets (e.g., tomorrow) with either a congruent (i.e., pointing forwards) or incongruent (i.e., pointing backwards) gesture. Two groups of college-age young adult participants (English and Chinese speakers) made temporal classifications of words after watching a gestural prime. If speakers represent time along the sagittal axis, they should respond faster if the auditory target is preceded with a gesture indicating a congruent vs. incongruent spatial location. Results showed that English speakers responded faster to congruent gesture-word pairs than to incongruent pairs, mirroring spatio-temporal metaphors commonly recruited to talk about time in their native language. However, such an effect of congruency was not found for Chinese speakers. These findings suggest that while the spatio-temporal metaphors commonly recruited to talk about time help to structure the mental timelines of English speakers, the varying instances in how time is represented along the sagittal axis in Chinese may lead to a more variable mental timeline as well. In addition, our findings demonstrate that gestures may not only be a means of accessing concrete concepts in the mind, as shown in previous studies, but may be used to access abstract ones as well.