Kazuki Sekine

Gesture production patterns in aphasic discourse: in-depth description and preliminary predictions

Background: Gesture frequently accompanies speech in healthy speakers. For many individuals with aphasia, gestures are a target of speech-language pathology intervention, either as an alternative form of communication or as a facilitative device for language restoration. The patterns of gesture production for people with aphasia and the participant variables that predict these patterns remain unclear. Aims: We aimed to examine gesture production during conversational discourse in a large sample of individuals with aphasia. We used a detailed gesture coding system to determine patterns of gesture production associated with specific aphasia types and severities. Methods & Procedures: We analysed conversation samples from AphasiaBank, gathered from 46 people with post-stroke aphasia and 10 healthy matched controls all of whom had gestured at least once during a story re-tell task. Twelve gesture types were coded. Descriptive statistics were used to describe the patterns of gesture production. Possible significant differences in production patterns according to aphasia type and severity were examined with a series of analyses of variance (ANOVA) statistics, and multiple regression analysis was used to examine these potential predictors of gesture production patterns. Outcomes & Results: Individuals with aphasia gestured significantly more frequently than healthy controls. Aphasia type and severity impacted significantly on gesture type in specific identified patterns detailed here, especially on the production of meaning-laden gestures. Conclusions: These patterns suggest the opportunity for gestures as targets of aphasia therapy. Aphasia fluency accounted for a greater degree of data variability than aphasia severity or naming skills. More work is required to delineate predictive factors.

The Development of the Ability to Semantically Integrate Information in Speech and Iconic Gesture in Comprehension.

We examined whether childrens ability to integrate speech and gesture follows the pattern of a broader developmental shift between 3- and 5-year-old children (Ramscar & Gitcho, 2007) regarding the ability to process two pieces of information simultaneously. In Experiment 1, 3-year-olds, 5-year-olds, and adults were presented with either an iconic gesture or a spoken sentence or a combination of the two on a computer screen, and they were instructed to select a photograph that best matched the message. The 3-year-olds did not integrate information in speech and gesture, but 5-year-olds and adults did. In Experiment 2, 3-year-old children were presented with the same speech and gesture as in Experiment 1 that were produced live by an experimenter. When presented live, 3-year-olds could integrate speech and gesture. We concluded that development of the integration ability is a part of the broader developmental shift; however, live-presentation facilitates the nascent integration ability in 3-year-olds.

Communicative effectiveness of pantomime gesture in people with aphasia

BACKGROUND Human communication occurs through both verbal and visual/motoric modalities. Simultaneous conversational speech and gesture occurs across all cultures and age groups. When verbal communication is compromised, more of the communicative load can be transferred to the gesture modality. Although people with aphasia produce meaning-laden gestures, the communicative value of these has not been adequately investigated. AIMS To investigate the communicative effectiveness of pantomime gesture produced spontaneously by individuals with aphasia during conversational discourse. METHODS & PROCEDURES Sixty-seven undergraduate students wrote down the messages conveyed by 11 people with aphasia that produced pantomime while engaged in conversational discourse. Students were presented with a speech-only, a gesture-only and a combined speech and gesture condition and guessed messages in both a free description and a multiple-choice task. OUTCOMES & RESULTS As hypothesized, listener comprehension was more accurate in the combined pantomime gesture and speech condition as compared with the gesture- or speech-only conditions. Participants achieved greater accuracy in the multiple-choice task as compared with the free-description task, but only in the gesture-only condition. The communicative effectiveness of the pantomime gestures increased as the fluency of the participants with aphasia decreased. CONCLUSIONS & IMPLICATIONS These results indicate that when pantomime gesture was presented with aphasic speech, the combination had strong communicative effectiveness. Future studies could investigate how pantomimes can be integrated into interventions for people with aphasia, particularly emphasizing elicitation of pantomimes in as natural a context as possible and highlighting the opportunity for efficient message repair.

Development of multimodal discourse comprehension: cohesive use of space by gestures

This study examined how well 5-, 6-, 10-year-olds and adults integrated information from spoken discourse with cohesive use of space in gesture, in comprehension. In Experiment 1, participants were presented with a combination of spoken discourse and a sequence of cohesive gestures, which consistently located each of the two protagonists in two distinct locations in gesture space. Participants were asked to select an interpretation of the final sentence that best matched the preceding spoken and gestural contexts. Adults and 10-year-olds performed better than 5-year-olds, who were at chance level. In Experiment 2, another group of 5-year-olds was presented with the same stimuli as in Experiment 1, except that the actor showed hand-held pictures, instead of producing cohesive gestures. Unlike cohesive gestures, one set of pictures was self-explanatory and did not require integration with the concurrent speech to derive the referent. With these pictures, 5-year-olds performed nearly perfectly and their performance in the identifiable pictures was significantly better than those in the unidentifiable pictures. These results suggest that young children failed to integrate spoken discourse and cohesive use of space in gestures, because they cannot derive a referent of cohesive gestures from the local speech context.

Part of the message comes in gesture: how people with aphasia convey information in different gesture types as compared with information in their speech

ABSTRACT Background: Studies have shown that the gestures produced by people with aphasia (PWA) can convey information useful for their communication. However, the exact significance of the contribution to message communication via gesture remains unclear. Furthermore, it remains unclear how different gesture types and representation techniques impact message conveyance. Aims: The present study aimed to investigate the contribution of gesture to PWA’s communication. We specifically focussed on the degree to which different gesture types and representation techniques convey information absent in the speech of PWA. Methods & Procedure: We studied the gestures produced by 46 PWA and nine non-brain-damaged participants (NBDP) during semi-structured conversation. For each of the different types of gestures and representation techniques we identified whether these conveyed essential information, that is information that was absent in speech. Rather than looking at information that was either similar to information in speech or additional to information in speech, we focused on the essential gestures only. Outcomes & Results: For PWA, a fifth of their gestures were Essential. Despite individual differences between PWA, the majority produced more Essential gestures than NBDP, who produced limited amounts of Essential gestures. Essential information was mostly conveyed by specific gesture types: Pointing, Emblems and Iconic gesture. Within the group of iconic gestures, not only Handling and Enact but also Object and Shape gestures, were often Essential. Conclusions: Our findings suggest that a great proportion of gestures produced by most PWA convey information essential for understanding their communication. In their communication advice, speech language therapists could draw attention to specific gesture types to make sure that interlocutors pay more attention to these gestures when communicating with PWA.

The parallel development of the form and meaning of two-handed gestures and linguistic information packaging within a clause in narrative

Abstract We examined how two-handed gestures and speech with equivalent contents that are used in narrative develop during childhood. The participants were 40 native speakers of English consisting of four different age groups: 3-, 5-, 9-year-olds, and adults. A set of 10 video clips depicting motion events were used to elicit speech and gesture. There are two findings. First, two types of two-handed gestures showed different developmental changes: those with a single-handed stroke with a simultaneous hold increased with age, while those with a two handed-stroke decreased with age. Second, representational gesture and speech developed in parallel at the discourse level. More specifically, the ways in which information is packaged in a gesture and in a clause are similar for a given age group; that is, gesture and speech develop hand-in-hand.