Hedda Lausberg

Coding gestural behavior with the NEUROGES-ELAN system

Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands We present a coding system combined with an annotation tool for the analysis of gestural behavior. The NEUROGES coding system consists of three modules that progress from gesture kinetics to gesture function. Grounded on empirical neuropsychological and psychological studies, the theoretical assumption behind NEUROGES is that its main kinetic and functional movement categories are differentially associated with specific cognitive, emotional, and interactive functions. ELAN is a free, multimodal annotation tool for digital audio and video media. It supports multileveled transcription and complies with such standards as XML and Unicode. ELAN allows gesture categories to be stored with associated vocabularies that are reusable by means of template files. The combination of the NEUROGES coding system and the annotation tool ELAN creates an effective tool for empirical research on gestural behavior.

Co-speech hand movements during narrations: What is the impact of right vs. left hemisphere brain damage?

Persons with brain damage show deviant patterns of co-speech hand movement behaviour in comparison to healthy speakers. It has been claimed by several authors that gesture and speech rely on a single production mechanism that depends on the same neurological substrate while others claim that both modalities are closely related but separate production channels. Thus, findings so far are contradictory and there is a lack of studies that systematically analyse the full range of hand movements that accompany speech in the condition of brain damage. In the present study, we aimed to fill this gap by comparing hand movement behaviour in persons with unilateral brain damage to the left and the right hemisphere and a matched control group of healthy persons. For hand movement coding, we applied Module I of NEUROGES, an objective and reliable analysis system that enables to analyse the full repertoire of hand movements independent of speech, which makes it specifically suited for the examination of persons with aphasia. The main results of our study show a decreased use of communicative conceptual gestures in persons with damage to the right hemisphere and an increased use of these gestures in persons with left brain damage and aphasia. These results not only suggest that the production of gesture and speech do not rely on the same neurological substrate but also underline the important role of right hemisphere functioning for gesture production.

The revised NEUROGES–ELAN system: An objective and reliable interdisciplinary analysis tool for nonverbal behavior and gesture

As visual media spread to all domains of public and scientific life, nonverbal behavior is taking its place as an important form of communication alongside the written and spoken word. An objective and reliable method of analysis for hand movement behavior and gesture is therefore currently required in various scientific disciplines, including psychology, medicine, linguistics, anthropology, sociology, and computer science. However, no adequate common methodological standards have been developed thus far. Many behavioral gesture-coding systems lack objectivity and reliability, andxa0automated methods that register specific movement parameters often fail to show validity with regard to psychological and social functions. To address these deficits, we have combined two methods, an elaborated behavioral coding system and an annotation tool for video and audio data. The NEUROGES–ELAN system is an effective and user-friendly research tool for the analysis of hand movement behavior, including gesture, self-touch, shifts, and actions. Since its first publication in 2009 in Behavior Research Methods, the tool has been used in interdisciplinary research projects to analyze a total of 467 individuals from different cultures, including subjects with mental disease and brain damage. Partly on the basis of new insights from these studies, the system has been revised methodologically and conceptually. The article presents the revised version of the system, including a detailed study of reliability. The improved reproducibility of the revised version makes NEUROGES–ELAN a suitable system for basic empirical research into the relation between hand movement behavior and gesture and cognitive, emotional, and interactive processes and for the development of automated movement behavior recognition methods.

Hemispheric Differences of Motor Execution: A Near-Infrared Spectroscopy Study

Distal movements of the limbs are predominantly controlled by the contralateral hemisphere. However, functional neuroimaging studies do not unequivocally demonstrate a lateralization of the cerebral activation during hand movements. While some studies show a predominant activation of the contralateral hemisphere, other studies provide evidence for a symmetrically distributed bihemispheric activation. However, the divergent results may also be due to methodological shortcomings. Therefore, the present study using functional near-infrared spectroscopy examines cerebral activation in both hemispheres during motor actions of the right and left hands. Twenty participants performed a flexion/extension task with the right- or left-hand thumb. Cerebral oxygenation changes were recorded from 48 channels over the primary motor, pre-motor, supplementary motor, primary somatosensory cortex, subcentral area, and the supramarginal gyrus of each hemisphere. A consistent increase of cerebral oxygenation was found for oxygenated and for total hemoglobin in the hemisphere contralateral to the moving hand, regardless of the laterality. These findings are in line with previous data from localization [1-3] and brain imaging studies [4-6]. The present data support the proposition that there is no hemispheric specialization for simple distal motor tasks. Both hemispheres are equally activated during movement of the contralateral upper limb.

Neural Control of Posture in Individuals with Persisting Postconcussion Symptoms.

INTRODUCTIONnPostural instability has been shown to characterize individuals who suffered from long-term symptoms after mild traumatic brain injury. However, recordings of neural processes during postural control are difficult to realize with standard neuroimaging techniques. Thus, we used functional nearinfrared spectroscopy to investigate brain oxygenation of individuals with persistent postconcussion symptoms (pPCS) during postural control in altered environments.nnnMETHODSnWe compared brain oxygenation and postural sway during balance control in three groups: individuals suffering from pPCS, individuals with a history of mild traumatic brain injury but without pPCS, and healthy controls. Individuals were investigated during postural control tasks with six different conditions: i) eyes opened, ii) eyes closed, and iii) blurred visual input, each while standing a) on a stable and b) an unstable surface.nnnRESULTSnIn all groups, during the eyes closed/unstable surface condition as compared with the other conditions, the postural sway increased as well as the brain oxygenation in frontal brain cortices. In the most difficult balance condition, as compared with the other two groups, subjects with pPCS applied more force over time to keep balance as measured by the force plate system with a significantly greater activation in frontopolar/orbitofrontal areas of the right hemisphere.nnnCONCLUSIONSnAs subjects with pPCS applied more force over time to control balance, we propose that with regard to cognitive processes, the increase of cerebral activation in these individuals indicates an increase of attention-demanding processes during postural control in altered environments.

Neuropsychological functions of hand movements and gestures change in the presence or absence of speech

In order to address previous controversies whether hand movements and gestures are linked to mental concepts or solely to the process of speaking, in the present study we investigate the neuropsychological functions of the entire spectrum of unimanual and bimanual hand movements and gestures when they either accompany speaking or when they act as the only means to communicate in the absence of speech. The results showed that the hand movement activity regarding all types of hand movements and gestures stayed constant with and without speaking. The analysis of the Structure of hand movements showed that executions shifted from in space hand movements with a phase structure during the condition without speech to more irregular on body hand movements without a phase structure during the co-speech condition. The gestural analysis revealed that pantomime gestures increase under conditions without speech whereas emotional motions and subject-oriented actions primarily occur when speaking. The present results provide evidence that the overall hand movement activity does not differ between co-speech conditions and conditions without speech, but that the hands adopt different neuropsychological functions. We conclude that the hands primarily externalise mental concepts in conditions without speaking but that their use shifts to more self-regulation and to endorsing verbal output with emotional connotations when they accompany speech.

Hand movements with a phase structure and gestures that depict action stem from a left hemispheric system of conceptualization.

The present study addresses the previously discussed controversy on the contribution of the right and left cerebral hemispheres to the production and conceptualization of spontaneous hand movements and gestures. Although it has been shown that each hemisphere contains the ability to produce hand movements, results of left hemispherically lateralized motor functions challenge the view of a contralateral hand movement production system. To examine hemispheric specialization in hand movement and gesture production, ten right-handed participants were tachistoscopically presented pictures of everyday life actions. The participants were asked to demonstrate with their hands, but without speaking what they had seen on the drawing. Two independent blind raters evaluated the videotaped hand movements and gestures employing the Neuropsychological Gesture Coding System. The results showed that the overall frequency of right- and left-hand movements is equal independent of stimulus lateralization. When hand movements were analyzed considering their Structure, the presentation of the action stimuli to the left hemisphere resulted in more hand movements with a phase structure than the presentation to the right hemisphere. Furthermore, the presentation to the left hemisphere resulted in more right and left-hand movements with a phase structure, whereas the presentation to the right hemisphere only increased contralateral left-hand movements with a phase structure as compared to hand movements without a phase structure. Gestures that depict action were primarily displayed in response to stimuli presented in the right visual field than in the left one. The present study shows that both hemispheres possess the faculty to produce hand movements in response to action stimuli. However, the left hemisphere dominates the production of hand movements with a phase structure and gestures that depict action. We therefore conclude that hand movements with a phase structure and gestures that represent action stem from a left hemispheric system of conceptualization.

Persistent Postconcussive Symptoms Are Accompanied by Decreased Functional Brain Oxygenation

Diagnostic methods are considered a major concern in the determination of mild traumatic brain injury. The authors examined brain oxygenation patterns in subjects with severe and minor persistent postconcussive difficulties and a healthy control group during working memory tasks in prefrontal brain regions using functional near-infrared spectroscopy. The results demonstrated decreased working memory performances among concussed subjects with severe postconcussive symptoms that were accompanied by decreased brain oxygenation patterns. An association appears to exist between decreased brain oxygenation, poor performance of working memory tasks, and increased symptom severity scores in subjects suffering from persistent postconcussive symptoms.

Brain oxygenation patterns during the execution of tool use demonstration, tool use pantomime, and body-part-as-object tool use

Divergent findings exist whether left and right hemispheric pre- and postcentral cortices contribute to the production of tool use related hand movements. In order to clarify the neural substrates of tool use demonstrations with tool in hand, tool use pantomimes without tool in hand, and body-part-as-object presentations of tool use (BPO) in a naturalistic mode of execution, we applied functional Near InfraRed Spectroscopy (fNIRS) in twenty-three right-handed participants. Functional NIRS techniques allow for the investigation of brain oxygenation during the execution of complex hand movements with an unlimited movement range. Brain oxygenation patterns were retrieved from 16 channels of measurement above pre- and postcentral cortices of each hemisphere. The results showed that tool use demonstration with tool in hand leads to increased oxygenation as compared to tool use pantomimes in the left hemispheric somatosensory gyrus. Left hand executions of the demonstration of tool use, pantomime of tool use, and BPO of tool use led to increased oxygenation in the premotor and somatosensory cortices of the left hemisphere as compared to right hand executions of either condition. The results indicate that the premotor and somatosensory cortices of the left hemisphere constitute relevant brain structures for tool related hand movement production when using the left hand, whereas the somatosensory cortex of the left hemisphere seems to provide specific mental representations when performing tool use demonstrations with the tool in hand.

Pantomiming tool use with an imaginary tool in hand as compared to demonstration with tool in hand specifically modulates the left middle and superior temporal gyri

Neuropsychological lesion studies evidence the necessity to differentiate between various forms of tool-related actions such as real tool use, tool use demonstration with tool in hand and without physical target object, and pantomime without tool in hand. However, thus far, neuroimaging studies have primarily focused only on investigating tool use pantomimes. The present fMRI study investigates pantomime without tool in hand as compared to tool use demonstration with tool in hand in order to explore patterns of cerebral signal modulation associated with acting with imaginary tools in hand. Fifteen participants performed with either hand (i) tool use pantomime with an imaginary tool in hand in response to visual tool presentation and (ii) tool use demonstration with tool in hand in response to visual-tactile tool presentation. In both conditions, no physical target object was present. The conjunction analysis of the right and left hands executions of tool use pantomime relative to tool use demonstration yielded significant activity in the left middle and superior temporal lobe. In contrast, demonstration relative to pantomime revealed large bihemispherically distributed homologous areas of activity. Thus far, fMRI studies have demonstrated the relevance of the left middle and superior temporal gyri in viewing, naming, and matching tools and related actions and contexts. Since in our study all these factors were equally (ir)relevant both in the tool use pantomime and the tool use demonstration conditions, the present findings enhance the knowledge about the function of these brain regions in tool-related cognitive processes. The two contrasted conditions only differ regarding the fact that the pantomime condition requires the individual to act with an imaginary tool in hand. Therefore, we suggest that the left middle and superior temporal gyri are specifically involved in integrating the projected mental image of a tool in the execution of a tool-specific movement concept.