Mikko Tiainen

Shared processing of planning articulatory gestures and grasping

It has been proposed that articulatory gestures are shaped by tight integration in planning mouth and hand acts. This hypothesis is supported by recent behavioral evidence showing that response selection between the precision and power grip is systematically influenced by simultaneous articulation of a syllable. For example, precision grip responses are performed relatively fast when the syllable articulation employs the tongue tip (e.g., [te]), whereas power grip responses are performed relatively fast when the syllable articulation employs the tongue body (e.g., [ke]). However, this correspondence effect, and other similar effects that demonstrate the interplay between grasping and articulatory gestures, has been found when the grasping is performed during overt articulation. The present study demonstrates that merely reading the syllables silently (Experiment 1) or hearing them (Experiment 2) results in a similar correspondence effect. The results suggest that the correspondence effect is based on integration in planning articulatory gestures and grasping rather than requiring an overt articulation of the syllables. We propose that this effect reflects partially overlapped planning of goal shapes of the two distal effectors: a vocal tract shape for articulation and a hand shape for grasping. In addition, the paper shows a pitch–grip correspondence effect in which the precision grip is associated with a high-pitched vocalization of the auditory stimuli and the power grip is associated with a low-pitched vocalization. The underlying mechanisms of this phenomenon are discussed in relation to the articulation–grip correspondence.

Interaction in planning vocalizations and grasping

Previous studies have shown a congruency effect between manual grasping and syllable articulation. For instance, a power grip is associated with syllables whose articulation involves the tongue body and/or large mouth aperture ([kɑ]) whereas a precision grip is associated with articulations that involve the tongue tip and/or small mouth aperture ([ti]). Previously, this effect has been observed in manual reaction times. The primary aim of the current study was to investigate whether this congruency effect also takes place in vocal responses and to investigate involvement of action selection processes in the effect. The congruency effect was found in vocal and manual responses regardless of whether or not the syllable or grip was known a priori, suggesting that the effect operates with minimal or absent action selection processes. In addition, the effect was observed in vocal responses even when the grip was only prepared but not performed, suggesting that merely planning a grip response primes the corresponding articulatory response. These results support the view that articulation and grasping are processed in a partially overlapping network.

Interaction in planning movement direction for articulatory gestures and manual actions

Some theories concerning speech mechanisms assume that overlapping representations are involved in programming certain articulatory gestures and hand actions. The present study investigated whether planning of movement direction for articulatory gestures and manual actions could interact. The participants were presented with written vowels (Experiment 1) or syllables (Experiment 2) that were associated with forward or backward movement of tongue (e.g., [i] vs. [ɑ] or [te] vs. [ke], respectively). They were required to pronounce the speech unit and simultaneously move the joystick forward or backward according to the color of the stimulus. Manual and vocal responses were performed relatively rapidly when the articulation and the hand action required movement into the same direction. The study suggests that planning horizontal tongue movements for articulation shares overlapping neural mechanisms with planning horizontal movement direction of hand actions.

Preparation and execution of teeth clenching and foot muscle contraction influence on corticospinal hand-muscle excitability

Contraction of a muscle modulates not only the corticospinal excitability (CSE) of the contracting muscle but also that of different muscles. We investigated to what extent the CSE of a hand muscle is modulated during preparation and execution of teeth clenching and ipsilateral foot dorsiflexion either separately or in combination. Hand-muscle CSE was estimated based on motor evoked potentials (MEPs) elicited by transcranial magnetic stimulation (TMS) and recorded from the first dorsal interosseous (FDI) muscle. We found higher excitability during both preparation and execution of all the motor tasks than during mere observation of a fixation cross. As expected, the excitability was greater during the execution phase than the preparation one. Furthermore, both execution and preparation of combined motor tasks led to higher excitability than individual tasks. These results extend our current understanding of the neural interactions underlying simultaneous contraction of muscles in different body parts.

Selective Influences of Precision and Power Grips on Speech Categorization.

Recent studies have shown that articulatory gestures are systematically associated with specific manual grip actions. Here we show that executing such actions can influence performance on a speech-categorization task. Participants watched and/or listened to speech stimuli while executing either a power or a precision grip. Grip performance influenced the syllable categorization by increasing the proportion of responses of the syllable congruent with the executed grip (power grip—[ke] and precision grip—[te]). Two follow-up experiments indicated that the effect was based on action-induced bias in selecting the syllable.

Sharp and round shapes of seen objects have distinct influences on vowel and consonant articulation

The shape and size-related sound symbolism phenomena assume that, for example, the vowel [i] and the consonant [t] are associated with sharp-shaped and small-sized objects, whereas [ɑ] and [m] are associated with round and large objects. It has been proposed that these phenomena are mostly based on the involvement of articulatory processes in representing shape and size properties of objects. For example, [i] might be associated with sharp and small objects, because it is produced by a specific front-close shape of articulators. Nevertheless, very little work has examined whether these object properties indeed have impact on speech sound vocalization. In the present study, the participants were presented with a sharp- or round-shaped object in a small or large size. They were required to pronounce one out of two meaningless speech units (e.g., [i] or [ɑ]) according to the size or shape of the object. We investigated how a task-irrelevant object property (e.g., the shape when responses are made according to size) influences reaction times, accuracy, intensity, fundamental frequency, and formant 1 and formant 2 of vocalizations. The size did not influence vocal responses but shape did. Specifically, the vowel [i] and consonant [t] were vocalized relatively rapidly when the object was sharp-shaped, whereas [u] and [m] were vocalized relatively rapidly when the object was round-shaped. The study supports the view that the shape-related sound symbolism phenomena might reflect mapping of the perceived shape with the corresponding articulatory gestures.

Excitability of hand motor areas during articulation of syllables.

It is known that articulating different syllables is linked to different grasp actions, e.g. [ti] is linked to precision grip, and [kɑ] to power grip. The aim of the present study was to test whether articulating or hearing these syllables would result in an increased activity in the representation of hand muscles involved in these two actions in a muscle-specific manner. To this end, we used transcranial magnetic stimulation (TMS) to investigate changes in the excitability of the left primary motor cortex (M1) innervating hand muscles while participants articulated or listened to meaningless syllables, listened to a metronome, or observed a fixation cross. The motor-evoked potentials of two hand muscles associated with either a precision or power grip exhibited significantly greater amplitudes during articulation than in passive listening, metronome, and fixation cross conditions. Moreover, these muscles exhibited similar patterns of excitability during articulation regardless of which syllable was articulated. The increased excitability of the left M1 hand area during articulation, but not during perception of the syllables, might be due to the cortico-cortical interaction between the motor representations of oral organs with the hand area.

Systematic Influence of Perceived Grasp Shape on Speech Production

Previous research has shown that precision and power grip performance is consistently influenced by simultaneous articulation. For example, power grip responses are performed relatively fast with the open-back vowel [a], whereas precision grip responses are performed relatively fast with the close-front vowel [i]. In the present study, the participants were presented with a picture of a hand shaped to the precision or power grip. They were required to pronounce speech sounds according to the front/above perspective of the hand. The results showed that not only the grip performance is affected by simultaneously pronouncing the speech sound but also the production of speech sound can be affected by viewing an image of a grip. The precision grip stimulus triggered relatively rapid production of the front-close vowel [i]. In contrast, the effect related to the power grip stimulus was mostly linked to the vertical dimension of the pronounced vowel since this stimulus triggered relatively rapid production of the back-open vowel [a] and back-mid-open vowel [o] while production of the back-close vowel [u] was not influenced by it. The fact that production of the dorsal consonant [k] or coronal consonant [t] were not influenced by these stimuli suggests that the effect was not associated with a relative front-back tongue shape of the articulation in the absence of changes in any vertical articulatory components. These findings provide evidence for an intimate interaction between certain articulatory gestures and grip types, suggesting that an overlapping visuomotor network operates for planning articulatory gestures and grasp actions.

Mismatch negativity (MMN) to speech sounds is modulated systematically by manual grip execution

Manual actions and speech are connected: for example, grip execution can influence simultaneous vocalizations and vice versa. Our previous studies show that the consonant [k] is associated with the power grip and the consonant [t] with the precision grip. Here we studied whether the interaction between speech sounds and grips could operate already at a pre-attentive stage of auditory processing, reflected by the mismatch-negativity (MMN) component of the event-related potential (ERP). Participants executed power and precision grips according to visual cues while listening to syllable sequences consisting of [ke] and [te] utterances. The grips modulated the MMN amplitudes to these syllables in a systematic manner so that when the deviant was [ke], the MMN response was larger with a precision grip than with a power grip. There was a converse trend when the deviant was [te]. These results suggest that manual gestures and speech can interact already at a pre-attentive processing level of auditory perception, and show, for the first time that manual actions can systematically modulate the MMN.

Connections of Grasping and Horizontal Hand Movements with Articulation in Czech Speakers

We have recently shown in Finnish speakers that articulation of certain vowels and consonants has a systematic influence on simultaneous grasp actions as well as on forward and backward hand movements. Here we studied whether these effects generalize to another language, namely Czech. We reasoned that if the results generalized to another language environment, it would suggest that the effects arise through other processes than language-dependent semantic associations. Rather, the effects would be likely to arise through language-independent interactions between processes that plan articulatory gestures and hand movements. Participants were presented with visual stimuli specifying articulations to be uttered (e.g., A or I), and they were required to produce a manual response concurrently with the articulation. In Experiment 1 they responded with a precision or a power grip, whereas in Experiment 2 they responded with a forward or a backward hand movement. The grip congruency effect was fully replicated: the consonant [k] and the vowel [α] were associated with power grip responses, while the consonant [t] and the vowel [i] were associated with precision grip responses. The forward/backward congruency effect was replicated with vowels [α], [o], which were associated with backward movement and with [i], which was associated with forward movement, but not with consonants [k] and [t]. These findings suggest that the congruency effects mostly reflect interaction between processes that plan articulatory gestures and hand movements with an exception that the forward/backward congruency effect might only work with vowel articulation.