Roland Thomaschke

From visuo-motor interactions to imitation learning: Behavioural and brain imaging studies

Abstract We review three areas of research and theory relating to the involvement of motor processing in action observation: behavioural studies on imitation learning, behavioural work on short-term visuomotor interactions, and related neurophysiological and neuroimaging work. A large number of behavioural studies now indicate bi-directional links between perception and action: visual processing can automatically induce related motor processes, and motor actions can direct future visual processing. The related concept of direct matching (Rizzolatti et al., 2001) does not, however, imply that observed actions are transduced into a corresponding motor representation that would guarantee an instant and accurate imitation. Rather, studies on the mirror neuron system indicate that action observation engages the observers own motor prototype of the observed action. This allows for enhanced action recognition, imitation recognition, and, predominantly in humans, imitation and observational learning. Despite the clear impact of action observation on motor representations, recent neuroimaging work also indicates the overlap of imitation learning with processes of non-imitative skill acquisition.

The Planning and Control Model (PCM) of Motorvisual Priming: Reconciling Motorvisual Impairment and Facilitation Effects

Previous research on dual-tasks has shown that, under some circumstances, actions impair the perception of action-consistent stimuli, whereas, under other conditions, actions facilitate the perception of action-consistent stimuli. We propose a new model to reconcile these contrasting findings. The planning and control model (PCM) of motorvisual priming proposes that action planning binds categorical representations of action features so that their availability for perceptual processing is inhibited. Thus, the perception of categorically action-consistent stimuli is impaired during action planning. Movement control processes, on the other hand, integrate multi-sensory spatial information about the movement and, therefore, facilitate perceptual processing of spatially movement-consistent stimuli. We show that the PCM is consistent with a wider range of empirical data than previous models on motorvisual priming. Furthermore, the model yields previously untested empirical predictions. We also discuss how the PCM relates to motorvisual research paradigms other than dual-tasks.

Temporal Predictability Facilitates Action, Not Perception

Expectancy for upcoming action requirements is a fundamental prerequisite for human control of action. In the research reported here, we investigated which part of cognitive processing benefits from temporal predictability. In a binary forced-choice paradigm, visual targets were preceded by different intervals. In one condition, targets could be predicted by the length of the intervals. In other conditions, response goals or response effectors could be predicted by the length of the intervals. Behavioral advantages were observed when response effectors were temporally predictable, whereas temporal predictability of response goals and target stimuli was not sufficient. The findings thus show that temporal expectancy in speeded choice-reaction tasks facilitates late, effector-specific motor processing. These findings are of importance not only for our basic understanding of action control but also for any human-machine interaction that involves system delays.

Response specific temporal expectancy: Evidence from a variable foreperiod paradigm

When a stimulus–response event is frequently paired with a specific foreperiod, response performance for this event is improved after this foreperiod. This phenomenon is referred to as specific temporal expectancy. In four experiments, we investigated whether stimulus- or response-related processing benefits from specific temporal expectancy. In a speeded choice reaction task, different features of the imperative stimuli were frequently paired with foreperiods in such a way that only in some experiments were the responses also frequently paired with foreperiods. Participants revealed evidence for specific temporal expectancy when responses were frequently paired with foreperiods, but not when only the stimuli were frequently paired with foreperiods. We concluded that specific temporal expectancy affects response-related processing.

Do endogenous and exogenous action control compete for perception

Human actions are guided either by endogenous action plans or by external stimuli in the environment. These two types of action control seem to be mediated by neurophysiologically and functionally distinct systems that interfere if an endogenously planned action suddenly has to be performed in response to an exogenous stimulus. In this case, the endogenous representation has to be deactivated first to give way to the exogenous system. Here we show that interference of endogenous and exogenous action control is not limited to motor-related aspects but also affects the perception of action-related stimuli. Participants associated two actions with contingent sensory effects in learning blocks. In subsequent test blocks, preparing one of these actions specifically impaired responding to the associated effect in an exogenous speeded detection task, yielding a blindness-like effect for arbitrary, learned action effects. In accordance with the theory of event coding, this finding suggests that action planning influences perception even in the absence of any physical similarities between action and to-be-perceived stimuli.

The time-event correlation effect is due to temporal expectancy, not to partial transition costs.

Humans are sensitive to temporal redundancies in their environment. When the identity of a target stimulus is correlated with the duration of the preceding interval, performance is better for frequent than for infrequent combinations of target and interval. This effect has been demonstrated several times in current timing research. However, it can be accounted for by 2 starkly contrasting explanations. The standard account has explained it in terms of learning associations between intervals and stimulus-response events. But, alternatively the effect might be due to partial trial transition costs, because infrequent time-event combinations are proportionally more often partial transitions (i.e., transitions of either interval, or target). We conducted 3 choice response time experiments to distinguish between both explanations. The results clearly show that the time-event correlation effect is due to learning, not to partial transition costs.

The role of cue-response mapping in motorvisual impairment and facilitation: evidence for different roles of action planning and action control in motorvisual dual-task priming.

Previous research has shown that actions impair the visual perception of categorically action-consistent stimuli. On the other hand, actions can also facilitate the perception of spatially action-consistent stimuli. We suggest that motorvisual impairment is due to action planning processes, while motorvisual facilitation is due to action control mechanisms. This implies that because action planning is sensitive to modulations by cue-response mapping so should motorvisual impairment, while motorvisual facilitation should be insensitive to manipulations of cue-response mapping as is action control. We tested this prediction in three dual-task experiments. The impact of performing left and right key presses on the perception of unrelated, categorically or spatially consistent, stimuli was studied. As expected, we found motorvisual impairment for categorically consistent stimuli and motorvisual facilitation for spatially consistent stimuli. In all experiments, we compared congruent with incongruent cue-key mappings. Mapping manipulations affected motorvisual impairment, but not motorvisual facilitation. The results support our suggestion that motorvisual impairment is due to action planning, and motorvisual facilitation to action control.

The specificity of temporal expectancy: Evidence from a variable foreperiod paradigm

In speeded choice tasks with variable foreperiods (FPs), individuals behaviourally adapt to various frequency manipulations. Adaptations have been shown to frequencies of different stimulus–response events, to frequencies of different foreperiods, and to frequencies of different event–foreperiod combinations. We have investigated how participants adapt to a situation where all three frequency manipulations are done simultaneously. Three variable foreperiod experiments are reported. In Experiment 1, one target (the peak distributed target) appeared particularly frequently after one particular FP (the peak foreperiod), while another target was less frequent and equally distributed over all foreperiods. In Experiment 2, the equally distributed target was overall more frequent than the peak distributed one. In both experiments, performance advantages for the peak distributed target were specific to the peak foreperiod, and performance advantages at the peak foreperiod were specific to the peak distributed targets. A third experiment showed that, when two differently frequent target are both equally distributed over FPs, the performance distribution over FPs is not significantly different between both targets. Together, the results suggest that participants were able to simultaneously and specifically adapt to frequency manipulations in events, foreperiods, and event–foreperiod combinations.

Time-based event expectations employ relative, not absolute, representations of time

When the timing of an event is predictable, humans automatically form implicit time-based event expectations. We investigated whether these expectations rely on absolute (e.g., 800 ms) or relative (e.g., a shorter duration) representations of time. In a choice-response task with two different pre-target intervals, participants implicitly learned that targets were predictable by interval durations. In a test phase, the two intervals were either considerably shortened or lengthened. In both cases, behavioral tendencies transferred from practice to test according to relative, not absolute, interval duration. We conclude that humans employ relative representations of time periods when forming time-based event expectations. These results suggest that learned time-based event expectations (e.g., in communication and human–machine interaction) should transfer to faster or slower environments if the relative temporal distribution of events is preserved.

Remapping motion across modalities: tactile rotations influence visual motion judgments

Multisensory interactions between haptics and vision remain poorly understood. Previous studies have shown that shapes, such as letters of the alphabet, when drawn on the skin, are differently perceived dependent upon which body part is stimulated and on how the stimulated body part, such as the hand, is positioned. Another line of research within this area has investigated multisensory interactions. Tactile perceptions, for example, have the potential to disambiguate visually perceived information. While the former studies focused on explicit reports about tactile perception, the latter studies relied on fully aligned multisensory stimulus dimensions. In this study, we investigated to what extent rotating tactile stimulations on the hand affect directional visual motion judgments implicitly and without any spatial stimulus alignment. We show that directional tactile cues and ambiguous visual motion cues are integrated, thus biasing the judgment of visually perceived motion. We further show that the direction of the tactile influence depends on the position and orientation of the stimulated part of the hand relative to a head-centered frame of reference. Finally, we also show that the time course of the cue integration is very versatile. Overall, the results imply immediate directional cue integration within a head-centered frame of reference.