Günther Knoblich

Predicting the Effects of Actions: Interactions of Perception and Action

Many theories in cognitive psychology assume that perception and action systems are clearly separated from the cognitive system. Other theories suggest that important cognitive functions reside in the interactions between these systems. One consequence of the latter claim is that the action system may contribute to predicting the future consequences of currently perceived actions. In particular, such predictions might be more accurate when one observes ones own actions than when one observes another persons actions, because in the former case the system that plans the action is the same system that contributes to predicting the actions effects. In the present study, participants (N = 104) watched video clips displaying either themselves or somebody else throwing a dart at a target board and predicted the darts landing position. The predictions were more accurate when participants watched themselves acting. This result provides evidence for the claim that perceptual input can be linked with the action system to predict future outcomes of actions.

An eye movement study of insight problem solving

The representational change theory of insight claims that insight problems cause impasses because they mislead problem solvers into constructing inappropriate initial representations. Insight is attained when the initial representation is changed. In the present study (N = 24), we tested three specific implications of these hypotheses against eye movements recorded while participants solved matchstick arithmetic problems. The results were consistent with the predictions, providing converging evidence with prior findings using solution rates and solution times. Alternative theories of insight can explain individual findings, but only the representational change theory accounts for both the performance data and the eye movement data. The present study also suggests that eye movement recordings provide an important new window into processes of insight problem solving.

The neural correlates of perceiving one's own movements

Feedforward mechanisms are important for movement control. They may also contribute to the identification of self-produced actions by attenuating the sensory consequences of self-produced movements. In our study, subjects opened and closed their hand slowly and continuously (0.5 Hz). This movement was filmed with an MRI compatible video camera and projected online onto a screen, viewed by the subject while BOLD contrast was measured with fMRI. The temporal delay between movement and feedback was parametrically varied (0-200 ms). In each trial, subjects judged whether there was a delay or not. There was a positive correlation between the extent of the temporal delay and activation in the right posterior superior temporal cortex (pSTS) and a negative correlation in the left putamen. A second analysis addressed the neural correlates of subjective judgement under conditions of uncertainty. This contrast showed a differential activation in the cerebellum. These results support the assumption of a forward model implying that predictions generated in motor areas attenuate sensory areas. They also suggest that efference copy mechanisms are not located within specific brain areas but are implemented as a specific form of interaction between perceptual and motor areas depending on the modalities and the type of actions involved. Further, conscious detection of small temporal deviations might be based on signals generated in the cerebellum which provide fine-grained temporal information. These results might be useful to refine theories about the role of forward mechanisms in the emergence of disorders of the self, such as in schizophrenia.

Twin Peaks: An ERP Study of Action Planning and Control in Coacting Individuals

Previous studies have shown that perceiving anothers actions activates corresponding representations in an observers action system. The present study investigated how performing a task with another person affects action planning and control. Reaction times (RTs) and event-related potentials were measured while participants performed a go/no-go task alone and with another person. Three effects of acting together were observed. First, RTs were slowed when individuals had to respond to a stimulus referring to the others action, suggesting that an action selection conflict occurred. Second, at frontal sites, a stimulus referring to the others action elicited a similar electrophysiological response as a stimulus referring to ones own action. Finally, on no-go trials, P300 amplitude was significantly larger in a group setting, indicating that an action was suppressed. These findings provide evidence that individuals acting in a social context form shared action representations.

Authorship effects in the prediction of handwriting strokes: evidence for action simulation during action perception.

Does the action system contribute to action perception? Recent evidence suggests that actions are simulated while being observed. Given that the planning and simulating system are the same only when one observes ones own actions, it might be easier to predict the future outcomes of actions when one has carried them out oneself earlier on. In order to test this hypothesis, three experiments were conducted in which participants observed parts of earlier self- and other-produced trajectories and judged whether another stroke would follow or not. When the trajectories were produced without constraints, participants accomplished this task only for self-produced trajectories. When the trajectories were produced under narrow constraints, the predictions were equally accurate for self- and for other-generated trajectories. These results support the action simulation assumption. The more the actions that one observes resemble the way one would carry them out oneself, the more accurate the simulation.

Recognition of self-generated actions from kinematic displays of drawing.

Five experiments addressed the question of whether individuals can distinguish between self-generated and other-generated actions when seeing their visual effects. Each experiment consisted of a recording session in which participants drew familiar and unfamiliar characters without receiving visual feedback and a recognition session in which they provided self-or-other judgments (SOJs) to indicate whether a kinematic display reproduced the visual effects of their own actions. The main results were that self-generated and other-generated drawing can be distinguished, that the familiarity of character shapes does not influence the accuracy of SOJs, and that velocity information is crucial for the identification of self-generated drawing. The ability to determine authorship from kinematic displays of drawing provides evidence for the contribution of action-planning structures to perception.

Deceiving Oneself about Being in Control: Conscious Detection of Changes in Visuomotor Coupling.

Previous research has demonstrated that compensatory movements for changes in visuomotor coupling often are not consciously detected. But what factors affect the conscious detection of such changes? This issue was addressed in 4 experiments. Participants carried out a drawing task in which the relative velocity between the actual movement and its visual consequences was perturbed. Unconscious compensatory movements and conscious detection rates were simultaneously recorded. There was an invariant relationship between the extent of the change and its conscious detection that was proportional to the initial drawing velocity. This suggests that conscious change detection relies on a system that integrates visual and motor information-as, for instance, suggested by the internal model theory of motor control. Figural discrepancies increased the detection rates, indicating that additional cues for the what system facilitate conscious change detection.

The joint flanker effect: sharing tasks with real and imagined co-actors

The Eriksen flanker task (Eriksen and Eriksen in Percept Psychophys 16:143–149, 1974) was distributed among pairs of participants to investigate whether individuals take into account a co-actor’s S–R mapping even when coordination is not required. Participants responded to target letters (Experiment 1) or colors (Experiment 2) surrounded by distractors. When performing their part of the task next to another person performing the complementary part of the task, participants responded more slowly to stimuli containing flankers that were potential targets for their co-actor (incompatible trials), compared to stimuli containing identical, compatible, or neutral flankers. This joint Flanker effect also occurred when participants merely believed to be performing the task with a co-actor (Experiment 3). Furthermore, Experiment 4 demonstrated that people form shared task representations only when they perceive their co-actor as intentionally controlling her actions. These findings substantiate and generalize earlier results on shared task representations and advance our understanding of the basic mechanisms subserving joint action.

Making oneself predictable: reduced temporal variability facilitates joint action coordination

Performing joint actions often requires precise temporal coordination of individual actions. The present study investigated how people coordinate their actions at discrete points in time when continuous or rhythmic information about others’ actions is not available. In particular, we tested the hypothesis that making oneself predictable is used as a coordination strategy. Pairs of participants were instructed to coordinate key presses in a two-choice reaction time task, either responding in synchrony (Experiments 1 and 2) or in close temporal succession (Experiment 3). Across all experiments, we found that coactors reduced the variability of their actions in the joint context compared with the same task performed individually. Correlation analyses indicated that the less variable the actions were, the better was interpersonal coordination. The relation between reduced variability and improved coordination performance was not observed when pairs of participants performed independent tasks next to each other without intending to coordinate. These findings support the claim that reducing variability is used as a coordination strategy to achieve predictability. Identifying coordination strategies contributes to the understanding of the mechanisms involved in real-time coordination.

Self-monitoring in patients with schizophrenia.

BACKGROUND The present study investigated whether a failure of self-monitoring contributes to core syndromes of schizophrenia. METHOD Three groups of patients with a DSM-IV diagnosis of schizophrenia (n = 27), with either prominent paranoid hallucinatory or disorganization syndrome, or without these symptoms, and a matched healthy control group (n = 23) drew circles on a writing pad connected to a PC monitor. Subjects were instructed to continuously monitor the relationship between their hand movements and their visual consequences. They were asked to detect gain changes in the mapping. Self-monitoring ability and the ability to automatically correct movements were assessed. RESULTS Patients with either paranoid-hallucinatory syndrome or formal thought disorder were selectively impaired in their ability to detect a mismatch between a self-generated movement and its consequences, but not impaired in their ability to automatically compensate for the gain change. CONCLUSIONS These results support the claim that a failure of self-monitoring may underlie the core symptoms of schizophrenia.