Jochen Müsseler

The Theory of Event Coding (TEC): a framework for perception and action planning.

Traditional approaches to human information processing tend to deal with perception and action planning in isolation, so that an adequate account of the perception-action interface is still missing. On the perceptual side, the dominant cognitive view largely underestimates, and thus fails to account for, the impact of action-related processes on both the processing of perceptual information and on perceptual learning. On the action side, most approaches conceive of action planning as a mere continuation of stimulus processing, thus failing to account for the goal-directedness of even the simplest reaction in an experimental task. We propose a new framework for a more adequate theoretical treatment of perception and action planning, in which perceptual contents and action plans are coded in a common representational medium by feature codes with distal reference. Perceived events (perceptions) and to-be-produced events (actions) are equally represented by integrated, task-tuned networks of feature codes--cognitive structures we call event codes. We give an overview of evidence from a wide variety of empirical domains, such as spatial stimulus-response compatibility, sensorimotor synchronization, and ideomotor action, showing that our main assumptions are well supported by the data.

Blindness to response-compatible stimuli

This contribution is devoted to the question of whether action-control processes may be demonstrated to influence perception. This influence is predicted from a framework in which stimulus processing and action control are assumed to share common codes, thus possibly interfering with each other. In 5 experiments, a paradigm was used that required a motor action during the presentation of a stimulus. The participants were presented with masked right- or left-pointing arrows shortly before executing an already prepared left or right keypress response. We found that the identification probability of the arrow was reduced when the to-be-executed reaction was compatible with the presented arrow. For example, the perception of a right-pointing arrow was impaired when presented during the execution of a right response as compared with that of a left response. The theoretical implications of this finding as well as its relation to other, seemingly similar phenomena (repetition blindness, inhibition of return, psychological refractory period) are discussed.

Codes and their vicissitudes

First, we discuss issues raised with respect to the Theory of Event Coding (TEC)s scope, that is, its limitations and possible extensions. Then, we address the issue of specificity, that is, the widespread concern that TEC is too unspecified and, therefore, too vague in a number of important respects. Finally, we elaborate on our views about TECs relations to other important frameworks and approaches in the field like stages models, ecological approaches, and the two-visual-pathways model.

Localizing the first position of a moving stimulus: The Fröhlich effect and an attention-shifting explanation

When subjects are asked to determine where a fast-moving stimulus enters a window, they typically do not localize the stimulus at the edge, but at some later position within that window (Fröhlich effect). We report five experiments that explored this illusion. An attentional account is tested, assuming that the entrance of the stimulus in the window initiates a focus shift toward it. While this shift is under way, the stimulus moves into the window. Because the first phenomenal (i.e., explicitly reportable) representation of the stimulus will not be available before the end of the focus shift, the stimulus is perceived at some later position. In Experiment 1, we established the Fröhlich effect and showed that its size depends on stimulus parameters such as movement speed and movement direction. In Experiments 2 and 3, we examined the influence of eye movements and tested whether the effect changed when the stimuli were presented within a structural background or when they started from different eccentricities. In Experiments 4 and 5, specific predictions from the attentional model were tested: In Experiment 4 we showed that the processing of the moving stimulus benefits from a preceding peripheral cue indicating the starting position of the subsequent movement, which induces a preliminary focus shift to the position where the moving stimulus would appear. As a consequence the Fröhlich effect was reduced. Using a detection task in Experiment 5, we showed that feature information about the moving stimulus is lost when it falls into the critical interval of the attention shift. In conclusion, the present attentional account shows that selection mechanisms are not exclusively space based; rather, they can establish a spatial representation that is also used for perceptual judgment—that is, selection mechanisms can bespace establishing as well.

Relative mislocalization of briefly presented stimuli in the retinal periphery

We studied the ability to localize flashed stimuli, using a relative judgment task. When observers are asked to localize the peripheral position of a probe with respect to the midposition of a spatially extended comparison stimulus, they tend to judge the probe as being more toward the periphery than is the midposition of the comparison stimulus. We report seven experiments in which this novel phenomenon was explored. They reveal that the mislocalization occurs only when the probe and the comparison stimulus are presented in succession, independent of whether the probe or the comparison stimulus comes first (Experiment 1). The size of the mislocalization is dependent on the stimulus onset asynchrony (Experiment 2) and on the eccentricity of presentation (Experiment 3). In addition, the illusion also occurs in an absolute judgment task, which links mislocalization with the general tendency to judge peripherally presented stimuli as being more foveal than they actually are (Experiment 4). The last three experiments reveal that relative mislocalization is affected by the amount of spatial extension of the comparison stimulus (Experiment 5) and by its structure (Experiments 6 and 7). This pattern of results allows us to evaluate possible explanations of the illusion and to relate it to comparable tendencies observed in eye movement behavior. It is concluded that the system in charge of the guidance of saccadic eye movements is also the system that provides the metric in perceived visual space.

Comparing mislocalizations with moving stimuli: The Fröhlich effect, the flash-lag, and representational momentum

When observers are asked to localize the onset or the offset position of a moving target, they typically make localization errors in the direction of movement. Similarly, when observers judge a moving target that is presented in alignment with a flash, the target appears to lead the flash. These errors are known as the Fröhlich effect, representational momentum, and flash-lag effect, respectively. This study compared the size of the three mislocalization errors. In Experiment 1, a flash appeared either simultaneously with the onset, the mid-position, or the offset of the moving target. Observers then judged the position where the moving target was located when the flash appeared. Experiments 2 and 3 are exclusively concerned with localizing the onset and the offset of the moving target. When observers localized the position with respect to the point in time when the flash was presented, a clear mislocalization in the direction of movement was observed at the initial position and the mid-position. In contrast, a mislocalization opposite to movement direction occurred at the final position. When observers were asked to ignore the flash (or when no flash was presented at all), a reduced error (or no error) was observed at the initial position and only a minor error in the direction of the movement occurred at the final position. An integrative model is proposed, which suggests a common underlying mechanism, but emphasizes the specific processing components of the Fröhlich effect, flash-lag effect, and representational momentum.

Spatial Compatibility Effects With Tool Use

Objective: We explored constraints in responding to spatially variable stimuli when hand movements are transformed into inverse movements of a tool. Background: Generally, the spatial compatibility between stimuli and responses is a powerful determinant of performance. However, many tasks require the use of simple tools such as first-class levers that transform hand movements into inverted movements of a tool. What types of compatibility effects arise with such tools? Method: Participants moved the tip of a pointer to the left or right according to the color of a stimulus. The pointer was manipulated either directly, so that a hand movement caused a pointer movement in the corresponding direction, or indirectly, so that the hand moved the pointer in the opposite direction. Results: Responding was faster when the location of stimulus and the movement direction of the tool corresponded than when they did not correspond, independent of the movement direction of the hand. This occurred when stimulus location was task relevant (Experiment 1) as well as when it was task irrelevant (Experiment 2). Furthermore, responding was delayed when the hand and the relevant end of the tool moved in noncorresponding rather than corresponding directions. Conclusion: These results point to two distinct compatibility effects in tool use: one that relates to the transformation of stimuli into goals and one that relates to the transformation of goals into movements. Application: Potential applications of this research include the prediction and possibly manipulation of unwanted “fulcrum effects” in laparoscopic surgery and other first-class lever movements.

Detecting and identifying response-compatible stimuli

Previous work indicates that action-control processes influence perceptual processes: The identification probability of a left- or right-pointing arrow is reduced when it appears during the execution of a compatible left-right-key press (Müsseler & Hommel, in press). The present study addresses the question of whether this effect would also be observed in a detection task—that is, with judgments that do not require discriminating between left- and right-pointing arrows. Indeed, we found comparable effects in both the identification task and the detection task. This outcome is interpreted within a commoncoding framework, which holds that stimulus processing and action control operate on the same codes.

Dissociations in the timing of stationary and moving stimuli

W hen participants are asked to localize the 1 st position of a moving stimulus, they mislocalize it in the direction of the movement (Frohlich effect; F. W. Frohlich, 1923). This mislocalization points to a delay in the temporal sensation of a moving stimulus, However, the delay is in contrast to findings indicating a faster processing of moving stimuli. This potential dissociation was studied in 6 experiments, After establishing the effect spatially, different temporal tasks were examined under otherwise identical conditions. Simple as well as choice reaction times were shorter to moving than to stationary stimuli. Other tasks (choice reaction to structural features, temporal order judgment, and synchronization), however, produced opposite effects. Results support a view that the output of early stimulus processing directly feeds into the motor system, whereas the processing stages used, for example, for localization judgments are based on later integrative mechanisms,

How independent from action control is perception? An event-coding account for more equally ranked crosstalks

There is an ongoing controversy in the study of visual perception as to how closely visual processes are tied to cognitive processes. The present paper extends this controversy in that it considers crosstalks between the starting and the end point of the information stream, that is, between visual and action-control processes. Though it is usually admitted that action-control processes affect which information is picked up, accentuated, disregarded, or rejected, the traditional view continues to espouse a predominantly one-way route for visual information processing. In the present contribution, an alternative view is developed that allows for more equally-ranked cros-stalks. It is based on the idea that actions are controlled by the anticipation of their intended effects, and that perception and action control therefore share codes in the same representational domain. As a consequence, action-control processes are capable affecting and modifying visual processes in a more elementary manner. Theoretical, neurophysiological as well as behavioral evidence supporting this view will be presented.