Stephen M. Kosslyn

Seeing and Imagining in the Cerebral Hemispheres: A Computational Approach.

Visual recognition, navigation, tracking, and imagery are posited to share certain high-level processing subsystems. In the first part of this article, a theory of some of these subsystems is formulated. This theory is developed in light of an analysis of problems that must be solved by the visual system and the constraints on the solutions to these problems; computational, neurological, and behavioral constraints are considered. In the second part, inferences about perceptual subsystems are used to develop a theory of how mental images are generated. Support for this theory is adduced from studies of split-brain patients and a review of relevant neuropsychological findings. In the third part, a computational mechanism is developed to account for how visual function becomes lateralized in the brain; this mechanism is used to predict how the hypothesized processing subsystems become lateralized. In the fourth part, some critical tests of the theory of lateralization of perceptual processing subsystems are reported, and in the fifth part the theory is extended to account for the lateralization of image-transformation subsystems. In the sixth part, the theory is used to account for the almost ubiquitous variability (both between subjects and within subjects) evident in the neuropsychological literature on lateralization. Finally, in the concluding part of the article, the computational-neuropsychological approach is discussed and evaluated.

Visual Images Preserve Metric Spatial Information: Evidence from Studies of Image Scanning

Four experiments demonstrated that more time is required to scan further distances across visual images, even when the same amount of material falls between the initial focus point and the target. Not only did times systematically increase with distance but subjectively larger images required more time to scan than did subjectively smaller ones. Finally, when subjects were not asked to base all judgments on examination of their images, the distance between an initial focus point and a target did not affect reaction times.

Visual mental imagery activates topographically organized visual cortex: Pet investigations

Cerebral blood flow was measured using positron emission tomography (PET) in three experiments while subjects performed mental imagery or analogous perceptual tasks. In Experiment 1, the subjects either visualized letters in grids and decided whether an X mark would have fallen on each letter if it were actually in the grid, or they saw letters in grids and decided whether an X mark fell on each letter. A region identified as part of area 17 by the Talairach and Tournoux (1988) atlas, in addition to other areas involved in vision, was activated more in the mental imagery task than in the perception task. In Experiment 2, the identical stimuli were presented in imagery and baseline conditions, but subjects were asked to form images only in the imagery condition; the portion of area 17 that was more active in the imagery condition of Experiment 1 was also more activated in imagery than in the baseline condition, as was part of area 18. Subjects also were tested with degraded perceptual stimuli, which caused visual cortex to be activated to the same degree in imagery and perception. In both Experiments 1 and 2, however, imagery selectively activated the extreme anterior part of what was identified as area 17, which is inconsistent with the relatively small size of the imaged stimuli. These results, then, suggest that imagery may have activated another region just anterior to area 17. In Experiment 3, subjects were instructed to close their eyes and evaluate visual mental images of upper case letters that were formed at a small size or large size. The small mental images engendered more activation in the posterior portion of visual cortex, and the large mental images engendered more activation in anterior portions of visual cortex. This finding is strong evidence that imagery activates topographically mapped cortex. The activated regions were also consistent with their being localized in area 17. Finally, additional results were consistent with the existence of two types of imagery, one that rests on allocating attention to form a pattern and one that rests on activating stored visual memories.

Mental rotation of objects versus hands: Neural mechanisms revealed by positron emission tomography

Twelve right-handed men participated in two mental rotation tasks as their regional cerebral blood flow (rCBF) was monitored using positron emission tomography. In one task, participants mentally rotated and compared figures composed of angular branching forms; in the other task, participants mentally rotated and compared drawings of human hands. In both cases, rCBF was compared with a baseline condition that used identical stimuli and required the same comparison, but in which rotation was not required. Mental rotation of branching objects engendered activation in the parietal lobe and Area 19. In contrast, mental rotation of hands engendered activation in the precentral gyrus (M1), superior and inferior parietal lobes, primary visual cortex, insula, and frontal Areas 6 and 9. The results suggest that at least two different mechanisms can be used in mental rotation, one mechanism that recruits processes that prepare motor movements and another mechanism that does not.

Information Representation in Visual Images.

Abstract How information is represented in visual images was explored in five experiments where subjects judged whether or not various properties were appropriate for given animals. It took more time to evaluate an animal when the subjective image of it was small, whether size was manipulated directly or indirectly (e.g., by having a target animal imaged at the correct relative size next to an elephant or a fly). More time also was required if the animal was imaged in a relatively “complex” environment (next to 4 vs. 2 digits painted on an imaginary wall, or next to a 16 cell vs. 4 cell matrix). Finally, subjectively larger images required more time to evoke than smaller images. These results support a constructivist notion of imagery, and the idea that images may act as ‘analogues’ to percepts.

Pictures and names: Making the connection ☆

Abstract In order to identify an object sensory input must somehow access stored information. A series of results supports two general assertions about this process: First, objects are identified first at a particular level of abstraction which is neither the most general nor the most specific possible. Time to provide names more general than “entry point” names is predicted by the degree of association between the “entry point” concept and the required name, not by perceptual factors. In contrast, providing more specific names than that corresponding to the “entry point” concept does require more detailed perceptual analysis. Second, the particular entry point for a given object covaries with its typicality, which affects whether or not the object will be identified at the “basic” level. Atypical objects have their entry point at a level subordinate to the basic level. The generality and usefulness of the notion of “basic level” is discussed in the face of these results.

Imagery, propositions and the form of internal representations

This paper has four major sections: First, we review the basic arguments offered by Pylyshyn (Psychological Bulletin, 1973, 80, 1–24) and others against using imagery as an explanatory construct in psychology. Second, we consider each of these points and find none that speak against any but the most primitive notions of imagery. Third, we review the results of various experiments on imagery. In each case, we compare two explanations of the findings: one which assumes the existence of a surface image manifesting emergent properties, and one which assumes that all internal representations are coded in terms of “abstract propositions.” We find imagery hypotheses to be at least as adequate as those based on propositional representation. Finally, we conclude that debate about the ultimate foundations of internal representation is fruitless; the empirical question is whether images have properties that cannot be derived directly from more abstract propositional structures.

Motor processes in mental rotation

Much indirect evidence supports the hypothesis that transformations of mental images are at least in part guided by motor processes, even in the case of images of abstract objects rather than of body parts. For example, rotation may be guided by processes that also prime one to see results of a specific motor action. We directly test the hypothesis by means of a dual-task paradigm in which subjects perform the Cooper-Shepard mental rotation task while executing an unseen motor rotation in a given direction and at a previously-learned speed. Four results support the inference that mental rotation relies on motor processes. First, motor rotation that is compatible with mental rotation results in faster times and fewer errors in the imagery task than when the two rotations are incompatible. Second, the angle through which subjects rotate their mental images, and the angle through which they rotate a joystick handle are correlated, but only if the directions of the two rotations are compatible. Third, motor rotation modifies the classical inverted V-shaped mental rotation response time function, favoring the direction of the motor rotation; indeed, in some cases motor rotation even shifts the location of the minimum of this curve in the direction of the motor rotation. Fourth, the preceding effect is sensitive not only to the direction of the motor rotation, but also to the motor speed. A change in the speed of motor rotation can correspondingly slow down or speed up the mental rotation.

When is early visual cortex activated during visual mental imagery

Although many neuroimaging studies of visual mental imagery have revealed activation in early visual cortex (Areas 17 or 18), many others have not. The authors review this literature and compare how well 3 models explain the disparate results. Each study was coded 1 or 0, indicating whether activation in early visual cortex was observed, and sets of variables associated with each model were fit to the observed results using logistic regression analysis. Three variables predicted all of the systematic differences in the probability of activation across studies. Two of these variables were identified with a perceptual anticipation theory, and the other was identified with a methodological factors theory. Thus, the variability in the literature is not random.

Form-Specific Visual Priming in the Right Cerebral Hemisphere

Results of 4 experiments indicate that both within-modality and case-specific visual priming for words are greater when test stimuli are presented initially to the right cerebral hemisphere (RH). In contrast, neither within-modality nor case-specific explicit memory for words is greater when stimuli are presented initially to the RH. Priming is measured using word-stem completion, and explicit memory is measured using word-stem cued recall. In both cases, Ss first rate how much they like words, and then word stems are presented briefly to the RH (in the left visual field) or to the left hemisphere (in the right visual field). Results suggest that at least 2 separate systems encode the visual representations that produce priming. The system that is more effective in the RH is better at representing form-specific information, whereas another system that is not more effective in the RH does not distinguish among distinct instances of word forms.