Rosa Lafer-Sousa

Parallel, multi-stage processing of colors, faces and shapes in macaque inferior temporal cortex

Visual-object processing culminates in inferior temporal cortex (IT). To assess the organization of IT, we measured functional magnetic resonance imaging responses in alert monkeys to achromatic images (faces, fruit, bodies and places) and colored gratings. IT contained multiple color-biased regions, which were typically ventral to face patches and yoked to them, spaced regularly at four locations predicted by known anatomy. Color and face selectivity increased for more anterior regions, indicative of a broad hierarchical arrangement. Responses to non-face shapes were found across IT, but were stronger outside color-biased regions and face patches, consistent with multiple parallel streams. IT also contained multiple coarse eccentricity maps: face patches overlapped central representations, color-biased regions spanned mid-peripheral representations and place-biased regions overlapped peripheral representations. These results show that IT comprises parallel, multi-stage processing networks subject to one organizing principle.

Color-Biased Regions of the Ventral Visual Pathway Lie between Face- and Place-Selective Regions in Humans, as in Macaques

The existence of color-processing regions in the human ventral visual pathway (VVP) has long been known from patient and imaging studies, but their location in the cortex relative to other regions, their selectivity for color compared with other properties (shape and object category), and their relationship to color-processing regions found in nonhuman primates remain unclear. We addressed these questions by scanning 13 subjects with fMRI while they viewed two versions of movie clips (colored, achromatic) of five different object classes (faces, scenes, bodies, objects, scrambled objects). We identified regions in each subject that were selective for color, faces, places, and object shape, and measured responses within these regions to the 10 conditions in independently acquired data. We report two key findings. First, the three previously reported color-biased regions (located within a band running posterior–anterior along the VVP, present in most of our subjects) were sandwiched between face-selective cortex and place-selective cortex, forming parallel bands of face, color, and place selectivity that tracked the fusiform gyrus/collateral sulcus. Second, the posterior color-biased regions showed little or no selectivity for object shape or for particular stimulus categories and showed no interaction of color preference with stimulus category, suggesting that they code color independently of shape or stimulus category; moreover, the shape-biased lateral occipital region showed no significant color bias. These observations mirror results in macaque inferior temporal cortex (Lafer-Sousa and Conway, 2013), and taken together, these results suggest a homology in which the entire tripartite face/color/place system of primates migrated onto the ventral surface in humans over the course of evolution. SIGNIFICANCE STATEMENT Here we report that color-biased cortex is sandwiched between face-selective and place-selective cortex on the bottom surface of the brain in humans. This face/color/place organization mirrors that seen on the lateral surface of the temporal lobe in macaques, suggesting that the entire tripartite system is homologous between species. This result validates the use of macaques as a model for human vision, making possible more powerful investigations into the connectivity, precise neural codes, and development of this part of the brain. In addition, we find substantial segregation of color from shape selectivity in posterior regions, as observed in macaques, indicating a considerable dissociation of the processing of shape and color in both species.

Psychophysical chromatic mechanisms in macaque monkey.

Chromatic mechanisms have been studied extensively with psychophysical techniques in humans, but the number and nature of the mechanisms are still controversial. Appeals to monkey neurophysiology are often used to sort out the competing claims and to test hypotheses arising from the experiments in humans, but psychophysical chromatic mechanisms have never been assessed in monkeys. Here we address this issue by measuring color-detection thresholds in monkeys before and after chromatic adaptation, employing a standard approach used to determine chromatic mechanisms in humans. We conducted separate experiments using adaptation configured as either flickering full-field colors or heterochromatic gratings. Full-field colors would favor activity within the visual system at or before the arrival of retinal signals to V1, before the spatial transformation of color signals by the cortex. Conversely, gratings would favor activity within the cortex where neurons are often sensitive to spatial chromatic structure. Detection thresholds were selectively elevated for the colors of full-field adaptation when it modulated along either of the two cardinal chromatic axes that define cone-opponent color space [L vs M or S vs (L + M)], providing evidence for two privileged cardinal chromatic mechanisms implemented early in the visual-processing hierarchy. Adaptation with gratings produced elevated thresholds for colors of the adaptation regardless of its chromatic makeup, suggesting a cortical representation comprised of multiple higher-order mechanisms each selective for a different direction in color space. The results suggest that color is represented by two cardinal channels early in the processing hierarchy and many chromatic channels in brain regions closer to perceptual readout.

Color-detection thresholds in rhesus macaque monkeys and humans

Macaque monkeys are a model of human color vision. To facilitate linking physiology in monkeys with psychophysics in humans, we directly compared color-detection thresholds in humans and rhesus monkeys. Colors were defined by an equiluminant plane of cone-opponent color space. All subjects were tested on an identical apparatus with a four-alternative forced-choice task. Targets were 2° square, centered 2° from fixation, embedded in luminance noise. Across all subjects, the change in detection thresholds from initial testing to plateau performance (“learning”) was similar for +L − M (red) colors and +M − L (bluish-green) colors. But the extent of learning was higher for +S (lavender) than for −S (yellow-lime); moreover, at plateau performance, the cone contrast at the detection threshold was higher for +S than for −S. These asymmetries may reflect differences in retinal circuitry for S-ON and S-OFF. At plateau performance, the two species also had similar detection thresholds for all colors, although monkeys had shorter reaction times than humans and slightly lower thresholds for colors that modulated L/M cones. We discuss whether these observations, together with previous work showing that monkeys have lower spatial acuity than humans, could be accounted for by selective pressures driving higher chromatic sensitivity at the cost of spatial acuity amongst monkeys, specifically for the more recently evolved L − M mechanism.

Facephenes and rainbows: Causal evidence for functional and anatomical specificity of face and color processing in the human brain

Significance Are some regions of the human brain exclusively engaged in a single specific mental process? Here we test this question in a neurosurgery patient implanted with electrodes for clinical reasons. When electrically stimulated in the fusiform face area while viewing objects, the patient reported illusory faces while the objects remained unchanged. When stimulated in nearby color-preferring sites, he reported seeing rainbows. The fact that stimulation of face-selective sites affected only face percepts and stimulation of color-preferring sites affected only color percepts, in both cases independent of the object being viewed, supports the view that some regions of cortex are indeed exclusively causally engaged in a single mental process and highlights the risks entailed in standard interpretations of neural decoding results. Neuroscientists have long debated whether some regions of the human brain are exclusively engaged in a single specific mental process. Consistent with this view, fMRI has revealed cortical regions that respond selectively to certain stimulus classes such as faces. However, results from multivoxel pattern analyses (MVPA) challenge this view by demonstrating that category-selective regions often contain information about “nonpreferred” stimulus dimensions. But is this nonpreferred information causally relevant to behavior? Here we report a rare opportunity to test this question in a neurosurgical patient implanted for clinical reasons with strips of electrodes along his fusiform gyri. Broadband gamma electrocorticographic responses in multiple adjacent electrodes showed strong selectivity for faces in a region corresponding to the fusiform face area (FFA), and preferential responses to color in a nearby site, replicating earlier reports. To test the causal role of these regions in the perception of nonpreferred dimensions, we then electrically stimulated individual sites while the patient viewed various objects. When stimulated in the FFA, the patient reported seeing an illusory face (or “facephene”), independent of the object viewed. Similarly, stimulation of color-preferring sites produced illusory “rainbows.” Crucially, the patient reported no change in the object viewed, apart from the facephenes and rainbows apparently superimposed on them. The functional and anatomical specificity of these effects indicate that some cortical regions are exclusively causally engaged in a single specific mental process, and prompt caution about the widespread assumption that any information scientists can decode from the brain is causally relevant to behavior.

#TheDress: Categorical perception of an ambiguous color image

We present a full analysis of data from our preliminary report (Lafer-Sousa, Hermann, & Conway, 2015) and test whether #TheDress image is multistable. A multistable image must give rise to more than one mutually exclusive percept, typically within single individuals. Clustering algorithms of color-matching data showed that the dress was seen categorically, as white/gold (W/G) or blue/black (B/K), with a blue/brown transition state. Multinomial regression predicted categorical labels. Consistent with our prior hypothesis, W/G observers inferred a cool illuminant, whereas B/K observers inferred a warm illuminant; moreover, subjects could use skin color alone to infer the illuminant. The data provide some, albeit weak, support for our hypothesis that day larks see the dress as W/G and night owls see it as B/K. About half of observers who were previously familiar with the image reported switching categories at least once. Switching probability increased with professional art experience. Priming with an image that disambiguated the dress as B/K biased reports toward B/K (priming with W/G had negligible impact); furthermore, knowledge of the dresss true colors and any prior exposure to the image shifted the population toward B/K. These results show that some people have switched their perception of the dress. Finally, consistent with a role of attention and local image statistics in determining how multistable images are seen, we found that observers tended to discount as achromatic the dress component that they did not attend to: B/K reporters focused on a blue region, whereas W/G reporters focused on a golden region.

Independence of color and shape processing in the ventral visual pathway of humans and macaques.

To what extent are color and shape processed independently of each other in the ventral visual pathway? Color is sometimes assumed to be just another stimulus feature for object recognition. On this view, there is a computational advantage to multiplexing color and shape processing. But the importance of color can sometimes be distinguished from shape: color signals a persons emotional state, independent of face features; similarly, color provides independent information about fruits edibility (bananas ripen, green to yellow, without changing shape). We asked whether distinct regions are sensitive to color and shape by scanning 13 humans and 2 rhesus monkeys with fMRI while they viewed full-color or achromatic movie clips of faces, objects, scenes, bodies, or scrambled objects. We defined color, face, place, and shape-selective regions within each subject, and measured the response magnitude of each region to the ten conditions in independently acquired data. We confirmed the existence of three color-biased regions in humans, and determined that none of these regions showed an interaction of color preference with stimulus category; moreover, two of the color-biased regions showed no higher response to intact versus scrambled objects, suggesting these regions code color independent of shape or stimulus category. Furthermore, shape-biased cortex (LO) did not show a bias for colored objects, providing a double dissociation of color and shape processing. Consistent with our previous report on responses to static stimuli in monkeys, a preliminary analysis of monkey data obtained using movie clips suggests that color-biased regions in inferior temporal cortex (IT) are no more responsive to intact than scrambled shapes, and that IT shape-selective regions are less sensitive to color compared to the color-biased regions. Taken together, these data support a dissociation of color and shape processing at certain stages of the visual-processing hierarchy in both humans and monkeys. Meeting abstract presented at VSS 2015.