Nathan Witthoft

Russian blues reveal effects of language on color discrimination

English and Russian color terms divide the color spectrum differently. Unlike English, Russian makes an obligatory distinction between lighter blues (“goluboy”) and darker blues (“siniy”). We investigated whether this linguistic difference leads to differences in color discrimination. We tested English and Russian speakers in a speeded color discrimination task using blue stimuli that spanned the siniy/goluboy border. We found that Russian speakers were faster to discriminate two colors when they fell into different linguistic categories in Russian (one siniy and the other goluboy) than when they were from the same linguistic category (both siniy or both goluboy). Moreover, this category advantage was eliminated by a verbal, but not a spatial, dual task. These effects were stronger for difficult discriminations (i.e., when the colors were perceptually close) than for easy discriminations (i.e., when the colors were further apart). English speakers tested on the identical stimuli did not show a category advantage in any of the conditions. These results demonstrate that (i) categories in language affect performance on simple perceptual color tasks and (ii) the effect of language is online (and can be disrupted by verbal interference).

Synesthetic colors determined by having colored refrigerator magnets in childhood

Synesthesia is a condition in which percepts in one modality reliably elicit secondary perceptions in the same or a different modality that are not in the stimulus. In a common manifestation, synesthetes see colors in response to spoken or written letters, words and numbers. In this paper we demonstrate that the particular colors seen by a grapheme-color synesthete AED were learned from a set of refrigerator magnets and that the synesthesia later transferred to Cyrillic in a systematic way, with the colors induced by the Cyrillic letters determined by their visual or phonetic similarity to English letters. Closer examination of the data reveals that letters of either language that are more visually similar to the English capitals in the magnet set are also more saturated. In order to differentiate AEDs synesthesia from ordinary memory, we use a novel psychophysical method to show that AEDs synesthetic colors are subject to ordinary lightness constancy mechanisms. This suggests that the level of representation at which her synesthesia arises is early in the stream of visual processing.

Global similarity and pattern separation in the human medial temporal lobe predict subsequent memory.

Intense debate surrounds the role of medial temporal lobe (MTL) structures in recognition memory. Using high-resolution fMRI and analyses of pattern similarity in humans, we examined the encoding computations subserved by MTL subregions. Specifically, we tested the theory that MTL cortex supports memory by encoding overlapping representations, whereas hippocampus supports memory by encoding pattern-separated representations. Consistent with this view, the relationship between encoding pattern similarity and subsequent memory dissociated MTL cortex and hippocampus: later memory was predicted by greater across-item pattern similarity in perirhinal cortex and in parahippocampal cortex, but greater pattern distinctiveness in hippocampus. Additionally, by comparing neural patterns elicited by individual stimuli regardless of subsequent memory, we found that perirhinal cortex and parahippocampal cortex exhibited differential content sensitivity for multiple stimulus categories, whereas hippocampus failed to demonstrate content sensitivity. These data provide novel evidence that complementary MTL encoding computations subserve declarative memory.

Functionally Defined White Matter Reveals Segregated Pathways in Human Ventral Temporal Cortex Associated with Category-Specific Processing

It is unknown if the white-matter properties associated with specific visual networks selectively affect category-specific processing. In a novel protocol we combined measurements of white-matter structure, functional selectivity, and behavior in the same subjects. We find two parallel white-matter pathways along the ventral temporal lobe connecting to either face-selective or place-selective regions. Diffusion properties of portions of these tracts adjacent to face- and place-selective regions of ventral temporal cortex correlate with behavioral performance for face or place processing, respectively. Strikingly, adults with developmental prosopagnosia (face blindness) express an atypical structure-behavior relationship near face-selective cortex, suggesting that white-matter atypicalities in this region may have behavioral consequences. These data suggest that examining the interplay between cortical function, anatomical connectivity, and visual behavior is integral to understanding functional networks and their role in producing visual abilities and deficits.

Learning, Memory, and Synesthesia

People with color-grapheme synesthesia experience color when viewing written letters or numerals, usually with a particular color evoked by each grapheme. Here, we report on data from 11 color-grapheme synesthetes who had startlingly similar color-grapheme pairings traceable to childhood toys containing colored letters. These are the first and only data to show learned synesthesia of this kind in more than a single individual. Whereas some researchers have focused on genetic and perceptual aspects of synesthesia, our results indicate that a complete explanation of synesthesia must also incorporate a central role for learning and memory. We argue that these two positions can be reconciled by thinking of synesthesia as the automatic retrieval of highly specific mnemonic associations, in which perceptual contents are brought to mind in a manner akin to mental imagery or the perceptual-reinstatement effects found in memory studies.

Corresponding ECoG and fMRI category-selective signals in human ventral temporal cortex.

Functional magnetic resonance imaging (fMRI) and electrocorticography (ECoG) research have been influential in revealing the functional characteristics of category-selective responses in human ventral temporal cortex (VTC). One important, but unanswered, question is how these two types of measurements might be related with respect to the VTC. Here we examined which components of the ECoG signal correspond to the fMRI response by using a rare opportunity to measure both fMRI and ECoG responses from the same individuals to images of exemplars of various categories including faces, limbs, cars and houses. Our data reveal three key findings. First, we discovered that the coupling between fMRI and ECoG responses is frequency and time dependent. The strongest and most sustained correlation is observed between fMRI and high frequency broadband (HFB) ECoG responses (30-160 hz). In contrast, the correlation between fMRI and ECoG signals in lower frequency bands is temporally transient, where the correlation is initially positive, but then tapers off or becomes negative. Second, we find that the strong and positive correlation between fMRI and ECoG signals in all frequency bands emerges rapidly around 100 ms after stimulus onset, together with the onset of the first stimulus-driven neural signals in VTC. Third, we find that the spatial topology and representational structure of category-selectivity in VTC reflected in ECoG HFB responses mirrors the topology and structure observed with fMRI. These findings of a strong and rapid coupling between fMRI and HFB responses validate fMRI measurements of functional selectivity with recordings of direct neural activity and suggest that fMRI category-selective signals in VTC are associated with feed-forward neural processing.

Where Is Human V4? Predicting the Location of hV4 and VO1 from Cortical Folding

A strong relationship between cortical folding and the location of primary sensory areas in the human brain is well established. However, it is unknown if coupling between functional responses and gross anatomy is found at higher stages of sensory processing. We examined the relationship between cortical folding and the location of the retinotopic maps hV4 and VO1, which are intermediate stages in the human ventral visual processing stream. Our data show a consistent arrangement of the eccentricity maps within hV4 and VO1 with respect to anatomy, with the consequence that the hV4/VO1 boundary is found consistently in the posterior transverse collateral sulcus (ptCoS) despite individual variability in map size and cortical folding. Understanding this relationship allowed us to predict the location of visual areas hV4 and VO1 in a separate set of individuals, using only their anatomies, with >85% accuracy. These findings have important implications for understanding the relation between cortical folding and functional maps as well as for defining visual areas from anatomical landmarks alone.

Prevalence of learned grapheme-color pairings in a large online sample of synesthetes.

In this paper we estimate the minimum prevalence of grapheme-color synesthetes with letter-color matches learned from an external stimulus, by analyzing a large sample of English-speaking grapheme-color synesthetes. We find that at least 6% (400/6588 participants) of the total sample learned many of their matches from a widely available colored letter toy. Among those born in the decade after the toy began to be manufactured, the proportion of synesthetes with learned letter-color pairings approaches 15% for some 5-year periods. Among those born 5 years or more before it was manufactured, none have colors learned from the toy. Analysis of the letter-color matching data suggests the only difference between synesthetes with matches to the toy and those without is exposure to the stimulus. These data indicate learning of letter-color pairings from external contingencies can occur in a substantial fraction of synesthetes, and are consistent with the hypothesis that grapheme-color synesthesia is a kind of conditioned mental imagery.

Gender aftereffects in face silhouettes reveal face-specific mechanisms

The present study investigated how object locations learned separately are integrated and represented as a single spatial layout in memory. Two experiments were conducted in which participants learned a room-sized spatial layout that was divided into two sets of five objects. Results suggested that integration across sets was performed efficiently when it was done during initial encoding of the environment but entailed cost in accuracy when it was attempted at the time of memory retrieval. These findings suggest that, once formed, spatial representations in memory generally remain independent and integrating them into a single representation requires additional cognitive processes.

Illusions of visual motion elicited by electrical stimulation of human MT complex.

Human cortical area MT+ (hMT+) is known to respond to visual motion stimuli, but its causal role in the conscious experience of motion remains largely unexplored. Studies in non-human primates demonstrate that altering activity in area MT can influence motion perception judgments, but animal studies are inherently limited in assessing subjective conscious experience. In the current study, we use functional magnetic resonance imaging (fMRI), intracranial electrocorticography (ECoG), and electrical brain stimulation (EBS) in three patients implanted with intracranial electrodes to address the role of area hMT+ in conscious visual motion perception. We show that in conscious human subjects, reproducible illusory motion can be elicited by electrical stimulation of hMT+. These visual motion percepts only occurred when the site of stimulation overlapped directly with the region of the brain that had increased fMRI and electrophysiological activity during moving compared to static visual stimuli in the same individual subjects. Electrical stimulation in neighboring regions failed to produce illusory motion. Our study provides evidence for the sufficient causal link between the hMT+ network and the human conscious experience of visual motion. It also suggests a clear spatial relationship between fMRI signal and ECoG activity in the human brain.