Tessa M. van Leeuwen

Effective Connectivity Determines the Nature of Subjective Experience in Grapheme-Color Synesthesia

Synesthesia provides an elegant model to investigate neural mechanisms underlying individual differences in subjective experience in humans. In grapheme–color synesthesia, written letters induce color sensations, accompanied by activation of color area V4. Competing hypotheses suggest that enhanced V4 activity during synesthesia is either induced by direct bottom-up cross-activation from grapheme processing areas within the fusiform gyrus, or indirectly via higher-order parietal areas. Synesthetes differ in the way synesthetic color is perceived: “projector” synesthetes experience color externally colocalized with a presented grapheme, whereas “associators” report an internally evoked association. Using dynamic causal modeling for fMRI, we show that V4 cross-activation during synesthesia was induced via a bottom-up pathway (within fusiform gyrus) in projector synesthetes, but via a top-down pathway (via parietal lobe) in associators. These findings show how altered coupling within the same network of active regions leads to differences in subjective experience. Our findings reconcile the two most influential cross-activation accounts of synesthesia.

Synaesthetic Colour in the Brain: Beyond Colour Areas. A Functional Magnetic Resonance Imaging Study of Synaesthetes and Matched Controls

Background In synaesthesia, sensations in a particular modality cause additional experiences in a second, unstimulated modality (e.g., letters elicit colour). Understanding how synaesthesia is mediated in the brain can help to understand normal processes of perceptual awareness and multisensory integration. In several neuroimaging studies, enhanced brain activity for grapheme-colour synaesthesia has been found in ventral-occipital areas that are also involved in real colour processing. Our question was whether the neural correlates of synaesthetically induced colour and real colour experience are truly shared. Methodology/Principal Findings First, in a free viewing functional magnetic resonance imaging (fMRI) experiment, we located main effects of synaesthesia in left superior parietal lobule and in colour related areas. In the left superior parietal lobe, individual differences between synaesthetes (projector-associator distinction) also influenced brain activity, confirming the importance of the left superior parietal lobe for synaesthesia. Next, we applied a repetition suppression paradigm in fMRI, in which a decrease in the BOLD (blood-oxygenated-level-dependent) response is generally observed for repeated stimuli. We hypothesized that synaesthetically induced colours would lead to a reduction in BOLD response for subsequently presented real colours, if the neural correlates were overlapping. We did find BOLD suppression effects induced by synaesthesia, but not within the colour areas. Conclusions/Significance Because synaesthetically induced colours were not able to suppress BOLD effects for real colour, we conclude that the neural correlates of synaesthetic colour experience and real colour experience are not fully shared. We propose that synaesthetic colour experiences are mediated by higher-order visual pathways that lie beyond the scope of classical, ventral-occipital visual areas. Feedback from these areas, in which the left parietal cortex is likely to play an important role, may induce V4 activation and the percept of synaesthetic colour.

Synaesthetic perception of colour and visual space in a blind subject: An fMRI case study

In spatial sequence synaesthesia (SSS) ordinal stimuli are perceived as arranged in peripersonal space. Using fMRI, we examined the neural bases of SSS and colour synaesthesia for spoken words in a late-blind synaesthete, JF. He reported days of the week and months of the year as both coloured and spatially ordered in peripersonal space; parts of the days and festivities of the year were spatially ordered but uncoloured. Words that denote time-units and triggered no concurrents were used in a control condition. Both conditions inducing SSS activated the occipito-parietal, infero-frontal and insular cortex. The colour area hOC4v was engaged when the synaesthetic experience included colour. These results confirm the continued recruitment of visual colour cortex in this late-blind synaesthetes. Synaesthesia also involved activation in inferior frontal cortex, which may be related to spatial memory and detection, and in the insula, which might contribute to audiovisual integration related to the processing of inducers and concurrents.

The merit of synesthesia for consciousness research

Synesthesia is a phenomenon in which additional perceptual experiences are elicited by sensory stimuli or cognitive concepts. Synesthetes possess a unique type of phenomenal experiences not directly triggered by sensory stimulation. Therefore, for better understanding of consciousness it is relevant to identify the mental and physiological processes that subserve synesthetic experience. In the present work we suggest several reasons why synesthesia has merit for research on consciousness. We first review the research on the dynamic and rapidly growing field of the studies of synesthesia. We particularly draw attention to the role of semantics in synesthesia, which is important for establishing synesthetic associations in the brain. We then propose that the interplay between semantics and sensory input in synesthesia can be helpful for the study of the neural correlates of consciousness, especially when making use of ambiguous stimuli for inducing synesthesia. Finally, synesthesia-related alterations of brain networks and functional connectivity can be of merit for the study of consciousness.

L2-Proficiency-Dependent Laterality Shift in Structural Connectivity of Brain Language Pathways

Diffusion tensor imaging (DTI) and a longitudinal language learning approach were applied to investigate the relationship between the achieved second language (L2) proficiency during L2 learning and the reorganization of structural connectivity between core language areas. Language proficiency tests and DTI scans were obtained from German students before and after they completed an intensive 6-week course of the Dutch language. In the initial learning stage, with increasing L2 proficiency, the hemispheric dominance of the Brodmann area (BA) 6-temporal pathway (mainly along the arcuate fasciculus) shifted from the left to the right hemisphere. With further increased proficiency, however, lateralization dominance was again found in the left BA6-temporal pathway. This result is consistent with reports in the literature that imply a stronger involvement of the right hemisphere in L2 processing especially for less proficient L2 speakers. This is the first time that an L2 proficiency-dependent laterality shift in the structural connectivity of language pathways during L2 acquisition has been observed to shift from left to right and back to left hemisphere dominance with increasing L2 proficiency. The authors additionally find that changes in fractional anisotropy values after the course are related to the time elapsed between the two scans. The results suggest that structural connectivity in (at least part of) the perisylvian language network may be subject to fast dynamic changes following language learning.

Nonrandom Associations of Graphemes with Colors in Arabic.

Numerous studies demonstrate people associate colors with letters and numbers in systematic ways. But most of these studies rely on speakers of English, or closely related languages. This makes it difficult to know how generalizable these findings are, or what factors might underlie these associations. We investigated letter-color and number-color associations in Arabic speakers, who have a different writing system and unusual word structure compared to Standard Average European languages. We also aimed to identify grapheme-color synaesthetes (people who have conscious color experiences with letters and numbers). Participants associated colors with 28 basic Arabic letters and ten digits by typing color names that best fit each grapheme. We found language-specific principles determining grapheme-color associations. For example, the word formation process in Arabic was relevant for color associations. In addition, psycholinguistic variables, such as letter frequency and the intrinsic order of graphemes influenced associations. Contrary to previous studies we found no evidence for sounds playing a role in letter-color associations for Arabic, and only a very limited role for shape influencing color associations. These findings highlight the importance of linguistic and psycholinguistic features in cross-modal correspondences, and illustrate why it is important to play close attention to each language on its own terms in order to disentangle language-specific from universal effects.

Phonological markers of information structure: an fMRI study

In this fMRI study we investigate the neural correlates of information structure integration during sentence comprehension in Dutch. We looked into how prosodic cues (pitch accents) that signal the information status of constituents to the listener (new information) are combined with other types of information during the unification process. The difficulty of unifying the prosodic cues into overall sentence meaning was manipulated by constructing sentences in which the pitch accent did (focus-accent agreement), and sentences in which the pitch accent did not (focus-accent disagreement) match the expectations for focus constituents of the sentence. In case of a mismatch, the load on unification processes increases. Our results show two anatomically distinct effects of focus-accent disagreement, one located in the posterior left inferior frontal gyrus (LIFG, BA6/44), and one in the more anterior-ventral LIFG (BA 47/45). Our results confirm that information structure is taken into account during unification, and imply an important role for the LIFG in unification processes, in line with previous fMRI studies.

Real color captures attention and overrides spatial cues in grapheme-color synesthetes but not in controls

Grapheme-color synesthetes perceive color when reading letters or digits. We investigated oscillatory brain signals of synesthetes vs. controls using magnetoencephalography. Brain oscillations specifically in the alpha band (∼10Hz) have two interesting features: alpha has been linked to inhibitory processes and can act as a marker for attention. The possible role of reduced inhibition as an underlying cause of synesthesia, as well as the precise role of attention in synesthesia is widely discussed. To assess alpha power effects due to synesthesia, synesthetes as well as matched controls viewed synesthesia-inducing graphemes, colored control graphemes, and non-colored control graphemes while brain activity was recorded. Subjects had to report a color change at the end of each trial which allowed us to assess the strength of synesthesia in each synesthete. Since color (synesthetic or real) might allocate attention we also included an attentional cue in our paradigm which could direct covert attention. In controls the attentional cue always caused a lateralization of alpha power with a contralateral decrease and ipsilateral alpha increase over occipital sensors. In synesthetes, however, the influence of the cue was overruled by color: independent of the attentional cue, alpha power decreased contralateral to the color (synesthetic or real). This indicates that in synesthetes color guides attention. This was confirmed by reaction time effects due to color, i.e. faster RTs for the color side independent of the cue. Finally, the stronger the observed color dependent alpha lateralization, the stronger was the manifestation of synesthesia as measured by congruency effects of synesthetic colors on RTs. Behavioral and imaging results indicate that color induces a location-specific, automatic shift of attention towards color in synesthetes but not in controls. We hypothesize that this mechanism can facilitate coupling of grapheme and color during the development of synesthesia.

Propagation of brain activity during audiovisual integration

An important question for research on audiovisual integration in humans is whether multisensory information is brought together in the primary sensory or association areas of the cortex. For example, can auditory information activate primary visual cortex directly, or must it first be processed by

Bridging the gap: Synaesthesia and multisensory processes

Although the study of multisensory processes and the study of synaesthesia both represent burgeoning fields of inquiry, there has been little attempt to bridge between these two research topics. This is somewhat surprising, as these two are undoubtedly heavily interrelated from both a psychological and neuroscience perspective. The goal of the present issue is explore these interrelationships, and focus on the state-of-the-field within these closely connected research domains. Synaesthesia is a fascinating phenomenon in which perception of different senses are mixed. For synaesthetes, specific sensory stimuli (automatically) trigger additional perceptual experiences that are not normally perceived by non-synaesthetes (e.g. letters elicit colours or words elicit tastes). Theories about the cause of this condition range from altered pruning and changes in anatomical connectivity to the disinhibition of typical feedback mechanisms common to all of us. The study of synaesthesia by itself has yielded many valuable insights over the past few decades (e.g. Ward, 2013), but a more current question in the field is how synaesthesia relates to other processes such as multisensory integration and crossmodal processing. The field of multisensory processing is in a period of rapid growth. Although the behavioral and perceptual benefits attributable to having information available from multiple senses has been long known (Welch and Warren, 1980; Sumby and Pollack, 1954), only recently has there been a concerted effort to better understand the psychological phenomena associated with multisensory functions and to link them to their neural correlates (Stein, 2012; Murray and Wallace, 2012). It is now well established that information from each of the senses converges at many sites within the brain, and that the product of this convergence is often a response that looks very different from the sum of the responses to the individual senses. Recent strides within this area have extended our understanding of the neural correlates of multisensory integration from the single cell to more distributed neural representations, have begun to provide evidence on the role of dynamic binding mechanisms in multisensory processing (e.g. Cappe et al., 2009, 2012; Carriere et al., 2008; Maier et al., 2008; Royal et al., 2009; Senkowski et al., 2007) and started to establish strong functional links between neural activity profiles and changes in behavior and perception (e.g. Murray et al., 2016). It is such a context that research into synaesthesia has become a focus for some interested in multisensory processes. With regard to relating the fields of multisensory processing and synaesthesia, one of the important issues that is being discussed is whether synaesthesia can be regarded as an extreme