Chai-Youn Kim

Psychophysical magic: rendering the visible ‘invisible’

What are the neural correlates of conscious visual awareness? Tackling this question requires contrasting neural correlates of stimulus processing culminating in visual awareness with neural correlates of stimulus processing unaccompanied by awareness. To produce these two neural states, one must be able to erase an otherwise visible stimulus from awareness. This article describes and assesses visual phenomena involving dissociation of physical stimulation and conscious awareness: degraded stimulation, visual masking, visual crowding, bistable figures, binocular rivalry, motion-induced blindness, inattentional blindness, change blindness and attentional blink. No single approach stands above the others, but those producing changing visual awareness despite invariant physical stimulation are clearly preferable. Such phenomena can help lead us ultimately to a comprehensive account of the neural correlates of conscious awareness.

Perceptual interaction between real and synesthetic colors

People with color-graphemic synesthesia experience vivid, reliable color upon viewing achromatic alphanumeric characters. Recent evidence indicates that synesthetic color experiences are as perceptually real as actual colors are for non-synesthetic observers. To investigate possible interactions between real and synesthetic colors, we tested two adult color-graphemic synesthetes on a pair of perceptual grouping tasks. In Experiment 1, we employed a well-known phenomenon of motion perception, bistable apparent motion, to explore whether synesthetic colors interact with real colors in grouping over time. Two-frame apparent motion sequences were presented with both path lengths and colors systematically manipulated. Results showed that synesthetic colors of motion tokens interacted with matching real colors of the corresponding motion tokens, which could subsequently bias perceived direction of motion. In Experiment 2, we exploited binocular rivalry, a condition under which two dissimilar monocular images compete with each other and result in perceptual switches, to explore whether synesthetic colors interact with real colors in grouping over space. Pairs of rival images with two different characters were presented dichoptically with colors of characters manipulated. Results showed that synesthetic and real colors of characters tended to group together, which, in turn, promoted the perceived global dominance during binocular rivalry. Therefore, the present results identify substantial interaction between synesthetic colors and real colors in perceptual grouping.

Brain activity accompanying perception of implied motion in abstract paintings

Early 20th century artists including Duchamp and Balla tried to portray moving objects on a static canvas by superimposing objects in successive portrayals of an action. We investigated whether implied motion in those paintings is associated with activation of motion-sensitive area MT+. In Experiment 1, we found that observers rated these kinds of paintings higher in portraying motion than they did other abstract paintings in which motion is not intended. We also found that observers who had previously experienced abstract paintings with implied motion tended to give higher motion ratings to that class of paintings. In Experiment 2, we used functional magnetic resonance imaging (fMRI) to measure brain activity of observers while viewing abstract paintings receiving the highest and the lowest motion rating scores in Experiment 1. We found MT+, but not primary visual cortex (V1), showed greater BOLD responses to abstract paintings with implied motion than to abstract paintings with little motion impression, but only in observers with prior experience viewing those kinds of paintings. These results imply that the neural machinery ordinarily engaged during perception of real visual motion is activated when people view paintings explicitly designed to convey a sense of visual motion. Experience, however, is necessary to achieve this sense of motion.

Auditory Attraction: Activation of Visual Cortex by Music and Sound in Williams Syndrome

Williams syndrome is a genetic neurodevelopmental disorder with a distinctive phenotype, including cognitive-linguistic features, nonsocial anxiety, and a strong attraction to music. we preformed functional MRI studies examining brain responses to musical and other types of stimuli in young adults with Williams syndrome and typically developing controls. In Study 1, the Williams syndrome group exhibited unforeseen activations of the visual cortex to musical stimuli, and it was this novel finding that became the focus of two subsequent studies. Using retinotopy, color localizers, and additional sound conditions, we identified specific visual areas in subjects with Williams syndrome that were activated by both musical and nonmusical auditory stimuli. The results, similar to synthetic-like experiences, have implications for cross-modal sensory processing in typical and atypical neurodevelopment.

Is “Σ” purple or green? Bistable grapheme-color synesthesia induced by ambiguous characters

People with grapheme-color synesthesia perceive specific colors when viewing different letters or numbers. Previous studies have suggested that synesthetic color experience can be bistable when induced by an ambiguous character. However, the exact relationship between processes underlying the identity of an alphanumeric character and the experience of the induced synesthetic color has not been examined. In the present study, we explored this by focusing on the temporal relation of inducer identification and color emergence using inducers whose identity could be rendered ambiguous upon rotation of the characters. Specifically, achromatic alphabetic letters (W/M) and digits (6/9) were presented at varying angles to 9 grapheme-color synesthetes. Results showed that grapheme identification and synesthetically perceived grapheme color covary with the orientation of the test stimulus and that synesthetes were slower naming the experienced color than identifying the character, particularly at intermediate angles where ambiguity was greatest.

Illusory colors promote interocular grouping during binocular rivalry

When dissimilar monocular images are presented separately to each of a person’s eyes, these images compete for visual dominance, with dominance of one image or the other alternating over time. While this phenomenon, called binocular rivalry, transpires, local image features distributed over space and between the eyes can become visually dominant at the same time; the resulting global figure implicates interocular grouping. Previous studies have suggested that color tends to influence the incidence of global dominance; in this study, we assess whether illusory color can also influence interocular grouping. To test this, we exploited the McCollough effect, an orientation-contingent color aftereffect induced by prolonged adaptation to different colors paired with different orientations. Results show that during binocular rivalry, illusory colors induced by the McCollough adaptation enhance strong interocular grouping relative to preadaptation testing, to an extent comparable in strength with the enhancement induced by real colors. Thus, illusory colors that are present only in an observer’s mind are sufficiently potent to influence low-level visual processes such as binocular rivalry.

Melodic sound enhances visual awareness of congruent musical notes, but only if you can read music.

Significance When left and right eyes disagree about what is being viewed, the brain resolves the disagreement by compromise: Visual awareness alternates between the two eyes’ views over time. Called “binocular rivalry,” these alternations in awareness are widely thought to reveal the usually implicit inferential nature of visual processing. In this study, we found that the perceptual dynamics defining rivalry are influenced by abstract, relational properties between visual and auditory sequences comprising musical melodies. However, this bisensory interaction impacts only the amount of time a visual sequence dominates awareness, with audiovisual interaction being powerless to hasten a visual sequence’s emergence from invisibility into awareness. These findings situate the inferential processing putatively transpiring during rivalry prior to extraction of abstract, semantic information. Predictive influences of auditory information on resolution of visual competition were investigated using music, whose visual symbolic notation is familiar only to those with musical training. Results from two experiments using different experimental paradigms revealed that melodic congruence between what is seen and what is heard impacts perceptual dynamics during binocular rivalry. This bisensory interaction was observed only when the musical score was perceptually dominant, not when it was suppressed from awareness, and it was observed only in people who could read music. Results from two ancillary experiments showed that this effect of congruence cannot be explained by differential patterns of eye movements or by differential response sluggishness associated with congruent score/melody combinations. Taken together, these results demonstrate robust audiovisual interaction based on high-level, symbolic representations and its predictive influence on perceptual dynamics during binocular rivalry.

Gender Differences in Neural Responses to Perceptually Invisible Fearful Face—An ERP Study

Women tend to respond to emotional stimuli differently from men. This study aimed at investigating whether neural responses to perceptually “invisible” emotional stimuli differ between men and women by exploiting event-related potential (ERP). Forty healthy participants (21 women) were recruited for the main experiment. A control experiment was conducted by excluding nine (7 women) participants from the main experiment and replacing them with additional ten (6 women) participants (total 41 participants) where Becks Anxiety Inventory (BAI) and Becks Depression Inventory (BDI) scores were controlled. Using the visual backward masking paradigm, either a fearful or a neutral face stimulus was presented in varied durations (subthreshold, near-threshold, or suprathreshold) followed by a mask. Participants performed a two-alternative forced choice (2-AFC) emotion discrimination task on each face. Behavioral analysis showed that participants were unaware of masked stimuli of which duration was the shortest and, therefore, processed at subthreshold. Nevertheless, women showed significantly larger response in P100 amplitude to subthreshold fearful faces than men. This result remained consistent in the control experiment. Our findings indicate gender-differences in neural response to subthreshold emotional face, which is reflected in the early processing stage.

Sequential effects in preference decision: Prior preference assimilates current preference

An important factor affecting preference formation is the context in which that preference decision takes place. The current research examined whether one’s preference formed for a previously presented stimulus influences the processing of a subsequent preference decision, henceforth referred to as the preference sequence effect. Using a novel sequential rating/judgment paradigm, the present study demonstrated the presence of a preference sequence effect using artistic photographs and face stimuli: A neutral stimulus was preferred more following a preferable stimulus than a less preferable stimulus. Furthermore, a similar trend was found even when the potential influence of response bias was controlled. These results suggest that an assimilative sequential effect exists even when sequential judgments are made solely based on one’s subjective feeling; preference formed for a preceding stimulus modulates preference for a subsequent stimulus. This implies the need for a consideration of trial sequence as a factor creating a psychological context affecting the subsequent preference decisions.

Graphemes Sharing Phonetic Features Tend to Induce Similar Synesthetic Colors

Individuals with grapheme-color synesthesia experience idiosyncratic colors when viewing achromatic letters or digits. Despite large individual differences in grapheme-color association, synesthetes tend to associate graphemes sharing a perceptual feature with similar synesthetic colors. Sound has been suggested as one such feature. In the present study, we investigated whether graphemes of which representative phonemes have similar phonetic features tend to be associated with analogous synesthetic colors. We tested five Korean multilingual synesthetes on a color-matching task using graphemes from Korean, English, and Japanese orthography. We then compared the similarity of synesthetic colors induced by those characters sharing a phonetic feature. Results showed that graphemes associated with the same phonetic feature tend to induce synesthetic color in both within- and cross-script analyses. Moreover, this tendency was consistent for graphemes that are not transliterable into each other as well as graphemes that are. These results suggest that it is the perceptual—i.e., phonetic—properties associated with graphemes, not just conceptual associations such as transliteration, that determine synesthetic color.