Ai Koizumi

I Feel Your Voice Cultural Differences in the Multisensory Perception of Emotion

Cultural differences in emotion perception have been reported mainly for facial expressions and to a lesser extent for vocal expressions. However, the way in which the perceiver combines auditory and visual cues may itself be subject to cultural variability. Our study investigated cultural differences between Japanese and Dutch participants in the multisensory perception of emotion. A face and a voice, expressing either congruent or incongruent emotions, were presented on each trial. Participants were instructed to judge the emotion expressed in one of the two sources. The effect of to-be-ignored voice information on facial judgments was larger in Japanese than in Dutch participants, whereas the effect of to-be-ignored face information on vocal judgments was smaller in Japanese than in Dutch participants. This result indicates that Japanese people are more attuned than Dutch people to vocal processing in the multisensory perception of emotion. Our findings provide the first evidence that multisensory integration of affective information is modulated by perceivers’ cultural background.

Confidence Leak in Perceptual Decision Making

People live in a continuous environment in which the visual scene changes on a slow timescale. It has been shown that to exploit such environmental stability, the brain creates a continuity field in which objects seen seconds ago influence the perception of current objects. What is unknown is whether a similar mechanism exists at the level of metacognitive representations. In three experiments, we demonstrated a robust intertask confidence leak—that is, confidence in one’s response on a given task or trial influencing confidence on the following task or trial. This confidence leak could not be explained by response priming or attentional fluctuations. Better ability to modulate confidence leak predicted higher capacity for metacognition as well as greater gray matter volume in the prefrontal cortex. A model based on normative principles from Bayesian inference explained the results by postulating that observers subjectively estimate the perceptual signal strength in a stable environment. These results point to the existence of a novel metacognitive mechanism mediated by regions in the prefrontal cortex.

Does perceptual confidence facilitate cognitive control

Our visual perception is typically accompanied by a sense of subjective confidence. Since perceptual confidence is related to prefrontal activity, higher perceptual confidence may enhance cognitive control functions. To examine this interaction, we developed a novel method to selectively manipulate perceptual confidence while keeping stimulus discrimination accuracy constant. In a behavioral experiment, grating stimuli with different orientations were presented as go/no-go signals. Surprisingly, the results showed that confidence in visual discrimination of the signals on its own did not facilitate response inhibition, since when participants were presented with stimuli that yielded higher confidence, they were no better at performing a go/no-go task. These results were replicated with different (dot motion) stimuli, ruling out alternative explanations based on stimulus idiosyncrasy. In a different experiment, when the grating stimuli were presented as cues for task set preparation, we found that higher perceptual confidence also did not enhance task set preparation efficiency. This result was again replicated with dot motion stimuli. Since confidence may relate to perceptual awareness (Peirce & Jastrow, 1885), our findings may put current dominant theories in question, since these theories often suppose the critical involvement of consciousness in cognitive control. As a proof of concept, our method may also provide a new and powerful way to examine other functions of consciousness in future studies.

The effects of anxiety on the interpretation of emotion in the face–voice pairs

Anxious individuals have been shown to interpret others’ emotional states negatively. Since most studies have used facial expressions as emotional cues, we examined whether trait anxiety affects the recognition of emotion in a dynamic face and voice that were presented in synchrony. The face and voice cues conveyed either matched (e.g., happy face and voice) or mismatched emotions (e.g., happy face and angry voice). Participants with high or low trait anxiety were to indicate the perceived emotion using one of the cues while ignoring the other. The results showed that individuals with high trait anxiety were more likely to interpret others’ emotions in a negative manner, putting more weight on the to-be-ignored angry cues. This interpretation bias was found regardless of the cue modality (i.e., face or voice). Since trait anxiety did not affect recognition of the face or voice cues presented in isolation, this interpretation bias appears to reflect an altered integration of the face and voice cues among anxious individuals.

The pleasant heat? Evidence for thermal-emotional implicit associations occurring with semantic and physical thermal stimulation

The association between thermal and emotional experiences in interpersonal relations is intuitively apparent and has been confirmed by previous studies. However, research has not yet elucidated whether such an association is grounded in mental processes occurring at an intrapersonal (internal) level. In two experiments we examined whether the thermal-emotional associations can be observed at an intrapersonal level. We looked at the speed and accuracy of stimuli categorization. Experiment 1 examined the implicit semantic association between temperature (warm versus cold) and emotional valence (positive versus negative). Experiment 2 examined the association between experience of physical temperature and emotional valence. In both experiments warm-positive/cold-negative associations were demonstrated. These results suggest a conceptual and perceptual mapping in the mental representation of emotion and temperature, which occurs at an intrapersonal level, and which might serve as the ground to the interpersonal thermal-emotional interactions.

The effects of neurochemical balance in the anterior cingulate cortex and dorsolateral prefrontal cortex on volitional control under irrelevant distraction

Volitional control has been related to the excitatory/inhibitory (E/I) ratio of glutamate-glutamine to γ-aminobutyric acid concentration in the different parts of the frontal cortex. Yet, how the neurochemical balance in each of the brain areas modulates volitional control remains unclear. Here, participants performed an auditory Go/No-Go task with and without task-irrelevant face distractors. Neurochemical balance was measured with magnetic resonance spectroscopy at rest. Participants with higher E/I ratios in the dorsolateral prefrontal cortex (DLPFC) showed less control over No-Go cues under no distraction, whereas participants with higher E/I ratios in the anterior cingulate cortex (ACC) were more prompted to make speeded Go responses under distraction. Therefore, the neurochemical balance in the DLPFC and ACC may be involved in the control over task-relevant and -irrelevant cues respectively.

The pleasant heat? A study of thermal-emotion associations

Recent studies demonstrated that the physical feeling of warmth could make people judge others more favorably, act more generously (Williams and Bargh, 2008) and induce greater social proximity (IJzerman and Semin, 2009). In the present study, we examined whether temperature is implicitly associated with positive or negative valence. In Experiment 1, subjects judged the valence of the emotion words and pictures with two response buttons, of which one is physically warm and the other is physically cold, and measured the reaction time. The response button assignment can be either congruent (warm-positive/cold-negative) or incongruent (warm-negative/cold-positive). We found that for emotion words, the warm-positive/cold-negative congruence holds. However, for emotion pictures, reverse results were obtained. To further examine the thermo-valence association, follow-up implicit association tests (IATs) were conducted with positive/negative words and warm/cold words in Experiment 2, and positive/negative pictures with warm/cold pads in Experiment 3. The results from Experiment 2 show a tendency towards warm-positive/cold-negative congruence. However, such tendency was not found in Experiment 3. In summary, our results indicate that when the valence is presented semantically, it is implicitly associated with both physical thermal experience (EXP 1) and abstract thermal concept (EXP 2), and the association follows the common expectation of warm-positive/cold-negative congruence. However, when the valence is presented visually, the association is not consistent (EXP 1 and EXP 3). These findings suggest that temperature might interact differently with valences being elicited by semantic and visual information.