Tetsuto Minami

Effects of color information on face processing using event-related potentials and gamma oscillations.

In humans, face configuration, contour and color may affect face perception, which is important for social interactions. This study aimed to determine the effect of color information on face perception by measuring event-related potentials (ERPs) during the presentation of natural- and bluish-colored faces. Our results demonstrated that the amplitude of the N170 event-related potential, which correlates strongly with face processing, was higher in response to a bluish-colored face than to a natural-colored face. However, gamma-band activity was insensitive to the deviation from a natural face color. These results indicated that color information affects the N170 associated with a face detection mechanism, which suggests that face color is important for face detection.

The face-selective N170 component is modulated by facial color

Faces play an important role in social interaction by conveying information and emotion. Of the various components of the face, color particularly provides important clues with regard to perception of age, sex, health status, and attractiveness. In event-related potential (ERP) studies, the N170 component has been identified as face-selective. To determine the effect of color on face processing, we investigated the modulation of N170 by facial color. We recorded ERPs while subjects viewed facial color stimuli at 8 hue angles, which were generated by rotating the original facial color distribution around the white point by 45° for each human face. Responses to facial color were localized to the left, but not to the right hemisphere. N170 amplitudes gradually increased in proportion to the increase in hue angle from the natural-colored face. This suggests that N170 amplitude in the left hemisphere reflects processing of facial color information.

Emotion Voice Analysis System Connected to the Human Brain

To investigate human brain activities in association with emotional speech, we developed a novel voice analysis system connected to a functional magnetic resonance imaging (fMRI) machine. Participants spoke inside the MR magnet during which BOLD activities of the brain was measured. Speech voice was transmitted through the newly developed mask-microphone inside the magnet to the external computer and was processed by the emotional voice analysis system. Two participants conversed without hindrance and their emotional state was analyzed. Using the system, we were able to detect brain activities during speech and simultaneously evaluate the human emotional voice.

Facial color processing in the face-selective regions: An fMRI study

Facial color is important information for social communication as it provides important clues to recognize a persons emotion and health condition. Our previous EEG study suggested that N170 at the left occipito‐temporal site is related to facial color processing (Nakajima et al., [2012]: Neuropsychologia 50:2499–2505). However, because of the low spatial resolution of EEG experiment, the brain region is involved in facial color processing remains controversial. In the present study, we examined the neural substrates of facial color processing using functional magnetic resonance imaging (fMRI). We measured brain activity from 25 subjects during the presentation of natural‐ and bluish‐colored face and their scrambled images. The bilateral fusiform face (FFA) area and occipital face area (OFA) were localized by the contrast of natural‐colored faces versus natural‐colored scrambled images. Moreover, region of interest (ROI) analysis showed that the left FFA was sensitive to facial color, whereas the right FFA and the right and left OFA were insensitive to facial color. In combination with our previous EEG results, these data suggest that the left FFA may play an important role in facial color processing. Hum Brain Mapp 35:4958–4964, 2014.

Electrophysiological Differences in the Processing of Affect Misattribution

The affect misattribution procedure (AMP) was proposed as a technique to measure an implicit attitude to a prime image [1]. In the AMP, neutral symbols (e.g., a Chinese pictograph, called the target) are presented, following an emotional stimulus (known as the prime). Participants often misattribute the positive or negative affect of the priming images to the targets in spite of receiving an instruction to ignore the primes. The AMP effect has been investigated using behavioral measures; however, it is difficult to identify when the AMP effect occurs in emotional processing—whether the effect may occur in the earlier attention allocation stage or in the later evaluation stage. In this study, we examined the neural correlates of affect misattribution, using event-related potential (ERP) dividing the participants into two groups based on their tendency toward affect misattribution. The ERP results showed that the amplitude of P2 was larger for the prime at the parietal location in participants showing a low tendency to misattribution than for those showing a high tendency, while the effect of judging neutral targets amiss according to the primes was reflected in the late processing of targets (LPP). In addition, the topographic pattern analysis revealed that EPN-like component to targets was correlated with the difference of AMP tendency as well as P2 to primes and LPP to targets. Taken together, the mechanism of the affective misattribution was closely related to the attention allocation processing. Our findings provide neural evidence that evaluations of neutral targets are misattributed to emotional primes.

Semantic processing in subliminal face stimuli: an EEG and tDCS study.

Whether visual subliminal processing involves semantic processing is still being debated. To examine this, we combined a passive electroencephalogram (EEG) study with an application of transcranial direct current stimulation (tDCS). In the masked-face priming paradigm, we presented a subliminal prime preceding the target stimulus. Participants were asked to determine whether the target face was a famous face, indicated by a button press. The prime and target pair were either the same persons face (congruent) or different persons faces (incongruent), and were always both famous or both non-famous faces. Experiments were performed over 2 days: 1 day for a real tDCS session and another for a sham session as a control condition. In the sham session, a priming effect, reflected in the difference in amplitude of the late positive component (250-500 ms to target onset), was observed only in the famous prime condition. According to a previous study, this effect might indicate a subliminal semantic process [10]. Alternatively, a priming effect toward famous primes disappeared after tDCS stimulation. Our results suggested that a subliminal process might not be limited to processes in the occipital and temporal areas, but may proceed to the semantic level processed in prefrontal cortex.

Asymmetry of P3 amplitude during oddball tasks reflects the unnaturalness of visual stimuli.

This study examined the relationship between the ‘naturalness’ of a visual stimulus and the event-related potentials measured during an oddball task. The study focused on asymmetry of the P3 amplitude during an oddball task or P3 asymmetry. Participants performed two visual oddball tasks using a pair of stimuli (A and B): one in which A was the target stimulus and B was the standard stimulus and vice versa. The stimuli consisted of natural–unnatural pairs of visual stimuli (e.g. upright–inverted faces, possible–impossible human poses). As a result of comparing the amplitudes of the target stimuli, P3 asymmetry was found in natural–unnatural pairs; that is, their naturalness differentiated the target P3 amplitude: larger P3 to the unnatural target than to the natural one. This study showed that P3 asymmetry reflected unnaturalness and unfamiliarity of visual stimuli.

Interaction between facial expression and color

Facial color varies depending on emotional state, and emotions are often described in relation to facial color. In this study, we investigated whether the recognition of facial expressions was affected by facial color and vice versa. In the facial expression task, expression morph continua were employed: fear-anger and sadness-happiness. The morphed faces were presented in three different facial colors (bluish, neutral, and reddish color). Participants identified a facial expression between the two endpoints (e.g., fear or anger) regardless of its facial color. The results showed that the perception of facial expression was influenced by facial color. In the fear-anger morphs, intermediate morphs of reddish-colored and bluish colored faces had a greater tendency to be identified as angry faces and fearful faces, respectively. In the facial color task, two bluish-to-reddish colored face continua were presented in three different facial expressions (fear-neutral-anger and sadness-neutral-happiness). Participants judged whether the facial color was reddish or bluish regardless of its expression. The faces with sad expression tended to be identified as more bluish, while the faces with other expressions did not affect facial color judgment. These results suggest that an interactive but disproportionate relationship exists between facial color and expression in face perception.

The effects of facial color and inversion on the N170 event-related potential (ERP) component.

Faces are important for social interaction because much can be perceived from facial details, including a persons race, age, and mood. Recent studies have shown that both configural (e.g. face shape and inversion) and surface information (e.g. surface color and reflectance properties) are important for face perception. Therefore, the present study examined the effects of facial color and inverted face properties on event-related potential (ERP) responses, particularly the N170 component. Stimuli consisted of natural and bluish-colored faces. Faces were presented in both upright and upside down orientations. An ANOVA was used to analyze N170 amplitudes and verify the effects of the main independent variables. Analysis of N170 amplitude revealed the significant interactions between stimulus orientation and color. Subsequent analysis indicated that N170 was larger for bluish-colored faces than natural-colored faces, and N170 to natural-colored faces was larger in response to inverted stimulus as compared to upright stimulus. Additionally, a multivariate pattern analysis (MVPA) investigated face-processing dynamics without any prior assumptions. Results distinguished, above chance, both facial color and orientation from single-trial electroencephalogram (EEG) signals. Decoding performance for color classification of inverted faces was significantly diminished as compared to an upright orientation. This suggests that processing orientation is predominant over facial color. Taken together, the present findings elucidate the temporal and spatial distribution of orientation and color processing during face processing.

Identifying neural components of emotion in free conversation with fMRI

To evaluate neural components of emotional utterance, functional Magnetic Resonance Imaging (fMRI) was operated during free conversation. Timing and types of emotional elements such as anger, sorrow, joy, excitement and calmness were identified by the Voice Emotion Analysis (VEA) system. Conducting the modified event-related analysis, we found increased Blood Oxygen Level Dependent (BOLD) activities during free conversation in the lateral frontal cortex (BA47). Furthermore, the dorsolateral frontal cortex (BA45) and the limbic cortex were activated when the VEA system indicated excitement and anger, respectively. Since these areas are consistent with the neural circuits subserving emotional speech, the results confirm the neurophysiological correlates of emotion extracted by specific patterns of phonetic parameters during speech production.