Takao Noguchi

Crossmodal Adaptation in Right Posterior Superior Temporal Sulcus during Face–Voice Emotional Integration

The integration of emotional information from the face and voice of other persons is known to be mediated by a number of “multisensory” cerebral regions, such as the right posterior superior temporal sulcus (pSTS). However, whether multimodal integration in these regions is attributable to interleaved populations of unisensory neurons responding to face or voice or rather by multimodal neurons receiving input from the two modalities is not fully clear. Here, we examine this question using functional magnetic resonance adaptation and dynamic audiovisual stimuli in which emotional information was manipulated parametrically and independently in the face and voice via morphing between angry and happy expressions. Healthy human adult subjects were scanned while performing a happy/angry emotion categorization task on a series of such stimuli included in a fast event-related, continuous carryover design. Subjects integrated both face and voice information when categorizing emotion—although there was a greater weighting of face information—and showed behavioral adaptation effects both within and across modality. Adaptation also occurred at the neural level: in addition to modality-specific adaptation in visual and auditory cortices, we observed for the first time a crossmodal adaptation effect. Specifically, fMRI signal in the right pSTS was reduced in response to a stimulus in which facial emotion was similar to the vocal emotion of the preceding stimulus. These results suggest that the integration of emotional information from face and voice in the pSTS involves a detectable proportion of bimodal neurons that combine inputs from visual and auditory cortices.

In the attraction, compromise, and similarity effects, alternatives are repeatedly compared in pairs on single dimensions

In multi-alternative choice, the attraction, compromise, and similarity effects demonstrate that the value of an alternative is not independent of the other alternatives in the choice-set. Rather, these effects suggest that a choice is reached through the comparison of alternatives. We investigated exactly how alternatives are compared against each other using eye-movement data. The results indicate that a series of comparisons is made in each choice, with a pair of alternatives compared on a single attribute dimension in each comparison. We conclude that psychological models of choice should be based on these single-attribute pairwise comparisons.

Information overload or search-amplified risk? Set size and order effects on decisions from experience

How do changes in choice-set size influence information search and subsequent decisions? Moreover, does information overload influence information processing with larger choice sets? We investigated these questions by letting people freely explore sets of gambles before choosing one of them, with the choice sets either increasing or decreasing in number for each participant (from two to 32 gambles). Set size influenced information search, with participants taking more samples overall, but sampling a smaller proportion of gambles and taking fewer samples per gamble, when set sizes were larger. The order of choice sets also influenced search, with participants sampling from more gambles and taking more samples overall if they started with smaller as opposed to larger choice sets. Inconsistent with information overload, information processing appeared consistent across set sizes and choice order conditions, reliably favoring gambles with higher sample means. Despite the lack of evidence for information overload, changes in information search did lead to systematic changes in choice: People who started with smaller choice sets were more likely to choose gambles with the highest expected values, but only for small set sizes. For large set sizes, the increase in total samples increased the likelihood of encountering rare events at the same time that the reduction in samples per gamble amplified the effect of these rare events when they occurred—what we call search-amplified risk. This led to riskier choices for individuals whose choices most closely followed the sample mean.

Dissociating task difficulty from incongruence in face-voice emotion integration

In the everyday environment, affective information is conveyed by both the face and the voice. Studies have demonstrated that a concurrently presented voice can alter the way that an emotional face expression is perceived, and vice versa, leading to emotional conflict if the information in the two modalities is mismatched. Additionally, evidence suggests that incongruence of emotional valence activates cerebral networks involved in conflict monitoring and resolution. However, it is currently unclear whether this is due to task difficulty—that incongruent stimuli are harder to categorize—or simply to the detection of mismatching information in the two modalities. The aim of the present fMRI study was to examine the neurophysiological correlates of processing incongruent emotional information, independent of task difficulty. Subjects were scanned while judging the emotion of face-voice affective stimuli. Both the face and voice were parametrically morphed between anger and happiness and then paired in all audiovisual combinations, resulting in stimuli each defined by two separate values: the degree of incongruence between the face and voice, and the degree of clarity of the combined face-voice information. Due to the specific morphing procedure utilized, we hypothesized that the clarity value, rather than incongruence value, would better reflect task difficulty. Behavioral data revealed that participants integrated face and voice affective information, and that the clarity, as opposed to incongruence value correlated with categorization difficulty. Cerebrally, incongruence was more associated with activity in the superior temporal region, which emerged after task difficulty had been accounted for. Overall, our results suggest that activation in the superior temporal region in response to incongruent information cannot be explained simply by task difficulty, and may rather be due to detection of mismatching information between the two modalities.

Characterizing the Time-Perspective of Nations with Search Engine Query Data

Vast quantities of data on human behavior are being created by our everyday internet usage. Building upon a recent study by Preis, Moat, Stanley, and Bishop (2012), we used search engine query data to construct measures of the time-perspective of nations, and tested these measures against per-capita gross domestic product (GDP). The results indicate that nations with higher per-capita GDP are more focused on the future and less on the past, and that when these nations do focus on the past, it is more likely to be the distant past. These results demonstrate the viability of using nation-level data to build psychological constructs.

Eye movements in strategic choice

Abstract In risky and other multiattribute choices, the process of choosing is well described by random walk or drift diffusion models in which evidence is accumulated over time to threshold. In strategic choices, level‐k and cognitive hierarchy models have been offered as accounts of the choice process, in which people simulate the choice processes of their opponents or partners. We recorded the eye movements in 2 × 2 symmetric games including dominance‐solvable games like prisoners dilemma and asymmetric coordination games like stag hunt and hawk–dove. The evidence was most consistent with the accumulation of payoff differences over time: we found longer duration choices with more fixations when payoffs differences were more finely balanced, an emerging bias to gaze more at the payoffs for the action ultimately chosen, and that a simple count of transitions between payoffs—whether or not the comparison is strategically informative—was strongly associated with the final choice. The accumulator models do account for these strategic choice process measures, but the level‐k and cognitive hierarchy models do not.

Instantaneous Conventions: The Emergence of Flexible Communicative Signals.

Humans can communicate even with few existing conventions in common (e.g., when they lack a shared language). We explored what makes this phenomenon possible with a nonlinguistic experimental task requiring participants to coordinate toward a common goal. We observed participants creating new communicative conventions using the most minimal possible signals. These conventions, furthermore, changed on a trial-by-trial basis in response to shared environmental and task constraints. Strikingly, as a result, signals of the same form successfully conveyed contradictory messages from trial to trial. Such behavior is evidence for the involvement of what we term joint inference, in which social interactants spontaneously infer the most sensible communicative convention in light of the common ground between them. Joint inference may help to elucidate how communicative conventions emerge instantaneously and how they are modified and reshaped into the elaborate systems of conventions involved in human communication, including natural languages.

Multialternative decision by sampling : a model of decision making constrained by process data

Sequential sampling of evidence, or evidence accumulation, has been implemented in a variety of models to explain a range of multialternative choice phenomena. But the existing models do not agree on what, exactly, the evidence is that is accumulated. They also do not agree on how this evidence is accumulated. In this article, we use findings from process-tracing studies to constrain the evidence accumulation process. With these constraints, we extend the decision by sampling model and propose the multialternative decision by sampling (MDbS) model. In MDbS, the evidence accumulated is outcomes of pairwise ordinal comparisons between attribute values. MDbS provides a quantitative account of the attraction, compromise, and similarity effects equal to that of other models, and captures a wider range of empirical phenomena than other models.

Dynamic integration of conceptual information during learning

The development and application of concepts is a critical component of cognition. Although concepts can be formed on the basis of simple perceptual or semantic features, conceptual representations can also capitalize on similarities across feature relationships. By representing these types of higher-order relationships, concepts can simplify the learning problem and facilitate decisions. Despite this, little is known about the neural mechanisms that support the construction and deployment of these kinds of higher-order concepts during learning. To address this question, we combined a carefully designed associative learning task with computational model-based functional magnetic resonance imaging (fMRI). Participants were scanned as they learned and made decisions about sixteen pairs of cues and associated outcomes. Associations were structured such that individual cues shared feature relationships, operationalized as shared patterns of cue pair-outcome associations. In order to capture the large number of possible conceptual representational structures that participants might employ and to evaluate how conceptual representations are used during learning, we leveraged a well-specified Bayesian computational model of category learning [1]. Behavioral and model-based results revealed that participants who displayed a tendency to link experiences in memory benefitted from faster learning rates, suggesting that the use of the conceptual structure in the task facilitated decisions about cue pair-outcome associations. Model-based fMRI analyses revealed that trial-by-trial integration of cue information into higher-order conceptual representations was supported by an anterior temporal (AT) network of regions previously implicated in representing complex conjunctions of features and meaning-based information.