Tom Schonberg

Reinforcement Learning Signals in the Human Striatum Distinguish Learners from Nonlearners during Reward-Based Decision Making

The computational framework of reinforcement learning has been used to forward our understanding of the neural mechanisms underlying reward learning and decision-making behavior. It is known that humans vary widely in their performance in decision-making tasks. Here, we used a simple four-armed bandit task in which subjects are almost evenly split into two groups on the basis of their performance: those who do learn to favor choice of the optimal action and those who do not. Using models of reinforcement learning we sought to determine the neural basis of these intrinsic differences in performance by scanning both groups with functional magnetic resonance imaging. We scanned 29 subjects while they performed the reward-based decision-making task. Our results suggest that these two groups differ markedly in the degree to which reinforcement learning signals in the striatum are engaged during task performance. While the learners showed robust prediction error signals in both the ventral and dorsal striatum during learning, the nonlearner group showed a marked absence of such signals. Moreover, the magnitude of prediction error signals in a region of dorsal striatum correlated significantly with a measure of behavioral performance across all subjects. These findings support a crucial role of prediction error signals, likely originating from dopaminergic midbrain neurons, in enabling learning of action selection preferences on the basis of obtained rewards. Thus, spontaneously observed individual differences in decision making performance demonstrate the suggested dependence of this type of learning on the functional integrity of the dopaminergic striatal system in humans.

Characterization of displaced white matter by brain tumors using combined DTI and fMRI.

In vivo white matter tractography by diffusion tensor imaging (DTI) has become a popular tool for investigation of white matter architecture in the normal brain. Despite some unresolved issues regarding the accuracy of DTI, recent studies applied DTI for delineating white matter organization in the vicinity of brain lesions and especially brain tumors. Apart from the intrinsic limitations of DTI, the tracking of fibers in the vicinity or within lesions is further complicated due to changes in diseased tissue such as elevated water content (edema), tissue compression and degeneration. These changes deform the architecture of the white matter and in some cases prevent definite selection of the seed region of interest (ROI) from which fiber tracking begins. We show here that for displaced fiber systems, the use of anatomical approach for seed ROI selection yields insufficient results. Alternatively, we propose to select the seed points based on functional MRI activations which constrain the subjective seed ROI selection. The results are demonstrated on two major fiber systems: the pyramidal tract and the superior longitudinal fasciculus that connect critical motor and language areas, respectively. The fMRI based seed ROI selection approach enabled a more comprehensive mapping of these fiber systems. Furthermore, this procedure enabled the characterization of displaced white matter using the eigenvalue decomposition of DTI. We show that along the compressed fiber system, the diffusivity parallel to the fiber increases, while that perpendicular to the fibers decreases, leading to an overall increase in the fractional anisotropy index reflecting the compression of the fiber bundle. We conclude that definition of the functional network of a subject with deformed white matter should be done carefully. With fMRI, one can more accurately define the seed ROI for DTI based tractography and to provide a more comprehensive, functionally related, white matter mapping, a very important tool used in pre-surgical mapping.

Mind the gap: bridging economic and naturalistic risk-taking with cognitive neuroscience

Economists define risk in terms of the variability of possible outcomes, whereas clinicians and laypeople generally view risk as exposure to possible loss or harm. Neuroeconomic studies using relatively simple behavioral tasks have identified a network of brain regions that respond to economic risk, but these studies have had limited success predicting naturalistic risk-taking. By contrast, more complex behavioral tasks developed by clinicians (e.g. Balloon Analogue Risk Task and Iowa Gambling Task) correlate with naturalistic risk-taking but resist decomposition into distinct cognitive constructs. We propose here that to bridge this gap and better understand neural substrates of naturalistic risk-taking, new tasks are needed that: are decomposable into basic cognitive and/or economic constructs; predict naturalistic risk-taking; and engender dynamic, affective engagement.

Bihemispheric leftward bias in a visuospatial attention-related network.

Asymmetry of spatial attention has long been described in both disease (hemispatial neglect) and healthy (pseudoneglect) states. Although right-hemisphere specialization for spatial attention has been suggested, the exact neural mechanisms of asymmetry have not been deciphered yet. A recent functional magnetic resonance imaging study from our laboratory serendipitously revealed bihemispheric left-hemifield superiority in activation of a visuospatial attention-related network. Nineteen right-handed healthy adult females participated in two experiments of visual half-field presentation. Either facial expressions (experiment 1) or house images (experiment 2) were presented unilaterally and parafoveally for 150 ms while subjects were engaging a central fixation task. Brain regions previously associated with a visuospatial attention network, in both hemispheres, were found to be more robustly activated by left visual field stimuli. The consistency of this finding with manifestations of attention lateralization is discussed, and a revised model based on neural connectivity asymmetry is proposed. Support for the revised model is given by a dynamic causal modeling analysis. Unraveling the basis for attention asymmetry may lead to better understanding of the pathogenesis of attention disorders, followed by improved diagnosis and treatment. Additionally, the proposed model for asymmetry of visuospatial attention might provide important insights into the mechanisms underlying functional brain lateralization in general.

Decreasing ventromedial prefrontal cortex activity during sequential risk-taking: an FMRI investigation of the balloon analog risk task.

Functional imaging studies examining the neural correlates of risk have mainly relied on paradigms involving exposure to simple chance gambles and an economic definition of risk as variance in the probability distribution over possible outcomes. However, there is little evidence that choices made during gambling tasks predict naturalistic risk-taking behaviors such as drug use, extreme sports, or even equity investing. To better understand the neural basis of naturalistic risk-taking, we scanned participants using fMRI while they completed the Balloon Analog Risk Task, an experimental measure that includes an active decision/choice component and that has been found to correlate with a number of naturalistic risk-taking behaviors. In the task, as in many naturalistic settings, escalating risk-taking occurs under uncertainty and might be experienced either as the accumulation of greater potential rewards, or as exposure to increasing possible losses (and decreasing expected value). We found that areas previously linked to risk and risk-taking (bilateral anterior insula, anterior cingulate cortex, and right dorsolateral prefrontal cortex) were activated as participants continued to inflate balloons. Interestingly, we found that ventromedial prefrontal cortex (vmPFC) activity decreased as participants further expanded balloons. In light of previous findings implicating the vmPFC in value calculation, this result suggests that escalating risk-taking in the task might be perceived as exposure to increasing possible losses (and decreasing expected value) rather than the increasing potential total reward relative to the starting point of the trial. A better understanding of how neural activity changes with risk-taking behavior in the task offers insight into the potential neural mechanisms driving naturalistic risk-taking.

Changing value through cued approach: an automatic mechanism of behavior change

It is believed that choice behavior reveals the underlying value of goods. The subjective values of stimuli can be changed through reward-based learning mechanisms as well as by modifying the description of the decision problem, but it has yet to be shown that preferences can be manipulated by perturbing intrinsic values of individual items. Here we show that the value of food items can be modulated by the concurrent presentation of an irrelevant auditory cue to which subjects must make a simple motor response (i.e., cue-approach training). Follow-up tests showed that the effects of this pairing on choice lasted at least 2 months after prolonged training. Eye-tracking during choice confirmed that cue-approach training increased attention to the cued items. Neuroimaging revealed the neural signature of a value change in the form of amplified preference-related activity in ventromedial prefrontal cortex.

Predicting risky choices from brain activity patterns

Significance Previous studies have examined the neural correlates of risk, but it is unknown if patterns of brain activity can predict choices in risky decision-making. We used functional MRI data to predict choice behavior in subjects while they performed a naturalistic risk-taking task. We found choices on subsequent trials could be predicted with high accuracy when condensing each individuals brain activity to two values, indicating that choice behavior is encoded even in coarse activation patterns. A searchlight analysis demonstrated that choices can also be predicted based on localized activity patterns within neural networks involved in cognitive control. These regions show greater activation prior to safe choices than risky choices, suggesting that control systems play a key role in inhibiting risky choices. Previous research has implicated a large network of brain regions in the processing of risk during decision making. However, it has not yet been determined if activity in these regions is predictive of choices on future risky decisions. Here, we examined functional MRI data from a large sample of healthy subjects performing a naturalistic risk-taking task and used a classification analysis approach to predict whether individuals would choose risky or safe options on upcoming trials. We were able to predict choice category successfully in 71.8% of cases. Searchlight analysis revealed a network of brain regions where activity patterns were reliably predictive of subsequent risk-taking behavior, including a number of regions known to play a role in control processes. Searchlights with significant predictive accuracy were primarily located in regions more active when preparing to avoid a risk than when preparing to engage in one, suggesting that risk taking may be due, in part, to a failure of the control systems necessary to initiate a safe choice. Additional analyses revealed that subject choice can be successfully predicted with minimal decrements in accuracy using highly condensed data, suggesting that information relevant for risky choice behavior is encoded in coarse global patterns of activation as well as within highly local activation within searchlights.

Discovering Relations Between Mind, Brain, and Mental Disorders Using Topic Mapping

Neuroimaging research has largely focused on the identification of associations between brain activation and specific mental functions. Here we show that data mining techniques applied to a large database of neuroimaging results can be used to identify the conceptual structure of mental functions and their mapping to brain systems. This analysis confirms many current ideas regarding the neural organization of cognition, but also provides some new insights into the roles of particular brain systems in mental function. We further show that the same methods can be used to identify the relations between mental disorders. Finally, we show that these two approaches can be combined to empirically identify novel relations between mental disorders and mental functions via their common involvement of particular brain networks. This approach has the potential to discover novel endophenotypes for neuropsychiatric disorders and to better characterize the structure of these disorders and the relations between them.

Mind your left: Spatial bias in subcortical fear processing

Hemispheric lateralization of emotional processing has long been suggested, but its underlying neural mechanisms have not yet been defined. In this functional magnetic resonance imaging study, facial expressions were presented to 10 right-handed healthy adult females in an event-related visual half-field presentation paradigm. Differential activations to fearful versus neutral faces were observed in the amygdala, pulvinar, and superior colliculus only for faces presented in the left hemifield. Interestingly, the left hemifield advantage for fear processing was observed in both hemispheres. These results suggest a leftward bias in subcortical fear processing, consistent with the well-documented leftward bias of danger-associated behaviors in animals. The current finding highlights the importance of hemifield advantage in emotional lateralization, which might reflect the combination of hemispheric dominance and asymmetric interhemispheric information transfer.

Differences in neural activation as a function of risk-taking task parameters

Despite evidence supporting a relationship between impulsivity and naturalistic risk-taking, the relationship of impulsivity with laboratory-based measures of risky decision-making remains unclear. One factor contributing to this gap in our understanding is the degree to which different risky decision-making tasks vary in their details. We conducted an fMRI investigation of the Angling Risk Task (ART), which is an improved behavioral measure of risky decision-making. In order to examine whether the observed pattern of neural activation was specific to the ART or generalizable, we also examined correlates of the Balloon Analog Risk Taking (BART) task in the same sample of 23 healthy adults. Exploratory analyses were conducted to examine the relationship between neural activation, performance, impulsivity and self-reported risk-taking. While activation in a valuation network was associated with reward tracking during the ART but not the BART, increased fronto-cingulate activation was seen during risky choice trials in the BART as compared to the ART. Thus, neural activation during risky decision-making trials differed between the two tasks, and this observation was likely driven by differences in task parameters, namely the absence vs. presence of ambiguity and/or stationary vs. increasing probability of loss on the ART and BART, respectively. Exploratory association analyses suggest that sensitivity of neural response to the magnitude of potential reward during the ART was associated with a suboptimal performance strategy, higher scores on a scale of dysfunctional impulsivity (DI) and a greater likelihood of engaging in risky behaviors, while this pattern was not seen for the BART. Our results suggest that the ART is decomposable and associated with distinct patterns of neural activation; this represents a preliminary step toward characterizing a behavioral measure of risky decision-making that may support a better understanding of naturalistic risk-taking.