Erie D. Boorman

Frontal Cortex and Reward-Guided Learning and Decision-Making

Reward-guided decision-making and learning depends on distributed neural circuits with many components. Here we focus on recent evidence that suggests four frontal lobe regions make distinct contributions to reward-guided learning and decision-making: the lateral orbitofrontal cortex, the ventromedial prefrontal cortex and adjacent medial orbitofrontal cortex, anterior cingulate cortex, and the anterior lateral prefrontal cortex. We attempt to identify common themes in experiments with human participants and with animal models, which suggest roles that the areas play in learning about reward associations, selecting reward goals, choosing actions to obtain reward, and monitoring the potential value of switching to alternative courses of action.

How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action

Behavioral flexibility is the hallmark of goal-directed behavior. Whereas a great deal is known about the neural substrates of behavioral adjustment when it is explicitly cued by features of the external environment, little is known about how we adapt our behavior when such changes are made on the basis of uncertain evidence. Using a Bayesian reinforcement-learning model and fMRI, we show that frontopolar cortex (FPC) tracks the relative advantage in favor of switching to a foregone alternative when choices are made voluntarily. Changes in FPC functional connectivity occur when subjects finally decide to switch to the alternative behavior. Moreover, interindividual variation in the FPC signal predicts interindividual differences in effectively adapting behavior. By contrast, ventromedial prefrontal cortex (vmPFC) encodes the relative value of the current decision. Collectively, these findings reveal complementary prefrontal computations essential for promoting short- and long-term behavioral flexibility.

Consensus paper: Combining transcranial stimulation with neuroimaging

In the last decade, combined transcranial magnetic stimulation (TMS)-neuroimaging studies have greatly stimulated research in the field of TMS and neuroimaging. Here, we review how TMS can be combined with various neuroimaging techniques to investigate human brain function. When applied during neuroimaging (online approach), TMS can be used to test how focal cortex stimulation acutely modifies the activity and connectivity in the stimulated neuronal circuits. TMS and neuroimaging can also be separated in time (offline approach). A conditioning session of repetitive TMS (rTMS) may be used to induce rapid reorganization in functional brain networks. The temporospatial patterns of TMS-induced reorganization can be subsequently mapped by using neuroimaging methods. Alternatively, neuroimaging may be performed first to localize brain areas that are involved in a given task. The temporospatial information obtained by neuroimaging can be used to define the optimal site and time point of stimulation in a subsequent experiment in which TMS is used to probe the functional contribution of the stimulated area to a specific task. In this review, we first address some general methodologic issues that need to be taken into account when using TMS in the context of neuroimaging. We then discuss the use of specific brain mapping techniques in conjunction with TMS. We emphasize that the various neuroimaging techniques offer complementary information and have different methodologic strengths and weaknesses.

Functional specificity of human premotor-motor cortical interactions during action selection.

Functional connections between dorsal premotor cortex (PMd) and primary motor cortex (M1) have been revealed by paired‐pulse transcranial magnetic stimulation (TMS). We tested if such connections would be modulated during a cognitive process (response selection) known to rely on those circuits. PMd–M1 TMS applied 75 ms after a cue to select a manual response facilitated motor‐evoked potentials (MEPs). MEPs were facilitated at 50 ms in a control task of response execution, suggesting that PMd–M1 interactions at 75 ms are functionally specific to the process of response selection. At 100 ms, PMd–M1 TMS delayed choice reaction time (RT). Importantly, the MEP (at 75 ms) and the RT (at 100 ms) effects were correlated in a way that was hand‐specific. When the response was made with the M1‐contralateral hand, MEPs correlated with slower RTs. When the response was made with the M1‐ipsilateral hand, MEPs correlated with faster RTs. Paired‐pulse TMS confined to M1 did not produce these effects, confirming the causal influence of PMd inputs. This study shows that a response selection signal evolves in PMd early during the reaction period (75–100 ms), impacts on M1 and affects behaviour. Such interactions are temporally, anatomically and functionally specific, and have a causal role in choosing which movement to make.

Short-Latency Influence of Medial Frontal Cortex on Primary Motor Cortex during Action Selection under Conflict

Medial frontal cortex (MFC) is crucial when actions have to be inhibited, reprogrammed, or selected under conflict, but the precise mechanism by which it operates is unclear. Importantly, how and when the MFC influences the primary motor cortex (M1) during action selection is unknown. Using paired-pulse transcranial magnetic stimulation, we investigated functional connectivity between the presupplementary motor area (pre-SMA) part of MFC and M1. We found that functional connectivity increased in a manner dependent on cognitive context: pre-SMA facilitated the motor evoked-potential elicited by M1 stimulation only during action reprogramming, but not when otherwise identical actions were made in the absence of conflict. The effect was anatomically specific to pre-SMA; it was not seen when adjacent brain regions were stimulated. We discuss implications for the anatomical pathways mediating the observed effects.

A Network Centered on Ventral Premotor Cortex Exerts Both Facilitatory and Inhibitory Control over Primary Motor Cortex during Action Reprogramming

Ventral premotor cortex (PMv) is widely accepted to exert an important influence over primary motor cortex (M1) when hand movements are made. Although study of these interactions has typically focused on their excitatory nature, given its strong connections with both ventral and opercular frontal regions, one feature of the influence of PMv over M1 may be inhibitory. Paired-pulse transcranial magnetic stimulation (ppTMS) was used to examine functional interactions between human PMv and M1 during the selection and reprogramming of a naturalistic goal-directed action. One of two cylinders was illuminated on each trial. It was then grasped and picked up. On some trials, however, subjects had to reprogram the action as the illuminated cylinder was switched off and the other illuminated simultaneously with reach initiation. At a neurophysiological level, the PMv paired-pulse effect (PPE) on M1 corticospinal activity was facilitatory after the initial target presentation and during movement initiation. When reprogramming was required, however, the PPE became strongly inhibitory. This context-dependent change from facilitation to inhibition occurred within 75 ms of the change of target. Behaviorally, PMv-M1 ppTMS disrupted reprogramming. Diffusion-weighted magnetic resonance image scans were taken of each subject. Intersubject differences in the facilitation–inhibition contrast of PMv-M1 interactions were correlated with fractional anisotropy of white-matter in ventral prefrontal, premotor, and intraparietal brain areas. These results suggest that a network of brain areas centered on PMv inhibits M1 corticospinal activity associated with undesired movements when action plans change.

Individual differences in white-matter microstructure reflect variation in functional connectivity during choice.

The relation between brain structure and function is of fundamental importance in neuroscience. Comparisons between behavioral and brain-imaging measures suggest that variation in brain structure correlates with the presence of specific skills. Behavioral measures, however, reflect the integrated function of multiple brain regions. Rather than behavior, a physiological index of function could be a more sensitive and informative measure with which to compare structural measures. Here, we test for a relationship between a physiological measure of functional connectivity between two brain areas during a simple decision-making task and a measure of structural connectivity. Paired-pulse transcranial magnetic stimulation indexed functional connectivity between two regions important for action choices: the premotor and motor cortex. Fractional anisotropy (FA), a marker of microstructural integrity, indexed structural connectivity. Individual differences in functional connectivity during action selection show highly specific correlations with FA in localized regions of white-matter interconnecting regions, including the premotor and motor cortex. Probabilistic tractography, a technique for identifying fiber pathways from diffusion-weighted imaging (DWI), was used to reconstruct the anatomical networks linking the component brain regions involved in making decisions. These findings demonstrate a relationship between individual differences in functional and structural connectivity within human brain networks central to action choice.

Low GABA concentrations in occipital cortex and anterior cingulate cortex in medication-free, recovered depressed patients

Studies using proton magnetic resonance spectroscopy (1H-MRS) indicate that unmedicated, acutely depressed patients have decreased levels of gamma-aminobutyric acid (GABA) in the occipital cortex. The aim of this study was to use 1H-MRS to determine if changes in occipital and frontal cortical GABA levels were present in patients with a history of depression who had recovered and were no longer taking medication. We used 1H-MRS to measure levels of GABA in both occipital cortex and anterior cingulate cortex/prefrontal cortex in medication-free, fully recovered subjects with a history of recurrent unipolar depression. Levels of GABA in both occipital and anterior cingulate cortex were significantly lower in recovered depressed subjects than healthy controls. Our data provide preliminary evidence that a history of recurrent depression is associated with decreased GABA levels in anterior cingulate cortex and occipital cortex. These changes could represent part of the neurobiological vulnerability to recurrent depressive episodes.

Counterfactual Choice and Learning in a Neural Network Centered on Human Lateral Frontopolar Cortex

Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available—a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.

Ventromedial Prefrontal and Anterior Cingulate Cortex Adopt Choice and Default Reference Frames during Sequential Multi-Alternative Choice

Although damage to the medial frontal cortex causes profound decision-making impairments, it has been difficult to pinpoint the relative contributions of key anatomical subdivisions. Here we use function magnetic resonance imaging to examine the contributions of human ventromedial prefrontal cortex (vmPFC) and dorsal anterior cingulate cortex (dACC) during sequential choices between multiple alternatives—two key features of choices made in ecological settings. By carefully constructing options whose current value at any given decision was dissociable from their longer term value, we were able to examine choices in current and long-term frames of reference. We present evidence showing that activity at choice and feedback in vmPFC and dACC was tied to the current choice and the best long-term option, respectively. vmPFC, mid-cingulate, and posterior cingulate cortex encoded the relative value between the chosen and next best option at each sequential decision, whereas dACC encoded the relative value of adapting choices from the option with the highest value in the longer term. Furthermore, at feedback we identify temporally dissociable effects that predict repetition of the current choice and adaptation away from the long-term best option in vmPFC and dACC, respectively. These functional dissociations at choice and feedback suggest that sequential choices are subject to competing cortical mechanisms.