Massimo Silvetti

Damage to White Matter Pathways in Subacute and Chronic Spatial Neglect: A Group Study and 2 Single-Case Studies with Complete Virtual “In Vivo” Tractography Dissection

The exact anatomical localization of right hemisphere lesions that lead to left spatial neglect is still debated. The effect of confounding factors such as acute diaschisis and hypoperfusion, visual field defects, and lesion size may account for conflicting results that have been reported in the literature. Here, we present a comprehensive anatomical investigation of the gray- and white matter lesion correlates of left spatial neglect, which was run in a sample 58 patients with subacute or chronic vascular strokes in the territory of the right middle cerebral artery. Standard voxel-based correlates confirmed the role played by lesions in the posterior parietal cortex (supramarginal gyrus, angular gyrus, and temporal-parietal junction), in the frontal cortex (frontal eye field, middle and inferior frontal gyrus), and in the underlying parietal-frontal white matter. Using a new diffusion tensor imaging-based atlas of the human brain, we were able to run, for the first time, a detailed analysis of the lesion involvement of subcortical white matter pathways. The results of this analysis revealed that, among the different pathways linking parietal with frontal areas, damage to the second branch of the superior longitudinal fasciculus (SLF II) was the best predictor of left spatial neglect. The group study also revealed a subsample of patients with neglect due to focal lesion in the lateral-dorsal portion of the thalamus, which connects the premotor cortex with the inferior parietal lobule. The relevance of fronto-parietal disconnection was further supported by complete in vivo tractography dissection of white matter pathways in 2 patients, one with and the other without signs of neglect. These 2 patients were studied both in the acute phase and 1 year after stroke and were perfectly matched for age, handedness, stroke onset, lesion size, and for cortical lesion involvement. Taken together, the results of the present study support the hypothesis that anatomical disconnections leading to a functional breakdown of parietal-frontal networks are an important pathophysiological factor leading to chronic left spatial neglect. Here, we propose that different loci of SLF disconnection on the rostro-caudal axis can also be associated with disconnection of short-range white matter pathways within the frontal or parietal areas. Such different local disconnection patterns can play a role in the important clinical variability of the neglect syndrome.

Value and Prediction Error in Medial Frontal Cortex: Integrating the Single-Unit and Systems Levels of Analysis

The role of the anterior cingulate cortex (ACC) in cognition has been extensively investigated with several techniques, including single-unit recordings in rodents and monkeys and EEG and fMRI in humans. This has generated a rich set of data and points of view. Important theoretical functions proposed for ACC are value estimation, error detection, error-likelihood estimation, conflict monitoring, and estimation of reward volatility. A unified view is lacking at this time, however. Here we propose that online value estimation could be the key function underlying these diverse data. This is instantiated in the reward value and prediction model (RVPM). The model contains units coding for the value of cues (stimuli or actions) and units coding for the differences between such values and the actual reward (prediction errors). We exposed the model to typical experimental paradigms from single-unit, EEG, and fMRI research to compare its overall behavior with the data from these studies. The model reproduced the ACC behavior of previous single-unit, EEG, and fMRI studies on reward processing, error processing, conflict monitoring, error-likelihood estimation, and volatility estimation, unifying the interpretations of the role performed by the ACC in some aspects of cognition.

From conflict management to reward-based decision making: Actors and critics in primate medial frontal cortex

The role of the medial prefrontal cortex (mPFC) and especially the anterior cingulate cortex has been the subject of intense debate for the last decade. A number of theories have been proposed to account for its function. Broadly speaking, some emphasize cognitive control, whereas others emphasize value processing; specific theories concern reward processing, conflict detection, error monitoring, and volatility detection, among others. Here we survey and evaluate them relative to experimental results from neurophysiological, anatomical, and cognitive studies. We argue for a new conceptualization of mPFC, arising from recent computational modeling work. Based on reinforcement learning theory, these new models propose that mPFC is an Actor-Critic system. This system is aimed to predict future events including rewards, to evaluate errors in those predictions, and finally, to implement optimal skeletal-motor and visceromotor commands to obtain reward. This framework provides a comprehensive account of mPFC function, accounting for and predicting empirical results across different levels of analysis, including monkey neurophysiology, human ERP, human neuroimaging, and human behavior.

Overlapping Neural Systems Represent Cognitive Effort and Reward Anticipation

Anticipating a potential benefit and how difficult it will be to obtain it are valuable skills in a constantly changing environment. In the human brain, the anticipation of reward is encoded by the Anterior Cingulate Cortex (ACC) and Striatum. Naturally, potential rewards have an incentive quality, resulting in a motivational effect improving performance. Recently it has been proposed that an upcoming task requiring effort induces a similar anticipation mechanism as reward, relying on the same cortico-limbic network. However, this overlapping anticipatory activity for reward and effort has only been investigated in a perceptual task. Whether this generalizes to high-level cognitive tasks remains to be investigated. To this end, an fMRI experiment was designed to investigate anticipation of reward and effort in cognitive tasks. A mental arithmetic task was implemented, manipulating effort (difficulty), reward, and delay in reward delivery to control for temporal confounds. The goal was to test for the motivational effect induced by the expectation of bigger reward and higher effort. The results showed that the activation elicited by an upcoming difficult task overlapped with higher reward prospect in the ACC and in the striatum, thus highlighting a pivotal role of this circuit in sustaining motivated behavior.

Adaptive effort investment in cognitive and physical tasks: a neurocomputational model

Despite its importance in everyday life, the computational nature of effort investment remains poorly understood. We propose an effort model obtained from optimality considerations, and a neurocomputational approximation to the optimal model. Both are couched in the framework of reinforcement learning. It is shown that choosing when or when not to exert effort can be adaptively learned, depending on rewards, costs, and task difficulty. In the neurocomputational model, the limbic loop comprising anterior cingulate cortex (ACC) and ventral striatum in the basal ganglia allocates effort to cortical stimulus-action pathways whenever this is valuable. We demonstrate that the model approximates optimality. Next, we consider two hallmark effects from the cognitive control literature, namely proportion congruency and sequential congruency effects. It is shown that the model exerts both proactive and reactive cognitive control. Then, we simulate two physical effort tasks. In line with empirical work, impairing the models dopaminergic pathway leads to apathetic behavior. Thus, we conceptually unify the exertion of cognitive and physical effort, studied across a variety of literatures (e.g., motivation and cognitive control) and animal species.

Value and prediction error estimation account for volatility effects in ACC: A model-based fMRI study

In order to choose the best action for maximizing fitness, mammals can estimate the reward expectations (value) linked to available actions based on past environmental outcomes. Value updates are performed by comparing the current value with the actual environmental outcomes (prediction error). The anterior cingulate cortex (ACC) has been shown to be critically involved in the computation of value and its variability across time (volatility). Previously, we proposed a new neural model of the ACC based on single-unit ACC neurophysiology, the Reward Value and Prediction Model (RVPM). Here, using the RVPM in computer simulations and in a model-based fMRI study, we found that highly uncertain but non-volatile environments activate ACC more than volatile environments, demonstrating that value estimation by means of prediction error computation can account for the effect of volatility in ACC. These findings suggest that ACC response to volatility can be parsimoniously explained by basic ACC reward processing.

Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice

Acting in an uncertain environment requires estimating the probability and the value of potential outcomes. These computations are typically ascribed to various parts of the medial prefrontal cortex (mPFC), but the functional architecture of this region remains debated. The anterior cingulate cortex (ACC) encodes reward prediction and outcome (i.e. win vs lose, Silvetti, Seurinck, & Verguts, 2013. Cortex, 49(6), 1627-35. doi:10.1016/j.cortex.2012.05.008). An outcome-related value signal has also been reported in the ventromedial Prefrontal Cortex (vmPFC, Rangel & Hare, 2010. Current Opinion in Neurobiology, 20(2), 262-70. doi:10.1016/j.conb.2010.03.001). Whether a functional dissociation can be traced in these regions with respect to reward prediction and outcome has been suggested but not rigorously tested. Hence an fMRI study was designed to systematically examine the contribution of ACC and vmPFC to reward prediction and outcome. A striking dissociation was identified, with ACC coding for positive prediction errors and vmPFC responding to outcome, irrespective of probability. Moreover, ACC has been assigned a crucial role in the selection of intentional actions (decision-making) and computing the value associated to these actions (action-based value). Conversely, vmPFC seems to implement stimulus-based value processing (Rudebeck et al., 2008. Journal of Neuroscience, 28(51), 13775-85. doi:10.1523/JNEUROSCI.3541-08.2008; Rushworth, Behrens, Rudebeck, & Walton, 2007. Trends in Cognitive Sciences, 11(4), 168-76. doi:10.1016/j.tics.2007.01.004). Therefore, a decision-making factor (choice vs. no choice condition) was also implemented in the present paradigm to distinguish stimulus-based versus action-based value coding in the mPFC during both decision and outcome phase. We found that vmPFC was more activated during the outcome phase in the no-choice than in the choice condition, potentially confirming the role of this area in stimulus-based (more than action-based) value processing.

The influence of the noradrenergic system on optimal control of neural plasticity

Decision making under uncertainty is challenging for any autonomous agent. The challenge increases when the environment’s stochastic properties change over time, i.e., when the environment is volatile. In order to efficiently adapt to volatile environments, agents must primarily rely on recent outcomes to quickly change their decision strategies; in other words, they need to increase their knowledge plasticity. On the contrary, in stable environments, knowledge stability must be preferred to preserve useful information against noise. Here we propose that in mammalian brain, the locus coeruleus (LC) is one of the nuclei involved in volatility estimation and in the subsequent control of neural plasticity. During a reinforcement learning task, LC activation, measured by means of pupil diameter, coded both for environmental volatility and learning rate. We hypothesize that LC could be responsible, through norepinephrinic modulation, for adaptations to optimize decision making in volatile environments. We also suggest a computational model on the interaction between the anterior cingulate cortex (ACC) and LC for volatility estimation.

Reward expectation and prediction error in human medial frontal cortex: an EEG study.

The mammalian medial frontal cortex (MFC) is involved in reward-based decision making. In particular, in nonhuman primates this area constructs expectations about upcoming rewards, given an environmental state or a choice planned by the animal. At the same time, in both humans and nonhuman primates, the MFC computes the difference between such predictions and actual environmental outcomes (reward prediction errors). However, there is a paucity of evidence about the time course of MFC-related activity during reward prediction and prediction error in humans. Here we experimentally investigated this by recording the EEG during a reinforcement learning task. Our results support the hypothesis that human MFC codes for reward prediction during the cue period and for prediction error during the outcome period. Further, reward expectation (cue period) was positively correlated with prediction error (outcome period) in error trials but negatively in correct trials, consistent with updating of reward expectation by prediction error. This demonstrates in humans, like in nonhuman primates, a role of the MFC in the rapid updating of reward expectations through prediction errors.

Deficient reinforcement learning in medial frontal cortex as a model of dopamine-related motivational deficits in ADHD

Attention Deficit/Hyperactivity Disorder (ADHD) is a pathophysiologically complex and heterogeneous condition with both cognitive and motivational components. We propose a novel computational hypothesis of motivational deficits in ADHD, drawing together recent evidence on the role of anterior cingulate cortex (ACC) and associated mesolimbic dopamine circuits in both reinforcement learning and ADHD. Based on findings of dopamine dysregulation and ACC involvement in ADHD we simulated a lesion in a previously validated computational model of ACC (Reward Value and Prediction Model, RVPM). We explored the effects of the lesion on the processing of reinforcement signals. We tested specific behavioral predictions about the profile of reinforcement-related deficits in ADHD in three experimental contexts; probability tracking task, partial and continuous reward schedules, and immediate versus delayed rewards. In addition, predictions were made at the neurophysiological level. Behavioral and neurophysiological predictions from the RVPM-based lesion-model of motivational dysfunction in ADHD were confirmed by data from previously published studies. RVPM represents a promising model of ADHD reinforcement learning suggesting that ACC dysregulation might play a role in the pathogenesis of motivational deficits in ADHD. However, more behavioral and neurophysiological studies are required to test core predictions of the model. In addition, the interaction with different brain networks underpinning other aspects of ADHD neuropathology (i.e., executive function) needs to be better understood.