Eliana Vassena

Overlapping Neural Systems Represent Cognitive Effort and Reward Anticipation

Anticipating a potential benefit and how difficult it will be to obtain it are valuable skills in a constantly changing environment. In the human brain, the anticipation of reward is encoded by the Anterior Cingulate Cortex (ACC) and Striatum. Naturally, potential rewards have an incentive quality, resulting in a motivational effect improving performance. Recently it has been proposed that an upcoming task requiring effort induces a similar anticipation mechanism as reward, relying on the same cortico-limbic network. However, this overlapping anticipatory activity for reward and effort has only been investigated in a perceptual task. Whether this generalizes to high-level cognitive tasks remains to be investigated. To this end, an fMRI experiment was designed to investigate anticipation of reward and effort in cognitive tasks. A mental arithmetic task was implemented, manipulating effort (difficulty), reward, and delay in reward delivery to control for temporal confounds. The goal was to test for the motivational effect induced by the expectation of bigger reward and higher effort. The results showed that the activation elicited by an upcoming difficult task overlapped with higher reward prospect in the ACC and in the striatum, thus highlighting a pivotal role of this circuit in sustaining motivated behavior.

Adaptive effort investment in cognitive and physical tasks: a neurocomputational model

Despite its importance in everyday life, the computational nature of effort investment remains poorly understood. We propose an effort model obtained from optimality considerations, and a neurocomputational approximation to the optimal model. Both are couched in the framework of reinforcement learning. It is shown that choosing when or when not to exert effort can be adaptively learned, depending on rewards, costs, and task difficulty. In the neurocomputational model, the limbic loop comprising anterior cingulate cortex (ACC) and ventral striatum in the basal ganglia allocates effort to cortical stimulus-action pathways whenever this is valuable. We demonstrate that the model approximates optimality. Next, we consider two hallmark effects from the cognitive control literature, namely proportion congruency and sequential congruency effects. It is shown that the model exerts both proactive and reactive cognitive control. Then, we simulate two physical effort tasks. In line with empirical work, impairing the models dopaminergic pathway leads to apathetic behavior. Thus, we conceptually unify the exertion of cognitive and physical effort, studied across a variety of literatures (e.g., motivation and cognitive control) and animal species.

Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice

Acting in an uncertain environment requires estimating the probability and the value of potential outcomes. These computations are typically ascribed to various parts of the medial prefrontal cortex (mPFC), but the functional architecture of this region remains debated. The anterior cingulate cortex (ACC) encodes reward prediction and outcome (i.e. win vs lose, Silvetti, Seurinck, & Verguts, 2013. Cortex, 49(6), 1627-35. doi:10.1016/j.cortex.2012.05.008). An outcome-related value signal has also been reported in the ventromedial Prefrontal Cortex (vmPFC, Rangel & Hare, 2010. Current Opinion in Neurobiology, 20(2), 262-70. doi:10.1016/j.conb.2010.03.001). Whether a functional dissociation can be traced in these regions with respect to reward prediction and outcome has been suggested but not rigorously tested. Hence an fMRI study was designed to systematically examine the contribution of ACC and vmPFC to reward prediction and outcome. A striking dissociation was identified, with ACC coding for positive prediction errors and vmPFC responding to outcome, irrespective of probability. Moreover, ACC has been assigned a crucial role in the selection of intentional actions (decision-making) and computing the value associated to these actions (action-based value). Conversely, vmPFC seems to implement stimulus-based value processing (Rudebeck et al., 2008. Journal of Neuroscience, 28(51), 13775-85. doi:10.1523/JNEUROSCI.3541-08.2008; Rushworth, Behrens, Rudebeck, & Walton, 2007. Trends in Cognitive Sciences, 11(4), 168-76. doi:10.1016/j.tics.2007.01.004). Therefore, a decision-making factor (choice vs. no choice condition) was also implemented in the present paradigm to distinguish stimulus-based versus action-based value coding in the mPFC during both decision and outcome phase. We found that vmPFC was more activated during the outcome phase in the no-choice than in the choice condition, potentially confirming the role of this area in stimulus-based (more than action-based) value processing.

Anticipatory processes in brain state switching — Evidence from a novel cued-switching task implicating default mode and salience networks

The default mode network (DMN) is the core brain system supporting internally oriented cognition. The ability to attenuate the DMN when switching to externally oriented processing is a prerequisite for effective performance and adaptive self-regulation. Right anterior insula (rAI), a core hub of the salience network (SN), has been proposed to control the switching from DMN to task-relevant brain networks. Little is currently known about the extent of anticipatory processes subserved by DMN and SN during switching. We investigated anticipatory DMN and SN modulation using a novel cued-switching task of between-state (rest-to-task/task-to-rest) and within-state (task-to-task) transitions. Twenty healthy adults performed the task implemented in an event-related functional magnetic resonance imaging (fMRI) design. Increases in activity were observed in the DMN regions in response to cues signalling upcoming rest. DMN attenuation was observed for rest-to-task switch cues. Obversely, DMN was up-regulated by task-to-rest cues. The strongest rAI response was observed to rest-to-task switch cues. Task-to-task switch cues elicited smaller rAI activation, whereas no significant rAI activation occurred for task-to-rest switches. Our data provide the first evidence that DMN modulation occurs rapidly and can be elicited by short duration cues signalling rest- and task-related state switches. The role of rAI appears to be limited to certain switch types - those implicating transition from a resting state and to tasks involving active cognitive engagement.

Computational Models of Anterior Cingulate Cortex: At the Crossroads between Prediction and Effort

In the last two decades the anterior cingulate cortex (ACC) has become one of the most investigated areas of the brain. Extensive neuroimaging evidence suggests countless functions for this region, ranging from conflict and error coding, to social cognition, pain and effortful control. In response to this burgeoning amount of data, a proliferation of computational models has tried to characterize the neurocognitive architecture of ACC. Early seminal models provided a computational explanation for a relatively circumscribed set of empirical findings, mainly accounting for EEG and fMRI evidence. More recent models have focused on ACCs contribution to effortful control. In parallel to these developments, several proposals attempted to explain within a single computational framework a wider variety of empirical findings that span different cognitive processes and experimental modalities. Here we critically evaluate these modeling attempts, highlighting the continued need to reconcile the array of disparate ACC observations within a coherent, unifying framework.

Predicting Motivation: Computational Models of PFC Can Explain Neural Coding of Motivation and Effort-based Decision-making in Health and Disease

Human behavior is strongly driven by the pursuit of rewards. In daily life, however, benefits mostly come at a cost, often requiring that effort be exerted to obtain potential benefits. Medial PFC (MPFC) and dorsolateral PFC (DLPFC) are frequently implicated in the expectation of effortful control, showing increased activity as a function of predicted task difficulty. Such activity partially overlaps with expectation of reward and has been observed both during decision-making and during task preparation. Recently, novel computational frameworks have been developed to explain activity in these regions during cognitive control, based on the principle of prediction and prediction error (predicted response–outcome [PRO] model [Alexander, W. H., & Brown, J. W. Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14, 1338–1344, 2011], hierarchical error representation [HER] model [Alexander, W. H., & Brown, J. W. Hierarchical error representation: A computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Computation, 27, 2354–2410, 2015]). Despite the broad explanatory power of these models, it is not clear whether they can also accommodate effects related to the expectation of effort observed in MPFC and DLPFC. Here, we propose a translation of these computational frameworks to the domain of effort-based behavior. First, we discuss how the PRO model, based on prediction error, can explain effort-related activity in MPFC, by reframing effort-based behavior in a predictive context. We propose that MPFC activity reflects monitoring of motivationally relevant variables (such as effort and reward), by coding expectations and discrepancies from such expectations. Moreover, we derive behavioral and neural model-based predictions for healthy controls and clinical populations with impairments of motivation. Second, we illustrate the possible translation to effort-based behavior of the HER model, an extended version of PRO model based on hierarchical error prediction, developed to explain MPFC–DLPFC interactions. We derive behavioral predictions that describe how effort and reward information is coded in PFC and how changing the configuration of such environmental information might affect decision-making and task performance involving motivation.

Unsigned value prediction-error modulates the motor system in absence of choice.

Human actions are driven by the pursuit of goals, especially when achieving these goals entails a reward. Accordingly, recent work showed that anticipating a reward in a motor task influences the motor system, boosting motor excitability and increasing overall readiness. Attaining a reward typically requires some mental or physical effort. Recent neuroimaging evidence suggested that both reward expectation and effort requirements are encoded by a partially overlapping brain network. Moreover, reward and effort information are combined in an integrative value signal. However, whether and how mental effort is integrated with reward at the motor level during task preparation remains unclear. To address these issues, we implemented a mental effort task where reward expectation and effort requirements were manipulated. During task preparation, TMS was delivered on the motor cortex and motor-evoked potentials (MEPs) were recorded on the right hand muscles to probe motor excitability. The results showed an interaction of effort and reward in modulating the motor system, reflecting an unsigned value prediction-error signal. Crucially, this was observed in the motor system in absence of a value-based decision or value-driven action selection. This suggests a high-level cognitive factor such as unsigned value prediction-error can modulate the motor system. Interestingly, effort-related motor excitability was also modulated by individual differences in tendency to engage in (and enjoy) mental effort, as measured by the Need for Cognition questionnaire, underlining a role of subjective effort experience in value-driven preparation for action.

Integrative Modeling of Prefrontal Cortex

pFC is generally regarded as a region critical for abstract reasoning and high-level cognitive behaviors. As such, it has become the focus of intense research involving a wide variety of subdisciplines of neuroscience and employing a diverse range of methods. However, even as the amount of data on pFC has increased exponentially, it appears that progress toward understanding the general function of the region across a broad array of contexts has not kept pace. Effects observed in pFC are legion, and their interpretations are generally informed by a particular perspective or methodology with little regard with how those effects may apply more broadly. Consequently, the number of specific roles and functions that have been identified makes the region a very crowded place indeed and one that appears unlikely to be explained by a single general principle. In this theoretical article, we describe how the function of large portions of pFC can be accommodated by a single explanatory framework based on the computation and manipulation of error signals and how this framework may be extended to account for additional parts of pFC.

Unimodal and cross-modal prediction is enhanced in musicians.

Musical training involves exposure to complex auditory and visual stimuli, memorization of elaborate sequences, and extensive motor rehearsal. It has been hypothesized that such multifaceted training may be associated with differences in basic cognitive functions, such as prediction, potentially translating to a facilitation in expert musicians. Moreover, such differences might generalize to non-auditory stimuli. This study was designed to test both hypotheses. We implemented a cross-modal attentional cueing task with auditory and visual stimuli, where a target was preceded by compatible or incompatible cues in mainly compatible (80% compatible, predictable) or random blocks (50% compatible, unpredictable). This allowed for the testing of prediction skills in musicians and controls. Musicians showed increased sensitivity to the statistical structure of the block, expressed as advantage for compatible trials (disadvantage for incompatible trials), but only in the mainly compatible (predictable) blocks. Controls did not show this pattern. The effect held within modalities (auditory, visual), across modalities, and when controlling for short-term memory capacity. These results reveal a striking enhancement in cross-modal prediction in musicians in a very basic cognitive task.

Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner

Optimal decision-making is based on integrating information from several dimensions of decisional space (e.g., reward expectation, cost estimation, effort exertion). Despite considerable empirical and theoretical efforts, the computational and neural bases of such multidimensional integration have remained largely elusive. Here we propose that the current theoretical stalemate may be broken by considering the computational properties of a cortical-subcortical circuit involving the dorsal anterior cingulate cortex (dACC) and the brainstem neuromodulatory nuclei: ventral tegmental area (VTA) and locus coeruleus (LC). From this perspective, the dACC optimizes decisions about stimuli and actions, and using the same computational machinery, it also modulates cortical functions (meta-learning), via neuromodulatory control (VTA and LC). We implemented this theory in a novel neuro-computational model–the Reinforcement Meta Learner (RML). We outline how the RML captures critical empirical findings from an unprecedented range of theoretical domains, and parsimoniously integrates various previous proposals on dACC functioning.