Andrea Reiter

The interaction of acute and chronic stress impairs model-based behavioral control

It is suggested that acute stress shifts behavioral control from goal-directed, model-based toward habitual, model-free strategies. Recent findings indicate that interindividual differences in the cortisol stress response influence model-based decision-making. Although not yet investigated in humans, animal studies show that chronic stress also shifts decision-making toward more habitual behavior. Here, we ask whether acute stress and individual vulnerability factors, such as stress reactivity and previous exposure to stressful life events, impact the balance between model-free and model-based control systems. To test this, 39 male participants (21-30 years old) were exposed to a potent psychosocial stressor (Trier Social Stress Test) and a control condition in a within-subjects design before they performed a sequential decision-making task which evaluates the balance between the two systems. Physiological and subjective stress reactivity was assessed before, during, and after acute stress exposure. By means of computational modeling, we demonstrate that interindividual variability in stress reactivity predicts impairments in model-based decision-making. Whereas acute psychosocial stress did not alter model-based behavioral control, we found chronic and acute stress to interact in their detrimental effect on decision-making: subjects with high but not low chronic stress levels as indicated by stressful life events exhibited reduced model-based control in response to acute psychosocial stress. These findings emphasize that stress reactivity and chronic stress play an important role in mediating the relationship between stress and decision-making. Our results might stimulate new insights into the interplay between chronic and acute stress, attenuated model-based control, and the pathogenesis of various psychiatric diseases.

Lateral prefrontal model-based signatures are reduced in healthy individuals with high trait impulsivity.

High impulsivity is an important risk factor for addiction with evidence from endophenotype studies. In addiction, behavioral control is shifted toward the habitual end. Habitual control can be described by retrospective updating of reward expectations in ‘model-free’ temporal-difference algorithms. Goal-directed control relies on the prospective consideration of actions and their outcomes, which can be captured by forward-planning ‘model-based’ algorithms. So far, no studies have examined behavioral and neural signatures of model-free and model-based control in healthy high-impulsive individuals. Fifty healthy participants were drawn from the upper and lower ends of 452 individuals, completing the Barratt Impulsiveness Scale. All participants performed a sequential decision-making task during functional magnetic resonance imaging (fMRI) and underwent structural MRI. Behavioral and fMRI data were analyzed by means of computational algorithms reflecting model-free and model-based control. Both groups did not differ regarding the balance of model-free and model-based control, but high-impulsive individuals showed a subtle but significant accentuation of model-free control alone. Right lateral prefrontal model-based signatures were reduced in high-impulsive individuals. Effects of smoking, drinking, general cognition or gray matter density did not account for the findings. Irrespectively of impulsivity, gray matter density in the left dorsolateral prefrontal cortex was positively associated with model-based control. The present study supports the idea that high levels of impulsivity are accompanied by behavioral and neural signatures in favor of model-free behavioral control. Behavioral results in healthy high-impulsive individuals were qualitatively different to findings in patients with the same task. The predictive relevance of these results remains an important target for future longitudinal studies.

State- and trait-greed, its impact on risky decision-making and underlying neural mechanisms

We investigated whether greed would predict risky decision-making and recorded neural responses during a monetary gambling task using the electroencephalogram. We found that individuals high in trait-greed took higher risks to maximize monetary outcome. Furthermore, this relation was moderated by state-greed; specifically, trait-greed had a stronger impact on risky decision-making when activated by situational characteristics. On the neural level, greedy individuals showed a specific response to favorable and unfavorable outcomes. Specifically, they had a reduced feedback-related negativity-difference score to these events, indicating that they might have difficulty in learning from experience, especially from mistakes and negative feedback. It is concluded that greed may explain risky and reckless behavior in diverse settings, such as investment banking, and may account for phenomena such as stock market bubbles.

Impaired Flexible Reward-Based Decision-Making in Binge Eating Disorder: Evidence from Computational Modeling and Functional Neuroimaging

Despite its clinical relevance and the recent recognition as a diagnostic category in the DSM-5, binge eating disorder (BED) has rarely been investigated from a cognitive neuroscientific perspective targeting a more precise neurocognitive profiling of the disorder. BED patients suffer from a lack of behavioral control during recurrent binge eating episodes and thus fail to adapt their behavior in the face of negative consequences, eg, high risk for obesity. To examine impairments in flexible reward-based decision-making, we exposed BED patients (n=22) and matched healthy individuals (n=22) to a reward-guided decision-making task during functional resonance imaging (fMRI). Performing fMRI analysis informed via computational modeling of choice behavior, we were able to identify specific signatures of altered decision-making in BED. On the behavioral level, we observed impaired behavioral adaptation in BED, which was due to enhanced switching behavior, a putative deficit in striking a balance between exploration and exploitation appropriately. This was accompanied by diminished activation related to exploratory decisions in the anterior insula/ventro-lateral prefrontal cortex. Moreover, although so-called model-free reward prediction errors remained intact, representation of ventro–medial prefrontal learning signatures, incorporating inference on unchosen options, was reduced in BED, which was associated with successful decision-making in the task. On the basis of a computational psychiatry account, the presented findings contribute to defining a neurocognitive phenotype of BED.

Model-based control in dimensional psychiatry

We use parallel interacting goal-directed and habitual strategies to make our daily decisions. The arbitration between these strategies is relevant to inflexible repetitive behaviors in psychiatric disorders. Goal-directed control, also known as model-based control, is based on an affective outcome relying on a learned internal model to prospectively make decisions. In contrast, habit control, also known as model-free control, is based on an integration of previous reinforced learning autonomous of the current outcome value and is implicit and more efficient but at the cost of greater inflexibility. The concept of model-based control can be further extended into pavlovian processes. Here we describe and compare tasks that tap into these constructs and emphasize the clinical relevance and translation of these tasks in psychiatric disorders. Together, these findings highlight a role for model-based control as a transdiagnostic impairment underlying compulsive behaviors and representing a promising therapeutic target.

The feedback-related negativity codes components of abstract inference during reward-based decision-making

Behavioral control is influenced not only by learning from the choices made and the rewards obtained but also by “what might have happened,” that is, inference about unchosen options and their fictive outcomes. Substantial progress has been made in understanding the neural signatures of direct learning from choices that are actually made and their associated rewards via reward prediction errors (RPEs). However, electrophysiological correlates of abstract inference in decision-making are less clear. One seminal theory suggests that the so-called feedback-related negativity (FRN), an ERP peaking 200–300 msec after a feedback stimulus at frontocentral sites of the scalp, codes RPEs. Hitherto, the FRN has been predominantly related to a so-called “model-free” RPE: The difference between the observed outcome and what had been expected. Here, by means of computational modeling of choice behavior, we show that individuals employ abstract, “double-update” inference on the task structure by concurrently tracking values of chosen stimuli (associated with observed outcomes) and unchosen stimuli (linked to fictive outcomes). In a parametric analysis, model-free RPEs as well as their modification because of abstract inference were regressed against single-trial FRN amplitudes. We demonstrate that components related to abstract inference uniquely explain variance in the FRN beyond model-free RPEs. These findings advance our understanding of the FRN and its role in behavioral adaptation. This might further the investigation of disturbed abstract inference, as proposed, for example, for psychiatric disorders, and its underlying neural correlates.

Risk factors for addiction and their association with model-based behavioral control

Addiction shows familial aggregation and previous endophenotype research suggests that healthy relatives of addicted individuals share altered behavioral and cognitive characteristics with individuals suffering from addiction. In this study we asked whether impairments in behavioral control proposed for addiction, namely a shift from goal-directed, model-based toward habitual, model-free control, extends toward an unaffected sample (n = 20) of adult children of alcohol-dependent fathers as compared to a sample without any personal or family history of alcohol addiction (n = 17). Using a sequential decision-making task designed to investigate model-free and model-based control combined with a computational modeling analysis, we did not find any evidence for altered behavioral control in individuals with a positive family history of alcohol addiction. Independent of family history of alcohol dependence, we however observed that the interaction of two different risk factors of addiction, namely impulsivity and cognitive capacities, predicts the balance of model-free and model-based behavioral control. Post-hoc tests showed a positive association of model-based behavior with cognitive capacity in the lower, but not in the higher impulsive group of the original sample. In an independent sample of particularly high- vs. low-impulsive individuals, we confirmed the interaction effect of cognitive capacities and high vs. low impulsivity on model-based control. In the confirmation sample, a positive association of omega with cognitive capacity was observed in highly impulsive individuals, but not in low impulsive individuals. Due to the moderate sample size of the study, further investigation of the association of risk factors for addiction with model-based behavior in larger sample sizes is warranted.

The Aging of the Social Mind - Differential Effects on Components of Social Understanding

Research in younger adults dissociates cognitive from affective facets of social information processing, rather than promoting a monolithic view of social intelligence. An influential theory on adult development suggests differential effects of aging on cognitive and affective functions. However, this dissociation has not been directly tested in the social domain. Employing a newly developed naturalistic paradigm that disentangles facets of the social mind within an individual, we show multi-directionality of age-related differences. Specifically, components of the socio-cognitive route – Theory of Mind and metacognition – are impaired in older relative to younger adults. Nevertheless, these social capacities are still less affected by aging than factual reasoning and metacognition regarding non-social content. Importantly, the socio-affective route is well-functioning, with no decline in empathy and elevated compassion in the elderly. These findings contribute to an integrated theory of age-related change in social functioning and inform interventions tailored to specifically reinstate socio-cognitive skills in old age.

Altered Medial Frontal Feedback Learning Signals in Anorexia Nervosa

BACKGROUND In their relentless pursuit of thinness, individuals with anorexia nervosa (AN) engage in maladaptive behaviors (restrictive food choices and overexercising) that may originate in altered decision making and learning. METHODS In this functional magnetic resonance imaging study, we employed computational modeling to elucidate the neural correlates of feedback learning and value-based decision making in 36 female patients with AN and 36 age-matched healthy volunteers (12-24 years). Participants performed a decision task that required adaptation to changing reward contingencies. Data were analyzed within a hierarchical Gaussian filter model that captures interindividual variability in learning under uncertainty. RESULTS Behaviorally, patients displayed an increased learning rate specifically after punishments. At the neural level, hemodynamic correlates for the learning rate, expected value, and prediction error did not differ between the groups. However, activity in the posterior medial frontal cortex was elevated in AN following punishment. CONCLUSIONS Our findings suggest that the neural underpinning of feedback learning is selectively altered for punishment in AN.

Linking social context and addiction neuroscience: a computational psychiatry approach

In their recent article (Time to connect: bringing social context into addiction neuroscience (Nat. Rev. Neurosci. 17, 592–599 (2016))1, Heilig et al. suggest that the lack of progress towards treatment and prevention of addiction is partly due to a neglect of social factors in neuroscientific research of addiction. We share the authors’ disappointment but argue that merely broadening the focus towards social context is not sufficient to close the “large gap [that] exists between the promise of neuroscientific approaches to addiction and what they have delivered”. To date, we lack sufficient transfer of powerful theoretical accounts and methodological resources to understand the multiple facets of addiction. Thus, adding yet another obviously important component to the heterogeneity that characterizes addiction might be insufficient to make addiction neuroscience more clinically relevant. Rather, a conceptual shift in ‘addiction neuroscience’ may be warranted. Seminal epidemiological and neuroimaging studies have established associations between neurobiological measures, addiction and social status2–4, thereby generating hypotheses on the physiological, psychological and social components of addiction. In our view, the next step should be to determine how different neural computations in modulatory circuits give rise to certain psychological phenomena of learning and decision making that are associated with addictive behaviours, such as cue-induced drug craving and habitual drug intake despite negative consequences. To integrate these interacting physiological, behavioural and social levels of description5, a generative model of addictive behaviours is needed. To achieve this goal, we propose that coherent theoretical accounts be formalized and applied in stringent and translational experimental design. A developmental perspective is necessary to track the onset, maintenance and relapse of addiction in longitudinal studies. Computational models might prove valuable for any of these processes. Computational models aid theory building by formalizing hypotheses and rigorously capture relations between latent factors and observations. They can inform on particularly meaningful manipulations and enable mechanistic interpretation of experimental results. Indeed, addictive behaviours and social cognition can be tied together through existing theoretical accounts of addictive behaviours — for example, those based on reinforcement learning6 and the ‘Bayesian brain’ hypothesis7. Computational modelling has begun to inform empirical studies on decision making in addiction8,9, putatively addiction-like disorders9,10 and risk factors for addiction11–13. Likewise, computational accounts of social cognition have been tested in neuroscience. Research combining computational modelling of social behaviours with neuroimaging suggests that social information may be processed by mechanisms similar to those involved in (non-social) reward-based learning and decision making14. Thus, establishing a mechanistic theory of addictive behaviours should advance empirical knowledge of whether (or not) social adversities are indeed specific factors that contribute to certain aspects of addictive behaviours. Ultimately, this might allow one to define patient-specific combinations of various model parameters and model evidences for alternative (social and non-social) disease mechanisms15. Crucially, such quantifiable ‘computational fingerprints’ have to be examined rigorously regarding their predictive power in longitudinal studies that include children or teenagers before the onset of addiction. This might prompt targeted intervention and prevention — be it in the social or the nonsocial domain — informed by the mechanisms that give rise to addiction.