Ben Eppinger

Neuromodulation of reward-based learning and decision making in human aging.

In this paper, we review the current literature to highlight relations between age‐associated declines in dopaminergic and serotonergic neuromodulation and adult age differences in adaptive goal‐directed behavior. Specifically, we focus on evidence suggesting that deficits in neuromodulation contribute to older adults’ behavioral disadvantages in learning and decision making. These deficits are particularly pronounced when reward information is uncertain or the task context requires flexible adaptations to changing stimulus–reward contingencies. Moreover, emerging evidence points to age‐related differences in the sensitivity to rewarding and aversive outcomes during learning and decision making if the acquisition of behavior critically depends on outcome processing. These age‐related asymmetries in outcome valuation may be explained by age differences in the interplay of dopaminergic and serotonergic neuromodulation. This hypothesis is based on recent neurocomputational and psychopharmacological approaches, which suggest that dopamine and serotonin serve opponent roles in regulating the balance between approach behavior and inhibitory control. Studying adaptive regulation of behavior across the adult life span may shed new light on how the aging brain changes functionally in response to its diminishing resources.

Reduced Striatal Responses to Reward Prediction Errors in Older Compared with Younger Adults

We examined whether older adults differ from younger adults in how they learn from rewarding and aversive outcomes. Human participants were asked to either learn to choose actions that lead to monetary reward or learn to avoid actions that lead to monetary losses. To examine age differences in the neurophysiological mechanisms of learning, we applied a combination of computational modeling and fMRI. Behavioral results showed age-related impairments in learning from reward but not in learning from monetary losses. Consistent with these results, we observed age-related reductions in BOLD activity during learning from reward in the ventromedial PFC. Furthermore, the model-based fMRI analysis revealed a reduced responsivity of the ventral striatum to reward prediction errors during learning in older than younger adults. This age-related reduction in striatal sensitivity to reward prediction errors may result from a decline in phasic dopaminergic learning signals in the elderly.

Reduced Sensitivity to Immediate Reward during Decision-Making in Older than Younger Adults

We examined whether older adults differ from younger adults in the degree to which they favor immediate over delayed rewards during decision-making. To examine the neural correlates of age-related differences in delay discounting we acquired functional MR images while participants made decisions between smaller but sooner and larger but later monetary rewards. The behavioral results show age-related reductions in delay discounting. Less impulsive decision-making in older adults was associated with lower ventral striatal activations to immediate reward. Furthermore, older adults showed an overall higher percentage of delayed choices and reduced activity in the dorsal striatum than younger adults. This points to a reduced reward sensitivity of the dorsal striatum in older adults. Taken together, our findings indicate that less impulsive decision-making in older adults is due to a reduced sensitivity of striatal areas to reward. These age-related changes in reward sensitivity may result from transformations in dopaminergic neuromodulation with age.

Of goals and habits: age-related and individual differences in goal-directed decision-making

In this study we investigated age-related and individual differences in habitual (model-free) and goal-directed (model-based) decision-making. Specifically, we were interested in three questions. First, does age affect the balance between model-based and model-free decision mechanisms? Second, are these age-related changes due to age differences in working memory (WM) capacity? Third, can model-based behavior be affected by manipulating the distinctiveness of the reward value of choice options? To answer these questions we used a two-stage Markov decision task in in combination with computational modeling to dissociate model-based and model-free decision mechanisms. To affect model-based behavior in this task we manipulated the distinctiveness of reward probabilities of choice options. The results show age-related deficits in model-based decision-making, which are particularly pronounced if unexpected reward indicates the need for a shift in decision strategy. In this situation younger adults explore the task structure, whereas older adults show perseverative behavior. Consistent with previous findings, these results indicate that older adults have deficits in the representation and updating of expected reward value. We also observed substantial individual differences in model-based behavior. In younger adults high WM capacity is associated with greater model-based behavior and this effect is further elevated when reward probabilities are more distinct. However, in older adults we found no effect of WM capacity. Moreover, age differences in model-based behavior remained statistically significant, even after controlling for WM capacity. Thus, factors other than decline in WM, such as deficits in the in the integration of expected reward value into strategic decisions may contribute to the observed impairments in model-based behavior in older adults.

Dopaminergic and Prefrontal Contributions to Reward-Based Learning and Outcome Monitoring during Child Development and Aging.

In many instances, children and older adults show similar difficulties in reward-based learning and outcome monitoring. These impairments are most pronounced in situations in which reward is uncertain (e.g., probabilistic reward schedules) and if outcome information is ambiguous (e.g., the relative value of outcomes has to be learned). Furthermore, whereas children show a greater sensitivity to external outcome information, older adults focus less on a rapid differentiation of rewarding outcomes. In this article, we review evidence for the idea that these phenomenologically similar impairments in learning and outcome monitoring in children and older adults can be attributed to deficits in different underlying neurophysiological mechanisms. We propose that in older adults learning impairments are the result of reduced dopaminergic projections to the ventromedial prefrontal cortex, which lead to less differentiated representations of reward value. In contrast, in children, impairments in learning can be primarily attributed to deficits in executive control, which may be due to a protracted development of the dorsal medial and lateral prefrontal cortices. We think that this framework maps well onto recent neurophysiological models of reward processing and is plausible from a broader developmental perspective.

We remember the good things: Age differences in learning and memory.

We combined a feedback-based learning task with a recognition memory paradigm to investigate how reward-based learning affects the event-related potential (ERP) correlates of recognition memory in younger and older adults. We found that positive, but not negative learning improves memory and results in an increased early ERP old-new effect, which is typically associated with familiarity-based memory. This indicates that reward-based learning supports a fast and relatively automatic memory retrieval process. Furthermore, we found age-related impairments in reward-based learning, whereas memory for the learned information was intact in the elderly, suggesting that declarative memory might be less affected by aging.

Younger but Not Older Adults Benefit from Salient Feedback during Learning.

Older adults are impaired in reinforcement learning (RL) when feedback is partially ambiguous (e.g., Eppinger and Kray, 2011). In this study we examined whether older adults benefit from salient feedback information during learning. We used an electrophysiological approach and investigated 15 younger and 15 older adults with a RL task in which they had to learn stimulus–response associations under two learning conditions. In the positive learning conditions, participants could gain 50 Cents for a correct response but did not gain or lose money (*00 Cent) for an incorrect response. In negative learning conditions, they could lose 50 Cents for an incorrect response but did not gain or lose money (*00 Cent) for a correct response. As the identical outcome “00 Cent” is either better or worse than the alternative outcome depending on the learning condition, this feedback type is ambiguous. To examine the influence of feedback salience we compared this condition with a condition in which positive and negative outcomes were color-coded and thereby clearly separable. The behavioral results indicated that younger adults reached higher accuracy levels under salient feedback conditions. Moreover, the error-related negativity and the feedback-related negativity for losses were larger if the good–bad dimension of feedback was salient. Hence, in younger adults salient feedback facilitates the rapid evaluation of outcomes on a good–bad dimension and by this supports learning. In contrast, for older adults we obtained neither behavioral nor electrophysiological effects of feedback salience. The older adults’ performance monitoring system therefore appears less flexible in integrating additional information in this evaluation process.

Developing developmental cognitive neuroscience: From agenda setting to hypothesis testing.

In this issue of Developmental cognitive neuroscience Shulman nd colleagues (n.d.) and Nelson and colleagues (n.d.) present wo heuristic models of cognitive development. Shulman and coleagues review the current evidence in favor of dual systems (DS) odels, which suggests that enhanced risk taking in adolescents s the consequence of an imbalance between an early maturing otivational system involved in reward processing and a later aturing cognitive control system. They conclude with the viewoint that the current literature seems to reaffirm the usefulness f these models. In a similar fashion, Nelson and colleagues (n.d.) resented an updated version of the social information processing odel (SIP), a heuristic framework which links facets of social evelopment (ranging form infant caregiver interactions to intiate relationships during adolescence) with functional changes in he developing brain. Models are one of the central instruments of modern science e.g. the double helix model of DNA, the billiard ball model of a as, or the mind as a computer). However, not all models are alike nd different types of models serve different functions in the proess of scientific discovery (Frigg and Hartmann, 2006). The current odels of adolescent brain development, including the ones preented in this issue, are often labeled as heuristic models (Casey, 014; Crone and Dahl, 2012; Nelson et al., n.d.; Richards et al., 2013; hulman et al., n.d.). However, it is not always clear what heuristic odels are and what role they have. It is currently unclear if, and ow, different heuristic models can be meaningfully compared, or o what extend they aid the formulation of testable hypotheses. oreover, although there are many results that can be interpreted s being consistent with heuristic models, we will argue that such omparisons have only limited value and may even hamper further nvestigation of the underlying developmental mechanisms.

Age differences in learning emerge from an insufficient representation of uncertainty in older adults

Healthy aging can lead to impairments in learning that affect many laboratory and real-life tasks. These tasks often involve the acquisition of dynamic contingencies, which requires adjusting the rate of learning to environmental statistics. For example, learning rate should increase when expectations are uncertain (uncertainty), outcomes are surprising (surprise) or contingencies are more likely to change (hazard rate). In this study, we combine computational modelling with an age-comparative behavioural study to test whether age-related learning deficits emerge from a failure to optimize learning according to the three factors mentioned above. Our results suggest that learning deficits observed in healthy older adults are driven by a diminished capacity to represent and use uncertainty to guide learning. These findings provide insight into age-related cognitive changes and demonstrate how learning deficits can emerge from a failure to accurately assess how much should be learned.

Reward speeds up and increases consistency of visual selective attention: A lifespan comparison

Children and older adults often show less favorable reward-based learning and decision making, relative to younger adults. It is unknown, however, whether reward-based processes that influence relatively early perceptual and attentional processes show similar lifespan differences. In this study, we investigated whether stimulus–reward associations affect selective visual attention differently across the human lifespan. Children, adolescents, younger adults, and older adults performed a visual search task in which the target colors were associated with either high or low monetary rewards. We discovered that high reward value speeded up response times across all four age groups, indicating that reward modulates attentional selection across the lifespan. This speed-up in response time was largest in younger adults, relative to the other three age groups. Furthermore, only younger adults benefited from high reward value in increasing response consistency (i.e., reduction of trial-by-trial reaction time variability). Our findings suggest that reward-based modulations of relatively early and implicit perceptual and attentional processes are operative across the lifespan, and the effects appear to be greater in adulthood. The age-specific effect of reward on reducing intraindividual response variability in younger adults likely reflects mechanisms underlying the development and aging of reward processing, such as lifespan age differences in the efficacy of dopaminergic modulation. Overall, the present results indicate that reward shapes visual perception across different age groups by biasing attention to motivationally salient events.