Peter Smittenaar

Dopamine Enhances Model-Based over Model-Free Choice Behavior

Summary Decision making is often considered to arise out of contributions from a model-free habitual system and a model-based goal-directed system. Here, we investigated the effect of a dopamine manipulation on the degree to which either system contributes to instrumental behavior in a two-stage Markov decision task, which has been shown to discriminate model-free from model-based control. We found increased dopamine levels promote model-based over model-free choice.

Disruption of Dorsolateral Prefrontal Cortex Decreases Model-Based in Favor of Model-free Control in Humans

Summary Human choice behavior often reflects a competition between inflexible computationally efficient control on the one hand and a slower more flexible system of control on the other. This distinction is well captured by model-free and model-based reinforcement learning algorithms. Here, studying human subjects, we show it is possible to shift the balance of control between these systems by disruption of right dorsolateral prefrontal cortex, such that participants manifest a dominance of the less optimal model-free control. In contrast, disruption of left dorsolateral prefrontal cortex impaired model-based performance only in those participants with low working memory capacity.

Widespread age-related differences in the human brain microstructure revealed by quantitative magnetic resonance imaging

A pressing need exists to disentangle age-related changes from pathologic neurodegeneration. This study aims to characterize the spatial pattern and age-related differences of biologically relevant measures in vivo over the course of normal aging. Quantitative multiparameter maps that provide neuroimaging biomarkers for myelination and iron levels, parameters sensitive to aging, were acquired from 138 healthy volunteers (age range: 19–75 years). Whole-brain voxel-wise analysis revealed a global pattern of age-related degeneration. Significant demyelination occurred principally in the white matter. The observed age-related differences in myelination were anatomically specific. In line with invasive histologic reports, higher age-related differences were seen in the genu of the corpus callosum than the splenium. Iron levels were significantly increased in the basal ganglia, red nucleus, and extensive cortical regions but decreased along the superior occipitofrontal fascicle and optic radiation. This whole-brain pattern of age-associated microstructural differences in the asymptomatic population provides insight into the neurobiology of aging. The results help build a quantitative baseline from which to examine and draw a dividing line between healthy aging and pathologic neurodegeneration.

Crowdsourcing for cognitive science--the utility of smartphones.

By 2015, there will be an estimated two billion smartphone users worldwide. This technology presents exciting opportunities for cognitive science as a medium for rapid, large-scale experimentation and data collection. At present, cost and logistics limit most study populations to small samples, restricting the experimental questions that can be addressed. In this study we investigated whether the mass collection of experimental data using smartphone technology is valid, given the variability of data collection outside of a laboratory setting. We presented four classic experimental paradigms as short games, available as a free app and over the first month 20,800 users submitted data. We found that the large sample size vastly outweighed the noise inherent in collecting data outside a controlled laboratory setting, and show that for all four games canonical results were reproduced. For the first time, we provide experimental validation for the use of smartphones for data collection in cognitive science, which can lead to the collection of richer data sets and a significant cost reduction as well as provide an opportunity for efficient phenotypic screening of large populations.

Preparing for Selective Inhibition within Frontostriatal Loops

Action inhibition can globally prevent all motor output or selectively cancel specific actions during concurrent motor output. Here we examine the behavioral and neural basis of selective inhibition focusing on the role of preparation. In 18 healthy human participants we manipulated the extent to which they could prepare for selective inhibition by providing or withholding information on what actions might need to be stopped. We show that, on average, information improves both speed and selectivity of inhibition. Functional magnetic resonance imaging data show that preparation for selective inhibition engages the inferior frontal gyrus, supplementary motor area, and striatum. Examining interindividual differences, we find the benefit of proactive control to speed and selectivity of inhibition trade off against each other, such that an improvement in stopping speed leads to a deterioration of selectivity of inhibition, and vice versa. This trade-off is implemented through engagement of the dorsolateral prefrontal cortex and putamen. Our results suggest proactive selective inhibition is implemented within frontostriatal structures, and we provide evidence that a speed-selectivity trade-off might underlie a range of findings reported previously.

Age-related changes in working memory and the ability to ignore distraction

Significance We reveal a novel and highly significant change in how items are held in mind in healthy aging. Using smartphones, data were collected from 29,631 participants, between the ages of 18–69 y. We compare the ability to exclude distractors when items are entered into working memory (WM) (encoding distraction, ED) and when items are held in mind (delay distraction, DD). In older adults, WM in the absence of distraction was more similar to ED exclusion than DD exclusion. A greater reliance on focused attention during encoding may reflect compensation for the more pronounced deterioration we observed in DD exclusion in older age. This can inform other areas of cognition and strategies to ameliorate or manage debilitating age-related cognitive decline. A weakened ability to effectively resist distraction is a potential basis for reduced working memory capacity (WMC) associated with healthy aging. Exploiting data from 29,631 users of a smartphone game, we show that, as age increases, working memory (WM) performance is compromised more by distractors presented during WM maintenance than distractors presented during encoding. However, with increasing age, the ability to exclude distraction at encoding is a better predictor of WMC in the absence of distraction. A significantly greater contribution of distractor filtering at encoding represents a potential compensation for reduced WMC in older age.

Adaptive integration of habits into depth-limited planning defines a habitual-goal–directed spectrum

Significance Solving complex tasks often requires estimates of the future consequences of current actions. Estimates could be learned from past experience, but they then risk being out of date, or they could be calculated by a form of planning into the future, a process that is computationally taxing. We show that humans integrate learned estimates into their planning calculations, saving mental effort and time. We also show that increasing time pressure leads to reliance on learned estimates after fewer steps of planning. We suggest a normative rationale for this effect using a computational model. Our results provide a perspective on how the brain combines different decision processes collaboratively to exploit their comparative computational advantages. Behavioral and neural evidence reveal a prospective goal-directed decision process that relies on mental simulation of the environment, and a retrospective habitual process that caches returns previously garnered from available choices. Artificial systems combine the two by simulating the environment up to some depth and then exploiting habitual values as proxies for consequences that may arise in the further future. Using a three-step task, we provide evidence that human subjects use such a normative plan-until-habit strategy, implying a spectrum of approaches that interpolates between habitual and goal-directed responding. We found that increasing time pressure led to shallower goal-directed planning, suggesting that a speed-accuracy tradeoff controls the depth of planning with deeper search leading to more accurate evaluation, at the cost of slower decision-making. We conclude that subjects integrate habit-based cached values directly into goal-directed evaluations in a normative manner.

Proactive and reactive response inhibition across the lifespan

One expression of executive control involves proactive preparation for future events, and this contrasts with stimulus driven reactive control exerted in response to events. Here we describe findings from a response inhibition task, delivered using a smartphone-based platform, that allowed us to index proactive and reactive inhibitory self-control in a large community sample (n = 12,496). Change in stop-signal reaction time (SSRT) when participants are provided with advance information about an upcoming trial, compared to when they are not, provides a measure of proactive control while SSRT in the absence of advance information provides a measure of reactive control. Both forms of control rely on overlapping frontostriatal pathways known to deteriorate in healthy aging, an age-related decline that occurs at an accelerated rate in men compared to women. Here we ask whether these patterns of age-related decline are reflected in similar changes in proactive and reactive inhibitory control across the lifespan. As predicted, we observed a decline in reactive control with natural aging, with a greater rate of decline in men compared to women (~10 ms versus ~8 ms per decade of adult life). Surprisingly, the benefit of preparation, i.e. proactive control, did not change over the lifespan and women showed superior proactive control at all ages compared to men. Our results suggest that reactive and proactive inhibitory control partially rely on distinct neural substrates that are differentially sensitive to age-related change.

Risk Taking for Potential Reward Decreases across the Lifespan

Summary The extent to which aging affects decision-making is controversial. Given the critical financial decisions that older adults face (e.g., managing retirement funds), changes in risk preferences are of particular importance [1]. Although some studies have found that older individuals are more risk averse than younger ones [2, 3, 4], there are also conflicting results, and a recent meta-analysis found no evidence for a consistent change in risk taking across the lifespan [5]. There has as yet been little examination of one potential substrate for age-related changes in decision-making, namely age-related decline in dopamine, a neuromodulator associated with risk-taking behavior. Here, we characterized choice preferences in a smartphone-based experiment (n = 25,189) in which participants chose between safe and risky options. The number of risky options chosen in trials with potential gains but not potential losses decreased gradually over the lifespan, a finding with potentially important economic consequences for an aging population. Using a novel approach-avoidance computational model, we found that a Pavlovian attraction to potential reward declined with age. This Pavlovian bias has been linked to dopamine, suggesting that age-related decline in this neuromodulator could lead to the observed decrease in risk taking.

Transcranial Direct Current Stimulation of Right Dorsolateral Prefrontal Cortex Does Not Affect Model-Based or Model-Free Reinforcement Learning in Humans

There is broad consensus that the prefrontal cortex supports goal-directed, model-based decision-making. Consistent with this, we have recently shown that model-based control can be impaired through transcranial magnetic stimulation of right dorsolateral prefrontal cortex in humans. We hypothesized that an enhancement of model-based control might be achieved by anodal transcranial direct current stimulation of the same region. We tested 22 healthy adult human participants in a within-subject, double-blind design in which participants were given Active or Sham stimulation over two sessions. We show Active stimulation had no effect on model-based control or on model-free (‘habitual’) control compared to Sham stimulation. These null effects are substantiated by a power analysis, which suggests that our study had at least 60% power to detect a true effect, and by a Bayesian model comparison, which favors a model of the data that assumes stimulation had no effect over models that assume stimulation had an effect on behavioral control. Although we cannot entirely exclude more trivial explanations for our null effect, for example related to (faults in) our experimental setup, these data suggest that anodal transcranial direct current stimulation over right dorsolateral prefrontal cortex does not improve model-based control, despite existing evidence that transcranial magnetic stimulation can disrupt such control in the same brain region.