Ian C. Ballard

Age Differences in Striatal Delay Sensitivity during Intertemporal Choice in Healthy Adults

Intertemporal choices are a ubiquitous class of decisions that involve selecting between outcomes available at different times in the future. We investigated the neural systems supporting intertemporal decisions in healthy younger and older adults. Using functional neuroimaging, we find that aging is associated with a shift in the brain areas that respond to delayed rewards. Although we replicate findings that brain regions associated with the mesolimbic dopamine system respond preferentially to immediate rewards, we find a separate region in the ventral striatum with very modest time dependence in older adults. Activation in this striatal region was relatively insensitive to delay in older but not younger adults. Since the dopamine system is believed to support associative learning about future rewards over time, our observed transfer of function may be due to greater experience with delayed rewards as people age. Identifying differences in the neural systems underlying these decisions may contribute to a more comprehensive model of age-related change in intertemporal choice.

Hippocampal Networks Habituate as Novelty Accumulates.

Novelty detection, a critical computation within the medial temporal lobe (MTL) memory system, necessarily depends on prior experience. The current study used functional magnetic resonance imaging (fMRI) in humans to investigate dynamic changes in MTL activation and functional connectivity as experience with novelty accumulates. fMRI data were collected during a target detection task: Participants monitored a series of trial-unique novel and familiar scene images to detect a repeating target scene. Even though novel images themselves did not repeat, we found that fMRI activations in the hippocampus and surrounding cortical MTL showed a specific, decrementing response with accumulating exposure to novelty. The significant linear decrement occurred for the novel but not the familiar images, and behavioral measures ruled out a corresponding decline in vigilance. Additionally, early in the series, the hippocampus was inversely coupled with the dorsal striatum, lateral and medial prefrontal cortex, and posterior visual processing regions; this inverse coupling also habituated as novelty accumulated. This novel demonstration of a dynamic adjustment in neural responses to novelty suggests a similarly dynamic allocation of neural resources based on recent experience.

Hippocampus and Prefrontal Cortex Predict Distinct Timescales of Activation in the Human Ventral Tegmental Area

Abstract The mesolimbic dopamine system contributes to a remarkable variety of behaviors at multiple timescales. Midbrain neurons have fast and slow signaling components, and specific afferent systems, such as the hippocampus (HPC) and prefrontal cortex (PFC), have been demonstrated to drive these components in anesthetized animals. Whether these interactions exist during behavior, however, is unknown. To address this question, we developed a novel analysis of human functional magnetic resonance imaging data that fits models of network excitation and inhibition on ventral tegmental area (VTA) activation. We show that specific afferent systems predict distinct temporal components of midbrain VTA signal. We found that PFC, but not HPC, positively predicted transient, event‐evoked VTA activation. In contrast, HPC, but not PFC, positively predicted slow shifts in VTA baseline variability. Thus, unique functional contributions of afferent systems to VTA physiology are detectable at the network level in behaving humans. The findings support models of dopamine function in which dissociable neural circuits support different aspects of motivated behavior via active regulation of tonic and phasic signals.

More Is Meaningful: The Magnitude Effect in Intertemporal Choice Depends on Self-Control

Impulsivity is a variable behavioral trait that depends on numerous factors. For example, increasing the absolute magnitude of available choice options promotes farsighted decisions. We argue that this magnitude effect arises in part from differential exertion of self-control as the perceived importance of the choice increases. First, we demonstrated that frontal executive-control areas were more engaged for more difficult decisions and that this effect was enhanced for high-magnitude rewards. Second, we showed that increased hunger, which is associated with lower self-control, reduced the magnitude effect. Third, we tested an intervention designed to increase self-control and showed that it reduced the magnitude effect. Taken together, our findings challenge existing theories about the magnitude effect and suggest that visceral and cognitive factors affecting choice may do so by influencing self-control.

Mere Exposure: Preference Change for Novel Drinks Reflected in Human Ventral Tegmental Area

Preferences for novel stimuli tend to develop slowly over many exposures. Psychological accounts of this effect suggest that it depends on changes in the brains valuation system. Participants consumed a novel fluid daily for 10 days and underwent fMRI on the first and last days. We hypothesized that changes in activation in areas associated with the dopamine system would accompany changes in preference. The change in activation in the ventral tegmental area (VTA) between sessions scaled with preference change. Furthermore, a network comprising the sensory thalamus, posterior insula, and ventrolateral striatum showed differential connectivity with the VTA that correlated with individual changes in preference. Our results suggest that the VTA is centrally involved in both assigning value to sensory stimuli and influencing downstream regions to translate these value signals into subjective preference. These results have important implications for models of dopaminergic function and behavioral addiction.

Joint Modeling of Reaction Times and Choice Improves Parameter Identifiability in Reinforcement Learning Models

Reinforcement learning models provide excellent descriptions of learning in a variety of tasks. Many researchers are interested in relating parameters of reinforcement learning models to psychological or neural variables of interest. We demonstrate that parameter identification is difficult due to the fact that a range of parameter values provide approximately equal quality fits to data. This identification problem has a large impact on power: we show that a researcher who wants to detect a medium sized correlation (r = .3) with 80% power between a psychological/neural variable and learning rate must collect 60% more subjects than specified by a typical power analysis in order to account for the noise introduced by model fitting. We introduce a method that exploits the information contained in reaction times to constrain model fitting and show using simulation and empirical data that it improves the ability to recover learning rates.

Causal Evidence for the Dependence of the Magnitude Effect on Dorsolateral Prefrontal Cortex

Impulsivity refers to the tendency to insufficiently consider alternatives or to overvalue rewards that are available sooner. The latter form of impulsivity – present bias – is a hallmark of human decision making with well documented health and financial ramifications. Numerous contextual changes and framing manipulations can powerfully influence present bias. One of the most robust such phenomenon is the finding that people are more patient as the values of choice options are increased. This magnitude effect has been related to cognitive control mechanisms in the dorsal lateral prefrontal cortex (dlPFC). We used repetitive transcranial magnetic stimulation (rTMS) to transiently disrupt neural activity in dlPFC. This manipulation dramatically reduced the magnitude effect, establishing causal evidence that the magnitude effect depends on dlPFC.

Hippocampal Pattern Separation Supports Reinforcement Learning

Animals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three-leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multifeatural conjunctive representations. We tested if the hippocampus formed separable conjunctive representations that enabled learning of response contingencies for stimuli of the form: AB+, B-, AC-, C+. Pattern analyses on functional MRI data showed the hippocampus formed conjunctive representations that were dissociable from feature components and that these representations influenced striatal PEs. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.

Toward an integrative perspective on the neural mechanisms underlying persistent maladaptive behaviors

A host of public health problems-from drug addiction to obesity-are associated with persistent, maladaptive behaviors. The underlying causes of such behaviors have received considerable attention from psychologists, clinicians, computational theorists, and neuroscientists. These diverse perspectives were showcased in a symposium at the University of Rochester entitled Persistent, Maladaptive Behaviors: Why We Make Bad Choices. Here, we synthesize novel findings and perspectives arising from the symposium and integrate those findings within the broader literature. This article is protected by copyright. All rights reserved.