Andrew T. Marshall

Mechanisms of impulsive choice: I. Individual differences in interval timing and reward processing

Impulsive choice behavior incorporates the psychological mechanisms involved in the processing of the anticipated magnitude and delay until reward. The goal of the present experiment was to determine whether individual differences in such processes related to individual differences in impulsive choice behavior. Two groups of rats (Delay Group and Magnitude Group) were initially exposed to an impulsive choice task with choices between smaller-sooner (SS) and larger-later (LL) rewards. The Delay Group was subsequently exposed to a temporal discrimination task followed by a progressive interval task, whereas the Magnitude Group was exposed to a reward magnitude sensitivity task followed by a progressive ratio task. Intertask correlations revealed that the rats in the Delay Group that made more self-controlled (LL) choices also displayed lower standard deviations in the temporal bisection task and greater delay tolerance in the progressive interval task. Impulsive choice behavior in the Magnitude Group did not display any substantial correlations with the reward magnitude sensitivity and progressive ratio tasks. The results indicate the importance of core timing processes in impulsive choice behavior, and encourage further research examining the effects of changes in core timing processes on impulsive choice.

Motivation and timing: clues for modeling the reward system.

There is growing evidence that a change in reward magnitude or value alters interval timing, indicating that motivation and timing are not independent processes as was previously believed. The present paper reviews several recent studies, as well as presenting some new evidence with further manipulations of reward value during training vs. testing on a peak procedure. The combined results cannot be accounted for by any of the current psychological timing theories. However, in examining the neural circuitry of the reward system, it is not surprising that motivation has an impact on timing because the motivation/valuation system directly interfaces with the timing system. A new approach is proposed for the development of the next generation of timing models, which utilizes knowledge of the neuroanatomy and neurophysiology of the reward system to guide the development of a neurocomputational model of the reward system. The initial foundation along with heuristics for proceeding with developing such a model is unveiled in an attempt to stimulate new theoretical approaches in the field.

Mechanisms of impulsive choice: II. Time-based interventions to improve self-control.

Impulsive choice behavior has been proposed as a primary risk factor for other maladaptive behaviors (e.g., gambling, substance abuse). Recent research has suggested that timing processes may play a key role in impulsive choice behavior, and could provide an avenue for altering impulsive choice. Accordingly, the current experiments assessed a set of time-based behavioral interventions to increase self-control while simultaneously assessing effects on timing processes within the impulsive choice task. Three experiments assessed temporal interventions using a differential reinforcement of low rates task (Experiment 1) and exposure to either a variable or fixed interval schedule (Experiments 2-3). The efficacy of the interventions was assessed in Sprague-Dawley (Experiments 1-2) and Lewis (Experiment 3) rat strains. Impulsive choice behavior was assessed by measuring preferences of a smaller-sooner (SS) versus a larger-later (LL) reward, while timing of the SS and LL durations was measured during peak trials within the impulsive choice procedure. The rats showed an increased preference for the LL following all three time-based interventions and also displayed increased temporal precision. These results add to the increasing evidence that supports a possible role for temporal processing in impulsive choice behavior and supply novel behavioral interventions to decrease impulsive behavior.

Individual differences in impulsive and risky choice: effects of environmental rearing conditions

The present experiment investigated early-rearing environment modulation of individual differences in impulsive and risky choice. Rats were reared in an isolated condition (IC; n=12), in which they lived alone without novel stimuli, or an enriched condition (EC; n=11), in which they lived among conspecifics with novel stimuli. The impulsive choice task involved choices between smaller-sooner (SS) versus larger-later (LL) rewards. The risky choice task involved choices between certain-smaller (C-S) versus uncertain-larger (U-L) rewards. Following choice testing, incentive motivation to work for food was measured using a progressive ratio task and correlated with choice behavior. HPLC analyses were conducted to determine how monoamine concentrations within the prefrontal cortex (PFC) and nucleus accumbens (NAC) related to behavior in different tasks. IC rats were more impulsive than EC rats, but they did not differ in risky choice behavior. However, choice behavior across tasks was significantly correlated (i.e., the more impulsive rats were also riskier). There were no group differences in monoamine levels, but noradrenergic and serotonergic concentrations were significantly correlated with impulsive and risky choice. Furthermore, serotonin and norepinephrine concentrations in the NAC significantly correlated with incentive motivation and the timing of the reward delays within the choice tasks. These results suggest a role for domain general processes in impulsive and risky choice and indicate the importance of the NAC and/or PFC in timing, reward processing, and choice behavior.

Environmental rearing effects on impulsivity and reward sensitivity

Previous research has indicated that rearing in an enriched environment may promote self-control in an impulsive choice task. To further assess the effects of rearing environment on impulsivity, 2 experiments examined locomotor activity, impulsive action, impulsive choice, and different aspects of reward sensitivity and discrimination. In Experiment 1, rats reared in isolated or enriched conditions were tested on an impulsive choice procedure with a smaller-sooner versus a larger-later reward, revealing that the isolated rats valued the smaller-sooner reward more than the enriched rats. A subsequent reward challenge was presented in which the delay to the 2 rewards was the same but the magnitude difference remained. The enriched rats did not choose the larger reward as often as the isolated rats, reflecting poorer reward discrimination. Impulsive action was assessed using a differential-reinforcement-of-low-rate task, which revealed deficits in the enriched rats. In Experiment 2, rats reared in isolated, standard, or enriched conditions were tested on reward contrast and reward magnitude sensitivity procedures. The rats were presented with 2 levers that delivered different magnitudes of food on variable interval 30-s schedules. Across all tests, the enriched and social rats displayed more generalized responding to the small-reward lever, but a similar response to the large-reward lever, compared with the isolated rats. This confirmed the results of Experiment 1, indicating poorer reward discrimination in the enriched condition compared with the isolated condition. The results suggest that enrichment may moderate reward generalization/discrimination processes through alterations in incentive motivational processes.

Relative gains, losses, and reference points in probabilistic choice in rats.

Theoretical reference points have been proposed to differentiate probabilistic gains from probabilistic losses in humans, but such a phenomenon in non-human animals has yet to be thoroughly elucidated. Three experiments evaluated the effect of reward magnitude on probabilistic choice in rats, seeking to determine reference point use by examining the effect of previous outcome magnitude(s) on subsequent choice behavior. Rats were trained to choose between an outcome that always delivered reward (low-uncertainty choice) and one that probabilistically delivered reward (high-uncertainty). The probability of high-uncertainty outcome receipt and the magnitudes of low-uncertainty and high-uncertainty outcomes were manipulated within and between experiments. Both the low- and high-uncertainty outcomes involved variable reward magnitudes, so that either a smaller or larger magnitude was probabilistically delivered, as well as reward omission following high-uncertainty choices. In Experiments 1 and 2, the between groups factor was the magnitude of the high-uncertainty-smaller (H-S) and high-uncertainty-larger (H-L) outcome, respectively. The H-S magnitude manipulation differentiated the groups, while the H-L magnitude manipulation did not. Experiment 3 showed that manipulating the probability of differential losses as well as the expected value of the low-uncertainty choice produced systematic effects on choice behavior. The results suggest that the reference point for probabilistic gains and losses was the expected value of the low-uncertainty choice. Current theories of probabilistic choice behavior have difficulty accounting for the present results, so an integrated theoretical framework is proposed. Overall, the present results have implications for understanding individual differences and corresponding underlying mechanisms of probabilistic choice behavior.

Mechanisms of impulsive choice: III. The role of reward processes.

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10 s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30 s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10 s) and an LL reward (2 and 4 pellets, 30 s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

Sex-specific enhancement of palatability-driven feeding in adolescent rats

It has been hypothesized that brain development during adolescence perturbs reward processing in a way that may ultimately contribute to the risky decision making associated with this stage of life, particularly in young males. To investigate potential reward dysfunction during adolescence, Experiment 1 examined palatable fluid intake in rats as a function of age and sex. During a series of twice-weekly test sessions, non-food-deprived rats were given the opportunity to voluntarily consume a highly palatable sweetened condensed milk (SCM) solution. We found that adolescent male, but not female, rats exhibited a pronounced, transient increase in SCM intake (normalized by body weight) that was centered around puberty. Additionally, adult females consumed more SCM than adult males and adolescent females. Using a well-established analytical framework to parse the influences of reward palatability and satiety on the temporal structure of feeding behavior, we found that palatability-driven intake at the outset of the meal was significantly elevated in adolescent males, relative to the other groups. Furthermore, although we found that there were some group differences in the onset of satiety, they were unlikely to contribute to differences in intake. Experiment 2 confirmed that adolescent male rats exhibit elevated palatable fluid consumption, relative to adult males, even when a non-caloric saccharin solution was used as the taste stimulus, demonstrating that these results were unlikely to be related to age-related differences in metabolic need. These findings suggest that elevated palatable food intake during adolescence is sex specific and driven by a fundamental change in reward processing. As adolescent risk taking has been hypothesized as a potential result of hypersensitivity to and overvaluation of appetitive stimuli, individual differences in reward palatability may factor into individual differences in adolescent risky decision making.

I Can’t Wait: Methods for Measuring and Moderating Individual Differences in Impulsive Choice

Abstract Impulsive choice behavior occurs when individuals make choices without regard for future consequences. This behavior is often maladaptive and is a common symptom in many disorders, including drug abuse, compulsive gambling, and obesity. Several proposed mechanisms may influence impulsive choice behavior. These mechanisms provide a variety of pathways that may provide the basis for individual differences that are often evident when measuring choice behavior. This review provides an overview of these different pathways to impulsive choice, and the behavioral intervention strategies being developed to moderate impulsive choice. Because of the compelling link between impulsive choice behavior and the near-epidemic pervasiveness of obesity in the United States, we focus on the relationship between impulsive choice behavior and obesity as a test case for application of the multiple pathways approach. Choosing immediate gratification over healthier long term food choices is a contributing factor to the obesity crisis. Behavioral interventions can lead to more self-controlled choices in a rat pre-clinical model, suggesting a possible gateway for translation to human populations. Designing and implementing effective impulsive choice interventions is crucial to improving the overall health and well-being of impulsive individuals.

A simultaneous temporal processing account of response rate

The goal was to determine whether a signal (e.g., a click) at food availability affects timing behavior in rats. Twenty-four rats were trained on an appetitive lever-press procedure that varied on two dimensions: shape of the interreinforcer distribution (i.e., fixed-interval 60s or random-interval 60s) and number of signals (i.e., the presence or absence of a click at the time of reinforcer availability). The rats were randomly partitioned into one of four groups (each group had six rats): Fixed, Signaled-Fixed, Random, and Signaled-Random. The shape of the interreinforcer distribution affected the response pattern; the presence of the click affected response rate. These results provide support for a simultaneous temporal processing account of behavior.