Andrew R. Delamater

Experimental extinction in Pavlovian conditioning: behavioural and neuroscience perspectives.

This paper reviews the behavioural and neuroscience literatures on extinction in Pavlovian conditioning with a view towards finding possible points of contact between these two often independent lines of investigation. Recent discoveries at the behavioural level indicate (1) that conditioned stimulus (CS)–unconditioned stimulus (US) associations specific in their sensory content are fully preserved during extinction, (2) that inhibitory stimulus-response associations appear to be learned during extinction, (3) that extinction is influenced by the level of activation of the US representation during nonreinforced trials, (4) that decreases in attention can influence conditioned performance during extinction, and (5) that contexts acquire an ability to modulate learning during both conditioning and extinction. Recent discoveries at the neural systems level suggest (1) that the hippocampus is important in context-specific learning during extinction, (2) that the prefrontal cortex is possibly important in long-term memory for extinction, (3) that the basolateral amygdala may be important in sustaining attention to a CS during extinction, (4) that NMDA receptors are important either in neural plasticity during extinction or by affecting the value of the US representation during extinction, and (5) that the GABAergic system may partially mediate inhibitory learning during extinction. It is concluded that both of these levels of analysis can benefit the other in the pursuit of a more comprehensive understanding of extinction.

Effects of several extinction treatments upon the integrity of Pavlovian stimulus–outcome associations.

Pavlovian-to-instrumental transfer tests were used to assess the sensitivity of Pavlovian stimulus-outcome (S-O) associations to various extinction treatments in four appetitive conditioning experiments with rats. In Experiment 1, simple nonreinforcement of a stimulus was shown to have little impact on the ability of that stimulus to display outcome-specific transfer of control. Extinguishing a stimulus by pairing the stimulus with an alternative reinforcement in Experiment 2 also had no detectable effect on the S-O associations as assessed with the outcome-specific transfer measure. The third and fourth experiments, respectively, examined the impact of postconditioning exposures to random and explicitly unpaired S-O contingencies upon previously learned S-O associations. These treatments, as well, had no detectable harmful effects upon the integrity of the S-O associations. In contrast to the consistent failures of various extinction treatments to influence the ability of stimuli to display outcome-specific transfer, these treatments often did reduce the strength of conditioned responding initially trained to these stimuli. These results support the view that extinction entails the preservation of S-O associations as well as the parallel development of inhibitory stimulus-response associations. Other notions of extinction are also discussed.

D1 but not D2 dopamine receptor antagonism blocks the acquisition of a flavor preference conditioned by intragastric carbohydrate infusions.

The effects of dopamine D1 (SCH23390) and D2 (raclopride) receptor antagonists on the acquisition and expressions of flavor preferences conditioned by the postingestive actions of sucrose were investigated. Food-restricted rats were trained in one-bottle sessions to associate one flavored saccharin solution (CS+) with intragastric (i.g.) infusions of 16% sucrose, and another flavored saccharin solution (CS-) with water infusions. Flavor preferences were then measured in two-bottle tests. In Experiment 1A, rats that received the D2 antagonist (raclopride, 200 nmol/kg; RAC group) throughout training consumed less CS+ and CS- than did saline-treated Control rats; a saline-treated Yoked group had its intake limited to that of the RAC group. All three groups displayed CS+ preferences during two-bottle tests when treated with saline or raclopride, except at doses that greatly suppressed intake. Experiment 1B obtained similar results with rats treated with 400 nmol/kg raclopride throughout training. In Experiment 2, rats that received the D1 antagonist (SCH23390, 200 nmol/kg; SCH group) throughout training consumed less CS+ and CS- than did saline-treated Control rats; a saline-treated Yoked group had its intake limited to that of the SCH group. Unlike the Control and Yoked groups, the SCH group failed to prefer the CS+ to the CS- in two bottle tests. SCH23390 treatment during two-bottle testing did not block CS+ preference in the Control or Yoked groups, except at doses that greatly suppressed intake. We conclude that D1, but not D2, dopamine receptors are critically involved in the acquisition of a sucrose-conditioned flavor preference, and both receptor subtypes have a more limited role in the expression of this preference.

Pharmacology of Flavor Preference Conditioning in Sham-Feeding Rats: Effects of Naltrexone

Relatively little is known about the neurochemical and pharmacological mechanisms involved in flavor preference learning. The present study examined the ability of the opioid antagonist, naltrexone to alter the acquisition and expression of flavor preferences conditioned by the sweet taste of sucrose. This was accomplished by adding a novel flavor (the CS+) to a sucrose solution, and a different flavor (the CS-) to a less-preferred saccharin solution. Rats were trained to drink these solutions with an open gastric fistula (sham-feeding), which minimized postingestive actions. Food-restricted (Experiments 1 and 2A) and ad lib-fed (Experiment 2B) rats were given either limited (Experiment 1) or unlimited (Experiment 2) access to the CS+ and CS- solutions during one-bottle training. Preferences were assessed in two-bottle tests (with the CS+ and CS- flavors presented in mixed sucrose-saccharin solutions) following vehicle or naltrexone (0.1-10 mg/kg, SC) treatment. The rats displayed significant CS+ preferences following vehicle, particularly after unlimited access training. In four of five experiments, naltrexone significantly reduced total intakes during the two-bottle, sham-feeding tests. Except for one instance, however, the drug failed to block the preference for the CS+ flavor over the CS- flavor. The effects of naltrexone (0.1 mg/kg) on the acquisition of flavor preferences were studied in sham-feeding rats under limited (Experiment 3A) and unlimited (Experiment 3B) training access conditions. Rats treated with naltrexone during training displayed similar CS+ preferences as did saline-treated rats, even though they consumed less CS+ during training. The naltrexone-trained rats also displayed smaller reductions in total or CS+ intakes than did saline-trained rats when all rats were treated with a 2.5 mg/kg dose of naltrexone during testing. As in previous studies, these results show that naltrexone significantly reduces the intake of sweet solutions, yet it has little or no effect on the acquisition or expression of flavor preferences conditioned by sucrose in sham-feeding rats.

Naltrexone fails to block the acquisition or expression of a flavor preference conditioned by intragastric carbohydrate infusions

The effects of naltrexone on the expression and acquisition of flavor preferences conditioned by the postingestive actions of carbohydrates were investigated. Food-restricted rats (Experiment 1) were given one-bottle training with one flavored saccharin solution (CS+) paired with intragastric (IG) infusions of 16% sucrose, and another flavored saccharin solution (CS-) paired with water infusions. In two-bottle tests CS+ was preferred to CS-, and naltrexone (1.0-5.0 mg/kg) reduced total intake but not CS+ preference. In Experiment 2 food-restricted rats that received naltrexone (0.1 or 1.0 mg/kg; NTX group) throughout one-bottle training consumed less CS+ and CS- than did saline-treated control rats. Yet, the NTX and control groups displayed similar CS+ preferences during two-bottle tests when treated with saline or naltrexone (0.1-5.0 mg/kg). In Experiment 3, rats were trained to accept more CS+ than CS- in one-bottle tests. Naltrexone (0.1-2.5 mg/kg) reduced the one-bottle intakes of both solutions, and the rats continued to consume more CS+ than CS-. We conclude that the opioid system modulates the consumption of flavored solutions, but is not critically involved in the acquisition or expression of flavor preferences conditioned by IG carbohydrate.

The influence of CS-US interval on several different indices of learning in appetitive conditioning.

Four experiments examined the effects of varying the conditioned stimulus-unconditioned stimulus (CS-US) interval (and US density) on learning in an appetitive magazine approach task with rats. Learning was assessed with conditioned response (CR) measures, as well as measures of sensory-specific stimulus-outcome associations (Pavlovian-instrumental transfer, potentiated feeding, and US devaluation). The results from these studies indicate that there exists an inverse relation between CS-US interval and magazine approach CRs, but that sensory-specific stimulus-outcome associations are established over a wide range of relatively long, but not short, CS-US intervals. These data suggest that simple CR measures provide different information about what is learned than measures of the specific stimulus-outcome association, and that time is a more critical variable for the former than latter component of learning.

Pharmacology of Flavor Preference Conditioning in Sham-Feeding Rats: Effects of Dopamine Receptor Antagonists

Opioid and dopamine systems are both implicated in the response to sweet solutions. Our laboratory previously reported that the opioid antagonist, naltrexone, reduced the intake of sweet solutions, yet had little or no effect on sucrose-conditioned flavor preferences in sham-feeding rats. The present study examined the role of dopamine D(1) and D(2) receptors in the expression of flavor preferences conditioned by the sweet taste of sucrose. All sessions were conducted under sham-feeding conditions to minimize postingestive influences. Training was accomplished by adding a novel flavor (CS+) to a 16% sucrose solution, a different flavor (CS-) to a less-preferred 0.2% saccharin solution in alternating, one-bottle sessions. Preferences were assessed in two-bottle tests with the CS+ and CS- flavors presented in mixed sucrose (8%)-saccharin (0.1%) solutions following systemic doses of 0, 50, 200, 400, or 800 nmol/kg of the D(2) antagonist, raclopride (Experiment 1) or the D(1) antagonist, SCH23390 (Experiment 2) under either food-restricted or unrestricted conditions. Rats significantly preferred the CS+ solutions in vehicle tests, and displayed equipotent and dose-dependent reductions in total intake and CS+ preference following either D(1) or D(2) receptor antagonism. Similar results were obtained with SCH23390 and raclopride in Experiment 3 conducted with water-restricted rats. These data indicate that dopaminergic D(1) and D(2) receptors play pivotal and functionally equivalent roles in the expression of flavor preferences conditioned by the sweet taste of sucrose.

Role of D1 and D2 dopamine receptors in the acquisition and expression of flavor-preference conditioning in sham-feeding rats

The present study examined the effects of D(1) and D(2) antagonists on flavor-preference conditioning by the sweet taste of sucrose. All sessions were conducted under sham-feeding conditions to minimize post-ingestive influences. The rats were trained in alternating, one-bottle sessions to sham-feed a 16% sucrose solution containing one novel flavor (CS+) and a less-preferred 0.2% saccharin solution containing a different flavor (CS-). Three groups of food-restricted rats were treated with either vehicle (control group), the D(1) antagonist, SCH23390 (200 nmol/kg), or the D(2) antagonist, raclopride (200 nmol/kg) during one-bottle training. A fourth group (yoked group) was vehicle-treated and its training intakes were matched to that of the D(1) and D(2) drug groups. Preferences were assessed in two-bottle tests with the CS+ and CS- flavors presented in mixed 8% sucrose+0.1% saccharin solutions following systemic doses of 0, 200, or 800 nmol/kg of either the D(1) or D(2) antagonists. All groups significantly preferred the CS+ flavor in vehicle tests, although the preferences were weaker in the D(1), D(2), and yoked groups compared to the control group. All groups selectively reduced their CS+ intakes when treated with either D(1) or D(2) antagonists during two-bottle testing, and the CS+ preference was blocked at the higher doses. These data show that D(1) and D(2) receptor antagonists block the expression of a sucrose-conditioned preference, but produces substantially lesser effects upon the acquisition of this form of flavor conditioning.

Psychological and neural mechanisms of experimental extinction: a selective review.

The present review examines key psychological concepts in the study of experimental extinction and implications these have for an understanding of the underlying neurobiology of extinction learning. We suggest that many of the signature characteristics of extinction learning (spontaneous recovery, renewal, reinstatement, rapid reacquisition) can be accommodated by the standard associative learning theory assumption that extinction results in partial erasure of the original learning together with new inhibitory learning. Moreover, we consider recent behavioral and neural evidence that supports the partial erasure view of extinction, but also note shortcomings in our understanding of extinction circuits as these relate to the negative prediction error concept. Recent work suggests that common prediction error and stimulus-specific prediction error terms both may be required to explain neural plasticity both in acquisition and extinction learning. In addition, we suggest that many issues in the content of extinction learning have not been fully addressed in current research, but that neurobiological approaches should be especially helpful in addressing such issues. These include questions about the nature of extinction learning (excitatory CS-No US, inhibitory CS-US learning, occasion setting processes), especially as this relates to studies of the micro-circuitry of extinction, as well as its representational content (sensory, motivational, response). An additional understudied problem in extinction research is the role played by attention processes and their underlying neural networks, although some research and theory converge on the idea that extinction is accompanied by attention decrements (i.e., habituation-like processes).

Learning about Multiple Attributes of Reward in Pavlovian Conditioning

Abstract:  The nature of the reward representation in Pavlovian conditioning has been of perennial interest to students of associative learning theory. We consider the view that it consists of a range of different attributes, each of which may be governed by different learning rules. We investigated this issue through a series of experiments using a time‐sensitive Pavlovian‐to‐instrumental transfer procedure, aiming to dissociate learning about temporal and specific sensory features of a reward. Our results successfully demonstrated that learning about these different features appears to be dissociable, with learning about the specific sensory features of a Pavlovian unconditioned stimulus (US) occurring very rapidly across a wide range of experimental procedures, while learning about the temporal features of the US occurred slightly less quickly and was more sensitive to parametric disruption. These results are discussed with regard to the potential independence or interdependence of the relevant learning processes, and to some recent neurophysiological recording and brain lesion work, which provide additional means to investigate these dissociations.