Travis P. Todd

Renewal after the extinction of free operant behavior

Four experiments were performed to explore the role of context in operant extinction. In all experiments, leverpressing in rats was first reinforced with food pellets on a variable interval 30-s schedule, then extinguished, and finally tested in the same and a different physical context. The experiments demonstrated a clear ABA renewal effect, a recovery of extinguished responding when conditioning, extinction, and testing occurred in contexts A, B, and A, respectively. They also demonstrated ABC renewal (where conditioning extinction and testing occurred in contexts A, B, and C) and, for the first time in operant conditioning, AAB renewal (where conditioning, extinction, and testing occurred in contexts A, A, and B). The latter two phenomena indicate that tests outside the extinction context are sufficient to cause a recovery of extinguished operant behavior and, thus, that operant extinction, like Pavlovian extinction, is relatively specific to the context in which it is learned. AAB renewal was not weakened by tripling the amount of extinction training. ABA renewal was stronger than AAB, but not merely because of context A’s direct association with the reinforcer.

Relapse processes after the extinction of instrumental learning: Renewal, resurgence, and reacquisition.

It is widely recognized that extinction (the procedure in which a Pavlovian conditioned stimulus or an instrumental action is repeatedly presented without its reinforcer) weakens behavior without erasing the original learning. Most of the experiments that support this claim have focused on several relapse effects that occur after Pavlovian extinction, which collectively suggest that the original learning is saved through extinction. However, although such effects do occur after instrumental extinction, they have not been explored there in as much detail. This article reviews recent research in our laboratory that has investigated three relapse effects that occur after the extinction of instrumental (operant) learning. In renewal, responding returns after extinction when the behavior is tested in a different context; in resurgence, responding recovers when a second response that has been reinforced during extinction of the first is itself put on extinction; and in rapid reacquisition, extinguished responding returns rapidly when the response is reinforced again. The results provide new insights into extinction and relapse, and are consistent with principles that have been developed to explain extinction and relapse as they occur after Pavlovian conditioning. Extinction of instrumental learning, like Pavlovian learning, involves new learning that is relatively dependent on the context for expression.

Behavioral and Neurobiological Mechanisms of Extinction in Pavlovian and Instrumental Learning

This article reviews research on the behavioral and neural mechanisms of extinction as it is represented in both Pavlovian and instrumental learning. In Pavlovian extinction, repeated presentation of a signal without its reinforcer weakens behavior evoked by the signal; in instrumental extinction, repeated occurrence of a voluntary action without its reinforcer weakens the strength of the action. In either case, contemporary research at both the behavioral and neural levels of analysis has been guided by a set of extinction principles that were first generated by research conducted at the behavioral level. The review discusses these principles and illustrates how they have informed the study of both Pavlovian and instrumental extinction. It shows that behavioral and neurobiological research efforts have been tightly linked and that their results are readily integrated. Pavlovian and instrumental extinction are also controlled by compatible behavioral and neural processes. Since many behavioral effects observed in extinction can be multiply determined, we suggest that the current close connection between behavioral-level and neural-level analyses will need to continue.

A fundamental role for context in instrumental learning and extinction.

The purpose of this article is to review recent research that has investigated the effects of context change on instrumental (operant) learning. The first part of the article discusses instrumental extinction, in which the strength of a reinforced instrumental behavior declines when reinforcers are withdrawn. The results suggest that extinction of either simple or discriminated operant behavior is relatively specific to the context in which it is learned: As in prior studies of Pavlovian extinction, ABA, ABC, and AAB renewal effects can all be observed. Further analysis supports the idea that the organism learns to refrain from making a specific response in a specific context, or in more formal terms, an inhibitory context-response association. The second part of the article then discusses research suggesting that the context also controls instrumental behavior before it is extinguished. Several experiments demonstrate that a context switch after either simple or discriminated operant training causes a decrement in the strength of the response. Over a range of conditions, the animal appears to learn a direct association between the context and the response. Under some conditions, it can also learn a hierarchical representation of context and the response-reinforcer relation. Extinction is still more context-specific than conditioning, as indicated by ABC and AAB renewal. Overall, the results establish that the context can play a significant role in both the acquisition and extinction of operant behavior.

Chemogenetic Silencing of Neurons in Retrosplenial Cortex Disrupts Sensory Preconditioning

An essential aspect of episodic memory is the formation of associations between neutral sensory cues in the environment. In light of recent evidence that this critical aspect of learning does not require the hippocampus, we tested the involvement of the retrosplenial cortex (RSC) in this process using a chemogenetic approach that allowed us to temporarily silence neurons along the entire rostrocaudal extent of the RSC. A viral vector containing the gene for a synthetic inhibitory G-protein-coupled receptor (hM4Di) was infused into RSC. When the receptor was later activated by systemic injection of clozapine-N-oxide, neural activity in RSC was transiently silenced (confirmed using a patch-clamp procedure). Rats expressing hM4Di and control rats were trained in a sensory preconditioning procedure in which a tone and light were paired on some trials and a white noise stimulus was presented alone on the other trials during the Preconditioning phase. Thus, rats were given the opportunity to form an association between a tone and a light in the absence of reinforcement. Later, the light was paired with food. During the test phase when the auditory cues were presented alone, controls exhibited more conditioned responding during presentation of the tone compared with the white noise reflecting the prior formation of a tone-light association. Silencing RSC neurons during the Preconditioning phase prevented the formation of an association between the tone and light and eliminated the sensory preconditioning effect. These findings indicate that RSC may contribute to episodic memory formation by linking essential sensory stimuli during learning.

Mechanisms of renewal after the extinction of instrumental behavior

Four experiments with rats examined renewal of extinguished instrumental behavior when the reinforcement histories of the contexts were equated by giving complementary training and extinction of a different response (lever press and chain pull) in each context. In Experiments 1 through 3, renewal occurred when the response was tested in the acquisition context (ABA) or outside the extinction context (AAB and ABC). Further, in Experiments 1 through 3, when both responses were simultaneously available, there was a clear preference for the response that was not in its extinction context. In Experiment 4, renewal was not reduced when testing occurred in a context that had been associated with extinction of the other instrumental response. The experimental designs rule out differential context-reinforcer associations being the only contributing mechanism of renewal, and also raise questions about configural and occasion-setting accounts. The results are consistent with the idea that during extinction an inhibitory association is formed between the context and the response.

Effects of the amount of acquisition and contextual generalization on the renewal of instrumental behavior after extinction

Four experiments with rat subjects examined the role of context during the extinction of instrumental (free-operant) behavior. In all experiments, leverpressing was first reinforced on a variable-interval 30-s schedule and then extinguished before being tested in the extinction and renewal contexts. The results identified three important variables affecting the renewal effect after instrumental extinction. First, ABA and ABC forms of renewal were strengthened by increasing the amount of acquisition training. This suggests that the strength of the association learned during acquisition, or the final level of performance, influences the degree of renewal after extinction. The effect of the amount of training was modulated by the second factor, the degrees of generalization from the acquisition and extinction contexts to the test context. The third variable was acquisition training in multiple contexts, which was shown to strengthen ABC renewal. Methodological, theoretical, and practical implications are discussed.

Contextual control of appetite. Renewal of inhibited food-seeking behavior in sated rats after extinction

Obesity and overeating have become fundamental problems in modern society. This article studies the inhibition of food-seeking behavior, and how contextual cues can control it. Rats that had free food in the home cage nevertheless learned to lever press for sucrose or high-fat pellets in a distinctive context (a Skinner box). Lever pressing was then inhibited by extinction, in which lever presses no longer produced food. After extinction, inhibited responding was renewed when the rats were switched to a different context: in the new context, the rats lever-pressed again, and worked more for food when food was made available. These effects were observed when conditioning, extinction and testing occurred in contexts A, B, and A (respectively) or in A, A, and B. Thus, mere removal from the context in which food-seeking was inhibited initiated a return to food-seeking. The contextual control of extinction may help explain why food seeking and consumption seem so persistent.

Context Change and Associative Learning.

This article reviews the effects of changing the background context on performance in associative learning tasks in humans and animals. The findings are complementary and consistent over animal conditioning (Pavlovian and instrumental learning) and human predictive learning and memory paradigms. In many cases, a context change after learning can have surprisingly little disruptive influence on performance. Extinction, or retroactive interference treatments more generally, is more context-specific than the initial learning. Contexts become important if the participant is exposed to any of several treatments that involve prediction error, which may serve to increase attention to the context. Contexts also become important if they are given predictive or informational value. Studies of instrumental (operant) learning are further consistent with the idea that the context might also influence affordances that support voluntary actions. Context switch effects are not universal, but mainly occur when certain attention and perception processes can come into play. WIREs Cogn Sci 2013, 4:237-244. doi: 10.1002/wcs.1225 This article is categorized under: Psychology > Learning.

Mechanisms of renewal after the extinction of discriminated operant behavior.

Three experiments demonstrated, and examined the mechanisms that underlie, the renewal of extinguished discriminated operant behavior. In Experiment 1, rats were trained to perform 1 response (lever press or chain pull) in the presence of one discriminative stimulus (S; light or tone) in Context A, and to perform the other response in the presence of the other S in Context B. Next, each of the original S/response combinations was extinguished in the alternate context. When the S/response combinations were tested back in the context in which they had been trained, responding in the presence of S returned (an ABA renewal effect was observed). This renewal could not be due to differential context-reinforcer associations, suggesting instead that the extinction context inhibits either the response and/or the effectiveness of the S. Consistent with the latter mechanism, in Experiment 2, ABA renewal was still observed when both the extinction and renewal contexts inhibited the same response. However, in Experiment 3, previous extinction of the response in the renewing context (occasioned by a different S) reduced AAB renewal more than did extinction of the different response. Taken together, the results suggest at least 2 mechanisms of renewal after instrumental extinction. First, extinction performance is at least partly controlled by a direct inhibitory association that is formed between the context and the response. Second, in the discriminated operant procedure, extinction performance can sometimes be partly controlled by a reduction in the effectiveness of the S in the extinction context. Renewal of discriminated operant behavior can be produced by a release from either of these forms of inhibition.