Eric A. Thrailkill

Contextual control of instrumental actions and habits.

After a relatively small amount of training, instrumental behavior is thought to be an action under the control of the motivational status of its goal or reinforcer. After more extended training, behavior can become habitual and insensitive to changes in reinforcer value. Recently, instrumental responding has been shown to weaken when tested outside of the training context. The present experiments compared the sensitivity of instrumental responding in rats with a context switch after training procedures that might differentially generate actions or habits. In Experiment 1, lever pressing was decremented in a new context after either short, medium, or long periods of training on either random-ratio or yoked random-interval reinforcement schedules. Experiment 2 found that more minimally trained responding was also sensitive to a context switch. Moreover, Experiment 3 showed that when the goal-directed component of responding was removed by devaluing the reinforcer, the residual responding that remained was still sensitive to the change of context. Goal-directed responding, in contrast, transferred across contexts. Experiment 4 then found that after extensive training, a habit that was insensitive to reinforcer devaluation was still decremented by a context switch. Overall, the results suggest that a context switch primarily influences instrumental habit rather than action. In addition, even a response that has received relatively minimal training may have a habit component that is insensitive to reinforcer devaluation but sensitive to the effects of a context switch.

Occasion setting, inhibition, and the contextual control of extinction in Pavlovian and instrumental (operant) learning.

An occasion setter is a stimulus that modulates the ability of another stimulus to control behavior. A rich history of experimental investigation has identified several important properties that define occasion setters and the conditions that give rise to occasion setting. In this paper, we first consider the basic hallmarks of occasion setting in Pavlovian conditioning. We then review research that has examined the mechanisms underlying the crucial role of context in Pavlovian and instrumental extinction. In Pavlovian extinction, evidence suggests that the extinction context can function as a negative occasion setter whose role is to disambiguate the current meaning of the conditioned stimulus; the conditioning context can also function as a positive occasion setter. In operant extinction, in contrast, the extinction context may directly inhibit the response, and the conditioning context can directly excite it. We outline and discuss the key results supporting these distinctions.

Extinction of Chained Instrumental Behaviors: Effects of Procurement Extinction on Consumption Responding

Instrumental behavior often consists of sequences or chains of responses that minimally include procurement behaviors that enable subsequent consumption behaviors. In such chains, behavioral units are linked by access to one another and eventually to a primary reinforcer, such as food or a drug. The present experiments examined the effects of extinguishing procurement responding on consumption responding after training of a discriminated heterogeneous instrumental chain. Rats learned to make a procurement response (e.g., pressing a lever) in the presence of a distinctive discriminative stimulus; making that response led to the presentation of a second discriminative stimulus that set the occasion for a consumption response (e.g., pulling a chain), which then produced a food-pellet reinforcer. Experiment 1 showed that extinction of either the full procurement-consumption chain or procurement alone weakened the consumption response tested in isolation. Experiment 2 replicated the procurement extinction effect and further demonstrated that the opportunity to make the procurement response, as opposed to simple exposure to the procurement stimulus alone, was required. In Experiment 3, rats learned 2 distinct discriminated heterogeneous chains; extinction of 1 procurement response specifically weakened the consumption response that had been associated with it. The results suggest that learning to inhibit the procurement response may produce extinction of consumption responding through mediated extinction. The experiments suggest the importance of an associative analysis of instrumental behavior chains. (PsycINFO Database Record

Extinction of chained instrumental behaviors: Effects of consumption extinction on procurement responding

Operant behavior is typically organized into sequences of responses that eventually lead to a reinforcer. Response elements can be categorized as those that directly lead to reward consumption (i.e., a consumption response) and those that lead to the opportunity to make the consumption response (i.e., a procurement response). These responses often differ topographically and in terms of the discriminative stimuli that set the occasion for them. We have recently shown that extinction of the procurement response acts to weaken the specific associated consumption response, and that active inhibition of the procurement response is required for this effect. To expand the analysis of the associative structure of chains, in the present experiments we asked the reverse question: whether extinction of consumption behavior results in a decrease in the associated procurement response in a discriminated heterogeneous chain. In Experiment 1, extinction of consumption alone led to an attenuation of the associated procurement response only when rats were allowed to make the consumption response in extinction. Exposure to the consumption stimulus alone was not sufficient to produce weakened procurement responding. In Experiment 2, rats learned two distinct heterogeneous chains, and extinction of one consumption response specifically weakened the procurement response associated with it. The results add to the evidence suggesting that rats learn a highly specific associative structure in behavior chains, emphasizing the role of learning response inhibition in extinction.

Pituitary Adenylate Cyclase-Activating Peptide in the Bed Nucleus of the Stria Terminalis Mediates Stress-Induced Reinstatement of Cocaine Seeking in Rats

Stressors often contribute to difficulties in maintaining behavior change following a period of abstinence, and may play a significant role in drug relapse. The activation of pituitary adenylate cyclase-activating peptide (PACAP) systems in the bed nucleus of the stria terminalis (BNST) mediates many consequences of chronic stressor exposure. Here we ask whether PACAP is also involved in producing reinstatement in a model of stress-induced relapse to drug taking. Rats self-administered cocaine for 1 h daily over 10 days that was followed by 20 days of extinction training in which lever pressing no longer produced cocaine. In experiment 1, quantitative PCR (qPCR) was performed at several stages to determine transcript levels of PACAP and corresponding receptors. Reinstatement of cocaine seeking was then tested after footshock exposure in different groups of rats that were pretreated with vehicle solution, a PAC1 receptor antagonist (experiment 2), or a PACAP agonist (experiment 3) without footshock. In experiment 1, cocaine self-administration increased BNST PACAP transcript levels similar to what we have previously reported with chronic stress. In experiment 2, intra-BNST infusions of the PAC1/VPAC2 antagonist, PACAP 6-38, prevented footshock-induced reinstatement of extinguished cocaine seeking. In experiment 3, intra-BNST PACAP infusion reinstated previously extinguished cocaine-seeking behavior in the absence of footshock. Cocaine self-administration elevated BNST PACAP, and BNST PACAP receptor activation was necessary and sufficient for stress-induced reinstatement of cocaine seeking. These data suggest that BNST PACAP systems may be viable targets for relapse prevention.

Factors that influence the persistence and relapse of discriminated behavior chains

Behavior chains are composed of sequences of behaviors that minimally include procurement and then consumption. This review surveys recent research from this laboratory that has examined the properties of discriminated heterogeneous behavior chains. In contrast to another review (Thrailkill and Bouton, 2016a), it discusses work examining what makes chained behavior persistent, and what makes it relapse. Results suggest that responses in a discriminated heterogeneous behavior chain may become associated, so that extinction of either one reduces the strength of the other. Evidence also suggests that the goal of the first (procurement) response may be the next (consumption) response (rather than the upcoming discriminative stimulus, a putative conditioned reinforcer, or the primary reinforcer at the end of the chain). Further studies suggest that methods that promote generalization across acquisition and extinction (partial reinforcement and delivery of noncontingent reinforcers during extinction) lead to greater persistence of the procurement response. A third set of studies analyzed the contextual control and relapse of chained behaviors. The context controls both the acquisition and extinction of chained behaviors. In addition, a separately-extinguished consumption response is renewed when returned to the context of the chain. The research expands our general understanding of the learning processes that govern instrumental behavior as well as our understanding of chains.

Extinction and the associative structure of heterogeneous instrumental chains.

Drug abuse, overeating, and smoking are all examples of instrumental behaviors that often involve chains or sequences of behavior. A behavior chain is minimally composed of a procurement response that is required in order for a subsequent consumption response to be reinforced. Despite the translational importance of behavior chains, few studies have attempted to understand what binds them together and takes them apart. This article surveys the development of the heterogeneous instrumental chain method and introduces recent findings that have used extinction to analyze the associative content of (what is learned in) the chain. Chained responses that are occasion-set by their own discriminative stimuli may be directly associated; extinction of the procurement response weakens its associated consumption response, and extinction of the consumption response weakens its associated procurement response. Extinction itself involves learning to inhibit the response. Extinguished chained responses are subject to renewal when they are tested either back in the acquisition context or in a new context. In addition, a consumption response that is extinguished outside its chain is renewed when returned to the context of the preceding response in the chain. Research on heterogeneous behavior chains can provide important insights into an important but often overlooked aspect of instrumental learning.

Effects of outcome devaluation on instrumental behaviors in a discriminated heterogeneous chain.

Operant behavior often takes place in a sequence, or chain, of linked responses that lead to a reinforcer. We have recently studied rats performing a discriminated heterogeneous behavior chain that involves the presentation of a discriminative stimulus (e.g., a panel light) to set the occasion for a procurement behavior (e.g., a lever press) that leads to a second stimulus (e.g., a second panel light) that indicates that a consumption response (e.g., a chain pull) will be reinforced. The present study assessed the role played by a representation of the reinforcer in controlling the performance of the responses in this chain. After acquisition of the chain, rats received a reinforcer devaluation treatment in the form of repeated paired, or unpaired, presentations of the food-pellet outcome and lithium-chloride illness. Once paired rats came to reject the pellets, half the animals in each group were tested on procurement, and the other half were tested on consumption. Neither response was affected by the outcome devaluation treatment, although entries into the food cup were suppressed. Combined with other results, the findings suggest that the “goal” for goal-directed procurement responding in a discriminated heterogeneous chain may be the consumption response rather than the primary reinforcer.

Preventing relapse after incentivized choice treatment: A laboratory model☆

Two experiments with rats examined relapse of an operant behavior that occurred after the behavior was suppressed by reinforcing (incentivizing) an alternative behavior. In the first phase, a target response (R1) was reinforced. In a treatment phase, R1 was still reinforced, but a new response (R2) was introduced and associated with a larger reinforcer. As in human contingency management treatments, incentivizing R2 this way was effective at suppressing R1. However, when R2s reinforcement was discontinued, there was a robust and immediate relapse to R1. Experiment 1 found that the strength of R1 during relapse testing was not different from that seen in a no treatment control. Experiment 2 found that relapse could nevertheless be reduced by presenting reinforcers not contingent on responding during the test. Either the reinforcer for R1 or the reinforcer for R2 (which were qualitatively different types of food pellets) were effective. The experiments introduce a laboratory method for studying relapse and how to prevent it after contingency management treatments, and suggest at least one treatment that discourages relapse. The incentivized choice paradigm differs from other models of relapse of operant behavior (e.g., resurgence, renewal, reinstatement) in that it does not focus on the return of behaviors that are inhibited by extinction.

Increasing the persistence of a heterogeneous behavior chain: Studies of extinction in a rat model of search behavior of working dogs

Dogs trained to search for contraband perform a chain of behavior in which they first search for a target and then make a separate response that indicates to the trainer that they have found one. The dogs often conduct multiple searches without encountering a target and receiving the reinforcer (i.e., no contraband is present). Understanding extinction (i.e., the decline in work rate when reinforcers are no longer encountered) may assist in training dogs to work in conditions where targets are rare. We therefore trained rats on a search-target behavior chain modeled on the search behavior of working dogs. A discriminative stimulus signaled that a search response (e.g., chain pull) led to a second stimulus that set the occasion for a target response (e.g., lever press) that was reinforced by a food pellet. In Experiment 1 training with longer search durations and intermittent (partial) reinforcement of searching (i.e. some trials had no target present) both led to more persistent search responding in extinction. The loss of search behavior in extinction was primarily dependent on the number of non-reinforced searches rather than time searching without reinforcement. In Experiments 2 and 3, delivery of non-contingent reinforcers during extinction increased search persistence provided they had also been presented during training. Thus, results with rats suggest that the persistence of working dog performance (or chained behavior generally) may be improved by training with partial reinforcement of searching and non-contingent reinforcement during both training and work (extinction).