Andrew R. Craig

Resurgence as Choice.

Resurgence is typically defined as an increase in a previously extinguished target behavior when a more recently reinforced alternative behavior is later extinguished. Some treatments of the phenomenon have suggested that it might also extend to circumstances where either the historic or more recently reinforced behavior is reduced by other non-extinction related means (e.g., punishment, decreases in reinforcement rate, satiation, etc.). Here we present a theory of resurgence suggesting that the phenomenon results from the same basic processes governing choice. In its most general form, the theory suggests that resurgence results from changes in the allocation of target behavior driven by changes in the values of the target and alternative options across time. Specifically, resurgence occurs when there is an increase in the relative value of an historically effective target option as a result of a subsequent devaluation of a more recently effective alternative option. We develop a more specific quantitative model of how extinction of the target and alternative responses in a typical resurgence paradigm might produce such changes in relative value across time using a temporal weighting rule. The example model does a good job in accounting for the effects of reinforcement rate and related manipulations on resurgence in simple schedules where Behavioral Momentum Theory has failed. We also discuss how the general theory might be extended to other parameters of reinforcement (e.g., magnitude, quality), other means to suppress target or alternative behavior (e.g., satiation, punishment, differential reinforcement of other behavior), and other factors (e.g., non- contingent versus contingent alternative reinforcement, serial alternative reinforcement, and multiple schedules).

Behavioral momentum theory fails to account for the effects of reinforcement rate on resurgence.

The behavioral-momentum model of resurgence predicts reinforcer rates within a resurgence preparation should have three effects on target behavior. First, higher reinforcer rates in baseline (Phase 1) produce more persistent target behavior during extinction plus alternative reinforcement. Second, higher rate alternative reinforcement during Phase 2 generates greater disruption of target responding during extinction. Finally, higher rates of either reinforcement source should produce greater responding when alternative reinforcement is suspended in Phase 3. Recent empirical reports have produced mixed results in terms of these predictions. Thus, the present experiment further examined reinforcer-rate effects on persistence and resurgence. Rats pressed target levers for high-rate or low-rate variable-interval food during Phase 1. In Phase 2, target-lever pressing was extinguished, an alternative nose-poke became available, and nose-poking produced either high-rate variable-interval, low-rate variable-interval, or no (an extinction control) alternative reinforcement. Alternative reinforcement was suspended in Phase 3. For groups that received no alternative reinforcement, target-lever pressing was less persistent following high-rate than low-rate Phase-1 reinforcement. Target behavior was more persistent with low-rate alternative reinforcement than with high-rate alternative reinforcement or extinction alone. Finally, no differences in Phase-3 responding were observed for groups that received either high-rate or low-rate alternative reinforcement, and resurgence occurred only following high-rate alternative reinforcement. These findings are inconsistent with the momentum-based model of resurgence. We conclude this model mischaracterizes the effects of reinforcer rates on persistence and resurgence of operant behavior.

Quantitative models of persistence and relapse from the perspective of behavioral momentum theory: Fits and misfits☆

We review quantitative accounts of behavioral momentum theory (BMT), its application to clinical treatment, and its extension to post-intervention relapse of target behavior. We suggest that its extension can account for relapse using reinstatement and renewal models, but that its application to resurgence is flawed both conceptually and in its failure to account for recent data. We propose that the enhanced persistence of target behavior engendered by alternative reinforcers is limited to their concurrent availability within a distinctive stimulus context. However, a failure to find effects of stimulus-correlated reinforcer rates in a Pavlovian-to-Instrumental Transfer (PIT) paradigm challenges even a straightforward Pavlovian account of alternative reinforcer effects. BMT has been valuable in understanding basic research findings and in guiding clinical applications and accounting for their data, but alternatives are needed that can account more effectively for resurgence while encompassing basic data on resistance to change as well as other forms of relapse.

Effects of signaled and unsignaled alternative reinforcement on persistence and relapse in children and pigeons.

Three experiments explored the impact of different reinforcer rates for alternative behavior (DRA) on the suppression and post-DRA relapse of target behavior, and the persistence of alternative behavior. All experiments arranged baseline, intervention with extinction of target behavior concurrently with DRA, and post-treatment tests of resurgence or reinstatement, in two- or three-component multiple schedules. Experiment 1, with pigeons, arranged high or low baseline reinforcer rates; both rich and lean DRA schedules reduced target behavior to low levels. When DRA was discontinued, the magnitude of relapse depended on both baseline reinforcer rate and the rate of DRA. Experiment 2, with children exhibiting problem behaviors, arranged an intermediate baseline reinforcer rate and rich or lean signaled DRA. During treatment, both rich and lean DRA rapidly reduced problem behavior to low levels, but post-treatment relapse was generally greater in the DRA-rich than the DRA-lean component. Experiment 3, with pigeons, repeated the low-baseline condition of Experiment 1 with signaled DRA as in Experiment 2. Target behavior decreased to intermediate levels in both DRA-rich and DRA-lean components. Relapse, when it occurred, was directly related to DRA reinforcer rate as in Experiment 2. The post-treatment persistence of alternative behavior was greater in the DRA-rich component in Experiment 1, whereas it was the same or greater in the signaled-DRA-lean component in Experiments 2 and 3. Thus, infrequent signaled DRA may be optimal for effective clinical treatment.

Higher rate alternative non-drug reinforcement produces faster suppression of cocaine seeking but more resurgence when removed

Relapse following removal of an alternative source of reinforcement introduced during extinction of a target behavior is called resurgence. This form of relapse may be related to relapse of drug taking following loss of alternative non-drug reinforcement in human populations. Laboratory investigations of factors mediating resurgence with food-maintained behavior suggest higher rates of alternative reinforcement produce faster suppression of target behavior but paradoxically generate more relapse when alternative reinforcement is discontinued. At present, it is unknown if a similar effect occurs when target behavior is maintained by drug reinforcement and the alternative is a non-drug reinforcer. In the present experiment three groups of rats were trained to lever press for infusions of cocaine during baseline. Next, during treatment, cocaine reinforcement was suspended and an alternative response was reinforced with either high-rate, low-rate, or no alternative food reinforcement. Finally, all reinforcement was suspended to test for relapse of cocaine seeking. Higher rate alternative reinforcement produced faster elimination of cocaine seeking than lower rates or extinction alone, but when treatment was suspended resurgence of cocaine seeking occurred following only high-rate alternative reinforcement. Thus, although higher rate alternative reinforcement appears to more effectively suppress drug seeking, should it become unavailable, it can have the unfortunate effect of increasing relapse.

Behavioral momentum and accumulation of mass in multiple schedules

Behavioral momentum theory suggests that the relation between a discriminative-stimulus situation and reinforcers obtained in that context (i.e., the Pavlovian stimulus-reinforcer relation) governs persistence of operant behavior. Within the theory, a mass-like aspect of behavior has been shown to be a power function of predisruption reinforcement rates. Previous investigations of resistance to change in multiple schedules, however, have been restricted to examining response persistence following protracted periods of stability in reinforcer rates within a discriminative situation. Thus, it is unclear how long a stimulus-reinforcer relation must be in effect prior to disruption in order to affect resistance to change. The present experiment examined resistance to change of pigeons key pecking following baseline conditions where reinforcer rates that were correlated with discriminative-stimulus situations changed. Across conditions, one multiple-schedule component arranged either relatively higher rates or lower rates of variable-interval food delivery, while the other component arranged the opposite rate. These schedules alternated between multiple-schedule components across blocks of sessions such that reinforcer rates in the components were held constant for 20, 5, 3, 2, or 1 session(s) between alternations. Resistance to extinction was higher in the component that most recently was associated with higher rates of food delivery in all conditions except when schedules alternated daily or every other day. These data suggest that resistance to change in multiple schedules is related to recently experienced reinforcer rates but only when multiple-schedule components are associated with specific reinforcer rates for several sessions.

EXPERIENCE WITH DYNAMIC REINFORCEMENT RATES DECREASES RESISTANCE TO EXTINCTION

The ability of organisms to detect reinforcer-rate changes in choice preparations is positively related to two factors: the magnitude of the change in rate and the frequency with which rates change. Gallistel (2012) suggested similar rate-detection processes are responsible for decreases in responding during operant extinction. Although effects of magnitude of change in reinforcer rate on resistance to extinction are well known (e.g., the partial-reinforcement-extinction effect), effects of frequency of changes in rate prior to extinction are unknown. Thus, the present experiments examined whether frequency of changes in baseline reinforcer rates impacts resistance to extinction. Pigeons pecked keys for variable-interval food under conditions where reinforcer rates were stable and where they changed within and between sessions. Overall reinforcer rates between conditions were controlled. In Experiment 1, resistance to extinction was lower following exposure to dynamic reinforcement schedules than to static schedules. Experiment 2 showed that resistance to presession feeding, a disruptor that should not involve change-detection processes, was unaffected by baseline-schedule dynamics. These findings are consistent with the suggestion that change detection contributes to extinction. We discuss implications of change-detection processes for extinction of simple and discriminated operant behavior and relate these processes to the behavioral-momentum based approach to understanding extinction.

Resurgence and alternative-reinforcer magnitude

Resurgence is defined as an increase in the frequency of a previously reinforced target response when an alternative source of reinforcement is suspended. Despite an extensive body of research examining factors that affect resurgence, the effects of alternative-reinforcer magnitude have not been examined. Thus, the present experiments aimed to fill this gap in the literature. In Experiment 1, rats pressed levers for single-pellet reinforcers during Phase 1. In Phase 2, target-lever pressing was extinguished, and alternative-lever pressing produced either five-pellet, one-pellet, or no alternative reinforcement. In Phase 3, alternative reinforcement was suspended to test for resurgence. Five-pellet alternative reinforcement produced faster elimination and greater resurgence of target-lever pressing than one-pellet alternative reinforcement. In Experiment 2, effects of decreasing alternative-reinforcer magnitude on resurgence were examined. Rats pressed levers and pulled chains for six-pellet reinforcers during Phases 1 and 2, respectively. In Phase 3, alternative reinforcement was decreased to three pellets for one group, one pellet for a second group, and suspended altogether for a third group. Shifting from six-pellet to one-pellet alternative reinforcement produced as much resurgence as suspending alternative reinforcement altogether, while shifting from six pellets to three pellets did not produce resurgence. These results suggest that alternative-reinforcer magnitude has effects on elimination and resurgence of target behavior that are similar to those of alternative-reinforcer rate. Thus, both suppression of target behavior during alternative reinforcement and resurgence when conditions of alternative reinforcement are altered may be related to variables that affect the value of the alternative-reinforcement source.

Resurgence of sucrose and cocaine seeking in free-feeding rats.

Resurgence is relapse of an extinguished operant response following the removal of alternative reinforcement. In animal models of resurgence to date, rats have been food deprived and food is used as the source of alternative reinforcement. Thus, when the alternative reinforcer is removed, the only remaining source of food during experimental sessions is no longer available. Acute food deprivation is known to produce reinstatement of drug seeking, thus such deprivation has been suggested a potential mechanism of resurgence. The present experiments examined whether resurgence of sucrose and cocaine seeking could be obtained with rats that were not food deprived. Free-feeding rats were trained to press a lever for either sucrose (Experiment 1) or cocaine infusions (Experiment 2). Next, lever pressing was extinguished and an alternative response (nose poking) was reinforced with sucrose. When nose poking was also placed on extinction, resurgence of both sucrose and cocaine seeking was observed. Thus, resurgence of both sucrose and cocaine seeking can be obtained in rats that are not food restricted and it appears unlikely that an acute hunger state is responsible for resurgence. In addition, the present procedures for studying resurgence in the absence of interpretive complexities introduced by the use of food-deprivation may prove useful for further investigations of the neurobiological mechanisms of resurgence.

Stimuli previously associated with reinforcement mitigate resurgence

Resurgence refers to the recurrence of an extinguished target behavior following subsequent suspension of alternative reinforcement. Delivery of reinforcers during extinction of alternative behavior has been shown to mitigate resurgence. The present experiment aimed to determine whether delivering stimuli associated with reinforcers during resurgence testing similarly mitigates resurgence. Three groups of rats pressed target levers for food according to variable-interval 15-s schedules during Phase 1. In Phase 2, lever pressing was extinguished, and an alternative nose-poke response produced alternative reinforcement according to a variable-interval 15-s schedule. Food reinforcement was always associated with illumination of the food aperture and an audible click from the pellet dispenser during Phases 1 and 2. Phase 3 treatments differed between groups. For one group, nose poking continued to produce food and food-correlated stimuli. Both of these consequences were suspended for a second group. Finally, nose poking produced food-correlated stimuli but not food for a third group. Target-lever pressing resurged in the group that received no consequences and in the group that received only food-correlated stimuli for nose poking. Resurgence, however, was smaller for the group that received food-correlated stimuli than for the group that received no consequences for nose poking. Target-lever pressing did not increase between phases in the group that continued to receive food and associated stimuli. Thus, delivery of stimuli associated with food reinforcement after suspension of food reduced but did not eliminate resurgence of extinguished lever pressing. These findings contribute to potential methodologies for preventing relapse of extinguished problem behavior in clinical settings.