Timothy A. Shahan

Behavioral momentum and relapse of extinguished operant responding.

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.

A MODEL OF RESURGENCE BASED ON BEHAVIORAL MOMENTUM THEORY

Resurgence is the reappearance of an extinguished behavior when an alternative behavior reinforced during extinction is subsequently placed on extinction. Resurgence is of particular interest because it may be a source of relapse to problem behavior following treatments involving alternative reinforcement. In this article we develop a quantitative model of resurgence based on the augmented model of extinction provided by behavioral momentum theory. The model suggests that alternative reinforcement during extinction of a target response acts as both an additional source of disruption during extinction and as a source of reinforcement in the context that increases the future strength of the target response. The model does a good job accounting for existing data in the resurgence literature and makes novel and testable predictions. Thus, the model appears to provide a framework for understanding resurgence and serves to integrate the phenomenon into the existing theoretical account of persistence provided by behavioral momentum theory. In addition, we discuss some potential implications of the model for further development of behavioral momentum theory.

Behavioral momentum theory: equations and applications.

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings.

Extinction, relapse, and behavioral momentum.

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a discriminative stimulus depends upon the baseline rate or magnitude of reinforcement associated with that stimulus (i.e., the Pavlovian stimulus-reinforcer relation). Recently, we have shown that relapse of operant behavior in reinstatement, resurgence, and context renewal preparations also is a function of baseline stimulus-reinforcer relations. In this paper we present new data examining the role of baseline stimulus-reinforcer relations on resistance to extinction and relapse using a variety of baseline training conditions and relapse operations. Furthermore, we evaluate the adequacy of a behavioral momentum based model in accounting for the results. The model suggests that relapse occurs as a result of a decrease in the disruptive impact of extinction precipitated by a change in circumstances associated with extinction, and that the degree of relapse is a function of the pre-extinction baseline Pavlovian stimulus-reinforcer relation. Across experiments, relative resistance to extinction and relapse were greater in the presence of stimuli associated with more favorable conditions of reinforcement and were positively related to one another. In addition, the model did a good job in accounting for these effects. Thus, behavioral momentum theory may provide a useful quantitative approach for characterizing how differential reinforcement conditions contribute to relapse of operant behavior.

Conditioned Reinforcement and Response Strength

Stimuli associated with primary reinforcers appear themselves to acquire the capacity to strengthen behavior. This paper reviews research on the strengthening effects of conditioned reinforcers within the context of contemporary quantitative choice theories and behavioral momentum theory. Based partially on the finding that variations in parameters of conditioned reinforcement appear not to affect response strength as measured by resistance to change, long-standing assertions that conditioned reinforcers do not strengthen behavior in a reinforcement-like fashion are considered. A signposts or means-to-an-end account is explored and appears to provide a plausible alternative interpretation of the effects of stimuli associated with primary reinforcers. Related suggestions that primary reinforcers also might not have their effects via a strengthening process are explored and found to be worthy of serious consideration.

Effects of high, low, and thinning rates of alternative reinforcement on response elimination and resurgence.

A common treatment for operant problem behavior is alternative reinforcement. When alternative reinforcement is removed or reduced, however, resurgence of the target behavior can occur. Shahan and Sweeney (2011) developed a quantitative model of resurgence based on behavioral momentum theory that suggests higher rates of alternative reinforcement result in faster response elimination and greater resurgence when removed, whereas lower rates of alternative reinforcement result in slower response elimination but are followed by less resurgence. Thus, the present study was designed to examine whether faster target response elimination and less resurgence could be achieved by beginning with a high rate of alternative reinforcement and gradually thinning it such that a low rate is ultimately removed during a simulated treatment lapse. Results showed that high rates of alternative reinforcement were more effective than low or thinning rates at target response suppression but resulted in resurgence when discontinued. Low and thinning rates, on the other hand, were less effective at response suppression but target responding did not increase when alternative reinforcement was discontinued. The quantitative model cannot currently account for the finding that lower-rate alternative reinforcement may not effectively disrupt behavior relative to an extinction only control. Relative advantages of high, low, thinning, or no alternative reinforcement are discussed with respect to suppression of target response rate during treatment, resurgence when alternative reinforcement is removed, and alternative response persistence, while taking into account differences between this animal model and modern applied behavior analytic treatments.

Novelty, Stimulus Control, and Operant Variability

Although behavior analysis has been criticized for failure to account for response novelty, many common behavior-analytic concepts and processes (e.g., selectionism, the operant, reinforcement, and stimulus control) assume variability both in the environment and in behavior. The importance of the relation between variability and novelty, particularly for verbal behavior, is discussed, and concepts used to account for novel behavior are examined. Experimental findings also are reviewed that suggest that variability in behavior can come under discriminative control, and these findings are applied to describe novel instances of behavior that may arise during problem solving. We conclude that variations provided and selected by the terms of the three-term contingency are powerful means for understanding novel behavior.

Loss of Alternative Non-Drug Reinforcement Induces Relapse of Cocaine-Seeking in Rats: Role of Dopamine D1 Receptors

Animal models of relapse to drug seeking have focused primarily on relapse induced by exposure to drugs, drug-associated cues or contexts, and foot-shock stress. However, relapse in human drug abusers is often precipitated by loss of alternative non-drug reinforcement. The present experiment used a novel ‘resurgence’ paradigm to examine relapse to cocaine seeking of rats as a result of loss of an alternative source of non-drug reinforcement. Rats were first trained to press a lever for intravenous infusions of cocaine. Next, cocaine deliveries were omitted and food pellets were provided for an alternative nose-poke response. Once cocaine seeking was reduced to low levels, food pellets for the alternative response were also omitted. Cocaine seeking increased with the loss of the alternative non-drug reinforcer (ie, resurgence occurred) despite continued extinction conditions. The increase in cocaine seeking did not occur in another group of rats injected with SCH 23390 before the loss of the alternative reinforcer. These results suggest that removal of an alternative source of reinforcement may induce relapse of cocaine seeking and that the dopamine D1 receptor may have a role in this effect.

Behavioral momentum and resurgence: Effects of time in extinction and repeated resurgence tests

Resurgence is an increase in a previously extinguished operant response that occurs if an alternative reinforcement introduced during extinction is removed. Shahan and Sweeney (2011) developed a quantitative model of resurgence based on behavioral momentum theory that captures existing data well and predicts that resurgence should decrease as time in extinction and exposure to the alternative reinforcement increases. Two experiments tested this prediction. The data from Experiment 1 suggested that without a return to baseline, resurgence decreases with increased exposure to alternative reinforcement and to extinction of the target response. Experiment 2 tested the predictions of the model across two conditions, one with constant alternative reinforcement for five sessions, and the other with alternative reinforcement removed three times. In both conditions, the alternative reinforcement was removed for the final test session. Experiment 2 again demonstrated a decrease in relapse across repeated resurgence tests. Furthermore, comparably little resurgence was observed at the same time point in extinction in the final test, despite dissimilar previous exposures to alternative reinforcement removal. The quantitative model provided a good description of the observed data in both experiments. More broadly, these data suggest that increased exposure to extinction may be a successful strategy to reduce resurgence. The relationship between these data and existing tests of the effect of time in extinction on resurgence is discussed.

Response-Reinforcer Relations and Resistance to Change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.