Christopher A. Podlesnik

Behavioral momentum and relapse of extinguished operant responding.

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.

Extinction, relapse, and behavioral momentum.

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a discriminative stimulus depends upon the baseline rate or magnitude of reinforcement associated with that stimulus (i.e., the Pavlovian stimulus-reinforcer relation). Recently, we have shown that relapse of operant behavior in reinstatement, resurgence, and context renewal preparations also is a function of baseline stimulus-reinforcer relations. In this paper we present new data examining the role of baseline stimulus-reinforcer relations on resistance to extinction and relapse using a variety of baseline training conditions and relapse operations. Furthermore, we evaluate the adequacy of a behavioral momentum based model in accounting for the results. The model suggests that relapse occurs as a result of a decrease in the disruptive impact of extinction precipitated by a change in circumstances associated with extinction, and that the degree of relapse is a function of the pre-extinction baseline Pavlovian stimulus-reinforcer relation. Across experiments, relative resistance to extinction and relapse were greater in the presence of stimuli associated with more favorable conditions of reinforcement and were positively related to one another. In addition, the model did a good job in accounting for these effects. Thus, behavioral momentum theory may provide a useful quantitative approach for characterizing how differential reinforcement conditions contribute to relapse of operant behavior.

Response-Reinforcer Relations and Resistance to Change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.

The effects of nociceptin/orphanin FQ receptor agonist Ro 64-6198 and diazepam on antinociception and remifentanil self-administration in rhesus monkeys

RationaleThe synthetic nonpeptide NOP (nociceptin/orphanin FQ peptide) receptor agonist Ro 64-6198 produces antinociception in rhesus monkeys. In rodents, it has much more variable effects on pain responses, but has response rate-increasing effects on punished operant behavior and decreases drug reward.ObjectivesThe aim of this study was to compare Ro 64-6198 with the benzodiazepine diazepam in tests of analgesia, drug self-administration, and response-increasing effects in rhesus monkeys.ResultsRo 64-6198 (0.001–0.01xa0mg/kg, i.v.) produced antinociception against an acute noxious stimulus (50°C water) in the absence of sedation, whereas diazepam (0.32–3.2xa0mg/kg, i.v.) did not have analgesic effects without sedation. Diazepam (1.0–5.6xa0mg/kg, i.v.) and the largest dose of Ro 64-6198 (0.32xa0mg/kg, i.v.) decreased lever pressing maintained by intravenous self-administration of the mu-opioid agonist, remifentanil, but neither effect could be distinguished from sedative effects. Although neither drug consistently increased responding during nonreinforcement, such effects were observed more frequently following diazepam administration. The effects of Ro 64-6198 on lever pressing were blocked by the NOP-receptor antagonist, J-113397, but not by the benzodiazepine antagonist, flumazenil.ConclusionsThese findings suggest that the effects of Ro 64-6198 on operant lever pressing are mediated by NOP receptors and that larger doses are required to impact operant behavior when compared directly with those that produce antinociception. Therefore, the present findings support previous literature suggesting NOP receptors are a viable target for pain management.

Divided Attention Performance and the Matching Law

This experiment examined the applicability of the generalized matching law to changes in divided-attention performance produced by variations in relative reinforcement rate. Pigeons were exposed to a delayed matching-to-sample procedure in which compound samples (color 1 line orientation) and element comparisons (two colors or two line orientations) were used. The relative rate of reinforcement for accurate matches on the two types of comparison trials was varied. Variations in relative accuracy with changes in relative reinforcement were well described by the generalized matching law. We suggest that the bias and sensitivity parameters of the generalized matching law may measure the contributions of sensory activation and stimulus pertinence, respectively, to divided-attention performance. Additional work using similar analyses might contribute to a more general quantitative account of how differential consequences contribute to decision processes involved in the allocation of attention.

Quantitative Analyses of Observing and Attending

We review recent experiments examining whether simple models of the allocation and persistence of operant behavior are applicable to attending. In one series of experiments, observing responses of pigeons were used as an analog of attending. Maintenance of observing is often attributed to the conditioned reinforcing effects of a food-correlated stimulus (i.e., S+), so these experiments also may inform our understanding of conditioned reinforcement. Rates and allocations of observing were governed by rates of food or S+ delivery in a manner consistent with the matching law. Resistance to change of observing was well described by behavioral momentum theory only when rates of primary reinforcement in the context were considered. Rate and value of S+ deliveries did not affect resistance to change. Thus, persistence of attending to stimuli appears to be governed by primary reinforcement rates in the training context rather than conditioned reinforcing effects of the stimuli. An additional implication of these findings is that conditioned reinforcers may affect response rates through some mechanism other than response-strengthening. In a second series of experiments, we examined the applicability of the matching law to the allocation of attending to the elements of compound stimuli in a divided-attention task. The generalized matching law described performance well, and sensitivity to relative reinforcement varied with sample duration. The bias and sensitivity terms of the generalized matching law may provide measures of stimulus-driven and goal-driven control of divided attention. Further application of theories of operant behavior to performance on attention tasks may provide insights into what is referred to variously as endogenous, top-down, or goal-directed control of attention.

Divided attention and the matching law: Sample duration affects sensitivity to reinforcement allocation

Previously, we have shown that changes in pigeons’ divided attention performance resulting from changes in relative reinforcement are well described by the generalized matching law. In the present experiment, we examined whether sensitivity of performance to variations in relative reinforcement would be dependent upon sample duration. Pigeons responded on a delayed matching-to-sample procedure with compound samples (color 1 line orientation) and element comparison stimuli (two colors or two line orientations). Relative reinforcement for accurate matches on the two types of comparison trials varied across conditions. Sample duration was short (i.e., 0.75 sec) for half of the trials in a session and longer (i.e., 2.25 sec) for the other half. Sensitivity of accuracy to changes in relative reinforcement was greater with the longer sample than with the shorter sample, suggesting that differential reinforcement alters the allocation of attending to the elements of compound stimuli. Continued examination of the applicability of well-established theories of goal-directed behavior to the allocation of attention may provide further insights into what is variously referred to as goal-directed, voluntary, endogenous, or top-down control of attention.

A choice procedure to assess the aversive effects of drugs in rodents.

The goal of this series of experiments was to develop an operant choice procedure to examine rapidly the punishing effects of intravenous drugs in rats. First, the cardiovascular effects of experimenter-administered intravenous histamine, a known aversive drug, were assessed to determine a biologically active dose range. Next, rats responded on each of two levers with concurrently available fixed-ratio 1 schedules of food reinforcement. Intravenous histamine was delivered along with food when responses were made on one of the options, and the lever on which both food and histamine were contingent was switched on a regular basis. A dose of 1.0xa0mg/kg/inj of histamine was effective in moving responding to the alternate lever, whereas saline, 0.1, or 0.3xa0mg/kg/inj of histamine were not. Histamine injections produced reliable selection of the alternate lever when they were presented on the same lever for three consecutive sessions, but not when they were switched between levers on each session. In addition, histamine produced greater selection of the alternate lever when it was presented with shorter intertrial interval durations. These findings indicate that, with appropriate parameters, the aversive effects of histamine and perhaps other drugs can be established rapidly using a concurrent choice procedure.

Repeated extinction and reversal learning of an approach response supports an arousal-mediated learning model

We assessed the effects of repeated extinction and reversals of two conditional stimuli (CS+/CS-) on an appetitive conditioned approach response in rats. Three results were observed that could not be accounted for by a simple linear operator model such as the one proposed by Rescorla and Wagner (1972): (1) responding to a CS- declined faster when a CS+ was simultaneously extinguished; (2) reacquisition of pre-extinction performance recovered rapidly within one session; and (3) reversal of CS+/CS- contingencies resulted in a more rapid recovery to the current CS- (former CS+) than the current CS+, accompanied by a slower acquisition of performance to the current CS+. An arousal parameter that mediates learning was introduced to a linear operator model to account for these effects. The arousal-mediated learning model adequately fit the data and predicted data from a second experiment with different rats in which only repeated reversals of CS+/CS- were assessed. According to this arousal-mediated learning model, learning is accelerated by US-elicited arousal and it slows down in the absence of US. Because arousal varies faster than conditioning, the model accounts for the decline in responding during extinction mainly through a reduction in arousal, not a change in learning. By preserving learning during extinction, the model is able to account for relapse effects like rapid reacquisition, renewal, and reinstatement.

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers.

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.