John A. Nevin

Behavioral momentum and the Law of Effect

In the metaphor of behavioral momentum, the rate of a free operant in the presence of a discriminative stimulus is analogous to the velocity of a moving body, and resistance to change measures an aspect of behavior that is analogous to its inertial mass. An extension of the metaphor suggests that preference measures an analog to the gravitational mass of that body. The independent functions relating resistance to change and preference to the conditions of reinforcement may be construed as convergent measures of a single construct, analogous to physical mass, that represents the effects of a history of exposure to the signaled conditions of reinforcement and that unifies the traditionally separate notions of the strength of learning and the value of incentives. Research guided by the momentum metaphor encompasses the effects of reinforcement on response rate, resistance to change, and preference and has implications for clinical interventions, drug addiction, and self-control. In addition, its principles can be seen as a modern, quantitative version of Thorndikes (1911) Law of Effect, providing a new perspective on some of the challenges to his postulation of strengthening by reinforcement.

Behavioral contrast and behavioral momentum

In Experiment 1, 5 pigeons were trained to peck a key on multiple schedules of food reinforcement. The reinforcer rate was constant in 1 component and varied between conditions in the alternated component. In the constant component, steady-state response rate and its resistance to both prefeeding and extinction were inversely related to the reinforcer rate in the alternated component. Thus, resistance to both prefeeding and to extinction, like response rate, exhibits behavioral contrast

Behavioral momentum theory: equations and applications.

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings.

Resistance to extinction: Contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.

Measuring behavioral momentum

The metaphor of behavioral momentum proposes that when ongoing operant behavior is disrupted, changes in response rate are directly related to a force-like aspect of the disruptor and inversely proportional to behavioral mass. Several data sets suggest that differential resistance to change between the components of a multiple schedule satisfies the requirements of a ratio scale and is additive when different disruptors and different dimensions of reinforcement are combined. Differential resistance also provides a basis for scaling force in relation to rate of food presentation between components as a disruptor, and for scaling mass in relation to food rate within a component as a reinforcer. Preference in concurrent chains with terminal links identical to multiple-schedule components also meets the requirements of ratio-scale measurement, is additive when different dimensions of reinforcement are combined, and provides convergent measurement of behavioral mass.

Pavlovian determiners of behavioral momentum

Pigeons pecked keys on a six-component multiple schedule of food reinforcement arranged in successive pairs on three keys. On all three keys, the schedule in the first component was variable-interval 120 sec. On one key, the second component was variable-time 24 sec; on another, it was variable-time 120 sec; and on the third, extinction. Pairs of components were separated by timeout. In this arrangement, second-component stimuli were differentially correlated with food in their presence, and first-component stimuli were differentially correlated with second-component food in serial fashion. After baseline performance stabilized, resistance to change was assessed by prefeeding, by discontinuing food, and by presenting free food at random times throughout sessions. There was no consistent relation between baseline response rates in the first component and reinforcement rate in the second component. When assessed by prefeeding or extinction, resistance to change in the first component was positively related to reinforcement rate in the second component, demonstrating that serial stimulus-reinforcer contingencies affected resistance to change.

Resistance to extinction and behavioral momentum.

In the metaphor of behavioral momentum, reinforcement is assumed to strengthen discriminated operant behavior in the sense of increasing its resistance to disruption, and extinction is viewed as disruption by contingency termination and reinforcer omission. In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.

Classically conditioned enhancement of antibody production

In Experiments 1 and 2, rats were exposed to a classical conditioning procedure in which saccharin and lithium chloride (LiCl) were paired with an injection of antigen. Relative to control groups, the conditioned animals that received additional saccharin and LiC1 presentations showed significantly higher antibody levels. These experiments demonstrate that classical conditioning procedures are sufficient to establish conditioned reflexes in which stimuli (which were previously neutral with respect to the immune system) enhance antibody production.

Resistance to extinction in the steady state and in transition.

Three experiments with pigeons explored the constancy of reinforcer omission during extinction conjectured by rate estimation theory. Experiment 1 arranged 3-component multiple variable-interval (VI) schedules with a mixture of food and extinction trials within each session. Reinforcers omitted to an extinction criterion increased with food-trial reinforcer rate. Experiment 2 arranged 3-component multiple VI schedules where components differed in rate or number of reinforcers. Resistance to extinction depended on the training reinforcer rate but not on the number of reinforcers omitted. Experiment 3 replicated the partial-reinforcement extinction effect within subjects in a discrete-trial procedure and found that more reinforcers were omitted in continuous- than in partial-reinforcement trials. A model of extinction based on behavioral momentum theory accounted for all the data.

Does the context of reinforcement affect resistance to change

Eight pigeons were trained on multiple schedules of reinforcement where pairs of components alternated in blocks on different keys to define 2 local contexts. On 1 key, components arranged 160 and 40 reinforcers/hr; on the other, components arranged 40 and 10 reinforcers/hr. Response rates in the 40/hr component were higher in the latter pair. Within pairs, resistance to prefeeding and resistance to extinction were generally greater in the richer component. The two 40/hr components did not differ in resistance to prefeeding, but the 40/hr component that alternated with 10/hr was more resistant to extinction. This discrepancy was interpreted by an algebraic model relating response strength to component reinforcer rate, including generalization decrement. According to this model, strength is independent of context, consistent with research on schedule preference.