Alliston K. Reid

Sensitivity to Molar Feedback Functions: A Test of Molar Optimality Theory

Molar optimality models assume that any reward schedule can be described by a molar feedback function, which is the relation between average response rates and average reinforcement rates enforced by that particular schedule. This molar feedback function is considered, by optimality models, to be a sufficient description of the schedule for the prediction of steady-state molar performance. In this article we challenge the fundamental assumption of all molar optimality modelsthat animals are directly sensitive to this molar feedback function. We found that animals were sensitive to the schedule conditions in effect, especially at the molecular level of postfood time, but they were not directly sensitive to the slopes of any ofthe molar feedback functions that we manipulated. Our data do not simply represent a failure to maximize a particular utility function so that this form of the function requires alteration. Rather, they demonstrate that animals may not be sensitive to the molar rates of responding and reinforcement described by the molar feedback functions. Our animals were sensitive to the schedules at a molecular level, and it is to this molecular level that we should direct our attention.

Adaptation to Reward

For some years ethologists, working largely in the field, and psychologists, working largely in the laboratory, had little to say to one another about learning and adaptive behavior in animals. Ethologists tend to explain behavior in terms of Darwinian fitness and adaptive function. Psychologists are concerned with mechanisms, physiological or formal. Animals in the laboratory often behave in apparently maladaptive ways, supporting psychologists’ suspicion of accounts in terms of adaptation. Ethologists, noting the artificiality of some laboratory procedures, have sometimes regarded situations far from the natural as essentially irrelevant to understanding animal behavior.

Route finding by rats in an open arena

Rats were repeatedly exposed to an open arena containing two depletable food sources in a discrete-trials procedure. Their movement patterns were recorded and compared to adaptive foraging tactics such as minimizing distance or energy expenditure, thigmotaxis, and trail following. They were also compared to the predictions of the associative route-finder model of Reid and Staddon [Reid, A.K., Staddon, J.E.R., 1998. A dynamic route finder for the cognitive map. Psychol. Rev. 105 (3), 585-601]. We manipulated the presence/absence of food, goal cups, and a wooden runway to determine the influence of local and distal stimuli (visual, olfactory, and tactile) on movement patterns. Increased experience in the arena produced decreases in travel distance and time to the food sources. Local and distal stimuli influenced movement patterns in ways compatible with visual beacons and trail following. The route-finder model accurately predicted movement patterns except those that were influenced by local and distal stimuli. These results show how certain stimuli influence movement and provide a guide for the incorporation of local and distal stimuli in a future version of the dynamic route-finder model.

Schedule induction and the temporal distributions of adjunctive behavior on periodic water schedules

The ability of periodic water reinforcement schedules to induce or entrain activities was investigated by having observers classify mutually exclusive and exhaustive activities of 10 rats placed in a rich environment with various fixed-time schedules of water delivery. For each activity measured, three characteristics of induced behavior were examined: (1) its excessiveness; (2) the commonly observed inverted-U relation between rate of induced behavior and rate of reinforcement; and (3) the observation that induced activities occur earlier in interreinforcement intervals than do facultative activities, producing multimodal activity distributions. All activities were demonstrated to be noninduced, facultative activities, and none could be classified as schedule-induced behavior by any of the three criteria. The factors that determine the temporal distributions of all activities appeared to be equivalent to the factors that determine the distributions of scheduleinduced activities in situations in which only the induced activities are available.

Learning new response sequences

Four rats were required to press either a right or left lever to complete various three-response sequences. After extended exposure to a training sequence, subjects were shifted to a new target sequence. The new target sequences always differed from the previous sequence by the response required in the first or last position of the sequence. Subjects were repeatedly exposed to all possible combinations of training and new target sequences. Learning of new sequences occurred more rapidly when the change in the new target sequence was in the last position. Errors persisted longer in new sequences in which the change was in the first position. Extinction of the training sequence occured faster when the change was in the last position. Responses in the last position were considerably more sensitive to the shift to new target sequences than were responses in the first position. Even though response sequences may form new behavioral units from the training sequence, reinforcement and extinction acted differentially on the individual lever presses within new target sequences rather than on the sequences as a whole. These findings support the hypothesis that response strength is determined by contiguity to reinforcement.

Changes in stimulus control during guided skill learning in rats

We examined the changes in stimulus control occurring during guided skill learning in rats. Twenty rats were trained to complete a left-right sequence of lever presses guided by the onset and offset of panel lights over their respective levers. Once sequence accuracy was high and stable, the rats were divided into two groups. For the No-Lights group, the lights were eliminated without changing the response requirements. Sequence accuracy decreased in all subjects, but accuracy was higher than that predicted by random chance. More practice produced greater autonomy and reduced dependence on the guiding lights. For the Reversed-Lights group, the lights were presented in reversed order without changing the response requirements. Sequence accuracy immediately plummeted and did not recover, violating expectations of automatization. The guiding lights appeared to overshadow other sources of stimulus control.

Resistance to change within heterogeneous response sequences.

Three experiments investigated how instrumental and Pavlovian contingencies contribute to resistance to change (RTC) in different ordinal response positions within heterogeneous response sequences in pigeons. RTC in the initial and terminal response positions of a three-response sequence were compared in Experiment 1, which presented three colored key lights in succession in each trial; and in Experiment 2, which severely degraded Pavlovian contingencies by presenting the lights simultaneously at each ordinal position. Experiment 3 eliminated the instrumental contingency in a high-order sign-tracking procedure. When the instrumental contingency was in effect, RTC of the initial position was greater than the terminal position (Initial RTC > Terminal RTC) when the Pavlovian contingencies were strong and when they were degraded. When the instrumental contingency was eliminated, RTC patterns reversed, producing a graded pattern of RTC (Initial < Middle < Terminal). Current theoretical approaches (e.g., behavioral momentum theory, conditioned reinforcement, and motivational control of instrumental conditioning) cannot account for these results. An alternative approach (a gradient model) shows that obtained measures of RTC in heterogeneous sequences may reflect a combination of three dissociable processes.

Why don't guiding cues always guide in behavior chains?

This research focused on the changes in stimulus control that influence an animal’s ability to master a behavioral skill. We assessed stimulus control by (a) predictive environmental cues (panel lights) and (b) practice cues resulting from the subject’s own behavior, as rats learned to complete a left–right lever-press sequence. Following a demonstration of overshadowing by Reid, Nill, and Getz (Behavioural Processes 84: 511–515, 2010), in which stimulus control by the panel lights overshadowed control by practice cues, four additional experiments replicated and assessed this overshadowing effect. In Experiment 1, we discovered a powerful asymmetry: Rats failed to adapt to a lights → reversed-lights transition, but adapted immediately to a reversed-lights → lights transition. Experiment 2 was designed to measure the interactions between these stimulus conditions and practice cues. In Experiment 3, we measured the effect of these stimulus conditions on acquisition rates. Finally, in Experiment 4 an ABA design was used to assess the effects of prior exposure to condition A on B → A transitions, and we found that prior exposure generally reversed the effects observed in B → A transitions presented first or in isolation. We discuss feature-positive bias and spatial S–R compatibility as potential explanations of the observed insensitivity to cues that should be, at face value, highly predictive of food during the acquisition of a behavioral skill. Perfectly predictive cues in behavior chains do not always guide behavior.

A reader for the cognitive map

Abstract A local diffusion model (Staddon and Reid, 1990) can reproduce exponential and Gaussian stimulus-generalization gradients. We show that a two-dimensional diffusion model, together with simple reinforcement assumptions, can reproduce many of the empirical properties of goal-directed spatial search, including area-restricted search, open-field foraging, barrier and detour problems, maze learning and spatial “insight.” The model provides a simple, associationistic “reader” for Tolmans cognitive map.

On the dynamics of stimulus control during guided skill learning in nonhumans

This study measured skill acquisition in the presence and absence of guiding cues in pigeons. It asked whether the speed of development of autonomy for the motor skill is influenced by the difficulty level of two guiding-cue conditions requiring the same left-right response sequence. The Follow-Red condition required a simple go, no-go discrimination (red=S+, green=S-), whereas the Red-Green condition was a more difficult simultaneous chain requiring sensitivity to the serial order of key colors (red=S+, green=S- for the first peck, but red=S-, green=S+ for the second peck). Pigeons exposed to the difficult Red-Green condition displayed significantly higher accuracy levels during no-cues conditions earlier in training than those exposed to the easier Follow-Red condition. A modified Power Law of Practice was used to evaluate the null hypothesis that autonomy develops equally in explicit guiding-cues conditions and no-cues conditions. This hypothesis was retained in the Follow-Red condition but rejected in the Red-Green condition. Practice completing the response sequence in the Follow-Red and no-cues conditions both contributed equally to autonomy. Autonomy developed faster in the Red-Green group in both conditions, and it developed unexpectedly rapidly during the second guiding-cues condition, implying the involvement of a second process for the Red-Green condition. We discuss the implications of these results to prompt dependence in children with learning disabilities, the transfer of stimulus control, and potential behavioral interventions.