E. James Kehoe

Summation and configuration in patterning schedules with the rat and rabbit

Discrimination between a tone + light compound and its components in positive and negative patterning schedules was examined. In the positive schedule, reinforced compound presentations (C+) were intermixed with unreinforced component presentations (T−, L−). In the negative schedule, the compound was unreinforced (C−) and the components were reinforced (T+, L+). In Experiment 1, appetitive conditioning of rats’ anticipatory magazine responses was used, and in Experiment 2, aversive conditioning of the rabbit’s nictitating membrane response was used. Both experiments revealed that the positive patterning schedule consistently produced rapid acquisition of appropriate discriminative responding. The results of the negative patterning schedule were more complex. Specifically, the results of Experiment 1 demonstrated that naive rats initially showed rapid acquisition of the negative patterning discrimination. However, schedule reversals revealed that experience with the positive patterning schedule virtually abolished subsequent acquisition of discriminative responding under the negative patterning schedule. The results of Experiment 2 revealed that naive rabbits showed very slow acquisition of discriminative responding under the negative patterning schedule. The results are discussed in relation to the unique-stimulus hypothesis, a contextual encoding hypothesis, and a configural hypothesis.

Summation and configuration between and within sensory modalities in classical conditioning of the rabbit

Three experiments examined “atomistic” and “configurai” processes in stimulus compounding using the rabbit’s conditioned nictitating membrane response. Two conditioned stimuli (CSs) were trained separately and then tested together in a compound. Animals trained with CSs from different modalities—namely, tone and light—showed summation in both acquisition and extinction. That is, the probability of a response to the compound could be predicted by the statistical sum of responding to the CSs. In contrast, animals trained with CSs from the auditory modality, tone and noise, showed a level of responding to the tone + noise compound that was the same as that of the CSs, well under the level predicted by the statistical sum of responding to the CSs. In conclusion, atomistic processes appear to predominate in cross-modal compounding. Configurai processes may occur during compounding within the auditory modality, but atomistic alternatives—namely, common elements and selective attention hypotheses—may be able to explain the results.

Stimulus representation and the timing of reward-prediction errors in models of the dopamine system

The phasic firing of dopamine neurons has been theorized to encode a reward-prediction error as formalized by the temporal-difference (TD) algorithm in reinforcement learning. Most TD models of dopamine have assumed a stimulus representation, known as the complete serial compound, in which each moment in a trial is distinctly represented. We introduce a more realistic temporal stimulus representation for the TD model. In our model, all external stimuli, including rewards, spawn a series of internal microstimuli, which grow weaker and more diffuse over time. These microstimuli are used by the TD learning algorithm to generate predictions of future reward. This new stimulus representation injects temporal generalization into the TD model and enhances correspondence between model and data in several experiments, including those when rewards are omitted or received early. This improved fit mostly derives from the absence of large negative errors in the new model, suggesting that dopamine alone can encode the full range of TD errors in these situations.

Associative transfer and stimulus selection in classical conditioning of the rabbit's nictitating membrane response to serial compound CSs.

Four experiments were conducted to determine whether in conditioning to a serial compound, CS1-CS2-UCS, there are (a) associative mechanisms operating to extend conditioning beyond the bounds of a CS-UCS contiguity gradient and (b) stimulus selection processes acting to attenuate the potency of CS-UCS contiguity. In Experiments 1 and 2, the CS2-UCS interval was held at .35 sec while the CS1-UCS interval was varied across groups from .75 to 2.75 sec. CS1 test trials revealed substantial CR acquisition at all CS1-UCS intervals. Moreover, Experiment 2 indicated that when the contribution of cross-modal generalization from CS2 to CS1 was factored out, there still remained a substantial level of conditioning, which Experiment 3 indicated was attributable to an associative mechanism like higher-order or sensory conditioning. The observation of CR acquisition at CS1-UCS intervals of 4.75, 8.75, and 18.75 sec in Experiment 4 suggested that serial compound training yields conditioning to CSs located well beyond the single CS contiguity gradient for the rabbits nictitating membrane response. Experiments 1 and 2 also indicated the presence of stimulus selection processes because, at the shorter CS1-UCS intervals (.75 and 1.25 sec), the levels of test-trial responding to CS2 fell below those observed to the less contiguous CS1.

Temporal primacy overrides prior training in serial compound conditioning of the rabbit's nictitating membrane response

Conditioning of the rabbit’s nictitating membrane response was conducted with a serial compound CSA-CSB-US. In the present experiments, prior training of CSB was pitted against the temporal primacy of CSA. Prior training of CSB was able to only weakly block CR acquisition to the added CSA, but CSA caused a pronounced decline in responding to the pretrained CSB. By the end of training, high levels of responding were sustained only in the final portion of the serial compound in which CSA or its traces coincided with CSB. These results provide support for real-time models as exemplified by Sutton and Barto (1981).

Fundamental Behavioral Methods and Findings in Classical Conditioning

This chapter is intended to provide a solid foundation in the methods and findings of classical conditioning at a behavioral level. We take readers from an introduction to the basics of classical conditioning through to its more complex and recent findings that appear related to higher-order processes that are commonly labeled as attention, perception, and even cognition. Many textbook descriptions of classical conditioning give the mistaken impression that it is an area of completed science. In fact, the methods of classical conditioning have continued to evolve, and the range of important phenomena has continued to expand. Although this chapter primarily describes the empirical methods and outcomes of classical conditioning, we introduce important theoretical principles along the way. These principles have proven crucial in organizing research findings and in linking classical conditioning to other learning phenomena. In addition, our introduction to behavioral theory should help the reader place the phenomena described in this chapter in the context of the theories described in more detail in the next chapter (Brandon, Vogel, & Wagner, this volume).

Cross-modal transfer as a function of similarities between training tasks in classical conditioning of the rabbit

The present experiments examined transfer of training from visual to auditory sensory modalities in classical conditioning of the rabbit’s nictitating membrane response. Experiment 1 examined transfer from initial training with a single visual CS to subsequent auditory discrimination training, and Experiment 2 examined transfer from visual discrimination training to auditory discrimination training. The major findings were that (1) initial conditioning of a visual CS facilitated the overall rate of CR acquisition to the auditory CSs separate from the requirements for discrimination learning (Experiments 1 and 2), and (2) initial visual discrimination training facilitated auditory discrimination learning (Experiment 2). Thus, the animals appeared to encode separately both contiguous CS-US relations and CS+ versus CS− relations during initial visual training. The results are discussed with respect to theories of extradimensional transfer.

Overshadowing and summation in compound stimulus conditioning of the rabbit's nictitating membrane response.

The present experiments examined acquisition of the rabbits nictitating membrane response to a light + tone simultaneous compound stimulus and its components as a function of the intensity of the tone. In Experiment 1, the tone intensity was varied across the values of 85, 89, and 93 dB, and the CS-US (conditioned stimulus-unconditioned stimulus) interval was 400 msec. In Experiment 2, the tone intensities were 73, 85, and 93 dB, and the CS-US interval was 800 msec. Experiments 3 and 4 further examined the effects of the 73-dB tone at CS-US intervals of 400 and 800 msec, respectively. All experiments included control groups, which were trained with either a light or a tone CS. In brief, the experiments revealed repeated instances of overshadowing, i.e., the impairment of conditioned response (CR) acquisition to one or both of the components of a compound. Moreover, two types of summation were obtained: within-subjects summation, in which rabbits trained with a compound showed a higher level of responding to the compound than to either of its component CSs (Experiments 2, 3, and 4), and between-groups summation, in which a group trained with a compound showed faster CR acquisition than either of its corresponding control groups trained with a single CS (Experiments 1 and 2). The results are discussed in terms of perceptual and distributive processing models of compound stimulus conditioning.

Transfer across CS-US intervals and sensory modalities in classical conditioning of the rabbit

Two experiments investigated transfer of the rabbit’s conditioned nictitating membrane response (NMR) from shorter to longer CS-US intervals in conjunction with a change in CS modality, for example, light to tone. In Experiment 1, three experimental groups received initial training with a 400-msec CS-US interval, which produced substantial CR acquisition, and three control groups received initial training with a 2,800-msec CS-US interval, which produced minimal CR acquisition. Subsequently, the experimental and control groups received training with an 800-, 1,800-, or 2,800-msec CS-US trace interval. At the same time, the modality of the CS was changed from tone to light (or vice versa). Experiment 2 contained three groups that received initial exposure to a 400-msec CS-US interval, a 2,800-msec CS-US interval, or just the experimental chambers. Subsequently, all three groups received training with an 800-msec CS-US interval in a different CS modality. The results of both experiments revealed substantial positive transfer across CS modalities from the 400-msec CS-US interval to the 800-msec CS-US interval. There was also significant transfer to the 1,800-msec but not the 2,800-msec CS-US interval. The transfer did not appear immediately on test presentations of the second CS. Rather, the transfer appeared as an enhancement in the rate of CR acquisition after reinforced training with the second CS had commenced. The results are discussed with respect to mechanisms of transfer and facilitation of trace conditioning.

CONNECTIONIST MODELS OF CONDITIONING: A TUTORIAL

Models containing networks of neuron-like units have become increasingly prominent in the study of both cognitive psychology and artificial intelligence. This article describes the basic features of connectionist models and provides an illustrative application to compound-stimulus effects in respondent conditioning. Connectionist models designed specifically for operant conditioning are not yet widely available, but some current learning algorithms for machine learning indicate that such models are feasible. Conversely, designers for machine learning appear to have recognized the value of behavioral principles in producing adaptive behavior in their creations.