Martin Hammer

The neural basis of associative reward learning in honeybees

Appetitive learning of food-predicting stimuli, an essential part of foraging behavior in honeybees, follows the rules of associative learning. In the learning of odors as reward-predicting stimuli, an individual neuron, one of a small group of large ascending neurons that serve principal brain neuropiles, mediates the reward and has experience-dependent response properties. This implies that this neuron functions as an integral part of associative memory, might underlie more complex features of learning, and could participate in the implementation of learning rules. Moreover, its structural properties suggest that it organizes the interaction of functionally different neural nets during learning and experience-dependent behavior.

Functional Organization of Appetitive Learning and Memory in a Generalist Pollinator, the Honey Bee

Individual experience with environmental stimuli leaves multiple traces of neuronal plasticities in the nervous system. Receptors adapt to prolonged stimulation; neural circuits habituate to repeated stimuli and dishabituate or sensitize to arousing stimuli; and new functional connections are formed or existing ones abolished by associative and latent learning. What are the rules of neural plasticity and how do they relate to the biological constraints under which they have evolved? The neuroethological approach taken in the study of honey bee learning and memory tries to understand the neuronal mechanisms of the multiple memory traces as adaptations to the particular demands of foraging by a generalist pollinating insect. The study of the functional dynamics of memory thus serves two goals: to unravel the informational sources which guide the sequences and time dependencies of the animal’s choice behavior, and to better understand the neural correlates of the various forms of memory.

Food-induced arousal and nonassociative learning in honeybees: dependence of sensitization on the application site and duration of food stimulation.

Stimulus-induced arousal (sensitization) of a component of appetitive behavior in honeybees, the proboscis extension reflex (PER), was used to investigate different aspects of nonassociative memory. The sensitizing stimulus (sucrose solution) was applied to one antenna, as a compound to antenna and proboscis, and to the proboscis. Stimulus duration was either 1 or 3 s. Sensitization was evaluated by monitoring PER toward an odor before (pretest) and after (test) application of the sensitizing stimulus. All responses were quantified by recording from muscle M17 which represents the motor program of PER. Data were analyzed by determining (1) the response probability to the odor and (2) the response strength by determining the number of M17-spikes and the percentage of licking bees per trial. The analysis of the response probability led to two main results: the proportion of animals responding to the test odor depended on stimulus site, and, dependent on stimulus site, a longer application of the sensitizing stimulus resulted in different sensitization rates. The strength of the sensitized response, however, did not correspond to the probability, with which it was elicited, but rather to the strength of the response to the sensitizing stimulus itself. Furthermore, the three groups were not equally affected by the short and long stimulation. The analysis of the proportion of animals licking during test confirmed the data obtained using the number of muscle spikes as a measure of response strength. These results suggest an internal evaluation of the sensitizing stimulus depending on its quality and intensity. The differential effects after antennal and proboscis stimulation may be realized via an arousal system which has two independent functions, a permissive one modulating response probability and one modulating response strength. The permissive function of arousal may be regulated via an intervening inhibitory system whose activation critically depends on the functional significance of the arousing stimulus. The content of this short-term form of memory may be interpreted as an expectation for food which is regulated according to experienced consequences.

Behavioral, neural and cellular components underlying olfactory learning in the honeybee.

A top-down approach as applied to learning and memory in honeybees provides the opportunity of relating different levels of complexity to each other, and of analyzing the rules and mechanisms from the viewpoint of the respective next higher level. Olfactory conditioning of harnessed bees exemplifies essential elements of associative learning and, in general, forms a bridge between the systems and the cellular levels of analysis. Intracellular recordings of identified neurons during olfactory conditioning play a key role in this effort. They allow testing of the assumptions made by modern behavioral theories of associative learning and provide access to cellular and molecular studies, owing to the identification of their transmitters and the peculiarities of the connectivities. Analysis at this intermediate level of complexity is particularly profitable in the bee, because essential neural elements of the associative network are known and can be tested during ongoing learning behavior. In this respect, the honeybee offers unique properties for the building of bridges between the molecular, cellular, neuronal, network and behavioral levels of associative learning.

Kinetic models of odor transduction implemented as artificial neural networks

We present a formal model of olfactory transduction corresponding to the biochemical reaction cascade found in chemosensory neurons. It assumes that odorants bind to receptor proteins which, in turn, activate transducer mechanisms corresponding to second messenger-mediated processes. The model is reformulated as a mathematically equivalent artificial neural network (ANN). To enable comparison of the computational power of our model, previously suggested models of chemosensory transduction are also presented in ANN versions. In ANNs, certain biological parameters, such as rate constants and affinities, are transformed into weights that can be fitted by training with a given experimental data set. After training, these weights do not necessarily equal the real biological parameters, but represent a set of values that is sufficient to simulate an experimental set of data. We used ANNs to simulate data recorded from bee subplacodes and compare the capacity of our model with ANN versions of other models. Receptor neurons of the nonpheromonal, general odor-processing subsystem of the honeybee are broadly tuned, have overlapping response spectra, and show highly nonlinear concentration dependencies and mixture interactions, i.e., synergistic and inhibitory effects. Our full model alone has the necessary complexity to simulate these complex response characteristics. To account for the complex response characteristics of honeybee receptor neurons, we suggest that several different receptor protein types and at least two second messenger systems are necessary that may interact at various levels of the transduction cascade and may eventually have opposing effects on receptor neuron excitability.

Real-time models of classical conditioning

Real-time models of classical conditioning simulate features of associative learning including its dependence on the timing of stimuli. We present the Sutton/Barto model, the TD model, the CP model, the drive-reinforcement model, and the SOP model in a framework of reinforcement learning rules. The role of eligibility and reinforcement is analyzed and the ability of the models to simulate time-dependent learning (e.g. inhibitory backward conditioning) and other conditioning phenomena is also compared. A new model is introduced, that is mathematically simple, and overcomes weaknesses of the other models. This model combines the two antagonistic US traces of the SOP model with the reinforcement term of the TD model.