Marianna Yanike

Dynamic Analysis of Learning in Behavioral Experiments

Understanding how an animals ability to learn relates to neural activity or is altered by lesions, different attentional states, pharmacological interventions, or genetic manipulations are central questions in neuroscience. Although learning is a dynamic process, current analyses do not use dynamic estimation methods, require many trials across many animals to establish the occurrence of learning, and provide no consensus as how best to identify when learning has occurred. We develop a state-space model paradigm to characterize learning as the probability of a correct response as a function of trial number (learning curve). We compute the learning curve and its confidence intervals using a state-space smoothing algorithm and define the learning trial as the first trial on which there is reasonable certainty (>0.95) that a subject performs better than chance for the balance of the experiment. For a range of simulated learning experiments, the smoothing algorithm estimated learning curves with smaller mean integrated squared error and identified the learning trials with greater reliability than commonly used methods. The smoothing algorithm tracked easily the rapid learning of a monkey during a single session of an association learning experiment and identified learning 2 to 4 d earlier than accepted criteria for a rat in a 47 d procedural learning experiment. Our state-space paradigm estimates learning curves for single animals, gives a precise definition of learning, and suggests a coherent statistical framework for the design and analysis of learning experiments that could reduce the number of animals and trials per animal that these studies require.

Functional Magnetic Resonance Imaging Activity during the Gradual Acquisition and Expression of Paired-Associate Memory

Recent neurophysiological findings from the monkey hippocampus showed dramatic changes in the firing rate of individual hippocampal cells as a function of learning new associations. To extend these findings to humans, we used blood oxygenation level-dependent (BOLD) functional magnetic resonance imaging (fMRI) to examine the patterns of brain activity during learning of an analogous associative task. We observed bilateral, monotonic increases in activity during learning not only in the hippocampus but also in the parahippocampal and right perirhinal cortices. In addition, activity related to simple novelty signals was observed throughout the medial temporal lobe (MTL) memory system and in several frontal regions. A contrasting pattern was observed in a frontoparietal network in which a high level of activity was sustained until the association was well learned, at which point the activity decreased to baseline. Thus, we found that associative learning in humans is accompanied by striking increases in BOLD fMRI activity throughout the MTL as well as in the cingulate cortex and frontal lobe, consistent with neurophysiological findings in the monkey hippocampus. The finding that both the hippocampus and surrounding MTL cortex exhibited similar associative learning and novelty signals argues strongly against the view that there is a clear division of labor in the MTL in which the hippocampus is essential for forming associations and the cortex is involved in novelty detection. A second experiment addressed a striking aspect of the data from the first experiment by demonstrating a substantial effect of baseline task difficulty on MTL activity capable of rendering mnemonic activity as either “positive” or “negative.”

Analysis of Between-Trial and Within-Trial Neural Spiking Dynamics

Recording single-neuron activity from a specific brain region across multiple trials in response to the same stimulus or execution of the same behavioral task is a common neurophysiology protocol. The raster plots of the spike trains often show strong between-trial and within-trial dynamics, yet the standard analysis of these data with the peristimulus time histogram (PSTH) and ANOVA do not consider between-trial dynamics. By itself, the PSTH does not provide a framework for statistical inference. We present a state-space generalized linear model (SS-GLM) to formulate a point process representation of between-trial and within-trial neural spiking dynamics. Our model has the PSTH as a special case. We provide a framework for model estimation, model selection, goodness-of-fit analysis, and inference. In an analysis of hippocampal neural activity recorded from a monkey performing a location-scene association task, we demonstrate how the SS-GLM may be used to answer frequently posed neurophysiological questions including, What is the nature of the between-trial and within-trial task-specific modulation of the neural spiking activity? How can we characterize learning-related neural dynamics? What are the timescales and characteristics of the neurons biophysical properties? Our results demonstrate that the SS-GLM is a more informative tool than the PSTH and ANOVA for analysis of multiple trial neural responses and that it provides a quantitative characterization of the between-trial and within-trial neural dynamics readily visible in raster plots, as well as the less apparent fast (1-10 ms), intermediate (11-20 ms), and longer (>20 ms) timescale features of the neurons biophysical properties.

Representation of well-learned information in the monkey hippocampus.

In the neocortex, extensive training results in enhanced neuronal selectivity for learned stimuli relative to novel stimuli. This enhanced selectivity has been taken as evidence for learning-related plasticity. Much less is known, in contrast, about the representation of well-learned information in the hippocampus. In this study, we examined the responses of individual hippocampal neurons to well-learned and novel stimuli presented in the context of an associative learning task. There was no difference in the response magnitude or visual response latency of hippocampal neurons to the well-learned and novel stimuli. In contrast, hippocampal neurons responded significantly more selectively to the well-learned stimuli relative to the novel stimuli. These findings show that hippocampal cells, like neocortical cells, show greater selectivity to well-learned stimuli compared to novel stimuli.

Comparison of Associative Learning-Related Signals in the Macaque Perirhinal Cortex and Hippocampus

Strong evidence suggests that the macaque monkey perirhinal cortex is involved in both the initial formation as well as the long-term storage of associative memory. To examine the neurophysiological basis of associative memory formation in this area, we recorded neural activity in this region as monkeys learned new conditional-motor associations. We report that a population of perirhinal neurons signal newly learned associations by changing their firing rate correlated with the animals behavioral learning curve. Individual perirhinal neurons signal learning of one or more associations concurrently and these neural changes could occur before, at the same time, or after behavioral learning was expressed. We also compared the associative learning signals in the perirhinal cortex to our previous findings in the hippocampus. We report global similarities in both the learning-related and task-related activity seen across these areas as well as clear differences in the within and across trial timing and relative proportion of different subtypes of learning-related signals. Taken together, these findings emphasize the important role of the perirhinal cortex in new associative learning and suggest that the perirhinal cortex together with the hippocampus contribute importantly to conditional-motor associative memory formation.

State-space algorithms for estimating spike rate functions

The accurate characterization of spike firing rates including the determination of when changes in activity occur is a fundamental issue in the analysis of neurophysiological data. Here we describe a state-space model for estimating the spike rate function that provides a maximum likelihood estimate of the spike rate, model goodness-of-fit assessments, as well as confidence intervals for the spike rate function and any other associated quantities of interest. Using simulated spike data, we first compare the performance of the state-space approach with that of Bayesian adaptive regression splines (BARS) and a simple cubic spline smoothing algorithm. We show that the state-space model is computationally efficient and comparable with other spline approaches. Our results suggest both a theoretically sound and practical approach for estimating spike rate functions that is applicable to a wide range of neurophysiological data.

Representation of Outcome Risk and Action in the Anterior Caudate Nucleus

The anterior caudate nucleus is essential for goal-directed behavior because it links outcome information to actions. It is well known that caudate neurons provide a variety of reward-related and action signals. However, it is still unclear how the two signals are integrated. We investigated whether and how outcome risk modulates spatial representation. We recorded neural activity in the anterior caudate nucleus while monkeys made saccades to multiple spatial targets, each associated with either fixed (safe) or variable (risky) amount of reward. We report that individual neurons combined the outcome reward signal with spatial information about the direction of saccades. These signals could be reliably read out from the populations of neurons. Moreover, the prospect of a risky outcome improved the quality of spatial information. These results provide direct evidence that global spatial representation in the caudate is modulated by outcome, which can be important for flexible control of behavior, particularly during learning and habit formation, when outcomes vary.

Characterizing learning by simultaneous analysis of continuous and binary measures of performance

Continuous observations, such as reaction and run times, and binary observations, such as correct/incorrect responses, are recorded routinely in behavioral learning experiments. Although both types of performance measures are often recorded simultaneously, the two have not been used in combination to evaluate learning. We present a state-space model of learning in which the observation process has simultaneously recorded continuous and binary measures of performance. We use these performance measures simultaneously to estimate the model parameters and the unobserved cognitive state process by maximum likelihood using an approximate expectation maximization (EM) algorithm. We introduce the concept of a reaction-time curve and reformulate our previous definitions of the learning curve, the ideal observer curve, the learning trial and between-trial comparisons of performance in terms of the new model. We illustrate the properties of the new model in an analysis of a simulated learning experiment. In the simulated data analysis, simultaneous use of the two measures of performance provided more credible and accurate estimates of the learning than either measure analyzed separately. We also analyze two actual learning experiments in which the performance of rats and of monkeys was tracked across trials by simultaneously recorded reaction and run times and the correct and incorrect responses. In the analysis of the actual experiments, our algorithm gave a straightforward, efficient way to characterize learning by combining continuous and binary measures of performance. This analysis paradigm has implications for characterizing learning and for the more general problem of combining different data types to characterize the properties of a neural system.

A mixed filter algorithm for cognitive state estimation from simultaneously recorded continuous and binary measures of performance

Continuous (reaction times) and binary (correct/ incorrect responses) measures of performance are routinely recorded to track the dynamics of a subject’s cognitive state during a learning experiment. Current analyses of experimental data from learning studies do not consider the two performance measures together and do not use the concept of the cognitive state formally to design statistical methods. We develop a mixed filter algorithm to estimate the cognitive state modeled as a linear stochastic dynamical system from simultaneously recorded continuous and binary measures of performance. The mixed filter algorithm has the Kalman filter and the more recently developed recursive filtering algorithm for binary processes as special cases. In the analysis of a simulated learning experiment the mixed filter algorithm provided a more accurate and precise estimate of the cognitive state process than either the Kalman or binary filter alone. In the analysis of an actual learning experiment in which a monkey’s performance was tracked by its series of reaction times, and correct and incorrect responses, the mixed filter gave a more complete description of the learning process than either the Kalman or binary filter. These results establish the feasibility of estimating cognitive state from simultaneously recorded continuous and binary performance measures and suggest a way to make practical use of concepts from learning theory in the design of statistical methods for the analysis of data from learning experiments.

Interpretive monitoring in the caudate nucleus

In a dynamic environment an organism has to constantly adjust ongoing behavior to adapt to a given context. This process requires continuous monitoring of ongoing behavior to provide its meaningful interpretation. The caudate nucleus is known to have a role in behavioral monitoring, but the nature of these signals during dynamic behavior is still unclear. We recorded neuronal activity in the caudate nucleus in monkeys during categorization behavior that changed rapidly across contexts. We found that neuronal activity maintained representation of the identity and context of a recently categorized stimulus, as well as interpreted the behavioral meaningfulness of the maintained trace. The accuracy of this cognitive monitoring signal was highest for behavior for which subjects were prone to make errors. Thus, the caudate nucleus provides interpretive monitoring of ongoing behavior, which is necessary for contextually specific decisions to adapt to rapidly changing conditions. DOI: http://dx.doi.org/10.7554/eLife.03727.001