Yonatan Loewenstein

Bistability of cerebellar Purkinje cells modulated by sensory stimulation.

A persistent change in neuronal activity after brief stimuli is a common feature of many neuronal microcircuits. This persistent activity can be sustained by ongoing reverberant network activity or by the intrinsic biophysical properties of individual cells. Here we demonstrate that rat and guinea pig cerebellar Purkinje cells in vivo show bistability of membrane potential and spike output on the time scale of seconds. The transition between membrane potential states can be bidirectionally triggered by the same brief current pulses. We also show that sensory activation of the climbing fiber input can switch Purkinje cells between the two states. The intrinsic nature of Purkinje cell bistability and its control by sensory input can be explained by a simple biophysical model. Purkinje cell bistability may have a key role in the short-term processing and storage of sensory information in the cerebellar cortex.

Temporal integration by calcium dynamics in a model neuron

The calculation and memory of position variables by temporal integration of velocity signals is essential for posture, the vestibulo-ocular reflex (VOR) and navigation. Integrator neurons exhibit persistent firing at multiple rates, which represent the values of memorized position variables. A widespread hypothesis is that temporal integration is the outcome of reverberating feedback loops within recurrent networks, but this hypothesis has not been proven experimentally. Here we present a single-cell model of a neural integrator. The nonlinear dynamics of calcium gives rise to propagating calcium wave-fronts along dendritic processes. The wave-front velocity is modulated by synaptic inputs such that the front location covaries with the temporal sum of its previous inputs. Calcium-dependent currents convert this information into concomitant persistent firing. Calcium dynamics in single neurons could thus be the physiological basis of the graded persistent activity and temporal integration observed in neurons during analog memory tasks.

Multiplicative Dynamics Underlie the Emergence of the Log-Normal Distribution of Spine Sizes in the Neocortex In Vivo

What fundamental properties of synaptic connectivity in the neocortex stem from the ongoing dynamics of synaptic changes? In this study, we seek to find the rules shaping the stationary distribution of synaptic efficacies in the cortex. To address this question, we combined chronic imaging of hundreds of spines in the auditory cortex of mice in vivo over weeks with modeling techniques to quantitatively study the dynamics of spines, the morphological correlates of excitatory synapses in the neocortex. We found that the stationary distribution of spine sizes of individual neurons can be exceptionally well described by a log-normal function. We furthermore show that spines exhibit substantial volatility in their sizes at timescales that range from days to months. Interestingly, the magnitude of changes in spine sizes is proportional to the size of the spine. Such multiplicative dynamics are in contrast with conventional models of synaptic plasticity, learning, and memory, which typically assume additive dynamics. Moreover, we show that the ongoing dynamics of spine sizes can be captured by a simple phenomenological model that operates at two timescales of days and months. This model converges to a log-normal distribution, bridging the gap between synaptic dynamics and the stationary distribution of synaptic efficacies.

Learning reward timing in cortex through reward dependent expression of synaptic plasticity

The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.

Bayesian inference underlies the contraction bias in delayed comparison tasks.

Delayed comparison tasks are widely used in the study of working memory and perception in psychology and neuroscience. It has long been known, however, that decisions in these tasks are biased. When the two stimuli in a delayed comparison trial are small in magnitude, subjects tend to report that the first stimulus is larger than the second stimulus. In contrast, subjects tend to report that the second stimulus is larger than the first when the stimuli are relatively large. Here we study the computational principles underlying this bias, also known as the contraction bias. We propose that the contraction bias results from a Bayesian computation in which a noisy representation of a magnitude is combined with a-priori information about the distribution of magnitudes to optimize performance. We test our hypothesis on choice behavior in a visual delayed comparison experiment by studying the effect of (i) changing the prior distribution and (ii) changing the uncertainty in the memorized stimulus. We show that choice behavior in both manipulations is consistent with the performance of an observer who uses a Bayesian inference in order to improve performance. Moreover, our results suggest that the contraction bias arises during memory retrieval/decision making and not during memory encoding. These results support the notion that the contraction bias illusion can be understood as resulting from optimality considerations.

Alternative Sites of Synaptic Plasticity in Two Homologous “Fan-out Fan-in” Learning and Memory Networks

BACKGROUND To what extent are the properties of neuronal networks constrained by computational considerations? Comparative analysis of the vertical lobe (VL) system, a brain structure involved in learning and memory, in two phylogenetically close cephalopod mollusks, Octopus vulgaris and the cuttlefish Sepia officinalis, provides a surprising answer to this question. RESULTS We show that in both the octopus and the cuttlefish the VL is characterized by the same simple fan-out fan-in connectivity architecture, composed of the same three neuron types. Yet, the sites of short- and long-term synaptic plasticity and neuromodulation are different. In the octopus, synaptic plasticity occurs at the fan-out glutamatergic synaptic layer, whereas in the cuttlefish plasticity is found at the fan-in cholinergic synaptic layer. CONCLUSIONS Does this dramatic difference in physiology imply a difference in function? Not necessarily. We show that the physiological properties of the VL neurons, particularly the linear input-output relations of the intermediate layer neurons, allow the two different networks to perform the same computation. The convergence of different networks to the same computational capacity indicates that it is the computation, not the specific properties of the network, that is self-organized or selected for by evolutionary pressure.

Reinforcement learning in professional basketball players

Reinforcement learning in complex natural environments is a challenging task because the agent should generalize from the outcomes of actions taken in one state of the world to future actions in different states of the world. The extent to which human experts find the proper level of generalization is unclear. Here we show, using the sequences of field goal attempts made by professional basketball players, that the outcome of even a single field goal attempt has a considerable effect on the rate of subsequent 3 point shot attempts, in line with standard models of reinforcement learning. However, this change in behaviour is associated with negative correlations between the outcomes of successive field goal attempts. These results indicate that despite years of experience and high motivation, professional players overgeneralize from the outcomes of their most recent actions, which leads to decreased performance.

Reinforcement Learning and Human Behavior

The dominant computational approach to model operant learning and its underlying neural activity is model-free reinforcement learning (RL). However, there is accumulating behavioral and neuronal-related evidence that human (and animal) operant learning is far more multifaceted. Theoretical advances in RL, such as hierarchical and model-based RL extend the explanatory power of RL to account for some of these findings. Nevertheless, some other aspects of human behavior remain inexplicable even in the simplest tasks. Here we review developments and remaining challenges in relating RL models to human operant learning. In particular, we emphasize that learning a model of the world is an essential step before or in parallel to learning the policy in RL and discuss alternative models that directly learn a policy without an explicit world model in terms of state-action pairs.

Robustness of Learning That Is Based on Covariance-Driven Synaptic Plasticity

It is widely believed that learning is due, at least in part, to long-lasting modifications of the strengths of synapses in the brain. Theoretical studies have shown that a family of synaptic plasticity rules, in which synaptic changes are driven by covariance, is particularly useful for many forms of learning, including associative memory, gradient estimation, and operant conditioning. Covariance-based plasticity is inherently sensitive. Even a slight mistuning of the parameters of a covariance-based plasticity rule is likely to result in substantial changes in synaptic efficacies. Therefore, the biological relevance of covariance-based plasticity models is questionable. Here, we study the effects of mistuning parameters of the plasticity rule in a decision making model in which synaptic plasticity is driven by the covariance of reward and neural activity. An exact covariance plasticity rule yields Herrnsteins matching law. We show that although the effect of slight mistuning of the plasticity rule on the synaptic efficacies is large, the behavioral effect is small. Thus, matching behavior is robust to mistuning of the parameters of the covariance-based plasticity rule. Furthermore, the mistuned covariance rule results in undermatching, which is consistent with experimentally observed behavior. These results substantiate the hypothesis that approximate covariance-based synaptic plasticity underlies operant conditioning. However, we show that the mistuning of the mean subtraction makes behavior sensitive to the mistuning of the properties of the decision making network. Thus, there is a tradeoff between the robustness of matching behavior to changes in the plasticity rule and its robustness to changes in the properties of the decision making network.

Contradictory Behavioral Biases Result from the Influence of Past Stimuli on Perception

Biases such as the preference of a particular response for no obvious reason, are an integral part of psychophysics. Such biases have been reported in the common two-alternative forced choice (2AFC) experiments, where participants are instructed to compare two consecutively presented stimuli. However, the principles underlying these biases are largely unknown and previous studies have typically used ad-hoc explanations to account for them. Here we consider human performance in the 2AFC tone frequency discrimination task, utilizing two standard protocols. In both protocols, each trial contains a reference stimulus. In one (Reference-Lower protocol), the frequency of the reference stimulus is always lower than that of the comparison stimulus, whereas in the other (Reference protocol), the frequency of the reference stimulus is either lower or higher than that of the comparison stimulus. We find substantial interval biases. Namely, participants perform better when the reference is in a specific interval. Surprisingly, the biases in the two experiments are opposite: performance is better when the reference is in the first interval in the Reference protocol, but is better when the reference is second in the Reference-Lower protocol. This inconsistency refutes previous accounts of the interval bias, and is resolved when experiments statistics is considered. Viewing perception as incorporation of sensory input with prior knowledge accumulated during the experiment accounts for the seemingly contradictory biases both qualitatively and quantitatively. The success of this account implies that even simple discriminations reflect a combination of sensory limitations, memory limitations, and the ability to utilize stimuli statistics.