David J. Herzfeld

A memory of errors in sensorimotor learning

Practice makes perfect — or does it? How do we learn from past errors? Herzfeld et al. found that when we practice a movement, the human brain has a memory for errors that is then used to learn faster in new conditions. This memory for error exists in parallel with motor memorys two traditional forms: memory of actions and memory of external perturbations. They also proposed a mathematical model for learning from errors. This model explained previous experimental results and predicted other major findings that they later verified experimentally. Science, this issue p. 1349 The history of previously experienced motor errors changes the amount the brain is willing to learn from an error. The current view of motor learning suggests that when we revisit a task, the brain recalls the motor commands it previously learned. In this view, motor memory is a memory of motor commands, acquired through trial-and-error and reinforcement. Here we show that the brain controls how much it is willing to learn from the current error through a principled mechanism that depends on the history of past errors. This suggests that the brain stores a previously unknown form of memory, a memory of errors. A mathematical formulation of this idea provides insights into a host of puzzling experimental data, including savings and meta-learning, demonstrating that when we are better at a motor task, it is partly because the brain recognizes the errors it experienced before.

Contributions of the cerebellum and the motor cortex to acquisition and retention of motor memories

We investigated the contributions of the cerebellum and the motor cortex (M1) to acquisition and retention of human motor memories in a force field reaching task. We found that anodal transcranial direct current stimulation (tDCS) of the cerebellum, a technique that is thought to increase neuronal excitability, increased the ability to learn from error and form an internal model of the field, while cathodal cerebellar stimulation reduced this error-dependent learning. In addition, cathodal cerebellar stimulation disrupted the ability to respond to error within a reaching movement, reducing the gain of the sensory-motor feedback loop. By contrast, anodal M1 stimulation had no significant effects on these variables. During sham stimulation, early in training the acquired motor memory exhibited rapid decay in error-clamp trials. With further training the rate of decay decreased, suggesting that with training the motor memory was transformed from a labile to a more stable state. Surprisingly, neither cerebellar nor M1 stimulation altered these decay patterns. Participants returned 24hours later and were re-tested in error-clamp trials without stimulation. The cerebellar group that had learned the task with cathodal stimulation exhibited significantly impaired retention, and retention was not improved by M1 anodal stimulation. In summary, non-invasive cerebellar stimulation resulted in polarity-dependent up- or down-regulation of error-dependent motor learning. In addition, cathodal cerebellar stimulation during acquisition impaired the ability to retain the motor memory overnight. Thus, in the force field task we found a critical role for the cerebellum in both formation of motor memory and its retention.

Encoding of action by the Purkinje cells of the cerebellum

Execution of accurate eye movements depends critically on the cerebellum, suggesting that the major output neurons of the cerebellum, Purkinje cells, may predict motion of the eye. However, this encoding of action for rapid eye movements (saccades) has remained unclear: Purkinje cells show little consistent modulation with respect to saccade amplitude or direction, and critically, their discharge lasts longer than the duration of a saccade. Here we analysed Purkinje-cell discharge in the oculomotor vermis of behaving rhesus monkeys (Macaca mulatta) and found neurons that increased or decreased their activity during saccades. We estimated the combined effect of these two populations via their projections to the caudal fastigial nucleus, and uncovered a simple-spike population response that precisely predicted the real-time motion of the eye. When we organized the Purkinje cells according to each cell’s complex-spike directional tuning, the simple-spike population response predicted both the real-time speed and direction of saccade multiplicatively via a gain field. This suggests that the cerebellum predicts the real-time motion of the eye during saccades via the combined inputs of Purkinje cells onto individual nucleus neurons. A gain-field encoding of simple spikes emerges if the Purkinje cells that project onto a nucleus neuron are not selected at random but share a common complex-spike property.

Improved multi-unit decoding at the brain?machine interface using population temporal linear filtering

Current efforts to decode control signals from multi-unit (MU) recordings rely on the use of spike sorting to differentiate neurons and the use of firing rates estimated over tens of milliseconds to reconstruct sensorimotor signals. The computational bottleneck associated with the need to identify and sort individual neuron responses poses challenges for the development of portable, real-time, neural decoding systems that can be incorporated into assistive and prosthetic devices for the disabled. Here, we investigate the ability of spike-based linear filtering to reduce computational overhead and improve the accuracy of decoding neuronal signals for populations of spiking neurons. Using a population temporal (PT) decoding framework, the speed and accuracy of spike-based MU decoding were compared with firing rate-based approaches using simulated populations of motor neurons tuned for the velocity of intended movement. For the two linear filtering approaches, the accuracy of decoded movements was examined as a function of the number of recorded neurons, amount of noise, with and without spike sorting, and for training and test motions whose statistics were either similar or dissimilar. Our results suggest that the use of a PT decoding framework can offset the loss in accuracy associated with decoding unsorted MU neural signals. Coupled with up to a 20-fold reduction in the number of decoding weights and the ability to implement the filtering in hardware, this approach could reduce the computational requirements and thus increase the portability of next generation brain-machine interfaces.

Encoding of error and learning to correct that error by the Purkinje cells of the cerebellum

The primary output cells of the cerebellar cortex, Purkinje cells, make kinematic predictions about ongoing movements via high-frequency simple spikes, but receive sensory error information about that movement via low-frequency complex spikes (CS). How is the vector space of sensory errors encoded by this low-frequency signal? Here we measured Purkinje cell activity in the oculomotor vermis of animals during saccades, then followed the chain of events from experience of visual error, generation of CS, modulation of simple spikes, and ultimately change in motor output. We found that while error direction affected the probability of CS, error magnitude altered its temporal distribution. Production of CS changed the simple spikes on the next trial, but regardless of the actual visual error, this change biased the movement only along a vector that was parallel to the Purkinje cell’s preferred error. From these results, we inferred the anatomy of a sensory-to-motor adaptive controller that transformed visual error vectors into motor-corrections.Herzfeld et al. examine how the cerebellum learns to correct movements. They find a timing code that links a Purkinje cell’s preference for error to its downstream projection on motor effectors that produce force to correct for that error.

Cerebellum estimates the sensory state of the body

A recent neurophysiology study provides data from the cerebellar vermis/nodulus, where neurons encode translation of the head, even when these translations are induced via an illusion. These data provide new neurophysiological evidence that the cerebellum is important for computations involving internal models of motion, estimating the state of the body.

Pools of virtual boxes: building campus grids with virtual machines

A campus grid is a critical component of research cyberinfrastructure. A grid facilitates resource sharing, improving collaboration and increasing the capability to address large scale scientific and engineering problems. Resources comprising a grid include research clusters, centrally managed clusters, opportunistic use of desktop computers, campus clouds, and commercial clouds. Advances in virtual machine technology and hypervisor availability has made the use of virtual machines an attractive tool for building campus grids. Pools of Virtual Boxes (POVB) is an open-source, dedicated virtual machine environment for rapidly deploying a campus grid. POVB is targeted at institutions where financial and administrative constraints prevent large scale changes in computational infrastructure. The POVB distribution includes: services that manage the virtual machine hypervisor, services that communicate select host information with the Linux guest for debugging and resource utilization policies, a bootstrapping framework for building virtual images, and a third-party package deployment framework that integrates with Condor to advertise available software services. We report on the design and implementation of POVB and examine a deployment of several hundred POVB instances, which has been operational for over a year. POVB has been released under the GNU Public License Version 3, and is available at http://poolsofvirtualb.sourceforge.net.

Movement vigor as a traitlike attribute of individuality

A common aspect of individuality is our subjective preferences in evaluation of reward and effort. The neural circuits that evaluate these commodities influence circuits that control our movements, raising the possibility that vigor differences between individuals may also be a trait of individuality, reflecting a willingness to expend effort. In contrast, classic theories in motor control suggest that vigor differences reflect a speed-accuracy trade-off, predicting that those who move fast are sacrificing accuracy for speed. Here we tested these contrasting hypotheses. We measured motion of the eyes, head, and arm in healthy humans during various elementary movements (saccades, head-free gaze shifts, and reaching). For each person we characterized their vigor, i.e., the speed with which they moved a body part (peak velocity) with respect to the population mean. Some moved with low vigor, while others moved with high vigor. Those with high vigor tended to react sooner to a visual stimulus, moving both their eyes and arm with a shorter reaction time. Arm and head vigor were tightly linked: individuals who moved their head with high vigor also moved their arm with high vigor. However, eye vigor did not correspond strongly with arm or head vigor. In all modalities, vigor had no impact on end-point accuracy, demonstrating that differences in vigor were not due to a speed-accuracy trade-off. Our results suggest that movement vigor may be a trait of individuality, not reflecting a willingness to accept inaccuracy but demonstrating a propensity to expend effort. NEW & NOTEWORTHY A common aspect of individuality is how we evaluate economic variables like reward and effort. This valuation affects not only decision making but also motor control, raising the possibility that vigor may be distinct between individuals but conserved across movements within an individual. Here we report conservation of vigor across elementary skeletal movements, but not eye movements, raising the possibility that the individuality of our movements may be driven by a common neural mechanism of effort evaluation across modalities of skeletal motor control.

Cerebellar output encodes a corrective saccadic command (Commentary on Sun et al.)

The integrity of the cerebellum is critical for accurate eye movements. Disruption of neurons in either the cerebellar oculomotor vermis or its projections to the most medial output nucleus of the cerebellum, the caudal fastigial nucleus (cFN), results in significant saccadic dysmetria (Ritchie, 1976; Ohtsuka et al., 1994; Goffart et al., 2004; Buzunov et al., 2013). The relationship between cFN neuron firing rates and saccade kinematic parameters is quite variable across neurons (Hepp et al., 1982; Fuchs et al., 1993), suggesting that a direct encoding of saccade parameters may not occur in the responses of individual neurons of cFN. However, previous studies have generally agreed that saccaderelated cFN neurons tend to fire earlier for horizontal saccades made in the contraversive vs. ipsiversive directions (Ohtsuka & Noda, 1991; Fuchs et al., 1993; Helmchen & B€uttner, 1995; Kleine et al., 2003). Taken together, these results have led to the hypothesis that saccade properties are related to the timing of responses in cFN rather than response magnitude. Under this hypothesis, neurons should fire early for contraversive saccades, helping to accelerate the eye, whereas the delayed firing of ipsiversive neurons serves to decelerate and stop the eye at saccade termination. However, new data reported by Sun et al. (2016) call this view into question. Sun and colleagues recorded single-unit cFN neuron activity while primates made saccades of various magnitudes. These saccades included very small saccades, made during periods of fixation (microsaccades), as well as larger magnitude goal-directed saccades to peripheral targets. Their results suggest that cFN neuron responses exist on a continuum between these two types of saccades. That is, microand macrosaccades likely share common neural mechanisms of generation. Therefore, the responses of cFN neurons can be interpreted similarly across a large range of saccadic amplitudes (e.g. 0.5–15°). The duration of a typical saccade is on the order of 60 ms. Taking advantage of the temporally short nature of saccades, Sun et al. combined the responses of individual cFN neurons recorded across different sessions to yield an estimate of the firing of a population of simultaneously recorded neurons. This population response represents an estimate of the combined response of all saccade-related cFN neurons during a saccade. Strikingly, the timing of the population response did not occur earlier for contraversive compared to ipsiversive saccades, as would be anticipated by previous single-unit studies. Rather, both directions of saccades resulted in a population response that preceded the start of the saccade, and began at approximately the same time. How can these two deep nuclei be involved in saccade acceleration and deceleration when the timing of the responses to contraversive and ipsiversive saccades are not different? We recently suggested that populations of Purkinje cells (P-cells) in the oculomotor vermis (OMV) of the cerebellum encode the velocity and direction of an impending eye movement as a gain-field (Herzfeld et al., 2015). In this encoding, the population response of P-cells in OMV increases linearly with increasing eye velocity, whereas the direction is encoded as a cosine in which the population response is smallest for the direction of error that produces the highest probability of complex spikes (called CS-on), and highest for the direction of error that produces the lowest probability of complex spikes (CS-off). Given that inferior olive neurons project to the contralateral P-cells, P-cell simple spike encoding has the highest gain for contraversive saccades, and lowest gain for ipsiversive saccades. These P-cells inhibit neurons in cFN. Data from Sun et al. are consistent with this P-cell encoding, demonstrating that the cFN population response is larger for ipsiversive saccades than contraversive, a property which is not reliably found in the responses of individual cells. Combining this new experimental evidence with previous studies, it seems likely that the encoding of saccade kinematic parameters in cFN is not temporal in nature but rather results from differences in the magnitude of the overall cFN response. How can we interpret the results of previous studies in this framework? Goffart et al. (2004) have previously suggested that cFN inactivation and lesion studies could be understood by interpreting experimental results in the context of the bilaterality of cFN. Under this encoding scheme, cFN activity does not strictly encode acceleration and deceleration, but rather the two nuclei act as antagonists, in which one cFN can be thought of as ‘pushing’ the eye while the other cFN ‘pulls.’ In this framework, equal activity in the two nuclei does not result in horizontal movement of the eye – cerebellar-dependent movement results only when the magnitude of the responses in the two nuclei are different. In this way, the output of the cerebellum is defined by the difference in the activities of the two cFNs, providing a ‘correction’ on top of the current movement (Fuchs et al., 1993). The population-level analysis presented by Sun et al. further clarifies this hypothesis. During a saccade, both sides of OMV are simultaneously active with a higher gain on the contralateral side (Herzfeld et al., 2015). This, in turn, results in the opposite scenario for the fastigial nuclei: higher firing rates for the ipsilateral vs. contralateral sides, without any differences in the timing of the response. Projections from cFN synapse throughout the brainstem saccadic circuitry, particularly on inhibitory and excitatory burst neurons, eventually acting on the motor

Impact of Human Behavioral Papers at Journal of Neuroscience

A recent policy change at Journal of Neuroscience (JN) has significantly increased editorial “desk rejections”, reducing the number of papers that are considered for publication. Survey results from 130 scientists suggested that the new policy may have had a particularly large impact on studies that focused on human behavioral techniques. To quantify the effects of the new policy, we gathered data on all papers ever published in JN (~35,000), as well as all papers that had cited the JN papers (~2.7 million papers). We found that the recent change in editorial policy had disproportionately affected rejection rates of human behavioral papers: since 2015, the number of human behavioral papers as a proportion of all papers published in JN has seen a 30% decline. While there has been a long-term declining trend in the journal’s impact factor, we found that this declining impact factor was shared by both human behavioral papers as well as other papers in the journal. This suggested that whatever may have been the source of the declining impact factor at JN, this source was affecting the various areas of research equally. That is, it was unlikely that papers in any one field were responsible for the declining impact factor number at JN. Indeed, when impact was measured over the long-term, we found that the average human behavioral paper at JN consistently outperformed other papers, generating a significantly higher number of citations per year at 5 and 10 years post publication.