Maurice A. Smith

Error Correction, Sensory Prediction, and Adaptation in Motor Control

Motor control is the study of how organisms make accurate goal-directed movements. Here we consider two problems that the motor system must solve in order to achieve such control. The first problem is that sensory feedback is noisy and delayed, which can make movements inaccurate and unstable. The second problem is that the relationship between a motor command and the movement it produces is variable, as the body and the environment can both change. A solution is to build adaptive internal models of the body and the world. The predictions of these internal models, called forward models because they transform motor commands into sensory consequences, can be used to both produce a lifetime of calibrated movements, and to improve the ability of the sensory system to estimate the state of the body and the world around it. Forward models are only useful if they produce unbiased predictions. Evidence shows that forward models remain calibrated through motor adaptation: learning driven by sensory prediction errors.

Motor Disorder in Huntington’s Disease Begins as a Dysfunction in Error Feedback Control

A steady progression of motor dysfunction takes place in Huntingtons disease (HD). The origin of this disturbance with relation to the motor control process is not understood. Here we studied reaching movements in asymptomatic HD gene-carriers (AGCs) and subjects with manifest HD. We found that movement jerkiness, which characterizes the smoothness and efficiency of motion, was a sensitive indicator of presymptomatic HD progression. A large fraction of AGCs displayed elevated jerk even when more than seven years remained until predicted disease onset. Movement termination was disturbed much more than initiation and was highly variable from trial to trial. Analysis of this variability revealed that the sensitivity of end-movement jerk to subtle, self-generated early-movement errors was greater in HD subjects than in controls. Additionally, we found that HD corrective responses to externally-generated force pulses were greatly disturbed, indicating that HD subjects display aberrant responses to both external and self-generated errors. Because feedback corrections are driven by error and are delayed such that they predominantly affect movement termination, these findings suggest that a dysfunction in error correction characterizes the motor control deficit in early HD. This dysfunction may be observed years before clinical disease onset and grows worse as the disease progresses.

Temporal structure of motor variability is dynamically regulated and predicts motor learning ability

Individual differences in motor learning ability are widely acknowledged, yet little is known about the factors that underlie them. Here we explore whether movement-to-movement variability in motor output, a ubiquitous if often unwanted characteristic of motor performance, predicts motor learning ability. Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning both across individuals and across tasks in two different paradigms, one relying on reward-based learning to shape specific arm movement trajectories and the other relying on error-based learning to adapt movements in novel physical environments. We proceeded to show that training can reshape the temporal structure of motor variability, aligning it with the trained task to improve learning. These results provide experimental support for the importance of action exploration, a key idea from reinforcement learning theory, showing that motor variability facilitates motor learning in humans and that our nervous systems actively regulate it to improve learning.

Shared Internal Models for Feedforward and Feedback Control

A child often learns to ride a bicycle in the driveway, free of unforeseen obstacles. Yet when she first rides in the street, we hope that if a car suddenly pulls out in front of her, she will combine her innate goal of avoiding an accident with her learned knowledge of the bicycle, and steer away or brake. In general, when we train to perform a new motor task, our learning is most robust if it updates the rules of online error correction to reflect the rules and goals of the new task. Here we provide direct evidence that, after a new feedforward motor adaptation, motor feedback responses to unanticipated errors become precisely task appropriate, even when such errors were never experienced during training. To study this ability, we asked how, if at all, do online responses to occasional, unanticipated force pulses during reaching arm movements change after adapting to altered arm dynamics? Specifically, do they change in a task-appropriate manner? In our task, subjects learned novel velocity-dependent dynamics. However, occasional force-pulse perturbations produced unanticipated changes in velocity. Therefore, after adaptation, task-appropriate responses to unanticipated pulses should compensate corresponding changes in velocity-dependent dynamics. We found that after adaptation, pulse responses precisely compensated these changes, although they were never trained to do so. These results provide evidence for a smart feedback controller which automatically produces responses specific to the learned dynamics of the current task. To accomplish this, the neural processes underlying feedback control must (1) be capable of accurate real-time state prediction for velocity via a forward model and (2) have access to recently learned changes in internal models of limb dynamics.

A Gain-Field Encoding of Limb Position and Velocity in the Internal Model of Arm Dynamics

Adaptability of reaching movements depends on a computation in the brain that transforms sensory cues, such as those that indicate the position and velocity of the arm, into motor commands. Theoretical consideration shows that the encoding properties of neural elements implementing this transformation dictate how errors should generalize from one limb position and velocity to another. To estimate how sensory cues are encoded by these neural elements, we designed experiments that quantified spatial generalization in environments where forces depended on both position and velocity of the limb. The patterns of error generalization suggest that the neural elements that compute the transformation encode limb position and velocity in intrinsic coordinates via a gain-field; i.e., the elements have directionally dependent tuning that is modulated monotonically with limb position. The gain-field encoding makes the counterintuitive prediction of hypergeneralization: there should be growing extrapolation beyond the trained workspace. Furthermore, nonmonotonic force patterns should be more difficult to learn than monotonic ones. We confirmed these predictions experimentally.

Long-Term Retention Explained by a Model of Short-Term Learning in the Adaptive Control of Reaching

Extensive theoretical, psychophysical, and neurobiological work has focused on the mechanisms by which short-term learning develops into long-term memory. Better understanding of these mechanisms may lead to the ability to improve the efficiency of training procedures. A key phenomenon in the formation of long-term memory is the effect of over learning on retention-discovered by Ebbinghaus in 1885: when the initial training period in a task is prolonged even beyond what is necessary for good immediate recall, long-term retention improves. Although this over learning effect has received considerable attention as a phenomenon in psychology research, the mechanisms governing this process are not well understood, and the ability to predict the benefit conveyed by varying degrees of over learning does not yet exist. Here we studied the relationship between the duration of an initial training period and the amount of retention 24 h later for the adaptation of human reaching arm movements to a novel force environment. We show that in this motor adaptation task, the amount of long-term retention is predicted not by the overall performance level achieved during the training period but rather by the level of a specific component process in a multi-rate model of short-term memory formation. These findings indicate that while multiple learning processes determine the ability to learn a motor adaptation, only one provides a gateway to long-term memory formation. Understanding the dynamics of this key learning process may allow for the rational design of training and rehabilitation paradigms that maximize the long-term benefit of each session.

Dissociable effects of the implicit and explicit memory systems on learning control of reaching

Adaptive control of reaching depends on internal models that associate states in which the limb experienced a force perturbation with motor commands that can compensate for it. Limb state can be sensed via both vision and proprioception. However, adaptation of reaching in novel dynamics results in generalization in the intrinsic coordinates of the limb, suggesting that the proprioceptive states in which the limb was perturbed dominate representation of limb state. To test this hypothesis, we considered a task where position of the hand during a reach was correlated with patterns of force perturbation. This correlation could be sensed via vision, proprioception, or both. As predicted, when the correlations could be sensed only via proprioception, learning was significantly better as compared to when the correlations could only be sensed through vision. We found that learning with visual correlations resulted in subjects who could verbally describe the patterns of perturbations but this awareness was never observed in subjects who learned the task with only proprioceptive correlations. We manipulated the relative values of the visual and proprioceptive parameters and found that the probability of becoming aware strongly depended on the correlations that subjects could visually observe. In all conditions, aware subjects demonstrated a small but significant advantage in their ability to adapt their motor commands. Proprioceptive correlations produced an internal model that strongly influenced reaching performance yet did not lead to awareness. Visual correlations strongly increased the probability of becoming aware, yet had a much smaller but still significant effect on reaching performance. Therefore, practice resulted in acquisition of both implicit and explicit internal models.

Reduction in Learning Rates Associated with Anterograde Interference Results from Interactions between Different Timescales in Motor Adaptation

Prior experiences can influence future actions. These experiences can not only drive adaptive changes in motor output, but they can also modulate the rate at which these adaptive changes occur. Here we studied anterograde interference in motor adaptation – the ability of a previously learned motor task (Task A) to reduce the rate of subsequently learning a different (and usually opposite) motor task (Task B). We examined the formation of the motor systems capacity for anterograde interference in the adaptive control of human reaching-arm movements by determining the amount of interference after varying durations of exposure to Task A (13, 41, 112, 230, and 369 trials). We found that the amount of anterograde interference observed in the learning of Task B increased with the duration of Task A. However, this increase did not continue indefinitely; instead, the interference reached asymptote after 15–40 trials of Task A. Interestingly, we found that a recently proposed multi-rate model of motor adaptation, composed of two distinct but interacting adaptive processes, predicts several key features of the interference patterns we observed. Specifically, this computational model (without any free parameters) predicts the initial growth and leveling off of anterograde interference that we describe, as well as the asymptotic amount of interference that we observe experimentally (R2 = 0.91). Understanding the mechanisms underlying anterograde interference in motor adaptation may enable the development of improved training and rehabilitation paradigms that mitigate unwanted interference.

Adaptation and generalization in acceleration-dependent force fields

Any passive rigid inertial object that we hold in our hand, e.g., a tennis racquet, imposes a field of forces on the arm that depends on limb position, velocity, and acceleration. A fundamental characteristic of this field is that the forces due to acceleration and velocity are linearly separable in the intrinsic coordinates of the limb. In order to learn such dynamics with a collection of basis elements, a control system would generalize correctly and therefore perform optimally if the basis elements that were sensitive to limb velocity were not sensitive to acceleration, and vice versa. However, in the mammalian nervous system proprioceptive sensors like muscle spindles encode a nonlinear combination of all components of limb state, with sensitivity to velocity dominating sensitivity to acceleration. Therefore, limb state in the space of proprioception is not linearly separable despite the fact that this separation is a desirable property of control systems that form models of inertial objects. In building internal models of limb dynamics, does the brain use a representation that is optimal for control of inertial objects, or a representation that is closely tied to how peripheral sensors measure limb state? Here we show that in humans, patterns of generalization of reaching movements in acceleration-dependent fields are strongly inconsistent with basis elements that are optimized for control of inertial objects. Unlike a robot controller that models the dynamics of the natural world and represents velocity and acceleration independently, internal models of dynamics that people learn appear to be rooted in the properties of proprioception, nonlinearly responding to the pattern of muscle activation and representing velocity more strongly than acceleration.

Motor Memory Is Encoded as a Gain-Field Combination of Intrinsic and Extrinsic Action Representations

Actions can be planned in either an intrinsic (body-based) reference frame or an extrinsic (world-based) frame, and understanding how the internal representations associated with these frames contribute to the learning of motor actions is a key issue in motor control. We studied the internal representation of this learning in human subjects by analyzing generalization patterns across an array of different movement directions and workspaces after training a visuomotor rotation in a single movement direction in one workspace. This provided a dense sampling of the generalization function across intrinsic and extrinsic reference frames, which allowed us to dissociate intrinsic and extrinsic representations and determine the manner in which they contributed to the motor memory for a trained action. A first experiment showed that the generalization pattern reflected a memory that was intermediate between intrinsic and extrinsic representations. A second experiment showed that this intermediate representation could not arise from separate intrinsic and extrinsic learning. Instead, we find that the representation of learning is based on a gain-field combination of local representations in intrinsic and extrinsic coordinates. This gain-field representation generalizes between actions by effectively computing similarity based on the (Mahalanobis) distance across intrinsic and extrinsic coordinates and is in line with neural recordings showing mixed intrinsic-extrinsic representations in motor and parietal cortices.