Reza Shadmehr

Error Correction, Sensory Prediction, and Adaptation in Motor Control

Motor control is the study of how organisms make accurate goal-directed movements. Here we consider two problems that the motor system must solve in order to achieve such control. The first problem is that sensory feedback is noisy and delayed, which can make movements inaccurate and unstable. The second problem is that the relationship between a motor command and the movement it produces is variable, as the body and the environment can both change. A solution is to build adaptive internal models of the body and the world. The predictions of these internal models, called forward models because they transform motor commands into sensory consequences, can be used to both produce a lifetime of calibrated movements, and to improve the ability of the sensory system to estimate the state of the body and the world around it. Forward models are only useful if they produce unbiased predictions. Evidence shows that forward models remain calibrated through motor adaptation: learning driven by sensory prediction errors.

Learning of action through adaptive combination of motor primitives.

Understanding how the brain constructs movements remains a fundamental challenge in neuroscience. The brain may control complex movements through flexible combination of motor primitives, where each primitive is an element of computation in the sensorimotor map that transforms desired limb trajectories into motor commands. Theoretical studies have shown that a systems ability to learn action depends on the shape of its primitives. Using a time-series analysis of error patterns, here we show that humans learn the dynamics of reaching movements through a flexible combination of primitives that have gaussian-like tuning functions encoding hand velocity. The wide tuning of the inferred primitives predicts limitations on the brains ability to represent viscous dynamics. We find close agreement between the predicted limitations and the subjects’ adaptation to new force fields. The mathematical properties of the derived primitives resemble the tuning curves of Purkinje cells in the cerebellum. The activity of these cells may encode primitives that underlie the learning of dynamics.

A COMPUTATIONAL NEUROANATOMY FOR MOTOR CONTROL

The study of patients to infer normal brain function has a long tradition in neurology and psychology. More recently, the motor system has been subject to quantitative and computational characterization. The purpose of this review is to argue that the lesion approach and theoretical motor control can mutually inform each other. Specifically, one may identify distinct motor control processes from computational models and map them onto specific deficits in patients. Here we review some of the impairments in motor control, motor learning and higher-order motor control in patients with lesions of the corticospinal tract, the cerebellum, parietal cortex, the basal ganglia, and the medial temporal lobe. We attempt to explain some of these impairments in terms of computational ideas such as state estimation, optimization, prediction, cost, and reward. We suggest that a function of the cerebellum is system identification: to build internal models that predict sensory outcome of motor commands and correct motor commands through internal feedback. A function of the parietal cortex is state estimation: to integrate the predicted proprioceptive and visual outcomes with sensory feedback to form a belief about how the commands affected the states of the body and the environment. A function of basal ganglia is related to optimal control: learning costs and rewards associated with sensory states and estimating the “cost-to-go” during execution of a motor task. Finally, functions of the primary and the premotor cortices are related to implementing the optimal control policy by transforming beliefs about proprioceptive and visual states, respectively, into motor commands.

Neural Correlates of Reach Errors

Reach errors may be broadly classified into errors arising from unpredictable changes in target location, called target errors, and errors arising from miscalibration of internal models (e.g., when prisms alter visual feedback or a force field alters limb dynamics), called execution errors. Execution errors may be caused by miscalibration of dynamics (e.g., when a force field alters limb dynamics) or by miscalibration of kinematics (e.g., when prisms alter visual feedback). Although all types of errors lead to similar on-line corrections, we found that the motor system showed strong trial-by-trial adaptation in response to random execution errors but not in response to random target errors. We used functional magnetic resonance imaging and a compatible robot to study brain regions involved in processing each kind of error. Both kinematic and dynamic execution errors activated regions along the central and the postcentral sulci and in lobules V, VI, and VIII of the cerebellum, making these areas possible sites of plastic changes in internal models for reaching. Only activity related to kinematic errors extended into parietal area 5. These results are inconsistent with the idea that kinematics and dynamics of reaching are computed in separate neural entities. In contrast, only target errors caused increased activity in the striatum and the posterior superior parietal lobule. The cerebellum and motor cortex were as strongly activated as with execution errors. These findings indicate a neural and behavioral dissociation between errors that lead to switching of behavioral goals and errors that lead to adaptation of internal models of limb dynamics and kinematics.

Computational nature of human adaptive control during learning of reaching movements in force fields

Abstract. Learning to make reaching movements in force fields was used as a paradigm to explore the system architecture of the biological adaptive controller. We compared the performance of a number of candidate control systems that acted on a model of the neuromuscular system of the human arm and asked how well the dynamics of the candidate system compared with the movement characteristics of 16 subjects. We found that control via a supra-spinal system that utilized an adaptive inverse model resulted in dynamics that were similar to that observed in our subjects, but lacked essential characteristics. These characteristics pointed to a different architecture where descending commands were influenced by an adaptive forward model. However, we found that control via a forward model alone also resulted in dynamics that did not match the behavior of the human arm. We considered a third control architecture where a forward model was used in conjunction with an inverse model and found that the resulting dynamics were remarkably similar to that observed in the experimental data. The essential property of this control architecture was that it predicted a complex pattern of near-discontinuities in hand trajectory in the novel force field. A nearly identical pattern was observed in our subjects, suggesting that generation of descending motor commands was likely through a control system architecture that included both adaptive forward and inverse models. We found that as subjects learned to make reaching movements, adaptation rates for the forward and inverse models could be independently estimated and the resulting changes in performance of subjects from movement to movement could be accurately accounted for. Results suggested that the adaptation of the forward model played a dominant role in the motor learning of subjects. After a period of consolidation, the rates of adaptation in the internal models were significantly larger than those observed before the memory had consolidated. This suggested that consolidation of motor memory coincided with freeing of certain computational resources for subsequent learning.

Motor Disorder in Huntington’s Disease Begins as a Dysfunction in Error Feedback Control

A steady progression of motor dysfunction takes place in Huntingtons disease (HD). The origin of this disturbance with relation to the motor control process is not understood. Here we studied reaching movements in asymptomatic HD gene-carriers (AGCs) and subjects with manifest HD. We found that movement jerkiness, which characterizes the smoothness and efficiency of motion, was a sensitive indicator of presymptomatic HD progression. A large fraction of AGCs displayed elevated jerk even when more than seven years remained until predicted disease onset. Movement termination was disturbed much more than initiation and was highly variable from trial to trial. Analysis of this variability revealed that the sensitivity of end-movement jerk to subtle, self-generated early-movement errors was greater in HD subjects than in controls. Additionally, we found that HD corrective responses to externally-generated force pulses were greatly disturbed, indicating that HD subjects display aberrant responses to both external and self-generated errors. Because feedback corrections are driven by error and are delayed such that they predominantly affect movement termination, these findings suggest that a dysfunction in error correction characterizes the motor control deficit in early HD. This dysfunction may be observed years before clinical disease onset and grows worse as the disease progresses.

Consolidation of motor memory

An issue of great recent interest is whether motor memory consolidates in a manner analogous to declarative memory--that is, with the formation of a memory that progresses over time from a fragile state, which is susceptible to interference by a lesion or a conflicting motor task, to a stabilized state, which is resistant to such interference. Here, we first review studies that examine the anatomical basis for motor consolidation. Evidence implicates cerebellar circuitry in two types of associative motor learning--eyelid conditioning and vestibulo-ocular reflex adaptation--and implicates primary motor cortex in skilled finger movements. We also review evidence for and against a consolidation process for adaptation of arm movements. We propose that contradictions have arisen because consolidation can be masked by inhibition of memory retrieval.

The coordination of movement: optimal feedback control and beyond

Optimal control theory and its more recent extension, optimal feedback control theory, provide valuable insights into the flexible and task-dependent control of movements. Here, we focus on the problem of coordination, defined as movements that involve multiple effectors (muscles, joints or limbs). Optimal control theory makes quantitative predictions concerning the distribution of work across multiple effectors. Optimal feedback control theory further predicts variation in feedback control with changes in task demands and the correlation structure between different effectors. We highlight two crucial areas of research, hierarchical control and the problem of movement initiation, that need to be developed for an optimal feedback control theory framework to characterise movement coordination more fully and to serve as a basis for studying the neural mechanisms involved in voluntary motor control.

The dynamics of memory as a consequence of optimal adaptation to a changing body.

There are many causes for variation in the responses of the motor apparatus to neural commands. Fast-timescale disturbances occur when muscles fatigue. Slow-timescale disturbances occur when muscles are damaged or when limb dynamics change as a result of development. To maintain performance, motor commands need to adapt. Computing the best adaptation in response to any performance error results in a credit assignment problem: which timescale is responsible for this disturbance? Here we show that a Bayesian solution to this problem accounts for numerous behaviors of animals during both short- and long-term training. Our analysis focused on characteristics of the oculomotor system during learning, including the effects of time passage. However, we suggest that learning and memory in other paradigms, such as reach adaptation, adaptation of visual neurons and retrieval of declarative memories, largely follow similar rules.

Motor Adaptation as a Process of Reoptimization

Adaptation is sometimes viewed as a process in which the nervous system learns to predict and cancel effects of a novel environment, returning movements to near baseline (unperturbed) conditions. An alternate view is that cancellation is not the goal of adaptation. Rather, the goal is to maximize performance in that environment. If performance criteria are well defined, theory allows one to predict the reoptimized trajectory. For example, if velocity-dependent forces perturb the hand perpendicular to the direction of a reaching movement, the best reach plan is not a straight line but a curved path that appears to overcompensate for the forces. If this environment is stochastic (changing from trial to trial), the reoptimized plan should take into account this uncertainty, removing the overcompensation. If the stochastic environment is zero-mean, peak velocities should increase to allow for more time to approach the target. Finally, if one is reaching through a via-point, the optimum plan in a zero-mean deterministic environment is a smooth movement but in a zero-mean stochastic environment is a segmented movement. We observed all of these tendencies in how people adapt to novel environments. Therefore, motor control in a novel environment is not a process of perturbation cancellation. Rather, the process resembles reoptimization: through practice in the novel environment, we learn internal models that predict sensory consequences of motor commands. Through reward-based optimization, we use the internal model to search for a better movement plan to minimize implicit motor costs and maximize rewards.