J. Randall Flanagan

Coding and use of tactile signals from the fingertips in object manipulation tasks

During object manipulation tasks, the brain selects and implements action-phase controllers that use sensory predictions and afferent signals to tailor motor output to the physical properties of the objects involved. Analysis of signals in tactile afferent neurons and central processes in humans reveals how contact events are encoded and used to monitor and update task performance.

Eye–Hand Coordination in Object Manipulation

We analyzed the coordination between gaze behavior, fingertip movements, and movements of the manipulated object when subjects reached for and grasped a bar and moved it to press a target-switch. Subjects almost exclusively fixated certain landmarks critical for the control of the task. Landmarks at which contact events took place were obligatory gaze targets. These included the grasp site on the bar, the target, and the support surface where the bar was returned after target contact. Any obstacle in the direct movement path and the tip of the bar were optional landmarks. Subjects never fixated the hand or the moving bar. Gaze and hand/bar movements were linked concerning landmarks, with gaze leading. The instant that gaze exited a given landmark coincided with a kinematic event at that landmark in a manner suggesting that subjects monitored critical kinematic events for phasic verification of task progress and subgoal completion. For both the obstacle and target, subjects directed saccades and fixations to sites that were offset from the physical extension of the objects. Fixations related to an obstacle appeared to specify a location around which the extending tip of the bar should travel. We conclude that gaze supports hand movement planning by marking key positions to which the fingertips or grasped object are subsequently directed. The salience of gaze targets arises from the functional sensorimotor requirements of the task. We further suggest that gaze control contributes to the development and maintenance of sensorimotor correlation matrices that support predictive motor control in manipulation.

Principles of sensorimotor learning

The exploits of Martina Navratilova and Roger Federer represent the pinnacle of motor learning. However, when considering the range and complexity of the processes that are involved in motor learning, even the mere mortals among us exhibit abilities that are impressive. We exercise these abilities when taking up new activities — whether it is snowboarding or ballroom dancing — but also engage in substantial motor learning on a daily basis as we adapt to changes in our environment, manipulate new objects and refine existing skills. Here we review recent research in human motor learning with an emphasis on the computational mechanisms that are involved.

Perspectives and problems in motor learning

Movement provides the only means we have to interact with both the world and other people. Such interactions can be hard-wired or learned through experience with the environment. Learning allows us to adapt to a changing physical environment as well as to novel conventions developed by society. Here we review motor learning from a computational perspective, exploring the need for motor learning, what is learned and how it is represented, and the mechanisms of learning. We relate these computational issues to empirical studies on motor learning in humans.

Action plans used in action observation.

How do we understand the actions of others? According to the direct matching hypothesis, action understanding results from a mechanism that maps an observed action onto motor representations of that action. Although supported by neurophysiological and brain-imaging studies, direct evidence for this hypothesis is sparse. In visually guided actions, task-specific proactive eye movements are crucial for planning and control. Because the eyes are free to move when observing such actions, the direct matching hypothesis predicts that subjects should produce eye movements similar to those produced when they perform the tasks. If an observer analyses action through purely visual means, however, eye movements will be linked reactively to the observed action. Here we show that when subjects observe a block stacking task, the coordination between their gaze and the actors hand is predictive, rather than reactive, and is highly similar to the gaze–hand coordination when they perform the task themselves. These results indicate that during action observation subjects implement eye motor programs directed by motor representations of manual actions and thus provide strong evidence for the direct matching hypothesis.

Independence of perceptual and sensorimotor predictions in the size–weight illusion

The smaller of two equally weighted objects is judged to be heavier when lifted. Here we disproved a leading hypothesis that this size–weight illusion is caused by a mismatch between predicted and actual sensory feedback. We showed that when subjects repeatedly lifted equally heavy large and small objects in alternation, they learned to scale their fingertip forces precisely for the true object weights and thus exhibited accurate sensorimotor prediction. The size–weight illusion nevertheless persisted, suggesting that the illusion can be caused by high-level cognitive and perceptual factors and indicating that the sensorimotor system can operate independently of the cognitive/perceptual system.

Control strategies in object manipulation tasks

The remarkable manipulative skill of the human hand is not the result of rapid sensorimotor processes, nor of fast or powerful effector mechanisms. Rather, the secret lies in the way manual tasks are organized and controlled by the nervous system. At the heart of this organization is prediction. Successful manipulation requires the ability both to predict the motor commands required to grasp, lift, and move objects and to predict the sensory events that arise as a consequence of these commands.

Failure to Consolidate the Consolidation Theory of Learning for Sensorimotor Adaptation Tasks

An influential idea in human motor learning is that there is a consolidation period during which motor memories are transformed from a fragile to a permanent state, no longer susceptible to interference from new learning. The evidence supporting this idea comes from studies showing that the motor memory of a task (A) is lost when an opposing task (B) is experienced soon after, but not if sufficient time is allowed to pass (∼6 hr). We report results from three laboratories challenging this consolidation idea. We used an ABA paradigm in the context of a reaching task to assess the influence of experiencing B after A on the retention of A. In two experiments using visuomotor rotations, we found that B fully interferes with the retention of A even when B is experienced 24 hr after A. Contrary to previous reports, in four experiments on learning force fields, we also observed full interference between A and B when they are separated by 24 hr or even 1 week. This latter result holds for both position-dependent and velocity-dependent force fields. For both the visuomotor and force-field tasks, complete interference is still observed when the possible affects of anterograde interference are controlled through the use of washout trials. Our results fail to support the idea that motor memories become consolidated into a protected state. Rather, they are consistent with recent ideas of memory formation, which propose that memories can shift between active and inactive states.

Control of Trajectory Modifications in Target-Directed Reaching

Human reaching movements to fixed and displaced visual targets were recorded and compared with simulated movements generated by using a two-joint arm model based on the equilibrium-point (EP) hypothesis (lambda model) of motor control (Feldman, 1986). The aim was to investigate the form of central control signals underlying these movements. According to this hypothesis, movements result from changes in control variables that shift the equilibrium position (EP) of the arm. At any time, muscle activations and forces will depend on the difference between the arms EP and its actual position and on the limbs velocity. In this article, we suggest that the direction of EP shift in reaching is specified at the hand level, whereas the rate of EP shift may be specified at the hand or joint level. A common mechanism underlying reaching to fixed and displaced targets is proposed whereby the EP of the hand shifts in a straight line toward the present target. After the target is displaced, the direction of the hand EP shift is modified toward the second target. The results suggest that the rate of shift of the hand EP may be modified for movements in different parts of the work space. The model, with control signals that vary in a simple fashion over time, is able to generate the kinematic patterns observed empirically.

Eye–Hand Coordination during Learning of a Novel Visuomotor Task

We investigated how gaze behavior and eye–hand coordination change when subjects learned a challenging visuomotor task that required acquisition of a novel mapping between bimanual actions and their visual sensory consequences. By applying isometric forces and torques to a rigid tool held freely between the two hands, subjects learned to control a cursor on a computer screen to hit successively displayed targets as quickly as possible. The learning occurred in stages that could be distinguished by changes in performance (target–hit rate) as well as by gaze behavior and eye–hand coordination. In a first exploratory stage, the hit rate was consistently low, the cursor position varied widely, and gaze typically pursued the cursor. In a second skill acquisition stage, the hit rate improved rapidly, and gaze fixations began to mark predictively desired cursor positions, indicating that subjects started to program spatially congruent eye and hand motor commands. In a third skill refinement stage, performance continued to improve gradually, and gaze shifted directly toward the target. We suggest that during the exploratory stage, the learner attempts to establish basic mapping rules between manual actions and eye-movement commands. In this process, subjects may establish correlations between hand motor commands and their visual sensory consequences, primarily in fovea-anchored, gaze-centered coordinates, and correlations between recent hand motor commands and eye motor commands. The established mapping rules are then implemented and refined in the skill acquisition and refinement stages.