Feedback Gains modulate with Motor Memory Uncertainty
Feedback Gains modulate with Motor Memory Uncertainty
Abbreviated title: Internal Model Uncertainty affects Feedback Gains Sae
Franklin & David W. Franklin Institute of Cognitive Systems, Department of Electrical and Computer Engineering, Technical University of Munich, Germany. Neuromuscular Diagnostics, Department of Sport and Health Science, Technical University of Munich, Germany. Correspondence: Dr. David Franklin Neuromuscular Diagnostics, Department of Sport and Health Science, Technical University of Munich, Campus D - Uptown München, Georg-Brauchle-Ring 60/62, 80992 Germany Telephone: +49.89.289.24536 Email: [email protected] Data and Code are available at: 10.6084/m9.figshare.12816086
Abstract
A sudden change in dynamics produces large errors leading to increases in muscle co-contraction and feedback gains during early adaptation. We previously proposed that internal model uncertainty drives these changes, whereby the sensorimotor system reacts to the change in dynamics by upregulating stiffness and feedback gains to reduce the effect of model errors. However, these feedback gain increases have also been suggested to represent part of the adaptation mechanism. Here, we investigate this by examining changes in visuomotor feedback gains during gradual or abrupt force field adaptation. Participants grasped a robotic manipulandum and reached while a curl force field was introduced gradually or abruptly. Abrupt introduction of dynamics elicited large initial increases in kinematic error, muscle co-contraction and visuomotor feedback gains, while gradual introduction showed little initial change in these measures despite evidence of adaptation. After adaptation had plateaued, there was a change in the co-contraction and visuomotor feedback gains relative to null field movements, but no differences (apart from the final muscle activation pattern) between the abrupt and gradual introduction of dynamics. This suggests that the initial increase in feedback gains is not part of the adaptation process, but instead an automatic reactive response to internal model uncertainty. In contrast, the final level of feedback gains is a predictive tuning of the feedback gains to the external dynamics as part of the internal model adaptation. Together, the reactive and predictive feedback gains explain the wide variety of previous experimental results of feedback changes during adaptation.
Introduction
Humans have exceptional abilities to skillfully manipulate and interact with objects in the environment. Our sensorimotor system constantly generates appropriate signals to control our musculoskeletal system, based on a prediction of the current dynamics. When these dynamics are stable and repeatable, we use an internal model or motor memory to produce an efficient and effective motion. When the external dynamics change, our sensorimotor control system adapts rapidly to the environmental disturbances in a manner that indicates a fundamental knowledge of the mechanics of the external world (Lackner & Dizio, 1994; Shadmehr & Mussa-Ivaldi, 1994; Conditt et al. , 1997). Any sudden change in the environmental dynamics during a movement causes large kinematic errors, leading to a rapid increase in muscle co-contraction (Thoroughman & Shadmehr, 1999; Osu et al. , 2002; Franklin et al. , 2003; Milner & Franklin, 2005). This co-contraction increases limb stiffness and acts to limit the perturbing effects of the dynamics until the sensorimotor control system is able to learn a motor memory or internal model that can predictively compensate for these dynamics. Once this motor memory is updated, this co-contraction is gradually decreased. Large kinematic errors which occur during the initial stages of adaptation to novel dynamics have also been shown to produce rapid increases in visuomotor (Franklin et al. , 2012) and long latency stretch (Coltman & Gribble, 2020) feedback responses. e previously proposed that this increase resulted from uncertainty in the internal model (Franklin et al. , 2012). During initial adaptation, the sensorimotor system receives unexpected error signals indicating that our current internal model no longer predicts the environment accurately, and that it either needs to update our current motor memory or select a new motor memory (Oh & Schweighofer, 2019). We suggest that along with co-contraction, the sensorimotor control system also upregulates feedback gains until the sensorimotor control system is able to relearn a new model that predictively compensates for these dynamics. These feedback gains are a reactive response to the internal model uncertainty to limit the perturbing effect of the novel dynamics during this initial phase of adaptation. However, even after adaptation to the new environment, these feedback gains are upregulated compared to the null field levels and tuned to the environmental dynamics (Franklin et al. , 2012, 2017; Cluff & Scott, 2013). That is, these feedback gains at the end of the adaptation process appear to arise as the sensorimotor control system regulates the gain of the feedback system as part of the adaptation process to novel dynamics, resulting in this final predictive component of feedback gains. We therefore suggested that there are two computational components of increased feedback gains; a reactive response to model uncertainty and a predictive response that is learned as part of the internal model (Franklin et al. , 2017). Here we examine whether the initial reactive component of the increased feedback gain is driven by internal model uncertainty or is simply learned as part of the adaptation process by contrasting the adaptation to abrupt changes in dynamics with the gradual introduction (Malfait & Ostry, 2004; Klassen et al. , 2005; Kluzik et al. , 2008; Huang & Shadmehr, 2009; Orban de Xivry et al. , 2011; Pekny et al. , 2011; Milner et al. , 2018). An abrupt introduction to a novel force field causes large error signals, whereas a gradual introduction to the same force field produces little or no error signals. Large errors would indicate that our current internal model no longer well predicts the environmental dynamics. In contrast, a gradual introduction of the force field provides only small errors within the natural variability of human reaching movements. As such, we predict that there will be little change in the uncertainty associated with the internal model, thereby causing little or no reactive increase in the feedback gains, despite adaptation continuing throughout the exposure phase. Instead, we predict only a gradual increase in the visuomotor feedback gains as the appropriate level of steady state feedback gains is learned for the dynamics. Here, we test these predictions by having participants adapt to both gradual and abrupt dynamics while measuring their visuomotor feedback gains to visual motion of the hand (Brenner & Smeets, 2003; Sarlegna et al. , 2003; Saunders & Knill, 2003; Franklin & Wolpert, 2008; Knill et al. , 2011).
Materials and Methods
Experimental Participants.
Twelve participants participated in the experiment (3 male and 9 female: aged 25.0 ± 4.4, mean ± SD). All participants were right-handed according to the Edinburgh handedness inventory (Oldfield, 1971) with no reported neurological disorders. Participants provided written informed consent, and the institutional ethics committee approved the experiments. One additional participant ook part in the experiment, but was excluded from the final analysis as the level of force field adaptation remained less than 40% at the end of the exposure period.
Apparatus.
Participants grasped the handle of the vBOT robotic manipulandum (Howard et al. , 2009) with their forearm supported against gravity with an air sled (Fig. 1A). The robotic manipulandum both generated the environmental dynamics (null field, force field or channel), and measured the participants’ behavior. Position and force data were sampled at 1KHz. Endpoint forces at the handle were measured using an ATI Nano 25 6-axis force-torque transducer (ATI Industrial Automation, NC, USA). The position of the vBOT handle was calculated from joint-position sensors (58SA; IED) on the motor axes. Visual feedback was provided using a computer monitor mounted above the vBOT and projected veridically to the participant via a mirror. This virtual reality system covers the manipulandum, arm and hand of the participant, preventing any visual information about their location. The exact time that the stimuli were presented visually to the participants was determined using the video card refresh rate and confirmed with an optical sensor to prevent a time delay. Participants performed right-handed forward reaching movements in the horizontal plane at approximately 10 cm below shoulder level.
Experimental Setup.
Participants were seated with their shoulders restrained against the back of a chair by a shoulder harness. Movements were made from a 1.0 cm diameter start circle centered approximately 28.0 cm in front of the participant to a 2.0 cm diameter target circle centered 25 cm in front of the start circle. The participant’s arm was hidden from view by the virtual reality visual system, on which the start and target circles as well as a 0.6 cm diameter cursor used to track instantaneous hand position were projected. Participants were instructed to perform successful movements to complete the experiment. A successful movement required the hand cursor to enter the target (without overshooting) within 700 ± 75 ms of movement initiation. Overshoot was defined as movements that exceeded the target in the direction of movement. When participants performed a successful movement they were provided with feedback as how close they were to the ideal movement time of 700ms (‘great’ or ‘good’) and a counter increased by one. Similarly, when they performed unsuccessful movements they were provided with feedback as to why the movement was not considered successful (“too fast”, “too slow” or “overshot target”). Trials were self-paced; participants initiated a trial by moving the hand cursor into the start circle and holding it within the target for 1000 ms. A beep then indicated that the participants could begin the movement to the target. The duration of the movement was determined from the time that the participants exited the starting position until the time that participants entered the target.
Electromyography.
Surface electromyography (EMG) was recorded from two mono-articular shoulder muscles (pectoralis major and posterior deltoid), two bi-articular muscles (biceps brachii and long head of the triceps) and two mono-articular elbow muscles (brachioradialis, and lateral head of the triceps). The EMG was recorded using the Delsys Bagnoli (DE-2.1 Single Differential Electrodes) electromyography system (Boston, MA). The electrode locations were chosen to maximize the signal from a particular muscle while avoiding cross-talk from other muscles. The skin was cleaned with alcohol and prepared by rubbing an abrasive gel into the skin. This was removed with a dry cotton pad and the gelled electrodes were secured to the skin sing double-sided tape. The EMG signals were analog band-pass filtered between 20 and 450 Hz (in the Delsys Bagnoli EMG system) and then sampled at 2.0 kHz.
Probe trials to measure reflex gain.
In order to assess reflex magnitude, visually induced motor responses were examined using perturbations of the visual system similar to those previously described (Franklin & Wolpert, 2008; Franklin et al. , 2012, 2014; Dimitriou et al. , 2013; Reichenbach et al. , 2014) throughout the experiments. On random trials, in the middle of a movement to the target, the cursor representing the hand position was jumped perpendicular to the direction of the movement (either to the left or to the right) by 2 cm for 250 ms and then returned to the true hand position for the rest of the movement (Fig. 1C). During these trials, the hand was physically constrained to the straight path between the starting position and the target using a mechanical channel, such that any force produced in response to the visual perturbation could be measured against the channel wall using the force sensor. The mechanical wall of the channel was implemented as a stiffness of 5,000 N/m and damping of 2 N∙m -1 ∙s for any movement lateral to the straight line joining the starting location and the middle of the target (Scheidt et al. , 2000; Milner & Franklin, 2005). As this visual perturbation was transitory, returning to the actual hand trajectory, participants were not required to respond to this visual perturbation to produce a successful trial. These visual perturbations were applied perpendicular to the direction of the movement (either to the left or the right). For comparison, a zero-perturbation trial was also included in which the hand was held to a straight-line trajectory to the target, but the visual cursor remained at the hand position throughout the trial. The onset of the displacements occurred starting at 7.5 cm (30% of the length of the movement). The perturbation trials were randomly applied during movements in a blocked fashion such that one of each of the three perturbations were applied within a block of twelve trials. Experimental Paradigm.
Our previous experiment suggested that the large feedback gains during initial adaptation were due to the uncertainty in the internal model (Franklin et al. , 2012), however this could not be dissociated from motor adaptation. Here we examine this phenomenon in detail, by contrasting the adaptation to abrupt changes in dynamics with the gradual introduction (Kluzik et al. , 2008; Orban de Xivry et al. , 2011). In this experiment, we examine the changes in the feedback gains while the reaching errors are presented both physically and visually to participants. While participants could not see their hands, a visual cursor (yellow circle) was presented which corresponded to their hand location. With the exception of 250ms on the probe trials in which this cursor was perturbed laterally by 2cm, the cursor always matched the physical location of their hand in both x- and y-axes (Fig. 1D).
Figure 1.
Experimental set-up. A: Participants grasp the robotic manipulandum (vBOT) while seated and visual feedback is presented veridically using a top mounted monitor viewed through a mirror so that it appears in the plane of movement. The participant’s forearm is supported by an airsled. B: Participants experienced either an abrupt (green) or gradual (blue) onset of a velocity dependent force field in different sessions. C: Throughout the experiment, on random trials visual perturbations (probe trials) were used to examine the magnitude of the visually induced motor response. During these trials, the hand (dark grey line) was physically constrained to the straight path between the initial starting position and the final target using a mechanical channel, such that any force produced in response to the visual perturbation (blue or red dotted lines) can be measured against the virtual channel wall (grey arrows) using the force sensor. D: In all trials except the probe trials, the hand cursor (orange line) is shown to the participant overlaid on the actual hand movement (black dotted line).
Experimental protocol.
Each participant performed two sessions, which were separated by a short break, in a single day. On one session, a dynamical force field was abruptly introduced (abrupt), while on the other session a directionally opposite force field was gradually applied (gradual). The order of both sessions and the force field directions were counterbalanced across participants. EMG electrodes remained in place throughout the entire experiment. Before each session, participants performed a practice of 61 null field trials in order to familiarize themselves with the movement criteria. Throughout the experiment, trials were arranged in blocks consisting of twelve trials; of which three were probe trials (visual perturbation and mechanical channel) and nine were normal reaching trials (null field or force field depending on the phase of the experiment). These probe trials were used to assess both the visuomotor feedback gain and the degree of learned force compensation. While lateral movement in the random probe trials was constrained by the mechanical channel, participants were free to move in any direction during all other trials. Each session consisted of 4 phases. A single movement was always performed first in any new phase such that a probe trial was never the first movement. First, participants experienced the pre-exposure phase of 121 null field trials (10 blocks of twelve trials plus one initial trial). Next, the initial exposure stage consisted of 361 force field trials (30 blocks plus one initial trial). In the abrupt condition, the full force field was applied from the first trial, whereas in the gradual condition the force field C Probe Trials -5 0 5
Position X-axis [cm] P o s i t i on Y - a x i s [ c m ] A B
Gradual F o r c e F i e l d Abrupt F o r c e F i e l d Force Field Conditions D -5 05 Position X-axis [cm] P o s i t i on Y - a x i s [ c m ] vBotmonitorairsled tablemirror Experimental Set-up hand positionvisual feedback as scaled up from one trial to the next over the 361 trials (Fig. 1B). The force field was a velocity-dependent curl force field where the force in N ( F x , F y ) on the hand was computed as: depending on the participants' hand velocity ( ) [m/s] and the scaling factor b which was either 0.16 (clockwise curl force field: CW) or -0.16 (counter-clockwise curl force field: CCW). Once the initial exposure phase was completed, both groups of participants performed the final exposure phase consisting of another 20 blocks of trials (241 trials). Finally, participants experienced the post-exposure phase in which 10 blocks (121 trials) of null field trials were performed. Participants were required to take short breaks every 200 movements throughout the experiment. They were also allowed to rest at any point they wished by releasing a safety switch on the handle. Analysis.
Analysis of the experimental data was performed using Matlab R2019a. EMG data were band-pass filtered (30 – 500Hz) with a fifth-order, zero phase-lag Butterworth filter and then rectified. Position, velocity and endpoint force were low-pass filtered at 40 Hz with a fifth-order, zero phase-lag Butterworth filter. Statistics were performed in Matlab and JASP 0.13 Statistical significance was considered at the p <0.05 level for all statistical tests. Hand path error.
The maximum perpendicular error (MPE) was used as a measure of the straightness of the hand trajectory. On each trial, the MPE is the maximum distance on the actual trajectory that the hand reaches perpendicular to the straight-line path joining the start and end targets (errors to the left are defined as negative and errors to the right are defined as positive). The MPE was calculated for each non-probe trial throughout the learning experiment.
Force compensation.
In order to examine the predictive forces exerted by the participants throughout the experiment, the forces against the channel walls on the probe trials were used. On each trial, the amount of force field compensation was calculated by linear regression of the measured lateral force against the channel wall onto the ideal force profile required for full force field compensation (Smith et al. , 2006). The ideal force field compensation was estimated as the product of the y-velocity and the force field scaling factor. In the null field, the ideal force field compensation is based upon the compensation required in the curl force field (Howard et al. , 2012). Therefore, values in the null force field before learning (pre-exposure phase) should be close to zero. In the gradual curl force field, the force compensation on a given trial was calculated based on the current strength of the force field at this specific trial.
Electromyographic activity.
For plotting purposes, the EMG was adjusted to the mean value of EMG in the null field trials prior to the force field exposure for each muscle of each participant prior to averaging. F x F y ⎡⎣⎢⎢ ⎤⎦⎥⎥ = b −
11 0 ⎡⎣⎢ ⎤⎦⎥ ! x ! y ⎡⎣⎢⎢ ⎤⎦⎥⎥ ! x , ! y o examine differences in the overall muscle activity across the experiment, the integral of the rectified EMG data was taken over 850 ms from 50 ms before movement start until 800 ms after movement start. The EMG data was averaged across all trials in a block. For display purposes the EMG was scaled and averaged across participants (but not across muscles). To do this for a particular muscle, a single scalar was calculated for each participant and used to scale the muscle’s EMG traces for all trials for that participant. The scalar was chosen so that the mean (across trials) of the EMG data averaged over the whole experiment was equal across participants (and set to the mean over all the participants). This puts each participant on an equal scale to influence any response seen in the data. For comparison across muscles, the EMG values were further scaled for each muscle relative to the mean value in the pre-exposure phase (expressed as a percentage value relative to the pre-exposure activity). The muscle activity was further separated into the amount related to co-contraction and the amount related to force production for each of the three muscle pairs. In order to examine the amount of co-contraction, the minimum value of EMG between the two muscles making up a muscle pair was determined and multiplied by 2 as both muscles would contribute to the increased stiffness. The activation that would correspond to the change in force was determined as the maximum muscle activity of the two muscles of the muscle pair subtracted by the minimum of the two muscle activities. Differences in these measures across the gradual and abrupt conditions were examined using a t-test in Matlab. Rapid visuomotor responses.
Individual probe trials were aligned on visual perturbation onset. The response to the right visual perturbation on probe trials was subtracted from the response to the left perturbation on probe trials in order to provide a single estimate of the motor response to the visual perturbation for each block. To examine the feedback gain, we calculated the average post-perturbation force over two intervals: the first corresponding to an rapid involuntary response (180–230 ms) (Franklin & Wolpert, 2008), and the second to a slower response (230-300 ms) (Franklin et al. , 2014, 2017). To examine the electromyographic responses to the visuomotor perturbations, the EMG traces were divided by the mean value of the muscle activity in the null force field (pre-exposure) for that muscle in that participant between -50 and 50 ms relative to the onset of the perturbation. Muscular responses were considered over two intervals: 90-120 ms and 120-180 ms as in previous studies (Dimitriou et al. , 2013; Franklin et al. , 2014; Gu et al. , 2016; Cross et al. , 2019).
Comparisons.
In order to compare the final values after learning, the mean measures (MPE, force compensation and visuomotor feedback gain) over the last 15 blocks in the final exposure phase were contrasted between abrupt and gradual conditions using frequentist repeated measures ANOVAs with a between subjects factor of condition order using JASP 0.13. esults
Participants performed forward reaching movements while grasping the handle of a robotic manipulandum (Fig. 1A). Participants were then presented with either an abrupt or gradual introduction of a velocity-dependent force field (Fig. 1B). Throughout the entire experiment we measured the feedback gains on random trials by presenting the participants with a brief visual perturbation of a hand cursor (to the right or left of the movement) while the physical hand was mechanically constrained to move within a channel to the target, termed probe trials, (Fig. 1C). On these probe trials we measured the lateral force produced by the participant’s hand into the channel providing information both about the learned predictive forces for compensating the force field and the magnitude of the rapid visuomotor feedback gains. Throughout the experiment, participants received visual feedback about their movement and the target via a computer screen in the plane of movement (Fig. 1D).
Behaviour.
All participants adapted to both an abrupt and a gradual introduction of a curl force field where both the order and the field direction were counterbalanced across participants. After a short pre-exposure phase where the initial trials were in the null field, a curl force field was applied unexpectedly. In the abrupt condition this caused a large increase in the kinematic error (Fig. 2A left, green). However, in the gradual condition, as the strength of the curl force field is gradually increased, the kinematic error (MPE) only slowly increased until the strength of the force field got close to the final level (Fig. 2A right, blue). Once the force field reached the final level, the MPE appeared to gradually reduce over the next twenty blocks of trials. Interestingly, after the experiment had finished, participants generally reported that they did not realize that a force field had been applied, instead only that something different was being done to their arm during these movements once they were halfway through the gradual loading of the force field. Comparing with a repeated measures ANOVA the last 15 blocks in the final exposure phase (dark gray shaded area), the MPE in the abrupt condition was not significantly different from that in the gradual condition (F =0.598; p=0.223) (Fig. 2A center bar graph) and there was no between subjects effect of condition order (F =0.984; p=0.345). For both groups, when the force field was removed (post-exposure phase) the maximum perpendicular error increased dramatically in the opposite direction to that of the force field (Fig. 2A, left and right), but was reduced quickly over the subsequent blocks.
Figure 2.
Comparison of adaptation between abrupt and gradual exposure to the curl force fields. A: The mean (solid line) and standard error of the mean (shaded region) of the maximum perpendicular error (MPE) of the hand trajectory over the experiment. The sign of the MPE measurement from the CCW force fields was flipped so that all errors produced by the force field were shown to be positive. The gray shaded area (and darker green and blue colors) indicate the period over which the curl force field was applied. The vertical dotted line shows the time point at which the gradual force field was the same strength as the abrupt force field. Dark gray shaded area indicates the last 15 blocks of the full force field exposure phase over which the final levels of adaptation were compared. The center bar graph compares this final level of MPE across the two force fields. The error bar represents the standard error of the mean (s.e.m.). Each participant’s final value is shown with a point. A light grey point indicates the CW force field whereas a dark grey point indicates the CCW force field. The square indicates the first force field experienced whereas the circle indicates the second time participants experienced a force field. There were no significant differences between the conditions. B: Force compensation level over the experiment as measured on the channel trials. A value of 100% indicates perfect compensation for the force field. Force compensation in the null field was quantified with respect to the full force field value, so a value of 0 is expected. In the gradual condition, as the force field is ramped up, the force compensation is expressed as a percentage of the force field that has been applied to the participant at this point in the experiment. Values plotted as in A. Similar levels of force compensation were found for abrupt and gradual conditions with no significant difference over the last 15 blocks of exposure. Throughout the entire experiment, random trials were introduced in which participants performed movements in a mechanical channel constraining their hand
Block -4-3-2-101234 M a x i m u m P e r p e n d i c u l a r E rr o r [ c m ] Block -4-3-2-101234 M a x i m u m P e r p e n d i c u l a r E rr o r [ c m ] -3-2-10123 M a x i m u m P e r p e n d i c u l a r E rr o r [ c m ] Abrupt Gradual
Block Fo r ce C o m p e n s a t i o n [ % ] Block Fo r ce C o m p e n s a t i o n [ % ] Fo r ce C o m p e n s a t i o n [ % ] Abrupt Gradual
Abrupt Condition Final Exposure Level Gradual Condition AB o a straight movement to the target. Using these trials, we can estimate the predictive force compensation as participants adapt to the dynamics (Fig. 2B) indicating the percentage of perfect adaptation to the force field. Note that in the gradual condition, the percentage of adaptation is expressed as a function of the current level of force field during the ramp phase. Interestingly, the force compensation in the gradual force field adaptation appears to start from 60-70% perfect adaptation from the beginning and stay around this level throughout both the initial ramping phase and the later constant phase. In both the abrupt and gradual conditions, participants showed high levels of force compensation (over 70%) after they reached to 100% strength of the force field (blocks 40-60). When comparing the final levels of adaptation in the abrupt and gradual conditions (final 15 blocks of exposure phase), we again found no significant difference (dark gray shaded area in the figures) in the force compensation (Fig. 2B center) between the abrupt and the gradual conditions (F =0.962; p=0.350) and no between subjects effect of condition order (F =1.280; p=0.284). Therefore, the final adaptation level of participants in the abrupt and gradual conditions were similar, although the abrupt condition had a much greater number of trials in which they were presented with the full magnitude of the force field. Muscle Activity.
Muscle activity (sEMG) was recorded from six muscles and is shown separately for the counter-clockwise (Fig. 3A) and clockwise (Fig. 3B) force fields. The EMG values are normalized to the null field level for all muscles, and are only shown here on the mechanical channel trials in which the force field is not applied and no trajectory errors occur. In the abrupt condition (green), all muscles increase their activity when the force field was first applied, suggesting an initial large increase in co-contraction. However, the muscles slowly reduced their activity as learning proceeded, eventually reaching a plateau level late in adaptation. At the end of the exposure phase there are high activity levels in the muscles compensating for the force field: posterior deltoid and triceps longus for the CCW force field and pectoralis major and biceps brachii for the CW force field. In the gradual condition (blue), the muscle activity remained low throughout the early exposure period, only gradually increasing and reaching a plateau towards the end of the exposure phase. The final levels of muscle activity were similar between both the abrupt and gradual conditions for both force fields. The similar levels of final muscle activity were true for both the muscles compensating for the force field, and those acting as stabilizers – increasing the co-contraction. When the force field was suddenly removed (after block 60), there were further increases of muscle activity, especially in the antagonist muscles – the muscles that were not acting to compensate for the force fields (e.g. posterior deltoid and triceps longus in the CW force field for a forward movement). As the muscle activity on this (and subsequent figures) is obtained from the channel trials, we would expect that most of the activity is pre-planned and not a reaction to kinematic errors experienced on the specific trial.
Figure 3.
Comparison of muscle activity (sEMG) across the experiment for abrupt (green) and gradual (blue) exposure to curl force fields. Light blue and green indicate values in the null field whereas dark blue and green indicate values in the curl force field. Data is only from the randomly interspersed mechanical channel trials where the curl force field was not applied. A: Muscle activity (mean and s.e.m) during adaptation to the CCW curl force field. Muscle activity was normalized for each of the six muscles to the mean value in the pre-exposure phase before averaging across participants. EMG values were calculated as the integrated muscle activity from -100 to 600 ms relative to the start of the movement. Grey shaded region indicates the exposure phase and the vertical dotted line indicates the point at which the gradual force field is equal to the full force field value. B: EMG in the CW curl force field.
Across both force fields, high levels of muscle activation were initially observed when the force field was introduced abruptly (green) but not when introduced gradually (blue). The temporal profile of muscle activity after adaptation was examined for both the CCW and CW force fields (Fig. 4) on the channel trials, with the profile of activity on the null field trials shown for comparison. As expected the profile of muscle activity in the null field prior to either the abrupt or gradual force field application is similar. After adaptation, we also find similar temporal profiles of muscle activation for many of the muscles, but with specific differences between the conditions. In particular, in the CCW force field (Fig. 4A), there was a larger activation of the posterior deltoid muscle in the abrupt condition, whereas there was a larger triceps longus activation in the gradual condition. After adaptation to the CW force field (Fig. 4B) it appears that there is a larger pectoralis major and triceps longus muscle activation after the
Block
Block
Block
Block
BrachioradialisTriceps Lateralus
CW Force Field m u sc l ea c t i v i t y [ % nu ll a c t i v i t y ] B Block
Block
CCW Force Field m u sc l ea c t i v i t y [ % nu ll a c t i v i t y ] A brupt introduction of the force field, whereas the gradual condition shows a slightly larger biceps brachii activation. Therefore, across both force fields, the abrupt introduction of the force field produced slightly higher activation of the shoulder muscles, whereas gradual adaptation often recruited the biarticular muscles to a larger degree. It is therefore possible that the specific pattern of large directional errors in the abrupt conditions changes the overall recruitment pattern of muscles even after adaptation. Figure 4.
Temporal profiles of muscle activity in the null force field and after adaptation for the abrupt (green) and gradual (blue) conditions. Data is only from the mechanical channel trials where the curl force field was not applied. A: EMG profiles in the CCW curl force field. Null field activity (all 10 blocks in the pre-exposure phase) prior to adaptation is indicated by the light green and light blue traces. Final adaptation activity (all 20 blocks in the final exposure phase) is indicated by the dark green and dark blue traces. Muscle activity has been aligned to the start of the movement (0 s). Solid lines indicate mean across participants and shaded regions indicate s.e.m. B: EMG profiles in the CW curl force field. Similar effects can be seen when we quantify the increases in sEMG for both initial and final exposure (Fig. 5). As expected, the abrupt condition produced large increases in muscle activation for all six muscles (and therefore increased co-contraction) during the initial exposure (Fig. 5A). In contrast, only a small increase in muscle activity was seen initially in the gradual condition. However, during the final exposure phase we see a similar level of muscle activity in both abrupt and gradual conditions (Fig. 5B). Here again the difference between the abrupt and gradual
Time [s] Time [s] Time [s]Time [s] CW Force Field m u sc l ea c t i v i t y [ % nu ll a c t i v i t y ] CCW Force Field B Pectoralis MajorPosterior Deltoid Biceps BrachiiTriceps Longus BrachioradialisTriceps LateralusBiceps BrachiiTriceps Longus
Time [s] m u sc l ea c t i v i t y [ % nu ll a c t i v i t y ] A Pectoralis MajorPosterior Deltoid
Time [s] daptation is clear in our experiment. Abrupt presentation of the force field appears to recruit higher activation in the shoulder muscles whereas gradual adaptation recruited higher levels of biarticular muscle activity. As error bars reflect the 95% confidence intervals, some of these differences are significant. In order to quantify the degree of co-contraction and adaptation in the experiments across the gradual and abrupt conditions, we calculated the co-contraction and adaptation indices (Fig. 5C and 5D). The co-contraction index is a simple measure to capture the relative amount of activation in antagonistic muscle pairs, whereas the adaptation index is designed to indicate the amount of muscle activity in a specific direction (reciprocal activation) that might be directed to compensate for the force field. In the initial exposure phase (Fig. 5C), the co-contraction index was much higher in the abrupt condition than in the gradual condition (t =5.3314; p=0.0002). The adaptation index was also higher in the abrupt condition (t =4.1241; p=0.0017), but this difference was much smaller (approximately twice the level). However, in the final exposure phase (Fig. 5D), there were no significant differences in either the co-contraction (t =0.4797; p=0.6409) or adaptation (t =0.9499; p=0.3625) measures. The difference in the shoulder and biarticular muscle activity at the end of adaptation can again be seen in the adaptation index measure (Fig. 5D, right), but does not show up in the co-contraction index. Despite these differences, the overall muscle activity levels were similar across both presentations of the dynamics. Figure 5.
Muscle activity in initial and final exposure to abrupt (green) and gradual (blue) application of force fields. Data is only from the mechanical channel trials initial exposure
CCW Force Field flexors s h o u l d e r b i a r t i c u l a r C h a n g e i n m u s c l eac t i v i t y [ % o f nu ll fi e l d ] e l b o w s h o u l d e r b i a r t i c u l a r e l b o w extensors extensorsflexors CW Force Field A final exposure CCW Force Field flexors s h o u l d e r b i a r t i c u l a r C h a n g e i n m u s c l eac t i v i t y [ % o f nu ll fi e l d ] e l b o w s h o u l d e r b i a r t i c u l a r e l b o w extensors extensorsflexors CW Force Field BC GradualAbrupt (p=0.0002) initial exposure C o - c on t r a c t i on I nde x [ % ] (p=0.0017) GradualAbrupt A dap t a t i on I nde x [ % ] shoulderbiarticularelbow D GradualAbrupt (p=0.64) final exposure C o - c on t r a c t i on I nde x [ % ] (p=0.36) GradualAbrupt A dap t a t i on I nde x [ % ] shoulderbiarticularelbow here the curl force field was not applied. A: Initial exposure (first 10 blocks in initial exposure phase) to abrupt force field elicits large co-contraction in shoulder, biarticular and elbow muscles for both CCW and CW curl fields. Muscle activity is calculated as increase relative to the null field activity. Bars indicate mean (± 95% confidence intervals) integrated muscle activity from -100 to 600 ms from the start of the movement. B: Final exposure (last 10 blocks in final exposure phase) shows similar levels of change in muscle activity for both abrupt and gradual change in dynamics. C: The co-contraction index and adaptation index of muscle activity in the initial exposure. Bar indicates total values across the muscle pairs (± s.e.m.) and the colors indicate the relative contribution from the shoulder muscles (dark colors), biarticular muscles (medium colors) and elbow muscles (light colors). Values are across both the CCW and CW curl fields. Statistics indicate result of t-test. D: The co-contraction index and adaptation index of muscle activity in the final exposure periods. Visuomotor Feedback Responses.
Throughout the abrupt and gradual experiments, the visuomotor feedback responses were measured using probe trials in which the hand was constrained to a mechanical channel, but a visual perturbation of the cursor position was applied. The visuomotor feedback response was quantified over two intervals: an early interval between 180 and 230 ms (Fig. 6A) and a later interval between 230 and 300 ms (Fig. 6B). In both intervals the onset of the abrupt change in dynamics produced a rapid increase in the visuomotor gain (green traces) which then remained fairly high over the rest of the exposure phase, with a possible slight decrease over learning as seen in previous work (Franklin et al. , 2012). In contrast, when the curl force field was applied gradually, there was little to no initial increase in the visuomotor gains, which instead gradually increased over the whole exposure period and then plateaued as the full level of force field was applied in the final exposure trials. Despite the very different patterns of visuomotor gains during the learning phase, the visuomotor gains in the last 15 blocks of the exposure phase were not significantly different in either the early (F =1.697, p=0.122; Fig. 6A bar plot) or late intervals (F =0.542, p=0.334; Fig. 6B bar plot). For both cases, there was also no effect of condition order (early: F =0.409, p=0.576; late: F =0.531, p=0.519). Therefore, both conditions produced the same final level of visuomotor gains regardless of the large difference in kinematic errors during the adaptation. When the force field was removed abruptly in both conditions, we see an initially high visuomotor gain that decreased rapidly in these null field trials.
Figure 6 . Changes in visuomotor feedback gain during adaptation to abrupt (green) and gradual (blue) curl force fields. A: Visuomotor feedback gains during 180-230ms after the visual perturbation onset. Figure plotted as in Fig. 2. B: Visuomotor feedback gains during 230-300ms after the visual perturbation onset. Visuomotor feedback gains are measured on channel trials where the curl force fields were not applied. In order to contrast the visuomotor feedback gains at the end of the exposure period we plotted the lateral hand force as a function of the time from perturbation onset (Fig. 7) in these probe trials. The lateral force is the force produced by the participant against the wall of the virtual channel. The force produced after the abrupt introduction of the force field (Fig. 7A) looks similar to that after the gradual introduction of the force field (Fig. 7B). When we subtract the zero-perturbation condition, we can see that the visuomotor response is similar in both conditions, not only in the early and late intervals but across the whole response (Fig. 7C). Participants in these experiments adapted to both the CCW and CW force fields, so we also examined the force response in each of these force fields separately (Fig. 7D-I). We find similar lateral forces against the channel wall for both the abrupt and gradual conditions in the CCW force field, but when we subtract the zero-perturbation condition it looks as though the forces are slightly larger in the gradual condition (Fig. 7F). In the CW force field, the lateral forces are in the opposite directions, but here the comparison shows a similar response in both abrupt and gradual conditions (Fig. 7I). One interesting point is that the CCW and CW force fields require opposite adaptive forces which can be clearly seen in the force traces (e.g. compare Fig. 7 D and G). In each force field a perturbation in one direction would be resisted by highly active muscles whereas in the other direction these muscles would have much lower activation (e.g. Fig. 5). This would then be reversed A Block V i s u o m o t o r G a i n230 - m s [ N ] Block V i s u o m o t o r G a i n230 - m s [ N ] V i s u o m o t o r G a i n M a g n i t u d e [ N ] Abrupt Gradual B Block V i s u o m o t o r G a i n180 - m s [ N ]
40 60 0 20 40 60
Block V i s u o m o t o r G a i n180 - m s [ N ] V i s u o m o t o r G a i n M a g n i t u d e [ N ] Abrupt Gradual
Abrupt Condition Final Exposure Level Gradual Condition n the opposite force field. However, despite these differences the visuomotor force response is roughly equal in both perturbation directions. This further supports our previous claim that visuomotor feedback responses do not exhibit gain scaling (Franklin et al. , 2012, 2017), at least at the level of force responses.
Figure 7.
Comparison of final visuomotor feedback responses after adaptation to the abrupt (green) and gradual (blue) curl force fields. A: Mean (± s.e.m.) lateral force produced after rightward, zero and leftward visual perturbations across both CCW and CW fields in the abrupt condition. Lateral force was adjusted by subtracting the mean lateral force across all channel trials in each force field. B: Lateral force in response to visual perturbations across both CCW and CW fields in the gradual condition. C: Visuomotor responses (zero perturbation subtracted) across both CCW and CW fields. Light grey shaded region indicates the early visuomotor response interval (180-230 ms) while the dark grey region indicates the late visuomotor response interval (230-300 ms). D: Lateral force produced in response to visual A d j u s t e d L a t e r a l Fo r ce [ N ] V i s u o m o t o r R e s p o n s e [ N ] L a t e r a l Fo r ce [ N ] V i s u o m o t o r R e s p o n s e [ N ] L a t e r a l Fo r ce [ N ] V i s u o m o t o r R e s p o n s e [ N ] Time from perturbation onset [ms] A d j u s t e d L a t e r a l Fo r ce [ N ] Time from perturbation onset [ms] L a t e r a l Fo r ce [ N ] Time from perturbation onset [ms] L a t e r a l Fo r ce [ N ] AD BE CFG H I
ComparisonAbrupt GradualAbrupt Gradual ComparisonAbrupt Gradual Comparison
CCW Force FieldMean across both Force FieldsCW Force Field erturbations after adaptation to the abrupt onset of the CCW force field. E: Lateral force produced in response to visual perturbations after adaptation to the gradual onset of the CCW force field. F: Visuomotor responses in the CCW force field after abrupt and gradual adaptation. G-I: Visuomotor force responses after adaptation to the CW force field. Finally, we examined the muscle responses to the visual perturbation, particularly those in the pectoralis major (Fig. 8) and posterior deltoid (Fig. 9); the major muscles to correct lateral perturbations in this movement. As the background load and muscle activity are different across force fields, the muscle responses to the visual perturbation is shown separately for the CCW and CW fields for both the abrupt and gradual conditions (Fig. 8A-D). Visual perturbations produce clear excitation or inhibition of the muscular activity. If we average across the two force fields we see that there are similar responses in the pectoralis major in both the abrupt and gradual conditions (Fig. 8E), with no differences across either the early or late visuomotor response windows (error bars represent 95% confidence intervals). If instead, we average across the abrupt and gradual conditions, we can directly compare the muscular responses in the CCW versus the CW force fields (Fig. 8E). Here again there were no differences across the temporal response or within the early or late visuomotor response windows. That is, despite differences in the background loads of the muscles, the visuomotor response at the end of the learning the force fields were similar for both CCW and CW force fields. Similar responses were observed in the posterior deltoid (Fig. 9). When averaged across the force fields, we found no differences in the muscular responses between the abrupt and gradual conditions (Fig. 9E). However, when averaging across abrupt and gradual conditions, we found apparent differences in the muscular responses between the CCW and CW force fields, although these differences were not statistically significant (Fig. 9F).
Figure 8.
Visuomotor feedback responses in the pectoralis major muscle after force field adaptation. A: Pectoralis major activity to leftward, zero, and rightward visual perturbations after abrupt adaptation to the CCW force field. Activity is scaled according to the level of muscle activity in the null field (mean between -50 and +50 ms prior to the perturbation time) which is represented by the dotted black line. Shaded region indicates the s.e.m. B: Muscle activity after gradual adaptation to the CCW force field. C: Muscle activity after abrupt adaptation to the CW force field. D: Muscle activity after gradual adaptation to the CW force field. E: Visuomotor responses (perturbation – zero perturbation) averaged across the CCW and CW force fields for the abrupt (green) and gradual (blue) conditions. Rightward perturbations produce an excitatory response whereas leftward perturbations inhibit the muscle activity. Light grey and dark grey bars indicate the early (90-120 ms after perturbation onset) and late (120-180 ms) visuomotor response time windows. Bar plot quantify the responses over the early and late windows. Error bars indicate 95% confidence intervals. F: Visuomotor responses averaged across abrupt and gradual conditions to examine differences between the CCW (pink) and CW (brown) force fields. P ec t o r a li s M a j o r [ % N F ] P ec t o r a li s M a j o r [ % N F ] Time from perturbation onset [ms]
Time from perturbation onset [ms] ea r l y ( - m s ) l a t e ( - m s ) -200-1000100200 -200-1000100200 Abrupt Abrupt versus Gradual
Time from perturbation onset [ms] ea r l y ( - m s ) l a t e ( - m s ) -200-1000100200 -200-1000100200 CCW versus CWGradual CC W Fo r ce F i e l d P ec t o r a li s M a j o r [ % N F ] P ec t o r a li s M a j o r [ % N F ] Time from perturbation onset [ms] C W Fo r ce F i e l d B FA DC E P ec t o r a li s M a j o r [ % N F ] P ec t o r a li s M a j o r [ % N F ] Time from perturbation onset [ms]
Time from perturbation onset [ms] ea r l y ( - m s ) l a t e ( - m s ) -200-1000100200 -200-1000100200 Abrupt Abrupt versus Gradual
Time from perturbation onset [ms] ea r l y ( - m s ) l a t e ( - m s ) -200-1000100200 C h a n g e i n m u s c l eac t i v i t y [ % N F ] C h a n g e i n m u s c l eac t i v i t y [ % N F ] -200-1000100200 CCW versus CWGradual CC W Fo r ce F i e l d P ec t o r a li s M a j o r [ % N F ] P ec t o r a li s M a j o r [ % N F ] Time from perturbation onset [ms] C W Fo r ce F i e l d B FA DC E
Figure 9.
Visuomotor feedback responses in the posterior deltoid muscle after force field adaptation. Responses plotted as in Fig. 8. A: Posterior deltoid activity to leftward, zero, and rightward visual perturbations after abrupt adaptation to the CCW force field. B: Gradual adaptation to the CCW force field. C: Abrupt adaptation to the CW force field. D: Gradual adaptation to the CW force field. E: Visuomotor responses (perturbation – zero perturbation) averaged across the CCW and CW force fields for the abrupt (green) and gradual (blue) conditions. Bar plot shows responses over the early and late windows, with error bars indicating 95% confidence intervals. F: Visuomotor responses averaged across abrupt and gradual conditions to examine differences between the CCW (pink) and CW (brown) force fields.
Discussion
The goal of this study was to examine whether internal model uncertainty drives changes in the feedback gains during adaptation. In order to do this, participants
Time from perturbation onset [ms] -5000500 -300-200-1000100200300 ea r l y ( - m s ) l a t e ( - m s ) Time from perturbation onset [ms] -5000500 -300-200-1000100200300 ea r l y ( - m s ) l a t e ( - m s ) Abrupt versus Gradual E P o s t e r i o r D e l t o i d [ % N F ] P o s t e r i o r D e l t o i d [ % N F ] Time from perturbation onset [ms] P o s t e r i o r D e l t o i d [ % N F ] P o s t e r i o r D e l t o i d [ % N F ] Time from perturbation onset [ms]
Abrupt Gradual CC W Fo r ce F i e l d C W Fo r ce F i e l d BA DC
CCW versus CW F Time from perturbation onset [ms] -5000500 -300-200-1000100200300 ea r l y ( - m s ) l a t e ( - m s ) Time from perturbation onset [ms] -5000500 -300-200-1000100200300 ea r l y ( - m s ) l a t e ( - m s ) Abrupt versus Gradual E P o s t e r i o r D e l t o i d [ % N F ] P o s t e r i o r D e l t o i d [ % N F ] Time from perturbation onset [ms] P o s t e r i o r D e l t o i d [ % N F ] P o s t e r i o r D e l t o i d [ % N F ] Time from perturbation onset [ms]
Abrupt Gradual CC W Fo r ce F i e l d C W Fo r ce F i e l d BA DC
CCW versus CW F C h a n g e i n m u s c l eac t i v i t y [ % N F ] C h a n g e i n m u s c l eac t i v i t y [ % N F ] erformed reaching movements where the environmental dynamics were either changed abruptly (producing large kinematic errors) or gradually (producing small kinematic errors). Abrupt changes in dynamics produced large kinematic errors during the movements signaling that the internal model was incorrect – which should increase internal model uncertainty. In contrast, a gradual change in the dynamics produced only small kinematic errors and should produce much lower uncertainty in the internal model. In the initial exposure to the force fields, participants in the abrupt condition experienced large kinematic errors, extensive muscle co-contraction, increased visuomotor feedback gains, and a rapid increase in force compensation. In contrast, participants in the gradual condition experienced little to no kinematic errors, very little co-contraction and only small increases in the visuomotor feedback gains. Despite this, the force compensation was maintained at around 70% of the applied force field throughout the initial exposure period, and was associated with the respective changes in the muscle adaptation index. Although there were large differences in the initial exposure phase, participants in both abrupt and gradual conditions reached similar levels of kinematic error, force compensation, co-contraction, muscle activity and visuomotor feedback gains by the end of the exposure phase. Many studies have compared gradual adaptation with abrupt adaptation to force fields (Malfait & Ostry, 2004; Klassen et al. , 2005; Kluzik et al. , 2008; Huang & Shadmehr, 2009; Pekny et al. , 2011; Milner et al. , 2018; Alhussein et al. , 2019). It has been shown that the final level of adaptation is similar regardless of whether the novel dynamics are presented abruptly or gradually (Malfait & Ostry, 2004; Klassen et al. , 2005; Milner et al. , 2018; Alhussein et al. , 2019). Here we also found no difference in the final level of kinematic error (MPE) or force compensation, agreeing with these previous studies. It has been suggested that gradual presentation of novel dynamics drives changes in the internal model of the limbs dynamics rather than in the internal model of the tool (or robot) (Kluzik et al. , 2008), which could explain why the motor memory formed with gradual adaptation does not transfer bimanually to the other limb (Malfait & Ostry, 2004). Although it has been suggested that gradual presentation produces better retention (Huang & Shadmehr, 2009), more recent studies have shown that the retention rates are similar, and that the biggest effect on retention is simply the amount of training (Alhussein et al. , 2019). Nevertheless, as larger errors are more likely to induce adaptation of a new motor memory (Oh & Schweighofer, 2019) or adapt a specific tool related motor memory (Kluzik et al. , 2008), we might expect that there are subtle differences in the properties of the motor memories that are formed under these two different situations, such as the fact that gradually adapted motor memories do not transfer across limbs (Malfait & Ostry, 2004). Although earlier studies suggested that abrupt presentation of novel dynamics is learned faster (Huang & Shadmehr, 2009), more recently it was claimed that the learning rate in abrupt and gradual dynamics is similar (Milner et al. , 2018). Milner and colleagues found that while amplitude error decreased faster under abrupt conditions, the temporal error or smoothness decreased faster under gradual conditions. Here we did not compare the rate of adaptation under the two conditions, however the force compensation calculated relative to the presented dynamics was consistently 70% of the applied force field throughout both the gradual increase of the force field and the steady state phase. Under the abrupt change in dynamics, the orce compensation increased up to a similar level. Therefore, it appears that under both conditions, the adaptation mechanism is able to incorporate a similar level of the error to modifying the predictive force compensation. Previous work has shown that the adaptation system is more sensitive to small errors than larger errors (Fine & Thoroughman, 2006; Wei & Körding, 2009; Marko et al. , 2012; Hayashi et al. , 2020), which might suggest that gradual dynamics would be learned faster. Here we have shown that the large errors produced during abrupt onset of dynamics induces much higher levels of co-contraction, which have been suggested to increase the speed of adaptation (Heald et al. , 2018). It has been suggested that co-contraction might increase the rate of adaptation by concentrating the adaptation within the range of state space in which the dynamics must finally be learned (Heald et al. , 2018), as participants learn adaptation of dynamics as a function of the visited states rather than planned states (Gonzalez Castro et al. , 2011). However, the small kinematic errors that occur during gradual presentation of the force field also mean that participants are never perturbed away from the region of state space to be learned. Thus, gradual presentation could be learned just as quickly as abrupt presentation despite the absence of co-contraction. The similar rates of adaptation under these different conditions likely arises through trade-offs in the competing mechanisms of co-contraction, error sensitivity, and nearness to the learned state space. Abrupt adaptation to novel dynamics has been shown to cause a large increase in initial co-contraction which gradually reduces as the internal model is gradually learned (Thoroughman & Shadmehr, 1999; Franklin et al. , 2003, 2012; Milner & Franklin, 2005; Huang et al. , 2012). This increase in co-contraction from errors is so strong, that it has been shown as a response to an abrupt introduction of a visuomotor rotation (Huang & Ahmed, 2014), despite the fact that co-contraction cannot reduce this type of visual transformation errors. Here we also showed a large initial increase in co-contraction when the dynamics were abruptly applied, but little or no increase in co-contraction when the dynamics were only gradually applied, even though both conditions show adaptation to the force field in terms of force compensation and our EMG based adaptation index. This suggests that it is the errors that drive these changes in co-contraction and not the adaptation process. Despite these differences early in learning, we found similar levels of co-contraction at the end of the exposure phase in both conditions. Therefore, even when co-contraction is not induced through large kinematic errors, co-contraction gradually builds up as part of the adaptation process. The final levels of muscle co-contraction were similar after both gradual and abrupt adaptation to the dynamics, and clearly different from the levels of co-contraction in the null force field as shown previously (Darainy & Ostry, 2008; Franklin et al. , 2012). However, the actual pattern of muscle activation after adaptation varied depending on the manner in which the dynamics were learned. In the abrupt condition participants tended to have higher shoulder muscle activity, whereas in the gradual condition participants tended to have higher biarticular muscle activity. We suggest two possible reasons for these differences in the final pattern of muscle activation. First, it is possible that abrupt onset of novel dynamics produces large errors, but these errors can be clearly associated with a specific cause – the robot is producing a disturbance which is perturbing specific muscles more than others (in this case single joint shoulder muscles). The sensorimotor control system therefore associates the errors with this novel task, forming motor memories of the tool rather than the ody (Berniker & Körding, 2008; Kluzik et al. , 2008). On the other hand, in the gradual condition there are small continuous errors which reduce the task success (Pekny et al. , 2011) but cannot be associated with any specific cause, inducing participants to adapt their baseline or limb dynamics model (Kluzik et al. , 2008; Oh & Schweighofer, 2019). These small errors and reduced task success may be signals of instability (Crevecoeur et al. , 2010) that drive specific increases in biarticular muscle activation as these have specific critical roles in limb stability (McIntyre et al. , 1996; Franklin & Milner, 2003). As there is extensive redundancy in the arm muscles that could be used to compensate for the external loads of the force fields, different conditions would drive different final adaptation measures. The second possible explanation for these findings is that adaptation occurs through a feedback error learning mechanism (Kawato et al. , 1987; Franklin et al. , 2008; Albert & Shadmehr, 2016). In this case, different errors produce different long latency feedback signals which are then incorporated into the feedforward motor command on the subsequent trial. A different pattern of feedback responses would therefore result in a different final pattern of muscle activity. These two possibilities are not mutually exclusive, so the different patterns of muscle activity could be driven by both. However, further experiments are required to determine whether these results are a general result of gradual versus abrupt adaptation or if they are specific to the movement direction studied in this experiment. A consistent finding of higher biarticular muscle activation would be important for use in rehabilitation studies (Patton & Mussa-Ivaldi, 2004; Huang & Patton, 2012; Reinkensmeyer et al. , 2016) where the goal is training new patterns of muscle activity. Gradual adaptation in a split-belt treadmill has already shown to generalize better to normal walking (Torres-Oviedo & Bastian, 2012), and an associated increase in stabilizing biarticular muscle activation could be an additional helpful result of such training if this is a consistent finding. Throughout the experiment, we assessed the gain of the visuomotor feedback response using rapid perturbations of the visual representation of the hand (Brenner & Smeets, 2003; Sarlegna et al. , 2003; Saunders & Knill, 2003). These visuomotor feedback gains demonstrate task modulation (Knill et al. , 2011) and are tuned to changes in the dynamics of the environment (Franklin et al. , 2017). Our previous work showed that early in learning novel dynamics (or when these dynamics are suddenly removed) the visuomotor feedback gains are upregulated (Franklin et al. , 2012) and then gradually reduce to a plateau after adaptation is completed. In the curl force field, the final level of visuomotor feedback gain was higher than in the null field, which was interpreted as adaptation to the increased uncertainty in the curl force field. We hypothesized that this initial increase in feedback gains was driven by increased uncertainty in the internal model of the dynamics (Franklin et al. , 2012), causing large feedback gains similar to the increased co-contraction seen in early adaptation (Thoroughman & Shadmehr, 1999; Franklin et al. , 2003). Here we tested this theory by having participants adapt to abrupt and gradual force field presentations. As in our previous work, the abrupt onset or removal of the force field elicited large increases in feedback gains. However, when the force field was only applied gradually these feedback gains remained low and only increased slowly to the same plateau level after adaptation was complete. However, both the force field compensation and adaptation index show that adaptation to the force field already occurs early in gradual learning, demonstrating that this initial upregulation of feedback gains is not just a side effect of adaptation. Instead, we suggest that it epresents a reactive increase in feedback gains due to the uncertainty in the internal model as signaled by large kinematic errors. Feedback gains during adaptation have also been studied using stretch reflexes (Cluff & Scott, 2013; Coltman & Gribble, 2020), although examining the changes in stretch reflex gains independent of their gain scaling (Pruszynski et al. , 2009) is more difficult. By examining changes in the long latency stretch reflex response for a movement direction in between two movements in which force fields were learned, (Cluff & Scott, 2013) showed that the long latency stretch reflex increased slowly as learning occurred, paralleling the slow increase in the visuomotor feedback gain that we found during gradual adaptation. On the other hand, a recent study (Coltman & Gribble, 2020) found only initial increases in the feedback gain during early learning, but no long-term changes after complete adaptation. However, the perturbation to assess the feedback responses was performed in the dwell period prior to the movement onset, so no long-term upregulation of the feedback gain during this time period would be needed for adaptation. Although this early increase in the feedback gains was attributed to the fast process of a two-state model of motor adaptation (Coltman & Gribble, 2020), we believe that the rapid increase in feedback gains, particularly as seen here and previously (Franklin et al. , 2012), could be better described as a reactive increase in response to error rather than as part of the adaptation process itself. This is particularly likely for feedback gains seen prior to movement initiation when little or no adaptation is ever found in the predictive force against the channel wall (Joiner et al. , 2011; Alhussein et al. , 2019). Further supporting our interpretation, we find identical levels of visuomotor feedback gain to perturbations to the left or right (see Fig. 7), even though adaptation as predicted by the fast process shows a directionality according to the force field. Overall, feedback gains during adaptation to novel dynamics demonstrate a clear pattern of modulation (Fig. 10A), with an initial rapid increase associated with large kinematic errors, followed by a gradual reduction to a new baseline level during the exposure. We suggest that this pattern is actually composed of two complementary processes (Fig. 10B) which we term reactive and predictive feedback gains (Franklin et al. , 2017). Any large errors, signaling model uncertainty, produce rapid increases in reactive feedback gains, which are gradually reduced as learning occurs. We propose that after adaptation is complete, these contribute little or nothing to the overall feedback responses, where this pattern matches perfectly the changes found in the recent paper of Coltman and Gribble (Coltman & Gribble, 2020). However, throughout the adaptation process, feedback responses are also learned and tuned according to the dynamics of the environment (Franklin et al. , 2017), becoming predictive feedback gains which are part of the learned motor memory to compensate for the force field (Franklin et al. , 2007; Wagner & Smith, 2008; Ahmadi-Pajouh et al. , 2012; Maeda et al. , 2018). The time course of these predictive change in the feedback gains matches well the changes seen in Cluff and Scott (Cluff & Scott, 2013) where no reactive responses were required as no errors were experienced on these trials. These two contributions to the overall pattern of feedback control would explain the different results found in a variety of experiments (Franklin et al. , 2012; Cluff & Scott, 2013; Coltman & Gribble, 2020). Furthermore, we predict that each of these two components would have different properties; reactive feedback gains are likely to be broader in terms of temporal timing (before and after the movement) whereas predictive feedback gains may be more likely to eneralize spatially to nearby movements similar to the generalization of predictive force (Shadmehr & Mussa-Ivaldi, 1994; Malfait et al. , 2002; Berniker et al. , 2014). Figure 10.
Schematic of feedback changes during adaptation to novel dynamics. A: The pattern of feedback gain modulation during adaptation and de-adaptation to changes in environmental dynamics. Increases in feedback gain occur when the dynamics are changed (onset or offset), and the final level of feedback gain after adaptation to the force field is different to that in the null force field. B: We propose that the total feedback gain is comprised of the reactive (purple) and predictive (blue) changes in feedback gains during adaptation. The reactive feedback gains increase immediately in response to large errors (model predictive errors) and gradually reduce as learning occurs. The predictive feedback gains gradually increase (or decrease) during adaptation as they are tuned to the environment. Abrupt and gradual adaptation produce very different initial patterns of force compensation, muscle activity, co-contraction and feedback gains, but finally result in a similar pattern after adaptation despite the different time course of errors. Although the final adaptation is similar, there still remain subtle differences in terms of the pattern of muscle activity (Fig. 4&5) and generalization (Malfait & Ostry, 2004). However, the different patterns of visuomotor feedback regulation allow us to separate out two components of feedback regulation: reactive and predictive. Here we argue that internal model uncertainty drives upregulation of the reactive feedback gains, and that adaptation tunes the predictive feedback gains according to the environment.
Acknowledgements:
The authors declare no competing financial interests.
Block V i s u o m o t o r G a i n Block V i s u o m o t o r G a i n BA exposuretotal feedback predictivereactiveexposure eferences Ahmadi-Pajouh, M.A., Towhidkhah, F., & Shadmehr, R. (2012) Preparing to reach: selecting an adaptive long-latency feedback controller.
Journal of Neuroscience , , 9537–9545. Albert, S.T. & Shadmehr, R. (2016) The Neural Feedback Response to Error As a Teaching Signal for the Motor Learning System. Journal of Neuroscience , , 4832–4845. Alhussein, L., Hosseini, E.A., Nguyen, K.P., Smith, M.A., & Joiner, W.M. (2019) Dissociating effects of error size, training duration, and amount of adaptation on the ability to retain motor memories. Journal of Neurophysiology , , 2027–2042. Berniker, M., Franklin, D.W., Flanagan, J.R., Wolpert, D.M., & Kording, K. (2014) Motor learning of novel dynamics is not represented in a single global coordinate system: evaluation of mixed coordinate representations and local learning. Journal of neurophysiology , , 1165–1182. Berniker, M. & Körding, K.P. (2008) Estimating the sources of motor errors for adaptation and generalization. Nature Neuroscience , , 1454–1461. Brenner, E. & Smeets, J.B.J. (2003) Fast corrections of movements with a computer mouse. Spatial vision , , 365–376. Cluff, T. & Scott, S.H. (2013) Rapid feedback responses correlate with reach adaptation and properties of novel upper limb loads. J Neurosci , , 15903–15914. Coltman, S.K. & Gribble, P.L. (2020) Time course of changes in the long-latency feedback response parallels the fast process of short-term motor adaptation. Journal of Neurophysiology , , 388–399. Conditt, M.A., Gandolfo, F., & Mussa-Ivaldi, F.A. (1997) The motor system does not learn the dynamics of the arm by rote memorization of past experience. Journal of neurophysiology , , 554–560. Crevecoeur, F., McIntyre, J., Thonnard, J.-L., & Lefèvre, P. (2010) Movement stability under uncertain internal models of dynamics. Journal of neurophysiology , , 1301–1313. Cross, K.P., Cluff, T., Takei, T., & Scott, S.H. (2019) Visual Feedback Processing of the Limb Involves Two Distinct Phases. J Neurosci , , 6751–6765. Darainy, M. & Ostry, D.J. (2008) Muscle cocontraction following dynamics learning. Experimental brain research , , 153–163. Dimitriou, M., Wolpert, D.M., & Franklin, D.W. (2013) The Temporal Evolution of Feedback Gains Rapidly Update to Task Demands. J Neurosci , , 10898–10909. Fine, M.S. & Thoroughman, K.A. (2006) Motor adaptation to single force pulses: sensitive to direction but insensitive to within-movement pulse placement and magnitude. Journal of neurophysiology , , 710–720. Franklin, D.W., Burdet, E., Tee, K.P., Osu, R., Chew, C.-M., Milner, T.E., & Kawato, M. (2008) CNS learns stable, accurate, and efficient movements using a simple algorithm. The Journal of neuroscience , 11165–11173. Franklin, D.W., Franklin, S., & Wolpert, D.M. (2014) Fractionation of the visuomotor feedback response to directions of movement and perturbation. J Neurophysiol , , 2218–2233. ranklin, D.W., Liaw, G., Milner, T.E., Osu, R., Burdet, E., & Kawato, M. (2007) Endpoint stiffness of the arm is directionally tuned to instability in the environment. J Neurosci , , 7705–7716. Franklin, D.W. & Milner, T.E. (2003) Adaptive control of stiffness to stabilize hand position with large loads. Experimental brain research , , 211–220. Franklin, D.W., Osu, R., Burdet, E., Kawato, M., & Milner, T.E. (2003) Adaptation to stable and unstable dynamics achieved by combined impedance control and inverse dynamics model. J Neurophysiol , , 3270–3282. Franklin, D.W. & Wolpert, D.M. (2008) Specificity of reflex adaptation for task-relevant variability. J Neurosci , , 14165–14175. Franklin, S., Wolpert, D.M., & Franklin, D.W. (2012) Visuomotor feedback gains upregulate during the learning of novel dynamics. J Neurophysiol , , 467–478. Franklin, S., Wolpert, D.M., & Franklin, D.W. (2017) Rapid visuomotor feedback gains are tuned to the task dynamics. J Neurophysiol , , 2711–2726. Gonzalez Castro, L.N., Monsen, C.B., & Smith, M.A. (2011) The binding of learning to action in motor adaptation. PLoS computational biology , , e1002052. Gu, C., Wood, D.K., Gribble, P.L., & Corneil, B.D. (2016) A Trial-by-Trial Window into Sensorimotor Transformations in the Human Motor Periphery. J Neurosci , , 8273–8282. Hayashi, T., Kato, Y., & Nozaki, D. (2020) Divisively Normalized Integration of Multisensory Error Information Develops Motor Memories Specific to Vision and Proprioception. J. Neurosci. , , 1560–1570. Heald, J.B., Franklin, D.W., & Wolpert, D.M. (2018) Increasing muscle co-contraction speeds up internal model acquisition during dynamic motor learning. Sci Rep , , 16355. Howard, I.S., Ingram, J.N., Franklin, D.W., & Wolpert, D.M. (2012) Gone in 0.6 seconds: the encoding of motor memories depends on recent sensorimotor States. Journal of Neuroscience , , 12756–12768. Howard, I.S., Ingram, J.N., & Wolpert, D.M. (2009) A modular planar robotic manipulandum with end-point torque control. J Neurosci Meth , , 199–211. Huang, F. & Patton, J. (2012) Augmented dynamics and motor exploration as training for stroke. Ieee Transactions on Biomedical Engineering ,. Huang, H.J. & Ahmed, A.A. (2014) Reductions in muscle coactivation and metabolic cost during visuomotor adaptation.
Journal of neurophysiology , , 2264–2274. Huang, H.J., Kram, R., & Ahmed, A.A. (2012) Reduction of metabolic cost during motor learning of arm reaching dynamics. The Journal of neuroscience , , 2182–2190. Huang, V.S. & Shadmehr, R. (2009) Persistence of motor memories reflects statistics of the learning event. Journal of neurophysiology , , 931–940. Joiner, W.M., Ajayi, O., Sing, G.C., & Smith, M.A. (2011) Linear hypergeneralization of learned dynamics across movement speeds reveals anisotropic, gain-encoding primitives for motor adaptation. Journal of neurophysiology , , 45–59. Kawato, M., Furukawa, K., & Suzuki, R. (1987) A hierarchical neural-network model for control and learning of voluntary movement. Biological cybernetics , , 169–185. lassen, J.J., Tong, C.C., & Flanagan, J.R.J. (2005) Learning and recall of incremental kinematic and dynamic sensorimotor transformations. Experimental Brain Research , , 250–259. Kluzik, J., Diedrichsen, J., Shadmehr, R., & Bastian, A.J. (2008) Reach adaptation: what determines whether we learn an internal model of the tool or adapt the model of our arm? Journal of neurophysiology , , 1455–1464. Knill, D.C., Bondada, A., & Chhabra, M. (2011) Flexible, task-dependent use of sensory feedback to control hand movements. The Journal of neuroscience, , 1219–1237. Lackner, J.R. & Dizio, P. (1994) Rapid adaptation to Coriolis force perturbations of arm trajectory. Journal of neurophysiology , , 299–313. Maeda, R.S., Cluff, T., Gribble, P.L., & Pruszynski, J.A. (2018) Feedforward and Feedback Control Share an Internal Model of the Arm’s Dynamics. J. Neurosci. , , 10505–10514. Malfait, N. & Ostry, D.J. (2004) Is interlimb transfer of force-field adaptation a cognitive response to the sudden introduction of load? The Journal of neuroscience , , 8084–8089. Malfait, N., Shiller, D.M., & Ostry, D.J. (2002) Transfer of motor learning across arm configurations. The Journal of neuroscience , , 9656–9660. Marko, M.K., Haith, A.M., Harran, M.D., & Shadmehr, R. (2012) Sensitivity to prediction error in reach adaptation. Journal of Neurophysiology , , 1752–1763. McIntyre, J., Mussa-Ivaldi, F.A., & Bizzi, E. (1996) The control of stable postures in the multijoint arm. Experimental brain research , , 248–264. Milner, T.E., Firouzimehr, Z., Babadi, S., & Ostry, D.J. (2018) Different adaptation rates to abrupt and gradual changes in environmental dynamics. Exp Brain Res , , 2923–2933. Milner, T.E. & Franklin, D.W. (2005) Impedance control and internal model use during the initial stage of adaptation to novel dynamics in humans. The Journal of physiology , , 651–664. Oh, Y. & Schweighofer, N. (2019) Minimizing Precision-Weighted Sensory Prediction Errors via Memory formation and switching in motor adaptation. Journal of Neuroscience ,. Oldfield, R.C. (1971) The assessment and analysis of handedness: the Edinburgh inventory.
Neuropsychologia , , 97–113. Orban de Xivry, J.-J., Criscimagna-Hemminger, S.E., & Shadmehr, R. (2011) Contributions of the motor cortex to adaptive control of reaching depend on the perturbation schedule. Cerebral cortex (New York, N.Y. : 1991) , , 1475–1484. Osu, R., Franklin, D.W., Kato, H., Gomi, H., Domen, K., Yoshioka, T., & Kawato, M. (2002) Short- and long-term changes in joint co-contraction associated with motor learning as revealed from surface EMG. Journal of neurophysiology , , 991–1004. Patton, J.L. & Mussa-Ivaldi, F.A. (2004) Robot-assisted adaptive training: custom force fields for teaching movement patterns. Ieee Transactions on Biomedical Engineering , , 636–646. Pekny, S.E., Criscimagna-Hemminger, S.E., & Shadmehr, R. (2011) Protection and expression of human motor memories. Journal of Neuroscience , , 13829–13839. ruszynski, J.A., Kurtzer, I., Lillicrap, T.P., & Scott, S.H. (2009) Temporal evolution of "automatic gain-scaling". Journal of neurophysiology , , 992–1003. Reichenbach, A., Franklin, D.W., Zatka-Haas, P., & Diedrichsen, J. (2014) A dedicated binding mechanism for the visual control of movement. Curr Biol , , 780–785. Reinkensmeyer, D.J., Burdet, E., Casadio, M., Krakauer, J.W., Kwakkel, G., Lang, C.E., Swinnen, S.P., Ward, N.S., & Schweighofer, N. (2016) Computational neurorehabilitation: modeling plasticity and learning to predict recovery. Journal of neuroengineering and rehabilitation , , 1–26. Sarlegna, F., Blouin, J., Bresciani, J.-P., Bourdin, C., Vercher, J.-L., & Gauthier, G.M. (2003) Target and hand position information in the online control of goal-directed arm movements. Experimental brain research , , 524–535. Saunders, J.A. & Knill, D.C. (2003) Humans use continuous visual feedback from the hand to control fast reaching movements. Experimental brain research , , 341–352. Scheidt, R.A., Reinkensmeyer, D.J., Conditt, M.A., Rymer, W.Z., & Mussa-Ivaldi, F.A. (2000) Persistence of motor adaptation during constrained, multi-joint, arm movements. Journal of neurophysiology , , 853–862. Shadmehr, R. & Mussa-Ivaldi, F.A. (1994) Adaptive representation of dynamics during learning of a motor task. The Journal of neuroscience , , 3208–3224. Smith, M.A., Ghazizadeh, A., & Shadmehr, R. (2006) Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biology , , e179. Thoroughman, K.A. & Shadmehr, R. (1999) Electromyographic correlates of learning an internal model of reaching movements. The Journal of neuroscience , , 8573–8588. Torres-Oviedo, G. & Bastian, A.J. (2012) Natural error patterns enable transfer of motor learning to novel contexts. Journal of neurophysiology , , 346–356. Wagner, M.J. & Smith, M.A. (2008) Shared internal models for feedforward and feedback control. Journal of Neuroscience , , 10663–10673. Wei, K. & Körding, K. (2009) Relevance of Error: What Drives Motor Adaptation? Journal of Neurophysiology ,101