The type of motor task and nature of the instruction can have varying effects on motor execution and learning [1] — [3]. In the serial reaction time task SRT , participants produce a sequence of cued button presses. If the participant is informed of the underlying sequence, learning occurs much more rapidly compared to when sequential learning arises from repeated performance [4].

However, learning in the SRT task entails the linkage of a series of discrete actions. Explicit instructions of the sequence structure may be viewed as a way to create a working memory representation of the series.

Many skills lack such a clear elemental partition and, as such, participants cannot easily verbalize what a successful movement entails.

For example, the pattern of forces required to move the hand in a straight line in a novel force field [5] — [7] would be hard to verbalize.

Various studies have examined the role of explicit strategies in tasks involving sensorimotor adaptation [8] — [11]. The benefits of an explicit strategy may be illusory with adaptive processes arising from automatic and incremental updating of a motor system that is impenetrable to conscious intervention [12] — [16].

However, performance measures indicate that adaptation may differ between conditions in which participants are either aware or unaware of the changes in the environment [17].

For example, a large visuomotor rotation can be introduced abruptly, in which case, awareness is likely, or introduced incrementally such that participants are unaware of the rotation. The abrupt onset of large unexpected errors may promote the use of cognitive strategies [18] — [20]. Participants who gain explicit knowledge of an imposed visuomotor rotation show better performance during learning than participants who report little or no awareness of the rotation [10].

Moreover, the rate of learning, at least in the early phase of adaptation, correlates positively with spatial working memory span [21] , suggesting that strategic compensation may be dependent on working memory capacity.

Studies of sensorimotor adaptation during aging also indicate that the rate of learning is slower in older adults compared to young adults, despite similar aftereffects [22] — [24]. This cost is absent in older adults who report awareness of the rotation [25].

In many of the studies cited above, the assumption has been that the development of awareness can lead to the utilization of compensatory strategies.

However, few studies have directly sought to manipulate strategic control during sensorimotor adaptation. One striking exception is a study by Mazzoni and Krakauer [9].

Participants viewed a display of eight small circles, or visual landmarks, that were evenly spaced by 45° to form a large, implicit ring.

The target location was specified by presenting a bullseye within one of the eight circles. After an initial training phase in which the visuomotor mapping was unaltered, a 45° rotation in the counterclockwise direction CCW was introduced.

In the standard condition in which no instructions were provided, participants gradually reduced endpoint error by altering their movement heading in the clockwise direction CW.

In the strategy condition, participants were given explicit instructions to move to the circle located 45° clockwise to the target. This strategy enabled these participants to immediately eliminate all error. However, as training continued, the participants progressively increased their movement heading in the clockwise direction.

As such, the endpoint location of the feedback cursor drifted further from the actual target location and, thus, performance showed an increase in error over training, a rather counterintuitive result [26].

Mazzoni and Krakauer [9] proposed that this drift arises from the implicit adaptation of an internal forward model. Importantly, the error signal for this learning process is not based on difference between the observed visual feedback and target location. Rather, it is based on the difference between the observed visual feedback and strategic aiming location.

Even though participants aim to a clockwise location of the target as instructed , the motor system experiences a mismatch between the predicted state and the visual feedback.

This mismatch defines an error signal that is used to recalibrate the internal model. Reducing the mismatch results in an adjustment of the internal model such that the next movement will be even further in the clockwise direction.

Thus, the operation of an implicit learning process that is impervious to the strategy produces the paradoxical deterioration in performance over time. In the present paper, we start by asking how this hypothesis could be formalized in a computational model of motor learning.

State space modeling techniques have successfully described adaptation and generalization during motor learning [27] — [29].

These models focus on how learning mechanisms minimize error from trial to trial. Variants of these models postulate multiple learning mechanisms that operate at different time scales [28].

Within this framework, strategic factors might be associated with fast learning processes that rapidly reduce error. However, such models are unable to account for the drift that arises following the deployment of a strategy. To address these issues, we developed a series of setpoint state-space models of adaptation to quantitatively explore how strategic control and implicit adaptation interact.

Assuming a fixed strategy, adaptation should continue to occur until the error signal, the difference between the feedback location and the aiming location is zero; that is, the visual feedback matches the intended aim of the reach.

As such, drift arising from implicit adaptation should continue to rise until it offsets the adopted strategy. To test this prediction, we increased the length of the adaptation phase. Moreover, we manipulated the salience of the visual landmarks used to support the strategy.

We hypothesized that these landmarks served as a proxy for the aiming location. If this assumption is correct, then elimination of the visual landmarks should weaken the error signal, given uncertainty concerning the aiming location, and drift should be attenuated.

We test this prediction by comparing performance with and without visual landmarks. Current models of sensorimotor adaptation have not addressed the effect of explicit strategies.

Therefore, we started with the standard state-space model Eq 1 and 2 , and incrementally modified it to accommodate the use of an explicit strategy.

The standard model for target error is given as: 1 2 where is the target endpoint error on movement n, is the rotation, is the internal model's estimation of the rotation, A is a memory term, and B represents either a learning rate or sensitivity to error [27] — [30].

A Simulated target error for four state space models. C Effect of a variable strategy on target error. When informed of an appropriate strategy that will compensate for the rotation, participants immediately counteract the rotation and show on-target accuracy.

The standard model as formulated above does not provide a mechanism to implement an explicit strategy. To allow immediate implementation of the strategy, we postulate that there is direct feedthrough of the strategy s to the target error equation equation 1 : 3.

Direct feedthrough allows the strategy to contribute to the target error equation without directly influencing the updating of the internal model. If the strategy operated through the internal model, then the impact of the strategy would take time to evolve, assuming there is substantial memory of the internal model's estimation of the rotation i.

With direct feedthrough, the implementation of an appropriate strategy can immediately compensate for the rotation. In the current arrangement, the appropriate strategy is fixed at 45° in the CW direction from the cued target.

Once the strategy is implemented, performance should remain stable since the error term is small. Indeed, a model based on Eq. The target error, the difference between the feedback location and target location, is essentially zero on the first trial with the strategy, and remains so throughout the rotation block Figure 1B — green line.

However, this model fails to match the empirical results observed by Mazzoni and Krakauer [9] : performance drifts over time with an increase in errors in the direction of the strategy. This phenomenon led the authors to suggest that the prediction error signal to the internal model is not based on target error.

Instead, the error signal should be defined by the difference between the feedback location and aiming location see Figure 2E :. The experiment workspace consisted of 8 empty blue circles separated by 45° three locations are shown here. The target was defined when a green circle appeared at one of the locations.

The hand was occluded by the apparatus and on feedback trials, a red cursor appeared as soon as the participant crossed a virtual ring, cm from the start location.

A In the baseline block, participants moved towards the cued green target. B In the strategy-only block, participants moved to the blue circle located 45° in the clockwise direction. Feedback was presented at the veridical hand position.

C For the two rotation probes, participants were instructed to move to the green target, but feedback of hand position was rotated 45° in the counter-clockwise direction. D In the rotation plus strategy block, participants were instructed to move to the blue circle located 45° clockwise direction from the target.

The feedback of hand position was rotated 45° counter-clockwise. E Two sources of movement error: a target error between the feedback location and target location and an aiming error between the feedback location and aiming location.

In typical motor learning studies, the setpoint is to reach to the target. The input error to update the internal model's estimate of the rotation becomes: 5. This model shows immediate compensation for the visuomotor rotation, and more importantly, produces a gradual deterioration in performance over the course of continued training with the reaching error drifting in the direction of the strategy Figure 1A — red line , consistent with the results reported by Mazzoni and Krakauer [9].

It is important to emphasize that the error signal for sensorimotor recalibration in Eq. Rather, the error signal is defined by the difference between the feedback location and aiming location, or what we will refer to as aiming error.

When a fixed strategy is adopted throughout training Figure 1B — blue line , the aiming error is initially quite large given that the predicted hand location is far from the location of visual feedback, even though the feedback cursor may be close to the actual target.

In its simplest form, the setpoint model predicts that, as the internal model minimizes this error Figure 1B — red line , drift will continue until the observed feedback of the hand matches the aiming location. That is, the magnitude of the drift should equal the size of the strategic adjustment.

In the Mazzoni and Krakauer set-up [9] , the drift would eventually reach 45° in the CW direction Figure 1A — red line. A second prediction can be derived by considering that the error signal in Eq.

We assume that a visual landmark in the display can be used as a reference point for strategy implementation e. This landmark can serve as a proxy for the aiming location. The salience of this landmark provides an accurate estimate of the aiming location and, from Eq.

However, if these landmarks are not available, then the estimate of the aiming location will be less certain. Previous studies have shown that adaptation is attenuated when sensory feedback is noisy [31] , [32].

One approach for modeling the effect of changing the availability or certainty of the strategy defined aiming location would be to vary the adaptation rate B. For example, B could be smaller if there is a decrease in certainty of the aiming location, and correspondingly, a decrease in the certainty of the aiming error.

This model predicts that the rate of drift is directly related to B: if B is lower due to decreased certainty of the aiming location, then the rate of drift will be attenuated Figure 1A — cyan line.

To evaluate the predictions of this setpoint model, participants were tested in an extended visuomotor rotation task in which we varied the visual displays used to define the target and strategic landmarks see methods.

The target was defined as a green circle, appearing at one of eight possible locations, separated by 45° Figure 2 , only three shown here. We assume that participants mostly relied on feedforward control given the ballistic nature of the movements and absence of continuous online feedback.

For the aiming-target group AT , the blue circles were always visible, similar to the method used by Mazzoni and Krakauer. For the disappearing aiming-target group AT , the blue circles were visible at the start of the trial and disappeared when the movement was initiated.

For the no aiming-target group NoAT , the blue landmarks were not included in the display. The participants were initially required to reach to the green target Figure 2A. Movement duration, measured when the hand crossed the target ring, averaged ± Following the initial familiarization block, participants were trained to use a strategy of moving 45° in the CW direction from the green target location, Figure 2B.

This location corresponded to the position of the neighboring blue circle. Feedback was veridical in this phase e. To help participants in the NoAT group learn to move at 45°, the blue circles were also presented on half of the trials for this group in this phase only.

The mean angular shifts, relative to the green target, were For the NoAT group, the mean angular shift was Participants first practiced moving to the cued target without a rotation black and while using the strategy without a rotation orange. The rotation was turned on between movements and dashed vertical lines.

For the first two of these trials, the rotation probes, the participants had not been given the strategy X's. For the next rotation trials, participants were instructed to use the strategy. Following this, the rotation was turned off and participants were instructed to move towards the cued target, first without endpoint feedback X's and then with endpoint feedback circles.

A Aiming-Target Group blue. B Disappearing Aiming-Target Group magenta. C No Aiming-Target Group red. Practicing the 45° CW strategy did not produce interference on a subsequent baseline block in which participants were again instructed to reach to the cued, green target Figure 3 — black.

Without warning, the CCW rotation was introduced Figure 2C. As expected, the introduction of the CCW rotation induced a large target error. After the participants were instructed to use the clockwise strategy Figure 2D , the target error was reduced immediately to 3.

The participants were then instructed to use the strategy and required to produce a total of reaching movements under the CCW rotation. This extended phase allowed us to a verify that error increased over time, drifting in the direction of the strategy, and b determine if the magnitude of the drift would approximate the magnitude of the rotation, a prediction of the simplest form of the setpoint model.

Consistent with the results of Mazzoni and Krakauer [9] , error increased in the direction of the strategy over the initial phase of the rotation block.

However, the extent of the drift fell far short of the magnitude of the rotation. To quantify the peak drift, each participant's time series of endpoint errors was averaged over 10 movements and we identified the bin with the largest error.

This is consistent with the prediction of the model based that the salience of the aiming targets would influence the estimation of the aiming location. Drift was largest when the aiming targets were always visible, and progressively less for the DAT and NoAT groups.

Drift was not isolated to particular target locations Figure 4B. A Average endpoint angular error relative to the target for the three groups, binned by averaging over epochs of ten movements AT group in blue, DAT group in magenta, NoAT group in red. B Peak drift with respect to the eight target location for the three groups.

The empty circles are the target locations. To identify peak drift, 10 bins of four movements were calculated for each direction. C Angular error after the rotation was turned off and participants were instructed to stop using the strategy. Triangles are average of the first eight post-rotation trials, performed without visual feedback.

Squares are washout block with feedback. D Relationship of drift and aftereffect based on the estimated peak drift for each participant and the first eight post-rotation trials. Our rotation plus strategy block lasted trials, nearly four times the number of trials used by Mazzoni and Krakauer [9].

This larger window provides an interesting probe on learning given that the participants become progressively worse in performance with respect to the target over the drift phase. While the AT group had the largest drift, they eventually showed a change in performance such that the heading angle at the end of the rotation block was close to 45° CW from the green target Figure 3A.

By the end of training, their target error was only 0. We did not observe a consistent pattern in how these participants counteracted the drift Figure 5. Two participants showed clear evidence of an abrupt change in their performance, suggesting a discrete change in their aiming strategy.

For the other eight AT participants, the changes in performance were more gradual. A and B are from the AT group; C is from the DAT group. A Drift followed by large fluctuations in error.

B Drift followed by an abrupt change in target error. C Continuous drift across training. The drift persisted over the trials of the rotation block for participants in the DAT group Figure 3C.

The average drift was 5. Given that the NoAT group showed minimal drift, we did not observe any consistent changes in performance over the block. At the end of training, the mean target error was only 1. The availability or certainty in the estimate of the aiming location was manipulated by altering the presence of the aiming target across the groups.

As predicted by the setpoint model, the degree of drift was attenuated as the availability of the aiming targets decreased. In the current implementation of our model, this decrease in drift rate is captured by a decrease in the adaptation rate B : with greater uncertainty, the weight given to the error term for updating the internal model is reduced.

However, one prediction of this model is at odds with the empirical results. Variation in the adaptation rate not only predicts a change in drift rate, but also predicts a change in the washout period.

Specifically, decreasing the adaptation rate should produce a slower washout, or extended aftereffect Figure 1B — cyan. This prediction was not supported. One could hypothesize different adaptation rates during the rotation and washout phases, with the effect of target certainty only relevant for the former.

However, a post hoc hypothesis along these lines is hard to justify. Alternatively, it is possible that the adaptation rate B is similar for the three groups and that the variation in drift rate arises from another process.

One possibility is that the manipulation of the availability of the aiming targets influences the certainty of the desired strategy term in Equation 4, and correspondingly, modifies the aiming error term: 6.

Consequently, the error used to adjust the internal model will be smaller and produce attenuated drift Figure 1B — magenta line. Moreover, because the strategy is no longer used during the washout phase, the K term is no longer relevant.

Thus, the washout rates should be identical across the three groups, assuming a constant value of B. In sum, while variation in B or K can capture the group differences in drift rate, only the latter accounts for the similar rates of washout observed across groups.

When the availability of the aiming targets is reduced, either by flashing them briefly or eliminating them entirely, the participants' certainty of the aiming location is attenuated. This hypothesis is consistent with the notion that the aiming locations serve as a proxy for the predicted aiming location.

As noted above, none of the participants showed drift approaching 45°. Even those exhibiting the largest drift eventually reversed direction such that they became more accurate over time in terms of reducing endpoint error with respect to the target location.

To capture this feature of the results, we considered how participants might vary their strategy over time as performance deteriorates. It is reasonable to assume that the participant may recognize that the adopted strategy should be modified to offset the rising error.

One salient signal that could be used to adjust the strategy is the target error, the difference between the target location and the visual feedback. To capture this idea, we modified the setpoint model, setting the strategy as a function of target error Figure 2E : 7 where E defines the retention of the state of the strategy and the F defines the rate of strategic adjustment.

As target error grows i. In our initial implementation of the setpoint model, the strategy term was fixed at 45°. Equation 7 allows the strategy term to vary, taking on any value between 0° and °.

The availability of the aiming targets, captured by K in Eq. Thus, the relative values of K and F determine the degree of performance error that is tolerated before strategic adjustments compensate to offset the drift Figure 1C. This setpoint model Eqs. When the aiming targets remain visible, the aiming error signal is readily available, and the weight given to the strategic aiming location, K, is larger.

These results are consistent with the hypothesis that participants in the NoAT group rely more on target errors because the absence of the aiming targets removes a reference point for generating a reliable aiming error Eq.

The setpoint model with direct-feedthrough equations 8—11 was fit to each group's data from figure 3 via bootstrapping. A—C: Solid functions are averaged model fits for the AT group blue , DAT group magenta , and NoAT group red in comparison with actual group averaged data black.

D The target error is the combination of learning within the sensorimotor recalibration process solid and the aiming location associated with the strategy dashed , shown here fro the AT group blue , DAT group magenta , and NoAT group red. The dynamics of the recalibration process and strategy state Eqs.

These parameters, along with the other parameters that represent the memory of the internal model A , the adaptation rate B , and the memory of the strategy E are listed in Table 1. Following the rotation block, we instructed the participants that the rotation would be turned off and they should reach to the cued green target.

For the first eight trials, no endpoint feedback was presented. This provided a measure of the degree of sensorimotor recalibration in the absence of learning Figure 4C — triangles. Aftereffects were observed in all three groups. In comparisons between the groups, the AT group showed the largest aftereffect of The mean aftereffects for the DAT and NoAT groups were When endpoint feedback was again provided, the size of the aftereffect diminished over the course of the washout block Figure 4C - squares.

In the setpoint model, the internal model will continue to adapt even in the face of strategic adjustments adopted to improve endpoint accuracy. As such, the model predicts that the size of the aftereffect should be larger than the degree of drift. To test this prediction, we compared the peak drift during the rotation block to the aftereffect.

In the preceding analysis, we had estimated peak drift for each participant by averaging over 10 movements and identifying the bin with the largest error. However, a few errant movements could easily bias the estimate of drift within a movement bin. As an alternative procedure, we used a bootstrapping procedure to identify the bin with the largest angular error for each group.

This method should decrease the effect of noise because the estimate of peak drift is selected from an averaged sample of the participants' data. Moreover, any bias in the estimate of the magnitude of the peak should be uniform across the three groups of participants.

For consistency, we estimated the aftereffect the first 8 trials without feedback using the same bootstrap procedure. For the AT group, the peak drift was For the DAT group, the peak drift was For the NoAT group, peak drift was only 3.

The difference between the degree of peak drift and aftereffect in the DAT group was not reliable. It is important to emphasize that estimates of the time of peak drift should be viewed cautiously, especially in terms of comparisons between the three groups.

These estimates have lower variance for the AT group because it was easier to detect the point of peak drift in this group compared to the DAT and NoAT groups. Visuomotor rotation tasks are well-suited to explore how explicit cognitive strategies influence sensorimotor adaptation.

Between groups, we manipulated the information available to support the strategy by either constantly providing an aiming target, blanking the aiming target at movement initiation, or never providing an aiming target.

In all groups, the strategy was initially effective, resulting in the rapid elimination of the rotation-induced endpoint error. However, when the aiming target was present, participants showed a drift in the direction of the strategy, replicating the behavior observed in Mazzoni and Krakauer [9].

This effect was markedly attenuated when the aiming target was not present suggesting that an accurate estimate of the strategic aiming location is responsible for causing the drift. In addition, when the drift became quite large as in the AT group , participants begin to adjust their strategy to offset the implicit drift.

Mathematical models of sensorimotor adaptation have not explicitly addressed how a strategy influences learning and performance. By formalizing the effect of strategy usage into the standard state-space model of motor learning, we can begin to evaluate qualitative hypotheses that have been offered to account for the influence of strategies on motor learning.

Mazzoni and Krakauer [9] suggested that drift reflects the interaction of the independent contribution of strategic and implicit learning processes in movement execution.

Current models of adaptation cannot be readily modified to account for this interaction. Rather, we had to consider more substantive architectural changes. Borrowing from engineering control theory, we used a setpoint model in which the internal model can be recalibrated around any given reach location.

The idea of a setpoint is generally implicit in most models of learning, but this component does not come into play since the regression is around zero. However, simply making the setpoint explicit is not sufficient to capture the drift phenomenon. The strategy must have direct feedthrough to the output equation in order to implement the explicit strategy while allowing for an internal model to implicitly learn the visuomotor rotation.

This simple setpoint model was capable of completely eliminating error on the first trial and capture the deterioration of performance with increased training. Drift arises because the error signal is driven by the difference between the internal model's prediction of the aiming location and the actual, endpoint feedback.

The idea that an aiming error signal is the source of drift is consistent with the conjecture of Mazzoni and Krakauer [9]. An important observation in the current study is that, given uncertainty in the prediction of the aiming location, participants use external cues as a proxy in generating this prediction.

This hypothesis accounts for the observation that drift was largest when the aiming target was always visible, intermediate when the aiming target was only visible at the start of the trial, and negligible when the aiming target was never visible.

The aiming target, when present, served as a proxy for predicted hand position, and helped define the error between the feedback cursor and aiming location in visual coordinates.

When the aiming target was not present, the aiming location was less well-defined in visual coordinates, and thus, the relationship between the aiming location and feedback cursor was less certain.

Under this condition, the participant's certainty of the error was reduced and adaptation based of this signal was attenuated.

Quantitatively, progressively smaller values of K were observed with decreasing availability of the aiming targets. The attenuation of adaptation with increasing uncertainty as reflected by reduced drift is similar to the effects on adaptation predicted by a Kalman filter when measurement noise is large.

Several studies have shown that adaptation rates can change when the certainty of sensory information is manipulated [31] , [32]. In our study, variation in certainty of the desired aiming location K influenced the magnitude of drift. As the availability of the aiming targets was reduced, the corresponding estimate of the aiming error became less certain, producing slower adaptation of the internal model, or reduced drift.

Moreover, since K directly operates on the estimate of the strategic aiming location, this parameter does not affect the rate of washout since the strategy is no longer used.

Consistent with this prediction, the rate of washout was similar across the three groups. The effect of the visual landmarks on adaptation also provides insight into why other studies have not observed drift, even when participants develop some explicit awareness of the rotation, and presumably, use that knowledge [8] , [10] , [11] , [24] , [25] to improve performance rapidly.

Several key methodological differences are relevant. First, in most visuomotor rotation studies, online visual feedback is provided during the movements. This may impede drift because participants observe the casual relationship between movement of their hand and the endpoint, cursor feedback [33].

Drift itself could be corrected by online feedback. Second, participants in the earlier studies were not given a clear, explicit strategy, and importantly, were not provided with visual landmarks that could support a self-generated strategy.

Under such conditions, participants face a difficult estimation process. The absence of landmarks would increase uncertainty in implementing a self-generated strategy. Moreover, the motor system would not have a salient visual signal for grounding the comparison of feedback and aiming location. As shown by our no-aiming target condition, drift is minimal when the landmarks are absent.

Thus, the absence of drift in the visuomotor adaptation literature cannot be taken as evidence that strategies are not relevant. It is likely that, when initial error signals are large, learning involves a combination of strategic and recalibration processes.

Our model entails two types of error signals: an aiming prediction error between the feedback location and aiming location, and performance error between the feedback location and the target location Figure 2E. The aiming error drives the drift phenomenon while the target error is used to restore performance.

Intuitively, the motor system should be able to recalibrate the internal model around any desired reach location, a feature captured by the setpoint model. When there is an accurate estimate of the strategy the setpoint , then the strategy naturally falls out of the error equation, allowing the internal model to recalibrate around any position.

The setpoint mechanism is revealed when a strategy is imposed to counteract a visuomotor rotation. A counterintuitive consequence of this process is the rise in error over time because the motor system is recalibrating around the strategic aiming location or its proxy and not the target location.

Interestingly, while there was an initial rise in endpoint error, this function eventually reversed, returning close to zero endpoint error by the end of the strategy phase for the AT group. We assume that at some point, the size of the endpoint error exceeded the participant's self-defined tolerance for errors and caused them to modify the strategy.

Unfortunately, we do not have a direct measure of strategy change. Examination of the learning profiles revealed considerable variability across individual participants Figure 4A and Figure 5. This variability likely reflects multiple sources of noise, as well as instability in the use of a strategy.

We obtained self-reports in a debriefing session at the end of the experiment. A few subjects in the AT and DAT groups reported adjusting their strategy such that they reached to a location between the cued target and aiming target, or that shifted to reach straight to the cued target.

At a minimum, multiple processes are required to capture this nonmonotonic learning function. In our initial modeling efforts, we fixed the strategy for the entire training process. Under this assumption, the system should exhibit drift that is equal in size to the rotation, an effect never observed.

Thus, the final version of our model is a variant of a two-rate state space model [28] , but with the two rates reflecting different error sources. As described above, adaptation of an implicit model is driven by the aiming error.

In contrast, the strategy is adjusted on a trial-by-trial basis as a function of the current target error. Target errors are initially quite small and, thus have little effect on performance. However, as the target errors become large due to adaptation of the internal model, adjustments in the strategy are required to improve endpoint accuracy.

Aiming to a new location resets the recalibration around a new setpoint. To reach a stable state, participants would need to progressively adjust their strategic aiming location to a point where aiming error and target error cancel each other out.

It is reasonable to assume that our manipulation of the availability of the aiming locations influenced the degree of certainty associated with the desired aiming location. When certainty is reduced, adaptation arising from the aiming error signal is slower, and in our two-process model, the level of adaptation achieved by the motor system is lowered.

Moreover, the model does not predict that drift will reach 45°. The strategy is adjusted, reaching a point where it offsets the drift arising from adaptation of the internal model. The interplay of these two processes is complex Figure 6D. With both occurring continuously during training, the system reaches a pseudo-equilibrium state at which additional changes to both processes becomes relatively small.

Linking the strategy adjustment to the target error signal offers a process-based approach to capture flexibility in strategy use. Our setpoint model captures this through the strategy adjustment parameter F , a weighting term on target error.

The NoAT group appears to give more weight to target error than the AT and DAT group. Interestingly, the modeling results indicate that the AT group showed more utilization of the target errors than the DAT group. We assume this arises because the AT group eventually offset the relatively large drift to restore on-target accuracy.

In contrast, the DAT group never corrected for drift, suggesting that the weight given to target errors for this group was nearly zero. It is important to highlight one difference in how we conceptualize changes in the rate of strategy adjustment F compared to changes in the rate of adaptation B.

Adjustments in a strategy can occur on very fast timescale; for example, once instructed, participants were able to immediately offset the full rotation. Variation in F refers to the rate at which participants change where to aim. In contrast, B reflects a gradual process, reflecting the rate of change in a system designed to reach a desired location.

In many sensorimotor adaptation tasks, variable learning rates are used to model the substantial variability observed in individual learning curves.

In a similar manner, our setpoint model captures individual differences in strategy utilization by varying the strategy adjustment rate F. Split-belt adaptation has a notable after-effect period limbs driven at the same speed in which the interlimb coordination pattern remains altered from that during the pre-adaptation period for some time after the split-belt perturbation period.

The after-effect, however, is context-dependent and therefore, will only exist in the same locomotor environment in which the adaptation had occurred. Moreover, split-belt adaptation has spatial placement of the limb and temporal timing of limb movement components that are dissociable at the behavioral and circuit level.

The adaptation rates of the two components are different where the adaptation of the temporal component is faster than that of spatial component. In vertebrates, the cerebellum is suggested to facilitate split-belt adaptation, and in mice, the interposed cerebellar nucleus is particularly crucial for this form of adaptation.

Additionally, somatomotor regions of cerebral cortex in mice are shown to be not involved in split-belt adaptation. The split-belt adaptation paradigm is clinically important for aiding in the adjustment or recovery of impaired limb coordination patterns resulting from injury or pathologies , as well as understanding the specific aspects e.

temporal or spatial components of gait that are disrupted in gait pathologies. As demonstrated in the chart, when the environmental forces are removed, the subject reserves, for a limited time, the adaptive movement pattern stage 4. This motor after-effect demonstrates that the learner does not merely react to environmental changes but also anticipates the expected dynamics of the new environment and moves according to a new set of expectations.

Therefore, motor adaptation appears to rely on an update in the internal representation internal model of the external environment.

The after-effects phenomena suggests that prior to the movement, the CNS generates an internal-model , a sort of internal-map that guides the body in the course of the movement, and adapt to environmental forces. This observation suggests that in programming the motor output to the muscles of the arm, the CNS uses an internal model Wolpert et al.

Using optogenetics the study, done by Dr. Mackenzie Mathis at Harvard University, using mice could also show that somatosensory cortex is involved in updating the internal model.

Contents move to sidebar hide. Article Talk. Read Edit View history. Tools Tools. What links here Related changes Upload file Special pages Permanent link Page information Cite this page Get shortened URL Download QR code Wikidata item.

Under this assumption, the system should exhibit drift that is equal in size to the rotation, an effect never observed. Thus, the final version of our model is a variant of a two-rate state space model [28] , but with the two rates reflecting different error sources.

As described above, adaptation of an implicit model is driven by the aiming error. In contrast, the strategy is adjusted on a trial-by-trial basis as a function of the current target error.

Target errors are initially quite small and, thus have little effect on performance. However, as the target errors become large due to adaptation of the internal model, adjustments in the strategy are required to improve endpoint accuracy.

Aiming to a new location resets the recalibration around a new setpoint. To reach a stable state, participants would need to progressively adjust their strategic aiming location to a point where aiming error and target error cancel each other out. It is reasonable to assume that our manipulation of the availability of the aiming locations influenced the degree of certainty associated with the desired aiming location.

When certainty is reduced, adaptation arising from the aiming error signal is slower, and in our two-process model, the level of adaptation achieved by the motor system is lowered. Moreover, the model does not predict that drift will reach 45°. The strategy is adjusted, reaching a point where it offsets the drift arising from adaptation of the internal model.

The interplay of these two processes is complex Figure 6D. With both occurring continuously during training, the system reaches a pseudo-equilibrium state at which additional changes to both processes becomes relatively small. Linking the strategy adjustment to the target error signal offers a process-based approach to capture flexibility in strategy use.

Our setpoint model captures this through the strategy adjustment parameter F , a weighting term on target error. The NoAT group appears to give more weight to target error than the AT and DAT group.

Interestingly, the modeling results indicate that the AT group showed more utilization of the target errors than the DAT group. We assume this arises because the AT group eventually offset the relatively large drift to restore on-target accuracy.

In contrast, the DAT group never corrected for drift, suggesting that the weight given to target errors for this group was nearly zero. It is important to highlight one difference in how we conceptualize changes in the rate of strategy adjustment F compared to changes in the rate of adaptation B.

Adjustments in a strategy can occur on very fast timescale; for example, once instructed, participants were able to immediately offset the full rotation. Variation in F refers to the rate at which participants change where to aim. In contrast, B reflects a gradual process, reflecting the rate of change in a system designed to reach a desired location.

In many sensorimotor adaptation tasks, variable learning rates are used to model the substantial variability observed in individual learning curves.

In a similar manner, our setpoint model captures individual differences in strategy utilization by varying the strategy adjustment rate F. Nonetheless, this formulation does not adequately capture the full range of behavior observed in the current study. In particular, this approach is insufficient to account for abrupt changes in performance.

For example, the learning profile shown in Figure 5B suggests a categorical change in strategy. That is, the participant abandoned what was becoming an unacceptable strategy to search for a new strategy.

Indeed, in a post-test interview, this participant reported changing the aiming location to a position halfway between the cued target and the aiming location. An alternative approach to model strategy change could be derived from models of reinforcement learning [34] — [37].

In such models, participants explore different regions of a strategy space, attempting to quickly identify the policy that results in small target error.

In our task, a shift in policy might occur when the rise in target error due to adaptation exceeds a threshold. That is, when a chosen action fails to achieve the predicted reward, a new strategy is adopted.

This approach would provide a way to fit the data of the few participants who exhibited categorical-like changes in performance. A reinforcement learning approach based on a discrete set of strategies is problematic with the current data set. At one extreme, one might suppose that such values could take on the locations of the aiming targets e.

At the other extreme, the set might consist of a large set of values. Choosing a sparse set of potential actions will result in more abrupt changes in performance, while choosing a finer set of potential actions will allow for more gradual changes.

Studies designed to explore reinforcement learning models generally use a limited set of choices and performance thus entails discrete shifts in behavior. In our task, reach direction spans a continuous space, and in fact, for most of our participants, the changes in performance were gradual.

Future experiments that constrain the set of potential actions and manipulate reward may be better suited for employing a reinforcement learning perspective to explore strategy change.

Qualitative changes in performance may also indicate that the participants have fundamentally changed their conceptualization of the task. For example, rather than view the task goal as one involving reaching to targets, the participant may have switched to an orientation in which the task goal involved mastering a game in which the hand is a tool [38] — [40].

By this account, the initial drift would result from the operation of implicit adaptation of an internal model of the arm as described above. However, when this drift became too large, the participant switched to treating the task as a game, with the arm now conceptualized in a manner similar to how we view a computer mouse.

Accurate performance now required learning the appropriate transformation between the movements of the tool and the task workspace.

Rather, the error signal here is the difference between the cued target location and the feedback location. An error signal of this form would not produce drift. The reconceptualization hypothesis would predict that peak drift should equal or be greater than the aftereffect.

This follows from the idea that adaptation of the internal model should cease at the time the task goal changes from reaching to tool mastery. Once the participant switches from learning about their arm to learning how to play the visuomotor game, then there the internal model would not continue to learn.

The target error gains emphasis and the aiming error falls out. As such, the aftereffect should equal the drift value or be lower if there is some time-dependent decay of the adaptation effects [41]. While this hypothesis is plausible, there are also some limitations.

First, it is important to keep in mind that in almost all adaptation studies, the only visual signals are the target location and a feedback cursor. Under such conditions, aftereffects are prominent, indicating adaptation of an internal model and not just learning a game.

One would have to assume that tool conceptualization was more pronounced in the present study because of the strategic instructions. Second, our estimate of the aftereffect is actually larger than the peak drift for two of the three groups Figure 4D.

This observation, while at odds with the reconceptualization hypothesis, is consistent with the setpoint model. In our model, the aiming error signal will continue to modify the internal model even as strategy adjustments reduce target error.

As such, the aftereffect, an estimator of implicit adaptation should be equal to or larger than peak drift. While future research will be required to explore the mechanisms of strategy change, the current study advances our understanding of the interactions that arise between explicit, strategic processes and implicit, motor adaptation.

Consistent with Mazzoni and Krakauer [9] , the results make clear that strategies should not be viewed simply as representations that can facilitate implicit learning mechanisms. Rather, implicit learning mechanisms operate with a considerable degree of autonomy and, under certain conditions, can override the influence of an explicit strategy.

Nonetheless, the benefits of strategic capabilities are also borne out in the present work. When implicit mechanisms go awry, a strategic system can confer the flexibility required to ensure task success. The study was conducted according to the principles expressed in the Declaration of Helsinki and the protocol was approved by the University's IRB.

Thirty right-handed participants with no known neurological conditions were recruited from the University of California research participation pool. All participants provided informed consent prior to the start of the experiment.

The participant was seated in front of a table with her right hand comfortably positioned on a table surface. A horizontal, back-projection screen was positioned 48 cm above the table and a mirror was placed halfway between this screen and the table surface.

The displays were presented via an overhead projector. By having the participant view the mirror, the stimuli appeared to be presented on the table surface. Movements were tracked by a 3D motion tracking system miniBIRD, Ascension Technology, Burlington, VT, USA.

A sensor was placed on the tip of the index finger, and position information was sampled at Hz. The miniBirds have an approximate spatial resolution of 0. On each trial, the participant made a horizontal reaching movement to a visually displayed target, sliding their hand along the surface of the table.

The target was defined by the appearance of a green circle at one of eight possible locations and the eight locations were separated by 45° on a virtual ring with a radius of 10 cm, centered on the starting position. The targets were not at cardinal directions, but started at Participants were instructed to move quickly and were not provided with online visual feedback during the movement.

Once the hand crossed the virtual target ring, a stationary red feedback cursor was displayed for ms. Subsequent to the feedback interval, the participant was visually guided back to the starting location.

A white circle appeared, with the diameter corresponding to the distance of the hand from the starting position. The participant was trained to move so as to reduce the diameter of this circle. When the hand was within 10 pixels 8. When this position had been maintained for ms, the next target appeared.

The target, start region, and feedback cursor were all 8 pixels 7 mm in diameter. Testing began with a familiarization block in which participant was trained to make rapid reaching movements from the start location toward the target location Figure 2A.

The participants were instructed that the task goal was to make the red feedback cursor appear as close as possible to the green target. We did not impose any constraint on movement amplitude other than that the movement had to span at least 10 cm.

In addition to emphasizing the importance of directional accuracy, the participants were trained to complete the movement within ms. The quick movements were intended to minimize any within-movement control even though there was no online visual feedback provided.

The familiarization block lasted 40 trials, five for each of the eight target locations. Following the familiarization block, participants were trained to use a 45° clockwise CW strategy.

The participants were instructed to aim to the neighboring CW blue circle Figure 2B. The feedback cursor was veridical, appearing at the position of the hand when it crossed the virtual target ring.

Thus, during this block, the participant attempted to align the feedback cursor with the blue circle that was 45° clockwise from the green target. This strategy-only block consisted of 40 trials. The strategy-only block was followed by a trial baseline block in which participants were instructed to reach directly towards the green target.

The feedback remained veridical and thus, the participant's goal was to align the feedback cursor with the green target. Following these 40 trials, a visuomotor rotation was introduced without warning.

At the end of this rotation plus strategy block, there was a brief pause so that the experimenter could instruct the participants the rotation would no longer be present and that they should resume moving directly to the cued target location.

The purpose of this short block was to quantify the aftereffects of the rotation training, while not inducing any learning based on visual errors [28].

The experiment concluded with a washout block that was identical to the baseline block. The rotation remained off and participants were reminded to continue reaching towards the green target.

The feedback cursor was again visible, now providing an error in terms of the distance between this cursor and the green target. There was a short temporal delay less than 1 minute between the blocks so that the experimenter could load the new block.

The participants were divided into three experimental groups. The only difference between the groups was the status of the blue circles, the visual landmarks that provide an aiming target during the strategy-only and rotation blocks. In this block, the blue circles were presented on half of the trials to assist the participants in learning where 45° was in relation to the cued, green target.

During the other blocks familiarization, baseline, rotation, and washout blocks , the blue circles were absent at all times.

For these participants, the blue circles were presented at the start of the trial, followed shortly by a green target at one of the locations.

The blue circles remained visible until the hand had been displaced 1 cm from the starting position approximately 30 ms into the movement at which point they disappeared. Thus, the aiming targets were not visible when the feedback cursor appeared.

As in the other conditions, the green target remained on the screen until the end of the feedback interval. We opted to use blue circles on a black background as the visual landmarks to minimize visual aftereffects for the DAT group.

Kinematic information was analyzed with Matlab MathWorks, Natick, MA. Movement duration was defined as the interval from when the hand was 1 cm from the start position until it passed through the virtual target ring 10 cm radius.

We determined the heading of the hand at the point of intersection and used this to compute the endpoint hand angle, defined as the difference between this heading and a straight line connecting the starting position and the target green circle except for the strategy-only block. When there was no rotation, the target error was identical to the endpoint hand angle.

When the rotation was present, the target error was the endpoint hand angle plus 45°. The angular endpoint error was used to infer the motor plan plus noise since the movements were made without on-line feedback and at a speed that minimized corrective movements.

Since there was a substantial difference between groups in terms of drift, we measured the aftereffect relative to the target location. For the analyses of movement accuracy, movements within each block were averaged over trial bins.

However, we did not bin the first two movements when the rotation was first introduced pre-strategy , nor did we bin the first two movements after the strategy was introduced. Rather, these two movement pairs were averaged separately to quantify error introduced by the rotation and the initial success of the participant using the strategy, respectively.

A key dependent measure in this study is the magnitude of the drift exhibited during the rotation block. Estimating peak drift is difficult, not only because of noise in performance, but also because some participants exhibited non-monotonic drift functions.

To minimize these problems, we used a boostrapping [42] method to estimate peak drift. Using the group averaged data, we created bins of 10 movements each and then identified the bin with the largest angular deviation. The group averaged data was recompiled times by randomly resampling with replacement from the participant pool.

The estimate of the time of the peak drift was chosen as the movement number in the middle of this movement bin. We used a similar method to compute the aftereffect.

Here we focused exclusively on the first 8 trials following the end of the strategy plus rotation phase, trials in which no visual feedback was provided. The bootstrapping method here produces only a slightly different estimate of the aftereffect compared to a simple averaging across the observed data from these 8 trials.

To quantify the deadaptation rate during the washout phase, we fit an exponential function [26] to the time series of target endpoint errors. Specifically, we bootstrapped the washout data from each group to provide an estimate of the exponential decay rate. We compared these rates to determine if there was a difference in the rate of deadaptation.

To statistically evaluate the results of the bootstrapping procedures, the mean statistics of each resampled iteration were calculated and then used to determine p values [29] , [42].

All statistical analyses were performed in Matlab. For the analyses that did not involve bootstrapping, we report the degrees of freedom for the F-values when performing ANOVAs across groups and t-values when performing t-tests within groups.

Occasionally participants did not move to the cued, green target on baseline and washout blocks , mistakenly implemented the strategy in the wrong direction i. We eliminated trials in which the movement heading was more than three standard deviations from the mean for that block. The Nelder—Mead method or simplex method [43] , implemented in Matlab as fminsearch, was used to fit the data from the baseline, rotation, stop-strategy, and washout blocks.

We did not fit the data from the familiarization block and strategy-only block. The value was reset to 0 at the start of the washout block. While the simplex method can be sensitive to initial conditions, we obtained similar estimates of the parameters within the confidence intervals of those parameters with different starting values with the current data sets.

Thus, the same initial conditions values of zero for all parameters were used for each participant. The goodness of fit was measured by the root mean square error rms and Pearson's correlation coefficient r between the output of the model for endpoint hand angle and the participant's endpoint hand angle.

Custom software was written to bound the parameters between 0 and 1. The sign of the parameters is dependent on the convention we used for the target errors: CCW to the target was negative and CW was positive. The equations were adjusted to make all the parameters positive.

The parameter K was bound from 0. The data from the NoAT group is more difficult to fit because of the absence of drift and relatively small aftereffect. A, B, and E, the parameters characterizing the internal model memory, adaptation gain, and strategy memory, were fit for all the groups collectively.

K the parameter characterizing the availability of the aiming target strategic aiming location and F, the influence of target errors, were estimated separately for each group through bootstrapping.

The group's averaged data was computed by resampling with replacement the participant pool, repeating this times, and fitting the setpoint model Eqns 8—11 to each resampled average.

We would like to thank Azeen Ghorayshi for help with data collection. Thanks to John Schlerf for setting up the experimental equipment, Arne Ridderikhoff for help with data analysis, and Greg Wojaczynski for helpful comments. We are grateful to John Krakauer for his many comments throughout the course of this project and for suggesting the reconceptualization hypothesis.

Conceived and designed the experiments: JAT RBI. Performed the experiments: JAT. Analyzed the data: JAT. Wrote the paper: JAT RBI.

Article Authors Metrics Comments Media Coverage Reader Comments Figures. Abstract Visuomotor rotation tasks have proven to be a powerful tool to study adaptation of the motor system. Author Summary Motor learning has been modeled as an implicit process in which an error, signaling the difference between the predicted and actual outcome is used to modify a model of the actor-environment interaction.

Introduction When learning a new motor skill, verbal instruction often proves useful to hasten the learning process. Results Current models of sensorimotor adaptation have not addressed the effect of explicit strategies. Download: PPT. Modeling strategy use during visuomotor adaptation When informed of an appropriate strategy that will compensate for the rotation, participants immediately counteract the rotation and show on-target accuracy.

To allow immediate implementation of the strategy, we postulate that there is direct feedthrough of the strategy s to the target error equation equation 1 : 3 Direct feedthrough allows the strategy to contribute to the target error equation without directly influencing the updating of the internal model.

Figure 3. Group averaged endpoint error relative to the target for the three groups. Figure 4. Time course of drift and aftereffect, and the relationship of drift to target location and aftereffect. Figure 5.

Performance during the rotation block of three participants. The effect of aiming target availability The availability or certainty in the estimate of the aiming location was manipulated by altering the presence of the aiming target across the groups.

Strategy adjustment based on performance error As noted above, none of the participants showed drift approaching 45°. Table 1. Modeling results for each group based on the setpoint model Eqns 8— Relationship between drift and aftereffect In the setpoint model, the internal model will continue to adapt even in the face of strategic adjustments adopted to improve endpoint accuracy.

Discussion Behavioral summary Visuomotor rotation tasks are well-suited to explore how explicit cognitive strategies influence sensorimotor adaptation.

Incorporating a strategy into state-space models Mathematical models of sensorimotor adaptation have not explicitly addressed how a strategy influences learning and performance. Two sources of errors Our model entails two types of error signals: an aiming prediction error between the feedback location and aiming location, and performance error between the feedback location and the target location Figure 2E.

Alternative models of strategy change In many sensorimotor adaptation tasks, variable learning rates are used to model the substantial variability observed in individual learning curves.

Methods Ethical statement The study was conducted according to the principles expressed in the Declaration of Helsinki and the protocol was approved by the University's IRB. Experimental apparatus and procedures The participant was seated in front of a table with her right hand comfortably positioned on a table surface.

Movement analysis Kinematic information was analyzed with Matlab MathWorks, Natick, MA. Modeling The Nelder—Mead method or simplex method [43] , implemented in Matlab as fminsearch, was used to fit the data from the baseline, rotation, stop-strategy, and washout blocks.

Acknowledgments We would like to thank Azeen Ghorayshi for help with data collection. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This work was supported by grants from the McKnight Endowment for Neuroscience and the Alfred P. Sloan Foundation to M. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: The authors have declared that no competing interests exist.

When learning to swim, the proper stroke motion is usually taught on the pool deck. Although a student might seem to have mastered this motion on dry land, upon entering the water she will have difficulty in accurately reproducing it underwater.

However, after many laps, the student eventually learns to produce the pattern of motor output that leads to the proper stroke motion while swimming. This learning occurs via the formation of internal models of the physical dynamics experienced which allow the programming of movement to contend with the dynamics of the environment [1] — [4].

These internal models have been shown to predict the dynamics of the environment as a function of motion rather than as a function of time [5] — [8] — a strategy that makes sense in light of the viscoelastic and inertial physics of our own limbs and the objects we interact with.

Consequently, the neural plasticity which underlies this learning must establish associations between motion state i. Although the existence of these associations has been well established, the mechanism by which they form is not yet understood.

How does this state-dependent learning arise during the course of motor adaptation? One possibility is that on individual trials, an internal model of the environment is updated based on a combination of the errors experienced and the motion plans that led to those errors.

Another possibility is that internal models are updated based on errors experienced in combination with the actual motion states associated with those errors. It is remarkable that previous work on motor learning in neural systems has widely assumed the former [4] , [9] — [16] , despite the fact that direct evidence for this hypothesis is scant.

The idea that learning is associated with the motion that was planned plan-referenced learning is especially pervasive in the learning rules of the algorithms that have been proposed to model the process of adaptation in the neuromotor learning literature [4] , [9] , [11] — [12] , [15] , however it is difficult to find work that addresses the validity of this assumption, explores its implications or provides a clear rationale for its use.

The machine learning community has developed, in parallel, a series of algorithms for updating internal models in robotic systems.

Interestingly, these algorithms almost uniformly involve learning rules in which internal models are updated based on a combination of the errors experienced and the actual motion associated with those errors motion-referenced learning rather than the motions that were planned [17] — [21].

The choice of these learning rules is grounded in the idea that adaptive changes should be provably stable in the sense that, under a set of reasonable assumptions, updated internal models should never result in worse performance [17] — [21].

Here we ask the question: Do the associations between motor output and motion state formed during human motor learning arise from adaptation based on planned or actual motions? The answer to this question is important not only for theories of motor control, and issues of stability during learning, but also because knowledge of how associations are formed during motor learning can be leveraged to improve the efficiency of training procedures.

Motor adaptation can be described as the process of tuning motor output to reduce the errors between plan and action. Thus the associations between motion state and motor output formed during this process result from the way that responsibility for these errors is assigned. This is known as a credit assignment problem.

This problem can be posited as the task of assigning blame after an error is experienced to the set of actions that would be most likely to give rise to similar errors in the future. This set of actions could then be modified in order to improve performance in subsequent trials.

Viewed in this way, the distinction between plan-referenced learning PRL and motion-referenced learning MRL corresponds to whether the blame for motor errors should be assigned to the planned versus actual motion.

Consequently, the amount of adaptation on a given trial will be determined by the magnitude of the error, however the location of the adaptation which future motions will benefit from the adaptation will be determined by the credit assignment mechanism. Here we studied the generalization of motor adaptation to untrained conditions in order to elucidate the credit assignment mechanism used by the CNS, and then used our understanding of this mechanism to design a training paradigm that takes advantage of it to improve the efficiency of motor adaptation.

The adaptations that would occur at different stages of training for reaching arm movements in a velocity-dependent force-field FF for the PRL and MRL credit assignment hypotheses are shown in Figure 1. The green shaded region around the planned motion — which is essentially straight toward the target for short 10 cm movements [22] — represents the space of future motions which would benefit from the adaptation to the greatest degree under PRL Figure 1A.

Alternatively, each red shaded region represents the space of future motions which would benefit maximally under MRL. A more direct visualization of the adaptive changes predicted by each credit assignment hypothesis can be made by representing motion and the resulting adaptation in velocity-space rather than position-space, since the adaptation to the velocity-dependent dynamics studied in the current series of experiments is believed to be mediated by an internal model largely composed of velocity-dependent motor primitives [8] , [10] , [12] — [13] , [23].

These primitives are the learning elements which contribute to the compensatory motor output i. Figure 1B shows how individual motor primitives would adapt based on PRL versus MRL credit assignment early on in training. Here each circle represents a single motor primitive centered at its preferred velocity with a color intensity denoting the amount of adaptation that would arise from the illustrated trial.

The left and right panels of Figure 1B show the adaptations predicted by PRL green and MRL red , respectively. As in Figure 1A , adaptation is centered on the planned motion for PRL and centered on the actual motion for MRL. As training proceeds over the course of several trials, the activation levels of the adapted primitives would continue to increase.

This continued increase in activation not illustrated leads to increased compensatory force, resulting in greater compensation of the external dynamics and thus straighter trajectories. Note that the adapted primitives would be noticeably different for the two credit assignment hypotheses early in training, but would overlap late in training as force compensation increases and the planned and actual motions converge as illustrated in Figure 1A.

a Illustration of planned green dashed line and actual solid red line trajectories for early left and late right movements during adaptation to a velocity-dependent curl FF grey arrows.

Plan-referenced learning PRL would lead to adaptation associated with the planned motion green dashed line. In contrast, motion-referenced learning MRL would lead to adaptation associated with the actual motion solid red line.

The green- and red-shaded regions represent the space of motions that would experience the greatest amount of adaptation under PRL and MRL, respectively.

b Illustration of the adaptation of velocity-dependent motor primitives under PRL and MRL for early training. The preferred velocities centers of these motor primitives are tiled across velocity space as shown. Note that the planned and actual arm motions green dashed line and red solid line are replotted in velocity space here.

The interior of the circle representing each motor primitive is colored with an intensity proportional to the activation induced by the adaptation resulting from the illustrated trial. Under PRL left panel this activation is greatest for motor primitives which neighbor the motion plan in velocity space green shading , whereas under MRL this activation is greatest for motor primitives that neighbor the actual motion red shading.

Given the different implications that the PRL and MRL credit assignment mechanisms have for motor adaptation, we can assess which one is favored by the CNS by asking a simple question: After training, which motions gain the most benefit from the induced adaptation?

The motions that were planned or the motions that were experienced? Since the mechanism for credit assignment determines which motions will benefit from adaptation on a particular trial, we studied how motor adaptation to a single target direction generalizes to neighboring motion directions.

If a particular motion is trained, the pattern of generalization can be viewed as a record of the history of credit-assignment for the errors experienced during a training period. Specifically, the amount of generalization in the directions neighboring the trained movement constitutes the set of actions that the motor system believes should be adapted based on the history of errors experienced.

Therefore, PRL and MRL should give rise to different patterns of generalization. In order to cleanly distinguish between these hypotheses, we designed an experiment in which the planned motion and the actual motion were maintained to be distinct from one another during the entire dataset so that the patterns of generalization predicted by PRL vs.

MRL would be very different from one another. This is a challenge because, training a motor adaptation generally results in improved performance such that the actual motion converges onto the planned motion, and such a scenario could hamper the ability to clearly distinguish between the PRL and MRL hypotheses.

Thus, we designed an experiment in which actual motion would not converge onto planned motion during the course of training, resulting in enduring differences between the predictions of these two hypotheses.

To accomplish this, subjects were exposed to a training period consisting of short, successive blocks of movements towards a single target location with a force-field FF that alternated between clockwise CW and counterclockwise CCW directions from block to block see Figure 2A.

In these FFs, the peak force perturbations were 2. The FF blocks were short enough 7±2 trials that neither the CW nor the CCW FF could be learned very well before unlearning with the opposite FF occurred.

After subjects were exposed to a number of these interfering FF cycles, we measured the generalization of adaptation to untrained movement directions with error-clamp EC trials see Materials and Methods for details. a Experiment schematic. After a baseline period where subjects performed movements in nine different directions, subjects received training for a single target location the central one with alternating blocks of 7±2 force-field trials in CW blue and CCW orange FFs as illustrated.

After training, generalization of the force-field compensation was tested along the nine original directions practiced during the baseline period see Materials and Methods for details.

b Credit assignment predictions. If the motor primitives that are adapted during training are centered at the desired movement direction — as specified by PRL — the exposure to opposite force-fields would lead to opposite generalization patterns for the CW and CCW FFs orange vs.

blue in the top panel that would essentially cancel one another leading to a near-zero net generalization pattern green dashed line. In contrast, if the motor primitives that are adapted during training are centered at the actual movement directions — as specified by MRL — the exposure to the CW and CCW force-fields would lead to individual generalization patterns for these FFs that are misaligned orange vs.

blue in the bottom panel. The sum of these misaligned generalization patterns would result in a bimodal generalization pattern red dashed line.

c Experimental results. The error bars represent standard errors. The predictions of PRL and MRL are strikingly different for this experiment. For the PRL hypothesis, since the adaptation is associated with motor primitives centered at the same target direction for both FFs Figure 2B top panel, blue and orange traces , the balanced exposure to these opposite FFs would lead to cancellation of the CW and CCW FF learning resulting in near zero adaptation at the trained target direction and the adjacent directions Figure 2B , dashed green trace.

Note that although target locations are identical between CW and CCW FF trials, the actual movement directions differ. The CW FF perturbs motion towards smaller movement angles whereas the CCW FF does the opposite. Therefore, MRL predicts that smaller movement angles would be preferentially associated with adaptation appropriate for the CW FF blue trace in the bottom panel of Figure 2B , whereas higher movement angles would be preferentially associated with adaptation appropriate for the CCW FF orange trace in the bottom panel of Figure 2B.

This would lead to the bimodal pattern of generalization illustrated in Figure 2B red dashed trace. We trained one group of subjects in this FF interference paradigm at a target location of °.

We found that target directions smaller than the training direction consistently display generalization appropriate for the CW FF negative whereas target directions greater than the training direction display generalization appropriate for the CCW FF positive. These results provide direct evidence for MRL by matching the complex pattern of generalization predicted by it.

In our experiment we balanced the direction of the FF that was presented before testing generalization, nevertheless, we noticed a small bias in the generalization function at the training direction consistent with a bias in adaptation level that we observed during the training period see Figure S1 and Text S1.

This bias is compatible with other results showing somewhat faster learning for a CW FF [8]. In order to eliminate the possibility that this bias or the target location we chose for training ° might have somehow contributed to the generalization pattern we observed in the data, we trained a second group of subjects in a version of this experiment that was designed to eliminate the bias and provide training at another target location 60°.

We eliminated the bias by unbalancing the number of CW versus CCW FF trials in each cycle in this second group of subjects see Text S1. Together, these results provide compelling evidence for MRL as the mechanism for credit assignment in motor adaptation. We note that Equations 3 and 4 used for our simulations incorporate local motor primitives that are functions of the initial movement direction θ rather than of the full time series of the velocity vectors encountered during each trial.

This might seem an inappropriate choice since, as we discussed above, velocity-dependent motor primitives are thought to underlie the learning of velocity-dependent dynamics [8] , [10] , [12] — [13] , [23]. However this approximation is a good one when movements are approximately straight, which is essentially the case for the first ms of the movements considered in our study.

This approximation, of course, breaks down at the end of the movement when the initial movement direction no longer describes the velocities experienced. However, the amplitudes of the velocity vectors during the end-movement correction are quite low and so the unmodeled spread of learning to the actual motion experienced in this correction phase should have relatively little effect since at low velocities, viscous dynamics have small consequences.

This effect can be visualized in the left panel of Figure 1B which shows that the end-movement correction which has a velocity vector that points to the second quadrant would only excite velocity-dependent primitives near the origin under MRL.

Note that the separation of the peaks in the bimodal generalization pattern predicted by MRL red dashed line in Figure 2C results from the size of the errors experienced during training. Consequently, larger force-field perturbations which induce larger errors would result in greater separation between the peaks.

However, the separation between the peaks about 60° is predicted to be greater than the separation between the average errors experienced in the two force-fields about 25°. There are two reasons for this.

The first is that more adaptation occurs on trials with larger errors than those with smaller errors, skewing the center of adaptation for each force-field outwardly from the mean experienced error. The second reason is illustrated in the lower panel of Figure 2B : When the patterns of generalization for the positive and negative force-fields are summed, resulting in a bimodal generalization pattern for MRL, the peaks of this bimodal generalization pattern red are separated by an even greater distance than the peaks of the positive orange and negative blue components because the amount of cancellation between these components is greater at movement directions corresponding to smaller rather than larger errors resulting in further outward skew.

Previous work has attempted to measure the generalization functions GFs associated with learning a single FF. MRL predicts that these GFs will be shifted toward the motion directions experienced during training.

Many of these studies have estimated GFs from complex datasets using a system identification framework [10] , [12] — [13]. However the implementation of this framework assumed PRL in these studies, thus preventing a straightforward interpretation of their results. In one study [24] a simpler generalization experiment was conducted, in which subjects were trained with a single FF to a single target location, after which the resulting GF was measured.

Because the actual motions approached the planned motions late in training, the shifts predicted by MRL would be subtle.

Furthermore, the ability to detect shifts in the generalization function was hampered by a coarse sampling of the generalization function 45°. Nevertheless, careful inspection of these GFs consistently reveals subtle shifts towards the motions experienced during training as predicted by MRL.

However, it is difficult to be certain whether if the shifts observed in this study result from MRL rather than innate biases in generalization functions because only a single FF direction was studied. Innate biases might stem from biomechanical asymmetries or direction-related biases in adaptation.

We therefore performed a pair of single-target, single-FF experiments in order to compare the shifts in generalization associated with opposite FFs. The results of these experiments confirm the existence of subtle but significant shifts in generalization [25].

The magnitudes and the directions of these shifts are consistent with the MRL hypothesis [25]. Insights into the mechanisms for learning in the CNS can provide a platform for creating training procedures that leverage these insights to improve the rate of learning — an important goal for both motor skill training and neurologic rehabilitation.

With our new understanding of how the CNS solves the credit assignment problem, we looked into the possibility of designing a novel training paradigm to take advantage of this knowledge. A key consequence of plan-referenced learning is that this mechanism for credit assignment would result in a match between what is learned and what is commanded on the next trial if the same motion plan is repeated from one trial to the next during training — like when aiming a dart at the bull's eye repeatedly.

In contrast, motion-referenced learning would result in a mismatch. Motion-referenced learning, therefore, predicts that the process of training an accurate movement to a given target location in a novel dynamic environment would be inefficient if that target were repeatedly presented at the same location during training single-target training, STT as illustrated in Figure 3.

This inefficiency arises because the motion experienced during training does not coincide with the motion that is to be learned, resulting in limited overlap between the motion-referenced learning that occurs and the learning that is desired. Single-target training STT : a single target location is presented during the training period.

The PRL hypothesis predicts alignment of credit assignment across trials for STT, whereas MRL predicts misalignment. Left-shifted training LST : targets are initially presented leftward of the desired learning direction and are brought closer to it as training progresses so that the actual motion matches the desired learning direction throughout the training period.

The MRL hypothesis predicts alignment of credit assignment across trials for LST, whereas PRL predicts misalignment. Correspondingly, PRL predicts that STT will yield the greatest learning whereas MRL predicts that LST will yield the greatest learning.

Right-shifted training RST : the training targets are presented in a sequence that mirrors LST. Both the PRL and MRL hypotheses predict misalignment for RST. However, PRL predicts an identical amount of misalignment for LST and RST, whereas MRL predicts much greater misalignment for RST than LST.

Note that CW FF training is illustrated in all panels. The aforementioned inefficiency can be ameliorated by a paradigm which continually changes the locations of the targets presented during the training period as shown in Figure 3 , second column.

In this training paradigm, target directions would be shifted from one trial to the next so that the actual motion experienced repeatedly lines up with the motion to be learned.

For the CW FF depicted in Figure 3 , this corresponds to left-shifted training LST. Initial target locations are placed with large leftward shifts with respect to the desired learning direction — in anticipation of the large rightward initial errors with respect to the target location.

These leftward target shifts are then gradually reduced as learning proceeds and errors become smaller, in order to maintain alignment between the actual motion experienced and the movement to be learned. The MRL hypothesis predicts that the LST training paradigm should produce faster learning than the standard STT paradigm used in previous motor adaptation studies in which a single target direction was trained [24] , [26].

We tested this idea by comparing the learning curves associated with these training paradigms for adaptation to a clockwise viscous curl force-field.

A different group of subjects was studied on each paradigm to avoid the effects of savings [27] — [29]. As a control for a possible increase in attention associated with changing target locations in the LST paradigm, we tested a third group of subjects with a right-shifted training RST paradigm.

Here targets were shifted to the right, mirroring the target positions in the LST paradigm. The MRL hypothesis would predict slower learning for RST than STT or LST because right-shifted targets in a rightward pushing force-field would result in reaching movements even farther away from the desired learning direction than those expected in STT see Figure 3 , third column.

In contrast the PRL hypothesis would predict fastest learning for the STT paradigm and identical learning rates for the LST and RST paradigms because the STT paradigm creates perfect alignment between the desired learning and the planned motion whereas the LST and RST paradigms create misalignments between the desired learning direction and planned motion that are opposite in direction but equal in magnitude.

We used a FF magnitude of We first collected data from a subset of subjects in the STT paradigm in order to estimate the evolution of directional errors across trials.

We used this pattern of directional errors to determine the target shifts that would produce good alignment between experienced motion and desired learning direction for the LST paradigm see Materials and Methods.

As shown in Figure 4A , we obtained a good match between motion direction and the desired learning direction 90° throughout the training period for the LST paradigm, so that misalignment between these directions was dramatically reduced compared to the STT paradigm.

Correspondingly, the misalignment between motion direction and the desired learning direction was about twice as great for RST than for STT.

a Characterization of the STT, LST and RST training paradigms. Target directions dashed and actual movement directions solid during the training period are plotted against trial number.

Note that the LST paradigm achieves actual movement directions that are much more closely aligned with 90° than the other two paradigms. b Simulations of motor adaptation based on the PRL and MRL hypotheses for the three training paradigms.

The darkened dots at 90° indicate the desired learning direction and the coloring indicates the amount of adaptation predicted. Note that PRL predicts optimal alignment with STT while MRL predicts optimal alignment with LST. c and d Predicted learning at 90° for the PRL and MRL hypotheses.

Note that these traces represent slices at 90° through the 3-D plots in panel b , corresponding to the darkened dots. e Experimental results for all three training paradigms.

Note that over the first 10 trials, the LST paradigm produces the highest adaptation levels, and RST the lowest, as predicted by MRL.

The plots shown in Figure 4B illustrate how the adaptation patterns predicted by MRL and PRL would evolve as training proceeds for the training paradigms discussed above.

Note that adaptation spreads across a limited range of movement directions consistent with local generalization [24] — [26] , but the alignment between adaptation and the desired learning direction 90° varies from one paradigm to another STT vs.

LST vs. RST , and from one credit assignment hypothesis to another PRL vs. The darkened dots which highlight a slice through these plots at 90° illustrate the amount of adaptation associated with the desired learning direction.

These simulations show that the PRL hypothesis predicts that in the STT paradigm, credit assignment will be perfectly aligned with the desired learning direction 90° throughout training. PRL also predicts an equal but opposite pattern of misalignments between credit assignment and desired learning for the LST and RST paradigms Figure 4B.

These misalignments are initially large but become attenuated during the course of the training because planned and actual motions converge. This results in simulated learning rates that are highest for the STT paradigm and lower, but identical, for the LST and RST paradigms under PRL Figure 4B—C.

In contrast, the simulations for the MRL hypothesis show perfect alignment between the credit assignment and the desired learning direction for the LST paradigm. For STT, the MRL-based simulations show a gross misalignment between the credit assignment and the training direction.

An implicit plan overrides an explicit strategy during visuomotor adaptation. An explicit strategy prevails when the cerebellum fails to compute movement errors. Cerebellum 9 , — Article PubMed PubMed Central Google Scholar.

Benson, B. A spatial explicit strategy reduces error but interferes with sensorimotor adaptation. Bond, K. Flexible explicit but rigid implicit learning in a visuomotor adaptation task.

Krakauer, J. Learning of visuomotor transformations for vectorial planning of reaching trajectories. in Progress in Motor Control Vol. Sternad, D. Brayanov, J. Motor memory is encoded as a gain-field combination of intrinsic and extrinsic action representations.

Wu, H. The generalization of visuomotor learning to untrained movements and movement sequences based on movement vector and goal location remapping. Von Helmholtz, H.

Handbuch der Physiologischen Optik Vol. Held, R. Plasticity in human sensorimotor control. Science , — Shadmehr, R. Adaptive representation of dynamics during learning of a motor task.

Smith, M. Article PubMed Google Scholar. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. Article PubMed PubMed Central CAS Google Scholar. Joiner, W. Long-term retention explained by a model of short-term learning in the adaptive control of reaching.

Wagner, M. Shared internal models for feedforward and feedback control. Sing, G. Primitives for motor adaptation reflect correlated neural tuning to position and velocity. Neuron 64 , — Kagerer, F.

Adaptation to gradual as compared with sudden visuo-motor distortions. Brain Res. Trial-by-trial analysis of intermanual transfer during visuomotor adaptation. Alhussein, L. Dissociating effects of error size, training duration, and amount of adaptation on the ability to retain motor memories.

Flexible cognitive strategies during motor learning. PLoS Comput. Jakobson, L. Hudson, T. Measuring adaptation with a sinusoidal perturbation function. Methods , 48—58 Kettner, R. et al. Prediction of complex two-dimensional trajectories by a cerebellar model of smooth pursuit eye movement.

Stein, C. Structural equation modeling. Methods Mol. de Marco, G. Principle of structural equation modeling for exploring functional interactivity within a putative network of interconnected brain areas.

Imaging 27 , 1—12 Harris, C. Signal-dependent noise determines motor planning. Nature , — Jones, K. Sources of signal-dependent noise during isometric force production. Error correction, sensory prediction, and adaptation in motor control. Human sensorimotor learning: adaptation, skill, and beyond.

Hadjiosif, A. Cerebellar damage reduces the stability of motor memories. in Proc. pdf Haith, A. The influence of movement preparation time on the expression of visuomotor learning and savings. Fitts, P.

Anderson, J. Acquisition of cognitive skill. Beilock, S. Expertise, attention, and memory in sensorimotor skill execution: impact of novel task constraints on dual-task performance and episodic memory. A 55 , — Gray, R. Attending to the execution of a complex sensorimotor skill: expertise differences, choking, and slumps.

Jackson, R. Attentional focus, dispositional reinvestment, and skilled motor performance under pressure. Sport Exerc. Flegal, K. Poldrack, R.

Interactive memory systems in the human brain. Competition among multiple memory systems: converging evidence from animal and human brain studies.

Neuropsychologia 41 , — Packard, M. Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems.

Boyd, L.