Sensory prediction errors, not performance errors, update memories in visuomotor adaptation

Sensory prediction errors are thought to update memories in motor adaptation, but the role of performance errors is largely unknown. To dissociate these errors, we manipulated visual feedback during fast shooting movements under visuomotor rotation. Participants were instructed to strategically correct for performance errors by shooting to a neighboring target in one of four conditions: following the movement onset, the main target, the neighboring target, both targets, or none of the targets disappeared. Participants in all conditions experienced a drift away from the main target following the strategy. In conditions where the main target was shown, participants often tried to minimize performance errors caused by the drift by generating corrective movements. However, despite differences in performance during adaptation between conditions, memory decay in a delayed washout block was indistinguishable between conditions. Our results thus suggest that, in visuomotor adaptation, sensory predictions errors, but not performance errors, update the slow, temporally stable, component of motor memory.

What is the mechanism that enables adaptation in response to a sensory perturbation? A long-held theoretical view is that sensory-motor adaptation is due to the update of internal forward models 1,2 , which are neural processes that predict the sensory consequences of motor commands [3][4][5] . According to this view, when a sensory perturbation such as a visuomotor rotation alters the sensory outcome, the forward model is updated to minimize the sensory prediction error (SPE), that is, the error between the sensory outcome and its prediction 6 . Forward model update can happen continuously during daily activities via the individual's own movements ("self-supervised learning" 3,7 ). The output from the updated forward model can then be used to compute the motor command needed to compensate for the perturbation when a task goal is defined 8 .
Thus, in theory, SPEs alone in the absence of a task goal are sufficient for motor adaptation via forward model update. Decisively, movements during adaptation do not need to be goal-directed because only the difference between actual and predicted sensory consequences matters in the update of the forward model 9,10 . However, in most real-life activities and in most behavioural experiments, performance errors or rewards, whether explicit (e.g., score, evaluation) or implicit (e.g., self-evaluation), related to the goal are available. Because reward-related signals have been shown to influence motor adaptation 8,[11][12][13][14] , both SPEs and performance errors could simultaneously play a role in adaptation.
A seminal experiment by Mazzoni and Krakauer 15 suggested that SPEs in the absence of initial performance errors lead to a drift in hand direction. In this experiment, participants made movements to one of eight targets around a circle, spaced 45 degrees apart. A 45-degree visuomotor perturbation was then suddenly introduced, creating large performance errors between the target and the cursor. After the second adaptation trial, participants were told that shooting at the neighboring target placed clockwise of the main target would cancel the error. Indeed, in the trials following the introduction of this strategy, the performance error between the main target and the cursor was approximately zero. However, a drift appeared in later trials, with the cursor starting to rotate clockwise from the main target. After about 70 trials, participants were told to stop using the strategy and aim directly at the main target again. The following trials showed strong after-effects. These results suggest that the SPE, but not the performance error between the cursor and the main target, is involved in adaptation via updates of an internal model. This is because just after the strategy is introduced, the drift appears despite zero performance error. However, this experiment did not allow for a disambiguation of the roles of SPEs versus performance errors on updating motor memories, because both types of errors were present in the drift phase. An extension of this initial study by Taylor and Ivry 16 with a larger number of adaptation trials showed that the performance error between the cursor and the main target that appears during the drift phase influences performance during adaptation: when the number of trials increased, participants started to generate corrective movements to cancel the drift. In addition, a second performance error, the error between the cursor and the neighboring target, which corresponds to the aiming error, could also influence adaptation. Note that this second performance error would initially act on the drift in the same direction as the SPEs. We call the first type of performance error, the error between the main target and the cursor, PE1 (See Fig. 1A). We call the second type of performance error, the error between the neighboring target and the cursor, PE2 (See Fig. 1A).
Here, we extended the study by Taylor and Ivry 16 in two ways: First, we performed a two-by-two design experiment to distinguish the possible influences of SPE, PE1, and PE2 on adaptation: the main target, the neighboring target, both targets, or none disappear following the movement onset (Fig. 1A). When no targets disappear (Condition 1), the learner can rely on PE1, PE2, and SPE. When the main target disappears (Condition 2), the learner can only reliably rely on PE2 and SPE. When the neighboring target disappears (Condition 3), the learner can only reliably rely on PE1 and SPE. When both the main and the neighboring target disappear (Condition 4), estimation of both PE1 and PE2 becomes unreliable, and if adaptation occurs, we hypothesized that it is driven by SPE. Second, we tested after-effects in a delayed washout block, given after a no-feedback movement block in all four conditions, to probe whether the slow component of memory 17,18 is updated via performance errors or SPEs.
Our results show that participants in the two conditions with PE1 performance errors clearly improved performance during adaptation by reducing the magnitude of the drift, and exhibited smaller drifts than participants (C) Strategy given to the participants to counteract the perturbation: participants were told to shoot to the neighboring target in order to reach the main target. (D) The four experimental conditions. In Condition 1, both the main and the neighboring target remained on the screen following the shooting movement. In Conditions 2 and 3, the main target or the neighboring target disappears as the hand was at the 1/3 rd distance between the home position and the target circles, respectively. In Condition 4, both targets disappeared. (E) Experiment schedule common to all conditions. The explicit strategy was given at time ➀, and the participants were asked to stop using the strategy from time ➁. in the two conditions without PE1. However, in delayed washout, both the amplitude and the decay were not distinguishable in the four different conditions, despite differences in performance during adaptation. Thus, our results support the view that the slow, temporally stable, component of visuomotor adaptation in arm movements is updated by SPEs, not performance errors.

Materials and Methods
Detailed Experimental methods. Participants. Fifty-two young and right-handed participants (23.4 ± 0.6 years old, 35 females; all results are reported as mean ± standard errors) with no history of neurological disorders participated in this study. We randomly assigned the participants to one of four conditions (N = 13 per condition). All participants signed an informed consent form and were right-handed based on the Edinburgh handedness inventory. The study was approved by the Institutional Review Board at the University of Southern California (HS-08-00461) and was performed in accordance with the approved guidelines.

Set-up.
Participants were asked to sit on a fixed chair in front of a horizontal double-layered device (Fig. 1B). The device has a layer with a reflected monitor (visual space) and a tablet layer (hand space). Participants were instructed to move a digitizer pen (Wacom Intuos 7) on the tablet. When the lights were turned off in the experimental room, the participants' view of their forearm and hand was obscured by the mirror. Head and trunk movements were minimized by using a chin-rest and a fixed chair. A cursor (0.1 cm radius) representing the tip of the pen was displayed on the mirror.
Timing of a single trial. Before the start of each trial, a red circle was displayed on the reflected screen with a radius equal to the distance between the cursor and a white home circle (2.6 mm radius). Participants were instructed to move the digitizer pen to the white home circle by minimizing the radius of the red circle. At the onset of the trial, a cursor (red dot with 1 mm radius, whose position matched the tip of the digitizer pen) was displayed, and a main target (bull's eye shape) appeared at one of eight possible locations on a circle of 5.3 cm radius. Target locations were distributed evenly on the circle with 45 degrees between targets (except in the no-feedback block where no target and no feedback were shown). At each trial, the location of the main target was pseudo-random, with each target being presented once for each block of eight targets. Upon presentation of the main target, shown as a bull's eye, participants were asked to perform an outward shooting movement passing the target circle. The cursor disappeared when the distance between the home circle and the cursor was greater than one-third of the circle radius. A 4.2 mm sized red square was displayed on the target circle to indicate where the cursor has passed. Different types of visual feedback were given in different blocks and conditions (as discussed below). After each shooting movement, participants were instructed to perform an inward movement to the white home circle as before the start of each trial. When the duration of the shooting movement was outside the 50 to 300 ms range, the messages 'Too fast' or 'Too slow' was displayed.
Overall design common to all conditions. The experiment schedule consisted of a baseline block, an adaptation block, a no feedback block, and a washout block (Fig. 1F). Before the experiment, participants participated in a familiarization block without visuomotor rotation of 110 trials. Following familiarization, they performed a baseline block of 40 trials with no perturbation. Following baseline, a 45 degrees counter-clockwise adaptation was introduced (a small amount of noise, σ σ = . n N(0, ), 05 r r r 2 degree was introduced; pilot data, showed this noise slowed down adaptation slightly, possibly because it reduces the effectiveness of cognitive strategies. (Note, however, that this noise was small compared to the trial-by-trial standard deviation of 6.1 degrees in average for all participants at baseline). During the adaptation block, two targets were presented at each trial: a main (bull's eye) target and a neighboring target, shown at 45 degrees clockwise of the main target. As in Mazzoni and Krakauer's study 15 , participants were told about the rotation and were then instructed to use the following strategy after the second adaptation: "Counter the rotation by aiming at the clockwise neighboring target rather than at bull's eye" (Fig. 1C).
After 198 trials of adaptation trials with this strategy, participants performed a block of 10 shooting trials without feedback before performing a delayed washout block. The main goal of this no-feedback block was to erase the fast component of adaptation 17,18 , and therefore to test in the delayed washout block whether the slow component of adaptation had been updated via performance errors, SPEs, or both. During this no-feedback block, participants were instructed to shoot toward an arbitrary direction on a 180-degree arc displayed ahead of the home position, with the same radius as the target circle in the other blocks, and to return to the home position. No feedback was presented during and after the shooting movements.
We expected that the fast component of adaptation would be fully erased following these blocks for two reasons. First, whereas the block lasted 154 s on average, the time constant of the decay of the fast component of visuomotor adaptation has been estimated to be approximately 16.5 s 19 . Second, because participants performed active movements, forgetting of the fast component is faster than during an equivalent rest period 20,21 . Following this no-feedback block, participants were then instructed to stop using the explicit strategy and to shoot directly at the main target. The 80 trials following these new instructions formed the delayed washout block.
Experimental conditions. During the adaptation block, extending the method of Taylor and Ivry 16 , we manipulated the possible contributions of PE1 and PE2, by removing the bull's eye target, the neighboring target, or both, as soon as the cursor passed 1/3rd of the distance between the center of the home position and the target circle (corresponding to 1.77 cm away from the center home position). When the cursor re-appeared to show where it crossed the circle, both PE1 and PE2 were shown (Condition 1), only PE1 was shown (Condition 2), only PE2 was shown (Condition 3), or neither was shown (Condition 4) -see Fig. 1E. Statistical analysis. The angular error was computed as the difference between the main target angle and the shooting angle. The shooting angle was given by the line connecting the center position to the point on the invisible target circle given by interpolation of the two closest data points. To assess differences in adaptation and delayed retention among the four conditions, we performed one-way ANOVAs on the mean adaptation angular error of the middle 10 adaptation trials, last 10 adaptation trials, and first 10 washout trials in each condition.
To estimate the amplitude and rate of adaptation and washout, we then modeled the adaptation data and washout data with single exponential models of the form, Err(t) = A exp (−t/τ), where A is an amplitude parameter and τ a time constant parameter. Estimates and confidence intervals for A and τ and for the goodness of fit R 2 were obtained with a bootstrap procedure with 10000 samples by drawing participants with replacement.
Finally, to test whether the performance at the end of adaptation predicted performance at the beginning of washout, we correlated the average performance in the last 10 trials of adaptation and the first 10 trials of washout.

Results
Adaptation block. Figure 2 shows the performance error between the main target and the cursor for representative participants in each condition. Figure 3 shows the average error across participants. In all conditions, the strategy was effective at canceling the perturbation initially during the adaptation phase but participants subsequently exhibited a drift, as shown previously 15,16 . However, when PE1 was present, participants often attempted to compensate for the drift by generating corrections to reduce the performance error between the main target and the cursor (see Fig. 2). Such corrections, which were highly variable both within and between participants, resulted in smaller drifts overall (Fig. 3; compare Conditions 1 and 3 to Conditions 2 and 4). ANOVA and post-hoc t-tests confirmed this reduction in drifts during adaptation in conditions with PE1 present (Fig. 4(A); between-condition differences in error in mid and last 10 trials: ANOVAs p = 0.0001 and p = 0.003, respectively). The drift in the 10 middle adaptation trials of Condition 1 (with PE1 and PE2) was smaller than that in Condition 2 (PE2 only) and Condition 4 (neither PE1 nor PE2) (all p < 0.05; Tukey test). The drifts in the last 10 adaptation trials of Conditions 1 and 3 were smaller than in Conditions 2 and 4 (all p < 0.05; Tukey tests). However, the final level of adaptation was not different in Condition 4 (no feedback) from that of Condition 2 (with PE2, difference in error in middle and end of adaptation in Conditions 2 and 4, both p > 0.5). Thus, the presence of PE2 did not induce greater drift. Exponential fits to mean adaptation data (Fig. 5A) shows overall group similarities between adaptation in Conditions 1 and 3, on one hand, and Conditions 2 and 4, on the other hand. We note that the data of Condition 4 shows that neither PE1 nor PE2 was needed for the drift to be induced. This result is consistent with the drift being induced by the SPE only.
We note that the exponential fit during the adaptation phase is relatively good in Conditions 2 and 4 (R 2 estimated from bootstrap = 0.44 ± 0.11 SD, and 0.29 ± 0.12, respectively) was but poor in Conditions 1 and 3 (R 2 = 0.10 ± 0.05, and 0.16 ± 0.10). This suggests that performance in Conditions 2 and 4, was overall decreasing with an exponential-like shape, with relatively low noise. In contrast, in Conditions 1 and 3, high noise and corrections towards baseline, as seen in Figs 2 and 3, created performance patterns that were not well fit by exponential functions. Delayed washout block. As seen in Fig. 3, the after-effects in the washout periods were strikingly similar between conditions. Indeed, the mean of the first 10 trials of the washout was not different across conditions (ANOVA p > 0.5). Exponential curves fitted of the average washout data in each condition almost perfectly overlapped in all conditions (Fig. 5A). This similarity between decay during washout in the four conditions was verified by a bootstrap analysis. In contrast to the adaptation phase were exponential fit was poor for Conditions 1 and 3, performance during the washout phase was relatively well fit by exponential functions in all conditions (R 2 estimated from bootstrap: Condition 1: R 2 = 0. 48 ± 0. 05 SD, Condition 2: R 2 = 0. 44 ± 0.07; Condition 3: R 2 = 0. 54 ± 0.07, and Condition 4: R 2 = 0. 49 ± 0.07). Figure 5B,C shows that the 95% confidence intervals for the exponential amplitudes and time constants τ derived from the bootstrap analysis largely overlapped in all conditions. Performance at the end of adaptation predicted after-effects during washout in conditions without PE1 (Conditions 2 and 4), but not in conditions with PE1 (Conditions 1 and 3). This is shown by significant correlations between the average performance in the last 10 trials of adaptation and the first 10 trials of washout in Conditions 2 and 4 (p = 0.02, and p = 0.005), but not in Conditions 1 and 3 (p = 0.14 and p = 0.32), as shown in Fig. 6.
In sum, conditions with PE1 (Conditions 1 and 3) showed a prominent difference between performance during adaptation and performance in retention, as probed by the delayed washout test. In contrast, in conditions where a is a forgetting rate and b a learning rate. As can be seen, according to this simple model, the SPE, and memory update, are independent of the actual movements.
Such independence of internal model update from actual motor commands was partially supported in a previous study that studied differences in adaptation in shooting vs. reaching movements 23 . Whereas in the shooting condition, no motor correction was possible, participants could correct for on-line errors in the reaching condition. No differences in the rate of adaptation were found in healthy control participants, suggesting that neither actual motor movements nor motor errors were responsible for internal model update. However, the effect of the performance errors was unclear in this paradigm since the participants were always exposed to visual targets, which afforded them to compute performance errors. Here, because we manipulated the potential contribution of the targets by eliminating them after movement onset, we showed that SPEs, not performance errors, were the major error signals that updated the forward models. Despite important differences in experimental procedures, our results of identical washouts in the four conditions, despite large differences in behavior during adaptation, are in line with the results of two other types of arm visuomotor adaptation experiment. First, Miyamoto et al. 24 performed a visuomotor adaptation experiment in which participants adapted to a visuomotor perturbation to a single target. There was a remarkable convergence of the stable implicit component of adaptation after a one-minute break across participants. Because there was no relation between the stable component of adaptation and strategy (their figure Fig. 3l), it was concluded that the implicit stable component of adaptation is driven by SPEs. Note however the difference between the Miyamoto 24 and the current study. Whereas in their study the overall performance during adaptation was highly stereotyped between participants, it was highly variable in the present study. Second, in a type of experiment called "visual clamp", participants were instructed to perform straight ahead movements while the cursor was rotated to different angles 25 . Remarkably, the amount of adaptation, as measured by a drift in hand directions was highly similar across a number of cursor rotations. Note here however that it is unclear how SPEs are involved in the drift in visual clamp experiments, since the drift is constant despite largely variable SPEs across cursor rotation conditions.
How can we reconcile the view that reward-related signals play a role in motor adaptation 8,11-14 ? In the present study, no reward was given, thus performance error PE1 could have played the role of an implicit (e.g., self-evaluation) reward. Unlike actual extrinsic rewards, our results show that performance errors do not seem sufficient to influence the update of temporally stable component of motor memories in adaptation, at least in the time scale of the delayed washout period in our static arm reaching experiment. Note, however, that the effect of rewards on adaptation may be more pronounced in dynamic environment than in our static environment 26 . Another possibility is that performance errors act mostly as punishments in our study, since we do not provide explicit rewards when the cursor is within the target (such as commonly given "target explosions"). Unlike rewards which increase retention 13,14 , punishments appear to only modulate short-term increase in performance 13 and not retention. This is in line with our data because PE1 modulates performance during adaptation, but not performance during washout.
In summary, our results suggest that the temporally stable component of motor memory is formed by SPEs alone, whereas the strategy and the fast process 24 can be altered by performance errors during the period of motor adaptation. This differential update of the components of adaptation can account for the large "learning-performance distinction" 27 , according to which performance during training is a poor predictor of long-term performance. The results of the current study shed further light on the mechanism underlying the learning-performance distinction during learning of motor behaviors and can help with the development of algorithms designed to enhance motor learning in rehabilitation and sports 28 .