Introduction

Our brains communicate bi-directionally with our limbs during coordinated movement1,2,3 where descending motor commands and ascending feedback travel via the nerves in our extremities4. Sensory feedback encodes several aspects of body state during gross limb control (e.g. proprioception and kinesthesia), which guides the brain to make minute corrections during movements in a process called motor adaptation5. Concurrently, the brain develops and refines internal models of the motor commands required to produce desired limb movements using this sensory feedback6. Thus, lack of sensory feedback results in a drastic decrease in coordinated control over the limb2.

It is no surprise, then, that the lack of proprioceptive feedback is a major limitation for robotic prosthetic arms7. A commonly-researched method of restoring this missing branch of the communication loop is through sensory substitution8. Proprioceptive information such as limb position or speed are communicated to the user indirectly using separate sensory channels including vibration9,10,11,12,13 and audio14,15,16 cues, among others17. Recently, invasive techniques including nerve stimulation have also been investigated18,19,20,21. These technologies allow for more direct interfacing with the nervous system, can elicit a variety of sensations, and may be incorporated into devices taken out of the lab. However, although both sensory substitution and neural stimulation studies typically show improved limb control with sight of the prosthesis obscured22,23, these benefits do not always translate to tasks where the prosthesis is visible24—some studies demonstrate improvement25,26,27,28, while others show no change22,29,30,31,32.

Recent research suggests that the success or failure of artificial sensory feedback to confer improvements to prosthesis control is dependent on several connected factors, including the complexity of the task and the precision of feedforward internal models32. As feedforward internal models improve, tasks can be completed while relying less on feedback to make corrections. However, appropriately developing this feedforward controller requires accurate sensory feedback. One unexplored factor that may affect both the achieved benefit from artificial feedback and the speed of developing a feedforward controller is that of sensory fusion33.

When a single measurement (e.g. limb kinematics) can be observed simultaneously from two sources (e.g. vision and proprioception), the final estimated measurement is weighted according to each source’s uncertainty34,35. Thus, because vision is often significantly less uncertain than the modality used for sensory substitution, the contribution of these redundant proprioceptive cues may be negligible compared to vision. This phenomenon suggests that congruent sensory feedback will always improve perception, however the conferred benefit is dependent on its level of uncertainty compared to vision—minimal or negligible benefit with high uncertainty, and significant benefit with lower uncertainty.

Prior research suggests that vision has very low uncertainty when estimating position36,37, but higher uncertainty when estimating speed38,39. Because knowledge of limb speed is useful in forming internal models of movement40, it may also be useful for learning to control a prosthetic limb. In our previous study, we used psychophysics techniques to measure speed perception in vision and showed that visual estimates of joint speed are poorest when compared to absolute angular or linear speeds. Furthermore, we developed an audio feedback paradigm capable of providing prosthetic limb joint speed more precisely than vision41. Although we have demonstrated the capacity to augment vision with audio feedback in observational tasks, we have yet to investigate the effects of audio feedback during real-time reaching tasks. Additionally, reaching with a prosthesis requires simultaneous control of the body and the prosthesis, and differences in how feedback may affect control of these two domains remains to be investigated.

The purpose of this study is to evaluate augmented joint speed feedback's ability to reduce reaching errors and improve the rate of adaptation during reaching tasks. Subjects performed ballistic center-out reaches requiring coordinated movement of a positional- and myoelectric-controlled limb. We measured trial-by-trial adaptation to self-generated errors during steady-state reaches to determine the strength of the generated internal model. We also measured the adaptation rate across several trials immediately post-perturbation to understand the speed at which internal models update to changing system parameters.

Methods

Subjects

16 right hand-dominant, non-amputee subjects participated in this study, which was approved by the Northwestern University Institutional Review Board; all experiments were performed in accordance with relevant guidelines and regulations. The number of subjects was determined via power analysis to detect a large effect size (f = 0.4) with significance level α = 0.05, power (1 − β) = 0.80, and up to 12 planned comparisons with Bonferroni corrections. All subjects provided informed consent before starting the study.

Experimental setup

Subjects participated in two experimental sessions: one session with no audio feedback, and one session with frequency-modulated joint speed audio feedback. The order of these sessions was randomized across subjects using balanced block randomization.

Subjects sat in front of a computer monitor and placed their right arms in a wrist brace. The wrist brace was supported by a ball bearing cart on a table adjusted so the subject’s shoulder was abducted to 90°. The ball bearing cart allowed the brace to move freely across the surface of the table. A blanket was draped over the table to reduce noise from the ball bearings during movement (Fig. 1a).

Figure 1
figure 1

Center-Out Reaching Experiment Setup. (a) Subject arm is placed in a hand brace (pink), which rolls over across the table on a ball bearing cart. A goniometer (green) measures elbow angle, and EMG sensors (blue) measure EMG amplitude. (b) Goniometer angle (green) controls the position of the proximal link, and the EMG amplitude (blue) controls the speed of the distal link. Distal link speed is used for frequency-modulated audio feedback. (c) Subjects perform center-out reaches with the virtual limb (black), starting from the home circle (blue) and reaching for one of four targets (red), each of which can only be reached with a single limb configuration (dashed grey). Before each reach, a grey ball would appear above the target. When the limb endpoint left the home circle, the ball began to drop, centering on the target after half a second, signifying the end of the trial.

Full size image

Two Delsys Bagnoli electromyographic (EMG) sensors were placed over the flexor and extensor compartments of the forearm; muscles were not specifically targeted, but electrodes were generally placed over the flexor carpi radialis and extensor carpi radialis longus, respectively. The reference electrode was placed over the olecranon. EMG signals were high-pass filtered at 0.1 Hz, positive-rectified, and low-pass filtered at 5 Hz using 2nd order Butterworth filters. A Biometrics twin-axis electrogoniometer was secured to the upper and lower arm to measure the elbow flexion angle. Goniometer signals were low-pass filtered at 5 Hz using a 2nd order Butterworth filter. Data were acquired at 1000 Hz and, after filtering, downsampled to 100 Hz.

Subjects used the goniometer and their EMG signals to control a virtual two-link arm with 10 cm lengths. Goniometer measurements controlled the position of the angular proximal link, and EMG measurements controlled the angular velocity of the distal link using a typical two-site prosthesis control scheme; wrist extension drove the link clockwise, wrist flexion drove the link counterclockwise, and their difference in magnitudes determined the speed (Fig. 1b). This experimental setup emulates how reaches with a prosthesis involve both prosthetic movement and residual limb movement, and by extension investigates performance and adaptation of both types of movements.

The virtual arm started with the proximal link vertical (at 90°) and the distal link creating a 30° angle from the proximal link (black link, Fig. 1c). Throughout the experiment, targets appeared in one of four locations corresponding to limb positions relative to the home position: + 60° and − 60° from the starting proximal link position, and -45° and -90° from the starting distal link position (green crosses, Fig. 1c).

Subjects controlled the virtual arm to perform ballistic center-out reaches. A ball was shown above each target and dropped at a constant speed when the cursor left the home circle (blue circle, Fig. 1c). The ball aligned with the center of the target at 0.5 s; subjects were instructed to reach towards the target, stopping when the ball reached the target42.

If the proximal or distal links were moving faster than 45°/s at the end of the trial, the arm was considered to still be in motion, and the ball was colored red to indicate that the movement was not ballistic. Otherwise, if the cursor was inside of the target at the end of the trial, the ball was colored green to indicate a successful trial.

Audio feedback

During both experimental sessions, subjects wore noise-canceling headphones (Bose QuietComfort 35 II). During the No Feedback session, no sounds were played. During the Feedback session, frequency-modulated joint speed audio feedback was provided according to the following equation:

$$f={f}_{min}*{2}^{\frac{\omega }{{\omega }_{step}}}$$
(1)

where \(\omega\) is the angular speed of the distal link, \({f}_{min}\) is the minimum desired frequency (220 Hz), and \({\omega }_{step}\) is the angular speed increase that would result in a one-octave increase in pitch (60°/s). No sound played while the distal link was not moving. Audio feedback was provided during the entire session, including during training.

Familiarization

To learn to control the virtual arm, subjects completed 80 training center-out reaches (Fig. 2a). The first 40 trials had a specified reaching order (four sets of 10 reaches towards each target), and the second 40 trials had a balanced and randomized reaching order (10 reaches total towards each target).

Figure 2
figure 2

Non-Amputee Experimental Protocol. Subjects completed the experimental protocol twice—once with and once without audio feedback. The order of the feedback and no feedback sessions was randomized across subjects. (a) Familiarization involved a total of 80 reaches: four sets of 10 reaches towards each target, and 40 reaches towards targets in balanced random order. (b) The steady-state block involved a total of 180 reaches: four sets of 30 reaches towards each target, and 60 reaches towards targets in balanced random order. The order of same- or different-target groupings was randomized across subjects and consistent between subject visits. (c) The Perturbation block started with 20 reaches towards targets in random order. After these baseline trials, subjects did cycles of 8–12 reaches towards targets in random order, followed by either 8 reaches towards the same target, or 8 reaches towards targets in balanced random order. The order of these cycles was randomized across subjects and consistent between subject visits. Reaches towards different targets with a dashed border indicate that balanced randomization was not enforced, and the number of reaches towards targets could differ from one another.

Full size image

Steady-state block

To test trial-by-trial adaptation to self-generated errors, subjects completed 180 center-out reaches separated into one set of 60 and one set of 120 (Fig. 2b). The order of these sets was randomized across subjects using balanced block randomization. Subjects were allowed a short break between sets.

During the set of 60 trials, subjects reached towards targets in a balanced and randomized order. During the set of 120 trials, subjects completed four sets of 30 reaches towards each target. After each set, expanding window optimization separated initial trials from steady-state trials for post-experiment analysis43.

Trial-by-trial adaptation is defined as the amount of correction from one trial to the next, given the amount of error on the first trial:

$${\Delta }_{Error}\left(t+1\right)=a*Error\left(t\right)+{\Delta }_{0}$$
(2)

where \(a\) is the adaptation rate and \({\Delta }_{0}\) is the y-intercept, or the correction elicited when the previous trial has no error. While the y-intercept holds little real-world import in describing adaptation behavior, the x-intercept \(b\) describes the bias during reaches, or the error which on average elicits no correction:

$$b=\frac{-{\Delta }_{0}}{a}$$
(3)

Because of limb angle constraints, each position in the Cartesian reaching space can only be achieved with a single joint configuration. Thus, there exists a one-to-one mapping of Cartesian coordinates to joint configuration.

This one-to-one mapping allowed us to calculate trial-by-trial adaptation for the elbow and wrist joint angle deviations from the required angles needed to attain the target. The adaptation rate \(a\) and the bias \(b\) were compared between feedback conditions and between target order using the following model:

$$y\sim subject+feedback+target+feedback x target$$
(4)

where subject is coded as a categorical random variable and feedback and target are coded as categorical independent variables. Holm–Bonferroni corrections were made for the number of terms in each steady-state block model.

If reaches are drawn from a Gaussian distribution, traditional trial-by-trial analysis biases towards higher adaptation rates43. To account for this, we ran a secondary post-hoc trial-by-trial analysis using a secondary approach utilizing stochastic signal processing to filter out inherent motor control noise44. This approach models steady-state adaptation behavior to simulate state and internal model parameter estimation, providing an unbiased estimate of true adaptation behavior45. Because this analysis requires a consistent target, we only performed this analysis on steady-state reaches towards the same target.

Perturbation block

To test the speed of adaptation to external perturbations to the control system, subjects completed 20 practice trials followed by 24 sets of perturbation trials. During each set, subjects started by making 8–12 unperturbed reaches towards random targets. The system was then perturbed by doubling the EMG gain, increasing the speed of the distal link and making accurate and precise control more difficult. Subjects then made 8 reaches with the perturbed dynamics. These sets of 8 reaches fell into two categories: towards the same target, or towards different targets. Each category was tested in 12 sets of the perturbation trials (Fig. 2c). The order of these sets was determined randomly.

Perturbation adaptation of the Euclidean distance between the cursor and the target was estimated using an exponential decay model46,47,48:

$$y=\alpha {e}^{-\lambda t}+{\epsilon }_{\infty }$$
(5)

where \(t\) is the number of trials after perturbation onset, \(\alpha\) is the gain indicating the immediate increase in error upon perturbation, and \(\lambda\) is the decay rate indicating the speed at which the error converges to the baseline error \({\varepsilon }_{\infty }\).

To fit all three parameters for each feedback and target condition simultaneously, we used a hierarchical nonlinear mixed effects model. The top-level model was defined as the exponential decay model above [(5)]. The gain \(\alpha\), decay rate \(\lambda\), and baseline error \({\varepsilon }_{\infty }\) were each fit with the second-level model:

$$y\sim {\beta }_{0}+subject+feedback+target+feedback x target$$
(6)

where subject was coded as a categorical random variable and feedback and target were coded as categorical independent variables.

Results

Perturbation block

Perturbing reaches during ballistic movements creates a window through which we can measure adaptation to externally-generated errors. By measuring the initial and final errors after perturbing system control and calculating the rate at which one decays to the other, it is possible to gain insight into how quickly the sensorimotor system responds and adapts to novel conditions. Figure 3a and b shows target error during perturbed reaches. Our results suggest that joint speed feedback may lower the overall error rate after perturbations (offset). However, we found no significant differences in other aspects of adaptation behavior, including initial error increase upon sudden perturbation (gain) and adaptation rate.

Figure 3
figure 3

Non-Amputee Perturbation Adaptation. Average error traces during perturbation block show that feedback generally reduces errors during perturbation but does not affect the rate of adaptation (a, b) Error traces during perturbed reaches towards the same target (a) and different targets (b). (c–e) Hierarchical nonlinear mixed effects model exponential decay coefficients for the gain (c), rate (d), and offset (e). *Indicates p < 0.05 for simple main effects comparisons with Holm–Bonferroni corrections.

Full size image

A hierarchical nonlinear mixed effects model fit exponential decay coefficients to these data. Figure 3c–e shows these resulting coefficients. There was significant interaction between feedback and target conditions for the decay rate (p = 0.020). Thus, we ran a subsequent simple main effects model for each factor at each level of the other factor49; Table 1 summarizes these results. Of note is that feedback significantly reduced the offset when reaching towards different targets (p = 0.002), though this offset reduction was not significant when reaching towards the same target (p = 0.077). Furthermore, when feedback was available, reaching towards the same target resulted in a lower offset than reaching towards different targets (p = 0.020). Finally, a lower adaptation rate during reaches towards different targets was observed with no feedback available (p = 0.047); this result was unexpected, and is addressed further in the Discussions.

Table 1 Simple main effects analysis of exponential decay coefficients extracted from the hierarchical nonlinear mixed effects model.
Full size table

An analysis of the first and last perturbation trials provides a secondary perspective on these results. We found that providing joint speed feedback significantly reduced the initial error upon sudden perturbation (p = 0.017), but did not significantly reduce the final error achieved on the last trial (p = 0.141).

Steady-state block

Steady-state reaches provide insight into how subjects coordinate positional- and myoelectric-controlled joints during reaching tasks after adapting to a control scheme, and may be used to quantify compensatory movements in one joint arising from errors or poor control in the other. Figure 4 shows the errors in Euclidean (a, d) and joint (b, c, e, f) spaces.

Figure 4
figure 4

Non-Amputee Endpoint and Joint Angle Errors. Endpoint and joint angle errors vary moderately by target, and elbow errors were lower when making repeated reaches towards the same target, but no consistent differences were found between feedback conditions. Error bars indicate standard error of the mean. (a–c) Errors while reaching towards the same target for endpoint (a), position-controlled elbow (b), and myoelectric-controlled wrist (c). (d–f) Errors while reaching towards different targets for endpoint (d), elbow (e), and wrist (f).

Full size image

No significant interactions were found (pmin = 0.923), so interaction terms were removed and the models were rerun49. Joint speed feedback showed no significant differences in endpoint errors (p = 0.187), wrist angle errors (p = 0.177), or elbow angle errors (p = 0.590). Positional elbow errors were higher when reaching towards different targets than the same target (p = 0.03), however no such differences were found for endpoint errors (p = 0.187) or myoelectric wrist angle errors (p = 0.245).

We conducted an analysis of trial-by-trial adaptation to investigate differences in adaptation rates between feedback and target conditions, and to identify possible compensatory strategies in the reach biases. Our results showed no significant interactions between feedback and target for elbow bias or rate (pmin = 0.592), so the interaction terms were removed and the models rerun49. We found an improved adaptation rate during reaches towards different targets for the elbow (p < 0.001) and wrist (p < 0.001) (Fig. 5b), but no significant differences for the bias of the elbow (p = 0.079) and wrist (p > 0.999) (Fig. 5a). No differences were observed between feedback conditions for elbow (p = 0.136) or wrist bias (p > 0.999). No differences were observed between feedback conditions for elbow (p = 0.227) or wrist adaptation rates (p = 0.518). However, an interesting observation is that subjects tended to underreach with the wrist (demonstrated by the negative wrist bias) and overreach with the elbow (demonstrated by the positive elbow bias). Possible explanations for this reaching strategy are presented in the Discussions.

Figure 5
figure 5

Non-amputee trial-by-trial adaptation analysis. Trial-by-trial adaptation biases suggests the elbow overreaches to compensate for an underreaching wrist as shown by the opposite signs of elbow and wrist biases. However, no changes in trial-by-trial adaptation behavior was observed between feedback conditions. (a) Trial-by-trial adaptation bias (b) Trial-by-trial adaptation rate. ***Indicates p < 0.001.

Full size image

To supplement our traditional trial-by-trial analysis, we ran a secondary stochastic signal processing analysis. In contrast to traditional analysis, we calculated a significant increase in adaptation rate for the elbow (p < 0.001); adaptation rate also increased in the wrist, but this improvement was not significant (p = 0.077) (Fig. 6a). Analyzing the control noise (Q) revealed no significant differences between feedback conditions for elbow control noise (p = 0.055) or wrist control noise (p = 0.894) (Fig. 6b).

Figure 6
figure 6

Non-amputee secondary trial-by-trial analysis. A secondary trial-by-trial analysis using stochastic signal processing approach found that (a) joint-speed feedback significantly improved adaptation of positional elbow movements, though the improved adaptation of myoelectric wrist movements was not significant, and (b) that control noise was significantly higher for the myoelectric controlled wrist than the positional controlled elbow. ***Indicates p < 0.001.

Full size image

These results taken together suggest that joint speed feedback may have improved trial-by-trial adaptation behavior (Fig. 6), and that adaptation may have been improved when subjects made reaches relying on a generalized motor program (different targets), compared to target-specific motor plans (same targets) (Fig. 5).

Discussion

Artificial sensory feedback for prosthetic limbs is a frequent research topic, but results can vary widely. Studies that block vision and hearing consistently demonstrate the benefits of artificial sensory feedback, whereas study outcomes with these senses available are more inconsistent32. In this study, we investigated whether providing proprioceptive information interpreted imprecisely via these intact sensory modalities was a factor in determining the benefit of artificial sensory feedback. Using a Bayesian sensory integration framework, we developed an audio feedback paradigm to provide joint speed cues, a measurement not well estimated by vision. Subjects completed ballistic center-out reaches requiring simultaneous control of positional- and myoelectric-controlled joints, in a manner analogous to myoelectric prosthesis control. Our results confirmed that such simulated reaches were indeed possible, despite initial subject difficulty.

We also evaluated how joint speed feedback affects human adaptation to self-generated and externally-generated errors during ballistic reaching tasks. Investigating these differences in motor control necessitates separating feedforward and feedback segments of reach; thus, ballistic movement is typically enforced, such that reaches are completed before visual or proprioceptive feedback can be used to adjust the reach trajectory. Furthermore, a typical task paradigm used to test motor control is the center-out reaching task50; for arm reaches, this task requires coordinating shoulder and elbow movements to maneuver a cursor towards a target. However, completing such a reach with a prosthetic limb is unique in that it requires simultaneous control of the robotic limb and the residual joints. To our knowledge, no prior study has investigated this type of hybrid biological/robotic ballistic reaching task.

Joint speed feedback reduces errors during perturbed reaches, but not steady-state reaches

Subjects were able to adapt their reaches to a gain perturbation of the wrist controller, consequently reducing their reaching errors over the course of several trials. When these reaches were fit using an exponential decay model, subjects achieved a lower offset when reaching towards different targets, meaning that reaching errors were generally lower for all trials with the perturbed system (Fig. 3). It was also found that repeated reaches towards the same target had a lower offset than reaches towards different targets when feedback was available. However, the availability of joint speed feedback did not significantly affect the adaptation rate during these reaches. Further, during steady-state reaches with joint speed feedback available, we found no significant reductions in either Cartesian and joint-based reference frame errors compared to no feedback conditions (Fig. 4).

Exponential decay is a useful model for describing motor adaptation to perturbations, as each parameter quantifies a different aspect of the reaching behavior over time46,47,48. Offset describes the error rate that reaches converge to following many trials, Gain describes the initial jump in error at the onset of the perturbation compared to the Offset, and Rate describes how quickly error falls from the initial error to the Offset value. However, one limitation of this model is that it cannot account for error values which fall below the Offset due to inherent reach variability. Thus, when fitting an exponential decay model to a small subset of data, such as reaches from a single subject, the model may choose unrealistic values to most closely match the limited data available.

The most common occurrence of this model fitting error was when the variability of reaches during the last 7 trials overshadowed the average improvement during these same trials. When this happens, the model achieves a best fit by passing through the y-intercept (Gain), then immediately converging to the steady-state error (Offset), which results in an extremely high decay Rate. One way to address this is to constrain the model with a lower Offset estimate (for example, by assuming that the steady-state error following perturbation will be due only to reach variability) and removing the effect of reach bias51. However, the hierarchical non-linear mixed-effects model circumvents this issue by using a large amount of data and simultaneously fitting the three parameters across all conditions while still allowing for individual subject variability via random effects.

Stochastic signal processing analysis, but not traditional trial-by-trial analysis, demonstrates improved adaptation

Analyzing trial-by-trial adaptation behavior using two different methods, we found two different outcomes. Traditional trial-by-trial adaptation analysis demonstrated a higher adaptation rate when reaching towards different targets than when reaching towards the same target (Fig. 5b), which may suggest that subjects are less prone to modifying their reach behavior when relying on target-specific internal models. This is in line with previous research which has shown prior reaches affect the path of future reaches52 and the exploitation of path redundancy53. However, joint speed feedback did not significantly affect trial-by-trial adaptation behavior for the elbow or wrist-controlled joints.

A limitation of the trial-by-trial adaptation analysis is that the results describe a biased measure of the true adaptation to self-generated errors43. We partially accounted for this by only quantifying trial-by-trial adaptation on trials which had achieved steady-state error, however our estimates are still biased. The presence of control noise during a reaching trial obfuscates the true intended response to the previous trial’s reach error. In simulations for reaching behavior using a hierarchical Kalman filter model45, we found that control noise biases the calculated adaptation rates towards a slope of -1 (i.e. perfect adaptation). To address this bias in traditional trial-by-trial adaptation analysis, we ran a secondary analysis using a stochastic signal processing approach44. This approach models reach behavior and tuning in the internal model via a hierarchical Kalman filter paradigm45. From this analysis, we calculated that joint speed feedback did significantly improve steady-state adaptation for both the elbow and wrist (Fig. 6a). The trial-by-trial analysis of the x-intercept (bias, in this paper), on the other hand, is not affected by noise and therefore may still be a useful metric for observational trial-by-trial adaptation analysis; to our knowledge, this study is the first demonstration of this metric.

Subjects demonstrated consistent elbow compensation for poor myoelectric control

The difference in trial-by-trial adaptation bias suggests that underreaching of the wrist was compensated for by slight overreaching of the elbow (Fig. 5a). One possible explanation for this behavior is the higher control noise in the myoelectric control of that joint (Fig. 6b); to minimize discrepancy between expected and achieved wrist positions, subjects underreached with the wrist to reduce variability and compensated by overreaching with the elbow, which exhibits more consistent control. This strategy may result in lower and more consistent endpoint errors than a strategy focused on perfect placement of each joint.

Our analysis showed that the control noise for the myoelectric-controlled wrist was several times higher than the control noise for the position-controlled elbow, and that noise did not significantly differ between feedback conditions (Fig. 6b). The difference in control noise between elbow and wrist may provide an explanation for the difference in adaptation bias from the traditional trial-by-trial analysis, suggesting the elbow was used to compensate for wrist errors. The subjects’ goal during our center-out reaching task was to minimize the distance between linkage endpoint and target center. If elbow control is deemed “more reliable” (i.e. has lower control noise) than wrist control, subjects may have used the elbow more heavily to correct for errors. Although the joint-speed feedback provided information solely about wrist movements, this information may have been used to better guide the elbow to make corrective adjustments during steady-state reaches. Furthermore, if instead of a two-link arm, subjects controlled the position of the cursor using orthogonal controls (e.g. the elbow and wrist controlled the horizontal and vertical positions, respectively), then this compensation would not arise; the performance of each degree of freedom would be independent from one another. The two-link arm task was chosen because of the uncertainty of visual joint speed perception, however it is possible that errors would have been lower and adaptation would have been higher for wrist movements if this cross-joint compensation were not permitted by the task.

Limitations

A limitation of this study is that only non-amputee subjects were included, who have an intact biological communication loop between the brain and the limb. This biological feedback can reduce the variability of feedforward commands and result in more precise and accurate limb movements than would be generated by subjects with amputations2. Another difference is that intact-limb subjects may have felt increased tactile and proprioceptive cues during the study; wrist flexion and extension contractions were isometric, but the wrist brace exerted antagonistic forces during these contractions. Thus, it is possible our artificial sensory feedback system was integrating with not only vision, but also the magnitude of this restrictive force. This would theoretically improve task performance, but the reduced uncertainty of the estimate of limb movements could also potentially reduce the benefits of feedback34,35. Our results remain interpretable for the purposes of teleoperated robotics including industrial machinery and surgical robots, however further investigation is required for myoelectric prosthesis applications. Future studies will include trans-radial amputee subjects to control for this additional source of indirect proprioceptive feedback.

The uncharacteristically high adaptation rate observed for reaches towards the same target with no feedback warrants additional attention (Fig. 3d). On average, while reaching towards the same target, errors were higher on the first perturbation trial with no feedback available. However, the errors are about the same by the second trial, suggesting larger improvement with no feedback. Furthermore, this behavior is not seen when reaching towards different targets. One potential explanation of this behavior is that it is an artifact of the duration of our ballistic movements. In typical center-out reaching studies, ballistic movements are defined on the order of 200 ms or less42. However, to accommodate the myoelectric control used in this study, we defined our threshold of ballistic movement as 500 ms, which could be long enough for subjects to react to changes. Mean auditory simple reaction times are between 140 and 160 ms, and mean visual simple reaction times are between 180 and 200 ms54; choice reaction times are longer, but differences between modalities remain the same55. It is possible that the difference in reaction times between audio feedback and vision-only conditions (feedback vs. no feedback) were great enough to allow for error correction during the reach with feedback, which is suggested by the lower initial error rates (Fig. 3). This drastic observed increase in error during no feedback conditions combined with the ability to rely on a target-specific motor plan during the same target condition may explain the uncharacteristically high adaptation rate.

We found limited improvements in reducing average reaching errors, elbow control noise, and error after sudden controller perturbations. However, Bayesian sensory integration may still provide insight into creating beneficial sensory feedback. The determinants of sensory feedback improvement are complex and intertwined. To fit the constraints of ballistic reaching, our tasks were simple center-out reaches. However, recent studies suggest that the benefits of feedback are more pronounced when provided during complex tasks necessitating complex prosthesis coordination32,56. After practice, subjects may have developed a sufficiently strong internal model for the simple reaches, negating the benefits of feedback.

Frequency-modulated audio was selected as the modality of feedback in this study due to its resolution and ease of implementation. Human perception of frequency has a resolution of less than 1%57, compared to the 30% or greater for visual joint speed perception41. This feedback is not a strong candidate for everyday use, but rather serves as a best-case scenario for joint speed feedback investigation. The day-to-day benefits of feedback are dependent on numerous factors32, likely among these are feedback modality, encoding, location, intensity, and user preference. For nerve stimulation approaches in particular, additional factors could include stimulated receptive field, elicited sensation, pleasantness, and naturalness. We believe our results to be generalizable to other feedback methods, as our study was based on investigation of motor and adaptation behaviors, however individual results may vary depending on the resolution of other feedback methods. The specifics of how these variables might impact prosthesis control, performance, and use are questions that should be addressed as prosthesis technology advances.

Conclusions

Proprioception and kinesthesia are crucial senses for human limb control, and are currently lacking in modern prosthetic limbs. We developed an artificial sensory feedback system to improve the sense of joint speed and tested its effectiveness on improving control and adaptation to novel conditions. Our results suggest improvement in reaching performance following a perturbation, and a reaching strategy that may arise to compensate for high myoelectric control noise. This improved reaching strategy suggests that joint speed feedback may assist prosthesis users in recovering reach performance following sudden changes to prosthesis control. However, the effects of joint speed feedback during complex coordinated tasks, and its interaction with other determinants of sensory feedback improvement, remain to be investigated. We anticipate future studies will refine our knowledge of how to successfully implement artificial sensory feedback, leading to improved control for prosthesis user’s robotic limbs.