Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

On human-in-the-loop optimization of human–robot interaction

Nature volume 633, pages 779–788 (2024)Cite this article

Subjects

Abstract

From industrial exoskeletons to implantable medical devices, robots that interact closely with people are poised to improve every aspect of our lives. Yet designing these systems is very challenging; humans are incredibly complex and, in many cases, we respond to robotic devices in ways that cannot be modelled or predicted with sufficient accuracy. A new approach, human-in-the-loop optimization, can overcome these challenges by systematically and empirically identifying the device characteristics that result in the best objective performance for a specific user and application. This approach has enabled substantial improvements in human–robot performance in research settings and has the potential to speed development and enhance products. In this Perspective, we describe methods for applying human-in-the-loop optimization to new human–robot interaction problems, addressing each key decision in a variety of contexts. We also identify opportunities to develop new optimization techniques and answer underlying scientific questions. We anticipate that our readers will advance human-in-the-loop optimization and use it to design robotic devices that truly enhance the human experience.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Designing systems of human-in-the-loop optimization.
Fig. 2: Potential applications for human-in-the-loop optimization.
Fig. 3: Example parameterizations.
Fig. 4: Challenges for human-in-the-loop optimization algorithms.

Similar content being viewed by others

References

  1. Demir, K. A., Döven, G. & Sezen, B. Industry 5.0 and human-robot co-working. Procedia Comput. Sci. 158, 688–695 (2019).

    Article  Google Scholar 

  2. Farina, D. et al. Toward higher-performance bionic limbs for wider clinical use. Nat. Biomed. Eng. 7, 473–485 (2023).

    Article  PubMed  Google Scholar 

  3. Sawicki, G. S., Beck, O. N., Kang, I. & Young, A. J. The exoskeleton expansion: improving walking and running economy. J. Neuroeng. Rehabil. 17, 25 (2020). This review presents a timeline of lower-limb exoskeleton development and performance enhancements.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Crea, S. et al. Occupational exoskeletons: a roadmap toward large-scale adoption. Methodology and challenges of bringing exoskeletons to workplaces. Wearable Technol. 2, e11 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Uchida, T. K. & Delp, S. L. Biomechanics of Movement: The Science of Sports, Robotics, and Rehabilitation (MIT Press, 2021).

  6. Ghez, C. & Krakauer, J. in Principles of Neural Science 4th edn (eds Kandel, E. R., Schwartz, J. H. & Jessell, T. M.) 653–673 (McGraw-Hill, 2000).

  7. Halilaj, E. et al. Machine learning in human movement biomechanics: best practices, common pitfalls, and new opportunities. J. Biomech. 81, 1–11 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Alili, A. et al. A novel framework to facilitate user preferred tuning for a robotic knee prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 31, 895–903 (2023).

    Article  Google Scholar 

  9. Franks, P. W. et al. in Proc. 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob) 700–707 (IEEE, 2020). This study demonstrates the shortcomings of simulation-based optimization of human–robot interactions.

  10. Diaz, M. A. et al. Human-in-the-loop optimization of wearable robotic devices to improve human–robot interaction: a systematic review. IEEE Trans. Cybern. 53, 7483–7496 (2022).

    Article  Google Scholar 

  11. Zhang, J. et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science 356, 1280–1284 (2017). This study highlights the effectiveness of human-in-the-loop optimization for increasing the benefits of an exoskeleton.

    Article  ADS  CAS  PubMed  Google Scholar 

  12. Poggensee, K. L. & Collins, S. H. How adaptation, training, and customization contribute to benefits from exoskeleton assistance. Sci. Robot. 6, eabf1078 (2021). This study highlights the importance of human adaptation in achieving effective human–robot interaction.

    Article  PubMed  Google Scholar 

  13. Witte, K. A., Fiers, P., Sheets-Singer, A. L. & Collins, S. H. Improving the energy economy of human running with powered and unpowered ankle exoskeleton assistance. Sci. Robot. 5, eaay9108 (2020).

    Article  PubMed  Google Scholar 

  14. Bryan, G. M. et al. Optimized hip–knee–ankle exoskeleton assistance reduces the metabolic cost of walking with worn loads. J. Neuroeng. Rehabil. 18, 161 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  15. Song, S. & Collins, S. H. Optimizing exoskeleton assistance for faster self-selected walking. IEEE Trans. Neural Syst. Rehabil. Eng. 29, 786–795 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  16. Ding, Y., Kim, M., Kuindersma, S. & Walsh, C. J. Human-in-the-loop optimization of hip assistance with a soft exosuit during walking. Sci. Robot. 3, eaar5438 (2018). This study illustrates the use of Bayesian optimization for human-in-the-loop optimization.

    Article  PubMed  Google Scholar 

  17. Kim, J. et al. Reducing the energy cost of walking with low assistance levels through optimized hip flexion assistance from a soft exosuit. Sci. Rep. 12, 11004 (2018).

    Article  ADS  Google Scholar 

  18. Haufe, F., Wolf, P. & Riener, R. Human-in-the-loop optimization of a multi-joint wearable robot for movement assistance. Proc. Autom. Med. Eng. 1, 023 (2020).

    Google Scholar 

  19. Slade, P., Kochenderfer, M. J., Delp, S. L. & Collins, S. H. Personalizing exoskeleton assistance while walking in the real world. Nature 610, 277–282 (2022). This study demonstrates a data-driven method for human-in-the-loop optimization and provides an example of optimization under naturalistic conditions.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  20. Ingraham, K. A., Remy, C. D. & Rouse, E. J. The role of user preference in the customized control of robotic exoskeletons. Sci. Robot. 7, eabj3487 (2022).

    Article  CAS  PubMed  Google Scholar 

  21. Lee, U. H. et al. User preference optimization for control of ankle exoskeletons using sample efficient active learning. Sci. Robot. 8, eadg3705 (2023).

    Article  PubMed  Google Scholar 

  22. Kantharaju, P. et al. Reducing squat physical effort using personalized assistance from an ankle exoskeleton. IEEE Trans. Neural Syst. Rehabil. Eng. 30, 1786–1795 (2022).

    Article  PubMed  Google Scholar 

  23. Pang, M. et al. Stiffness optimization based on muscle fatigue and muscle synergy for passive waist assistive exoskeleton. Robotic Intell. Autom. 43, 209–224 (2023).

    Article  Google Scholar 

  24. Koginov, G. et al. Human-in-the-loop personalization of a bi-articular wearable robot’s assistance for downhill walking. IEEE Trans. Med. Robot. Bionics 6, 328–339 (2023).

    Article  Google Scholar 

  25. Hamaya, M., Matsubara, T., Noda, T., Teramae, T. & Morimoto, J. Learning task-parameterized assistive strategies for exoskeleton robots by multi-task reinforcement learning. In IEEE International Conference on Robotics and Automation (ICRA) 5907–5912 (IEEE, 2017).

  26. Liu, R. et al. Adaptive symmetry reference trajectory generation in shared autonomy for active knee orthosis. IEEE Robot. Autom. Lett. 8, 3118–3125 (2023).

    Article  Google Scholar 

  27. Li, Z., Li, Q., Huang, P., Xia, H. & Li, G. Human-in-the-loop adaptive control of a soft exo-suit with actuator dynamics and ankle impedance adaptation. IEEE Trans. Cybern. 53, 7920–7932 (2023).

    Article  PubMed  Google Scholar 

  28. Kantharaju, P. et al. Framework for personalizing wearable devices using real-time physiological measures. IEEE Access 11, 81389–81400 (2023).

    Article  Google Scholar 

  29. Wen, T. C., Jacobson, M., Zhou, X., Chung, H. J. & Kim, M. in Proc. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 3431–3436 (IEEE, 2020).

  30. Wen, Y., Si, J., Brandt, A., Gao, X. & Huang, H. H. Online reinforcement learning control for the personalization of a robotic knee prosthesis. IEEE Trans. Cybern. 50, 2346–2356 (2019).

    Article  PubMed  Google Scholar 

  31. Tankink, T., Carloni, R. & Hijmans, J. M. & Houdijk, H. Human-in-the-loop optimization of rocker shoes via different cost functions during walking. J. Biomech. 166, 112028 (2024). This study provides an example of human-in-the-loop optimization of a non-robotic device.

    Article  PubMed  Google Scholar 

  32. Tankink, T., Houdijk, H. & Hijmans, J. M. Human-in-the-loop optimized rocker profile of running shoes to enhance ankle work and running economy. Eur. J. Sport Sci. 24, 164–173 (2024).

    Article  PubMed Central  Google Scholar 

  33. Huang, G., Lin, S. & Xie, L. Human-in-the-loop optimization of knee-joint biomechanical energy harvester to maximize power generation with minimal user effort. Energy Convers. Manage. 283, 116913 (2023).

    Article  Google Scholar 

  34. Felt, W., Selinger, J. C., Donelan, J. M. & Remy, C. D. “Body-in-the-loop”: optimizing device parameters using measures of instantaneous energetic cost. PLoS One 10, e0135342 (2015). This study provides an example of an early, gradient-based approach to human-in-the-loop optimization.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Garcia-Rosas, R., Tan, Y., Oetomo, D., Manzie, C. & Choong, P. Personalized online adaptation of kinematic synergies for human-prosthesis interfaces. IEEE Tran. Cybern. 51, 1070–1084 (2019).

    Article  Google Scholar 

  36. Catkin, B. & Patoglu, V. Preference-based human-in-the-loop optimization for perceived realism of haptic rendering. IEEE Trans. Haptics 16, 470–476 (2023).

    Article  PubMed  Google Scholar 

  37. Fauvel, T. & Chalk, M. Human-in-the-loop optimization of visual prosthetic stimulation. J. Neural Eng. 19, 036038 (2022). This study provides an example of user preference as an optimization objective, in this case applied to a retinal prosthesis.

    Article  ADS  Google Scholar 

  38. Sánchez, N. et al. Multi-site identification and generalization of clusters of walking behaviors in individuals with chronic stroke and neurotypical controls. Neurorehabil. Neural Repair 37, 810–822 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Lamers, E. P., Yang, A. J. & Zelik, K. E. Feasibility of a biomechanically-assistive garment to reduce low back loading during leaning and lifting. IEEE Trans. Biomed. Eng. 65, 1674–1680 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Nuesslein, C. et al. Comparing metabolic cost and muscle activation for knee and back exoskeletons in lifting. IEEE Trans. Med. Robot. Bionics 6, 224–234 (2023).

    Article  Google Scholar 

  41. Kazerooni, H., Racine, J.-L., Huang, L. & Steger, R. in Proc. 2005 IEEE International Conference on Robotics and Automation 4353–4360 (IEEE, 2005). This study describes an early exoskeleton that did not improve user performance despite extensive investment, illustrating the risks of a traditional development approach.

  42. Garcia, M., Chatterjee, A., Ruina, A. & Coleman, M. The simplest walking model: stability, complexity, and scaling. J. Biomech. Eng. 120, 281–288 (1998).

    Article  CAS  PubMed  Google Scholar 

  43. Dembia, C. L., Silder, A., Uchida, T. K., Hicks, J. L. & Delp, S. L. Simulating ideal assistive devices to reduce the metabolic cost of walking with heavy loads. PLoS One 12, e0180320 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  44. Siviy, C. et al. Offline assistance optimization of a soft exosuit for augmenting ankle power of stroke survivors during walking. IEEE Robot. Autom. Lett. 5, 828–835 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  45. Jackson, R. W. & Collins, S. H. An experimental comparison of the relative benefits of work and torque assistance in ankle exoskeletons. J. Appl. Physiol. 119, 541–557 (2015).

    Article  PubMed  Google Scholar 

  46. Caputo, J. M. & Collins, S. H. A universal ankle–foot prosthesis emulator for human locomotion experiments. J. Biomech. Eng. 136, 035002 (2014).

    Article  PubMed  Google Scholar 

  47. Witte, K. A., Zhang, J., Jackson, R. W. & Collins, S. H. in Proc. 2015 IEEE International Conference on Robotics and Automation (ICRA) 1223–1228 (IEEE, 2015).

  48. Anderson, A. et al. A robotic emulator for the systematic exploration of transtibial biarticular prosthesis designs. Preprint at https://doi.org/10.36227/techrxiv.24417310.v1 (2023).

  49. Portnova, A. A., Mukherjee, G., Peters, K. M., Yamane, A. & Steele, K. M. Design of a 3D-printed, open-source wrist-driven orthosis for individuals with spinal cord injury. PLoS One 13, e0193106 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  50. Severin, A. C. et al. Case report: adjusting seat and backrest angle improves performance in an elite paralympic rower. Front. Sports Act. Living 3, 625656 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  51. Sanz-Pena, I., Jeong, H. & Kim, M. Personalized wearable ankle robot using modular additive manufacturing design. IEEE Robot. Autom. Lett. 8, 4935–4942 (2023).

    Article  Google Scholar 

  52. Sloot, L. H. et al. Effects of a soft robotic exosuit on the quality and speed of overground walking depends on walking ability after stroke. J. Neuroeng. Rehabil. 20, 113 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  53. Walsh, C. Human-in-the-loop development of soft wearable robots. Nat. Rev. Mater. 3, 78–80 (2018).

    Article  ADS  Google Scholar 

  54. Xu, L. et al. Reducing the muscle activity of walking using a portable hip exoskeleton based on human-in-the-loop optimization. Front. Bioeng. Biotechnol. 11, 1006326 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  55. Kong, H. M. A Personalized Quasi-passive Ankle Exoskeleton Using Human-in-the loop Optimization Approaches Doctoral dissertation, KTH Royal Institute of Technology (2023).

  56. Hybart, R., Villancio-Wolter, K. S. & Ferris, D. P. Metabolic cost of walking with electromechanical ankle exoskeletons under proportional myoelectric control on a treadmill and outdoors. PeerJ 11, e15775 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  57. Kinsey, H., Upton, E. & Young, A. Towards meaningful community ambulation in individuals post stroke through use of a smart hip exoskeleton: a preliminary investigation. Assist. Technol. 36, 198–208 (2023).

    Google Scholar 

  58. Fang, Y., Orekhov, G. & Lerner, Z. Improving the energy cost of incline walking and stair ascent with ankle exoskeleton assistance in cerebral palsy. IEEE Trans. Biomed. Eng. 69, 2143–2152 (2021).

    Article  Google Scholar 

  59. Caputo, J. M. et al. Robotic emulation of candidate prosthetic foot designs may enable efficient, evidence-based, and individualized prescriptions. J. Prosthet. Orthot. 34, 202–212 (2022).

    Article  PubMed  Google Scholar 

  60. Welker, C. G., Voloshina, A. S., Chiu, V. L. & Collins, S. H. Shortcomings of human-in-the-loop optimization of an ankle-foot prosthesis emulator: a case series. R. Soc. Open Sci. 8, 202020 (2021).

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  61. Arelekatti, V. N. M. & Winter, A. G. V. in Proc. 2015 IEEE International Conference on Rehabilitation Robotics (ICORR) 350–356 (IEEE, 2015).

  62. Mattson, C. A. & Winter, A. G. Why the developing world needs mechanical design. J. Mech. Des. 138, 070301 (2016).

    Article  Google Scholar 

  63. Eikevåg, S. W., Erichsen, J. F. & Steinert, M. in Proc. The Engineering of Sport 14 1–2 (International Sports Engineering Association, 2022).

  64. Quintero, D., Villarreal, D. J., Lambert, D. J., Kapp, S. & Gregg, R. D. Continuous-phase control of a powered knee–ankle prosthesis: amputee experiments across speeds and inclines. IEEE Trans. Robot. 34, 686–701 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  65. Geyer, H. & Herr, H. A muscle-reflex model that encodes principles of legged mechanics produces human walking dynamics and muscle activities. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 263–273 (2010).

    Article  PubMed  Google Scholar 

  66. Varol, H. A., Sup, F. & Goldfarb, M. Multiclass real-time intent recognition of a powered lower limb prosthesis. IEEE Trans. Biomed. Eng. 57, 542–551 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  67. Simon, A. M. et al. Configuring a powered knee and ankle prosthesis for transfemoral amputees within five specific ambulation modes. PLoS One 9, e99387 (2014).

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  68. Tran, M., Gabert, L., Cempini, M. & Lenzi, T. A lightweight, efficient fully powered knee prosthesis with actively variable transmission. IEEE Robot. Autom. Lett. 4, 1186–1193 (2019).

    Article  Google Scholar 

  69. Song, Y., Romero, A., Müller, M., Koltun, V. & Scaramuzza, D. Reaching the limit in autonomous racing: optimal control versus reinforcement learning. Sci. Robot. 8, eadg1462 (2023).

    Article  PubMed  Google Scholar 

  70. Slade, P., Kochenderfer, M. J., Delp, S. L. & Collins, S. H. Sensing leg movement enhances wearable monitoring of energy expenditure. Nat. Commun. 12, 4312 (2021).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  71. Revi, D. A., Alvarez, A. M., Walsh, C. J., De Rossi, S. M. & Awad, L. N. Indirect measurement of anterior-posterior ground reaction forces using a minimal set of wearable inertial sensors: from healthy to hemiparetic walking. J. Neuroeng. Rehabil. 17, 82 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  72. Ramadurai, S., Jeong, H. & Kim, M. Predicting the metabolic cost of exoskeleton-assisted squatting using foot pressure features and machine learning. Front. Robot. AI 10, 1166248 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  73. Flach, P. & Matsubara, E. in Dagstuhl Seminar Proceedings Vol. 7161 1–10 (Schloss Dagstuhl–Leibniz-Zentrum für Informatik, 2008).

  74. Wang, W., Raitor, M., Collins, S., Liu, C. K. & Kennedy, M. in Proc. 2023 IEEE International Conference on Robotics and Automation (ICRA) 10483–10489 (IEEE, 2023).

  75. Eveld, M. E., King, S. T., Vailati, L. G., Zelik, K. E. & Goldfarb, M. On the basis for stumble recovery strategy selection in healthy adults. J. Biomech. Eng. 143, 071003 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  76. Chasnov, B. J., Ratliff, L. J. & Burden, S. A. Human adaptation to adaptive machines converges to game-theoretic equilibria. Preprint at https://arxiv.org/abs/2305.01124 (2023).

  77. Snaterse, M., Ton, R., Kuo, A. D. & Donelan, J. M. Distinct fast and slow processes contribute to the selection of preferred step frequency during human walking. J. Appl. Physiol. 110, 1682–1690 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  78. Finley, J. M., Bastian, A. J. & Gottschall, J. S. Learning to be economical: the energy cost of walking tracks motor adaptation. J. Physiol. 591, 1081–1095 (2013).

    Article  CAS  PubMed  Google Scholar 

  79. Nikolaidis, S., Nath, S., Procaccia, A. D. & Srinivasa, S. in Proc. 2017 ACM/IEEE International Conference on Human-Robot Interaction 323–331 (IEEE, 2017).

  80. Medrano, R. L., Thomas, G. C., Margolin, D. & Rouse, E. J. The economic value of augmentative exoskeletons and their assistance. Commun. Eng. 2, 43 (2023).

    Article  PubMed Central  Google Scholar 

  81. Brown, G. L., Seethapathi, N. & Srinivasan, M. A unified energy-optimality criterion predicts human navigation paths and speeds. Proc. Natl Acad. Sci. 118, e2020327118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  82. IJmker, T., Lamoth, C. J., Houdijk, H., van der Woude, L. H. & Beek, P. J. Postural threat during walking: effects on energy cost and accompanying gait changes. J. Neuroeng. Rehabil. 11, 71 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  83. Park, K. W., Choi, J. & Kong, K. Iterative learning of human behavior for adaptive gait pattern adjustment of a powered exoskeleton. IEEE Trans. Robot. 38, 1395–1409 (2022). This study illustrates the potential for human–robot interaction to improve mobility for individuals with severe impairments.

    Article  Google Scholar 

  84. Antos, S. A., Kording, K. P. & Gordon, K. E. Energy expenditure does not solely explain step length–width choices during walking. J. Exp. Biol. 225, jeb243104 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  85. McDonald, K. A., Cusumano, J. P., Hieronymi, A. & Rubenson, J. Humans trade off whole-body energy cost to avoid overburdening muscles while walking. Proc. R. Soc. B 289, 20221189 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  86. Mombaur, K., Truong, A. & Laumond, J. P. From human to humanoid locomotion—an inverse optimal control approach. Auton. Robots 28, 369–383 (2010).

    Article  Google Scholar 

  87. Tucker, M. et al. in Proc. 2020 IEEE International Conference on Robotics and Automation (ICRA) 2351–2357 (IEEE, 2020).

  88. Ingraham, K. A., Tucker, M., Ame, A. D., Rouse, E. J. & Shepherd, M. K. Leveraging user preference in the design and evaluation of lower-limb exoskeletons and prostheses. Curr. Opin. Biomed. Eng. 28, 100487 (2023).

    Article  Google Scholar 

  89. Brunner, C., Fischer, A., Luig, K. & Thies, T. Pairwise support vector machines and their application to large scale problems. J. Mach. Learn. Res. 13, 2279–2292 (2012).

    MathSciNet  Google Scholar 

  90. Astudillo, R. et al. in Proc. ICML 2023 Workshop The Many Facets of Preference-Based Learning (ICML, 2023).

  91. Hansen, N. in Towards a New Evolutionary Computation. Studies in Fuzziness and Soft Computing, Vol. 192 (eds Lozano, J. A., Larrañaga, P., Inza, I. & Bengoetxea, E.) 75–102 (Springer, 2006).

  92. Kochenderfer, M. J. & Wheeler, T. A. Algorithms for Optimization (MIT Press, 2019).

  93. Lakmazaheri, A. et al. Optimizing exoskeleton assistance to improve walking speed and energy economy for older adults. J. Neuroeng. Rehabil. 21, 1 (2024).

    Article  PubMed  PubMed Central  Google Scholar 

  94. Han, H. et al. Selection of muscle-activity-based cost function in human-in-the-loop optimization of multi-gait ankle exoskeleton assistance. IEEE Trans. Neural Syst. Rehabil. Eng. 29, 944–952 (2021).

    Article  PubMed  Google Scholar 

  95. Kutulakos, Z. & Slade, P. Simulating human-in-the-loop optimization of exoskeleton assistance to compare optimization algorithm performance. Preprint at bioRxiv https://doi.org/10.1101/2024.04.05.587982 (2024).

  96. Antonova, R., Rai, A. & Atkeson, C. G. in Proc. 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids) 22–28 (IEEE, 2016).

  97. Kim, M. et al. Human-in-the-loop Bayesian optimization of wearable device parameters. PLoS One 12, e0184054 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  98. Kim, M. et al. in Proc. 2019 International Conference on Robotics and Automation (ICRA) 9173–9179 (IEEE, 2019).

  99. Denning, P. J. Working sets past and present. IEEE Trans. Softw. Eng. 1, 64–84 (1980).

    Article  ADS  Google Scholar 

  100. Franks, P. W. et al. Comparing optimized exoskeleton assistance of the hip, knee, and ankle in single and multi-joint configurations. Wearable Technol. 2, e16 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  101. Vasudevan, E. V., Torres-Oviedo, G., Morton, S. M., Yang, J. F. & Bastian, A. J. Younger is not always better: development of locomotor adaptation from childhood to adulthood. J. Neurosci. 31, 3055–3065 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  102. Macready, W. G. & Wolpert, D. H. Bandit problems and the exploration/exploitation tradeoff. IEEE Trans. Evol. Comput. 2, 2–22 (1998).

    Article  Google Scholar 

  103. McAllister, M. J., Blair, R. L., Donelan, J. M. & Selinger, J. C. Energy optimization during walking involves implicit processing. J. Exp. Biol. 224, jeb242655 (2021).

    Article  PubMed  Google Scholar 

  104. Hybart, R. & Ferris, D. Gait variability of outdoor vs treadmill walking with bilateral robotic ankle exoskeletons under proportional myoelectric control. PLoS One 18, e0294241 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  105. Waldherr, S., Romero, R. & Thrun, S. A gesture based interface for human-robot interaction. Auton. Robots 9, 151–173 (2000).

    Article  Google Scholar 

  106. Landi, C. T., Ferraguti, F., Fantuzzi, C. & Secchi, C. in Proc. 2018 IEEE International Conference on Robotics and Automation (ICRA) 3279–3284 (IEEE, 2018).

  107. Xiao, X. et al. APPL: adaptive planner parameter learning. Robot. Auton. Syst. 154, 104132 (2022).

    Article  Google Scholar 

  108. Kristoffersen, M. B., Franzke, A. W., van der Sluis, C. K., Murgia, A. & Bongers, R. M. The effect of feedback during training sessions on learning pattern-recognition-based prosthesis control. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 2087–2096 (2019).

    Article  PubMed  Google Scholar 

  109. Wong, J. D., Selinger, J. C. & Donelan, J. C. Is natural variability in gait sufficient to initiate spontaneous energy optimization in human walking? J. Neurophysiol. 121, 1848–1855 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  110. Abram, S. J. et al. General variability leads to specific adaptation toward optimal movement policies. Curr. Biol. 32, 2222–2232 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  111. Song, S., Haynes, C. A. & Bradford, J. C. Human cortical, muscular, and kinematic gait adaptation with novel use of an ankle exoskeleton. Preprint at Research Square https://doi.org/10.21203/rs.3.rs-2675191/v1 (2023).

  112. Jacobsen, N. A. & Ferris, D. P. Electrocortical activity correlated with locomotor adaptation during split‐belt treadmill walking. J. Physiol. 601, 3921–3944 (2023).

    Article  CAS  PubMed  Google Scholar 

  113. Mu, T., Goel, K. & Brunskill, E. in Proc. 31st Conference on Neural Information Processing Systems (NIPS 2017) (Curran Associates, 2017).

  114. Ghonasgi, K. et al. in Proc. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 771–776 (IEEE, 2021).

  115. Byeon, S., Choi, J., Zhang, Y. & Hwang, I. Stochastic-skill-level-based shared control for human training in urban air mobility scenario. ACM Trans. Hum.-Robot Interact. (in the press).

  116. Srivastava, M., Biyik, E., Mirchandani, S., Goodman, N. & Sadigh, D. Assistive teaching of motor control tasks to humans. Adv. Neural Inf. Process. Syst. 35, 28517–28529 (2022).

    Google Scholar 

  117. Kim, M. et al. Visual guidance can help with the use of a robotic exoskeleton during human walking. Sci. Rep. 12, 3881 (2022).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  118. Madden, J. D. Mobile robots: motor challenges and materials solutions. Science 318, 1094–1097 (2007).

    Article  ADS  CAS  PubMed  Google Scholar 

  119. Burden, S. A., Libby, T., Jayaram, K., Sponberg, S. & Donelan, J. Why animals can outrun robots. Sci. Robot. 9, eadi9754 (2024).

    Article  PubMed  Google Scholar 

  120. Riener, R., Rabezzana, L. & Zimmermann, Y. D. Do robots outperform humans in human-centered domains? Front. Robot. AI 10, 1223946 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  121. Collins, S. H., Wiggin, M. B. & Sawicki, G. S. Reducing the energy cost of human walking using an unpowered exoskeleton. Nature 522, 212–215 (2015).

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  122. Lee, H. J. et al. A wearable hip assist robot can improve gait function and cardiopulmonary metabolic efficiency in elderly adults. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 1549–1557 (2017).

    ADS  PubMed  Google Scholar 

  123. Mooney, L. M., Rouse, E. J. & Herr, H. M. Autonomous exoskeleton reduces metabolic cost of human walking during load carriage. J. Neuroeng. Rehabil. 11, 80 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank C. Walsh for editorial suggestions.

Author information

Authors and Affiliations

  1. Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA

    Patrick Slade

  2. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA

    Christopher Atkeson

  3. WearTech Labs, Simon Fraser University, Burnaby, British Columbia, Canada

    J. Maxwell Donelan

  4. Department of Human Movement Sciences, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands

    Han Houdijk

  5. Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA

    Kimberly A. Ingraham

  6. Mechanical and Industrial Engineering, University of Illinois Chicago, Chicago, IL, USA

    Myunghee Kim

  7. Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea

    Kyoungchul Kong

  8. Department of Rehabilitation Medicine, Erasmus Medical Center, Rotterdam, The Netherlands

    Katherine L. Poggensee

  9. Faculty of Mechanical Engineering, Delft University of Technology, Delft, The Netherlands

    Katherine L. Poggensee

  10. Sensory-Motor Systems Lab, ETH Zurich, Zürich, Switzerland

    Robert Riener

  11. Faculty of Medicine, University of Zurich, Zürich, Switzerland

    Robert Riener

  12. Department of Mechanical and Industrial Engineering, Norwegian University of Science and Technology (NTNU), Trondheim, Norway

    Martin Steinert

  13. College of Artificial Intelligence, Institute of Robotics and Automatic Information System, Nankai University, Tianjin, China

    Juanjuan Zhang

  14. Department of Mechanical Engineering, Stanford University, Stanford, CA, USA

    Steven H. Collins

Authors
  1. Patrick Slade
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Atkeson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Maxwell Donelan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Han Houdijk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kimberly A. Ingraham
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Myunghee Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kyoungchul Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Katherine L. Poggensee
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Robert Riener
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Martin Steinert
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Juanjuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Steven H. Collins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Patrick Slade or Steven H. Collins.

Ethics declarations

Competing interests

S.H.C. and P.S. are inventors on patents that cover the human-in-the-loop optimization concept. S.H.C. is an inventor on patents for the emulator systems discussed in this paper.

Peer review

Peer review information

Nature thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Slade, P., Atkeson, C., Donelan, J.M. et al. On human-in-the-loop optimization of human–robot interaction. Nature 633, 779–788 (2024). https://doi.org/10.1038/s41586-024-07697-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41586-024-07697-2

Search

Advanced search

Quick links

Nature Briefing: Translational Research

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Get what matters in translational research, free to your inbox weekly. Sign up for Nature Briefing: Translational Research