Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Differential game theory for versatile physical human–robot interaction

Nature Machine Intelligence volume 1, pages 36–43 (2019)Cite this article

Subjects

Abstract

The last decades have seen a surge of robots working in contact with humans. However, until now these contact robots have made little use of the opportunities offered by physical interaction and lack a systematic methodology to produce versatile behaviours. Here, we develop an interactive robot controller able to understand the control strategy of the human user and react optimally to their movements. We demonstrate that combining an observer with a differential game theory controller can induce a stable interaction between the two partners, precisely identify each other’s control law, and allow them to successfully perform the task with minimum effort. Simulations and experiments with human subjects demonstrate these properties and illustrate how this controller can induce different representative interaction strategies.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Simulation of an arm reaching with the GT interactive controller.
Fig. 2: Simulated arm reaching training for motor recovery.
Fig. 3: Adaptation of assistance to reaching experiment.

Similar content being viewed by others

Code availability

The code that supports the findings of this study is available from the corresponding authors upon reasonable request.

Data availability

The data that support the findings of this study are available from the corresponding authors upon reasonable request.

References

  1. Sawers, A. & Ting, L. H. Perspectives on human-human sensorimotor interactions for the design of rehabilitation robots. J. Neuroeng. Rehabil. 11, 142 (2014).

    Article  Google Scholar 

  2. Ganesh, G. et al. Two is better than one: physical interactions improve motor performance in humans. Sci. Rep. 4, 3824 (2014).

    Article  Google Scholar 

  3. Jarrassé, N., Sanguineti, V. & Burdet, E. Slaves no longer: review on role assignment for human-robot joint motor action. Adapt. Behav. 22, 70–82 (2014).

    Article  Google Scholar 

  4. Hesse, S. et al. Computerized arm training improves the motor control of the severely affected arm after stroke: a single-blinded randomized trial in two centers. Stroke 36, 1960–1966 (2005).

    Article  Google Scholar 

  5. Hokayem, P. F. & Spong, M. W. Bilateral teleoperation: an historical survey. Automatica 42, 2035–2057 (2006).

    Article  MathSciNet  Google Scholar 

  6. Passenberg, C., Peer, A. & Buss, M. A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Mechatronics 20, 787–801 (2010).

    Article  Google Scholar 

  7. Colombo, R. & Sanguineti, V. in Rehabilitation Robotics (eds Colombo, R. & Sanguineti, V.) 63–74 (Elsevier, 2018).

  8. Marchal-Crespo, L. et al. The effect of haptic guidance and visual feedback on learning a complex tennis task. Exp. Brain Res. 231, 277–291 (2013).

    Article  Google Scholar 

  9. Díaz, I., Gil, J. J. & Sánchez, E. Lower-limb robotic rehabilitation: literature review and challenges. J. Robot. 2011, 1–11 (2011).

    Article  Google Scholar 

  10. Na, X. & Cole, D. J. Linear quadratic game and noncooperative predictive methods for potential application to modelling driver-AFS interactive steering control. Veh. Sys. Dyn. 51, 165–198 (2013).

    Article  Google Scholar 

  11. Music, S. & Hirche, S. Control sharing in human-robot team interaction. Annu. Rev. Control 44, 342–354 (2017).

    Article  Google Scholar 

  12. Khoramshahi, M. & Billard, A. A dynamical system approach to task-adaptation in physical human–robot interaction. Auton. Robot. https://doi.org/10.1007/s10514-018-9764-z (2018).

  13. Jarrassé, N., Charalambous, T. & Burdet, E. A framework to describe, analyze and generate interactive motor behaviors. PLoS ONE 7, e49945 (2012).

    Article  Google Scholar 

  14. Starr, A. W. & Ho, Y.-C. Nonzero-sum differential games. J. Optim. Theory Appl. 3, 184–206 (1969).

    Article  MathSciNet  Google Scholar 

  15. Basar, T. & Olsder, G. J. Dynamic Noncooperative Game Theory 2nd edn (Society for Industrial and Applied Mathematics, Philadelphia, 1999).

  16. Takagi, A., Ganesh, G., Yoshioka, T., Kawato, M. & Burdet, E. Physically interacting individuals estimate their partner’s movement goal to enhance motor abilities. Nat. Hum. Behav. 1, 0054 (2017).

    Article  Google Scholar 

  17. Kiumarsi, B. et al. Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans. Neur. Netw. Learn. Syst. 29, 2042–2062 (2018).

    Article  MathSciNet  Google Scholar 

  18. Marden, J. R., Arslan, G. & Shamma, J. S. Joint strategy fictitious play with inertia for potential games. IEEE Trans. Autom. Contr. 54, 208–220 (2009).

    Article  MathSciNet  Google Scholar 

  19. Li, Y., Tee, K. P., Yan, R., Chan, W. L. & Wu, Y. A framework of human-robot coordination based on game theory and policy iteration. IEEE Trans. Robot. 32, 1408–1418 (2016).

    Article  Google Scholar 

  20. Reinkensmeyer, D. J. et al. Computational neurorehabilitation: modeling plasticity and learning to predict recoverys. J. Neuroeng. Rehabil. 13, 1–25 (2016).

    Article  Google Scholar 

  21. Nierhoff, T., Leibrandt, K., Lorenz, T. & Hirche, S. Robotic billiards: understanding humans in order to counter them. IEEE Trans. Cybern. 46, 1889–1899 (2016).

    Article  Google Scholar 

  22. Slotine, J.-J. E. & Li, W. Applied Nonlinear Control (Prentice-Hall, Upper Saddle River, 1991).

  23. Gajic, Z. & Qureshi, M. T. J. Lyapunov Matrix Equation in System Stability and Control (Elsevier, Amsterdam, 1995).

  24. Burdet, E., Franklin, D. W. & Milner, T. E. Human Robotics: Neuromechanics and Motor Control (MIT Press, Cambridge, MA, 2013).

  25. Engwerda, J. Algorithms for computing Nash equilibria in deterministic LQ games. Comput. Manag. Sci. 4, 113–140 (2007).

    Article  MathSciNet  Google Scholar 

  26. Evrard, P. & Kheddar, A. Homotopy switching model for dyad haptic interaction in physical collaborative tasks. In Proc. IEEE Worldhaptics 45–50 (2009).

  27. Emken, J. L., Benitez, R., Sideris, A., Bobrow, J. E. & Reinkensmeyer, D. J. Motor adaptation as a greedy optimization of error and effort. J. Neurophysiol. 97, 3997–4006 (2007).

    Article  Google Scholar 

  28. Franklin, D. W. et al. CNS learns stable, accurate, and efficient movements using a simple algorithm. J. Neurosci. 28, 11165–11173 (2008).

    Article  Google Scholar 

  29. Levin, M. F. et al. Deficits in the coordination of agonist and antagonist muscles in stroke patients: implications for normal motor control. Brain Res. 853, 352–369 (2000).

    Article  Google Scholar 

  30. Colgate, J. E. et al. Methods and apparatus for manipulation of heavy payloads with intelligent assist devices. US patent 7185774 (2007).

  31. Zoss, A. B., Kazerooni, H. & Chu, A. Biomechanical design of the Berkeley lower extremity exoskeleton (BLEEX). IEEE-ASME Trans. Mech. 11, 128–138 (2006).

    Article  Google Scholar 

  32. Peshkin, M. A. et al. Cobot architecture. IEEE Trans. Robot. Autom. 17, 377–390 (2001).

    Article  Google Scholar 

  33. Burgar, C. G. et al. Robot-assisted upper-limb therapy in acute rehabilitation setting following stroke: Department of Veterans Affairs multisite clinical trial. J. Rehabil. Res. Dev. 48, 445–458 (2011).

    Article  Google Scholar 

  34. Chackochan, V. T. Development of Collaborative Strategies in Joint Action. PhD thesis, University of Genoa, Italy (2018).

  35. Braun, D. A., Ortega, P. A. & Wolpert, D. M. Nash equilibria in multi-agent motor interactions. PLoS Comput. Biol. 5, e1000468 (2009).

    Article  MathSciNet  Google Scholar 

  36. Hogan, N. et al. Motions or muscles? Some behavioral factors underlying robotic assistance of motor recovery. J. Rehabil. Res. Dev. 43, 605 (2006).

    Article  Google Scholar 

  37. Kahn, L. E. et al. Robot-assisted movement training for the stroke-impaired arm: does it matter what the robot does? J. Rehabil. Res. Dev. 43, 619 (2006).

    Article  Google Scholar 

  38. Spong, M. & Vidyasagar, M. Robot Dynamics and Control (Wiley, Hoboken, 1989).

  39. Codourey, A. & Burdet, E. A body-oriented method for finding a linear form of the dynamic equation of fully parallel robots. Proc. IEEE Int. Conf. Robot. 2, 1612–1618 (1997).

    Google Scholar 

  40. Campolo, D. et al. H-Man: a planar, H-shape cabled differential robotic manipulandum for experiments on human motor control. J. Neurosci. Meth. 235, 285–297 (2014).

    Article  Google Scholar 

Download references

Acknowledgements

We thank J. Eden, T. Mylvaganam, N. P. Perez, Q.-C. Pham and K. P. Tee for their careful reading and comments on the manuscript. This research was supported in part by the European Commission grant EU-H2020 COGIMON (644727), UK EPSRC MOTION grant EP/NO29003/1 and Singapore MOE Tier1 grant RG48/17.

Author information

Authors and Affiliations

  1. Department of Bioengineering, Imperial College of Science, Technology and Medicine, London, UK

    Y. Li, G. Carboni, F. Gonzalez & E. Burdet

  2. Department of Engineering and Design, University of Sussex, Brighton, UK

    Y. Li

  3. School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore

    D. Campolo & E. Burdet

Authors
  1. Y. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Carboni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. F. Gonzalez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. D. Campolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. E. Burdet
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Control concepts: Y.L. and E.B.; algorithm and simulation: Y.L.; set-up: F.G. and D.C.; experiments: G.C.; results analysis: Y.L., G.C. and E.B.; manuscript writing: Y.L. and E.B. All authors have read and edited the manuscript, and agree with its content.

Corresponding authors

Correspondence to Y. Li or E. Burdet.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Identifiable images

Consent to publish identifiable images of research participants was obtained.

Ethical Compliance

We have complied with all relevant ethical regulations.

Ethics Committee

Guidelines for study procedures were provided by Imperial College London.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Carboni, G., Gonzalez, F. et al. Differential game theory for versatile physical human–robot interaction. Nat Mach Intell 1, 36–43 (2019). https://doi.org/10.1038/s42256-018-0010-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-018-0010-3

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics