Differential game theory for versatile physical human–robot interaction

Abstract

The last decades have seen a surge of robots working in contact with humans. However, until now these contact robots have made little use of the opportunities offered by physical interaction and lack a systematic methodology to produce versatile behaviours. Here, we develop an interactive robot controller able to understand the control strategy of the human user and react optimally to their movements. We demonstrate that combining an observer with a differential game theory controller can induce a stable interaction between the two partners, precisely identify each other’s control law, and allow them to successfully perform the task with minimum effort. Simulations and experiments with human subjects demonstrate these properties and illustrate how this controller can induce different representative interaction strategies.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Simulation of an arm reaching with the GT interactive controller.
Fig. 2: Simulated arm reaching training for motor recovery.
Fig. 3: Adaptation of assistance to reaching experiment.

Code availability

The code that supports the findings of this study is available from the corresponding authors upon reasonable request.

Data availability

The data that support the findings of this study are available from the corresponding authors upon reasonable request.

References

  1. 1.

    Sawers, A. & Ting, L. H. Perspectives on human-human sensorimotor interactions for the design of rehabilitation robots. J. Neuroeng. Rehabil. 11, 142 (2014).

  2. 2.

    Ganesh, G. et al. Two is better than one: physical interactions improve motor performance in humans. Sci. Rep. 4, 3824 (2014).

  3. 3.

    Jarrassé, N., Sanguineti, V. & Burdet, E. Slaves no longer: review on role assignment for human-robot joint motor action. Adapt. Behav. 22, 70–82 (2014).

  4. 4.

    Hesse, S. et al. Computerized arm training improves the motor control of the severely affected arm after stroke: a single-blinded randomized trial in two centers. Stroke 36, 1960–1966 (2005).

  5. 5.

    Hokayem, P. F. & Spong, M. W. Bilateral teleoperation: an historical survey. Automatica 42, 2035–2057 (2006).

  6. 6.

    Passenberg, C., Peer, A. & Buss, M. A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Mechatronics 20, 787–801 (2010).

  7. 7.

    Colombo, R. & Sanguineti, V. in Rehabilitation Robotics (eds Colombo, R. & Sanguineti, V.) 63–74 (Elsevier, 2018).

  8. 8.

    Marchal-Crespo, L. et al. The effect of haptic guidance and visual feedback on learning a complex tennis task. Exp. Brain Res. 231, 277–291 (2013).

  9. 9.

    Díaz, I., Gil, J. J. & Sánchez, E. Lower-limb robotic rehabilitation: literature review and challenges. J. Robot. 2011, 1–11 (2011).

  10. 10.

    Na, X. & Cole, D. J. Linear quadratic game and noncooperative predictive methods for potential application to modelling driver-AFS interactive steering control. Veh. Sys. Dyn. 51, 165–198 (2013).

  11. 11.

    Music, S. & Hirche, S. Control sharing in human-robot team interaction. Annu. Rev. Control 44, 342–354 (2017).

  12. 12.

    Khoramshahi, M. & Billard, A. A dynamical system approach to task-adaptation in physical human–robot interaction. Auton. Robot. https://doi.org/10.1007/s10514-018-9764-z (2018).

  13. 13.

    Jarrassé, N., Charalambous, T. & Burdet, E. A framework to describe, analyze and generate interactive motor behaviors. PLoS ONE 7, e49945 (2012).

  14. 14.

    Starr, A. W. & Ho, Y.-C. Nonzero-sum differential games. J. Optim. Theory Appl. 3, 184–206 (1969).

  15. 15.

    Basar, T. & Olsder, G. J. Dynamic Noncooperative Game Theory 2nd edn (Society for Industrial and Applied Mathematics, Philadelphia, 1999).

  16. 16.

    Takagi, A., Ganesh, G., Yoshioka, T., Kawato, M. & Burdet, E. Physically interacting individuals estimate their partner’s movement goal to enhance motor abilities. Nat. Hum. Behav. 1, 0054 (2017).

  17. 17.

    Kiumarsi, B. et al. Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans. Neur. Netw. Learn. Syst. 29, 2042–2062 (2018).

  18. 18.

    Marden, J. R., Arslan, G. & Shamma, J. S. Joint strategy fictitious play with inertia for potential games. IEEE Trans. Autom. Contr. 54, 208–220 (2009).

  19. 19.

    Li, Y., Tee, K. P., Yan, R., Chan, W. L. & Wu, Y. A framework of human-robot coordination based on game theory and policy iteration. IEEE Trans. Robot. 32, 1408–1418 (2016).

  20. 20.

    Reinkensmeyer, D. J. et al. Computational neurorehabilitation: modeling plasticity and learning to predict recoverys. J. Neuroeng. Rehabil. 13, 1–25 (2016).

  21. 21.

    Nierhoff, T., Leibrandt, K., Lorenz, T. & Hirche, S. Robotic billiards: understanding humans in order to counter them. IEEE Trans. Cybern. 46, 1889–1899 (2016).

  22. 22.

    Slotine, J.-J. E. & Li, W. Applied Nonlinear Control (Prentice-Hall, Upper Saddle River, 1991).

  23. 23.

    Gajic, Z. & Qureshi, M. T. J. Lyapunov Matrix Equation in System Stability and Control (Elsevier, Amsterdam, 1995).

  24. 24.

    Burdet, E., Franklin, D. W. & Milner, T. E. Human Robotics: Neuromechanics and Motor Control (MIT Press, Cambridge, MA, 2013).

  25. 25.

    Engwerda, J. Algorithms for computing Nash equilibria in deterministic LQ games. Comput. Manag. Sci. 4, 113–140 (2007).

  26. 26.

    Evrard, P. & Kheddar, A. Homotopy switching model for dyad haptic interaction in physical collaborative tasks. In Proc. IEEE Worldhaptics 45–50 (2009).

  27. 27.

    Emken, J. L., Benitez, R., Sideris, A., Bobrow, J. E. & Reinkensmeyer, D. J. Motor adaptation as a greedy optimization of error and effort. J. Neurophysiol. 97, 3997–4006 (2007).

  28. 28.

    Franklin, D. W. et al. CNS learns stable, accurate, and efficient movements using a simple algorithm. J. Neurosci. 28, 11165–11173 (2008).

  29. 29.

    Levin, M. F. et al. Deficits in the coordination of agonist and antagonist muscles in stroke patients: implications for normal motor control. Brain Res. 853, 352–369 (2000).

  30. 30.

    Colgate, J. E. et al. Methods and apparatus for manipulation of heavy payloads with intelligent assist devices. US patent 7185774 (2007).

  31. 31.

    Zoss, A. B., Kazerooni, H. & Chu, A. Biomechanical design of the Berkeley lower extremity exoskeleton (BLEEX). IEEE-ASME Trans. Mech. 11, 128–138 (2006).

  32. 32.

    Peshkin, M. A. et al. Cobot architecture. IEEE Trans. Robot. Autom. 17, 377–390 (2001).

  33. 33.

    Burgar, C. G. et al. Robot-assisted upper-limb therapy in acute rehabilitation setting following stroke: Department of Veterans Affairs multisite clinical trial. J. Rehabil. Res. Dev. 48, 445–458 (2011).

  34. 34.

    Chackochan, V. T. Development of Collaborative Strategies in Joint Action. PhD thesis, University of Genoa, Italy (2018).

  35. 35.

    Braun, D. A., Ortega, P. A. & Wolpert, D. M. Nash equilibria in multi-agent motor interactions. PLoS Comput. Biol. 5, e1000468 (2009).

  36. 36.

    Hogan, N. et al. Motions or muscles? Some behavioral factors underlying robotic assistance of motor recovery. J. Rehabil. Res. Dev. 43, 605 (2006).

  37. 37.

    Kahn, L. E. et al. Robot-assisted movement training for the stroke-impaired arm: does it matter what the robot does? J. Rehabil. Res. Dev. 43, 619 (2006).

  38. 38.

    Spong, M. & Vidyasagar, M. Robot Dynamics and Control (Wiley, Hoboken, 1989).

  39. 39.

    Codourey, A. & Burdet, E. A body-oriented method for finding a linear form of the dynamic equation of fully parallel robots. Proc. IEEE Int. Conf. Robot. 2, 1612–1618 (1997).

  40. 40.

    Campolo, D. et al. H-Man: a planar, H-shape cabled differential robotic manipulandum for experiments on human motor control. J. Neurosci. Meth. 235, 285–297 (2014).

Download references

Acknowledgements

We thank J. Eden, T. Mylvaganam, N. P. Perez, Q.-C. Pham and K. P. Tee for their careful reading and comments on the manuscript. This research was supported in part by the European Commission grant EU-H2020 COGIMON (644727), UK EPSRC MOTION grant EP/NO29003/1 and Singapore MOE Tier1 grant RG48/17.

Author information

Affiliations

Authors

Contributions

Control concepts: Y.L. and E.B.; algorithm and simulation: Y.L.; set-up: F.G. and D.C.; experiments: G.C.; results analysis: Y.L., G.C. and E.B.; manuscript writing: Y.L. and E.B. All authors have read and edited the manuscript, and agree with its content.

Corresponding authors

Correspondence to Y. Li or E. Burdet.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Identifiable images

Consent to publish identifiable images of research participants was obtained.

Ethical Compliance

We have complied with all relevant ethical regulations.

Ethics Committee

Guidelines for study procedures were provided by Imperial College London.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Carboni, G., Gonzalez, F. et al. Differential game theory for versatile physical human–robot interaction. Nat Mach Intell 1, 36–43 (2019). https://doi.org/10.1038/s42256-018-0010-3

Download citation

Further reading