The last decades have seen a surge of robots working in contact with humans. However, until now these contact robots have made little use of the opportunities offered by physical interaction and lack a systematic methodology to produce versatile behaviours. Here, we develop an interactive robot controller able to understand the control strategy of the human user and react optimally to their movements. We demonstrate that combining an observer with a differential game theory controller can induce a stable interaction between the two partners, precisely identify each other’s control law, and allow them to successfully perform the task with minimum effort. Simulations and experiments with human subjects demonstrate these properties and illustrate how this controller can induce different representative interaction strategies.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The code that supports the findings of this study is available from the corresponding authors upon reasonable request.
The data that support the findings of this study are available from the corresponding authors upon reasonable request.
Sawers, A. & Ting, L. H. Perspectives on human-human sensorimotor interactions for the design of rehabilitation robots. J. Neuroeng. Rehabil. 11, 142 (2014).
Ganesh, G. et al. Two is better than one: physical interactions improve motor performance in humans. Sci. Rep. 4, 3824 (2014).
Jarrassé, N., Sanguineti, V. & Burdet, E. Slaves no longer: review on role assignment for human-robot joint motor action. Adapt. Behav. 22, 70–82 (2014).
Hesse, S. et al. Computerized arm training improves the motor control of the severely affected arm after stroke: a single-blinded randomized trial in two centers. Stroke 36, 1960–1966 (2005).
Hokayem, P. F. & Spong, M. W. Bilateral teleoperation: an historical survey. Automatica 42, 2035–2057 (2006).
Passenberg, C., Peer, A. & Buss, M. A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Mechatronics 20, 787–801 (2010).
Colombo, R. & Sanguineti, V. in Rehabilitation Robotics (eds Colombo, R. & Sanguineti, V.) 63–74 (Elsevier, 2018).
Marchal-Crespo, L. et al. The effect of haptic guidance and visual feedback on learning a complex tennis task. Exp. Brain Res. 231, 277–291 (2013).
Díaz, I., Gil, J. J. & Sánchez, E. Lower-limb robotic rehabilitation: literature review and challenges. J. Robot. 2011, 1–11 (2011).
Na, X. & Cole, D. J. Linear quadratic game and noncooperative predictive methods for potential application to modelling driver-AFS interactive steering control. Veh. Sys. Dyn. 51, 165–198 (2013).
Music, S. & Hirche, S. Control sharing in human-robot team interaction. Annu. Rev. Control 44, 342–354 (2017).
Khoramshahi, M. & Billard, A. A dynamical system approach to task-adaptation in physical human–robot interaction. Auton. Robot. https://doi.org/10.1007/s10514-018-9764-z (2018).
Jarrassé, N., Charalambous, T. & Burdet, E. A framework to describe, analyze and generate interactive motor behaviors. PLoS ONE 7, e49945 (2012).
Starr, A. W. & Ho, Y.-C. Nonzero-sum differential games. J. Optim. Theory Appl. 3, 184–206 (1969).
Basar, T. & Olsder, G. J. Dynamic Noncooperative Game Theory 2nd edn (Society for Industrial and Applied Mathematics, Philadelphia, 1999).
Takagi, A., Ganesh, G., Yoshioka, T., Kawato, M. & Burdet, E. Physically interacting individuals estimate their partner’s movement goal to enhance motor abilities. Nat. Hum. Behav. 1, 0054 (2017).
Kiumarsi, B. et al. Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans. Neur. Netw. Learn. Syst. 29, 2042–2062 (2018).
Marden, J. R., Arslan, G. & Shamma, J. S. Joint strategy fictitious play with inertia for potential games. IEEE Trans. Autom. Contr. 54, 208–220 (2009).
Li, Y., Tee, K. P., Yan, R., Chan, W. L. & Wu, Y. A framework of human-robot coordination based on game theory and policy iteration. IEEE Trans. Robot. 32, 1408–1418 (2016).
Reinkensmeyer, D. J. et al. Computational neurorehabilitation: modeling plasticity and learning to predict recoverys. J. Neuroeng. Rehabil. 13, 1–25 (2016).
Nierhoff, T., Leibrandt, K., Lorenz, T. & Hirche, S. Robotic billiards: understanding humans in order to counter them. IEEE Trans. Cybern. 46, 1889–1899 (2016).
Slotine, J.-J. E. & Li, W. Applied Nonlinear Control (Prentice-Hall, Upper Saddle River, 1991).
Gajic, Z. & Qureshi, M. T. J. Lyapunov Matrix Equation in System Stability and Control (Elsevier, Amsterdam, 1995).
Burdet, E., Franklin, D. W. & Milner, T. E. Human Robotics: Neuromechanics and Motor Control (MIT Press, Cambridge, MA, 2013).
Engwerda, J. Algorithms for computing Nash equilibria in deterministic LQ games. Comput. Manag. Sci. 4, 113–140 (2007).
Evrard, P. & Kheddar, A. Homotopy switching model for dyad haptic interaction in physical collaborative tasks. In Proc. IEEE Worldhaptics 45–50 (2009).
Emken, J. L., Benitez, R., Sideris, A., Bobrow, J. E. & Reinkensmeyer, D. J. Motor adaptation as a greedy optimization of error and effort. J. Neurophysiol. 97, 3997–4006 (2007).
Franklin, D. W. et al. CNS learns stable, accurate, and efficient movements using a simple algorithm. J. Neurosci. 28, 11165–11173 (2008).
Levin, M. F. et al. Deficits in the coordination of agonist and antagonist muscles in stroke patients: implications for normal motor control. Brain Res. 853, 352–369 (2000).
Colgate, J. E. et al. Methods and apparatus for manipulation of heavy payloads with intelligent assist devices. US patent 7185774 (2007).
Zoss, A. B., Kazerooni, H. & Chu, A. Biomechanical design of the Berkeley lower extremity exoskeleton (BLEEX). IEEE-ASME Trans. Mech. 11, 128–138 (2006).
Peshkin, M. A. et al. Cobot architecture. IEEE Trans. Robot. Autom. 17, 377–390 (2001).
Burgar, C. G. et al. Robot-assisted upper-limb therapy in acute rehabilitation setting following stroke: Department of Veterans Affairs multisite clinical trial. J. Rehabil. Res. Dev. 48, 445–458 (2011).
Chackochan, V. T. Development of Collaborative Strategies in Joint Action. PhD thesis, University of Genoa, Italy (2018).
Braun, D. A., Ortega, P. A. & Wolpert, D. M. Nash equilibria in multi-agent motor interactions. PLoS Comput. Biol. 5, e1000468 (2009).
Hogan, N. et al. Motions or muscles? Some behavioral factors underlying robotic assistance of motor recovery. J. Rehabil. Res. Dev. 43, 605 (2006).
Kahn, L. E. et al. Robot-assisted movement training for the stroke-impaired arm: does it matter what the robot does? J. Rehabil. Res. Dev. 43, 619 (2006).
Spong, M. & Vidyasagar, M. Robot Dynamics and Control (Wiley, Hoboken, 1989).
Codourey, A. & Burdet, E. A body-oriented method for finding a linear form of the dynamic equation of fully parallel robots. Proc. IEEE Int. Conf. Robot. 2, 1612–1618 (1997).
Campolo, D. et al. H-Man: a planar, H-shape cabled differential robotic manipulandum for experiments on human motor control. J. Neurosci. Meth. 235, 285–297 (2014).
We thank J. Eden, T. Mylvaganam, N. P. Perez, Q.-C. Pham and K. P. Tee for their careful reading and comments on the manuscript. This research was supported in part by the European Commission grant EU-H2020 COGIMON (644727), UK EPSRC MOTION grant EP/NO29003/1 and Singapore MOE Tier1 grant RG48/17.
The authors declare no competing interests.
Consent to publish identifiable images of research participants was obtained.
We have complied with all relevant ethical regulations.
Guidelines for study procedures were provided by Imperial College London.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Li, Y., Carboni, G., Gonzalez, F. et al. Differential game theory for versatile physical human–robot interaction. Nat Mach Intell 1, 36–43 (2019). https://doi.org/10.1038/s42256-018-0010-3
Nature Machine Intelligence (2019)