Letter | Published:

Robots that can adapt like animals

Nature volume 521, pages 503507 (28 May 2015) | Download Citation

Subjects

Abstract

Robots have transformed many industries, most notably manufacturing1, and have the power to deliver tremendous benefits to society, such as in search and rescue2, disaster response3, health care4 and transportation5. They are also invaluable tools for scientific exploration in environments inaccessible to humans, from distant planets6 to deep oceans7. A major obstacle to their widespread adoption in more complex environments outside factories is their fragility6,8. Whereas animals can quickly adapt to injuries, current robots cannot ‘think outside the box’ to find a compensatory behaviour when they are damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes9, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots6,8. A promising approach to reducing robot fragility involves having robots learn appropriate behaviours in response to damage10,11, but current techniques are slow even with small, constrained search spaces12. Here we introduce an intelligent trial-and-error algorithm that allows robots to adapt to damage in less than two minutes in large search spaces without requiring self-diagnosis or pre-specified contingency plans. Before the robot is deployed, it uses a novel technique to create a detailed map of the space of high-performing behaviours. This map represents the robot’s prior knowledge about what behaviours it can perform and their value. When the robot is damaged, it uses this prior knowledge to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a behaviour that compensates for the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new algorithm will enable more robust, effective, autonomous robots, and may shed light on the principles that animals use to adapt to injury.

  • Subscribe to Nature for full access:

    $199

    Subscribe

Additional access options:

Already a subscriber?  Log in  now or  Register  for online access.

References

  1. 1.

    & Springer Handbook of Robotics (Springer, 2008).

  2. 2.

    Trial by fire. Robot. Automat. Mag. 11, 50–61 (2004).

  3. 3.

    et al. Emergency response to the nuclear accident at the Fukushima Daiichi nuclear power plants using mobile rescue robots. J. Field Robot. 30, 44–63 (2013).

  4. 4.

    , & Acceptance of healthcare robots for the older population: review and future directions. Int. J. Social Robot. 1, 319–330 (2009).

  5. 5.

    et al. Stanley: the robot that won the DARPA grand challenge. J. Field Robot. 23, 661–692 (2006).

  6. 6.

    Mars rover Spirit (2003–10). Nature 463, 600 (2010).

  7. 7.

    & in Springer Handbook of Robotics (eds & ) 987–1008 (Springer, 2008).

  8. 8.

    & How UGVs physically fail in the field. IEEE Trans. Robot. 21, 423–437 (2005).

  9. 9.

    , & Diagnosis and Fault-Tolerant Control (Springer, 2006).

  10. 10.

    , , & Learning to move in modular robots using central pattern generators and online optimization. Int. J. Robot. Res. 27, 423–443 (2008).

  11. 11.

    , & A distributed and morphology-independent strategy for adaptive locomotion in self-reconfigurable modular robots. Robot. Auton. Syst. 61, 1021–1035 (2013).

  12. 12.

    , & Reinforcement learning in robotics: a survey. Int. J. Robot. Res. 32, 1238–1274 (2013).

  13. 13.

    , , & Real-time fault diagnosis. Robot. Automat. Mag. 11, 56–66 (2004).

  14. 14.

    , & Resilient machines through continuous self-modeling. Science 314, 1118–1121 (2006).

  15. 15.

    & Apollo 13 (Mariner Books, 2006).

  16. 16.

    et al. Kinematic and kinetic analysis of dogs during trotting after amputation of a thoracic limb. Am. J. Vet. Res. 74, 1155–1163 (2013).

  17. 17.

    , , & Ground reaction force adaptations to tripedal locomotion in dogs. Vet. J. 201, 307–315 (2014).

  18. 18.

    , , & A survey of robot learning from demonstration. Robot. Auton. Syst. 57, 469–483 (2009).

  19. 19.

    , & Perspective and problems in motor learning. Trends Cogn. Sci. 5, 487–494 (2001).

  20. 20.

    Postural hand synergies for tool use. J. Neurosci. 18, 10105–10115 (1998).

  21. 21.

    & Gaussian Processes for Machine Learning (MIT Press, 2006).

  22. 22.

    Bayesian Approach to Global Optimization: Theory and Applications (Kluwer Academic, 2013).

  23. 23.

    & Bayesian optimization explains human active search. Adv. Neural Inform. Process. Syst. 26, 55–63 (2013).

  24. 24.

    The motor infrastructure: from ion channels to neuronal networks. Nature Rev. Neurosci. 4, 573–586 (2003).

  25. 25.

    & Innovative problem solving by wild spotted hyenas. Proc. R. Soc. Lond. B 279, 4087–4095 (2012).

  26. 26.

    , , & Probabilistic brains: knowns and unknowns. Nature Neurosci. 16, 1170–1178 (2013).

  27. 27.

    & Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).

  28. 28.

    , , , & How sleep affects the developmental learning of bird song. Nature 433, 710–716 (2005).

  29. 29.

    , , , & Sleep inspires insight. Nature 427, 352–355 (2004).

  30. 30.

    Control of mental activities by internal models in the cerebellum. Nature Rev. Neurosci. 9, 304–313 (2008).

  31. 31.

    & Policy gradient reinforcement learning for fast quadrupedal locomotion. In Proc. IEEE Int. Conf. on ‘Robotics and Automation’ (ICRA) 2619–2624 (IEEE, 2004).

  32. 32.

    & Automatic gait optimization with Gaussian process regression. In Proc. Int. Joint Conf. on ‘Artificial Intelligence’ (IJCAI) 944–949 (2007).

  33. 33.

    & Using response surfaces and expected improvement to optimize snake robot gait parameters. In Proc. IEEE/RSJ Int. Conf. on ‘Intelligent Robots and Systems (IROS)’ 1069–1074 (IEEE, 2011).

  34. 34.

    , & An experimental comparison of bayesian optimization for bipedal locomotion. In Proc. IEEE Int. Conf. on ‘Robotics and Automation’ (ICRA) 1951–1958 (IEEE, 2014).

  35. 35.

    & Illuminating search spaces by mapping elites. Preprint at (2015).

Download references

Acknowledgements

We thank L. Tedesco, S. Doncieux, N. Bredeche, S. Whiteson, R. Calandra, J. Droulez, P. Bessière, F. Lesaint, C. Thurat, S. Ivaldi, C. Lan Sun Luk, J. Li, J. Huizinga, R. Velez, H. Mengistu, M. Norouzzadeh, T. Clune, and A. Nguyen for feedback and discussions. This work has been funded by the ANR Creadapt project (ANR-12-JS03-0009), the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement number 637972), and a Direction Générale de l’Armement (DGA) scholarship to A.C.

Author information

Author notes

    • Danesh Tarapore
    •  & Jean-Baptiste Mouret

    Present addresses: Department of Electronics, University of York, York YO10 5DD, UK (D.T.); Inria, Villers-lès-Nancy, F-54600, France (J.-B.M.)

Affiliations

  1. Sorbonne Universités, Université Pierre et Marie Curie (UPMC), Paris 06, UMR 7222, Institut des Systèmes Intelligents et de Robotique (ISIR), F-75005, Paris, France

    • Antoine Cully
    • , Danesh Tarapore
    •  & Jean-Baptiste Mouret
  2. CNRS, UMR 7222, Institut des Systèmes Intelligents et de Robotique (ISIR), F-75005, Paris, France

    • Antoine Cully
    • , Danesh Tarapore
    •  & Jean-Baptiste Mouret
  3. Department of Computer Science, University of Wyoming, Laramie, Wyoming 82071, USA

    • Jeff Clune
  4. Inria, Team Larsen, Villers-lès-Nancy, F-54600, France

    • Jean-Baptiste Mouret
  5. CNRS, Loria, UMR 7503, Vandœuvre-lès-Nancy, F-54500, France

    • Jean-Baptiste Mouret
  6. Université de Lorraine, Loria, UMR 7503, Vandœuvre-lès-Nancy, F-54500, France

    • Jean-Baptiste Mouret

Authors

  1. Search for Antoine Cully in:

  2. Search for Jeff Clune in:

  3. Search for Danesh Tarapore in:

  4. Search for Jean-Baptiste Mouret in:

Contributions

A.C. and J.-B. M. designed the study. A.C. and D.T. performed the experiments. A.C., J.-B.M., D.T. and J.C. analysed the results, discussed additional experiments, and wrote the paper.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Jean-Baptiste Mouret.

Extended data

Supplementary information

PDF files

  1. 1.

    Supplementary Information

    This file contains Supplementary Methods, Supplementary Experiments 1 to 5, full captions for Supplementary Videos 1-2, and Supplementary References.

Videos

  1. 1.

    Damage Recovery in Robots via Intelligent Trial and Error

    This video shows the Intelligent Trial and Error Algorithm in action on the two experimental robots in this paper: a hexapod robot and a robotic arm (Fig. 3). The video shows several examples of the different types of behaviours that are produced during the behaviour-performance map creation step, from classic hexapod gaits to more unexpected forms of locomotion. Then, it shows how the hexapod robot uses that behaviour-performance map to adapt to damage that deprives one of its leg of power (Fig. 3a:C3). The video also illustrates how the Intelligent Trial and Error Algorithm also finds a compensatory behaviour for the robot arm. Finally, adaptation to a second damage condition is shown for both the hexapod and robotic arm.

  2. 2.

    A Behavior-Performance Map Containing Many Different Types of Walking Gaits.

    In the behavior-performance map creation step, the MAP-Elites algorithm produces a diverse collection of different types of walking gaits. The video shows several examples of the different types of behaviors that are produced, from classic hexapod gaits to more unexpected forms of locomotion.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nature14422

Rights and permissions

To obtain permission to re-use content from this article visit RightsLink.

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.