Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Developing the knowledge of number digits in a child-like robot

Nature Machine Intelligence volume 1, pages 594–605 (2019)Cite this article

Subjects

Abstract

Number knowledge can be boosted initially by embodied strategies such as the use of fingers. This Article explores the perceptual process of grounding number symbols in artificial agents, particularly the iCub robot—a child-like humanoid with fully functional, five-fingered hands. It studies the application of convolutional neural network models in the context of cognitive developmental robotics, where the training information is likely to be gradually acquired while operating, rather than being abundant and fully available as in many machine learning scenarios. The experimental analyses show increased efficiency of the training and similarities with studies in developmental psychology. Indeed, the proprioceptive information from the robot hands can improve accuracy in the recognition of spoken digits by supporting a quicker creation of a uniform number line. In conclusion, these findings reveal a novel way for the humanization of artificial training strategies, where the embodiment can make the robot’s learning more efficient and understandable for humans.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Schematics of the artificial neural network architecture.
Fig. 2: Accuracy rate on the test set over epochs.
Fig. 3: Accuracy rate on the test set over epochs.

Similar content being viewed by others

Data availability

The data for the models presented in this paper can be found in the GitHub repository: https://github.com/EPSRC-NUMBERS/EmbodiedCNN-Speech. A Supplementary Video of the iCub counting from 1 to 10 is also provided. The Google Tensorflow Speech Command database can be downloaded from http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz.

Code availability

The source code for the models presented in this paper can be found in the GitHub repository: https://github.com/EPSRC-NUMBERS/EmbodiedCNN-Speech.

References

  1. Glenberg, A. M. Embodiment as a unifying perspective for psychology. WIREs Cogn. Sci. 1, 586–596 (2010).

    Article  Google Scholar 

  2. Wilson, M. Six views of embodied cognition. Psychon. Bull. Rev. 9, 625–636 (2002).

    Article  Google Scholar 

  3. Pfeifer, R., Bongard, J. & Grand, S. How the Body Shapes the Way We Think: A New View of Intelligence (MIT Press, 2007).

  4. Shapiro, L. The Routledge Handbook of Embodied Cognition (Routledge, 2014).

  5. Dackermann, T., Fischer, U., Nuerk, H. C., Cress, U. & Moeller, K. Applying embodied cognition: from useful interventions and their theoretical underpinnings to practical applications. ZDM Math. Educ. 49, 545–557 (2017).

    Article  Google Scholar 

  6. Nieder, A. The neuronal code for number. Nat. Rev. Neurosci. 17, 366–382 (2016).

    Article  Google Scholar 

  7. Barrow, J. D. New Theories of Everything: The Quest for Ultimate Explanation (Oxford Univ. Press, 2008).

  8. Lakoff, G. & Nuñez, R. Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being (Basic Books, 2000).

  9. Soylu, F., Lester, F. K. Jr. & Newman, S. D. You can count on your fingers: the role of fingers in early mathematical development. J. Numer. Cogn. 4, 107–135 (2018).

    Article  Google Scholar 

  10. Goldin-Meadow, S., Levine, S. C. & Jacobs, S. in Emerging Perspectives on Gesture and Embodiment in Mathematics (eds Edwards, L. D. et al.) 50–64 (Information Age, 2014).

  11. Gunderson, E. A., Spaepen, E., Gibson, D., Goldin-Meadow, S. & Levine, S. C. Gesture as a window onto children’s number knowledge. Cognition 144, 14–28 (2015).

    Article  Google Scholar 

  12. Di Luca, S. & Pesenti, M. Masked priming effect with canonical finger numeral configurations. Exp. Brain Res. 185, 27–39 (2008).

    Article  Google Scholar 

  13. Domahs, F., Kaufmann, L. & Fischer, M. H. Handy Numbers: Finger Counting and Numerical Cognition (Frontiers, 2014).

  14. Alibali, M. W. & DiRusso, A. A. The function of gesture in learning to count: more than keeping track. Cogn. Dev. 14, 37–56 (1999).

    Article  Google Scholar 

  15. Di Luca, S. & Pesenti, M. Finger numeral representations: more than just another symbolic code. Front. Psychol. 2, 272 (2011).

    Google Scholar 

  16. Sixtus, E., Fischer, M. H. & Lindemann, O. Finger posing primes number comprehension. Cogn. Process. 18, 237–248 (2017).

    Article  Google Scholar 

  17. Klein, E., Moeller, K., Willmes, K., Nuerk, H.-C. & Domahs, F. The Influence of implicit hand-based representations on mental arithmetic. Front. Psychol. 2, 197 (2011).

    Article  Google Scholar 

  18. Butterworth, B. The Mathematical Brain (Macmillan, 1999).

  19. Peters, L. & De Smedt, B. Arithmetic in the developing brain: a review of brain imaging studies. Dev. Cogn. Neurosci. 30, 265–279 (2018).

    Article  Google Scholar 

  20. Andres, M., Michaux, N. & Pesenti, M. Common substrate for mental arithmetic and finger representation in the parietal cortex. Neuroimage 62, 1520–1528 (2012).

    Article  Google Scholar 

  21. Kaufmann, L. et al. A developmental fMRI study of nonsymbolic numerical and spatial processing. Cortex 44, 376–385 (2008).

    Article  Google Scholar 

  22. Gracia-Bafalluy, M. & Noël, M.-P. Does finger training increase young children’s numerical performance? Cortex 44, 368–375 (2008).

    Article  Google Scholar 

  23. Sato, M., Cattaneo, L., Rizzolatti, G. & Gallese, V. Numbers within our hands: modulation of corticospinal excitability of hand muscles during numerical judgment. J. Cogn. Neurosci. 19, 684–693 (2007).

    Article  Google Scholar 

  24. Tschentscher, N., Hauk, O., Fischer, M. H. & Pulvermüller, F. You can count on the motor cortex: finger counting habits modulate motor cortex activation evoked by numbers. Neuroimage 59, 3139–3148 (2012).

    Article  Google Scholar 

  25. Alibali, M. W. & Nathan, M. J. Embodiment in mathematics teaching and learning: evidence from learners’ and teachers’ gestures. J. Learn. Sci. 21, 247–286 (2012).

    Article  Google Scholar 

  26. Alibali, M. W. et al. How teachers link ideas in mathematics instruction using speech and gesture: a corpus. Anal. Cogn. Instr. 32, 65–100 (2014).

    Article  Google Scholar 

  27. Cook, S. W. & Goldin-Meadow, S. The role of gesture in learning: do children use their hands to change their minds? J. Cogn. Dev. 7, 211–232 (2006).

    Article  Google Scholar 

  28. Cook, S. W., Duffy, R. G. & Fenn, K. M. Consolidation and transfer of learning after observing hand gesture. Child Dev. 84, 1863–1871 (2013).

    Article  Google Scholar 

  29. Jordan, N. C., Kaplan, D., Ramineni, C. & Locuniak, M. N. Development of number combination skill in the early school years: when do fingers help? Dev. Sci. 11, 662–668 (2008).

    Article  Google Scholar 

  30. Moeller, K., Martignon, L., Wessolowski, S., Engel, J. & Nuerk, H.-C. Effects of finger counting on numerical development—the opposing views of neurocognition and mathematics education. Front. Psychol. 2, 328 (2011).

    Article  Google Scholar 

  31. Cangelosi, A. & Schlesinger, M. Developmental Robotics: From Babies to Robots (MIT Press, 2015).

  32. Sakagami, Y. et al. The intelligent ASIMO: system overview and integration. In IEEE/RSJ International Conference on Intelligent Robots and Systems Vol. 3 (ed. Dodds, G.) 2478–2483 (IEEE, 2002).

  33. Sandini, G., Metta, G. & Vernon, D. in 50 Years of Artificial Intelligence (eds Lungarella, M. et al.) 358–369 (Springer, 2007).

  34. Theodorou, A., Wortham, R. H. & Bryson, J. J. Designing and implementing transparency for real time inspection of autonomous robots. Conn. Sci. 29, 230–241 (2017).

    Article  Google Scholar 

  35. Asada, M. et al. Cognitive developmental robotics: a survey. IEEE Trans. Auton. Ment. Dev. 1, 12–34 (2009).

    Article  Google Scholar 

  36. Cangelosi, A. et al. in Conceptual and Interactive Embodiment: Foundations of Embodied Cognition Vol. 2 (eds Fischer, M. H. & Coello, Y.) 275–293 (Routledge, 2016).

  37. Di Nuovo, A., Marocco, D., Di Nuovo, S. & Cangelosi, A. Autonomous learning in humanoid robotics through mental imagery. Neural Netw. 41, 147–155 (2013).

    Article  Google Scholar 

  38. Di Nuovo, A., Marocco, D., Di Nuovo, S. & Cangelosi, A. in Springer Handbook of Model-Based Science (eds Magnani, L. & Bertolotti, T.) 619–637 (Springer, 2017).

  39. Di Nuovo, A. & Jay, T. The development of numerical cognition in children and artificial systems: a review of the current knowledge and proposals for multi-disciplinary research. IET Cogn. Comput. Syst. 1, 2–11 (2019).

    Google Scholar 

  40. Rucinski, M., Cangelosi, A. & Belpaeme, T. Robotic model of the contribution of gesture to learning to count. In IEEE International Conference on Development and Learning and Epigenetic Robotics (eds Morrison, C. & Nagai, Y.) 1–6 (IEEE, 2012).

  41. De La Cruz, V. M., Di Nuovo, A., Di Nuovo, S. & Cangelosi, A. Making fingers and words count in a cognitive robot. Front. Behav. Neurosci. 8, 13 (2014).

    Google Scholar 

  42. Di Nuovo, A., De La Cruz, V. M. & Cangelosi, A. Grounding fingers, words and numbers in a cognitive developmental robot. In IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (eds Perlovsky, L. et al.) 9–15 (IEEE, 2014).

  43. Di Nuovo, A., De La Cruz, V. M., Cangelosi, A. & Di Nuovo, S. The iCub learns numbers: an embodied cognition study. In International Joint Conference on Neural Networks (ed. Alippi, C.) 692–699 (IEEE, 2014).

  44. Di Nuovo, A., De La Cruz, V. M. & Cangelosi, A. A deep learning neural network for number cognition: a bi-cultural study with the iCub. In IEEE International Conference on Development and Learning and Epigenetic Robotics (ed Meeden, L.) 320–325 (2015).

  45. Di Nuovo, A. An embodied model for handwritten digits recognition in a cognitive robot. In IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (eds Perlovsky, L. et al.) 1–6 (IEEE, 2017).

  46. Zorzi, M., Stoianov, I. & Umiltà, C. in The Handbook of Mathematical Cognition (ed. Campbell, J.) 67–84 (Psychology, 2005).

  47. Di Nuovo, A. Long-short term memory networks for modelling embodied mathematical cognition in robots. In International Joint Conference on Neural Networks (ed. Ludermir, T.) 1–7 (IEEE, 2018).

  48. Domahs, F., Krinzinger, H. & Willmes, K. Mind the gap between both hands: evidence for internal finger-based number representations in children’s mental calculation. Cortex 44, 359–367 (2008).

    Article  Google Scholar 

  49. Davis, S., Tsagarakis, N. G. & Caldwell, D. G. The initial design and manufacturing process of a low cost hand for the robot iCub. In IEEE-RAS International Conference on Humanoid Robots (ed. Oh, J.-H.) 40–45 (IEEE, 2008).

  50. Warden, P. Speech commands: a dataset for limited-vocabulary speech recognition. Preprint at https://arxiv.org/abs/1804.03209 (2018).

  51. Sharif Razavian, A., Azizpour, H., Sullivan, J. & Carlsson, S. CNN features off-the-shelf: an astounding baseline for recognition. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (eds Betke, M. & Davis, J.) 806–813 (IEEE, 2014).

  52. Gallistel, C. R. & Gelman, R. Preverbal and verbal counting and computation. Cognition 44, 43–74 (1992).

    Article  Google Scholar 

  53. Szegedy, C. et al. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition (ed. Mortensen, E.) 1–9 (IEEE, 2015).

  54. Gelman, R. & Gallistel, C. R. The Child’s Understanding of Number (Harvard Univ. Press, 1986).

  55. Piantadosi, S. T. Zipf’s word frequency law in natural language: a critical review and future directions. Psychon. Bull. Rev. 21, 1112–1130 (2014).

    Article  Google Scholar 

  56. Laski, E. V. & Siegler, R. S. Is 27 a big number? Correlational and causal connections among numerical categorization, number line estimation, and numerical magnitude comparison. Child Dev. 78, 1723–1743 (2007).

    Article  Google Scholar 

  57. Gunderson, E. A., Spaepen, E. & Levine, S. C. Approximate number word knowledge before the cardinal principle. J. Exp. Child Psychol. 130, 35–55 (2015).

    Article  Google Scholar 

  58. Tensorflow Speech Recognition Challenge (Kaggle, 2018); https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/leaderboard

  59. Dehaene, S. & Mehler, J. Cross-linguistic regularities in the frequency of number words. Cognition 43, 1–29 (1992).

    Article  Google Scholar 

  60. Stoianov, I., Zorzi, M., Becker, S. & Umilta, C. Associative arithmetic with Boltzmann machines: the role of number representations. In International Conference on Artificial Neural Networks (ed. Dorronsoro, J. R.) 277–283 (Springer, 2002).

  61. Schmitz, A. et al. Design, realization and sensorization of the dexterous iCub hand. In IEEE-RAS International Conference on Humanoid Robots (ed. Wilkes, M.) 186–191 (IEEE, 2010).

  62. Lang, C. E. & Schieber, M. H. Human finger independence: limitations due to passive mechanical coupling versus active neuromuscular control. J. Neurophysiol. 92, 2802–2810 (2004).

    Article  Google Scholar 

  63. Chollet, F. Keras: The Python Deep Learning Library (GitHub respository, 2018); http://keras.io

  64. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (GoogleResearch, 2018); https://www.tensorflow.org

  65. Schmidhuber, J. Deep learning in neural networks: An overview. Neural Networks 61, 85–117 (2015).

    Article  Google Scholar 

  66. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  67. Fukushima, K. Artificial vision by multi-layered neural networks: neocognitron and its advances. Neural Networks 37, 103–119 (2013).

    Article  Google Scholar 

  68. Scherer, D., Müller, A. & Behnke, S. Evaluation of pooling operations in convolutional srchitectures for object recognition. In International Conference on Artificial Neural Networks (eds Diamantaras, K., Duch, W. & Iliadis, L.) 92–101 (Springer, 2010).

  69. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).

    MathSciNet  MATH  Google Scholar 

  70. Santurkar, S., Tsipras, D., Ilyas, A. & Madry, A. How does batch normalization help optimization? In Advances in Neural Information Processing Systems (eds Bengio, S. et al.) 2483–2493 (NIPS Foundation, 2018).

  71. He, K., Zhang, X., Ren, S. & Sun, J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In IEEE International Conference on Computer Vision (eds Mortensen, E. & Fidler, S.) 1026–1034 (IEEE, 2015).

  72. Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of Machine Learning Research (eds Teh, Y.W., & Titterington, M.) 249–256 (MLR Press, 2010).

  73. Riedmiller, M. & Braun, H. A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In IEEE International Conference on Neural Networks (ed. Ruspini, E.) 586–591 (IEEE, 1993).

  74. Ruder, S., An overview of gradient descent optimization algorithms. Preprint at https://arxiv.org/abs/1609.04747 (2017).

  75. Kingma, D. P. & Ba, J. L. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980v9 (2017).

Download references

Acknowledgements

A.D.N. acknowledges the support of the EPSRC through project grant no. EP/P030033/1 (NUMBERS). A.D.N. also acknowledges the support of the NVIDIA Corporation with the donation of the GeForce Titan X and the Tesla K40 GPUs used for this research.

Author information

Authors and Affiliations

  1. Sheffield Robotics, Department of Computing, Sheffield Hallam University, Sheffield, UK

    Alessandro Di Nuovo

  2. Department of Psychology, Center for Mind, Brain and Computation, Stanford University, Stanford, CA, USA

    James L. McClelland

Authors
  1. Alessandro Di Nuovo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James L. McClelland
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.D.N. conceptualized the experiment, developed the methodology and designed the baseline artificial neural network architecture. A.D.N. and J.L.M. collaborated on the design of the embodied model. A.D.N. implemented the source code, ran the simulations, validated the results and wrote the first draft of the article. J.L.M. provided relevant ideas from cognitive psychology and neuroscience and contributed to the discussion.

Corresponding author

Correspondence to Alessandro Di Nuovo.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Di Nuovo, A., McClelland, J.L. Developing the knowledge of number digits in a child-like robot. Nat Mach Intell 1, 594–605 (2019). https://doi.org/10.1038/s42256-019-0123-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-019-0123-3

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics