Number knowledge can be boosted initially by embodied strategies such as the use of fingers. This Article explores the perceptual process of grounding number symbols in artificial agents, particularly the iCub robot—a child-like humanoid with fully functional, five-fingered hands. It studies the application of convolutional neural network models in the context of cognitive developmental robotics, where the training information is likely to be gradually acquired while operating, rather than being abundant and fully available as in many machine learning scenarios. The experimental analyses show increased efficiency of the training and similarities with studies in developmental psychology. Indeed, the proprioceptive information from the robot hands can improve accuracy in the recognition of spoken digits by supporting a quicker creation of a uniform number line. In conclusion, these findings reveal a novel way for the humanization of artificial training strategies, where the embodiment can make the robot’s learning more efficient and understandable for humans.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Current Robotics Reports Open Access 21 January 2021
Subscribe to Nature+
Get immediate online access to Nature and 55 other Nature journal
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
The data for the models presented in this paper can be found in the GitHub repository: https://github.com/EPSRC-NUMBERS/EmbodiedCNN-Speech. A Supplementary Video of the iCub counting from 1 to 10 is also provided. The Google Tensorflow Speech Command database can be downloaded from http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz.
The source code for the models presented in this paper can be found in the GitHub repository: https://github.com/EPSRC-NUMBERS/EmbodiedCNN-Speech.
Glenberg, A. M. Embodiment as a unifying perspective for psychology. WIREs Cogn. Sci. 1, 586–596 (2010).
Wilson, M. Six views of embodied cognition. Psychon. Bull. Rev. 9, 625–636 (2002).
Pfeifer, R., Bongard, J. & Grand, S. How the Body Shapes the Way We Think: A New View of Intelligence (MIT Press, 2007).
Shapiro, L. The Routledge Handbook of Embodied Cognition (Routledge, 2014).
Dackermann, T., Fischer, U., Nuerk, H. C., Cress, U. & Moeller, K. Applying embodied cognition: from useful interventions and their theoretical underpinnings to practical applications. ZDM Math. Educ. 49, 545–557 (2017).
Nieder, A. The neuronal code for number. Nat. Rev. Neurosci. 17, 366–382 (2016).
Barrow, J. D. New Theories of Everything: The Quest for Ultimate Explanation (Oxford Univ. Press, 2008).
Lakoff, G. & Nuñez, R. Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being (Basic Books, 2000).
Soylu, F., Lester, F. K. Jr. & Newman, S. D. You can count on your fingers: the role of fingers in early mathematical development. J. Numer. Cogn. 4, 107–135 (2018).
Goldin-Meadow, S., Levine, S. C. & Jacobs, S. in Emerging Perspectives on Gesture and Embodiment in Mathematics (eds Edwards, L. D. et al.) 50–64 (Information Age, 2014).
Gunderson, E. A., Spaepen, E., Gibson, D., Goldin-Meadow, S. & Levine, S. C. Gesture as a window onto children’s number knowledge. Cognition 144, 14–28 (2015).
Di Luca, S. & Pesenti, M. Masked priming effect with canonical finger numeral configurations. Exp. Brain Res. 185, 27–39 (2008).
Domahs, F., Kaufmann, L. & Fischer, M. H. Handy Numbers: Finger Counting and Numerical Cognition (Frontiers, 2014).
Alibali, M. W. & DiRusso, A. A. The function of gesture in learning to count: more than keeping track. Cogn. Dev. 14, 37–56 (1999).
Di Luca, S. & Pesenti, M. Finger numeral representations: more than just another symbolic code. Front. Psychol. 2, 272 (2011).
Sixtus, E., Fischer, M. H. & Lindemann, O. Finger posing primes number comprehension. Cogn. Process. 18, 237–248 (2017).
Klein, E., Moeller, K., Willmes, K., Nuerk, H.-C. & Domahs, F. The Influence of implicit hand-based representations on mental arithmetic. Front. Psychol. 2, 197 (2011).
Butterworth, B. The Mathematical Brain (Macmillan, 1999).
Peters, L. & De Smedt, B. Arithmetic in the developing brain: a review of brain imaging studies. Dev. Cogn. Neurosci. 30, 265–279 (2018).
Andres, M., Michaux, N. & Pesenti, M. Common substrate for mental arithmetic and finger representation in the parietal cortex. Neuroimage 62, 1520–1528 (2012).
Kaufmann, L. et al. A developmental fMRI study of nonsymbolic numerical and spatial processing. Cortex 44, 376–385 (2008).
Gracia-Bafalluy, M. & Noël, M.-P. Does finger training increase young children’s numerical performance? Cortex 44, 368–375 (2008).
Sato, M., Cattaneo, L., Rizzolatti, G. & Gallese, V. Numbers within our hands: modulation of corticospinal excitability of hand muscles during numerical judgment. J. Cogn. Neurosci. 19, 684–693 (2007).
Tschentscher, N., Hauk, O., Fischer, M. H. & Pulvermüller, F. You can count on the motor cortex: finger counting habits modulate motor cortex activation evoked by numbers. Neuroimage 59, 3139–3148 (2012).
Alibali, M. W. & Nathan, M. J. Embodiment in mathematics teaching and learning: evidence from learners’ and teachers’ gestures. J. Learn. Sci. 21, 247–286 (2012).
Alibali, M. W. et al. How teachers link ideas in mathematics instruction using speech and gesture: a corpus. Anal. Cogn. Instr. 32, 65–100 (2014).
Cook, S. W. & Goldin-Meadow, S. The role of gesture in learning: do children use their hands to change their minds? J. Cogn. Dev. 7, 211–232 (2006).
Cook, S. W., Duffy, R. G. & Fenn, K. M. Consolidation and transfer of learning after observing hand gesture. Child Dev. 84, 1863–1871 (2013).
Jordan, N. C., Kaplan, D., Ramineni, C. & Locuniak, M. N. Development of number combination skill in the early school years: when do fingers help? Dev. Sci. 11, 662–668 (2008).
Moeller, K., Martignon, L., Wessolowski, S., Engel, J. & Nuerk, H.-C. Effects of finger counting on numerical development—the opposing views of neurocognition and mathematics education. Front. Psychol. 2, 328 (2011).
Cangelosi, A. & Schlesinger, M. Developmental Robotics: From Babies to Robots (MIT Press, 2015).
Sakagami, Y. et al. The intelligent ASIMO: system overview and integration. In IEEE/RSJ International Conference on Intelligent Robots and Systems Vol. 3 (ed. Dodds, G.) 2478–2483 (IEEE, 2002).
Sandini, G., Metta, G. & Vernon, D. in 50 Years of Artificial Intelligence (eds Lungarella, M. et al.) 358–369 (Springer, 2007).
Theodorou, A., Wortham, R. H. & Bryson, J. J. Designing and implementing transparency for real time inspection of autonomous robots. Conn. Sci. 29, 230–241 (2017).
Asada, M. et al. Cognitive developmental robotics: a survey. IEEE Trans. Auton. Ment. Dev. 1, 12–34 (2009).
Cangelosi, A. et al. in Conceptual and Interactive Embodiment: Foundations of Embodied Cognition Vol. 2 (eds Fischer, M. H. & Coello, Y.) 275–293 (Routledge, 2016).
Di Nuovo, A., Marocco, D., Di Nuovo, S. & Cangelosi, A. Autonomous learning in humanoid robotics through mental imagery. Neural Netw. 41, 147–155 (2013).
Di Nuovo, A., Marocco, D., Di Nuovo, S. & Cangelosi, A. in Springer Handbook of Model-Based Science (eds Magnani, L. & Bertolotti, T.) 619–637 (Springer, 2017).
Di Nuovo, A. & Jay, T. The development of numerical cognition in children and artificial systems: a review of the current knowledge and proposals for multi-disciplinary research. IET Cogn. Comput. Syst. 1, 2–11 (2019).
Rucinski, M., Cangelosi, A. & Belpaeme, T. Robotic model of the contribution of gesture to learning to count. In IEEE International Conference on Development and Learning and Epigenetic Robotics (eds Morrison, C. & Nagai, Y.) 1–6 (IEEE, 2012).
De La Cruz, V. M., Di Nuovo, A., Di Nuovo, S. & Cangelosi, A. Making fingers and words count in a cognitive robot. Front. Behav. Neurosci. 8, 13 (2014).
Di Nuovo, A., De La Cruz, V. M. & Cangelosi, A. Grounding fingers, words and numbers in a cognitive developmental robot. In IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (eds Perlovsky, L. et al.) 9–15 (IEEE, 2014).
Di Nuovo, A., De La Cruz, V. M., Cangelosi, A. & Di Nuovo, S. The iCub learns numbers: an embodied cognition study. In International Joint Conference on Neural Networks (ed. Alippi, C.) 692–699 (IEEE, 2014).
Di Nuovo, A., De La Cruz, V. M. & Cangelosi, A. A deep learning neural network for number cognition: a bi-cultural study with the iCub. In IEEE International Conference on Development and Learning and Epigenetic Robotics (ed Meeden, L.) 320–325 (2015).
Di Nuovo, A. An embodied model for handwritten digits recognition in a cognitive robot. In IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (eds Perlovsky, L. et al.) 1–6 (IEEE, 2017).
Zorzi, M., Stoianov, I. & Umiltà, C. in The Handbook of Mathematical Cognition (ed. Campbell, J.) 67–84 (Psychology, 2005).
Di Nuovo, A. Long-short term memory networks for modelling embodied mathematical cognition in robots. In International Joint Conference on Neural Networks (ed. Ludermir, T.) 1–7 (IEEE, 2018).
Domahs, F., Krinzinger, H. & Willmes, K. Mind the gap between both hands: evidence for internal finger-based number representations in children’s mental calculation. Cortex 44, 359–367 (2008).
Davis, S., Tsagarakis, N. G. & Caldwell, D. G. The initial design and manufacturing process of a low cost hand for the robot iCub. In IEEE-RAS International Conference on Humanoid Robots (ed. Oh, J.-H.) 40–45 (IEEE, 2008).
Warden, P. Speech commands: a dataset for limited-vocabulary speech recognition. Preprint at https://arxiv.org/abs/1804.03209 (2018).
Sharif Razavian, A., Azizpour, H., Sullivan, J. & Carlsson, S. CNN features off-the-shelf: an astounding baseline for recognition. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (eds Betke, M. & Davis, J.) 806–813 (IEEE, 2014).
Gallistel, C. R. & Gelman, R. Preverbal and verbal counting and computation. Cognition 44, 43–74 (1992).
Szegedy, C. et al. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition (ed. Mortensen, E.) 1–9 (IEEE, 2015).
Gelman, R. & Gallistel, C. R. The Child’s Understanding of Number (Harvard Univ. Press, 1986).
Piantadosi, S. T. Zipf’s word frequency law in natural language: a critical review and future directions. Psychon. Bull. Rev. 21, 1112–1130 (2014).
Laski, E. V. & Siegler, R. S. Is 27 a big number? Correlational and causal connections among numerical categorization, number line estimation, and numerical magnitude comparison. Child Dev. 78, 1723–1743 (2007).
Gunderson, E. A., Spaepen, E. & Levine, S. C. Approximate number word knowledge before the cardinal principle. J. Exp. Child Psychol. 130, 35–55 (2015).
Tensorflow Speech Recognition Challenge (Kaggle, 2018); https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/leaderboard
Dehaene, S. & Mehler, J. Cross-linguistic regularities in the frequency of number words. Cognition 43, 1–29 (1992).
Stoianov, I., Zorzi, M., Becker, S. & Umilta, C. Associative arithmetic with Boltzmann machines: the role of number representations. In International Conference on Artificial Neural Networks (ed. Dorronsoro, J. R.) 277–283 (Springer, 2002).
Schmitz, A. et al. Design, realization and sensorization of the dexterous iCub hand. In IEEE-RAS International Conference on Humanoid Robots (ed. Wilkes, M.) 186–191 (IEEE, 2010).
Lang, C. E. & Schieber, M. H. Human finger independence: limitations due to passive mechanical coupling versus active neuromuscular control. J. Neurophysiol. 92, 2802–2810 (2004).
Chollet, F. Keras: The Python Deep Learning Library (GitHub respository, 2018); http://keras.io
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (GoogleResearch, 2018); https://www.tensorflow.org
Schmidhuber, J. Deep learning in neural networks: An overview. Neural Networks 61, 85–117 (2015).
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
Fukushima, K. Artificial vision by multi-layered neural networks: neocognitron and its advances. Neural Networks 37, 103–119 (2013).
Scherer, D., Müller, A. & Behnke, S. Evaluation of pooling operations in convolutional srchitectures for object recognition. In International Conference on Artificial Neural Networks (eds Diamantaras, K., Duch, W. & Iliadis, L.) 92–101 (Springer, 2010).
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).
Santurkar, S., Tsipras, D., Ilyas, A. & Madry, A. How does batch normalization help optimization? In Advances in Neural Information Processing Systems (eds Bengio, S. et al.) 2483–2493 (NIPS Foundation, 2018).
He, K., Zhang, X., Ren, S. & Sun, J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In IEEE International Conference on Computer Vision (eds Mortensen, E. & Fidler, S.) 1026–1034 (IEEE, 2015).
Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of Machine Learning Research (eds Teh, Y.W., & Titterington, M.) 249–256 (MLR Press, 2010).
Riedmiller, M. & Braun, H. A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In IEEE International Conference on Neural Networks (ed. Ruspini, E.) 586–591 (IEEE, 1993).
Ruder, S., An overview of gradient descent optimization algorithms. Preprint at https://arxiv.org/abs/1609.04747 (2017).
Kingma, D. P. & Ba, J. L. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980v9 (2017).
A.D.N. acknowledges the support of the EPSRC through project grant no. EP/P030033/1 (NUMBERS). A.D.N. also acknowledges the support of the NVIDIA Corporation with the donation of the GeForce Titan X and the Tesla K40 GPUs used for this research.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Di Nuovo, A., McClelland, J.L. Developing the knowledge of number digits in a child-like robot. Nat Mach Intell 1, 594–605 (2019). https://doi.org/10.1038/s42256-019-0123-3
This article is cited by
Psychological Research (2022)
Current Robotics Reports (2021)