Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Lessons from infant learning for unsupervised machine learning

Abstract

The desire to reduce the dependence on curated, labeled datasets and to leverage the vast quantities of unlabeled data has triggered renewed interest in unsupervised (or self-supervised) learning algorithms. Despite improved performance due to approaches such as the identification of disentangled latent representations, contrastive learning and clustering optimizations, unsupervised machine learning still falls short of its hypothesized potential as a breakthrough paradigm enabling generally intelligent systems. Inspiration from cognitive (neuro)science has been based mostly on adult learners with access to labels and a vast amount of prior knowledge. To push unsupervised machine learning forward, we argue that developmental science of infant cognition might hold the key to unlocking the next generation of unsupervised learning approaches. We identify three crucial factors enabling infants’ quality and speed of learning: (1) babies’ information processing is guided and constrained; (2) babies are learning from diverse, multimodal inputs; and (3) babies’ input is shaped by development and active learning. We assess the extent to which these insights from infant learning have already been exploited in machine learning, examine how closely these implementations resemble the core insights, and propose how further adoption of these factors can give rise to previously unseen performance levels in unsupervised learning.

This is a preview of subscription content, access via your institution

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: Schematic overview of the connectivity within biological and artificial neural networks.
Fig. 2: Typical ANN training versus infant development.

References

  1. Bengio, Y., Lamblin, P., Popovici, D. & Larochelle, H. Greedy layer-wise training of deep networks. In Proc. Advances in Neural Information Processing Systems Vol. 19 (eds. Schölkopf, B., Platt, J. & Hoffman, T.) 153–160 (NIPS, 2006).

  2. Hinton, G. E., Osindero, S. & Teh, Y.-W. A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006).

    Article  MathSciNet  MATH  Google Scholar 

  3. Baldi, P. Autoencoders, unsupervised learning and deep architectures. In Proc. ICML Workshop on Unsupervised and Transfer Learning (eds. Guyon, I., Dror, G., Lemaire, V., Taylor, G. & Silver, D.) 37–49 (JMLR, 2012).

  4. Bengio, Y., Courville, A. & Vincent, P. Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013).

    Article  Google Scholar 

  5. Erhan, D. et al. Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11, 625–660 (2010).

    MathSciNet  MATH  Google Scholar 

  6. Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115, 211–252 (2015).

    Article  MathSciNet  Google Scholar 

  7. Carreira, J. & Zisserman, A. Quo vadis, action recognition? A new model and the kinetics dataset. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 6299–6308 (IEEE, 2017).

  8. Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 3431–3440 (IEEE, 2015).

  9. Ren, S., He, K., Girshick, R. & Sun, J. Faster R-CNN: towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems Vol. 28 (eds. Cortes, C., Lawrence, N., Lee, D., Sugiyama, M. & Garnett, R.) 91–99 (NIPS, 2015).

  10. He, K., Girshick, R. & Dollár, P. Rethinking ImageNet pre-training. In Proc. IEEE International Conference on Computer Vision 4918–4927 (IEEE, 2019).

  11. Huh, M., Agrawal, P. & Efros, A. A. What makes ImageNet good for transfer learning? Preprint at https://arxiv.org/abs/1608.08614 (2016).

  12. Recht, B., Roelofs, R., Schmidt, L. & Shankar, V. Do ImageNet classifiers generalize to imagenet? In Proc. 36th International Conference on Machine Learning (eds. Chaudhuri, K. & Salakhutdinov, R.) 5389–5400 (PMLR, 2019).

  13. Burgess, C. P. et al. Understanding disentangling in β-VAE. Preprint at https://arxiv.org/abs/1804.03599 (2018).

  14. Caron, M., Bojanowski, P., Joulin, A. & Douze, M. Deep clustering for unsupervised learning of visual features. In Proc. European Conference on Computer Vision (eds. Ferrari, V., Hebert, M. I., Sminchisescu, C. & Weiss, Y.) 132–149 (Springer, 2018).

  15. Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In Proc. 37th International Conference on Machine Learning (eds. Daumé, H. III & Singh, A.) 1597–1607 (PMLR, 2020).

  16. Zbontar, J., Jing, L., Misra, I., LeCun, Y. & Deny, S. Barlow twins: self-supervised learning via redundancy reduction. In Proc. 38th International Conference on Machine Learning (eds. Meila, M. & Zhang, T.) 12310–12320 (PMLR, 2021).

  17. Ma, W. J. & Peters, B. A neural network walks into a lab: towards using deep nets as models for human behavior. Preprint at https://arxiv.org/abs/2005.02181 (2020).

  18. Yamins, D. L. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19, 356–365 (2016).

    Article  Google Scholar 

  19. Macpherson, T. et al. Natural and artificial intelligence: a brief introduction to the interplay between AI and neuroscience research. Neural Networks 144, 603–613 (2021).

    Article  Google Scholar 

  20. Rosenblatt, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386–408 (1958).

    Article  Google Scholar 

  21. McCulloch, W. S. & Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943).

    Article  MathSciNet  MATH  Google Scholar 

  22. Hassabis, D., Kumaran, D., Summerfield, C. & Botvinick, M. Neuroscience-inspired artificial intelligence. Neuron 95, 245–258 (2017).

    Article  Google Scholar 

  23. Lake, B. M., Ullman, T. D., Tenenbaum, J. B. & Gershman, S. J. Building machines that learn and think like people. Behav. Brain Sci. 40, e253 (2017).

    Article  Google Scholar 

  24. Saxe, A., Nelli, S. & Summerfield, C. If deep learning is the answer, then what is the question? Nat. Rev. Neurosci. 22, 55–67 (2021).

    Article  Google Scholar 

  25. Sinz, F. H., Pitkow, X., Reimer, J., Bethge, M. & Tolias, A. S. Engineering a less artificial intelligence. Neuron 103, 967–979 (2019).

    Article  Google Scholar 

  26. Zador, A. M. A critique of pure learning and what artificial neural networks can learn from animal brains. Nat. Commun. 10, 3770 (2019).

    Article  Google Scholar 

  27. Bengio, Y. Deep learning of representations for unsupervised and transfer learning. In Proc. ICML Workshop on Unsupervised and Transfer Learning (eds. Guyon, I., Dror, G., Lemaire, V., Taylor, G. & Silver, D.) 17–36 (JMLR, 2012).

  28. Cangelosi, A. & Schlesinger, M. Developmental Robotics: From Babies to Robots (MIT Press, 2015).

  29. Kidd, C. How to know. In Proc. 33rd Conference on Neural Information Processing Systems (NIPS, 2019).

  30. Gopnik, A. An AI that knows the world like children do. Sci. Am. Mind 28, 21–28 (2017).

    Article  Google Scholar 

  31. Kosoy, E. et al. Exploring exploration: comparing children with RL agents in unified environments. Preprint at https://arxiv.org/abs/2005.02880 (2020).

  32. Smith, L. B. & Breazeal, C. The dynamic lift of developmental process. Dev. Sci. 10, 61–68 (2007).

    Article  Google Scholar 

  33. Smith, L. B. & Slone, L. K. A developmental approach to machine learning? Front. Psychol. 8, 2124 (2017).

    Article  Google Scholar 

  34. Tenenbaum, J. B., Kemp, C., Griffiths, T. L. & Goodman, N. D. How to grow a mind: statistics, structure and abstraction. Science 331, 1279–1285 (2011).

    Article  MathSciNet  MATH  Google Scholar 

  35. Adolph, K. E., Hoch, J. E. & Cole, W. G. Development (of walking): 15 suggestions. Trends Cogn. Sci. 22, 699–711 (2018).

    Article  Google Scholar 

  36. Byrge, L., Sporns, O. & Smith, L. B. Developmental process emerges from extended brain-body-behavior networks. Trends Cogn. Sci. 18, 395–403 (2014).

    Article  Google Scholar 

  37. Hunnius, S. Early cognitive development: five lessons from infant learning. In Oxford Research Encyclopedia of Psychology (ed. Braddick, O.) (Oxford Univ. Press, in the press).

  38. Karmiloff-Smith, A. An alternative to domain-general or domain-specific frameworks for theorizing about human evolution and ontogenesis. AIMS Neurosci. 2, 91–104 (2015).

    Article  Google Scholar 

  39. von Hofsten, C. & Rosander, K. The development of sensorimotor intelligence in infants. Adv. Child Dev. Behav. 55, 73–106 (2018).

    Article  Google Scholar 

  40. Dunsworth, H. M., Warrener, A. G., Deacon, T., Ellison, P. T. & Pontzer, H. Metabolic hypothesis for human altriciality. Proc. Natl Acad. Sci. USA 109, 15212–15216 (2012).

    Article  Google Scholar 

  41. Haeusler, M. et al. The obstetrical dilemma hypothesis: there’s life in the old dog yet. Biol. Rev. 96, 2031–2057 (2021).

    Article  Google Scholar 

  42. Bethlehem, R. A. et al. Brain charts for the human lifespan. Nature 604, 525–533 (2022).

    Article  Google Scholar 

  43. Huttenlocher, P. R. et al. Synaptic density in human frontal cortex-developmental changes and effects of aging. Brain Res. 163, 195–205 (1979).

    Article  Google Scholar 

  44. Deoni, S. C. et al. Mapping infant brain myelination with magnetic resonance imaging. J. Neurosci. 31, 784–791 (2011).

    Article  Google Scholar 

  45. Hill, J. et al. A surface-based analysis of hemispheric asymmetries and folding of cerebral cortex in term-born human infants. J. Neurosci. 30, 2268–2276 (2010).

    Article  Google Scholar 

  46. Clouchoux, C. et al. Quantitative in vivo MRI measurement of cortical development in the fetus. Brain Struct. Funct. 217, 127–139 (2012).

    Article  Google Scholar 

  47. Cabral, L., Zubiaurre, L., Wild, C., Linke, A. & Cusack, R. Category-selective visual regions have distinctive signatures of connectivity in early infancy. Prerpint at https://www.biorxiv.org/content/10.1101/675421v2.full (2019).

  48. Doria, V. et al. Emergence of resting state networks in the preterm human brain. Proc. Natl Acad. Sci. USA 107, 20015–20020 (2010).

    Article  Google Scholar 

  49. Kamps, F. S., Hendrix, C. L., Brennan, P. A. & Dilks, D. D. Connectivity at the origins of domain specificity in the cortical face and place networks. Proc. Natl Acad. Sci. USA 117, 6163–6169 (2020).

    Article  Google Scholar 

  50. Cusack, R., Wild, C. J., Zubiaurre-Elorza, L. & Linke, A. C. Why does language not emerge until the second year? Hearing Res. 366, 75–81 (2018).

    Article  Google Scholar 

  51. Deen, B. et al. Organization of high-level visual cortex in human infants. Nat. Commun. 8, 13995 (2017).

    Article  Google Scholar 

  52. Ellis, C. T. et al. Evidence of hippocampal learning in human infants. Curr. Biol. 31, 3358–3364 (2021).

    Article  Google Scholar 

  53. Ellis, C. T., Skalaban, L. J., Yates, T. S. & Turk-Browne, N. B. Attention recruits frontal cortex in human infants. Proc. Natl Acad. Sci. USA 118, e2021474118 (2021).

    Article  Google Scholar 

  54. Raz, G. & Saxe, R. Learning in infancy is active, endogenously motivated, and depends on the prefrontal cortices. Annu. Rev. Dev. Psychol. 2, 247–268 (2020).

    Article  Google Scholar 

  55. Linke, A. C. et al. Disruption to functional networks in neonates with perinatal brain injury predicts motor skills at 8 months. NeuroImage Clin. 18, 399–406 (2018).

    Article  Google Scholar 

  56. Marcus, G. F., Vijayan, S., Rao, S. B. & Vishton, P. M. Rule learning by seven-month-old infants. Science 283, 77–80 (1999).

    Article  Google Scholar 

  57. Elman, J. L. Finding structure in time. Cogn. Sci. 14, 179–211 (1990).

    Article  Google Scholar 

  58. Alhama, R. G. & Zuidema, W. Pre-wiring and pre-training: what does a neural network need to learn truly general identity rules? J. Artif. Intell. Res. 61, 927–946 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  59. Jeffress, L. A. A place theory of sound localization. J. Comp. Physiol. Psychol. 41, 35–39 (1948).

    Article  Google Scholar 

  60. Jaeger, H. The ‘Echo State’ Approach to Analysing and Training Recurrent Neural Networks—With an Erratum Note. Technical Report 148, 13 (German National Research Center for Information Technology (GMD), 2001).

  61. Smith, L. B. Do infants possess innate knowledge structures? The con side. Dev. Sci. 2, 133–144 (1999).

    Article  Google Scholar 

  62. Spelke, E. Initial knowledge: six suggestions. Cognition 50, 431–445 (1995).

    Article  Google Scholar 

  63. Stahl, A. E. & Feigenson, L. Observing the unexpected enhances infants’ learning and exploration. Science 348, 91–94 (2015).

    Article  Google Scholar 

  64. Simion, F., Di Giorgio, E., Leo, I. & Bardi, L. The processing of social stimuli in early infancy: from faces to biological motion perception. In Progress in Brain Research Vol. 189 (eds. Braddick, O., Atkinson, J. & Innocenti, G. M.) 173–193 (Elsevier, 2011).

  65. Reynolds, G. D. & Roth, K. C. The development of attentional biases for faces in infancy: a developmental systems perspective. Front. Psychol. 9, 222 (2018).

    Article  Google Scholar 

  66. Viola Macchi, C., Turati, C. & Simion, F. Can a nonspecific bias toward top-heavy patterns explain newborns’ face preference? Psychol. Sci. 15, 379–383 (2004).

    Article  Google Scholar 

  67. Chien, S. H.-L. No more top-heavy bias: Infants and adults prefer upright faces but not top-heavy geometric or face-like patterns. J. Vision 11, 13 (2011).

    Article  Google Scholar 

  68. Ichikawa, H., Tsuruhara, A., Kanazawa, S. & Yamaguchi, M. K. Two- to three-month-old infants prefer moving face patterns to moving top-heavy patterns. Jap. Psychol. Res. 55, 254–263 (2013).

    Article  Google Scholar 

  69. Cooper, R. P. & Aslin, R. N. Preference for infant-directed speech in the first month after birth. Child Dev. 61, 1584–1595 (1990).

    Article  Google Scholar 

  70. Peña, M. et al. Sounds and silence: an optical topography study of language recognition at birth. Proc. Natl Acad. Sci. USA 100, 11702–11705 (2003).

    Article  Google Scholar 

  71. Vouloumanos, A. & Werker, J. F. Listening to language at birth: evidence for a bias for speech in neonates. Dev. Sci. 10, 159–164 (2007).

    Article  Google Scholar 

  72. Mély, D. A., Linsley, D. & Serre, T. Complementary surrounds explain diverse contextual phenomena across visual modalities. Psychol. Rev. 125, 769 (2018).

    Article  Google Scholar 

  73. Linsley, D., Kim, J., Ashok, A. & Serre, T. Recurrent neural circuits for contour detection. In Proc. 8th International Conference on Learning Representations (ICLR, 2020).

  74. Michalski, R. S. in Machine Learning 83–134 (Morgan Kaufmann, 1983).

  75. Mitchell, T. The Need for Biases in Learning Generalizations. Rutgers Computer Science Technical Report cbm-tr-117 (Rutgers University, 1980).

  76. Feinman, R. & Lake, B. M. Learning inductive biases with simple neural networks. In Proc. 40th Annual Meeting of the Cognitive Science Society (eds. Kalish, C., Rau, M. A., Zhu, X. & Rogers, T. T.) (CSS, 2018).

  77. Kopparti, R. M. & Weyde, T. Weight priors for learning identity relations. In Advances in Neural Information Processing Systems Vol. 33 (eds. Wallach, H. M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F. E. Fox, A. & Garnett, R.) (NIPS, 2020).

  78. Weyde, T. & Kopparti, R. M. Modelling identity rules with neural networks. J. Appl. Logics 6, 745–769 (2019).

    MathSciNet  Google Scholar 

  79. Ullman, S., Harari, D. & Dorfman, N. From simple innate biases to complex visual concepts. Proc. Natl Acad. Sci. USA 109, 18215–18220 (2012).

    Article  Google Scholar 

  80. Felleman, D. J. & Van Essen, D. C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991).

    Article  Google Scholar 

  81. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012).

    Google Scholar 

  82. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016).

  83. Szegedy, C. et al. Going deeper with convolutions. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 1–9 (IEEE, 2015).

  84. Besold, T. R. et al. Neural-symbolic learning and reasoning: a survey and interpretation. In Neuro-Symbolic Artificial Intelligence: The State of the Art (eds. Hitzler, P. & Sarker, M. K.) 1–51 (IOS Press, 2021).

  85. d’Avila Garcez, A. S. & Gabbay, D. M. Fibring neural networks. In Proc. Nineteenth National Conference on Artificial Intelligence, Sixteenth Conference on Innovative Applications of Artificial Intelligence (eds. McGuinness, D. L. & Ferguson, G.) 342–347 (AAAI Press/MIT Press, 2004).

  86. Saffran, J. R. & Kirkham, N. Z. Infant statistical learning. Annu. Rev. Psychol. 69, 181–203 (2018).

    Article  Google Scholar 

  87. Teinonen, T., Fellman, V., Näätänen, R., Alku, P. & Huotilainen, M. Statistical language learning in neonates revealed by event-related brain potentials. BMC Neurosci. 10, 21 (2009).

    Article  Google Scholar 

  88. Jacquey, L., Fagard, J., Esseily, R. & O’Regan, J. K. Detection of sensorimotor contingencies in infants before the age of one year: a comprehensive review. Dev. Psychol 56, 1233–1251 (2020).

    Article  Google Scholar 

  89. Zaadnoordijk, L. et al. From movement to action: an EEG study into the emerging sense of agency in early infancy. Dev. Cogn. Neurosci. 42, 100760 (2020).

    Article  Google Scholar 

  90. Hunnius, S. & Bekkering, H. The early development of object knowledge: a study of infants’ visual anticipations during action observation. Dev. Psychol. 46, 446–454 (2010).

    Article  Google Scholar 

  91. Brookes, H. et al. Three-month-old infants learn arbitrary auditory-visual pairings between voices and faces. Infant Child Dev. 10, 75–82 (2001).

    Article  Google Scholar 

  92. Gómez, R. & Maye, J. The developmental trajectory of nonadjacent dependency learning. Infancy 7, 183–206 (2005).

    Article  Google Scholar 

  93. Maye, J., Werker, J. F. & Gerken, L. Infant sensitivity to distributional information can affect phonetic discrimination. Cognition 82, B101–B111 (2002).

    Article  Google Scholar 

  94. Saffran, J. R., Aslin, R. N. & Newport, E. L. Statistical learning by 8-month-old infants. Science 274, 1926–1928 (1996).

    Article  Google Scholar 

  95. Emberson, L. L., Misyak, J. B., Schwade, J. A., Christiansen, M. H. & Goldstein, M. H. Comparing statistical learning across perceptual modalities in infancy: an investigation of underlying learning mechanism(s). Dev. Sci. 22, e12847 (2019).

    Article  Google Scholar 

  96. Kirkham, N. Z., Slemmer, J. A. & Johnson, S. P. Visual statistical learning in infancy: evidence for a domain general learning mechanism. Cognition 83, B35–B42 (2002).

    Article  Google Scholar 

  97. Monroy, C. D. et al. Sensitivity to structure in action sequences: an infant event-related potential study. Neuropsychologia 126, 92–101 (2019).

    Article  Google Scholar 

  98. Stahl, A. E., Romberg, A. R., Roseberry, S., Golinkoff, R. M. & Hirsh-Pasek, K. Infants segment continuous events using transitional probabilities. Child Dev. 85, 1821–1826 (2014).

    Google Scholar 

  99. Tummeltshammer, K. S. & Kirkham, N. Z. Learning to look: probabilistic variation and noise guide infants’ eye movements. Dev. Sci. 16, 760–771 (2013).

    Article  Google Scholar 

  100. Ruffman, T., Taumoepeau, M. & Perkins, C. Statistical learning as a basis for social understanding in children. Br. J. Dev. Psychol. 30, 87–104 (2012).

    Article  Google Scholar 

  101. Bristow, D. et al. Hearing faces: how the infant brain matches the face it sees with the speech it hears. J. Cogn. Neurosci. 21, 905–921 (2008).

    Article  Google Scholar 

  102. Bremner, A. J., Mareschal, D., Lloyd-Fox, S. & Spence, C. Spatial localization of touch in the first year of life: early influence of a visual spatial code and the development of remapping across changes in limb position. J. Exp. Psychol. Gen. 137, 149–162 (2008).

    Article  Google Scholar 

  103. Zmyj, N., Jank, J., Schütz-Bosbach, S. & Daum, M. M. Detection of visual-tactile contingency in the first year after birth. Cognition 120, 82–89 (2011).

    Article  Google Scholar 

  104. Tanaka, Y., Kanakogi, Y., Kawasaki, M. & Myowa, M. The integration of audio-tactile information is modulated by multimodal social interaction with physical contact in infancy. Dev. Cogn. Neurosci. 30, 31–40 (2018).

    Article  Google Scholar 

  105. Lewkowicz, D. J. The development of intersensory temporal perception: an epigenetic systems/limitations view. Psychol. Bull. 126, 281–308 (2000).

    Article  Google Scholar 

  106. Landry, S. P., Guillemot, J.-P. & Champoux, F. Temporary deafness can impair multisensory integration: a study of cochlear-implant users. Psychol. Sci. 24, 1260–1268 (2013).

    Article  Google Scholar 

  107. Stevenson, R., Sheffield, S. W., Butera, I. M., Gifford, R. H. & Wallace, M. Multisensory integration in cochlear implant recipients. Ear Hearing 38, 521–538 (2017).

    Article  Google Scholar 

  108. Weatherhead, D. & White, K. S. Read my lips: visual speech influences word processing in infants. Cognition 160, 103–109 (2017).

    Article  Google Scholar 

  109. Cappagli, G., Cocchi, E. & Gori, M. Auditory and proprioceptive spatial impairments in blind children and adults. Dev. Sci. 20, e12374 (2017).

    Article  Google Scholar 

  110. Bruni, E., Tran, N.-K. & Baroni, M. Multimodal distributional semantics. J. Artif. Intell. Res. 49, 1–47 (2014).

    Article  MathSciNet  MATH  Google Scholar 

  111. Marton, Z.-C., Pangercic, D., Blodow, N. & Beetz, M. Combined 2D-3D categorization and classification for multimodal perception systems. Int. J. Robot. Res. 30, 1378–1402 (2011).

    Article  Google Scholar 

  112. Nakamura, T., Nagai, T. & Iwahashi, N. Multimodal object categorization by a robot. In Proc. 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems 2415–2420 (IEEE, 2007).

  113. Mogadala, A., Kalimuthu, M. & Klakow, D. Trends in integration of vision and language research: a survey of tasks, datasets, and methods. J. Artif. Intell. Res. 71, 1183–1317 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  114. Barbieri, F. et al. Towards a multimodal time-based empathy prediction system. In Proc. 2019 14th IEEE International Conference on Automatic Face and Gesture Recognition 1–5 (IEEE, 2019).

  115. Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B. W. & Zafeiriou, S. End-to-end multimodal emotion recognition using deep neural networks. IEEE J. Select. Top. Signal Process. 11, 1301–1309 (2017).

    Article  Google Scholar 

  116. Evangelopoulos, G. et al. Multimodal saliency and fusion for movie summarization based on aural, visual and textual attention. IEEE Trans. Multimedia 15, 1553–1568 (2013).

    Article  Google Scholar 

  117. de Sa, V. R. & Ballard, D. H. Category learning through multimodality sensing. Neural Comput. 10, 1097–1117 (1998).

    Article  Google Scholar 

  118. Radford, A., Metz, L. & Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. In Proc. 4th International Conference on Learning Representations (eds. Bengio, Y. & LeCun, Y.) (ICLR, 2016).

  119. Droniou, A., Ivaldi, S. & Sigaud, O. Deep unsupervised network for multimodal perception, representation and classification. Robot. Auton. Syst. 71, 83–98 (2015).

    Article  Google Scholar 

  120. Feng, Y., Ma, L., Liu, W. & Luo, J. Unsupervised image captioning. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 4125–4134 (IEEE, 2019).

  121. Ngiam, J. et al. Multimodal deep learning. In Proc. 28th International Conference on International Conference on Machine Learning (eds. Getoor, L. & Scheffer, T.) 689–696 (Omnipress, 2011).

  122. Srivastava, N. & Salakhutdinov, R. R. Multimodal learning with deep Boltzmann machines. In Advances in Neural Information Processing Systems (Pereira, F., Burges, C. J., Bottou, L. & Weinberger, K. Q.) 2222–2230 (NIPS, 2012).

  123. Bachman, P., Hjelm, R. D. & Buchwalter, W. Learning representations by maximizing mutual information across views. In Advances in Neural Information Processing Systems (eds. Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E. & Garnett, R.) 15509–15519 (NIPS, 2019).

  124. Tian, Y., Krishnan, D. & Isola, P. Contrastive multiview coding. In Proc. Computer Vision–ECCV 2020: 16th European Conference Part XI 16 (Vedaldi, A., Bischof, H., Brox, T. & Frahm, J.-M.) 776–794 (Springer, 2020).

  125. Roads, B. D. & Love, B. C. Learning as the unsupervised alignment of conceptual systems. Nat. Mach. Intell. 2, 76–82 (2020).

    Article  Google Scholar 

  126. Wang, C. & Mahadevan, S. Manifold alignment without correspondence. In Proc. 21st International Jont Conference on Artifical Intelligence (ed. Boutilier, C.) 1273–1278 (Morgan Kaufmann, 2009).

  127. Wang, C. & Mahadevan, S. Heterogeneous domain adaptation using manifold alignment. In Proc. Twenty-Second International Joint Conference on Artificial Intelligence Vol. 2 (ed. Walsh, T.) 1541–1546 (AAAI Press, 2011).

  128. Baltrušaitis, T., Ahuja, C. & Morency, L.-P. Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41, 423–443 (2018).

    Article  Google Scholar 

  129. Barros, P., Eppe, M., Parisi, G. I., Liu, X. & Wermter, S. Expectation learning for stimulus prediction across modalities improves unisensory classification. Front. Robot. AI 6, 137 (2019).

    Article  Google Scholar 

  130. Peterson, S. M., Rao, R. P. & Brunton, B. W. Learning neural decoders without labels using multiple data streams. Preprint at https://www.biorxiv.org/content/10.1101/2021.09.10.459775v1.full (2021).

  131. Ackman, J. B., Burbridge, T. J. & Crair, M. C. Retinal waves coordinate patterned activity throughout the developing visual system. Nature 490, 219–225 (2012).

    Article  Google Scholar 

  132. Moon, C., Lagercrantz, H. & Kuhl, P. K. Language experienced in utero affects vowel perception after birth: a two-country study. Acta Paediatrica 102, 156–160 (2013).

    Article  Google Scholar 

  133. DeCasper, A. J. & Spence, M. J. Prenatal maternal speech influences newborns’ perception of speech sounds. Infant Behav. Dev. 9, 133–150 (1986).

    Article  Google Scholar 

  134. Lobo, M. A., Kokkoni, E., de Campos, A. C. & Galloway, J. C. Not just playing around: infants’ behaviors with objects reflect ability, constraints and object properties. Infant Behav. Dev. 37, 334–351 (2014).

    Article  Google Scholar 

  135. Soska, K. C. & Adolph, K. E. Postural position constrains multimodal object exploration in infants. Infancy 19, 138–161 (2014).

    Article  Google Scholar 

  136. Campos, J. J. et al. Travel broadens the mind. Infancy 1, 149–219 (2000).

    Article  Google Scholar 

  137. Barsalou, L. W. Grounded cognition. Annu. Rev. Psychol. 59, 617–645 (2008).

    Article  Google Scholar 

  138. Dobson, V. & Teller, D. Y. Visual acuity in human infants: a review and comparison of behavioral and electrophysiological studies. Vision Res. 18, 1469–1483 (1978).

    Article  Google Scholar 

  139. Sokol, S. Measurement of infant visual acuity from pattern reversal evoked potentials. Vision Res. 18, 33–39 (1978).

    Article  Google Scholar 

  140. Fiser, J., Aslin, R., Lathrop, A., Rothkopf, C. & Markant, J. An infants’ eye view of the world: implications for learning in natural contexts. In Proc. International Conference on Infant Studies (2006).

  141. Franchak, J. M., Kretch, K. S., Soska, K. C. & Adolph, K. E. Head-mounted eye tracking: a new method to describe infant looking. Child Dev. 82, 1738–1750 (2011).

    Article  Google Scholar 

  142. Smith, L. B., Yu, C., Yoshida, H. & Fausey, C. M. Contributions of head-mounted cameras to studying the visual environments of infants and young children. J. Cogn. Dev. 16, 407–419 (2015).

    Article  Google Scholar 

  143. Yoshida, H. & Smith, L. B. What’s in view for toddlers? Using a head camera to study visual experience. Infancy 13, 229–248 (2008).

    Article  Google Scholar 

  144. Smith, L. B., Jayaraman, S., Clerkin, E. & Yu, C. The developing infant creates a curriculum for statistical learning. Trends Cogn. Sci. 22, 325–336 (2018).

    Article  Google Scholar 

  145. Fausey, C. M., Jayaraman, S. & Smith, L. B. From faces to hands: changing visual input in the first two years. Cognition 152, 101–107 (2016).

    Article  Google Scholar 

  146. Davis, J. et al. Does neonatal imitation exist? Insights from a meta-analysis of 336 effect sizes. Perspect. Psychol. Sci. https://doi.org/10.1177/1745691620959834 (2021).

  147. Hunnius, S. & Bekkering, H. What are you doing? How active and observational experience shape infants’ action understanding. Philos. Trans. R. Soc. B Biol. Sci. 369, 20130490 (2014).

    Article  Google Scholar 

  148. Meltzoff, A. N. & Moore, M. K. Explaining facial imitation: a theoretical model. Infant Child Dev. 6, 179–192 (1997).

    Google Scholar 

  149. Meltzoff, A. N. & Marshall, P. J. Human infant imitation as a social survival circuit. Curr. Opin. Behav. Sci. 24, 130–136 (2018).

    Article  Google Scholar 

  150. Ray, E. & Heyes, C. Imitation in infancy: the wealth of the stimulus. Dev. Sci. 14, 92–105 (2011).

    Article  Google Scholar 

  151. Soderstrom, M. Beyond babytalk: re-evaluating the nature and content of speech input to preverbal infants. Dev. Rev. 27, 501–532 (2007).

    Article  Google Scholar 

  152. Brand, R. J., Baldwin, D. A. & Ashburn, L. A. Evidence for ‘motionese’: modifications in mothers’ infant-directed action. Dev. Sci. 5, 72–83 (2002).

    Article  Google Scholar 

  153. van Schaik, J. E., Meyer, M., van Ham, C. R. & Hunnius, S. Motion tracking of parents’ infant-versus adult-directed actions reveals general and action-specific modulations. Dev. Sci. 23, e12869 (2020).

    Article  Google Scholar 

  154. Wass, S. V. et al. Infants’ visual sustained attention is higher during joint play than solo play: is this due to increased endogenous attention control or exogenous stimulus capture? Dev. Sci. 21, e12667 (2018).

    Article  Google Scholar 

  155. Yu, C. & Smith, L. B. The social origins of sustained attention in one-year-old human infants. Curr. Biol. 26, 1235–1240 (2016).

    Article  Google Scholar 

  156. Yu, Y. et al. The theoretical and methodological opportunities afforded by guided play with young children. Front. Psychol. 9, 1152 (2018).

    Article  Google Scholar 

  157. Bazhydai, M., Westermann, G. & Parise, E. ‘I don’t know but I know who to ask’: 12-month-olds actively seek information from knowledgeable adults. Dev. Sci. 23, e12938 (2020).

    Article  Google Scholar 

  158. Poulin-Dubois, D. & Brosseau-Liard, P. The developmental origins of selective social learning. Curr. Directions Psychol. Sci. 25, 60–64 (2016).

    Article  Google Scholar 

  159. Berlyne, D. E. Conflict, Arousal and Curiosity (McGraw-Hill, 1960).

  160. Day, H. I. Curiosity and the interested explorer. Performance & Instruction 21, 19–22 (1982).

    Article  Google Scholar 

  161. Kidd, C., Piantadosi, S. T. & Aslin, R. N. The Goldilocks effect: human infants allocate attention to visual sequences that are neither too simple nor too complex. PLoS ONE 7, e36399 (2012).

    Article  Google Scholar 

  162. Kidd, C., Piantadosi, S. T. & Aslin, R. N. The Goldilocks effect in infant auditory attention. Child Dev. 85, 1795–1804 (2014).

    Google Scholar 

  163. Poli, F., Serino, G., Mars, R. & Hunnius, S. Infants tailor their attention to maximize learning. Sci. Adv. 6, eabb5053 (2020).

    Article  Google Scholar 

  164. Cohen, L. B. Uses and misuses of habituation and related preference paradigms. Infant Child Dev. 13, 349–352 (2004).

    Article  Google Scholar 

  165. Hunter, M. A. & Ames, E. W. A multifactor model of infant preferences for novel and familiar stimuli. Adv. Infancy Res 5, 69–95 (1988).

    Google Scholar 

  166. Aslin, R. N. What’s in a look? Dev. Sci. 10, 48–53 (2007).

    Article  Google Scholar 

  167. Haith, M. M. Who put the cog in infant cognition? Is rich interpretation too costly? Infant Behav. Dev. 21, 167–179 (1998).

    Article  Google Scholar 

  168. Adolph, K. E. et al. How do you learn to walk? Thousands of steps and dozens of falls per day. Psychol. Sci. 23, 1387–1394 (2012).

    Article  Google Scholar 

  169. Hoch, J. E., O’Grady, S. M. & Adolph, K. E. It’s the journey, not the destination: locomotor exploration in infants. Dev. Sci. 22, e12740 (2019).

    Article  Google Scholar 

  170. Oakes, L. M. & Baumgartner, H. A. Manual object exploration and learning about object features in human infants. In Proc. 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics1–6 (IEEE, 2012).

  171. Elman, J. L. Learning and development in neural networks: the importance of starting small. Cognition 48, 71–99 (1993).

    Article  Google Scholar 

  172. Bengio, Y., Louradour, J., Collobert, R. & Weston, J. Curriculum learning. In Proc. 26th Annual International Conference on Machine Learning (eds. Pohoreckyj, A., Danyluk, L. Bottou, M. & Littman, L.) 41–48 (ACM, 2009).

  173. Vogelsang, L. et al. Potential downside of high initial visual acuity. Proc. Natl Acad. Sci. USA 115, 11333–11338 (2018).

    Article  Google Scholar 

  174. Orhan, A. E., Gupta, V. V. & Lake, B. M. Self-supervised learning through the eyes of a child. In Advances in Neural Information Processing Systems Vol. 33 (eds. Wallach, H. M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E. A. & Garnett, R.) 9960–9971 (NIPS, 2020).

  175. Newport, E. L., Bavelier, D. & Neville, H. J. Critical thinking about critical periods: perspectives on a critical period for language acquisition. In Language, Brain and Cognitive Development: Essays in Honor of Jacques Mehler (ed. Dupoux, E.) 481–502 (MIT Press, 2001).

  176. Molnár, Z., Luhmann, H. J. & Kanold, P. O. Transient cortical circuits match spontaneous and sensory-driven activity during development. Science 370, eabb2153 (2020).

    Article  Google Scholar 

  177. Kostovic, I. & Rakic, P. Developmental history of the transient subplate zone in the visual and somatosensory cortex of the macaque monkey and human brain. J. Comp. Neurol. 297, 441–470 (1990).

    Article  Google Scholar 

  178. Achille, A., Rovere, M. & Soatto, S. Critical learning periods in deep neural networks. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).

  179. Carpenter, G. A. & Grossberg, S. The art of adaptive pattern recognition by a self-organizing neural network. Computer 21, 77–88 (1988).

    Article  Google Scholar 

  180. French, R. M. Catastrophic forgetting in connectionist networks. Trends Cogn. Sci. 3, 128–135 (1999).

    Article  Google Scholar 

  181. Parisi, G. I., Kemker, R., Part, J. L., Kanan, C. & Wermter, S. Continual lifelong learning with neural networks: a review. Neural Networks 113, 54–71 (2019).

    Article  Google Scholar 

  182. Robins, A. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Sci. 7, 123–146 (1995).

    Article  Google Scholar 

  183. Hinton, G. E. & Plaut, D. C. Using fast weights to deblur old memories. In Proc. 9th Annual Conference of the Cognitive Science Society 177–186 (Erlbaum, 1987).

  184. Kemker, R. & Kanan, C. FearNet: brain-inspired model for incremental learning. In Proc. 6th International Conference on Learning Representations (ICLR, 2018).

  185. Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl Acad. Sci. USA 114, 3521–3526 (2017).

    Article  MathSciNet  MATH  Google Scholar 

  186. Rannen, A., Aljundi, R., Blaschko, M. B. & Tuytelaars, T. Encoder based lifelong learning. In Proc. IEEE International Conference on Computer Vision 1320–1328 (IEEE, 2017).

  187. Draelos, T. J. et al. Neurogenesis deep learning: extending deep networks to accommodate new classes. In Proc. 2017 International Joint Conference on Neural Networks 526–533 (IEEE, 2017).

  188. Javed, K. & White, M. Meta-learning representations for continual learning. In Advances in Neural Information Processing Systems (eds. Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E. & Garnett, R.) 1818–1828 (NIPS, 2019).

  189. Kemker, R., McClure, M., Abitino, A., Hayes, T. L. & Kanan, C. Measuring catastrophic forgetting in neural networks. In Proc. Thirty-Second AAAI Conference on Artificial Intelligence Vol. 415 (eds. McIlraith, S. A. & Weinberger, K. Q.) 3390–3398 (AAAI, 2018).

  190. Settles, B. Active Learning Literature Survey (Univ. Wisconsin-Madison Department of Computer Sciences, 2009).

  191. Settles, B. From theories to queries: active learning in practice. In Proc. Active Learning and Experimental Design Workshop in Conjunction with AISTATS 2010 Vol. 16 (eds. Guyon, I., Cawley, G., Dror, G., Lemaire, V. & Statnikov, A.) 1–18 (MLR, 2011).

  192. Botvinick, M. M., Niv, Y. & Barto, A. G. Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113, 262–280 (2009).

    Article  Google Scholar 

  193. Lefort, M. & Gepperth, A. Active learning of local predictable representations with artificial curiosity. In Proc. 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics 228–233 (IEEE, 2015).

  194. Graves, A., Bellemare, M. G., Menick, J., Munos, R. & Kavukcuoglu, K. Automated curriculum learning for neural networks. In Proc. 34th International Conference on Machine Learning Vol. 70 (eds. Precup, D. & Teh, Y. W.) 1311–1320 (JMLR, 2017).

  195. Schmidhuber, J. Driven by compression progress: a simple principle explains essential aspects of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. In Proc. Workshop on Anticipatory Behavior in Adaptive Learning Systems (eds. Pezzulo, G. Butz, M. V., Sigaud, O. & Baldassarre, G.) 48–76 (Springer, 2008).

  196. Oudeyer, P.-Y. Computational theories of curiosity-driven learning. In The New Science of Curiosity (ed. Gordon, G.) 43–72 (Nova Science, 2018).

  197. Oudeyer, P.-Y., Kaplan, F. & Hafner, V. V. Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evolution. Comput. 11, 265–286 (2007).

    Article  Google Scholar 

  198. Twomey, K. E. & Westermann, G. Curiosity-based learning in infants: a neurocomputational approach. Dev. Sci. 21, e12629 (2018).

    Article  Google Scholar 

  199. Haber, N., Mrowca, D., Fei-Fei, L. & Yamins, D. L. Emergence of structured behaviors from curiosity-based intrinsic motivation. Preprint at https://arxiv.org/abs/1802.07461 (2018).

  200. Soltoggio, A., Stanley, K. O. & Risi, S. Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks. Neural Networks 108, 48–67 (2018).

    Article  Google Scholar 

  201. Arandjelovic, R. & Zisserman, A. Look, listen and learn. In Proc. IEEE International Conference on Computer Vision 609–617 (IEEE, 2017).

  202. Barros, P., Parisi, G. I., Weber, C. & Wermter, S. Emotion-modulated attention improves expression recognition: a deep learning model. Neurocomputing 253, 104–114 (2017).

    Article  Google Scholar 

  203. Senocak, A., Oh, T.-H., Kim, J., Yang, M.-H. & Kweon, I. S. Learning to localize sound source in visual scenes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 4358–4366 (IEEE, 2018).

  204. Ashby, F. G. & Vucovich, L. E. The role of feedback contingency in perceptual category learning. J. Exp. Psychol. Learn. Mem. Cogn. 42, 1731–1746 (2016).

    Article  Google Scholar 

  205. Parisi, G. I., Tani, J., Weber, C. & Wermter, S. Lifelong learning of human actions with deep neural network self-organization. Neural Networks 96, 137–149 (2017).

    Article  Google Scholar 

  206. Bonawitz, E. & Shafto, P. Computational models of development, social influences. Curr. Opin. Behav. Sci. 7, 95–100 (2016).

    Article  Google Scholar 

  207. Brockman, G. et al. OpenAI gym. Preprint at https://arxiv.org/abs/1606.01540 (2016).

  208. Geirhos, R. et al. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In Proc. 7th International Conference on Learning Representations (2019).

Download references

Acknowledgements

This work was supported by an ERC Advanced Grant (FOUNDCOG, #787981) awarded to R.C. and an MSCA Individual Fellowship (InterPlay, #891535) awarded to L.Z. We thank K. Storrs and K. Körding as well as the Nature Machine Intelligence editorial team for their helpful feedback.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Lorijn Zaadnoordijk, Tarek R. Besold or Rhodri Cusack.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zaadnoordijk, L., Besold, T.R. & Cusack, R. Lessons from infant learning for unsupervised machine learning. Nat Mach Intell 4, 510–520 (2022). https://doi.org/10.1038/s42256-022-00488-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-022-00488-2

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing