Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks

Nature Human Behaviour volume 6, pages 720–731 (2022)Cite this article

Subjects

An Author Correction to this article was published on 28 April 2022

This article has been updated

Abstract

A framework to pinpoint the scope of unconscious processing is critical to improve models of visual consciousness. Previous research observed brain signatures of unconscious processing in visual cortex, but these were not reliably identified. Further, whether unconscious contents are represented in high-level stages of the ventral visual stream and linked parieto-frontal areas remains unknown. Using a within-subject, high-precision functional magnetic resonance imaging approach, we show that unconscious contents can be decoded from multi-voxel patterns that are highly distributed alongside the ventral visual pathway and also involving parieto-frontal substrates. Classifiers trained with multi-voxel patterns of conscious items generalized to predict the unconscious counterparts, indicating that their neural representations overlap. These findings suggest revisions to models of consciousness such as the neuronal global workspace. We then provide a computational simulation of visual processing/representation without perceptual sensitivity by using deep neural networks performing a similar visual task. The work provides a framework for pinpointing the representation of unconscious knowledge across different task domains.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Example sequence of events within an experimental trial.
Fig. 2: Behavioural performance.
Fig. 3: Classification performance (ROC-AUC) for out-of-sample images for each observer across the unconscious and conscious trials.
Fig. 4: Classification performance.
Fig. 5: Image classification.

Similar content being viewed by others

Data availability

The fMRI data can be found at https://openneuro.org/datasets/ds003927.

Code availability

Full scripts for the fMRI analyses and deep neural network simulations are available at https://github.com/nmningmei/unconfeats.

Change history

References

  1. Dehaene, S. Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts (Penguin, 2014).

  2. Lamme, V. A. F. Visual functions generating conscious seeing. Front. Psychol. https://doi.org/10.3389/fpsyg.2020.00083 (2020).

  3. Van Gaal, S. & Lamme, V. A. F. Unconscious high-level information processing: implication for neurobiological theories of consciousness. Neuroscientist 18, 287–301 (2012).

    Article  PubMed  Google Scholar 

  4. Soto, D., Mäntylä, T. & Silvanto, J. Working memory without consciousness. Curr. Biol. 21, 912–913 (2011).

    Article  CAS  Google Scholar 

  5. Trübutschek, D. et al. A theory of working memory without consciousness or sustained activity. eLife 6, e23871 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Rosenthal, C. R., Andrews, S. K., Antoniades, C. A., Kennard, C. & Soto, D. Learning and recognition of a non-conscious sequence of events in human primary visual cortex. Curr. Biol. 26, 834–841 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Chong, T. T.-J., Husain, M. & Rosenthal, C. R. Recognizing the unconscious. Curr. Biol. 24, 1033–1035 (2014).

    Article  CAS  Google Scholar 

  8. Wuethrich, S., Hannula, D. E., Mast, F. W. & Henke, K. Subliminal encoding and flexible retrieval of objects in scenes. Hippocampus 28, 633–643 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Hassin, R. R. Yes it can. Perspect. Psychol. Sci. 8, 195–207 (2013).

    Article  PubMed  Google Scholar 

  10. Rabagliati, H., Robertson, A. & Carmel, D. The importance of awareness for understanding language. J. Exp. Psychol. Gen. 147, 190 (2018).

    Article  PubMed  Google Scholar 

  11. Kouider, S. & Dehaene, S. Levels of processing during non-conscious perception: a critical review of visual masking. Philos. Trans. R. Soc. B 362, 857–875 (2007).

    Article  Google Scholar 

  12. Stein, T., Utz, V. & van Opstal, F. Unconscious semantic priming from pictures under backward masking and continuous flash suppression. Conscious. Cogn. 78, 102864 (2020).

    Article  PubMed  Google Scholar 

  13. Soto, D., Sheikh, U. A. & Rosenthal, C. R. A novel framework for unconscious processing. Trends Cogn. Sci. 23, 372–376 (2019).

    Article  PubMed  Google Scholar 

  14. Overgaard, M., Timmermans, B., Sandberg, K. & Cleeremans, A. Optimizing subjective measures of consciousness. Conscious. Cogn. 19, 682–684 (2010).

    Article  PubMed  Google Scholar 

  15. Peters, M.A.K. & Lau, H. Human observers have optimal introspective access to perceptual processes even for visually masked stimuli. eLife https://doi.org/10.7554/elife.09651 (2015).

  16. Sterzer, P., Haynes, J. D. & Rees, G. Fine-scale activity patterns in high-level visual areas encode the category of invisible objects. J. Vis. 8, 10–10 (2008).

    Article  PubMed  Google Scholar 

  17. Haynes, J.-D. & Rees, G. Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 8, 686–691 (2005).

    Article  CAS  PubMed  Google Scholar 

  18. Jiang, Y., Zhou, K. & He, S. Human visual cortex responds to invisible chromatic flicker. Nat. Neurosci. 10, 657–662 (2007).

    Article  CAS  PubMed  Google Scholar 

  19. Dehaene, S. et al. Cerebral mechanisms of word masking and unconscious repetition priming. Nat. Neurosci. 4, 752–758 (2001a).

    Article  CAS  PubMed  Google Scholar 

  20. Fang, F. & He, S. Cortical responses to invisible objects in the human dorsal and ventral pathways. Nat. Neurosci. 8, 1380–1385 (2005).

    Article  CAS  PubMed  Google Scholar 

  21. Hesselmann, G., Hebart, M. & Malach, R. Differential BOLD activity associated with subjective and objective reports during ‘blindsight’ in normal observers. J. Neurosci. 31, 12936–12944 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Ludwig, K. & Hesselmann, G. Weighing the evidence for a dorsal processing bias under continuous flash suppression. Conscious. Cogn. 35, 251–259 (2015).

    Article  PubMed  Google Scholar 

  23. Ludwig, K., Kathmann, N., Sterzer, P. & Hesselmann, G. Investigating category-and shape-selective neural processing in ventral and dorsal visual stream under interocular suppression. Human Brain Mapp. 36, 137–149 (2015).

    Article  Google Scholar 

  24. Macmillan, N. A. The psychophysics of subliminal perception. Behav. Brain Sci. 9, 38–39 (1986).

    Article  Google Scholar 

  25. Newell, B. R. & Shanks, D. R. Unconscious influences on decision making: a critical review. Behav. Brain Sci. 37, 1–19 (2014).

    Article  PubMed  Google Scholar 

  26. Kravitz, D. J., Saleem, K. S., Baker, C. I., Ungerleider, L. G. & Mishkin, M. The ventral visual pathway: an expanded neural framework for the processing of object quality. Trends Cogn. Sci. 17, 26–49 (2013).

    Article  PubMed  Google Scholar 

  27. Haxby, J. V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).

    Article  CAS  PubMed  Google Scholar 

  28. Naselaris, T., Kay, K. N., Nishimoto, S. & Gallant, J. L. Encoding and decoding in fMRI. NeuroImage 56, 400–410 (2011).

    Article  PubMed  Google Scholar 

  29. Kriegeskorte, N. Pattern-information analysis: from stimulus decoding to computational-model testing. NeuroImage 56, 411–421 (2011).

    Article  PubMed  Google Scholar 

  30. Ester, E. F., Sprague, T. C. & Serences, J. T. Parietal and frontal cortex encode stimulus-specific mnemonic representations during visual working memory. Neuron 87, 893–905 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Christophel, T. B., Hebart, M. N. & Haynes, J.-D. Decoding the contents of visual short-term memory from human visual and parietal cortex. J. Neurosci. 32, 12983–12989 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Kapoor, V. et al. Decoding the contents of consciousness from prefrontal ensembles. Preprint at bioRxiv https://doi.org/10.1101/2020.01.28.921841 (2020).

  33. Schurger, A., Pereira, F., Treisman, A. & Cohen, J. D. Reproducibility distinguishes conscious from nonconscious neural representations. Science 327, 97–99 (2010).

    Article  CAS  PubMed  Google Scholar 

  34. Dehaene, S., Changeux, J.-P., Naccache, L., Sackur, J. & Sergent, C. Conscious, preconscious, and subliminal processing: a testable taxonomy. Trends Cogn. Sci. 10, 204–211 (2006).

    Article  PubMed  Google Scholar 

  35. G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. Preprint at https://arxiv.org/abs/1503.02531 (2015).

  36. LeCun, Y. & Bengio, Y. in The Handbook of Brain Theory and Neural Networks (ed. Arbib, M. A.) (MIT Press, 1995).

  37. McFee, B., Salamon, J. & Bello, J. P. Adaptive pooling operators for weakly labeled sound event detection. IEEE/ACM Trans. Audio Speech Lang. Process. 26, 2180–2193 (2018).

    Article  Google Scholar 

  38. Marr, D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (MIT Press, 1982).

  39. Kriegeskorte, N. & Douglas, P. K. Cognitive computational neuroscience. Nat. Neurosci. 21, 1148–1160 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Kietzmann, T. C., McClure, P., & Kriegeskorte, N. in Oxford Research Encyclopaedia of Neuroscience https://doi.org/10.1093/acrefore/9780190264086.013.46 (2019a).

  41. Yamins, D. L. K. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19, 356 (2016).

    Article  CAS  PubMed  Google Scholar 

  42. Kriegeskorte, N. Deep neural networks: a new framework for modeling biological vision and brain information processing. Annu. Rev. Vis. Sci. 1, 417–446 (2015).

    Article  PubMed  Google Scholar 

  43. Moreno-Martínez, F. J. & Montoro, P. R. An ecological alternative to Snodgrass & Vanderwart: 360 high quality colour images with norms for seven psycholinguistic variables. PloS ONE 7, e37527 (2012).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  44. Aru, J., Bachmann, T., Singer, W. & Melloni, L. Distilling the neural correlates of consciousness. Neurosci. Biobehav. Rev. 36, 737–746 (2012).

    Article  PubMed  Google Scholar 

  45. Zhang, J. & Mueller, S. T. A note on ROC analysis and non-parametric estimate of sensitivity. Psychometrika 70, 203–212 (2005).

    Article  Google Scholar 

  46. Stein, T., Kaiser, D., Fahrenfort, J. J. & Van Gaal, S. The human visual system differentially represents subjectively and objectively invisible stimuli. PLoS Biol. 19, e3001241 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Jamalabadi, H., Alizadeh, S., Schönauer, M., Leibold, C. & Gais, S. Classification based hypothesis testing in neuroscience: below-chance level classification rates and overlooked statistical properties of linear parametric classifiers. Human Brain Mapp. 37, 1842–1855 (2016).

    Article  Google Scholar 

  48. Snoek, L., Miletić, S. & Scholte, H. S. How to control for confounds in decoding analyses of neuroimaging data. NeuroImage 184, 741–760 (2019).

    Article  PubMed  Google Scholar 

  49. Ben-David, S. et al. A theory of learning from different domains. Mach. Learn. 79, 151–175 (2010).

    Article  Google Scholar 

  50. Fukushima, K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36, 193–202 (1980).

    Article  CAS  PubMed  Google Scholar 

  51. Kubilius, J., Bracci, S. & Op de Beeck, H. P. Deep neural networks as a computational model for human shape sensitivity. PLoS Comput. Biol. 12, e1004896 (2016).

  52. Khaligh-Razavi, S.-M. & Kriegeskorte, N. Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Comput. Biol. 10, e1003915 (2014).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  53. Güçlü, U. & van Gerven, M. A. J. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J. Neurosci. 35, 10005–10014 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  54. Kriegeskorte, N. et al. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60, 1126–1141 (2008a).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Fisher, A., Rudin, C. & Dominici, F. All models are wrong but many are useful: variable importance for black-box, proprietary, or misspecified prediction models, using model class reliance. Preprint at https://arxiv.org/abs/1801.01489 (2018).

  56. Altmann, A., Toloşi, L., Sander, O. & Lengauer, T. Permutation importance: a corrected feature importance measure. Bioinformatics 26, 1340–1347 (2010).

    Article  CAS  PubMed  Google Scholar 

  57. Kriegeskorte, N., Mur, M. & Bandettini, P. A. Representational similarity analysis-connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2, 4 (2008b).

    PubMed  PubMed Central  Google Scholar 

  58. Michel, M. Consciousness science underdetermined: a short history of endless debates. Ergo https://doi.org/10.3998/ergo.12405314.0006.028 (2019)

  59. Gayet, S. et al. No evidence for mnemonic modulation of interocularly suppressed visual input. NeuroImage 215, 116801 (2020).

    Article  PubMed  Google Scholar 

  60. Kriegeskorte, N., Goebel, R. & Bandettini, P. Information-based functional brain mapping. Proc. Natl Acad. Sci. USA 103, 3863–3868 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Dehaene, S. & Naccache, L. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition 79, 1–37 (2001).

    Article  CAS  PubMed  Google Scholar 

  62. Dehaene, S. & Changeux, J.-P. Neural mechanisms for access to consciousness. Cogn. Neurosci. 3, 1145–58 (2004).

    Google Scholar 

  63. Dehaene, S. et al. Cerebral mechanisms of word masking and unconscious repetition priming. Nat. Neurosci. 4, 752–758 (2001b).

    Article  CAS  PubMed  Google Scholar 

  64. Haynes, J.-D., Driver, J. & Rees, G. Visibility reflects dynamic changes of effective connectivity between v1 and fusiform cortex. Neuron 46, 811–821 (2005).

    Article  CAS  PubMed  Google Scholar 

  65. Beck, D. M., Rees, G., Frith, C. D. & Lavie, N. Neural correlates of change detection and change blindness. Nat. Neurosci. 4, 645–650 (2001).

    Article  CAS  PubMed  Google Scholar 

  66. Pessoa, L. & Ungerleider, L. G. Neural correlates of change detection and change blindness in a working memory task. Cereb. Cortex 14, 511–520 (2004).

    Article  PubMed  Google Scholar 

  67. Kranczioch, C., Debener, S., Schwarzbach, J., Goebel, R. & Engel, A. K. Neural correlates of conscious perception in the attentional blink. NeuroImage 24, 704–714 (2005).

    Article  PubMed  Google Scholar 

  68. Yamins, D. L., Hong, H., Cadieu, C. & DiCarlo, J. J. Hierarchical modular optimization of convolutional networks achieves representations similar to macaque IT and human ventral stream. In Advances in Neural Information Processing Systems 26 (eds Burges, C. J. C., Bottou, L., Welling, M., Ghahramani, Z. & Weinberger, K. Q.) 3093–3101 (Curran Associates, Inc., 2013).

  69. Geirhos, R. et al. Comparing deep neural networks against humans: object recognition when the signal gets weaker. Preprint at https://arxiv.org/abs/1706.06969 (2017).

  70. Wichmann, F. A. et al. Methods and measurements to compare men against machines. Electron. Imaging 2017, 36–45 (2017).

    Article  Google Scholar 

  71. Geirhos, R. et al. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. Preprint at https://arxiv.org/abs/1811.12231 (2018).

  72. Ghodrati, M., Farzmahdi, A., Rajaei, K., Ebrahimpour, R. & Khaligh-Razavi, S.-M. Feedforward object-vision models only tolerate small image variations compared to human. Front. Comput. Neurosci. 8, 74 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  73. Kriegeskorte, N. & Diedrichsen, J. Peeling the onion of brain representations. Annu. Rev. Neurosci. 42, 407–432 (2019).

    Article  CAS  PubMed  Google Scholar 

  74. Shea, N. Representation in Cognitive Science (Oxford Univ. Press, 2018).

  75. Lamme, V. A. F. & Roelfsema, P. R. The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23, 571–579 (2000).

    Article  CAS  PubMed  Google Scholar 

  76. Fahrenfort, J. J., Scholte, H. S. & Lamme, V. A. F. Masking disrupts reentrant processing in human visual cortex. J. Cogn. Neurosci. 19, 1488–1497 (2007).

    Article  CAS  PubMed  Google Scholar 

  77. DiCarlo, J. J., Zoccolan, D. & Rust, N. C. How does the brain solve visual object recognition? Neuron 73, 415–434 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  78. Nayebi, A. et al. Task-driven convolutional recurrent models of the visual system. Preprint at https://arxiv.org/abs/1807.00053 (2018).

  79. Bullier, J. Feedback connections and conscious vision. Trends Cogn. Sci. 5, 369–370 (2001).

    Article  PubMed  Google Scholar 

  80. Pascual-Leone, A. & Walsh, V. Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science 292, 510–512 (2001).

    Article  CAS  PubMed  Google Scholar 

  81. Huang, L. et al. A source for awareness-dependent figure–ground segregation in human prefrontal cortex. Proc. Natl Acad. Sci. USA. 117, 30836–30847 (2020).

  82. Cohen, M. A. & Dennett, D. C. Consciousness cannot be separated from function. Trends Cogn. Sci. 15, 358–364 (2011).

    Article  PubMed  Google Scholar 

  83. Soto, D. & Silvanto, J. Reappraising the relationship between working memory and conscious awareness. Trends Cogn. Sci. 18, 520–525 (2014).

    Article  PubMed  Google Scholar 

  84. Melloni, L. et al. Synchronization of neural activity across cortical areas correlates with conscious perception. J. Neurosci. 27, 2858–2865 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  85. Koivisto, M., Mäntylä, T. & Silvanto, J. The role of early visual cortex (v1/v2) in conscious and unconscious visual perception. NeuroImage 51, 828–834 (2010).

    Article  PubMed  Google Scholar 

  86. Cohen, M. X., Van Gaal, S., Ridderinkhof, K. R. & Lamme, V. Unconscious errors enhance prefrontal-occipital oscillatory synchrony. Front. Human Neurosci. 3, 54 (2009).

    Article  Google Scholar 

  87. Zwickel, T., Wachtler, T. & Eckhorn, R. Coding the presence of visual objects in a recurrent neural network of visual cortex. Biosystems 89, 216–226 (2007).

    Article  PubMed  Google Scholar 

  88. Spoerer, C. J., McClure, P. & Kriegeskorte, N. Recurrent convolutional neural networks: a better model of biological object recognition. Front. Psychol. 8, 1551 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  89. Shi, J., Wen, H., Zhang, Y., Han, K. & Liu, Z. Deep recurrent neural network reveals a hierarchy of process memory during dynamic natural vision. Human Brain Mapp. 39, 2269–2282 (2018).

    Article  Google Scholar 

  90. Kietzmann, T. C. et al. Recurrence is required to capture the representational dynamics of the human visual system. Proc. Natl Acad. Sci. USA 116, 21854–21863 (2019b).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  91. Zamir, A. R. et al. Feedback networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 1308–1317 (IEEE, 2017).

  92. Liao, Q. & Poggio, T. Bridging the gaps between residual learning, recurrent neural networks and visual cortex. Preprint at https://arxiv.org/abs/1604.03640 (2016).

  93. Grodan, E. M. et al. Precision functional mapping of individual human brains. Neuron 95, 791–807 (2017).

  94. Gratton, C. et al. Defining individual-specific functional neuroanatomy for precision psychiatry. Biol. Psychiatry 88, 28–39 (2020).

  95. Chollet, F. et al. Keras: the Python deep learning library. Astrophysics Source Code Library, ascl-1806.022 (2018).

  96. Peirce, J. W. Psychopy—Psychophysics software in Python. J. Neurosci. Methods 162, 8–13 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  97. Horowitz, J. L. in Handbook of Econometrics (eds Heckman, J. J. & Leamer, E.) vol. 5, pp. 3159–3228 (Elsevier, 2001).

  98. Ojala, M. & Garriga, G. C. Permutation tests for studying classifier performance. J. Mach. Learn. Res. 11, 1833–1863 (2010).

    Google Scholar 

  99. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

    Google Scholar 

  100. Abraham, A. et al. Machine learning for neuroimaging with scikit-learn. Front. Neuroinform. 8, 14 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  101. Pereira, F. & Botvinick, M. Information mapping with pattern classifiers: a comparative study. NeuroImage 56, 476–496 (2011).

    Article  PubMed  Google Scholar 

  102. Lewis-Peacock, J. A. & Norman, K. A. Multi-voxel pattern analysis of fmri data. Cogn. Neurosci. 512, 911–920 (2014).

    Google Scholar 

  103. Bruha, I. From machine learning to knowledge discovery: survey of preprocessing and postprocessing. Intell. Data Anal. 4, 363–374 (2000).

    Article  Google Scholar 

  104. King, J.-R. & Dehaene, S. Characterizing the dynamics of mental representations: the temporal generalization method. Trends Cogn. Sci. 18, 203–210 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  105. Paszke, A. et al. Automatic Differentiation in pytorch. Neural Information Processing Systems Workshop Autodiff, https://openreview.net/forum?id=BJJsrmfCZ (2017).

  106. Krizhevsky, A., Sutskever, I. & Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25 (eds Pereira, F., Burges, C. J. C., Bottou, L. & Weinberger, K. Q.) 25, 1097–1105 (Curran Associates, Inc., 2012).

  107. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at https://arxiv.org/abs/1409.1556 (2015).

  108. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 770–778 (IEEE, 2016).

  109. Howard, A.G. et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. Preprint at https://arxiv.org/abs/1704.04861 (2017).

  110. Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 4700–4708 (IEEE, 2017).

  111. Deng, J. et al. ImageNet: A large-scale hierarchical image database. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 248–255 (IEEE, 2009).

  112. Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. How transferable are features in deep neural networks? Advances in Neural Information Processing Systems 27 (eds Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. & Weinberger, K. Q.) 3320–3328 (Curran Associates, Inc., 2014).

  113. Specht, D. F. Probabilistic neural networks and the polynomial adaline as complementary techniques for classification. IEEE Trans. Neural Netw. 1, 111–121 (1990).

    Article  CAS  PubMed  Google Scholar 

  114. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014).

  115. Smith, S. M. Fast robust automated brain extraction. Human Brain Mapp. 17, 143–155 (2002).

    Article  Google Scholar 

  116. Jenkinson, M., Bannister, P., Brady, M. & Smith, S. M. Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage 17, 825–841 (2002).

    Article  PubMed  Google Scholar 

  117. Pruim, R. H. R. et al. Ica-aroma: A robust ica-based strategy for removing motion artifacts from fmri data. NeuroImage 112, 267–277 (2015).

    Article  PubMed  Google Scholar 

  118. Jenkinson, M., Beckmann, C. F., Behrens, T. E. J., Woolrich, M. W. & Smith, S. M. FSL. NeuroImage 62, 782–790 (2012).

    Article  PubMed  Google Scholar 

  119. Gorgolewski, K. et al. Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python. Front. Neuroinform. 5, 13 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

D.S. acknowledges support from the Basque Government through the BERC 2018-2021 programme, from the Spanish Ministry of Economy and Competitiveness, through the ‘Severo Ochoa’ Programme for Centres/Units of Excellence in R & D (CEX2020-001010-S) and also from project grants PSI2016-76443-P and PID2019-105494GB-I00 from MINECO. R.S. acknowledges support by the Basque Government (IT1244-19 and ELKARTEK programmes), and the Spanish Ministry of Economy and Competitiveness MINECO (project TIN2016-78365-R). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Basque Center on Cognition, Brain and Language, San Sebastian, Spain

    Ning Mei & David Soto

  2. Computer Science and Artificial Intelligence Department, University of Basque Country, San Sebastian, Spain

    Roberto Santana

  3. Ikerbasque, Basque Foundation for Science, Bilbao, Spain

    David Soto

Authors
  1. Ning Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roberto Santana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Soto
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

N.M. and D.S. designed the study. N.M. analysed the data under the guidance of R.S. and D.S. N.M. prepared a first draft of the paper. All authors discussed the results and contributed towards the writing of the final version of the manuscript. D.S. supervised the project.

Corresponding authors

Correspondence to Ning Mei or David Soto.

Ethics declarations

Competing interests

The authors declare no competing interests. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Peer review

Peer review information

Nature Human Behaviour thanks Steven Scholte and Axel Cleeremans for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information

Supplementary Tables 1–5 and Figs. 1–18.

Reporting summary.

Peer reviewer reports.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mei, N., Santana, R. & Soto, D. Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks. Nat Hum Behav 6, 720–731 (2022). https://doi.org/10.1038/s41562-021-01274-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-021-01274-7

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing