Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

The ConTraSt database for analysing and comparing empirical studies of consciousness theories

Abstract

Understanding how consciousness arises from neural activity remains one of the biggest challenges for neuroscience. Numerous theories have been proposed in recent years, each gaining independent empirical support. Currently, there is no comprehensive, quantitative and theory-neutral overview of the field that enables an evaluation of how theoretical frameworks interact with empirical research. We provide a bird’s eye view of studies that interpreted their findings in light of at least one of four leading neuroscientific theories of consciousness (N = 412 experiments), asking how methodological choices of the researchers might affect the final conclusions. We found that supporting a specific theory can be predicted solely from methodological choices, irrespective of findings. Furthermore, most studies interpret their findings post hoc, rather than a priori testing critical predictions of the theories. Our results highlight challenges for the field and provide researchers with an open-access website (https://ContrastDB.tau.ac.il) to further analyse trends in the neuroscience of consciousness.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: Database papers selection.
Fig. 2: Number of experiments per theory overall and over time.
Fig. 3: Distribution of experiment types according to theoretical interpretations.
Fig. 4: Predicting support for theories based on methodological choices.
Fig. 5: Distribution of methodological parameters in experiments supporting the theories.
Fig. 6: Spatial findings.
Fig. 7: Temporal findings of EEG, iEEG and MEG components related to conscious perception.

Data availability

The database we collected is shared on the Open Science Framework96 (https://osf.io/avz8b/). Source data are provided with this paper.

Code availability

All analysis scripts used in this paper are shared on the Open Science Framework96 (https://osf.io/avz8b/).

References

  1. Crick, F. & Koch, C. Towards a neurobiological theory of consciousness. Seminars Neurosci. 2, 263–275 (1990).

    Google Scholar 

  2. Crick, F. & Koch, C. A framework for consciousness. Nat. Neurosci. 6, 119–126 (2003).

    CAS  Article  PubMed  Google Scholar 

  3. Northoff, G. & Lamme, V. Neural signs and mechanisms of consciousness: is there a potential convergence of theories of consciousness in sight? Neurosci. Biobehav. Rev. 118, 568–587 (2020).

    Article  PubMed  Google Scholar 

  4. Doerig, A., Schurger, A. & Herzog, M. H. Hard criteria for empirical theories of consciousness. Cogn. Neurosci. 12, 41–62 (2020).

    Article  PubMed  Google Scholar 

  5. Signorelli, C. M., Szczotka, J. & Prentner, R. Explanatory profiles of models of consciousness—towards a systematic classification. Neurosci. Conscious. 2021, niab021 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Sattin, D. et al. Theoretical models of consciousness: a scoping review. Brain Sci. 11, 535 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Doerig, A., Schurger, A., Hess, K. & Herzog, M. H. The unfolding argument: why IIT and other causal structure theories cannot explain consciousness. Conscious. Cogn. 72, 49–59 (2019).

    Article  PubMed  Google Scholar 

  8. Michel, M. et al. An informal internet survey on the current state of consciousness science. Front. Psychol. 9, 2134 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Odegaard, B., Knight, R. T. & Lau, H. Should a few null findings falsify prefrontal theories of conscious perception? J. Neurosci. 37, 9593–9602 (2017).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  10. Tsuchiya, N., Andrillon, T. & Haun, A. A reply to “the unfolding argument”: beyond functionalism/behaviorism and towards a science of causal structure theories of consciousness. Conscious. Cogn. 79, 102877 (2020).

    Article  PubMed  Google Scholar 

  11. Lamme, V. A. F. How neuroscience will change our view on consciousness. Cogn. Neurosci. 1, 204–220 (2010).

    Article  PubMed  Google Scholar 

  12. Dehaene, S. & Naccache, L. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition 79, 1–37 (2001).

    CAS  Article  PubMed  Google Scholar 

  13. Lau, H. & Rosenthal, D. M. Empirical support for higher-order theories of conscious awareness. Trends Cogn. Sci. 15, 365–373 (2011).

    Article  PubMed  Google Scholar 

  14. Breitmeyer, B. G. Visual masking: past accomplishments, present status, future developments. Adv. Cogn. Psychol. 3, 9–20 (2007).

    Article  Google Scholar 

  15. Tsuchiya, N. & Koch, C. Continuous flash suppression reduces negative afterimages. Nat. Neurosci. 8, 1096–1101 (2005).

    CAS  Article  PubMed  Google Scholar 

  16. Levelt, W. J. M. On Binocular Rivalry (Van Gorcum Assen, 1965).

  17. Mack, A. et al. Inattentional Blindness (MIT Press, 1998).

  18. Simons, D. J. & Levin, D. T. Change blindness. Trends Cogn. Sci. 1, 261–267 (1997).

    CAS  Article  PubMed  Google Scholar 

  19. Kunst-Wilson, W. R. & Zajonc, R. B. Affective discrimination of stimuli that cannot be recognized. Science 207, 557–558 (1980).

    CAS  Article  PubMed  Google Scholar 

  20. Hobson, J. A. & Pace-Schott, E. F. The cognitive neuroscience of sleep: neuronal systems, consciousness and learning. Nat. Rev. Neurosci. 3, 679–693 (2002).

    CAS  Article  PubMed  Google Scholar 

  21. Alkire, M. T., Hudetz, A. G. & Tononi, G. Consciousness and anesthesia. Science 322, 876–880 (2008).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  22. Ro, T. What can TMS tell us about visual awareness? Cortex 46, 110–113 (2010).

    Article  PubMed  Google Scholar 

  23. Schiff, N. D., Giacino, J. T. & Fins, J. J. Deep brain stimulation, neuroethics, and the minimally conscious state. Arch. Neurol. 66, 697–702 (2009).

    Article  PubMed  Google Scholar 

  24. Tsuchiya, N., Wilke, M., Frässle, S. & Lamme, V. A. F. No-report paradigms: extracting the true neural correlates of consciousness. Trends Cogn. Sci. 19, 757–770 (2015).

    Article  PubMed  Google Scholar 

  25. Sergent, C. et al. Bifurcation in brain dynamics reveals a signature of conscious processing independent of report. Nat. Commun. 12, 1149 (2021).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  26. Frassle, S., Sommer, J., Jansen, A., Naber, M. & Einhauser, W. Binocular rivalry: frontal activity relates to introspection and action but not to perception. J. Neurosci. 34, 1738–1747 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Pitts, M. A., Padwal, J., Fennelly, D., Martínez, A. & Hillyard, S. A. Gamma band activity and the P3 reflect post-perceptual processes, not visual awareness. Neuroimage 101, 337–350 (2014).

    Article  PubMed  Google Scholar 

  28. Boly, M. et al. Are the neural correlates of consciousness in the front or in the back of the cerebral cortex? Clinical and neuroimaging evidence. J. Neurosci. 37, 9603–9613 (2017).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  29. Michel, M. & Morales, J. Minority reports: consciousness and the prefrontal cortex. Mind Lang. 35, 493–513 (2020).

    Article  Google Scholar 

  30. Derda, M. et al. The role of levels of processing in disentangling the ERP signatures of conscious visual processing. Conscious. Cogn. 73, 102767 (2019).

    Article  PubMed  Google Scholar 

  31. Windey, B., Gevers, W. & Cleeremans, A. Subjective visibility depends on level of processing. Cognition 129, 404–409 (2013).

    Article  PubMed  Google Scholar 

  32. Mashour, G. A., Roelfsema, P. R., Changeux, J. P. & Dehaene, S. Conscious processing and the global neuronal workspace hypothesis. Neuron 105, 776–798 (2020).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  33. Brown, R., Lau, H. & LeDoux, J. E. Understanding the higher-order approach to consciousness. Trends Cogn. Sci. 23, 754–768 (2019).

    Article  PubMed  Google Scholar 

  34. Tononi, G. An information integration theory of consciousness. BMC Neurosci. https://doi.org/10.1186/1471-2202-5-42 (2004).

  35. Tononi, G., Boly, M., Massimini, M. & Koch, C. Integrated information theory: from consciousness to its physical substrate. Nat. Rev. Neurosci. 17, 450–461 (2016).

    CAS  Article  PubMed  Google Scholar 

  36. Supèr, H., Spekreijse, H. & Lamme, V. A. F. Two distinct modes of sensory processing observed in monkey primary visual cortex (VI). Nat. Neurosci. 4, 304–310 (2001).

    Article  PubMed  Google Scholar 

  37. Lamme, V. A. F. The crack of dawn—perceptual functions and neural mechanisms that mark the transition from unconscious processing to conscious vision. In Open MIND Vol. 22 (eds Metzinger, T. K. & Windt, J. M.) (MIND Group, 2015); https://doi.org/10.15502/9783958570092

  38. Dehaene, S. & Changeux, J. P. Neural mechanisms for access to consciousness. Cogn. Neurosci. 3, 1145–1158 (2004).

    Google Scholar 

  39. Oizumi, M., Albantakis, L. & Tononi, G. From the phenomenology to the mechanisms of consciousness: integrated information theory 3.0. PLoS Comput. Biol. 10, e1003588 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Koch, C., Massimini, M., Boly, M. & Tononi, G. Posterior and anterior cortex—where is the difference that makes the difference? Nat. Rev. Neurosci. 17, 666 (2016).

    Article  CAS  PubMed  Google Scholar 

  41. Moher, D. et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. https://doi.org/10.1371/journal.pmed.1000097 (2009).

  42. Churchland, P. S. in Consciousness in Contemporary Science (eds Marcel, A. J. and Bisiach, E.) Ch. 13 (Clarendon Press, 1988).

  43. Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).

    Article  Google Scholar 

  44. Farrar, D. E. & Glauber, R. R. Multicollinearity in regression analysis: the problem revisited. Rev. Econ. Stat. 49, 92–107 (1967).

    Article  Google Scholar 

  45. Green, D. M. et al. Signal Detection Theory and Psychophysics Vol. 1 (Wiley New York, 1966).

  46. Reingold, E. M. & Merikle, P. M. Using direct and indirect measures to study perception without awareness. Percept. Psychophys. 44, 563–575 (1988).

    CAS  Article  PubMed  Google Scholar 

  47. Sandberg, K., Timmermans, B., Overgaard, M. & Cleeremans, A. Measuring consciousness: is one measure better than the other? Conscious. Cogn. 19, 1069–1078 (2010).

    Article  PubMed  Google Scholar 

  48. Ramsøy, T. Z. & Overgaard, M. Introspection and subliminal perception. Phenomenol. Cogn. Sci. 3, 1–23 (2004).

    Article  Google Scholar 

  49. Ojanen, V., Revonsuo, A. & Sams, M. Visual awareness of low-contrast stimuli is reflected in event-related brain potentials. Psychophysiology 40, 192–197 (2003).

    Article  PubMed  Google Scholar 

  50. Genetti, M., Britz, J., Michel, C. M. & Pegna, A. J. An electrophysiological study of conscious visual perception using progressively degraded stimuli. J. Vis. 10, 1–14 (2010).

    Article  Google Scholar 

  51. Watanabe, T., Náñez, J. E. & Sasaki, Y. Perceptual learning without perception. Nature 413, 844–848 (2001).

    CAS  Article  PubMed  Google Scholar 

  52. Koivisto, M. & Revonsuo, A. Event-related brain potential correlates of visual awareness. Neurosci. Biobehav. Rev. 34, 922–934 (2010).

    Article  PubMed  Google Scholar 

  53. Picton, T. W. The P300 wave of the human event-related potential. J. Clin. Neurophysiol. 9, 456–479 (1992).

    CAS  Article  PubMed  Google Scholar 

  54. Dehaene, S. & Changeux, J. P. Experimental and theoretical approaches to conscious processing. Neuron 70, 200–227 (2011).

    CAS  Article  PubMed  Google Scholar 

  55. Firestone, C. & Scholl, B. J. Cognition does not affect perception: evaluating the evidence for “top-down” effects. Behav. Brain Sci. 39, e229 (2016).

    Article  PubMed  Google Scholar 

  56. Irvine, E. Explaining what? Topoi 36, 95–106 (2017).

    Article  Google Scholar 

  57. Mynatt, C. R., Doherty, M. E. & Tweney, R. D. Confirmation bias in a simulated research environment: an experimental study of scientific inference. Q. J. Exp. Psychol. 29, 85–95 (1977).

    Article  Google Scholar 

  58. Greenwald, A. G., Pratkanis, A. R., Leippe, M. R. & Baumgardner, M. H. Under what conditions does theory obstruct research progress? Psychol. Rev. 93, 216–229 (1986).

    CAS  Article  PubMed  Google Scholar 

  59. Fraser, H., Parker, T., Nakagawa, S., Barnett, A. & Fidler, F. Questionable research practices in ecology and evolution. PLoS ONE 13, e0200303 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Yom, S. From methodology to practice. Comp. Polit. Stud. 48, 616–644 (2015).

    Article  Google Scholar 

  61. Fanelli, D. Negative results are disappearing from most disciplines and countries. Scientometrics 90, 891–904 (2012).

    Article  Google Scholar 

  62. Aru, J., Bachmann, T., Singer, W. & Melloni, L. Distilling the neural correlates of consciousness. Neurosci. Biobehav. Rev. 36, 737–746 (2012).

    Article  PubMed  Google Scholar 

  63. Popper, K. R. The Logic of Scientific Discovery (Basic Books, 1959).

  64. Melloni, L., Mudrik, L., Pitts, M. & Koch, C. Making the hard problem of consciousness easier. Science 372, 911–912 (2021).

    CAS  Article  PubMed  Google Scholar 

  65. Nickerson, R. S. Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 175–220 (1998).

    Article  Google Scholar 

  66. Koch, C., Massimini, M., Boly, M. & Tononi, G. Neural correlates of consciousness: progress and problems. Nat. Rev. Neurosci. 17, 307–321 (2016).

    CAS  Article  PubMed  Google Scholar 

  67. Baars, B. J. A thoroughly empirical approach to consciousness. Psyche (Stuttg.). 1, 1–18 (1994).

    Google Scholar 

  68. Sandberg, K., Andersen, L. M. & Overgaard, M. Using multivariate decoding to go beyond contrastive analyses in consciousness research. Front. Psychol. 5, 1250 (2014).

    PubMed  PubMed Central  Google Scholar 

  69. Hohwy, J. The neural correlates of consciousness: new experimental approaches needed? Conscious. Cogn. 18, 428–438 (2009).

    Article  PubMed  Google Scholar 

  70. De Graaf, T. A., Hsieh, P. J. & Sack, A. T. The ‘correlates’ in neural correlates of consciousness. Neurosci. Biobehav. Rev. 36, 191–197 (2012).

    Article  PubMed  Google Scholar 

  71. Tononi, G. Consciousness and complexity. Science 282, 1846–1851 (1998).

    CAS  Article  PubMed  Google Scholar 

  72. Dehaene, S., Sergent, C. & Changeux, J. P. A neuronal network model linking subjective reports and objective physiological data during conscious perception. Proc. Natl Acad. Sci. USA 100, 8520–8525 (2003).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  73. Del Cul, A., Baillet, S. & Dehaene, S. Brain dynamics underlying the nonlinear threshold for access to consciousness. PLoS Biol. 5, 2408–2423 (2007).

    Google Scholar 

  74. Lamme, V. A. F. Challenges for theories of consciousness: seeing or knowing, the missing ingredient and how to deal with panpsychism. Phil. Trans. R. Soc. B https://doi.org/10.1098/rstb.2017.0344 (2018).

  75. Lau, H. C. A higher order Bayesian decision theory of consciousness. Prog. Brain Res. https://doi.org/10.1016/S0079-6123(07)68004-2 (2007).

  76. Block, N. Finessing the bored monkey problem. Trends Cogn. Sci. 24, 167–168 (2020).

    Article  PubMed  Google Scholar 

  77. Block, N. What is wrong with the no-report paradigm and how to fix it. Trends Cogn. Sci. 23, 1003–1013 (2019).

    Article  PubMed  Google Scholar 

  78. Borenstein, M., Hedges, L. V., Higgins, J. P. T. & Rothstein, H. R. Introduction to Meta-analysis (John Wiley & Sons, 2011).

  79. Rolls, E. T., Huang, C.-C., Lin, C.-P., Feng, J. & Joliot, M. Automated anatomical labelling atlas 3. NeuroImage 206, 116189 (2020).

    Article  PubMed  Google Scholar 

  80. Chuang, Y.-S. label4MNI. GitHub https://github.com/yunshiuan/label4MRI (2019).

  81. MNI <-> TAL online converter v.1.0 https://bioimagesuiteweb.github.io/webapp/mni2tal.html (BioImage Suite Web, 2019).

  82. Lacadie, C. M., Fulbright, R. K., Rajeevan, N., Constable, R. T. & Papademetris, X. More accurate Talairach coordinates for neuroimaging using non-linear registration. NeuroImage 42, 717–725 (2008).

    Article  PubMed  Google Scholar 

  83. Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C. & Wager, T. D. Large-scale automated synthesis of human functional neuroimaging data. Nat. Methods 8, 665–670 (2011).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  84. Matlab 2020b (The MathWorks Inc., 2020).

  85. Yushkevich, P. A. et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage 31, 1116–1128 (2006).

    Article  PubMed  Google Scholar 

  86. Ahrens, J., Geveci, B. & Law, C. Paraview: an end-user tool for large data visualization. Vis. Handb. 717, 8 (2005).

    Google Scholar 

  87. Madan, C. R. Creating 3D visualizations of MRI data: a brief guide. F1000Research 4, 466 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  88. R Core Team. R: A Language and Environment for Statistical Computing v.4.0.0 (R Foundation for Statistical Computing, 2020).

  89. Van Rossum, G. & Drake, F. L. Python 3 Reference Manual (CreateSpace, 2009).

  90. Quinlan, J. R. Induction of decision trees. Mach. Learn. 1, 81–106 (1986).

    Google Scholar 

  91. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

    Google Scholar 

  92. Hair, J. F., Risher, J. J., Sarstedt, M. & Ringle, C. M. When to use and how to report the results of PLS-SEM. Eur. Bus. Rev. 31, 2–24 (2019).

    Article  Google Scholar 

  93. Akinwande, M. O., Dikko, H. G. & Samson, A. Variance inflation factor: as a condition for the inclusion of suppressor variable(s) in regression analysis. Open J. Stat. 5, 754–767 (2015).

    Article  Google Scholar 

  94. Altmann, A., Toloşi, L., Sander, O. & Lengauer, T. Permutation importance: a corrected feature importance measure. Bioinformatics 26, 1340–1347 (2010).

    CAS  Article  PubMed  Google Scholar 

  95. Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57, 289–300 (1995).

    Google Scholar 

  96. Foster, E. D. & Deardorff, A. Open Science Framework (OSF). J. Med. Libr. Assoc. 105, 203–206 (2017).

    Article  PubMed Central  Google Scholar 

Download references

Acknowledgements

This project was made possible through the support of grants from Templeton World Charity Foundation Inc. (no. TWCF0389 to L. Melloni, M.P. and L. Mudrik; no. TWCF0599 to L. Mudrik) and the National Science Foundation (no. BCS1829470 to M.P.). The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of Templeton World Charity Foundation Inc. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. L.Mudrik is a CIFAR Tanenbaum Fellow in the Brain, Mind, and Consciousness programme.

Author information

Authors and Affiliations

Authors

Contributions

The ConTraSt database was conceived by all authors. I.Y. collected and classified the papers in consultation with L. Mudrik and, when necessary, with L. Melloni and M.P. I.Y. performed all the analyses, created the website and drafted the manuscript. L. Mudrik, L. Melloni and M.P. edited the manuscript.

Corresponding author

Correspondence to Itay Yaron.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Ned Block, Axel Cleeremans and Boris Kotchoubey for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–11, Tables 1–9, Notes and Results.

Reporting Summary.

Peer Reviewer Reports.

Source data

Source Data Fig. 1

Number of collected papers after (‘post’) and before (‘pre’) the screening stage of our paper selection process.

Source Data Fig. 2

Number of experiments supporting and challenging each theory (Fig. 2a) and the cumulative number of papers mentioning, supporting and challenging the theories for each year (Fig. 2b–d, respectively).

Source Data Fig. 3

Number of experiments mentioning the theories in the introduction, being a priori designed to test theory predictions (theory driven) or post-hoc interpreting their findings in light of the theories, in experiments supporting and challenging the theories (Fig. 3a,b, respectively), and across theories for each year (Fig. 3c).

Source Data Fig. 4

Accuracy and AUC of the ROCs of the random forest classifier analysis (Fig. 4a,b, respectively). On the last sheet, the full data on the classification of theory support for each experiment in the database are provided.

Source Data Fig. 5

Frequency of each parameter value within experiments supporting each theory (separate sheet for each parameter).

Source Data Fig. 6

Brain areas reported in each panel, split according to the ‘consciousness type’ being studied (including all of the papers supporting each theory, or limiting the data to experiments focusing on content consciousness).

Source Data Fig. 7

Findings in the temporal domain for experiments using EEG, MEG or iEEG across the database (Fig. 7a), split according to the supported theory (Fig. 7b,c) and in theory-driven experiments supporting each theory (Fig. 7d,e).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yaron, I., Melloni, L., Pitts, M. et al. The ConTraSt database for analysing and comparing empirical studies of consciousness theories. Nat Hum Behav 6, 593–604 (2022). https://doi.org/10.1038/s41562-021-01284-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-021-01284-5

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing