Conscious experiences involve subjective qualities, such as colours, sounds, smells and emotions. In this Perspective, we argue that these subjective qualities can be understood in terms of their similarity to other experiences. This account highlights the role of memory in conscious experience, even for simple percepts. How an experience feels depends on implicit memory of the relationships between different perceptual representations within the brain. With more complex experiences such as emotions, explicit memories are also recruited. We draw inspiration from work in machine learning as well as the cognitive neuroscience of learning and decision making to make our case and discuss how the account could be tested in future experiments. The resulting findings might help to reveal the functions of subjective experience and inform current theoretical debates on consciousness.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
The complexity of the stream of consciousness
Communications Biology Open Access 03 November 2022
Subscribe to this journal
Receive 12 digital issues and online access to articles
$59.00 per year
only $4.92 per issue
Rent or buy this article
Get just this article for as long as you need it
Prices may be subject to local taxes which are calculated during checkout
Rosenthal, D. How to think about mental qualities. Phil. Issues 20, 368–393 (2010).
Clark, A. A Theory of Sentience (Oxford Univ. Press, 2000).
Sellars, W. Science, Perception and Reality (Humanities Press, 1963).
Squire, L. R. Memory and Brain vol. 315 (Oxford Univ. Press, 1987).
Schacter, D. L., Buckner, R. L. & Koutstaal, W. Memory, consciousness and neuroimaging. Phil. Trans. R. Soc. Lond. B 353, 1861–1878 (1998).
Tulving, E. in The Missing Link In Cognition: Origins Of Self-reflective Consciousness Vol. 364 (ed. Terrace, H. S.) 3–56 (Oxford Univ. Press, 2005).
LeDoux, J. E. & Lau, H. Seeing consciousness through the lens of memory. Curr. Biol. 30, R1018–R1022 (2020).
Bruner, J. S. & Leigh Minturn, A. Perceptual identification and perceptual organization. J. Gen. Psychol. 53, 21–28 (1955).
Allport, F. H. Theories of perception and the concept of structure: a review and critical analysis with an introduction to a dynamic-structural theory of behavior. Optom. Vis. Sci. 33, 216 (1955).
Gregory, R. L. Concepts And Mechanisms Of Perception (Charles Scribner’s Sons, 1974).
Graham, D. J., Friedenberg, J. D., Rockmore, D. N. & Field, D. J. Mapping the similarity space of paintings: image statistics and visual perception. Vis. Cogn. 18, 559–573 (2010).
Summerfield, C. & de Lange, F. P. Expectation in perceptual decision making: neural and computational mechanisms. Nat. Rev. Neurosci. 15, 745–756 (2014).
Murray, E. A., Wise, S. P. & Graham, K. S. The Evolution of Memory Systems: Ancestors, Anatomy, and Adaptations (Oxford Univ. Press, 2017).
Lamy, D., Carmel, T. & Peremen, Z. Prior conscious experience enhances conscious perception but does not affect response priming. Cognition 160, 62–81 (2017).
Marks, L. E. et al. Magnitude-matching: the measurement of taste and smell. Chem. Senses 13, 63–87 (1988).
Olshausen, B. A. & Field, D. J. Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 14, 481–487 (2004).
Rigotti, M. et al. The importance of mixed selectivity in complex cognitive tasks. Nature 497, 585–590 (2013).
Bernardi, S. et al. The geometry of abstraction in the hippocampus and prefrontal cortex. Cell 183, 954–967.e21 (2020).
Rosca, M., Weber, T., Gretton, A. & Mohamed, S. A case for new neural network smoothness constraints. Preprint at arXiv https://arxiv.org/abs/2012.07969 (2020).
Jin, P., Lu, L., Tang, Y. & Karniadakis, G. E. Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness. Neural Netw. 130, 85–99 (2020).
Bauer, A.-K. R., Debener, S. & Nobre, A. C. Synchronisation of neural oscillations and cross-modal influences. Trends Cogn. Sci. 24, 481–495 (2020).
Thoen, H. H., How, M. J., Chiou, T.-H. & Marshall, J. A different form of color vision in mantis shrimp. Science 343, 411–413 (2014).
Brown, R., Lau, H. & LeDoux, J. E. Understanding the higher-order approach to consciousness. Trends Cogn. Sci. 23, 754–768 (2019).
Lau, H. & Rosenthal, D. The higher-order view does not require consciously self-directed introspection: response to Malach. Trends Cogn. Sci. 15, 508–509 (2011).
Michel, M. & Lau, H. On the dangers of conflating strong and weak versions of a theory of consciousness. PhiMiSci https://doi.org/10.33735/phimisci.2020.II.54 (2020).
Lamme, V. A. F. Towards a true neural stance on consciousness. Trends Cogn. Sci. 10, 494–501 (2006).
Lamme, V. A. F. Why visual attention and awareness are different. Trends Cogn. Sci. 7, 12–18 (2003).
Malach, R. Conscious perception and the frontal lobes: comment on Lau and Rosenthal. Trends Cogn. Sci. 15, 507 (2011).
Zeki, S. The disunity of consciousness. Trends Cogn. Sci. 7, 214–218 (2003).
Zeki, S. Localization and globalization in conscious vision. Annu. Rev. Neurosci. 24, 57–86 (2001).
Macknik, S. L. Visual masking approaches to visual awareness. Prog. Brain Res. 155, 177–215 (2006).
Macknik, S. L. & Martinez-Conde, S. in The Cognitive Neurosciences (ed. Gazzaniga, M. S.) 1165–1175 (MIT Press, 2009).
Malach, R. Local neuronal relational structures underlying the contents of human conscious experience. Neurosci. Conscious. 2021, niab028 (2021).
Kouider, S. & Dehaene, S. Levels of processing during non-conscious perception: a critical review of visual masking. Phil. Trans. R. Soc. Lond. B 362, 857–875 (2007).
Mazzi, C., Savazzi, S. & Silvanto, J. On the ‘blindness’ of blindsight: what is the evidence for phenomenal awareness in the absence of primary visual cortex (V1)? Neuropsychologia 128, 103–108 (2019).
Silvanto, J., Cowey, A., Lavie, N. & Walsh, V. Making the blindsighted see. Neuropsychologia 45, 3346–3350 (2007).
Weil, R. S., Plant, G. T., James-Galton, M. & Rees, G. Neural correlates of hemianopic completion across the vertical meridian. Neuropsychologia 47, 457–464 (2009).
Weiskrantz, L. Prime-sight and blindsight. Conscious. Cogn. 11, 568–581 (2002).
Dehaene, S., Lau, H. & Kouider, S. What is consciousness, and could machines have it? Science 358, 486–492 (2017).
Dehaene, S. Consciousness And The Brain: Deciphering How the Brain Codes Our Thoughts (Penguin, 2014).
Baars, B. J. A Cognitive Theory of Consciousness (Cambridge Univ. Press, 1988).
Mashour, G. A., Roelfsema, P., Changeux, J.-P. & Dehaene, S. Conscious processing and the global neuronal workspace hypothesis. Neuron 105, 776–798 (2020).
Pounder, Z., Jacob, J., Evans, S., Loveday, C., Eardley, A. F., & Silvanto, J. Only minimal differences between individuals with congenital aphantasia and those with typical imagery on neuropsychological tasks that involve imagery. Cortex 148, 180–192 (2022).
Keogh, R. & Pearson, J. The blind mind: no sensory visual imagery in aphantasia. Cortex 105, 53–60 (2018).
Kay, L., Keogh, R., Andrillion, T. & Pearson, J. The eyes have it: the pupillary light response as a physiological index of aphantasia, sensory and phenomenological imagery strength. Elife 11, e72484 (2022).
Dehaene, S. & Naccache, L. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition 79, 1–37 (2001).
Lau, H. Volition and the functions of consciousness. Neurosci. Res. 65, S28 (2009).
Rosenthal, D. Consciousness And Mind (Clarendon, 2005).
Lycan, W. in The Stanford Encyclopedia of Philosophy (Stanford University, 2019).
Johnson, M. K. & Raye, C. L. Reality monitoring. Psychol. Rev. 88, 67–85 (1981).
Johnson, M. K. Reality monitoring: an experimental phenomenological approach. J. Exp. Psychol. Gen. 117, 390–394 (1988).
Simons, J. S., Garrison, J. R. & Johnson, M. K. Brain mechanisms of reality monitoring. Trends Cogn. Sci. 21, 462–473 (2017).
Lau, H. Consciousness, metacognition, & perceptual reality monitoring. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/ckbyf (2019).
McCurdy, L. Y. et al. Anatomical coupling between distinct metacognitive systems for memory and visual perception. J. Neurosci. 33, 1897–1906 (2013).
Morales, J., Lau, H. & Fleming, S. M. Domain-general and domain-specific patterns of activity supporting metacognition in human prefrontal cortex. J. Neurosci. 38, 3534–3546 (2018).
Garrison, J. R. et al. Testing continuum models of psychosis: no reduction in source monitoring ability in healthy individuals prone to auditory hallucinations. Cortex 91, 197–207 (2017).
Dijkstra, N., Mazor, M., Kok, P. & Fleming, S. Mistaking imagination for reality: congruent mental imagery leads to more liberal perceptual detection. Cognition 212, 104719 (2021).
Voss, U. et al. Induction of self awareness in dreams through frontal low current stimulation of gamma activity. Nat. Neurosci. 17, 810–812 (2014).
Fleming, S. M., Ryu, J., Golfinos, J. G. & Blackmon, K. E. Domain-specific impairment in metacognitive accuracy following anterior prefrontal lesions. Brain 137, 2811–2822 (2014).
Ye, Q., Zou, F., Lau, H., Hu, Y. & Kwok, S. C. Causal evidence for mnemonic metacognition in human precuneus. J. Neurosci. 38, 6379–6387 (2018).
Miyamoto, K. et al. Causal neural network of metamemory for retrospection in primates. Science 355, 188–193 (2017).
Cleeremans, A. et al. Learning to be conscious. Trends Cogn. Sci. 24, 112–123 (2020).
Fleming, S. M. Awareness as inference in a higher-order state space. Neurosci. Conscious 2020, niz020 (2020).
Whyte, C. J. & Smith, R. The predictive global neuronal workspace: a formal active inference model of visual consciousness. Prog. Neurobiol. 199, 101918 (2021).
Koenig-Robert, R. & Pearson, J. Why do imagery and perception look and feel so different? Phil. Trans. R. Soc. Lond. B 376, 20190703 (2021).
Muzur, A., Pace-Schott, E. F. & Hobson, J. A. The prefrontal cortex in sleep. Trends Cogn. Sci. 6, 475–481 (2002).
Zmigrod, L., Garrison, J. R., Carr, J. & Simons, J. S. The neural mechanisms of hallucinations: a quantitative meta-analysis of neuroimaging studies. Neurosci. Biobehav. Rev. 69, 113–123 (2016).
Narayanan, N. S., Rodnitzky, R. L. & Uc, E. Y. Prefrontal dopamine signaling and cognitive symptoms of Parkinson’s disease. Rev. Neurosci. 24, 267–278 (2013).
Fazekas, P. & Nemeth, G. Dream experiences and the neural correlates of perceptual consciousness and cognitive access. Phil. Trans. R. Soc. Lond. B 373, 20170356 (2018).
Mendoza-Halliday, D. & Martinez-Trujillo, J. C. Neuronal population coding of perceived and memorized visual features in the lateral prefrontal cortex. Nat. Commun. 8, 15471 (2017).
Gershman, S. J. The generative adversarial brain. Front. Artif. Intell. Appl. 2, 18 (2019).
Barceló, F., Suwazono, S. & Knight, R. T. Prefrontal modulation of visual processing in humans. Nat. Neurosci. 3, 399–403 (2000).
Bar, M. et al. Top-down facilitation of visual recognition. Proc. Natl Acad. Sci. USA 103, 449–454 (2006).
Bar, M. A cortical mechanism for triggering top-down facilitation in visual object recognition. J. Cogn. Neurosci. 15, 600–609 (2003).
Tsushima, Y., Sasaki, Y. & Watanabe, T. Greater disruption due to failure of inhibitory control on an ambiguous distractor. Science 314, 1786–1788 (2006).
Kriete, T., Noelle, D. C., Cohen, J. D. & O’Reilly, R. C. Indirection and symbol-like processing in the prefrontal cortex and basal ganglia. Proc. Natl Acad. Sci. USA 110, 16390–16395 (2013).
Held, R. & Hein, A. Movement-produced stimulation in the development of visually guided behavior. J. Comp. Physiol. Psychol. 56, 872–876 (1963).
Hubel, D. H., Wiesel, T. N. & LeVay, S. Plasticity of ocular dominance columns in monkey striate cortex. Phil. Trans. R. Soc. Lond. B 278, 377–409 (1977).
King, A. J., Hutchings, M. E., Moore, D. R. & Blakemore, C. Developmental plasticity in the visual and auditory representations in the mammalian superior colliculus. Nature 332, 73–76 (1988).
Weinberger, D. R. From neuropathology to neurodevelopment. Lancet 346, 552–557 (1995).
Merzenich, M. M. & Sameshima, K. Cortical plasticity and memory. Curr. Opin. Neurobiol. 3, 187–196 (1993).
Fiser, J. & Aslin, R. N. Unsupervised statistical learning of higher-order spatial structures from visual scenes. Psychol. Sci. 12, 499–504 (2001).
Recanzone, G. H. Rapidly induced auditory plasticity: the ventriloquism aftereffect. Proc. Natl Acad. Sci. USA 95, 869–875 (1998).
Moser, E. I., Kropff, E. & Moser, M.-B. Place cells, grid cells, and the brain’s spatial representation system. Annu. Rev. Neurosci. 31, 69–89 (2008).
O’Keefe, J. & Burgess, N. Dual phase and rate coding in hippocampal place cells: theoretical significance and relationship to entorhinal grid cells. Hippocampus 15, 853–866 (2005).
Hafting, T., Fyhn, M., Molden, S., Moser, M.-B. & Moser, E. I. Microstructure of a spatial map in the entorhinal cortex. Nature 436, 801–806 (2005).
Qasim, S. E., Fried, I. & Jacobs, J. Phase precession in the human hippocampus and entorhinal cortex. Cell 184, 3242–3255.e10 (2021).
Doeller, C. F., Barry, C. & Burgess, N. Evidence for grid cells in a human memory network. Nature 463, 657–661 (2010).
Bellmund, J. L. S., Gärdenfors, P., Moser, E. I. & Doeller, C. F. Navigating cognition: spatial codes for human thinking. Science 362, eaat6766 (2018).
Nau, M., Navarro Schröder, T., Bellmund, J. L. S. & Doeller, C. F. Hexadirectional coding of visual space in human entorhinal cortex. Nat. Neurosci. 21, 188–190 (2018).
Bellmund, J. L., Deuker, L., Navarro Schröder, T. & Doeller, C. F. Grid-cell representations in mental simulation. eLife 5, e17089 (2016).
Constantinescu, A. O., O’Reilly, J. X. & Behrens, T. E. J. Organizing conceptual knowledge in humans with a gridlike code. Science 352, 1464–1468 (2016).
Mark, S., Moran, R., Parr, T., Kennerley, S. W. & Behrens, T. E. J. Transferring structural knowledge across cognitive maps in humans and models. Nat. Commun. 11, 4783 (2020).
Aronov, D., Nevers, R. & Tank, D. W. Mapping of a non-spatial dimension by the hippocampal–entorhinal circuit. Nature 543, 719–722 (2017).
Bao, X. et al. Grid-like neural representations support olfactory navigation of a two-dimensional odor space. Neuron 102, 1066–1075.e5 (2019).
LeDoux, J. The Deep History of Ourselves: the Four-Billion-Year Story of How We Got Conscious Brains (Penguin, 2019).
LeDoux, J. E. Thoughtful feelings. Curr. Biol. 30, R619–R623 (2020).
Johnson-Laird, P. N. Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness (Harvard Univ. Press, 1983).
LeDoux, J. E. What emotions might be like in other animals. Curr. Biol. 31, R824–R829 (2021).
Passingham, R. E., Bengtsson, S. L. & Lau, H. C. Medial frontal cortex: from self-generated action to reflection on one’s own performance. Trends Cogn. Sci. 14, 16–21 (2010).
Wagner, D. D., Haxby, J. V. & Heatherton, T. F. The representation of self and person knowledge in the medial prefrontal cortex. Wiley Interdiscip. Rev. Cogn. Sci. 3, 451–470 (2012).
Sutherland, K. & Bryant, R. A. Autobiographical memory and the self-memory system in posttraumatic stress disorder. J. Anxiety Disord. 22, 555–560 (2008).
Olsson, A., Knapska, E. & Lindström, B. The neural and computational systems of social learning. Nat. Rev. Neurosci. 21, 197–212 (2020).
Isoda, M. The role of the medial prefrontal cortex in moderating neural representations of self and other in primates. Annu. Rev. Neurosci. 44, 295–313 (2021).
Ritchey, M., Libby, L. A. & Ranganath, C. Cortico-hippocampal systems involved in memory and cognition: the PMAT framework. Prog. Brain Res. 219, 45–64 (2015).
Gilboa, A. & Marlatte, H. Neurobiology of schemas and schema-mediated memory. Trends Cogn. Sci. 21, 618–631 (2017).
Damasio, A. & Carvalho, G. B. The nature of feelings: evolutionary and neurobiological origins. Nat. Rev. Neurosci. 14, 143–152 (2013).
Yeterian, E. H., Pandya, D. N., Tomaiuolo, F. & Petrides, M. The cortical connectivity of the prefrontal cortex in the monkey brain. Cortex 48, 58–81 (2012).
Curtis, C. E. Prefrontal and parietal contributions to spatial working memory. Neuroscience 139, 173–180 (2006).
Vilberg, K. L. & Rugg, M. D. Memory retrieval and the parietal cortex: a review of evidence from a dual-process perspective. Neuropsychologia 46, 1787–1799 (2008).
Cabeza, R., Ciaramelli, E., Olson, I. R. & Moscovitch, M. The parietal cortex and episodic memory: an attentional account. Nat. Rev. Neurosci. 9, 613–625 (2008).
Fischer, M., Moscovitch, M. & Alain, C. A systematic review and meta-analysis of memory-guided attention: frontal and parietal activation suggests involvement of fronto-parietal networks. Wiley Interdiscip. Rev. Cogn. Sci. 12, e1546 (2021).
Wagner, A. D., Shannon, B. J., Kahn, I. & Buckner, R. L. Parietal lobe contributions to episodic memory retrieval. Trends Cogn. Sci. 9, 445–453 (2005).
Berryhill, M. E., Picasso, L., Arnold, R., Drowos, D. & Olson, I. R. Similarities and differences between parietal and frontal patients in autobiographical and constructed experience tasks. Neuropsychologia 48, 1385–1393 (2010).
Katsuki, F. & Constantinidis, C. Unique and shared roles of the posterior parietal and dorsolateral prefrontal cortex in cognitive functions. Front. Integr. Neurosci. 6, 17 (2012).
Berryhill, M. E., Phuong, L., Picasso, L., Cabeza, R. & Olson, I. R. Parietal lobe and episodic memory: bilateral damage causes impaired free recall of autobiographical memory. J. Neurosci. 27, 14415–14423 (2007).
Tanaka, K. Z. & McHugh, T. J. The hippocampal engram as a memory index. J. Exp. Neurosci. 12, 1179069518815942 (2018).
Teyler, T. J. & Rudy, J. W. The hippocampal indexing theory and episodic memory: updating the index. Hippocampus 17, 1158–1169 (2007).
Wittkuhn, L., Chien, S., Hall-McMaster, S. & Schuck, N. W. Replay in minds and machines. Neurosci. Biobehav. Rev. https://doi.org/10.1016/j.neubiorev.2021.08.002 (2021).
Endo, K., Tsuchimoto, Y. & Kazama, H. Synthesis of conserved odor object representations in a random, divergent-convergent network. Neuron 108, 367–381.e5 (2020).
Pashkovski, S. L. et al. Structure and flexibility in cortical representations of odour space. Nature 583, 253–258 (2020).
Dasgupta, S., Stevens, C. F. & Navlakha, S. A neural algorithm for a fundamental computing problem. Science 358, 793–796 (2017).
Dasgupta, S., Sheehan, T. C., Stevens, C. F. & Navlakha, S. A neural data structure for novelty detection. Proc. Natl Acad. Sci. USA 115, 13093–13098 (2018).
Dunsmoor, J. E. & Murphy, G. L. Categories, concepts, and conditioning: how humans generalize fear. Trends Cogn. Sci. 19, 73–77 (2015).
Tolman, E. C. Cognitive maps in rats and men. Psychol. Rev. 55, 189–208 (1948).
Gershman, S. J., Markman, A. B. & Otto, A. R. Retrospective revaluation in sequential decision making: a tale of two systems. J. Exp. Psychol. Gen. 143, 182–194 (2014).
Momennejad, I., Otto, A. R., Daw, N. D. & Norman, K. A. Offline replay supports planning in human reinforcement learning. eLife 7, e32548 (2018).
Schacter, D. L. & Addis, D. R. On the constructive episodic simulation of past and future events. Behav. Brain Sci. 30, 331–332 (2007).
Schacter, D. L. Constructive memory: past and future. Dialogues Clin. Neurosci. 14, 7–18 (2012).
Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L. & Platt, M. L. A primer on foraging and the explore/exploit trade-off for psychiatry research. Neuropsychopharmacology 42, 1931–1939 (2017).
Mazor, M., Friston, K. J. & Fleming, S. M. Distinct neural contributions to metacognition for detecting, but not discriminating visual stimuli. eLife 9, e53900 (2020).
Miyamoto, K., Setsuie, R., Osada, T. & Miyashita, Y. Reversible silencing of the frontopolar cortex selectively impairs metacognitive judgment on non-experience in primates. Neuron 97, 980–989.e6 (2018).
Martino, B. D., De Martino, B., Fleming, S. M., Garrett, N. & Dolan, R. J. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).
Donoso, M., Collins, A. G. E. & Koechlin, E. Foundations of human reasoning in the prefrontal cortex. Science 344, 1481–1486 (2014).
Morales, J. & Lau, H. in Qualitative Consciousness: Themes from the Philosophy of David Rosenthal (ed. Weisberg, J.) (Cambridge Univ. Press, 2021).
Gherman, S. & Philiastides, M. G. Human VMPFC encodes early signatures of confidence in perceptual decisions. eLife 7, e38293 (2018).
Bang, D. & Fleming, S. M. Distinct encoding of decision confidence in human medial prefrontal cortex. Proc. Natl Acad. Sci. USA 115, 6082–6087 (2018).
Wittmann, M. K. et al. Self-other mergence in the frontal cortex during cooperation and competition. Neuron 91, 482–493 (2016).
S.M.F. is funded by a Wellcome/Royal Society Sir Henry Dale Fellowship (206648/Z/17/Z) and a Philip Leverhulme Prize from the Leverhulme Trust. The Wellcome Centre for Human Neuroimaging is supported by core funding from the Wellcome Trust (203147/Z/16/Z). The Max Planck UCL Centre is a joint initiative supported by UCL and the Max Planck Society.
The authors declare no competing interests.
Peer review information
Nature Reviews Psychology thanks Peter Fazekas, Rafael Malach and the other, anonymous, reviewer for their contribution to the peer review of this work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lau, H., Michel, M., LeDoux, J.E. et al. The mnemonic basis of subjective experience. Nat Rev Psychol 1, 479–488 (2022). https://doi.org/10.1038/s44159-022-00068-6
This article is cited by
The complexity of the stream of consciousness
Communications Biology (2022)