Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Neuroimaging

Decoding mental states from brain activity in humans

Nature Reviews Neuroscience volume 7, pages 523–534 (2006)Cite this article

Key Points

  • Understanding whether cognitive and perceptual states can be decoded from brain activity alone is a fundamental question in cognitive neuroscience. It is not only relevant for scientific theories of how information is encoded in the brain, but also has important practical and ethical implications.

  • Non-invasive techniques such as functional MRI (fMRI) can be used to record signals related to brain activity in humans from many locations in the brain simultaneously. However, many conventional approaches to analysing these data rely on considering signal changes at each location independently of all the other locations in the brain

  • These conventional approaches have proven successful in elucidating many aspects of the relationship between cognitive and mental states and brain activity. However, recent advances in data analysis procedures raise the possibility of deciphering additional and complementary information from neuroimaging data.

  • Recently, a powerful approach has emerged that applies pattern-recognition techniques to neuroimaging data. The new strategy is to decode a person's current mental state by learning to recognize characteristic spatial patterns of brain activity associated with different mental states. This takes into account not just activity at single locations but the full spatial pattern of activity. Such pattern-based decoding reveals that substantially more information is encoded in fMRI signals than was previously recognised.

  • These new approaches have a particular use in addressing the question of how information regarding perceptual and cognitive states is encoded in the human brain. Pattern-based decoding has now been successfully used to reveal the principles underlying the representation of objects in the ventral visual pathway. It can also reveal conscious and unconscious sensory representations of individual features, and can be used to track dynamic changes in the contents of consciousness over time.

  • Decoding approaches therefore provide a particularly sensitive way to determine what types of information are represented in the spatially distributed pattern of brain responses recorded with current neuroimaging techniques. However, for more general applications, important technical and methodological barriers remain to be overcome, including the ability of such approaches to generalize across individuals and different cognitive and perceptual states.

  • As these techniques have the possibility to reveal covert or unconscious mental states, they raise important ethical and privacy concerns. These can be addressed within existing ethical frameworks, but nevertheless necessitate careful and considered engagement by the neuroimaging community.

Abstract

Recent advances in human neuroimaging have shown that it is possible to accurately decode a person's conscious experience based only on non-invasive measurements of their brain activity. Such 'brain reading' has mostly been studied in the domain of visual perception, where it helps reveal the way in which individual experiences are encoded in the human brain. The same approach can also be extended to other types of mental state, such as covert attitudes and lie detection. Such applications raise important ethical issues concerning the privacy of personal thought.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Decoding visual object perception from fMRI responses.
Figure 2: Decoding perceived orientation from sampling patterns in the early visual cortex.
Figure 3: Tracking dynamic mental processes.
Figure 4: Decoding unconscious processing.

References

  1. Farah, M. J. Emerging ethical issues in neuroscience. Nature Neurosci. 5, 1123–1129 (2002).

    Article  CAS  PubMed  Google Scholar 

  2. Cox, D. D. & Savoy, R. L. Functional magnetic resonance imaging (fMRI) 'brain reading': detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270 (2003). This study compares various classification techniques and outlines important principles of decoding-based fMRI research.

    Article  PubMed  Google Scholar 

  3. O'Craven, K. M. & Kanwisher, N. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. J. Cogn. Neurosci. 12, 1013–1023 (2000).

    Article  CAS  PubMed  Google Scholar 

  4. Haxby, J. V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001). One of the first studies using pattern-based analysis to investigate the nature of object representations in the human ventral visual cortex.

    Article  CAS  PubMed  Google Scholar 

  5. Carlson, T. A., Schrater, P. & He, S. Patterns of activity in the categorical representation of objects. J. Cogn. Neurosci. 15, 704–717 (2003).

    Article  PubMed  Google Scholar 

  6. Mitchell, T. M. et al. Classifying instantaneous cognitive states from FMRI data. AMIA Annu. Symp. Proc. 465–469 (2003).

  7. Mitchell, T. M., Hutchinson, R., Niculescu, R. S., Pereira, F. & Wang, X. Learning to decode cognitive states from brain images. Machine Learning 57, 145–175 (2004).

    Article  Google Scholar 

  8. Kamitani, Y. & Tong, F. Decoding the visual and subjective contents of the human brain. Nature Neurosci. 8, 679–685 (2005). The first application of multivariate classification to reveal processing of features in the primary visual cortex represented below the resolution of fMRI.

    Article  CAS  PubMed  Google Scholar 

  9. Haynes, J. D. & Rees, G. Predicting the orientation of invisible stimuli from activity in primary visual cortex. Nature Neurosci. 8, 686–691 (2005). This study directly compares perceptual performance with the performance of a decoder trained on fMRI-signals from the early visual cortex.

    Article  CAS  PubMed  Google Scholar 

  10. Haynes, J. D. & Rees, G. Predicting the stream of consciousness from activity in human visual cortex. Curr. Biol. 15, 1301–1307 (2005). This work reveals the potential power of multivariate decoding to track perception quasi-online on a second-to-second basis.

    Article  CAS  PubMed  Google Scholar 

  11. Kamitani, Y. & Tong, F. Decoding motion direction from activity in human visual cortex. J. Vision 5, 152a (2005).

    Article  Google Scholar 

  12. Kriegeskorte, N., Goebel, R. & Bandettini, P. Information-based functional brain mapping. Proc. Natl Acad. Sci. USA 103, 3863–3868 (2006). This study introduces the 'searchlight' approach that searches across the entire brain for specific local patterns that encode information about a cognitive or perceptual state.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. LaConte, S., Strother, S., Cherkassky, V., Anderson, J. & Hu, X. Support vector machines for temporal classification of block design fMRI data. Neuroimage 26, 317–329 (2005).

    Article  PubMed  Google Scholar 

  14. Mourao-Miranda, J., Bokde, A. L., Born, C., Hampel, H. & Stetter, M. Classifying brain states and determining the discriminating activation patterns: support vector machine on functional MRI data. Neuroimage 28, 980–995 (2005).

    Article  PubMed  Google Scholar 

  15. O'Toole, A., Jiang, F., Abdi, H. & Haxby, J. V. Partially distributed representation of objects and faces in ventral temporal cortex. J. Cogn. Neurosci. 17, 580–590 (2005).

    Article  PubMed  Google Scholar 

  16. Polyn, S. M., Natu, V. S., Cohen, J. D. & Norman, K. A. Category-specific cortical activity precedes retrieval during memory search. Science 310, 1963–1966 (2005).

    Article  CAS  PubMed  Google Scholar 

  17. Sidtis, J. J., Strother, S. C. & Rottenberg, D. A. Predicting performance from functional imaging data: methods matter. Neuroimage 20, 615–624 (2003).

    Article  PubMed  Google Scholar 

  18. Logothetis, N. K. & Pfeuffer, J. On the nature of the BOLD fMRI contrast mechanism. Magn. Reson. Imaging 22, 1517–1531 (2004).

    Article  PubMed  Google Scholar 

  19. Allison, T. et al. Face recognition in human extrastriate cortex. J. Neurophysiol. 71, 821–825 (1994).

    Article  CAS  PubMed  Google Scholar 

  20. Kanwisher, N., McDermott, J. & Chun, M. M. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 17, 4302–4311 (1997).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Epstein, R. & Kanwisher, N. A cortical representation of the local visual environment. Nature 392, 598–601 (1999).

    Article  CAS  Google Scholar 

  22. Engel, S. A. et al. FMRI of human visual cortex. Nature 369, 525 (1994).

    Article  CAS  PubMed  Google Scholar 

  23. Sereno, M. I. et al. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 268, 889–893 (1995).

    Article  CAS  PubMed  Google Scholar 

  24. Dehaene, S. et al. Inferring behavior from functional brain images. Nature Neurosci. 1, 549–550 (1998).

    Article  CAS  PubMed  Google Scholar 

  25. Tsao, D. Y., Freiwald, W. A., Tootell, R. B. & Livingstone, M. S. A cortical region consisting entirely of face-selective cells. Science 311, 670–674 (2006).

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  26. Downing, P. E., Jiang, Y., Shuman, M. & Kanwisher, N. A cortical area selective for visual processing of the human body. Science 293, 2470–2473 (2001).

    Article  CAS  PubMed  Google Scholar 

  27. Cohen, L. et al. The visual word form area: spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients, Brain 123, 291–307 (2000).

    Article  PubMed  Google Scholar 

  28. Downing, P. E., Chan, A. W. Y., Peelen, M. V., Dodds, C. M. & Kanwisher, N. Domain specificity in visual cortex. Cereb. Cortex Dec 7 2005 (doi: 10.1093/cercor/bhj086).

  29. Ishai, A., Schmidt, C. F. & Boesiger, P. Face perception is mediated by a distributed cortical network. Brain Res. Bull. 67, 87–93 (2005).

    Article  PubMed  Google Scholar 

  30. Edelman, S., Grill-Spector, K., Kushnir, T. & Malach, R. Towards direct visualization of the internal shape space by fMRI. Psychobiology 26, 309–321 (1998). This early work is the first application of multivariate techniques to the study of object representation. Of special interest is the demonstration that shape space is reflected in the similarity space between evoked cortical responses.

    Article  Google Scholar 

  31. Spiridon, M. & Kanwisher, N. How distributed is visual category information in human occipito-temporal cortex? An fMRI study. Neuron 35, 1157–1165 (2002).

    Article  CAS  PubMed  Google Scholar 

  32. Tanaka, K. Mechanisms of visual object recognition: monkey and human studies. Curr. Opin. Neurobiol. 7, 523–529 (1997).

    Article  CAS  PubMed  Google Scholar 

  33. Wang, G., Tanaka, K. & Tanifuji, M. Optical imaging of functional organization in the monkey inferotemporal cortex. Science 272, 1665–1668 (1996).

    Article  CAS  PubMed  Google Scholar 

  34. Obermayer, K. & Blasdel, G. G. Geometry of orientation and ocular dominance columns in monkey striate cortex. J. Neurosci. 13, 4114–4129 (1993).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. James, W. The Principles of Psychology (Henry Holt, New York, 1890).

    Google Scholar 

  36. Blake, R. & Logothetis, N. K. Visual competition. Nature Rev. Neurosci. 3, 13–21 (2002).

    Article  CAS  Google Scholar 

  37. Haynes, J. D., Deichmann, R. & Rees, G. Eye-specific effects of binocular rivalry in the human lateral geniculate nucleus. Nature 438, 496–499 (2005).

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  38. Brown, R. J. & Norcia, A. M. A method for investigating binocular rivalry in real-time with the steady-state VEP. Vision Res. 37, 2401–2408 (1997).

    Article  CAS  PubMed  Google Scholar 

  39. Hung, C. P., Kreiman, G., Poggio, T. & DiCarlo, J. J. Fast readout of object identity from macaque inferior temporal cortex. Science 310, 863–866 (2005).

    Article  CAS  PubMed  Google Scholar 

  40. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G. & Malach, R. Intersubject synchronization of cortical activity during natural vision. Science 303, 1634–1640 (2004).

    Article  CAS  PubMed  Google Scholar 

  41. Bartels, A. & Zeki, S. Functional brain mapping during free viewing of natural scenes. Hum. Brain Mapp. 21, 75–85 (2004),

    Article  PubMed  Google Scholar 

  42. Yarbus, A. L. Eye Movements and Vision (Plenum, New York, 1967).

    Book  Google Scholar 

  43. Duhamel, J. R., Colby, C. L. & Goldberg, M. E. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255, 90–92 (1992).

    Article  CAS  PubMed  Google Scholar 

  44. Davatzikos, C. et al. Classifying spatial patterns of brain activity with machine learning methods: application to lie detection. Neuroimage 28, 663–668 (2005). This study is the first demonstration that multivariate decoding can be applied to lie detection.

    Article  CAS  PubMed  Google Scholar 

  45. Langleben, D. D. et al. Telling truth from lie in individual subjects with fast event-related fMRI. Hum. Brain Mapp. 26, 262–272 (2005).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Kozel, F. A. et al. Detecting deception using functional magnetic resonance imaging. Biol. Psychiatry 58, 605–613 (2005).

    Article  PubMed  Google Scholar 

  47. Marcel, A. J. Conscious and unconscious perception: experiments on visual masking and word recognition. Cognit. Psychol. 15, 197–237 (1983).

    Article  CAS  PubMed  Google Scholar 

  48. Reingold, E. M. & Merikle, P. M. Using direct and indirect measures to study perception without awareness. Percept. Psychophys. 44, 563–575 (1988).

    Article  CAS  PubMed  Google Scholar 

  49. Dehaene, S. et al. Imaging unconscious semantic priming. Nature 395, 597–600 (1998)

    Article  CAS  PubMed  Google Scholar 

  50. Crick, F., & Koch, C. Are we aware of neural activity in primary visual cortex? Nature 375, 121–123 (1995)

    Article  CAS  PubMed  Google Scholar 

  51. Phelps, E. A. et al. Performance on indirect measures of race evaluation predicts amygdala activation. J. Cogn. Neurosci. 12, 729–738 (2000).

    Article  CAS  PubMed  Google Scholar 

  52. Libet, B., Gleason, C. A., Wright, E. W., Pearl, D. K. Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential). The unconscious initiation of a freely voluntary act. Brain 106, 623–642 (1983).

    Article  PubMed  Google Scholar 

  53. Haggard, P. & Eimer, M. On the relation between brain potentials and the awareness of voluntary movements. Exp. Brain. Res. 126, 128–133 (1999).

    Article  CAS  PubMed  Google Scholar 

  54. Tesla, N. Tremendous New Power to be Unleashed. Kansas City Journal-Post (10 Sep 1933).

    Google Scholar 

  55. Obrig, H. & Villringer, A. Beyond the visible — imaging the human brain with light. J. Cereb. Blood Flow Metab. 23, 1–18 (2003).

    Article  PubMed  Google Scholar 

  56. Suppes, P., Han, B., Epelboim, J. & Lu, Z. L. Invariance between subjects of brain wave representations of language. Proc. Natl Acad. Sci. USA 96, 12953–12958 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Friston, K. J. et al. Spatial registration and normalisation of images. Hum. Brain Mapp. 2, 165–189 (1995).

    Article  Google Scholar 

  58. Fischl, B., Sereno, M. I., Tootell, R. B. & Dale, A. M. High resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 8, 272–284 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Grill-Spector, K., Henson, R. & Martin, A. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn. Sci. 10, 14–23 (2006).

    Article  PubMed  Google Scholar 

  60. Mountcastle, V. B. The columnar organization of the neocortex. Brain 120, 701–722 (1997).

    Article  PubMed  Google Scholar 

  61. Horton, J. C. & Adams, D. L. The cortical column: a structure without a function. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 837–862 (2005).

    Article  PubMed Central  PubMed  Google Scholar 

  62. Poldrack, R. A. Can cognitive processes be inferred from neuroimaging data? Trends Cogn. Sci. 10, 59–63 (2006).

    Article  PubMed  Google Scholar 

  63. Cunningham, W. A. et al. Separable neural components in the processing of black and white faces. Psychol. Sci. 15, 806–813 (2004).

    Article  PubMed  Google Scholar 

  64. Richeson, J. A. et al. An fMRI investigation of the impact of interracial contact on executive function. Nature Neurosci. 6, 1323–1328 (2003).

    Article  CAS  PubMed  Google Scholar 

  65. Beauregard, M., Levesque, J. & Bourgouin, P. Neural correlates of conscious self-regulation of emotion. J. Neurosci. 21, RC165 (2001).

  66. Phan, K. L., Wager, T., Taylor, S. F. & Liberzon, I. Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI. Neuroimage 16, 331–348 (2002).

    Article  PubMed  Google Scholar 

  67. Canli, T. & Amin, Z. Neuroimaging of emotion and personality: scientific evidence and ethical considerations. Brain Cogn. 50, 414–431 (2002).

    Article  PubMed  Google Scholar 

  68. McCloskey, M. S., Phan, K. L. & Coccaro, E. F. Neuroimaging and personality disorders. Curr. Psychiatry Rep. 7, 65–72 (2005).

    Article  PubMed  Google Scholar 

  69. Pridmore, S., Chambers, A. & McArthur, M. Neuroimaging in psychopathy. Aust. N. Z. J. Psychiatry 39, 856–865 (2005).

    Article  PubMed  Google Scholar 

  70. Raine, A. et al. Reduced prefrontal and increased subcortical brain functioning assessed using positron emission tomography in predatory and affective murderers. Behav. Sci. Law 16, 319–332 (1998).

    Article  CAS  PubMed  Google Scholar 

  71. Childress, A. R. et al. Limbic activation during cue-induced cocaine craving. Am. J. Psychiatry 156, 11–18 (1999).

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  72. McClure, S. M. et al. Neural correlates of behavioral preference for culturally familiar drinks. Neuron 44, 379–387 (2004).

    Article  CAS  PubMed  Google Scholar 

  73. Heekeren, H. R., Marrett, S., Bandettini, P. A. & Ungerleider, L. G. A general mechanism for perceptual decision-making in the human brain. Nature 431, 859–862 (2004).

    Article  CAS  PubMed  Google Scholar 

  74. Levy, D. E. et al. Differences in cerebral blood flow and glucose utilization in vegetative versus locked-in patients. Ann. Neurol. 22, 673–682 (1987).

    Article  CAS  PubMed  Google Scholar 

  75. Laureys, S., Perrin, F., Schnakers, C., Boly, M. & Majerus, S. Residual cognitive function in comatose, vegetative and minimally conscious states. Curr. Opin. Neurol. 18, 726–733 (2005).

    Article  PubMed  Google Scholar 

  76. Farah, M. J. Neuroethics: the practical and the philosophical. Trends Cogn. Sci. 9, 34–40 (2005).

    Article  PubMed  Google Scholar 

  77. Brammer, M. Brain scam? Nature Neurosci. 7, 1015 (2004).

    Article  CAS  PubMed  Google Scholar 

  78. Dickson, K. & McMahon, M. Will the law come running? The potential role of 'brain fingerprinting' in crime investigation and adjudication in Australia. J. Law Med. 13, 204–222 (2005).

    PubMed  Google Scholar 

  79. Dewan, E. M. Occipital alpha rhythm, eye position and lens accommodation. Nature 214, 975–977 (1967).

    Article  CAS  PubMed  Google Scholar 

  80. Nicolelis, M. A. Actions from thoughts. Nature 409, 403–407 (2001).

    Article  CAS  PubMed  Google Scholar 

  81. Andersen, R. A., Burdick, J. W., Musallam, S., Pesaran, B. & Cham, J. G. Cognitive neural prosthetics. Trends Cogn. Sci. 8, 486–493 (2004).

    Article  CAS  PubMed  Google Scholar 

  82. Blankertz, B. et al. Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 127–131 (2003).

    Article  PubMed  Google Scholar 

  83. Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G. & Vaughan, T. M. Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002).

    Article  PubMed  Google Scholar 

  84. Wolpaw, J. R. & McFarland, D. J. Control of a two-dimensional movement signal by a noninvasive brain–computer interface in humans. Proc. Natl Acad. Sci. USA 101, 17849–17854 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  85. Kreiman, G., Koch, C. & Fried, I. Category-specific visual responses of single neurons in the human median temporal lobe. Nature Neurosci. 3, 946–953 (2000).

    Article  CAS  PubMed  Google Scholar 

  86. Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C. & Fried, I. Invariant visual representation by single neurons in the human brain. Nature 435, 1102–1107 (2005). Here, multivariate decoding is applied to simultaneous recordings of spike trains from the human medial temporal lobe.

    Article  CAS  PubMed  Google Scholar 

  87. Kennedy, P. R. & Bakay, R. A. Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9, 1707–1711 (1998).

    Article  CAS  PubMed  Google Scholar 

  88. Birbaumer, N. et al. A spelling device for the paralysed. Nature 398, 297–298 (1999).

    Article  CAS  PubMed  Google Scholar 

  89. Weiskopf, N. et al. Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI). J. Physiol. (Paris) 98, 357–373 (2004).

    Article  Google Scholar 

  90. Stanley, G. B., Li, F. F. & Dan, Y. Reconstruction of natural scenes from ensemble responses in the lateral geniculate nucleus. J. Neurosci. 19, 8036–8042 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  91. Ekman, P. & O'Sullivan, M. Who can catch a liar? Am. Psychol. 46, 913–920 (1991).

    Article  CAS  PubMed  Google Scholar 

  92. Marston, W. M. The systolic blood pressure symptoms of deception. J. Exp. Psy. 2, 117–163 (1917).

    Article  Google Scholar 

  93. Geddes, L. A. History of the polygraph, an instrument for the detection of deception. Biomed. Eng. 8, 154–156 (1973).

    CAS  PubMed  Google Scholar 

  94. Burtt, H. E. The inspiration–expiration ratio during truth and falsehood. J. Exp. Psy. 4, 1–23 (1921).

    Article  Google Scholar 

  95. Thackeray, R. J. & Orne, M. T. A comparison of physiological indices in detection of deception. Psychophysiol. 4, 329–339 (1968).

    Article  Google Scholar 

  96. Horvath, F. Detecting deception: the promise and the reality of voice stress analysis. J. Forensic Sci. 27, 340–351 (1982).

    Article  CAS  PubMed  Google Scholar 

  97. Pavlidis, I., Eberhardt, N. L. & Levine, J. A. Seeing through the face of deception. Nature 415, 35 (2002).

    Article  CAS  PubMed  Google Scholar 

  98. Pollina, D. A., Dollins, A. B., Senter, S. M., Krapohl, D. J. & Ryan, A. H. Comparison of polygraph data obtained from individuals involved in mock crimes and actual crime investigations. J. Appl. Psychol. 89, 1099–1105 (2004).

    Article  PubMed  Google Scholar 

  99. Lykken, D. T. Tremor in the Blood: Uses and Abuses of the Lie Detector (McGraw-Hill, New York, 1981).

    Google Scholar 

  100. Honts, C. R., Raskin, D. C. & Kircher, J. C. Mental and physical countermeasures reduce the accuracy of polygraph tests. J. App. Psy. 79, 252–259 (1994).

    Article  CAS  Google Scholar 

  101. Farwell, L. A. & Smith, S. S. Using brain MERMER testing to detect knowledge despite efforts to conceal. J. Forensic Sci. 46, 135–143 (2001).

    Article  CAS  PubMed  Google Scholar 

  102. Spence, S. A. et al. Behavioral and functional anatomical correlates of deception in humans. Neuroreport 12, 2849–2853 (2001).

    Article  CAS  PubMed  Google Scholar 

  103. Phan, K. L. et al. Neural correlates of telling lies: a functional magnetic resonance imaging study at 4 Tesla. Acad. Radiol. 12, 164–172 (2005).

    Article  PubMed  Google Scholar 

  104. Ganis, G., Kosslyn, S. M., Stose, S., Thompson, W. L. & Yurgelun, T. Neural correlates of different types of deception: an fMRI investigation. Cereb. Cortex 13, 830–836 (2003).

    Article  CAS  PubMed  Google Scholar 

  105. Lee, T. M. et al. Lie detection by functional magnetic resonance imaging. Hum. Brain Mapp. 15, 157–164 (2002).

    Article  PubMed  PubMed Central  Google Scholar 

  106. Boynton, G. Imaging orientation selectivity: decoding conscious perception in V1. Nature Neurosci. 8, 541–542 (2005).

    Article  CAS  PubMed  Google Scholar 

  107. Nevado, Y., Young, M. P. & Panzeri, S. Functional imaging and neural information coding. Neuroimage 21, 1083–1095 (2004).

    Article  PubMed  Google Scholar 

  108. Turner, R. How much cortex can a vein drain? Downstream dilution of activation-related cerebral blood oxygenation changes. Neuroimage 16, 1062–1067 (2002).

    Article  PubMed  Google Scholar 

  109. Duvernoy, H. The Human Brain (Springer, New York, 1999).

    Book  Google Scholar 

Download references

Acknowledgements

This work was supported by the Wellcome Trust and the Mind–Science Foundation. We thank V. Lamme for bringing the reference to Nikola Tesla to our attention, and thank J. Driver, C. Frith and K.-E. Stephan for helpful comments on the manuscript.

Author information

Authors and Affiliations

  1. Max Planck Institute for Cognitive and Brain Sciences, Stephanstrasse 1a, Leipzig, 04103, Germany

    John-Dylan Haynes

  2. Wellcome Department of Imaging Neuroscience, Institute of Neurology, University College London, 12 Queen Square, London, WC1N 3BG, UK

    John-Dylan Haynes & Geraint Rees

  3. Institute of Cognitive Neuroscience, University College London, Alexandra House, 17 Queen Square, London, WC1N 3AR, UK

    John-Dylan Haynes & Geraint Rees

Authors
  1. John-Dylan Haynes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geraint Rees
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John-Dylan Haynes.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Related links

Related links

FURTHER INFORMATION

Haynes's homepage

Rees's laboratory

Glossary

Multivariate analysis

An analytical technique that considers (or solves) multiple decision variables. In the present context, multivariate analysis takes into account patterns of information that might be present across multiple voxels measured by neuroimaging techniques.

Univariate analysis

Univariate statistical analysis considers only single-decision variables at any one time. Conventional brain imaging data analyses are mass univariate in that they consider how responses vary at very many single voxels, but consider each individual voxel separately.

Blood-oxygen-leveldependent (BOLD) signal

Functional MRI measures local changes in the proportion of oxygenated blood in the brain; the BOLD signal. This proportion changes in response to neural activity. Therefore, the BOLD signal, or haemodynamic response, indicates the location and magnitude of neural activity.

Primary visual cortex

Considered to be the first visual cortical area in primates, and receives the majority of its input from the retina via the lateral geniculate nucleus.

Voxel

A voxel is the three-dimensional (3D) equivalent of a pixel; a finite volume within 3D space. This corresponds to the smallest element measured in a 3D anatomical or functional brain image volume.

Pattern vector

A vector is a set of one or more numerical elements. Here, a pattern vector is the set of values that together represent the value of each individual voxel in a particular spatial pattern.

Orientation tuning

Many neurons in the mammalian early visual cortex evoke spikes at a greater rate when the animal is presented with visual stimuli of a particular orientation. The stimulus orientation that evokes the greatest firing rate for a particular cell is known as its preferred orientation, and the orientation tuning curve of a cell describes how that firing rate changes as the orientation of the stimulus is varied away from the preferred orientation.

Spatial anisotropy

An anisotropic property is one where a measurement made in one direction differs from the measurement made in another direction. For example, the orientation tuning preferences of neurons in V1 change in a systematic but anisotropic way across the surface of the cortex.

Electroencephalogram

(EEG). The continuously changing electrical signal recorded from the scalp in humans that reflects the summated postsynaptic potentials of cortical neurons in response to changing cognitive or perceptual states. The EEG can be measured with extremely high temporal resolution.

Magnetoencephalography

A non-invasive technique that allows the detection of the changing magnetic fields that are associated with brain activity on the timescale of milliseconds.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Haynes, JD., Rees, G. Decoding mental states from brain activity in humans. Nat Rev Neurosci 7, 523–534 (2006). https://doi.org/10.1038/nrn1931

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn1931

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing