Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Mobile cognition: imaging the human brain in the ‘real world’

Nature Reviews Neuroscience volume 24, pages 347–362 (2023)Cite this article

Subjects

Abstract

Cognitive neuroscience studies in humans have enabled decades of impactful discoveries but have primarily been limited to recording the brain activity of immobile participants in a laboratory setting. In recent years, advances in neuroimaging technologies have enabled recordings of human brain activity to be obtained during freely moving behaviours in the real world. Here, we propose that these mobile neuroimaging methods can provide unique insights into the neural mechanisms of human cognition and contribute to the development of novel treatments for neurological and psychiatric disorders. We further discuss the challenges associated with studying naturalistic human behaviours in complex real-world settings as well as strategies for overcoming them. We conclude that mobile neuroimaging methods have the potential to bring about a new era of cognitive neuroscience in which neural mechanisms can be studied with increased ecological validity and with the ability to address questions about natural behaviour and cognitive processes in humans engaged in dynamic real-world experiences.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Experimental environments and factors impacting human cognition and behaviour.
Fig. 2: Recent advances in mobile neuroimaging methods.
Fig. 3: Studying spatial navigation and social interaction with mobile neuroimaging.
Fig. 4: Studying motor-related functions and emotional affect with mobile neuroimaging.

Similar content being viewed by others

References

  1. Poldrack, R. A. & Farah, M. J. Progress and challenges in probing the human brain. Nature 526, 371–379 (2015).

    Article  CAS  PubMed  Google Scholar 

  2. Johnson, K. T. & Picard, R. W. Advancing neuroscience through wearable devices. Neuron 108, 8–12 (2020).

    Article  CAS  PubMed  Google Scholar 

  3. Mobbs, D. et al. Promises and challenges of human computational ethology. Neuron 109, 2224–2238 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Chow, T. E. & Rissman, J. Neurocognitive mechanisms of real-world autobiographical memory retrieval: insights from studies using wearable camera technology. Ann. NY Acad. Sci. 1396, 202–221 (2017).

    Article  PubMed  Google Scholar 

  5. Harari, G. M. et al. Using smartphones to collect behavioral data in psychological science: opportunities, practical considerations, and challenges. Perspect. Psychol. Sci. 11, 838–854 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Miller, G. The smartphone psychology manifesto. Perspect. Psychol. Sci. 7, 221–237 (2012).

    Article  PubMed  Google Scholar 

  7. Nastase, S. A., Goldstein, A. & Hasson, U. Keep it real: rethinking the primacy of experimental control in cognitive neuroscience. NeuroImage 222, 117254 (2020).

    Article  PubMed  Google Scholar 

  8. Musall, S., Kaufman, M. T., Juavinett, A. L., Gluf, S. & Churchland, A. K. Single-trial neural dynamics are dominated by richly varied movements. Nat. Neurosci. 22, 1677–1686 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Stringer, C. et al. Spontaneous behaviors drive multidimensional, brainwide activity. Science 364, eaav7893 (2019).

    Article  CAS  Google Scholar 

  10. David, S. V., Vinje, W. E. & Gallant, J. L. Natural stimulus statistics alter the receptive field structure of V1 neurons. J. Neurosci. 24, 6991–7006 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Angelaki, D. E. & Cullen, K. E. Vestibular system: the many facets of a multimodal sense. Annu. Rev. Neurosci. 31, 125–150 (2008).

    Article  CAS  PubMed  Google Scholar 

  12. Taube, J. S., Valerio, S. & Yoder, R. M. Is navigation in virtual reality with FMRI really navigation? J. Cogn. Neurosci. 25, 1008–1019 (2013).

    Article  PubMed  Google Scholar 

  13. Steel, A., Robertson, C. E. & Taube, J. S. Current promises and limitations of combined virtual reality and functional magnetic resonance imaging research in humans: a commentary on Huffman and Ekstrom (2019). J. Cogn. Neurosci. 33, 159–166 (2021).

    Article  PubMed  Google Scholar 

  14. Mao, D. et al. Spatial modulation of hippocampal activity in freely moving macaques. Neuron 109, 3521–3534 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Chen, G., King, J. A., Burgess, N. & O’Keefe, J. How vision and movement combine in the hippocampal place code. Proc. Natl Acad. Sci. USA 110, 378–383 (2013).

    Article  CAS  PubMed  Google Scholar 

  16. Bohbot, V. D., Copara, M. S., Gotman, J. & Ekstrom, A. D. Low-frequency theta oscillations in the human hippocampus during real-world and virtual navigation. Nat. Commun. 8, 14415 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Aghajan, Z. M. et al. Impaired spatial selectivity and intact phase precession in two-dimensional virtual reality. Nat. Neurosci. 18, 121–128 (2015).

    Article  CAS  PubMed  Google Scholar 

  18. Aronov, D. & Tank, D. W. Engagement of neural circuits underlying 2D spatial navigation in a rodent virtual reality system. Neuron 84, 442–456 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Anderson, W. S. & Lenz, F. A. Review of motor and phantom related imagery. Neuroreport 22, 939–942 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Lyu, Y., Guo, X., Bekrater-Bodmann, R., Flor, H. & Tong, S. Phantom limb perception interferes with motor imagery after unilateral upper-limb amputation. Sci. Rep. 6, 21100 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Thomas, E., Dyson, M. & Clerc, M. An analysis of performance evaluation for motor-imagery based BCI. J. Neural Eng. 10, 031001 (2013).

    Article  PubMed  Google Scholar 

  22. Hochberg, L. R. et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Hochberg, L. R. et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006).

    Article  CAS  PubMed  Google Scholar 

  24. Hardwick, R. M., Caspers, S., Eickhoff, S. B. & Swinnen, S. P. Neural correlates of action: comparing meta-analyses of imagery, observation, and execution. Neurosci. Biobehav. Rev. 94, 31–44 (2018).

    Article  PubMed  Google Scholar 

  25. Crammond, D. J. Motor imagery: never in your wildest dream. Trends Neurosci. 20, 54–57 (1997).

    Article  CAS  PubMed  Google Scholar 

  26. Grèzes, J. & Decety, J. Functional anatomy of execution, mental simulation, observation, and verb generation of actions: a meta-analysis. Hum. Brain Mapp. 12, 1–19 (2001).

    Article  PubMed  Google Scholar 

  27. Jeannerod, M. Neural simulation of action: a unifying mechanism for motor cognition. Neuroimage 14, S103–109 (2001).

    Article  CAS  PubMed  Google Scholar 

  28. Bohil, C. J., Alicea, B. & Biocca, F. A. Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12, 752–762 (2011).

    Article  CAS  PubMed  Google Scholar 

  29. Parsons, T. D. Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci. 9, 660 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  30. Huffman, D. J. & Ekstrom, A. D. A modality-independent network underlies the retrieval of large-scale spatial environments in the human brain. Neuron 104, 611–622 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Niso, G., Romero, E., Moreau, J. T., Araujo, A. & Krol, L. R. Wireless EEG: a survey of systems and studies. NeuroImage 269, 119774 (2023).

    Article  PubMed  Google Scholar 

  32. Debener, S., Minow, F., Emkes, R., Gandras, K. & de Vos, M. How about taking a low-cost, small, and wireless EEG for a walk? Psychophysiology 49, 1617–1621 (2012).

    Article  PubMed  Google Scholar 

  33. Krugliak, A. & Clarke, A. Towards real-world neuroscience using mobile EEG and augmented reality. Sci. Rep. 12, 2291 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. King, J. L. & Parada, F. J. Using mobile brain/body imaging to advance research in arts, health, and related therapeutics. Eur. J. Neurosci. 54, 8364–8380 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Gwin, J. T., Gramann, K., Makeig, S. & Ferris, D. P. Removal of movement artifact from high-density EEG recorded during walking and running. J. Neurophysiol. 103, 3526–3534 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Delorme, A., Sejnowski, T. & Makeig, S. Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis. NeuroImage 34, 1443–1449 (2007).

    Article  PubMed  Google Scholar 

  37. Gramann, K. et al. Human brain dynamics accompanying use of egocentric and allocentric reference frames during navigation. J. Cogn. Neurosci. 22, 2836–2849 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  38. Seeber, M. et al. Subcortical electrophysiological activity is detectable with high-density EEG source imaging. Nat. Commun. 10, 753 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Delaux, A. et al. Mobile brain/body imaging of landmark-based navigation with high-density EEG. Eur. J. Neurosci. 54, 8256–8282 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Gehrke, L. & Gramann, K. Single-trial regression of spatial exploration behavior indicates posterior EEG alpha modulation to reflect egocentric coding. Eur. J. Neurosci. 54, 8318–8335 (2021).

    Article  PubMed  Google Scholar 

  41. McKendrick, R. et al. Into the wild: neuroergonomic differentiation of hand-held and augmented reality wearable displays during outdoor navigation with functional near infrared spectroscopy. Front. Hum. Neurosci. 10, 216 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  42. Piper, S. K. et al. A wearable multi-channel fNIRS system for brain imaging in freely moving subjects. NeuroImage 85, 64–71 (2014).

    Article  PubMed  Google Scholar 

  43. Brigadoi, S. et al. Motion artifacts in functional near-infrared spectroscopy: a comparison of motion correction techniques applied to real cognitive data. NeuroImage 85, 181–191 (2014).

    Article  PubMed  Google Scholar 

  44. Koenraadt, K. L. M., Roelofsen, E. G. J., Duysens, J. & Keijsers, N. L. W. Cortical control of normal gait and precision stepping: an fNIRS study. NeuroImage 85, 415–422 (2014).

    Article  PubMed  Google Scholar 

  45. Takizawa, R. et al. Neuroimaging-aided differential diagnosis of the depressive state. NeuroImage 85, 498–507 (2014).

    Article  CAS  PubMed  Google Scholar 

  46. Suda, M. et al. Frontopolar activation during face-to-face conversation: an in situ study using near-infrared spectroscopy. Neuropsychologia 48, 441–447 (2010).

    Article  PubMed  Google Scholar 

  47. Suda, M. et al. Autistic traits and brain activation during face-to-face conversations in typically developed adults. PLoS ONE 6, e20021 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Boto, E. et al. Moving magnetoencephalography towards real-world applications with a wearable system. Nature 555, 657–661 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Brookes, M. J. et al. Magnetoencephalography with optically pumped magnetometers (OPM-MEG): the next generation of functional neuroimaging. Trends Neurosci. 45, 621–634 (2022).

    Article  CAS  PubMed  Google Scholar 

  50. Krauss, J. K. et al. Technology of deep brain stimulation: current status and future directions. Nat. Rev. Neurol. 17, 75–87 (2021).

    Article  PubMed  Google Scholar 

  51. Morrell, M. J., RNS System in Epilepsy Study Group. Responsive cortical stimulation for the treatment of medically intractable partial epilepsy. Neurology 77, 1295–1304 (2011).

    Article  PubMed  Google Scholar 

  52. Lozano, A. M. et al. Deep brain stimulation: current challenges and future directions. Nat. Rev. Neurol. 15, 148–160 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  53. Scangos, K. W., Makhoul, G. S., Sugrue, L. P., Chang, E. F. & Krystal, A. D. State-dependent responses to intracranial brain stimulation in a patient with depression. Nat. Med. 27, 229–231 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Scangos, K. W. et al. Closed-loop neuromodulation in an individual with treatment-resistant depression. Nat. Med. 27, 1696–1700 (2021).

    Article  CAS  PubMed  Google Scholar 

  55. US National Library of Medicine. ClinicalTrials.gov https://clinicaltrials.gov/ct2/show/NCT04152993 (2021).

  56. Provenza, N. R. et al. Long-term ecological assessment of intracranial electrophysiology synchronized to behavioral markers in obsessive-compulsive disorder. Nat. Med. 27, 2154–2164 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Shivacharan, R. S. et al. Pilot study of responsive nucleus accumbens deep brain stimulation for loss-of-control eating. Nat. Med. 28, 1791–1796 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Topalovic, U. et al. Wireless programmable recording and stimulation of deep brain activity in freely moving humans. Neuron 108, 322–334 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Meisenhelter, S. et al. Cognitive tasks and human ambulatory electrocorticography using the RNS System. J. Neurosci. Methods 311, 408–417 (2019).

    Article  PubMed  Google Scholar 

  60. Buzsáki, G. & Moser, E. I. Memory, navigation and theta rhythm in the hippocampal-entorhinal system. Nat. Neurosci. 16, 130–138 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  61. Behrens, T. E. J. et al. What is a cognitive map? Organizing knowledge for flexible behavior. Neuron 100, 490–509 (2018).

    Article  CAS  PubMed  Google Scholar 

  62. Buzsáki, G., McKenzie, S. & Davachi, L. Neurophysiology of remembering. Annu. Rev. Psychol. 73, 187–215 (2022).

    Article  PubMed  Google Scholar 

  63. Coughlan, G., Laczó, J., Hort, J., Minihane, A.-M. & Hornberger, M. Spatial navigation deficits — overlooked cognitive marker for preclinical Alzheimer disease? Nat. Rev. Neurol. 14, 496–506 (2018).

    Article  PubMed  Google Scholar 

  64. Lester, A. W., Moffat, S. D., Wiener, J. M., Barnes, C. A. & Wolbers, T. The aging navigational system. Neuron 95, 1019–1035 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  65. Epstein, R. A., Patai, E. Z., Julian, J. B. & Spiers, H. J. The cognitive map in humans: spatial navigation and beyond. Nat. Neurosci. 20, 1504–1513 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  66. Ekstrom, A. D. & Ranganath, C. Space, time, and episodic memory: the hippocampus is all over the cognitive map. Hippocampus 28, 680–687 (2018).

    Article  PubMed  Google Scholar 

  67. Jacobs, J. Hippocampal theta oscillations are slower in humans than in rodents: implications for models of spatial navigation and memory. Philos. Trans. R. Soc. B Biol. Sci. 369, 20130304 (2014).

    Article  Google Scholar 

  68. Aghajan, Z. M. et al. Theta oscillations in the human medial temporal lobe during real-world ambulatory movement. Curr. Biol. 27, 3743–3751 (2017).

    Article  CAS  PubMed Central  Google Scholar 

  69. Courellis, H. S. et al. Spatial encoding in primate hippocampus during free navigation. PLoS Biol. 17, e3000546 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Yartsev, M. M., Witter, M. P. & Ulanovsky, N. Grid cells without theta oscillations in the entorhinal cortex of bats. Nature 479, 103–107 (2011).

    Article  CAS  PubMed  Google Scholar 

  71. Stangl, M. et al. Boundary-anchored neural mechanisms of location-encoding for self and others. Nature 589, 420–425 (2021).

    Article  CAS  PubMed  Google Scholar 

  72. Lin, M.-H., Liran, O., Bauer, N. & Baker, T. E. Scalp recorded theta activity is modulated by reward, direction, and speed during virtual navigation in freely moving humans. Sci. Rep. 12, 2041 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  73. Ehinger, B. et al. Kinesthetic and vestibular information modulate alpha activity during spatial navigation: a mobile EEG study. Front. Hum. Neurosci. 8, 71 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Do, T.-T. N., Lin, C.-T. & Gramann, K. Human brain dynamics in active spatial navigation. Sci. Rep. 11, 13036 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Liang, M., Starrett, M. J. & Ekstrom, A. D. Dissociation of frontal-midline delta-theta and posterior alpha oscillations: a mobile EEG study. Psychophysiology 55, e13090 (2018).

    Article  PubMed  Google Scholar 

  76. Piñeyro Salvidegoitia, M. et al. Out and about: subsequent memory effect captured in a natural outdoor environment with smartphone EEG. Psychophysiology 56, e13331 (2019).

    Article  PubMed  Google Scholar 

  77. Park, J. L. & Donaldson, D. I. Detecting the neural correlates of episodic memory with mobile EEG: recollecting objects in the real world. NeuroImage 193, 1–9 (2019).

    Article  PubMed  Google Scholar 

  78. Griffiths, B., Mazaheri, A., Debener, S. & Hanslmayr, S. Brain oscillations track the formation of episodic memories in the real world. NeuroImage 143, 256–266 (2016).

    Article  PubMed  Google Scholar 

  79. Parvizi, J. & Kastner, S. Promises and limitations of human intracranial electroencephalography. Nat. Neurosci. 21, 474–483 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  80. Mankin, E. A. & Fried, I. Modulation of human memory by deep brain stimulation of the entorhinal-hippocampal circuitry. Neuron 106, 218–235 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Suthana, N. & Fried, I. Deep brain stimulation for enhancement of learning and memory. NeuroImage 85, 996–1002 (2014).

    Article  PubMed  Google Scholar 

  82. Polanía, R., Nitsche, M. A. & Ruff, C. C. Studying and modifying brain function with non-invasive brain stimulation. Nat. Neurosci. 21, 174–187 (2018).

    Article  PubMed  Google Scholar 

  83. Kunz, L. et al. Reduced grid-cell-like representations in adults at genetic risk for Alzheimer’s disease. Science 350, 430–433 (2015).

    Article  CAS  PubMed  Google Scholar 

  84. Kasai, K., Fukuda, M., Yahata, N., Morita, K. & Fujii, N. The future of real-world neuroscience: imaging techniques to assess active brains in social environments. Neurosci. Res. 90, 65–71 (2015).

    Article  PubMed  Google Scholar 

  85. Adolphs, R. Conceptual challenges and directions for social neuroscience. Neuron 65, 752–767 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  86. van Dijk, E. & De Dreu, C. K. W. Experimental games and social decision making. Annu. Rev. Psychol. 72, 415–438 (2021).

    Article  PubMed  Google Scholar 

  87. Spears, R. Social influence and group identity. Annu. Rev. Psychol. 72, 367–390 (2021).

    Article  PubMed  Google Scholar 

  88. Amodio, D. M. Social cognition 2.0: an interactive memory systems account. Trends Cogn. Sci. 23, 21–33 (2019).

    Article  PubMed  Google Scholar 

  89. Alcalá-López, D., Vogeley, K., Binkofski, F. & Bzdok, D. Building blocks of social cognition: mirror, mentalize, share? Cortex 118, 4–18 (2019).

    Article  PubMed  Google Scholar 

  90. Sherman, J. W., Gawronski, B. & Trope, Y. Dual-Process Theories of the Social Mind (Guilford Publications, 2014).

  91. Greenwald, A. G. & Lai, C. K. Implicit social cognition. Annu. Rev. Psychol. 71, 419–445 (2020).

    Article  PubMed  Google Scholar 

  92. Frith, C. & Frith, U. Theory of mind. Curr. Biol. 15, R644–646 (2005).

    Article  CAS  PubMed  Google Scholar 

  93. Aureli, F. & Schino, G. Social complexity from within: how individuals experience the structure and organization of their groups. Behav. Ecol. Sociobiol. 73, 6 (2019).

    Article  Google Scholar 

  94. Lombardi, F., Wang, J. W. J. L., Zhang, X. & Ivanov, P. C. Power-law correlations and coupling of active and quiet states underlie a class of complex systems with self-organization at criticality. EPJ Web Conf. 230, 00005 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  95. Tamariz, M. Replication and emergence in cultural transmission. Phys. Life Rev. 30, 47–71 (2019).

    Article  PubMed  Google Scholar 

  96. De Vincenzo, I., Giannoccaro, I., Carbone, G. & Grigolini, P. Criticality triggers the emergence of collective intelligence in groups. Phys. Rev. E 96, 022309 (2017).

    Article  PubMed  Google Scholar 

  97. Corrêa, U. C., Alegre, F. A. M., Freudenheim, A. M., Dos Santos, S. & Tani, G. The game of futsal as an adaptive process. Nonlinear Dyn. Psychol. Life Sci. 16, 185–203 (2012).

    Google Scholar 

  98. Horsevad, N., Mateo, D., Kooij, R. E., Barrat, A. & Bouffanais, R. Transition from simple to complex contagion in collective decision-making. Nat. Commun. 13, 1442 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  99. Vilone, D., Realpe-Gómez, J. & Andrighetto, G. Evolutionary advantages of turning points in human cooperative behaviour. PLoS ONE 16, e0246278 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  100. Ribeiro, T. L., Chialvo, D. R. & Plenz, D. Scale-free dynamics in animal groups and brain networks. Front. Syst. Neurosci. 14, 591210 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  101. Marshall, J. A. R., Reina, A. & Bose, T. Multiscale modelling tool: mathematical modelling of collective behaviour without the maths. PLoS ONE 14, e0222906 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  102. Fan, S., Dal Monte, O. & Chang, S. W. C. Levels of naturalism in social neuroscience research. iScience 24, 102702 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  103. Kingsbury, L. & Hong, W. A multi-brain framework for social interaction. Trends Neurosci. 43, 651–666 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  104. Schippers, M. B., Roebroeck, A., Renken, R., Nanetti, L. & Keysers, C. Mapping the information flow from one brain to another during gestural communication. Proc. Natl Acad. Sci. USA 107, 9388–9393 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  105. Anders, S., Heinzle, J., Weiskopf, N., Ethofer, T. & Haynes, J.-D. Flow of affective information between communicating brains. NeuroImage 54, 439–446 (2011).

    Article  PubMed  Google Scholar 

  106. Stephens, G. J., Silbert, L. J. & Hasson, U. Speaker-listener neural coupling underlies successful communication. Proc. Natl Acad. Sci. USA 107, 14425–14430 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  107. Watanabe, H. et al. Construction of a fiber-optically connected MEG hyperscanning system for recording brain activity during real-time communication. PLoS ONE 17, e0270090 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  108. Dumas, G., Nadel, J., Soussignan, R., Martinerie, J. & Garnero, L. Inter-brain synchronization during social interaction. PLoS ONE 5, e12166 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  109. Leong, V. et al. Speaker gaze increases information coupling between infant and adult brains. Proc. Natl Acad. Sci. USA 114, 13290–13295 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  110. Hirsch, J., Zhang, X., Noah, J. A. & Ono, Y. Frontal temporal and parietal systems synchronize within and across brains during live eye-to-eye contact. NeuroImage 157, 314–330 (2017).

    Article  PubMed  Google Scholar 

  111. Babiloni, F. & Astolfi, L. Social neuroscience and hyperscanning techniques: past, present and future. Neurosci. Biobehav. Rev. 44, 76–93 (2014).

    Article  PubMed  Google Scholar 

  112. Dikker, S. et al. Brain-to-brain synchrony tracks real-world dynamic group interactions in the classroom. Curr. Biol. 27, 1375–1380 (2017).

    Article  CAS  PubMed  Google Scholar 

  113. Dikker, S. et al. Crowdsourcing neuroscience: inter-brain coupling during face-to-face interactions outside the laboratory. NeuroImage 227, 117436 (2021).

    Article  PubMed  Google Scholar 

  114. Braha, D. Global civil unrest: contagion, self-organization, and prediction. PLoS ONE 7, e48596 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  115. Cotter, J. et al. Social cognitive dysfunction as a clinical marker: a systematic review of meta-analyses across 30 clinical conditions. Neurosci. Biobehav. Rev. 84, 92–99 (2018).

    Article  PubMed  Google Scholar 

  116. Santamaría-García, H. et al. The role of social cognition skills and social determinants of health in predicting symptoms of mental illness. Transl Psychiatry 10, 165 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  117. Rokita, K. I., Dauvermann, M. R. & Donohoe, G. Early life experiences and social cognition in major psychiatric disorders: a systematic review. Eur. Psychiatry 53, 123–133 (2018).

    Article  PubMed  Google Scholar 

  118. Tanabe, H. C. et al. Hard to ‘tune in’: neural mechanisms of live face-to-face interaction with high-functioning autistic spectrum disorder. Front. Hum. Neurosci. 6, 268 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  119. Kruppa, J. A. et al. Brain and motor synchrony in children and adolescents with ASD-a fNIRS hyperscanning study. Soc. Cogn. Affect. Neurosci. 16, 103–116 (2021).

    Article  PubMed  Google Scholar 

  120. Gvirts Problovski, H. Z. et al. Impairments of interpersonal synchrony evident in attention deficit hyperactivity disorder (ADHD). Acta Psychol. 212, 103210 (2021).

    Article  Google Scholar 

  121. Saul, M. A., He, X., Black, S. & Charles, F. A two-person neuroscience approach for social anxiety: a paradigm with interbrain synchrony and neurofeedback. Front. Psychol. 12, 568921 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  122. Ospina, L. H. et al. Social cognition moderates the relationship between neurocognition and community functioning in bipolar disorder. J. Affect. Disord. 235, 7–14 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  123. Pijnenborg, G. H. M. et al. The predictive value of measures of social cognition for community functioning in schizophrenia: implications for neuropsychological assessment. J. Int. Neuropsychol. Soc. 15, 239–247 (2009).

    Article  CAS  PubMed  Google Scholar 

  124. Bilek, E. et al. State-dependent cross-brain information flow in borderline personality disorder. JAMA Psychiatry 74, 949–957 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  125. Lau-Zhu, A., Lau, M. P. H. & McLoughlin, G. Mobile EEG in research on neurodevelopmental disorders: opportunities and challenges. Dev. Cogn. Neurosci. 36, 100635 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  126. Nguyen, J. P. et al. Whole-brain calcium imaging with cellular resolution in freely behaving Caenorhabditis elegans. Proc. Natl Acad. Sci. USA 113, E1074–E1081 (2016).

    Article  CAS  PubMed  Google Scholar 

  127. Svoboda, K. & Li, N. Neural mechanisms of movement planning: motor cortex and beyond. Curr. Opin. Neurobiol. 49, 33–41 (2018).

    Article  CAS  PubMed  Google Scholar 

  128. Seeber, M., Scherer, R., Wagner, J., Solis-Escalante, T. & Müller-Putz, G. R. High and low gamma EEG oscillations in central sensorimotor areas are conversely modulated during the human gait cycle. NeuroImage 112, 318–326 (2015).

    Article  PubMed  Google Scholar 

  129. Wagner, J. et al. Level of participation in robotic-assisted treadmill walking modulates midline sensorimotor EEG rhythms in able-bodied subjects. NeuroImage 63, 1203–1211 (2012).

    Article  PubMed  Google Scholar 

  130. McCrimmon, C. M. et al. Electrocorticographic encoding of human gait in the leg primary motor cortex. Cereb. Cortex 28, 2752–2762 (2018).

    Article  PubMed  Google Scholar 

  131. Fischer, P. et al. Alternating modulation of subthalamic nucleus beta oscillations during stepping. J. Neurosci. 38, 5111–5121 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  132. Wagner, F. B. et al. Targeted neurotechnology restores walking in humans with spinal cord injury. Nature 563, 65–71 (2018).

    Article  CAS  PubMed  Google Scholar 

  133. Yokoyama, H. et al. Gait-phase-dependent and gait-phase-independent cortical activity across multiple regions involved in voluntary gait modifications in humans. Eur. J. Neurosci. 54, 8092–8105 (2021).

    Article  PubMed  Google Scholar 

  134. Thenaisie, Y. et al. Principles of gait encoding in the subthalamic nucleus of people with Parkinson’s disease. Sci. Transl Med. 14, eabo1800 (2022).

    Article  PubMed  Google Scholar 

  135. Holtzer, R., Epstein, N., Mahoney, J. R., Izzetoglu, M. & Blumen, H. M. Neuroimaging of mobility in aging: a targeted review. J. Gerontol. Ser. A 69, 1375–1388 (2014).

    Article  Google Scholar 

  136. Peterson, D. S. & Horak, F. B. Neural control of walking in people with Parkinsonism. Physiology 31, 95–107 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  137. Mustile, M. et al. Mobile EEG reveals functionally dissociable dynamic processes supporting real-world ambulatory obstacle avoidance: evidence for early proactive control. Eur. J. Neurosci. 54, 8106–8119 (2021).

    Article  PubMed  Google Scholar 

  138. Meng, J. et al. Noninvasive electroencephalogram based control of a robotic arm for reach and grasp tasks. Sci. Rep. 6, 38565 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  139. Barnstaple, R., Protzak, J., DeSouza, J. F. X. & Gramann, K. Mobile brain/body Imaging in dance: a dynamic transdisciplinary field for applied research. Eur. J. Neurosci. 54, 8355–8363 (2021).

    Article  PubMed  Google Scholar 

  140. Reiser, J. E., Wascher, E. & Arnau, S. Recording mobile EEG in an outdoor environment reveals cognitive-motor interference dependent on movement complexity. Sci. Rep. 9, 13086 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  141. Protzak, J. & Gramann, K. EEG beta-modulations reflect age-specific motor resource allocation during dual-task walking. Sci. Rep. 11, 16110 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  142. De Sanctis, P., Butler, J. S., Malcolm, B. R. & Foxe, J. J. Recalibration of inhibitory control systems during walking-related dual-task interference: a mobile brain-body imaging (MOBI) study. NeuroImage 94, 55–64 (2014).

    Article  PubMed  Google Scholar 

  143. Peterson, S. M. & Ferris, D. P. Differentiation in theta and beta electrocortical activity between visual and physical perturbations to walking and standing balance. eNeuro 5, ENEURO.0207-18.2018 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  144. Wagner, J., Makeig, S., Gola, M., Neuper, C. & Müller-Putz, G. Distinct β band oscillatory networks subserving motor and cognitive control during gait adaptation. J. Neurosci. 36, 2212–2226 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  145. Gilron, R. et al. Long-term wireless streaming of neural recordings for circuit discovery and adaptive stimulation in individuals with Parkinson’s disease. Nat. Biotechnol. 39, 1078–1085 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  146. Ansó, J. et al. Concurrent stimulation and sensing in bi-directional brain interfaces: a multi-site translational experience. J. Neural Eng. 19, 026025 (2022).

    Article  Google Scholar 

  147. Johnson, V. et al. Embedded adaptive deep brain stimulation for cervical dystonia controlled by motor cortex theta oscillations. Exp. Neurol. 345, 113825 (2021).

    Article  PubMed  Google Scholar 

  148. Louie, K. H. et al. Cortico-subthalamic field potentials support classification of the natural gait cycle in Parkinson’s disease and reveal individualized spectral signatures. eNeuro 9, ENEURO.0325-22.2022 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  149. Cagnan, H. et al. Stimulating at the right time: phase-specific deep brain stimulation. Brain 140, 132–145 (2017).

    Article  PubMed  Google Scholar 

  150. Morinan, G. et al. Computer-vision based method for quantifying rising from chair in Parkinson’s disease patients. Intell. Based Med. 6, 100046 (2022).

    Article  Google Scholar 

  151. Fox, E. Perspectives from affective science on understanding the nature of emotion. Brain Neurosci. Adv. 2, 2398212818812628 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  152. Pessoa, L. On the relationship between emotion and cognition. Nat. Rev. Neurosci. 9, 148–158 (2008).

    Article  CAS  PubMed  Google Scholar 

  153. Shamay-Tsoory, S. G. & Mendelsohn, A. Real-Life neuroscience: an ecological approach to brain and behavior research. Perspect. Psychol. Sci. 14, 841–859 (2019).

    Article  PubMed  Google Scholar 

  154. Barrett, L. F. The theory of constructed emotion: an active inference account of interoception and categorization. Soc. Cogn. Affect. Neurosci. 12, 1–23 (2017).

    Article  PubMed  Google Scholar 

  155. Bhanot, S. P., Chang, D., Lee Cunningham, J. & Ranson, M. Emotions and decisions in the real world: what can we learn from quasi-field experiments? PLoS ONE 15, e0243044 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  156. Mavros, P., Austwick, M. Z. & Smith, A. H. Geo-EEG: towards the use of EEG in the study of urban behaviour. Appl. Spat. Anal. 9, 191–212 (2016).

    Google Scholar 

  157. Aspinall, P., Mavros, P., Coyne, R. & Roe, J. The urban brain: analysing outdoor physical activity with mobile EEG. Br. J. Sports Med. 49, 272–276 (2015).

    Article  PubMed  Google Scholar 

  158. Mavros, P., Wälti, M. J., Nazemi, M., Ong, C. H. & Hölscher, C. A mobile EEG study on the psychophysiological effects of walking and crowding in indoor and outdoor urban environments. Sci. Rep. 12, 18476 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  159. Neale, C. et al. The impact of walking in different urban environments on brain activity in older people. Cities Health 4, 94–106 (2020).

    Article  Google Scholar 

  160. Lin, W. et al. Sitting or walking? Analyzing the neural emotional indicators of urban green space behavior with mobile EEG. J. Urban. Health 97, 191–203 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  161. Suhaimi, N. S., Mountstephens, J. & Teo, J. EEG-Based emotion recognition: a state-of-the-art review of current trends and opportunities. Comput. Intell. Neurosci. 2020, e8875426 (2020).

    Article  Google Scholar 

  162. Voineskos, D., Daskalakis, Z. J. & Blumberger, D. M. Management of treatment-resistant depression: challenges and strategies. Neuropsychiatr. Dis. Treat. 16, 221–234 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  163. Figee, M. et al. Deep brain stimulation for depression. Neurotherapeutics 19, 1229–1245 (2022).

    Article  PubMed  Google Scholar 

  164. Miller, C. T. et al. Natural behavior is the language of the brain. Curr. Biol. 32, R482–R493 (2022).

    Article  CAS  PubMed  Google Scholar 

  165. Hardcastle, K., Ganguli, S. & Giocomo, L. M. Environmental boundaries as an error correction mechanism for grid cells. Neuron 86, 827–839 (2015).

    Article  CAS  PubMed  Google Scholar 

  166. Ledergerber, D. et al. Task-dependent mixed selectivity in the subiculum. Cell Rep. 35, 109175 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  167. Yu, Z. et al. Beyond t test and ANOVA: applications of mixed-effects models for more rigorous statistical analysis in neuroscience research. Neuron 110, 21–35 (2022).

    Article  CAS  PubMed  Google Scholar 

  168. Bonnefon, J.-F., Hopfensitz, A. & De Neys, W. Can we detect cooperators by looking at their face? Curr. Dir. Psychol. Sci. 26, 276–281 (2017).

    Article  Google Scholar 

  169. Javadi, A.-H. et al. Hippocampal and prefrontal processing of network topology to simulate the future. Nat. Commun. 8, 14652 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  170. Antony, J. W. et al. Behavioral, physiological, and neural signatures of surprise during naturalistic sports viewing. Neuron 109, 377–390 (2021).

    Article  CAS  PubMed  Google Scholar 

  171. Topalovic, U. et al. A wearable platform for closed-loop stimulation and recording of single-neuron and local field potential activity in freely moving humans. Nat. Neurosci. 26, 517–527 (2023).

    CAS  PubMed  PubMed Central  Google Scholar 

  172. Geller, E. B. et al. Brain-responsive neurostimulation in patients with medically intractable mesial temporal lobe epilepsy. Epilepsia 58, 994–1004 (2017).

    Article  PubMed  Google Scholar 

  173. Ma, B. B. & Rao, V. R. Responsive neurostimulation: candidates and considerations. Epilepsy Behav. 88, 388–395 (2018).

    Article  PubMed  Google Scholar 

  174. Gelinas, J. N., Khodagholy, D., Thesen, T., Devinsky, O. & Buzsáki, G. Interictal epileptiform discharges induce hippocampal-cortical coupling in temporal lobe epilepsy. Nat. Med. 22, 641–648 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  175. Spiers, H. J. Brain rhythms that help us to detect borders. Nature 589, 353–354 (2021).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Institute of Neurological Disorders and Stroke (NINDS) and the National Institute of Mental Health (NIMH) of the National Institutes of Health (NIH) under award numbers R01MH124761 and UO1NS103780 (to N.S.), K99NS126715 (to M.S.) and F30MH125534 (to S.L.M.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. N.S. was also supported by the McKnight Foundation (Technological Innovations in Neuroscience Award) and the Keck Foundation (UCLA DGSOM Junior Faculty Award). The authors thank Leonardo Christov-Moore and Martin Seeber for helpful feedback on the manuscript, and Tyler Wishard, Leonardo Christov-Moore, Jason Whisman and Emi Jenkens-Drake for contributing to the design and creation of figure illustrations. Lastly, the authors also thank all members of the Suthana Lab for helpful discussions and particularly Sonja Hiller for additional general assistance.

Author information

Authors and Affiliations

  1. Department of Psychiatry and Biobehavioral Sciences, Jane and Terry Semel Institute for Neuroscience and Human Behaviour, University of California, Los Angeles, Los Angeles, CA, USA

    Matthias Stangl & Nanthia Suthana

  2. Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA

    Sabrina L. Maoz & Nanthia Suthana

  3. Department of Neurosurgery, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA

    Nanthia Suthana

  4. Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA

    Nanthia Suthana

Authors
  1. Matthias Stangl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sabrina L. Maoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nanthia Suthana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors contributed equally to all aspects of the article.

Corresponding authors

Correspondence to Matthias Stangl or Nanthia Suthana.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Neuroscience thanks A. Clarke, H. Spiers and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Related links

Mo-DBRS platform: https://github.com/suthanalab/Mo-DBRS

Open Mind Consortium: https://openmind-consortium.github.io/

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stangl, M., Maoz, S.L. & Suthana, N. Mobile cognition: imaging the human brain in the ‘real world’. Nat Rev Neurosci 24, 347–362 (2023). https://doi.org/10.1038/s41583-023-00692-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41583-023-00692-y

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing