Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Multi-timescale neural dynamics for multisensory integration

Nature Reviews Neuroscience volume 25, pages 625–642 (2024)Cite this article

Subjects

Abstract

Carrying out any everyday task, be it driving in traffic, conversing with friends or playing basketball, requires rapid selection, integration and segregation of stimuli from different sensory modalities. At present, even the most advanced artificial intelligence-based systems are unable to replicate the multisensory processes that the human brain routinely performs, but how neural circuits in the brain carry out these processes is still not well understood. In this Perspective, we discuss recent findings that shed fresh light on the oscillatory neural mechanisms that mediate multisensory integration (MI), including power modulations, phase resetting, phase–amplitude coupling and dynamic functional connectivity. We then consider studies that also suggest multi-timescale dynamics in intrinsic ongoing neural activity and during stimulus-driven bottom–up and cognitive top–down neural network processing in the context of MI. We propose a new concept of MI that emphasizes the critical role of neural dynamics at multiple timescales within and across brain networks, enabling the simultaneous integration, segregation, hierarchical structuring and selection of information in different time windows. To highlight predictions from our multi-timescale concept of MI, real-world scenarios in which multi-timescale processes may coordinate MI in a flexible and adaptive manner are considered.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Multiple timescales and neural mechanisms relevant for multisensory processing.
Fig. 2: Multisensory processing and multi-timescale dynamics in ongoing neural activity.
Fig. 3: Multi-timescale dynamics during multisensory cognitive processing.
Fig. 4: Scenarios for multi-timescale multisensory integration.

Similar content being viewed by others

References

  1. Auvray, M. & Spence, C. The multisensory perception of flavor. Conscious. Cogn. 17, 1016–1031 (2008).

    Article  PubMed  Google Scholar 

  2. Stein, B. E. & Meredith, M. A. The Merging of the Senses (MIT Press, 1993).

  3. Stein, B. E. & Stanford, T. R. Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, 255–266 (2008).

    Article  CAS  PubMed  Google Scholar 

  4. Welch, R. B. & Warren, D. H. Immediate perceptual response to intersensory discrepancy. Psychol. Bull. 88, 638–667 (1980).

    Article  CAS  PubMed  Google Scholar 

  5. Talsma, D., Senkowski, D., Soto-Faraco, S. & Woldorff, M. G. The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 14, 400–410 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Spence, C. & Squire, S. Multisensory integration: maintaining the perception of synchrony. Curr. Biol. 13, R519–R521 (2003).

    Article  CAS  PubMed  Google Scholar 

  7. Panzeri, S., Brunel, N., Logothetis, N. K. & Kayser, C. Sensory neural codes using multiplexed temporal scales. Trends Neurosci. 33, 111–120 (2010).

    Article  CAS  PubMed  Google Scholar 

  8. Buzsáki, G. & Draguhn, A. Neuronal oscillations in cortical networks. Science 304, 1926–1929 (2004).

    Article  PubMed  Google Scholar 

  9. Singer, W. & Gray, C. M. Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci. 18, 555–586 (1995).

    Article  CAS  PubMed  Google Scholar 

  10. Singer, W. Neuronal synchrony: a versatile code for the definition of relations? Neuron 24, 49–65 (1999).

    Article  CAS  PubMed  Google Scholar 

  11. Engel, A. K., Fries, P. & Singer, W. Dynamic predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci. 2, 704–716 (2001).

    Article  CAS  PubMed  Google Scholar 

  12. Fries, P., Nikolić, D. & Singer, W. The gamma cycle. Trends Neurosci. 30, 309–316 (2007).

    Article  CAS  PubMed  Google Scholar 

  13. Senkowski, D., Schneider, T. R., Foxe, J. J. & Engel, A. K. Crossmodal binding through neural coherence: implications for multisensory processing. Trends Neurosci. 31, 401–409 (2008).

    Article  CAS  PubMed  Google Scholar 

  14. Engel, A. K. & Fries, P. Beta-band oscillations — signalling the status quo? Curr. Opin. Neurobiol. 20, 156–165 (2010).

    Article  CAS  PubMed  Google Scholar 

  15. Jensen, O., Bonnefond, M. & VanRullen, R. An oscillatory mechanism for prioritizing salient unattended stimuli. Trends Cogn. Sci. 16, 200–206 (2012).

    Article  PubMed  Google Scholar 

  16. Lisman, J. The theta/gamma discrete phase code occuring during the hippocampal phase precession may be a more general brain coding scheme. Hippocampus 15, 913–922 (2005).

    Article  PubMed  Google Scholar 

  17. Fries, P. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn. Sci. 9, 474–480 (2005).

    Article  PubMed  Google Scholar 

  18. Jensen, O., Gips, B., Bergmann, T. O. & Bonnefond, M. Temporal coding organized by coupled alpha and gamma oscillations prioritize visual processing. Trends Neurosci. 37, 357–369 (2014).

    Article  CAS  PubMed  Google Scholar 

  19. Hasson, U., Chen, J. & Honey, C. J. Hierarchical process memory: memory as an integral component of information processing. Trends Cogn. Sci. 19, 304–313 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Wang, X.-J. & Kennedy, H. Brain structure and dynamics across scales: in search of rules. Curr. Opin. Neurobiol. 37, 92–98 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  21. Cavanagh, S. E., Hunt, L. T. & Kennerley, S. W. A diversity of intrinsic timescales underlie neural computations. Front. Neural Circuits 14, 615626 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Soltani, A., Murray, J. D., Seo, H. & Lee, D. Timescales of cognition in the brain. Curr. Opin. Behav. Sci. 41, 30–37 (2021). This review discusses how information processing on different neural timescales may contribute to sensory and reward integration in the brain.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Wolff, A. et al. Intrinsic neural timescales: temporal integration and segregation. Trends Cogn. Sci. 26, 159–173 (2022). This paper provides a comprehensive review on the role of intrinsic neural timescales, as measured by the autocorrelation window, for the integration and segregation of sensory input.

    Article  PubMed  Google Scholar 

  24. Singer, W. et al. Neuronal assemblies: necessity, signature and detectability. Trends Cogn. Sci. 1, 252–261 (1997).

    Article  CAS  PubMed  Google Scholar 

  25. Siegel, M., Donner, T. H. & Engel, A. K. Spectral fingerprints of large-scale neuronal interactions. Nat. Rev. Neurosci. 13, 121–134 (2012).

    Article  CAS  PubMed  Google Scholar 

  26. Engel, A., Gerloff, C., Hilgetag, C. & Nolte, G. Intrinsic coupling modes: multiscale interactions in ongoing brain activity. Neuron 80, 867–886 (2013). This review summarizes evidence for the functional importance of intrinsically generated neural coupling at multiple timescales.

    Article  CAS  PubMed  Google Scholar 

  27. Fries, P. Rhythms for cognition: communication through coherence. Neuron 88, 220–235 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Bastos, A. M. et al. Visual areas exert feedforward and feedback influences through distinct frequency channels. Neuron 85, 390–401 (2015).

    Article  CAS  PubMed  Google Scholar 

  29. Michalareas, G. et al. Alpha-beta and gamma rhythms subserve feedback and feedforward influences among human visual cortical areas. Neuron 89, 384–397 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Helfrich, R. F. & Knight, R. T. Oscillatory dynamics of prefrontal cognitive control. Trends Cogn. Sci. 20, 916–930 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Canolty, R. T. & Knight, R. T. The functional role of cross-frequency coupling. Trends Cogn. Sci. 14, 506–515 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  32. Hyafil, A., Giraud, A.-L., Fontolan, L. & Gutkin, B. Neural cross-frequency coupling: connecting architectures, mechanisms, and functions. Trends Neurosci. 38, 725–740 (2015).

    Article  CAS  PubMed  Google Scholar 

  33. Yamashita, Y. & Tani, J. in Computational and Robotic Models of the Hierarchical Organization of Behavior (ed. Baldassarre, G. M.) 47–62 (Springer, 2013).

  34. Buschman, T. J. & Kastner, S. From behavior to neural dynamics: an integrated theory of attention. Neuron 88, 127–144 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Fries, P. Rhythmic attentional scanning. Neuron 111, 954–970 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Bastos, A. et al. Canonical microcircuits for predictive coding. Neuron 76, 695–711 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Pomper, U., Keil, J., Foxe, J. J. & Senkowski, D. Intersensory selective attention and temporal orienting operate in parallel and are instantiated in spatially distinct sensory and motor cortices. Hum. Brain Mapp. 36, 3246–3259 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  38. Daume, J., Graetz, S., Gruber, T., Engel, A. K. & Friese, U. Cognitive control during audiovisual working memory engages frontotemporal theta-band interactions. Sci. Rep. 7, 12585 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Wang, P., Göschl, F., Friese, U., König, P. & Engel, A. K. Long-range functional coupling predicts performance: oscillatory EEG networks in multisensory processing. Neuroimage 196, 114–125 (2019). This EEG study demonstrates phase coupling in multiple frequency bands, as well as phase–amplitude coupling, during visuotactile pattern matching.

    Article  PubMed  Google Scholar 

  40. Lakatos, P., Chen, C.-M., O’Connell, M. N., Mills, A. & Schroeder, C. E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53, 279–292 (2007). This study demonstrates broadband crossmodal tactile-auditory phase resetting in the monkey primary auditory cortex.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Bauer, A. R., Debener, S. & Nobre, A. C. Synchronisation of neural oscillations and cross-modal influences. Trends Cogn. Sci. 24, 481–495 (2020). This review highlights the critical role of crossmodal phase resetting and neural entrainment in multisensory processing.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Galindo-Leon, E. E. et al. Context-specific modulation of intrinsic coupling modes shapes multisensory processing. Sci. Adv. 5, eaar7633 (2019). This study in the ferret cortex shows that phase and amplitude coupling in different frequency bands predicts specific aspects of multisensory processing.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Cooke, J., Poch, C., Gillmeister, H., Costantini, M. & Romei, V. Oscillatory properties of functional connections between sensory areas mediate cross-modal illusory perception. J. Neurosci. 39, 5711–5718 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Senkowski, D., Talsma, D., Herrmann, C. S. & Woldorff, M. G. Multisensory processing and oscillatory gamma responses: effects of spatial selective attention. Exp. Brain Res. 166, 411–426 (2005).

    Article  PubMed  Google Scholar 

  45. Senkowski, D., Talsma, D., Grigutsch, M., Herrmann, C. S. & Woldorff, M. G. Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia 45, 561–571 (2007).

    Article  PubMed  Google Scholar 

  46. Quinn, B. T. et al. Intracranial cortical responses during visual-tactile integration in humans. J. Neurosci. 34, 171–181 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Krebber, M., Harwood, J., Spitzer, B., Keil, J. & Senkowski, D. Visuotactile motion congruence enhances gamma-band activity in visual and somatosensory cortices. Neuroimage 117, 160–169 (2015).

    Article  PubMed  Google Scholar 

  48. Ferraro, S. et al. Stereotactic electroencephalography in humans reveals multisensory signal in early visual and auditory cortices. Cortex 126, 253–264 (2020).

    Article  PubMed  Google Scholar 

  49. Roa Romero, Y., Keil, J., Balz, J., Gallinat, J. & Senkowski, D. Reduced frontal theta oscillations indicate altered crossmodal prediction error processing in schizophrenia. J. Neurophysiol. 116, 1396–1407 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  50. Biau, E., Schultz, B. G., Gunter, T. C. & Kotz, S. A. Left motor δ oscillations reflect asynchrony detection in multisensory speech perception. J. Neurosci. 42, 2313–2326 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Mahjoory, K., Schoffelen, J. M., Keitel, A. & Gross, J. The frequency gradient of human resting-state brain oscillations follows cortical hierarchies. eLife 9, e53715 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Cao, Y., Summerfield, C., Park, H., Giordano, B. L. & Kayser, C. Causal inference in the multisensory brain. Neuron 102, 1076–1087.e8 (2019).

    Article  CAS  PubMed  Google Scholar 

  53. Rohe, T., Ehlis, A. C. & Noppeney, U. The neural dynamics of hierarchical Bayesian causal inference in multisensory perception. Nat. Commun. 10, 1907 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  54. Rohe, T. & Noppeney, U. Cortical hierarchies perform Bayesian causal inference in multisensory perception. PLoS Biol. 13, e1002073 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  55. Rohe, T. & Noppeney, U. Distinct computational principles govern multisensory integration in primary sensory and association cortices. Curr. Biol. 26, 509–514 (2016).

    Article  CAS  PubMed  Google Scholar 

  56. Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Balz, J. et al. GABA concentration in superior temporal sulcus predicts gamma power and perception in the sound-induced flash illusion. Neuroimage 125, 724–730 (2016).

    Article  CAS  PubMed  Google Scholar 

  58. Ozker, M., Schepers, I. M., Magnotti, J. F., Yoshor, D. & Beauchamp, M. S. A double dissociation between anterior and posterior superior temporal gyrus for processing audiovisual speech demonstrated by electrocorticography. J. Cogn. Neurosci. 29, 1044–1060 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  59. Michail, G., Senkowski, D., Holtkamp, M., Wächter, B. & Keil, J. Early beta oscillations in multisensory association areas underlie crossmodal performance enhancement. Neuroimage 257, 119307 (2022).

    Article  PubMed  Google Scholar 

  60. La Rocca, D., Ciuciu, P., Engemann, D. A. & van Wassenhove, V. Emergence of β and γ networks following multisensory training. Neuroimage 206, 116313 (2020). This MEG study demonstrates changes in stimulus-induced beta-band and gamma-band network oscillations after multisensory training.

    Article  PubMed  Google Scholar 

  61. Theves, S., Chan, J. S., Naumer, M. J. & Kaiser, J. Improving audio-visual temporal perception through training enhances beta-band activity. Neuroimage 206, 116312 (2020).

    Article  PubMed  Google Scholar 

  62. Göschl, F., Friese, U., Daume, J., König, P. & Engel, A. K. Oscillatory signatures of crossmodal congruence effects: an EEG investigation employing a visuotactile pattern matching paradigm. Neuroimage 116, 177–186 (2015).

    Article  PubMed  Google Scholar 

  63. Arnal, L. H., Wyart, V. & Giraud, A.-L. Transitions in neural oscillations reflect prediction errors generated in audiovisual speech. Nat. Neurosci. 14, 797–801 (2011). This MEG study shows that crossmodal prediction errors in audiovisual speech processing are expressed in multi-timescale neural dynamics.

    Article  CAS  PubMed  Google Scholar 

  64. Keil, J. & Senkowski, D. Neural oscillations orchestrate multisensory processing. Neuroscientist 24, 609–626 (2018). This review provides an overview of the functional role of neural oscillations with different timescales for multisensory processing.

    Article  PubMed  Google Scholar 

  65. Mercier, M. R. et al. Auditory-driven phase reset in visual cortex: human electrocorticography reveals mechanisms of early multisensory integration. Neuroimage 79, 19–29 (2013).

    Article  PubMed  Google Scholar 

  66. Kayser, C., Petkov, C. I. & Logothetis, N. K. Visual modulation of neurons in auditory cortex. Cereb. Cortex 18, 1560–1574 (2008).

    Article  PubMed  Google Scholar 

  67. Thorne, J. D., De Vos, M., Viola, F. C. & Debener, S. Cross-modal phase reset predicts auditory task performance in humans. J. Neurosci. 31, 3853–3861 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Mercier, M. R. et al. Neuro-oscillatory phase alignment drives speeded multisensory response times: an electro-corticographic investigation. J. Neurosci. 35, 8546–8557 (2015). This human electrocorticography study demonstrates the behavioural relevance of visual-auditory crossmodal phase resetting in auditory cortex.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  69. Daume, J., Wang, P., Maye, A., Zhang, D. & Engel, A. K. Non-rhythmic temporal prediction involves phase resets of low-frequency delta oscillations. Neuroimage 224, 117376 (2021).

    Article  PubMed  Google Scholar 

  70. Lakatos, P. et al. The leading sense: supramodal control of neurophysiological context by attention. Neuron 64, 419–430 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  71. Sieben, K., Röder, B. & Hanganu-Opatz, I. L. Oscillatory entrainment of primary somatosensory cortex encodes visual control of tactile processing. J. Neurosci. 33, 5736–5749 (2013). This study in rats shows that cortico-cortical interactions contribute to crossmodal phase resetting on multiple timescales in somatosensory cortex.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Kuroki, S. et al. Excitatory neuronal hubs configure multisensory integration of slow waves in association cortex. Cell Rep. 22, 2873–2885 (2018).

    Article  CAS  PubMed  Google Scholar 

  73. Mégevand, P. et al. Crossmodal phase reset and evoked responses provide complementary mechanisms for the influence of visual speech in auditory cortex. J. Neurosci. 40, 8530–8542 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Crosse, M. J., Butler, J. S. & Lalor, E. C. Congruent visual speech enhances cortical entrainment to continuous auditory speech in noise-free conditions. J. Neurosci. 35, 14195–14204 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Wang, D., Clouter, A., Chen, Q., Shapiro, K. L. & Hanslmayr, S. Single-trial phase entrainment of theta oscillations in sensory regions predicts human associative memory performance. J. Neurosci. 38, 6299–6309 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  76. Bauer, A. R., van Ede, F., Quinn, A. J. & Nobre, A. C. Rhythmic modulation of visual perception by continuous rhythmic auditory stimulation. J. Neurosci. 41, 7065–7075 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  77. Albouy, P., Martinez-Moreno, Z. E., Hoyer, R. S., Zatorre, R. J. & Baillet, S. Supramodality of neural entrainment: rhythmic visual stimulation causally enhances auditory working memory performance. Sci. Adv. 8, eabj9782 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  78. Maddox, R. K., Atilgan, H., Bizley, J. K. & Lee, A. K. Auditory selective attention is enhanced by a task-irrelevant temporally coherent visual stimulus in human listeners. eLife 4, e04995 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  79. Kösem, A. & van Wassenhove, V. Temporal structure in audiovisual sensory selection. PLoS ONE 7, e40936 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  80. Fu, X. & Riecke, L. Effects of continuous tactile stimulation on auditory-evoked cortical responses depend on the audio-tactile phase. Neuroimage 274, 120140 (2023).

    Article  PubMed  Google Scholar 

  81. Keil, J., Müller, N., Ihssen, N. & Weisz, N. On the variability of the McGurk effect: audiovisual integration depends on prestimulus brain states. Cereb. Cortex 22, 221–231 (2012).

    Article  PubMed  Google Scholar 

  82. Ghazanfar, A. A., Chandrasekaran, C. & Logothetis, N. K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. J. Neurosci. 28, 4457–4469 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  83. Maier, J. X., Chandrasekaran, C. & Ghazanfar, A. A. Integration of bimodal looming signals through neuronal coherence in the temporal lobe. Curr. Biol. 18, 963–968 (2008).

    Article  CAS  PubMed  Google Scholar 

  84. Kayser, C. & Logothetis, N. K. Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Front. Integr. Neurosci. 3, 7 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  85. Bieler, M. et al. Rate and temporal coding convey multisensory information in primary sensory cortices. eNeuro https://doi.org/10.1523/ENEURO.0037-17.2017 (2017).

  86. Olofsson, J. K., Zhou, G., East, B. S., Zelano, C. & Wilson, D. A. Odor identification in rats: behavioral and electrophysiological evidence of learned olfactory-auditory associations. eNeuro https://doi.org/10.1523/ENEURO.0102-19.2019 (2019).

  87. Clouter, A., Shapiro, K. L. & Hanslmayr, S. Theta phase synchronization is the glue that binds human associative memory. Curr. Biol. 27, 3143–3148 (2017).

    Article  CAS  PubMed  Google Scholar 

  88. Grabot, L., Kösem, A., Azizi, L. & van Wassenhove, V. Prestimulus alpha oscillations and the temporal sequencing of audiovisual events. J. Cogn. Neurosci. 29, 1566–1582 (2017).

    Article  PubMed  Google Scholar 

  89. Ikumi, N., Torralba, M., Ruzzoli, M. & Soto-Faraco, S. The phase of pre-stimulus brain oscillations correlates with cross-modal synchrony perception. Eur. J. Neurosci. 49, 150–164 (2019).

    Article  PubMed  Google Scholar 

  90. Kaiser, M., Senkowski, D., Busch, N. A., Balz, J. & Keil, J. Single trial prestimulus oscillations predict perception of the sound-induced flash illusion. Sci. Rep. 9, 5983 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  91. Kaiser, M., Senkowski, D. & Keil, J. Mediofrontal theta-band oscillations reflect top-down influence in the ventriloquist illusion. Hum. Brain Mapp. 42, 452–466 (2021).

    Article  PubMed  Google Scholar 

  92. Keil, J., Müller, N., Hartmann, T. & Weisz, N. Prestimulus beta power and phase synchrony influence the sound-induced flash illusion. Cereb. Cortex 24, 1278–1288 (2014).

    Article  PubMed  Google Scholar 

  93. Leonardelli, E. et al. Prestimulus oscillatory alpha power and connectivity patterns predispose perceptual integration of an audio and a tactile stimulus. Hum. Brain Mapp. 36, 3486–3498 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  94. London, R. E. et al. EEG alpha power predicts the temporal sensitivity of multisensory perception. Eur. J. Neurosci. 55, 3241–3255 (2022).

    Article  CAS  PubMed  Google Scholar 

  95. Buergers, S. & Noppeney, U. The role of alpha oscillations in temporal binding within and across the senses. Nat. Hum. Behav. 6, 732–742 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  96. Cecere, R., Rees, G. & Romei, V. Individual differences in alpha frequency drive crossmodal illusory perception. Curr. Biol. 25, 231–235 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  97. Keil, J. & Senkowski, D. Individual alpha frequency relates to the sound-induced flash illusion. Multisens. Res. 30, 565–578 (2017).

    Article  PubMed  Google Scholar 

  98. VanRullen, R. & Koch, C. Is perception discrete or continuous? Trends Cogn. Sci. 7, 207–213 (2003).

    Article  PubMed  Google Scholar 

  99. Lange, J., Keil, J., Schnitzler, A., van Dijk, H. & Weisz, N. The role of alpha oscillations for illusory perception. Behav. Brain Res. 271, 294–301 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  100. Samaha, J. & Romei, V. Alpha-band frequency and temporal windows in perception: a review and living meta-analysis of 27 experiments (and counting). J. Cogn. Neurosci. 36, 640–654 (2024).

    Article  PubMed  Google Scholar 

  101. Schoffelen, J. M., Pesci, U. G. & Noppeney, U. Alpha oscillations and temporal binding windows in perception — a critical review and best practice guidelines. J. Cogn. Neurosci. 36, 655–690 (2024).

    Article  PubMed  Google Scholar 

  102. Hipp, J. F., Engel, A. K. & Siegel, M. Oscillatory synchronization in large-scale cortical networks predicts perception. Neuron 69, 387–396 (2011).

    Article  CAS  PubMed  Google Scholar 

  103. Zhou, G. et al. Human olfactory-auditory integration requires phase synchrony between sensory cortices. Nat. Commun. 10, 1168 (2019). Using human intracranial recordings, this study shows that auditory–olfactory matching involves phase–amplitude coupling between auditory and olfactory cortical regions.

    Article  PubMed  PubMed Central  Google Scholar 

  104. Powers, A. R., Hillock, A. R. & Wallace, M. T. Perceptual training narrows the temporal window of multisensory binding. J. Neurosci. 29, 12265–12274 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  105. Lennert, T., Samiee, S. & Baillet, S. Coupled oscillations enable rapid temporal recalibration to audiovisual asynchrony. Commun. Biol. 4, 559 (2021). This MEG study demonstrates rapid temporal recalibration of alpha–gamma phase–amplitude coupling in response to audiovisual temporal asynchronies.

    Article  PubMed  PubMed Central  Google Scholar 

  106. Bidelman, G. M. Musicians have enhanced audiovisual multisensory binding: experience-dependent effects in the double-flash illusion. Exp. Brain Res. 234, 3037–3047 (2016).

    Article  PubMed  Google Scholar 

  107. Lee, H. & Noppeney, U. Long-term music training tunes how the brain temporally binds signals from multiple senses. Proc. Natl Acad. Sci. USA 108, E1441–E1450 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  108. O’Donohue, M., Lacherez, P. & Yamamoto, N. Musical training refines audiovisual integration but does not influence temporal recalibration. Sci. Rep. 12, 15292 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  109. Klein, C., Liem, F., Hänggi, J., Elmer, S. & Jäncke, L. The “silent” imprint of musical training. Hum. Brain Mapp. 37, 536–546 (2016). This EEG study in musicians shows that sustained training influences multi-timescale network dynamics in the brain.

    Article  PubMed  Google Scholar 

  110. Cocchi, L., Gollo, L. L., Zalesky, A. & Breakspear, M. Criticality in the brain: a synthesis of neurobiology, models and cognition. Prog. Neurobiol. 158, 132–152 (2017). This review highlights the important role of criticality, which is characterized by activity fluctuations that have no preferred timescale and allow the brain to rapidly adapt to environmental changes.

    Article  PubMed  Google Scholar 

  111. La Rocca, D., Zilber, N., Abry, P., van Wassenhove, V. & Ciuciu, P. Self-similarity and multifractality in human brain activity: a wavelet-based analysis of scale-free brain dynamics. J. Neurosci. Methods 309, 175–187 (2018).

    Article  PubMed  Google Scholar 

  112. Palva, J. M. et al. Neuronal long-range temporal correlations and avalanche dynamics are correlated with behavioral scaling laws. Proc. Natl Acad. Sci. USA 110, 3585–3590 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  113. Palva, S. & Palva, J. M. Roles of brain criticality and multiscale oscillations in temporal predictions for sensorimotor processing. Trends Neurosci. 41, 729–743 (2018). This review highlights the importance of scale-free neural dynamics for the role of neural oscillations and cross-frequency coupling in sensorimotor processing.

    Article  CAS  PubMed  Google Scholar 

  114. Linkenkaer-Hansen, K., Nikouline, V. V., Palva, J. M. & Ilmoniemi, R. J. Long-range temporal correlations and scaling behavior in human brain oscillations. J. Neurosci. 21, 1370–1377 (2001).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  115. Gross, J. et al. Speech rhythms and multiplexed oscillatory sensory coding in the human brain. PLoS Biol. 11, e1001752 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  116. Schmitt, L. M. et al. Predicting speech from a cortical hierarchy of event-based time scales. Sci. Adv. 7, eabi6070 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  117. Park, H., Kayser, C., Thut, G. & Gross, J. Lip movements entrain the observers’ low-frequency brain oscillations to facilitate speech intelligibility. eLife 5, e14521 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  118. Aller, M., Økland, H. S., MacGregor, L. J., Blank, H. & Davis, M. H. Differential auditory and visual phase-locking are observed during audio-visual benefit and silent lip-reading for speech perception. J. Neurosci. 42, 6108–6120 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  119. Perrodin, C., Kayser, C., Logothetis, N. K. & Petkov, C. I. Natural asynchronies in audiovisual communication signals regulate neuronal multisensory interactions in voice-sensitive cortex. Proc. Natl Acad. Sci. USA 112, 273–278 (2015).

    Article  CAS  PubMed  Google Scholar 

  120. Thézé, R., Giraud, A. L. & Mégevand, P. The phase of cortical oscillations determines the perceptual fate of visual cues in naturalistic audiovisual speech. Sci. Adv. 6, eabc6348 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  121. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A. & Ghazanfar, A. A. The natural statistics of audiovisual speech. PLoS Comput. Biol. 5, e1000436 (2009). This study in monkeys shows that multi-timescale neural responses, as reflected in different frequency band oscillations, match to the natural features of audiovisual speech.

    Article  PubMed  PubMed Central  Google Scholar 

  122. Chalas, N., Omigie, D., Poeppel, D. & van Wassenhove, V. Hierarchically nested networks optimize the analysis of audiovisual speech. iScience 26, 106257 (2023). This MEG study in humans demonstrates that hierarchically nested oscillatory brain networks track audiovisual speech asynchronies.

    Article  PubMed  PubMed Central  Google Scholar 

  123. Noppeney, U. Perceptual inference, learning, and attention in a multisensory world. Annu. Rev. Neurosci. 44, 449–473 (2021).

    Article  CAS  PubMed  Google Scholar 

  124. Feng, W., Störmer, V. S., Martinez, A., McDonald, J. J. & Hillyard, S. A. Involuntary orienting of attention to a sound desynchronizes the occipital alpha rhythm and improves visual perception. Neuroimage 150, 318–328 (2017).

    Article  PubMed  Google Scholar 

  125. Friese, U. et al. Oscillatory brain activity during multisensory attention reflects activation, disinhibition, and cognitive control. Sci. Rep. 6, 32775 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  126. Gomez-Ramirez, M. et al. Oscillatory sensory selection mechanisms during intersensory attention to rhythmic auditory and visual inputs: a human electrocorticographic investigation. J. Neurosci. 31, 18556–18567 (2011). This human intracranial recording study demonstrates a multi-timescale phase–amplitude coupling mechanism that supports multisensory attention.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  127. Keil, J., Pomper, U. & Senkowski, D. Distinct patterns of local oscillatory activity and functional connectivity underlie intersensory attention and temporal prediction. Cortex 74, 277–288 (2016).

    Article  PubMed  Google Scholar 

  128. Zuo, Y., Huang, Y., Wu, D., Wang, Q. & Wang, Z. Spike phase shift relative to beta oscillations mediates modality selection. Cereb. Cortex 30, 5431–5448 (2020).

    Article  PubMed  Google Scholar 

  129. Panichello, M. F. & Buschman, T. J. Shared mechanisms underlie the control of working memory and attention. Nature 592, 601–605 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  130. Quak, M., London, R. E. & Talsma, D. A multisensory perspective of working memory. Front. Hum. Neurosci. 9, 197 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  131. Fiebelkorn, I. C. & Kastner, S. A rhythmic theory of attention. Trends Cogn. Sci. 23, 87–101 (2019).

    Article  PubMed  Google Scholar 

  132. Axmacher, N. et al. Cross-frequency coupling supports multi-item working memory in the human hippocampus. Proc. Natl Acad. Sci. USA 107, 3228–3233 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  133. Xie, Y. et al. Theta oscillations and source connectivity during complex audiovisual object encoding in working memory. Front. Hum. Neurosci. 15, 614950 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  134. Michail, G., Senkowski, D., Niedeggen, M. & Keil, J. Memory load alters perception-related neural oscillations during multisensory integration. J. Neurosci. 41, 1505–1515 (2021). This EEG study shows that interactions between working memory and multisensory perception are expressed in multi-timescale interactions of neural oscillations.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  135. Spitzer, B. & Blankenburg, F. Supramodal parametric working memory processing in humans. J. Neurosci. 32, 3287–3295 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  136. Friston, K. The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127–138 (2010).

    Article  CAS  PubMed  Google Scholar 

  137. Arnal, L. H. & Giraud, A.-L. Cortical oscillations and sensory predictions. Trends Cogn. Sci. 16, 390–398 (2012).

    Article  PubMed  Google Scholar 

  138. Bastos, A. M., Lundqvist, M., Waite, A. S., Kopell, N. & Miller, E. K. Layer and rhythm specificity for predictive routing. Proc. Natl Acad. Sci. USA 117, 31459–31469 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  139. Fernandez-Ruiz, A., Sirota, A., Lopes-dos-Santos, V. & Dupret, D. Over and above frequency: gamma oscillations as units of neural circuit operations. Neuron 111, 936–953 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  140. Arnal, L. H., Doelling, K. B. & Poeppel, D. Delta-beta coupled oscillations underlie temporal prediction accuracy. Cereb. Cortex 25, 3077–3085 (2015).

    Article  PubMed  Google Scholar 

  141. Luo, H., Liu, Z. & Poeppel, D. Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation. PLoS Biol. 8, e1000445 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  142. Lakatos, P. et al. An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J. Neurophysiol. 94, 1904–1911 (2005).

    Article  PubMed  Google Scholar 

  143. Schroeder, C. E. & Lakatos, P. Low-frequency neuronal oscillations as instruments of sensory selection. Trends Neurosci. 32, 9–18 (2009).

    Article  CAS  PubMed  Google Scholar 

  144. van Atteveldt, N., Murray, M. M., Thut, G. & Schroeder, C. E. Multisensory integration: flexible use of general operations. Neuron 81, 1240–1253 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  145. Roopun, A. K. et al. Temporal interactions between cortical rhythms. Front. Neurosci. 2, 145–154 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  146. Yang, H., Shew, W. L., Roy, R. & Plenz, D. Maximal variability of phase synchrony in cortical networks with neuronal avalanches. J. Neurosci. 32, 1061–1072 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  147. Chandrasekaran, C. Computational principles and models of multisensory integration. Curr. Opin. Neurobiol. 43, 25–34 (2017).

    Article  CAS  PubMed  Google Scholar 

  148. Shams, L. & Beierholm, U. Bayesian causal inference: a unifying neuroscience theory. Neurosci. Biobehav. Rev. 137, 104619 (2022).

    Article  PubMed  Google Scholar 

  149. Parise, C. V. & Ernst, M. O. Correlation detection as a general mechanism for multisensory integration. Nat. Commun. 7, 11543 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  150. Pesnot Lerousseau, J., Parise, C. V., Ernst, M. O. & van Wassenhove, V. Multisensory correlation computations in the human brain identified by a time-resolved encoding model. Nat. Commun. 13, 2489 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  151. Kramer, M. A. et al. Rhythm generation through period concatenation in rat somatosensory cortex. PLoS Comput. Biol. 4, e1000169 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  152. Gelastopoulos, A., Whittington, M. A. & Kopell, N. J. Parietal low beta rhythm provides a dynamical substrate for a working memory buffer. Proc. Natl Acad. Sci. USA 116, 16613–16620 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  153. Schirner, M., McIntosh, A. R., Jirsa, V., Deco, G. & Ritter, P. Inferring multi-scale neural mechanisms with brain network modelling. eLife 7, e28927 (2018). This article presents a computational modelling approach for the analysis of neural activity from humans and animals on multiple spatial and temporal scales.

    Article  PubMed  PubMed Central  Google Scholar 

  154. D’Angelo, E. & Jirsa, V. The quest for multiscale brain modeling. Trends Neurosci. 45, 777–790 (2022).

    Article  PubMed  Google Scholar 

  155. Einevoll, G. T., Kayser, C., Logothetis, N. K. & Panzeri, S. Modelling and analysis of local field potentials for studying the function of cortical circuits. Nat. Rev. Neurosci. 14, 770–785 (2013).

    Article  CAS  PubMed  Google Scholar 

  156. Nasr, K. et al. Breaking the boundaries of interacting with the human brain using adaptive closed-loop stimulation. Progr. Neurobiol. 216, 102311 (2022).

    Article  Google Scholar 

  157. Haslacher, D. et al. In vivo phase-dependent enhancement and suppression of human brain oscillations by transcranial alternating current stimulation (tACS). Neuroimage 275, 120187 (2023).

    Article  PubMed  Google Scholar 

  158. Fenno, L., Yizhar, O. & Deisseroth, K. The development and application of optogenetics. Annu. Rev. Neurosci. 34, 389–412 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  159. Grosenick, L., Marshel, J. H. & Deisseroth, K. Closed-loop and activity-guided optogenetic control. Neuron 86, 106–139 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  160. Zhou, H.-y et al. Multisensory temporal binding window in autism spectrum disorders and schizophrenia spectrum disorders: a systematic review and meta-analysis. Neurosci. Biobehav. Rev. 86, 66–76 (2018).

    Article  PubMed  Google Scholar 

  161. Gröhn, C., Norgren, E. & Eriksson, L. A systematic review of the neural correlates of multisensory integration in schizophrenia. Schizophr. Res. Cogn. 27, 100219 (2022).

    Article  PubMed  Google Scholar 

  162. Feldman, J. I. et al. Audiovisual multisensory integration in individuals with autism spectrum disorder: a systematic review and meta-analysis. Neurosci. Biobehav. Rev. 95, 220–234 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  163. Wallace, M. T., Woynaroski, T. G. & Stevenson, R. A. Multisensory integration as a window into orderly and disrupted cognition and communication. Annu. Rev. Psychol. 71, 193–219 (2020).

    Article  PubMed  Google Scholar 

  164. van Laarhoven, T., Stekelenburg, J. J., Eussen, M. L. & Vroomen, J. Atypical visual-auditory predictive coding in autism spectrum disorder: electrophysiological evidence from stimulus omissions. Autism 24, 1849–1859 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  165. Hirano, S. et al. Phase-amplitude coupling of the electroencephalogram in the auditory cortex in schizophrenia. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 3, 69–76 (2018).

    PubMed  Google Scholar 

  166. Moran, J. K., Michail, G., Heinz, A., Keil, J. & Senkowski, D. Long-range temporal correlations in resting state beta oscillations are reduced in schizophrenia. Front. Psychiatry 10, 517 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  167. Cruz, G. et al. Long range temporal correlations (LRTCs) in MEG-data during emerging psychosis: relationship to symptoms, medication-status and clinical trajectory. Neuroimage Clin. 31, 102722 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  168. Rognini, G. et al. Multisensory bionic limb to achieve prosthesis embodiment and reduce distorted phantom limb perceptions. J. Neurol. Neurosurg. Psychiatry 90, 833–836 (2019).

    Article  PubMed  Google Scholar 

  169. Aurucci, G. V., Preatoni, G., Damiani, A. & Raspopovic, S. Brain-computer interface to deliver individualized multisensory intervention for neuropathic pain. Neurotherapeutics 20, 1316–1329 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  170. Lloyd-Esenkaya, T., Lloyd-Esenkaya, V., O’Neill, E. & Proulx, M. J. Multisensory inclusive design with sensory substitution. Cogn. Res. Princ. Implic. 5, 37 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  171. Mackin, A. & Bull, D. Characterizing the spatiotemporal envelope of the human visual system through the visibility of temporal aliasing artifacts. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 37, 1116–1127 (2020).

    Article  PubMed  Google Scholar 

  172. Mori, S., Kikuchi, Y., Hirose, N., Lepage, H. & Wong, W. Auditory gap detection: psychometric functions and insights into the underlying neural activity. Biol. Cybern. 112, 575–584 (2018).

    Article  PubMed  Google Scholar 

  173. Victor, J. D. How the brain uses time to represent and process visual information. Brain Res. 886, 33–46 (2000).

    Article  CAS  PubMed  Google Scholar 

  174. Uchida, N., Poo, C. & Haddad, R. Coding and transformations in the olfactory system. Annu. Rev. Neurosci. 37, 363–385 (2014).

    Article  CAS  PubMed  Google Scholar 

  175. Kayser, C., Montemurro, M. A., Logothetis, N. K. & Panzeri, S. Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns. Neuron 61, 597–608 (2009).

    Article  CAS  PubMed  Google Scholar 

  176. Chaudhuri, R., Knoblauch, K., Gariel, M.-A., Kennedy, H. & Wang, X.-J. A large-scale circuit mechanism for hierarchical dynamical processing in the primate cortex. Neuron 88, 419–431 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  177. Villalonga, M. B. & Sekuler, R. Keep your finger on the pulse: better rate perception and gap detection with vibrotactile compared to visual stimuli. Atten. Percept. Psychophys. 85, 2004–2017 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  178. Manger, P. R., Engler, G., Moll, C. K. E. & Engel, A. K. The anterior ectosylvian visual area of the ferret: a homologue for an enigmatic visual cortical area of the cat? Eur. J. Neurosci. 22, 706–714 (2005).

    Article  PubMed  Google Scholar 

  179. Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cogn. Psychol. 12, 97–136 (1980).

    Article  CAS  PubMed  Google Scholar 

  180. Biederman, I. Recognition-by-components: a theory of human image understanding. Psychol. Rev. 94, 115–147 (1987).

    Article  PubMed  Google Scholar 

  181. Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995).

    Article  CAS  PubMed  Google Scholar 

  182. Johansen-Berg, H. & Lloyd, D. M. The physiology and psychology of selective attention to touch. Front. Biosci. Landmark 5, 894–904 (2000).

    Article  Google Scholar 

  183. Fritz, J. B., Elhilali, M., David, S. V. & Shamma, S. A. Auditory attention — focusing the searchlight on sound. Curr. Opin. Neurobiol. 17, 437–455 (2007).

    Article  CAS  PubMed  Google Scholar 

  184. Freeman, W. J. Simulation of chaotic EEG patterns with a dynamic model of the olfactory system. Biol. Cybern. 56, 139–150 (1987).

    Article  CAS  PubMed  Google Scholar 

  185. Gray, C. M. & Singer, W. Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc. Natl Acad. Sci. USA 86, 1698–1702 (1989).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  186. Gray, C. M., König, P., Engel, A. K. & Singer, W. Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338, 334–337 (1989).

    Article  CAS  PubMed  Google Scholar 

  187. Fries, P., Reynolds, J. H., Rorie, A. E. & Desimone, R. Modulation of oscillatory neuronal synchronization by selective visual attention. Science 291, 1560–1563 (2001).

    Article  CAS  PubMed  Google Scholar 

  188. Ghazanfar, A. A. & Schroeder, C. E. Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278–285 (2006).

    Article  PubMed  Google Scholar 

  189. Voloh, B. & Womelsdorf, T. A role of phase-resetting in coordinating large scale neural networks during attention and goal-directed behavior. Front. Syst. Neurosci. 10, 18 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  190. Atilgan, H. et al. Integration of visual information in auditory cortex promotes auditory scene analysis through multisensory binding. Neuron 97, 640–655 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  191. Gingras, G., Rowland, B. A. & Stein, B. E. The differing impact of multisensory and unisensory integration on behavior. J. Neurosci. 29, 4897–4902 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  192. Pluta, S. R., Rowland, B. A., Stanford, T. R. & Stein, B. E. Alterations to multisensory and unisensory integration by stimulus competition. J. Neurophysiol. 106, 3091–3101 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

D.S. discloses support for this work from the Deutsche Forschungsgemeinschaft (DFG) (SE1859/10-1). A.K.E. acknowledges support for this work from the DFG (SFB936-178316478-A2/A3; TRR169-261402652-B1/B4/Z2) and from the European Union (project cICMs, ERC-2022-AdG-101097402). Views and opinions expressed in this article are those of the authors only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them.

Author information

Authors and Affiliations

  1. Department of Psychiatry and Neurosciences, Charité - Universitätsmedizin Berlin, Berlin, Germany

    Daniel Senkowski

  2. Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

    Andreas K. Engel

Authors
  1. Daniel Senkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas K. Engel
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors contributed equally to all aspects of the article.

Corresponding author

Correspondence to Andreas K. Engel.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Neuroscience thanks Nancy Kopell, Christoph Kayser and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Glossary

Alpha

The alpha band comprises frequencies between 8 and 12 Hz; rhythms in this frequency band are also called alpha oscillations; these relatively slow oscillations constitute processing time windows of about 100 ms.

Amplitude modulation

Change of the amplitude of a neural oscillation in response to an external input.

Autocorrelation

Correlation between values of a signal at different time points, as a function of the lag between these time points.

Beta

The beta band comprises frequencies between 13 and 30 Hz; rhythmic activity in this frequency band is also known as beta oscillations; the processing time window provided by these fast oscillations is in the range of 40–70 ms.

Criticality

State of a system close to a phase transition, in which spatiotemporal correlations are highly susceptible to perturbations.

Cross-frequency coupling

Coupling of the power and/or phase of neural oscillations across different frequency bands.

Crossmodal

An interaction between neural systems in which input from one sensory modality influences the processing in another sensory modality.

Delta

The delta band comprises frequencies below 3 Hz; rhythmic activity in this frequency band is also addressed as delta oscillations; these slow oscillations provide long processing time windows with durations of several hundreds of milliseconds.

Double-flash illusion

Illusory perception of two flashes when a single flash is presented together with either two auditory clicks or two tactile stimuli.

Entrainment

Synchronization of neural activity to a rhythmic external input.

Envelope coupling

Dynamic coupling based on the correlation of the amplitude envelope of neural signals.

Functional connectivity

Statistical relationship between signals recorded from different neurons or brain regions.

Gamma

The gamma band comprises oscillations at frequencies above 30 Hz, also called gamma oscillations; often, the gamma band is subdivided into low and high gamma, depending on whether the oscillation frequencies are below or above about 80 Hz; the processing time window provided by these fast oscillations has a duration below 20–30 ms.

Intrinsic coupling modes

Neural coupling patterns that are not imposed by external factors but are generated in the brain.

Intrinsic neural timescales

Processing time windows inferred from the autocorrelation of neural signals in a given brain region.

Inverse effectiveness

Multisensory interactions are strongest when the respective unisensory stimuli, presented alone, elicit weak neural responses.

Long-range temporal correlations

Persistence of correlations in time series data across multiple timescales.

McGurk effect

Fusion of an auditory syllable or word paired with a conflicting visual syllable or word into a combined multisensory percept.

Neural dynamics

Spatiotemporal change of activity patterns in neuronal populations.

Oscillation cycle

The repeatable part of an oscillatory waveform, comprising one trough and one peak; the duration of the cycle defines the length of the functional time window provided by the oscillation.

Oscillations

Rhythmic patterns of electrical activity generated by a cooperating group of neurons.

Oscillatory frequency bands

Range of frequencies that define the characteristic dynamics of a rhythmic activity pattern; neural oscillations are typically classified into five frequency bands: delta (<3 Hz), theta (4–7 Hz), alpha (8–12 Hz), beta (13–30 Hz) and gamma (>30 Hz).

Phase–amplitude coupling

Modulation of the amplitude of an oscillatory signal by the phase of another oscillatory signal.

Phase coupling

Dynamic coupling based on the correlation of the phase of neural signals.

Phase resetting

Shift of the phase of a neural oscillation in response to an external input.

Power

Square of the amplitude of a sinusoidal wave.

Power-law scaling

Relation between two variables in which one variable varies as a power of the other; this yields a linear relation when both variables are displayed in a logarithmic manner.

Predictive processing

Brain process involved in generating and updating predictions about sensory inputs or events.

Scale-free dynamics

Lack of a characteristic timescale in the dynamics of a neural process.

Supramodal

Cognitive process that operates across different sensory modalities.

Theta

The theta band comprises frequencies between 4 and 7 Hz; rhythmic activity in this frequency band is also denoted as theta oscillations; these relatively slow oscillations constitute processing time windows of about 150–250 ms.

Time windows

Temporal epochs in which information about sensory stimuli can be encoded or processed; such windows can vary in duration, giving rise to different processing timescales.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Senkowski, D., Engel, A.K. Multi-timescale neural dynamics for multisensory integration. Nat. Rev. Neurosci. 25, 625–642 (2024). https://doi.org/10.1038/s41583-024-00845-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41583-024-00845-7

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing