Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Closed-loop training of attention with real-time brain imaging

Nature Neuroscience volume 18, pages 470–475 (2015)Cite this article

Subjects

Abstract

Lapses of attention can have negative consequences, including accidents and lost productivity. Here we used closed-loop neurofeedback to improve sustained attention abilities and reduce the frequency of lapses. During a sustained attention task, the focus of attention was monitored in real time with multivariate pattern analysis of whole-brain neuroimaging data. When indicators of an attentional lapse were detected in the brain, we gave human participants feedback by making the task more difficult. Behavioral performance improved after one training session, relative to control participants who received feedback from other participants' brains. This improvement was largest when feedback carried information from a frontoparietal attention network. A neural consequence of training was that the basal ganglia and ventral temporal cortex came to represent attentional states more distinctively. These findings suggest that attentional failures do not reflect an upper limit on cognitive potential and that attention can be trained with appropriate feedback about neural signals.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Real-time pipeline.
Figure 2: Brain-behavior relationship.
Figure 3: Change in behavior.
Figure 4: Searchlight analyses.
Figure 5: Potential sources of feedback.

Similar content being viewed by others

References

  1. Chun, M.M., Golomb, J.D. & Turk-Browne, N.B. A taxonomy of external and internal attention. Annu. Rev. Psychol. 62, 73–101 (2011).

    Article  PubMed  Google Scholar 

  2. Mackworth, N.H. The breakdown of vigilance during prolonged visual search. Q. J. Exp. Psychol. 1, 6–21 (1948).

    Article  Google Scholar 

  3. Redelmeier, D.A. & Tibshirani, R.J. Association between cellular-telephone calls and motor vehicle collisions. N. Engl. J. Med. 336, 453–458 (1997).

    Article  CAS  PubMed  Google Scholar 

  4. Czeisler, C.A. et al. Modafinil for excessive sleepiness associated with shift-work sleep disorder. N. Engl. J. Med. 353, 476–486 (2005).

    Article  CAS  PubMed  Google Scholar 

  5. Dinges, D.F. & Powell, J.W. Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations. Behav. Res. Methods Instrum. Comput. 17, 652–655 (1985).

    Article  Google Scholar 

  6. Sarter, M., Givens, B. & Bruno, J.P. The cognitive neuroscience of sustained attention: where top-down meets bottom-up. Brain Res. Brain Res. Rev. 35, 146–160 (2001).

    Article  CAS  PubMed  Google Scholar 

  7. Wolfe, J.M., Horowitz, T.S. & Kenner, N.M. Rare items often missed in visual searches. Nature 435, 439–440 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Johnson, K.A. et al. Dissociation in performance of children with ADHD and high-functioning autism on a task of sustained attention. Neuropsychologia 45, 2234–2245 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Robertson, I.H., Manly, T., Andrade, J., Baddeley, B.T. & Yiend, J. 'Oops!': performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia 35, 747–758 (1997).

    Article  CAS  PubMed  Google Scholar 

  10. Drew, T. & Vogel, E.K. Neural measures of individual differences in selecting and tracking multiple moving objects. J. Neurosci. 28, 4183–4191 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Weiskopf, N. et al. Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Trans. Biomed. Eng. 51, 966–970 (2004).

    Article  PubMed  Google Scholar 

  12. LaConte, S.M. Decoding fMRI brain states in real-time. Neuroimage 56, 440–454 (2011).

    Article  PubMed  Google Scholar 

  13. Sulzer, J. et al. Real-time fMRI neurofeedback: progress and challenges. Neuroimage 76, 386–399 (2013).

    Article  CAS  PubMed  Google Scholar 

  14. Norman, K.A., Polyn, S.M., Detre, G.J. & Haxby, J.V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430 (2006).

    Article  PubMed  Google Scholar 

  15. deCharms, R.C. et al. Control over brain activation and pain learned by using real-time functional MRI. Proc. Natl. Acad. Sci. USA 102, 18626–18631 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Shibata, K., Watanabe, T., Sasaki, Y. & Kawato, M. Perceptual learning incepted by decoded fMRI neurofeedback without stimulus presentation. Science 334, 1413–1415 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Yoo, J.J. et al. When the brain is prepared to learn: enhancing human learning using real-time fMRI. Neuroimage 59, 846–852 (2012).

    Article  PubMed  Google Scholar 

  18. Hinds, O. et al. Roles of default-mode network and supplementary motor area in human vigilance performance: evidence from real-time fMRI. J. Neurophysiol. 109, 1250–1258 (2013).

    Article  CAS  PubMed  Google Scholar 

  19. Yoss, R.E., Moyer, N.J. & Hollenhorst, R.W. Pupil size and spontaneous pupillary waves associated with alertness, drowsiness, and sleep. Neurology 20, 545–554 (1970).

    Article  CAS  PubMed  Google Scholar 

  20. Rosenberg, M., Noonan, S., DeGutis, J. & Esterman, M. Sustaining visual attention in the face of distraction: a novel gradual-onset continuous performance task. Atten. Percept. Psychophys. 75, 426–439 (2013).

    Article  PubMed  Google Scholar 

  21. O'Craven, K.M., Downing, P.E. & Kanwisher, N. fMRI evidence for objects as the units of attentional selection. Nature 401, 584–587 (1999).

    Article  CAS  PubMed  Google Scholar 

  22. Al-Aidroos, N., Said, C.P. & Turk-Browne, N.B. Top-down attention switches coupling between low-level and high-level areas of human visual cortex. Proc. Natl. Acad. Sci. USA 109, 14675–14680 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Weissman, D.H., Roberts, K.C., Visscher, K.M. & Woldorff, M.G. The neural bases of momentary lapses in attention. Nat. Neurosci. 9, 971–978 (2006).

    Article  CAS  PubMed  Google Scholar 

  24. Leber, A.B., Turk-Browne, N.B. & Chun, M.M. Neural predictors of moment-to-moment fluctuations in cognitive flexibility. Proc. Natl. Acad. Sci. USA 105, 13592–13597 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Miller, E.K. & Cohen, J.D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

    Article  CAS  PubMed  Google Scholar 

  26. Noudoost, B., Chang, M.H., Steinmetz, N.A. & Moore, T. Top-down control of visual attention. Curr. Opin. Neurobiol. 20, 183–190 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Todd, M.T., Nystrom, L.E. & Cohen, J.D. Confounds in multivariate pattern analysis: theory and rule representation case study. Neuroimage 77, 157–165 (2013).

    Article  PubMed  Google Scholar 

  28. Haxby, J.V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).

    Article  CAS  PubMed  Google Scholar 

  29. Woolgar, A., Hampshire, A., Thompson, R. & Duncan, J. Adaptive coding of task-relevant information in human frontoparietal cortex. J. Neurosci. 31, 14592–14599 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Turk-Browne, N.B. Functional interactions as big data in the human brain. Science 342, 580–584 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Reddy, L., Kanwisher, N.G. & VanRullen, R. Attention and biased competition in multi-voxel object representations. Proc. Natl. Acad. Sci. USA 106, 21447–21452 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Baldauf, D. & Desimone, R. Neural mechanisms of object-based attention. Science 344, 424–427 (2014).

    Article  CAS  PubMed  Google Scholar 

  33. O'Reilly, R.C. & Frank, M. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283–328 (2006).

    Article  PubMed  Google Scholar 

  34. Kravitz, A.V. et al. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622–626 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Wiecki, T.V. & Frank, M.J. A computational model of inhibitory control in frontal cortex and basal ganglia. Psychol. Rev. 120, 329–355 (2013).

    Article  PubMed  Google Scholar 

  36. Foerde, K. & Shohamy, D. The role of the basal ganglia in learning and memory: insight from Parkinson's disease. Neurobiol. Learn. Mem. 96, 624–636 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Frank, M.J. & Badre, D. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cereb. Cortex 22, 509–526 (2012).

    Article  PubMed  Google Scholar 

  38. Biggs, A.T. & Mitroff, S.R. Different predictors of multiple-target search accuracy between nonprofessional and professional visual searchers. Q. J. Exp. Psychol. (Hove) 67, 1335–1348 (2014).

    Article  Google Scholar 

  39. Disner, S.G., Beevers, C.G., Haigh, E.A.P. & Beck, A.T. Neural mechanisms of the cognitive model of depression. Nat. Rev. Neurosci. 12, 467–477 (2011).

    Article  CAS  PubMed  Google Scholar 

  40. Shallice, T. et al. Executive function profile of children with attention deficit hyperactivity disorder. Dev. Neuropsychol. 21, 43–71 (2002).

    Article  PubMed  Google Scholar 

  41. Chadick, J.Z. & Gazzaley, A. Differential coupling of visual cortex with default network or frontal-parietal network based on goals. Nat. Neurosci. 14, 830–832 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Moore, K.S., Yi, D.-J. & Chun, M. The effect of attention on repetition suppression and multivoxel pattern similarity. J. Cogn. Neurosci. 25, 1305–1314 (2013).

    Article  PubMed  Google Scholar 

  43. Serences, J.T., Schwarzbach, J., Courtney, S.M., Golay, X. & Yantis, S. Control of object-based attention in human cortex. Cereb. Cortex 14, 1346–1357 (2004).

    Article  PubMed  Google Scholar 

  44. Efron, B. & Tibshirani, R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat. Sci. 1, 54–75 (1986).

    Google Scholar 

  45. Pernet, C.R., Wilcox, R.R. & Rousselet, G.A. Robust correlation analyses: false positive and power validation using a new open source Matlab toolbox. Front. Psychol. 3, 606 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Nichols, T.E. & Holmes, A.P. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum. Brain Mapp. 15, 1–25 (2002).

    Article  PubMed  Google Scholar 

  47. Smith, S.M. & Nichols, T.E. Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference. Neuroimage 44, 83–98 (2009).

    Article  PubMed  Google Scholar 

  48. Donaldson, W. Measuring recognition memory. J. Exp. Psychol. Gen. 121, 275–277 (1992).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by US National Institutes of Health grant R01EY021755, US National Science Foundation (NSF) grant BCS1229597, NSF fellowship DGE1148900 and the John Templeton Foundation. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of these funding agencies.

Author information

Authors and Affiliations

  1. Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA

    Megan T deBettencourt, Jonathan D Cohen, Ray F Lee, Kenneth A Norman & Nicholas B Turk-Browne

  2. Department of Psychology, Princeton University, Princeton, New Jersey, USA

    Jonathan D Cohen, Kenneth A Norman & Nicholas B Turk-Browne

Authors
  1. Megan T deBettencourt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonathan D Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ray F Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kenneth A Norman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicholas B Turk-Browne
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.T.dB., J.D.C., K.A.N. and N.B.T.-B. designed the experiment, discussed the data and wrote the paper. M.T.dB. and R.F.L. developed data acquisition and analysis tools. M.T.dB. collected and analyzed the data. All authors read and commented on the manuscript.

Corresponding author

Correspondence to Nicholas B Turk-Browne.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Study procedure.

(a) Participants completed three sessions on different days, with different numbers of task runs. (b) Each run contained eight blocks of the sustained attention task. The pre-training, post-training, and stable blocks of the rtfMRI training sessions contained composite stimuli with an equal mixture of faces and scenes. In the feedback blocks of the rtfMRI session, the mixture of images was determined by real-time analysis of brain activity. (c) Each block began with a cue (1 s) that indicated the attended category (e.g., scene) and target subcategory (e.g., indoor). This was followed by a brief fixation period (1 s) and then 50 sequential stimuli (1 s each, no interstimulus interval), of which 90% were targets and 10% were lures. The blocks ended with a fixation period (4–6 s).

Supplementary Figure 2 Average RTs surrounding lures.

Green lines correspond to trials around correct rejections (CRs), where there was no behavioral response to the lure (presented at time = 0). Red lines correspond to trials around false alarms (FAs), where participants mistakenly responded to the lure. RTs were significantly slower prior to CRs than FAs (all timepoints, ps < 0.00001), consistent with the idea that FAs occurred when participants started responding habitually and were less attentive to the task. Error bars represent +/–1 s.e.m.

Supplementary Figure 3 Removing the influence of RT.

Both RT and classifier evidence for the task-relevant vs. task-irrelevant category were higher preceding CRs. To verify that the classifier was predictive of behavioral accuracy and not merely correlated with RT, we regressed RT out of classifier output and behavioral accuracy across trials and performed a partial correlation analysis. Green lines correspond to regression fits for feedback participants, and blue lines correspond to regression fits for control participants. The black line is the average fit. The relationship between classifier output and behavioral accuracy remained robust (p < 0.00001).

Supplementary Figure 4 Classifier-to-stimulus transfer function.

The volume-by-volume classifier output for the task-relevant minus task-irrelevant category was mapped to the proportion of the image from the task-relevant category using a sigmoidal function. The inflection point on the classifier axis was set to 0.60, based on the average decoding accuracy in a pilot study. Given the nonlinearities in the function, this helped calibrate the feedback to a more sensitive range of classifier values. Image proportion ranged from 0.17 to 0.98, preventing the task-relevant image from ever disappearing completely, and providing a foothold for recovery from a serious lapse.

Supplementary Figure 5 Histogram of feedback.

The bottom x-axis refers to proportion of the image from the task-relevant category in each composite stimulus. The y-axis refers to the number of TRs that contained stimuli with this mixture (in bins of width = 0.04) across all feedback blocks from all participants in the feedback group. The top x-axis depicts the correspondence between classifier output and feedback (computed using the transfer function in Supplementary Fig. 4). The most frequent values were in the highest and lowest bins, as well as in the bin including feedback of 0.5. The highest bin reflects cases in which the image from the task-relevant category was 98% of the composite stimulus. The lowest bin reflects cases in which the image from the task-irrelevant category was 83% of the composite stimulus. The bin with 0.5 was frequent because every block began with an equal mixture of the images from the task-relevant and task-irrelevant categories.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–5 (PDF 1239 kb)

Supplementary Methods Checklist (PDF 484 kb)

Example neurofeedback block.

This video depicts the real-time data analysis and stimulus-updating procedure during a feedback block. In this block, the participant was instructed to attend to scenes and respond when a scene was indoors. The left window shows what the participant saw. The top-right window shows the real-time fMRI estimate of the participant's attentional state (classifier evidence for task-relevant minus task-irrelevant categories; here, scene minus face outputs). The bottom-right window shows the mixture proportions of the composite stimuli. The mixture was initialized at 50% face/50% scene, and then updated on the basis of a moving window of classifier evidence over the preceding three volumes using a transfer function. (MOV 5100 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

deBettencourt, M., Cohen, J., Lee, R. et al. Closed-loop training of attention with real-time brain imaging. Nat Neurosci 18, 470–475 (2015). https://doi.org/10.1038/nn.3940

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3940

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing