Abstract
Lapses of attention can have negative consequences, including accidents and lost productivity. Here we used closed-loop neurofeedback to improve sustained attention abilities and reduce the frequency of lapses. During a sustained attention task, the focus of attention was monitored in real time with multivariate pattern analysis of whole-brain neuroimaging data. When indicators of an attentional lapse were detected in the brain, we gave human participants feedback by making the task more difficult. Behavioral performance improved after one training session, relative to control participants who received feedback from other participants' brains. This improvement was largest when feedback carried information from a frontoparietal attention network. A neural consequence of training was that the basal ganglia and ventral temporal cortex came to represent attentional states more distinctively. These findings suggest that attentional failures do not reflect an upper limit on cognitive potential and that attention can be trained with appropriate feedback about neural signals.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Chun, M.M., Golomb, J.D. & Turk-Browne, N.B. A taxonomy of external and internal attention. Annu. Rev. Psychol. 62, 73–101 (2011).
Mackworth, N.H. The breakdown of vigilance during prolonged visual search. Q. J. Exp. Psychol. 1, 6–21 (1948).
Redelmeier, D.A. & Tibshirani, R.J. Association between cellular-telephone calls and motor vehicle collisions. N. Engl. J. Med. 336, 453–458 (1997).
Czeisler, C.A. et al. Modafinil for excessive sleepiness associated with shift-work sleep disorder. N. Engl. J. Med. 353, 476–486 (2005).
Dinges, D.F. & Powell, J.W. Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations. Behav. Res. Methods Instrum. Comput. 17, 652–655 (1985).
Sarter, M., Givens, B. & Bruno, J.P. The cognitive neuroscience of sustained attention: where top-down meets bottom-up. Brain Res. Brain Res. Rev. 35, 146–160 (2001).
Wolfe, J.M., Horowitz, T.S. & Kenner, N.M. Rare items often missed in visual searches. Nature 435, 439–440 (2005).
Johnson, K.A. et al. Dissociation in performance of children with ADHD and high-functioning autism on a task of sustained attention. Neuropsychologia 45, 2234–2245 (2007).
Robertson, I.H., Manly, T., Andrade, J., Baddeley, B.T. & Yiend, J. 'Oops!': performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia 35, 747–758 (1997).
Drew, T. & Vogel, E.K. Neural measures of individual differences in selecting and tracking multiple moving objects. J. Neurosci. 28, 4183–4191 (2008).
Weiskopf, N. et al. Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Trans. Biomed. Eng. 51, 966–970 (2004).
LaConte, S.M. Decoding fMRI brain states in real-time. Neuroimage 56, 440–454 (2011).
Sulzer, J. et al. Real-time fMRI neurofeedback: progress and challenges. Neuroimage 76, 386–399 (2013).
Norman, K.A., Polyn, S.M., Detre, G.J. & Haxby, J.V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430 (2006).
deCharms, R.C. et al. Control over brain activation and pain learned by using real-time functional MRI. Proc. Natl. Acad. Sci. USA 102, 18626–18631 (2005).
Shibata, K., Watanabe, T., Sasaki, Y. & Kawato, M. Perceptual learning incepted by decoded fMRI neurofeedback without stimulus presentation. Science 334, 1413–1415 (2011).
Yoo, J.J. et al. When the brain is prepared to learn: enhancing human learning using real-time fMRI. Neuroimage 59, 846–852 (2012).
Hinds, O. et al. Roles of default-mode network and supplementary motor area in human vigilance performance: evidence from real-time fMRI. J. Neurophysiol. 109, 1250–1258 (2013).
Yoss, R.E., Moyer, N.J. & Hollenhorst, R.W. Pupil size and spontaneous pupillary waves associated with alertness, drowsiness, and sleep. Neurology 20, 545–554 (1970).
Rosenberg, M., Noonan, S., DeGutis, J. & Esterman, M. Sustaining visual attention in the face of distraction: a novel gradual-onset continuous performance task. Atten. Percept. Psychophys. 75, 426–439 (2013).
O'Craven, K.M., Downing, P.E. & Kanwisher, N. fMRI evidence for objects as the units of attentional selection. Nature 401, 584–587 (1999).
Al-Aidroos, N., Said, C.P. & Turk-Browne, N.B. Top-down attention switches coupling between low-level and high-level areas of human visual cortex. Proc. Natl. Acad. Sci. USA 109, 14675–14680 (2012).
Weissman, D.H., Roberts, K.C., Visscher, K.M. & Woldorff, M.G. The neural bases of momentary lapses in attention. Nat. Neurosci. 9, 971–978 (2006).
Leber, A.B., Turk-Browne, N.B. & Chun, M.M. Neural predictors of moment-to-moment fluctuations in cognitive flexibility. Proc. Natl. Acad. Sci. USA 105, 13592–13597 (2008).
Miller, E.K. & Cohen, J.D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).
Noudoost, B., Chang, M.H., Steinmetz, N.A. & Moore, T. Top-down control of visual attention. Curr. Opin. Neurobiol. 20, 183–190 (2010).
Todd, M.T., Nystrom, L.E. & Cohen, J.D. Confounds in multivariate pattern analysis: theory and rule representation case study. Neuroimage 77, 157–165 (2013).
Haxby, J.V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).
Woolgar, A., Hampshire, A., Thompson, R. & Duncan, J. Adaptive coding of task-relevant information in human frontoparietal cortex. J. Neurosci. 31, 14592–14599 (2011).
Turk-Browne, N.B. Functional interactions as big data in the human brain. Science 342, 580–584 (2013).
Reddy, L., Kanwisher, N.G. & VanRullen, R. Attention and biased competition in multi-voxel object representations. Proc. Natl. Acad. Sci. USA 106, 21447–21452 (2009).
Baldauf, D. & Desimone, R. Neural mechanisms of object-based attention. Science 344, 424–427 (2014).
O'Reilly, R.C. & Frank, M. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283–328 (2006).
Kravitz, A.V. et al. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622–626 (2010).
Wiecki, T.V. & Frank, M.J. A computational model of inhibitory control in frontal cortex and basal ganglia. Psychol. Rev. 120, 329–355 (2013).
Foerde, K. & Shohamy, D. The role of the basal ganglia in learning and memory: insight from Parkinson's disease. Neurobiol. Learn. Mem. 96, 624–636 (2011).
Frank, M.J. & Badre, D. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cereb. Cortex 22, 509–526 (2012).
Biggs, A.T. & Mitroff, S.R. Different predictors of multiple-target search accuracy between nonprofessional and professional visual searchers. Q. J. Exp. Psychol. (Hove) 67, 1335–1348 (2014).
Disner, S.G., Beevers, C.G., Haigh, E.A.P. & Beck, A.T. Neural mechanisms of the cognitive model of depression. Nat. Rev. Neurosci. 12, 467–477 (2011).
Shallice, T. et al. Executive function profile of children with attention deficit hyperactivity disorder. Dev. Neuropsychol. 21, 43–71 (2002).
Chadick, J.Z. & Gazzaley, A. Differential coupling of visual cortex with default network or frontal-parietal network based on goals. Nat. Neurosci. 14, 830–832 (2011).
Moore, K.S., Yi, D.-J. & Chun, M. The effect of attention on repetition suppression and multivoxel pattern similarity. J. Cogn. Neurosci. 25, 1305–1314 (2013).
Serences, J.T., Schwarzbach, J., Courtney, S.M., Golay, X. & Yantis, S. Control of object-based attention in human cortex. Cereb. Cortex 14, 1346–1357 (2004).
Efron, B. & Tibshirani, R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat. Sci. 1, 54–75 (1986).
Pernet, C.R., Wilcox, R.R. & Rousselet, G.A. Robust correlation analyses: false positive and power validation using a new open source Matlab toolbox. Front. Psychol. 3, 606 (2013).
Nichols, T.E. & Holmes, A.P. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum. Brain Mapp. 15, 1–25 (2002).
Smith, S.M. & Nichols, T.E. Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference. Neuroimage 44, 83–98 (2009).
Donaldson, W. Measuring recognition memory. J. Exp. Psychol. Gen. 121, 275–277 (1992).
Acknowledgements
This work was supported by US National Institutes of Health grant R01EY021755, US National Science Foundation (NSF) grant BCS1229597, NSF fellowship DGE1148900 and the John Templeton Foundation. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of these funding agencies.
Author information
Authors and Affiliations
Contributions
M.T.dB., J.D.C., K.A.N. and N.B.T.-B. designed the experiment, discussed the data and wrote the paper. M.T.dB. and R.F.L. developed data acquisition and analysis tools. M.T.dB. collected and analyzed the data. All authors read and commented on the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Study procedure.
(a) Participants completed three sessions on different days, with different numbers of task runs. (b) Each run contained eight blocks of the sustained attention task. The pre-training, post-training, and stable blocks of the rtfMRI training sessions contained composite stimuli with an equal mixture of faces and scenes. In the feedback blocks of the rtfMRI session, the mixture of images was determined by real-time analysis of brain activity. (c) Each block began with a cue (1 s) that indicated the attended category (e.g., scene) and target subcategory (e.g., indoor). This was followed by a brief fixation period (1 s) and then 50 sequential stimuli (1 s each, no interstimulus interval), of which 90% were targets and 10% were lures. The blocks ended with a fixation period (4–6 s).
Supplementary Figure 2 Average RTs surrounding lures.
Green lines correspond to trials around correct rejections (CRs), where there was no behavioral response to the lure (presented at time = 0). Red lines correspond to trials around false alarms (FAs), where participants mistakenly responded to the lure. RTs were significantly slower prior to CRs than FAs (all timepoints, ps < 0.00001), consistent with the idea that FAs occurred when participants started responding habitually and were less attentive to the task. Error bars represent +/–1 s.e.m.
Supplementary Figure 3 Removing the influence of RT.
Both RT and classifier evidence for the task-relevant vs. task-irrelevant category were higher preceding CRs. To verify that the classifier was predictive of behavioral accuracy and not merely correlated with RT, we regressed RT out of classifier output and behavioral accuracy across trials and performed a partial correlation analysis. Green lines correspond to regression fits for feedback participants, and blue lines correspond to regression fits for control participants. The black line is the average fit. The relationship between classifier output and behavioral accuracy remained robust (p < 0.00001).
Supplementary Figure 4 Classifier-to-stimulus transfer function.
The volume-by-volume classifier output for the task-relevant minus task-irrelevant category was mapped to the proportion of the image from the task-relevant category using a sigmoidal function. The inflection point on the classifier axis was set to 0.60, based on the average decoding accuracy in a pilot study. Given the nonlinearities in the function, this helped calibrate the feedback to a more sensitive range of classifier values. Image proportion ranged from 0.17 to 0.98, preventing the task-relevant image from ever disappearing completely, and providing a foothold for recovery from a serious lapse.
Supplementary Figure 5 Histogram of feedback.
The bottom x-axis refers to proportion of the image from the task-relevant category in each composite stimulus. The y-axis refers to the number of TRs that contained stimuli with this mixture (in bins of width = 0.04) across all feedback blocks from all participants in the feedback group. The top x-axis depicts the correspondence between classifier output and feedback (computed using the transfer function in Supplementary Fig. 4). The most frequent values were in the highest and lowest bins, as well as in the bin including feedback of 0.5. The highest bin reflects cases in which the image from the task-relevant category was 98% of the composite stimulus. The lowest bin reflects cases in which the image from the task-irrelevant category was 83% of the composite stimulus. The bin with 0.5 was frequent because every block began with an equal mixture of the images from the task-relevant and task-irrelevant categories.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–5 (PDF 1239 kb)
Example neurofeedback block.
This video depicts the real-time data analysis and stimulus-updating procedure during a feedback block. In this block, the participant was instructed to attend to scenes and respond when a scene was indoors. The left window shows what the participant saw. The top-right window shows the real-time fMRI estimate of the participant's attentional state (classifier evidence for task-relevant minus task-irrelevant categories; here, scene minus face outputs). The bottom-right window shows the mixture proportions of the composite stimuli. The mixture was initialized at 50% face/50% scene, and then updated on the basis of a moving window of classifier evidence over the preceding three volumes using a transfer function. (MOV 5100 kb)
Rights and permissions
About this article
Cite this article
deBettencourt, M., Cohen, J., Lee, R. et al. Closed-loop training of attention with real-time brain imaging. Nat Neurosci 18, 470–475 (2015). https://doi.org/10.1038/nn.3940
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.3940
This article is cited by
-
Performance-linked visual feedback slows response times during a sustained attention task
Cognitive Research: Principles and Implications (2023)
-
Optimising the classification of feature-based attention in frequency-tagged electroencephalography data
Scientific Data (2022)
-
Basal ganglia-cortical connectivity underlies self-regulation of brain oscillations in humans
Communications Biology (2022)
-
Voluntary control of semantic neural representations by imagery with conflicting visual stimulation
Communications Biology (2022)
-
Inducing a mental context for associative memory formation with real-time fMRI neurofeedback
Scientific Reports (2022)