Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Harnessing reliability for neuroscience research

Neuroscientists are amassing the large-scale datasets needed to study individual differences and identify biomarkers. However, measurement reliability within individual samples is often suboptimal, thereby requiring unnecessarily large samples. We focus our comment on reliability in neuroimaging and provide examples of how the reliability can be increased.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type



Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Interactions between reliability, sample and effect size for a given statistical power.
Fig. 2: Reliability and validity of measurements of individual differences.

Code availability

All code employed in this effort can be found on GitHub at


  1. Laumann, T. O. et al. Neuron 87, 657–670 (2015).

    Article  CAS  Google Scholar 

  2. O’Connor, D. et al. Gigascience 6, 1–14 (2017).

    Article  Google Scholar 

  3. Xu, T., Opitz, A., Craddock, C., Zuo, X. N. & Milham, M. P. Cereb. Cortex 26, 4192–4211 (2016).

    Article  Google Scholar 

  4. Zuo, X. N. & Xing, X. X. Neurosci. Biobehav. Rev. 45, 100–118 (2014).

    Article  Google Scholar 

  5. Kanyongo, G. Y., Brook, G. P., Kyei-Blankson, L. & Gocmen, G. J. Mod. Appl. Stat. Methods 6, Article 9 (2007).

    Article  Google Scholar 

  6. Poldrack, R. A. et al. Nat. Rev. Neurosci. 18, 115–126 (2017).

    Article  CAS  Google Scholar 

  7. Bennett, C. M. & Miller, M. B. Ann. NY Acad. Sci. 1191, 133–155 (2010).

    Article  Google Scholar 

  8. Hedge, C., Powell, G. & Sumner, P. Behav. Res. Methods 50, 1166–1186 (2018).

    Article  Google Scholar 

  9. Button, K. S. et al. Nat. Rev. Neurosci. 14, 365–376 (2013).

    Article  CAS  Google Scholar 

  10. Zuo, X.N., Biswal, B.B. & Poldrack, R.A. Front. Neurosci. 13, 117 (2019).

    Article  Google Scholar 

  11. Tomasi, D. G., Shokri-Kojori, E. & Volkow, N. D. Cereb. Cortex 27, 4153–4165 (2017).

    PubMed  Google Scholar 

  12. Elliott, M. L. et al. Neuroimage 189, 516–532 (2019).

    Article  Google Scholar 

  13. Nichols, T. E. et al. Nat. Neurosci. 20, 299–303 (2017).

    Article  CAS  Google Scholar 

  14. Koo, T. K. & Li, M. Y. J. Chiropr. Med. 15, 155–163 (2016).

    Article  Google Scholar 

  15. Kraemer, H. C. Annu. Rev. Clin. Psychol. 10, 111–130 (2014).

    Article  Google Scholar 

  16. Yan, C. G. et al. Neuroimage 76, 183–201 (2013).

    Article  Google Scholar 

Download references


We thank X. Castellanos, A. Franco, H. Kimball, A. Nikolaidis and X.-X. Xing for their helpful comments in the preparation of this commentary, as well as D. Klein for his guidance and encouragement of our focus on issues of reliability over the years, X. Castellanos for his support along the way, and all the contributors from CoRR and R3BRAIN for their enthusiasm on open neuroscience and data sharing. The two consortia are supported in part by the National Basic Research (973) Program (2015CB351702).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Xi-Nian Zuo.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zuo, XN., Xu, T. & Milham, M.P. Harnessing reliability for neuroscience research. Nat Hum Behav 3, 768–771 (2019).

Download citation

  • Published:

  • Issue Date:

  • DOI:

This article is cited by


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing