HISTORICAL NEWS & VIEWS: REPRODUCIBILITY

Double-dipping revisited

Robust conclusions require rigorous statistics. In 2009 a seminal paper described the dangers and prevalence of double-dipping in neuroscience. Ten years on, I consider progress toward statistical rigor in neuroimaging.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Number of fMRI articles published per year from 1990 to 2018.

References

  1. 1.

    Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F. & Baker, C. I. Nat. Neurosci. 12, 535–540 (2009).

    CAS  Article  Google Scholar 

  2. 2.

    Ioannidis, J. P. PLoS Med. 2, e124 (2005).

    Article  Google Scholar 

  3. 3.

    Poldrack, R. A. et al. Nat. Rev. Neurosci. 18, 115–126 (2017).

    CAS  Article  Google Scholar 

  4. 4.

    Vul, E., Harris, C., Winkielman, P. & Pashler, H. Perspect. Psychol. Sci. 4, 274–290 (2009).

    Article  Google Scholar 

  5. 5.

    Bennett, C. M., Miller, M. B. & Wolford, G. L. Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: an argument for multiple comparisons correction. NeuroImage 47, S125 (2009).

    Article  Google Scholar 

  6. 6.

    Button, K. S. et al. Nat. Rev. Neurosci. 14, 365–376 (2013).

    CAS  Article  Google Scholar 

  7. 7.

    Thompson, P. M. et al. Alzheimer’s Disease Neuroimaging Initiative, EPIGEN Consortium, IMAGEN Consortium, Saguenay Youth Study (SYS) Group. Brain Imaging Behav. 8, 153–182 (2014).

    PubMed  PubMed Central  Google Scholar 

  8. 8.

    Biswal, B. B. et al. Proc. Natl. Acad. Sci. USA 107, 4734–4739 (2010).

    CAS  Article  Google Scholar 

  9. 9.

    Nichols, T. E. et al. Nat. Neurosci. 20, 299–303 (2017).

    CAS  Article  Google Scholar 

  10. 10.

    Dickersin, K. & Rennie, D. J. Am. Med. Assoc. 290, 516–523 (2003).

    Article  Google Scholar 

  11. 11.

    Munafò, M. R. et al. Nat. Hum. Behav. 1, 0021 (2017).

    Article  Google Scholar 

  12. 12.

    Carp, J. Neuroimage 63, 289–300 (2012).

    Article  Google Scholar 

  13. 13.

    Guo, Q. et al. PLoS One 9, e94412 (2014).

    Article  Google Scholar 

  14. 14.

    Carp, J. Front. Neurosci. 6, 149 (2012).

    Article  Google Scholar 

  15. 15.

    Ioannidis, J. P., Tarone, R. & McLaughlin, J. K. Epidemiology 22, 450–456 (2011).

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Katherine S. Button.

Ethics declarations

Competing interests

The author declares no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Button, K.S. Double-dipping revisited. Nat Neurosci 22, 688–690 (2019). https://doi.org/10.1038/s41593-019-0398-z

Download citation

Further reading

  • Exercise alters cerebellar and cortical activity related to working memory in phenotypes of Gulf War Illness

    • Stuart D Washington
    • , Rakib U Rayhan
    • , Richard Garner
    • , Destie Provenzano
    • , Kristina Zajur
    • , Florencia Martinez Addiego
    • , John W VanMeter
    •  & James N Baraniuk

    Brain Communications (2020)

  • Dataset decay and the problem of sequential analyses on open datasets

    • William Hedley Thompson
    • , Jessey Wright
    • , Patrick G Bissett
    •  & Russell A Poldrack

    eLife (2020)

  • ENIGMA and global neuroscience: A decade of large-scale studies of the brain in health and disease across more than 40 countries

    • Paul M. Thompson
    • , Neda Jahanshad
    • , Christopher R. K. Ching
    • , Lauren E. Salminen
    • , Sophia I. Thomopoulos
    • , Joanna Bright
    • , Bernhard T. Baune
    • , Sara Bertolín
    • , Janita Bralten
    • , Willem B. Bruin
    • , Robin Bülow
    • , Jian Chen
    • , Yann Chye
    • , Udo Dannlowski
    • , Carolien G. F. de Kovel
    • , Gary Donohoe
    • , Lisa T. Eyler
    • , Stephen V. Faraone
    • , Pauline Favre
    • , Courtney A. Filippi
    • , Thomas Frodl
    • , Daniel Garijo
    • , Yolanda Gil
    • , Hans J. Grabe
    • , Katrina L. Grasby
    • , Tomas Hajek
    • , Laura K. M. Han
    • , Sean N. Hatton
    • , Kevin Hilbert
    • , Tiffany C. Ho
    • , Laurena Holleran
    • , Georg Homuth
    • , Norbert Hosten
    • , Josselin Houenou
    • , Iliyan Ivanov
    • , Tianye Jia
    • , Sinead Kelly
    • , Marieke Klein
    • , Jun Soo Kwon
    • , Max A. Laansma
    • , Jeanne Leerssen
    • , Ulrike Lueken
    • , Abraham Nunes
    • , Joseph O’ Neill
    • , Nils Opel
    • , Fabrizio Piras
    • , Federica Piras
    • , Merel C. Postema
    • , Elena Pozzi
    • , Natalia Shatokhina
    • , Carles Soriano-Mas
    • , Gianfranco Spalletta
    • , Daqiang Sun
    • , Alexander Teumer
    • , Amanda K. Tilot
    • , Leonardo Tozzi
    • , Celia van der Merwe
    • , Eus J. W. Van Someren
    • , Guido A. van Wingen
    • , Henry Völzke
    • , Esther Walton
    • , Lei Wang
    • , Anderson M. Winkler
    • , Katharina Wittfeld
    • , Margaret J. Wright
    • , Je-Yeon Yun
    • , Guohao Zhang
    • , Yanli Zhang-James
    • , Bhim M. Adhikari
    • , Ingrid Agartz
    • , Moji Aghajani
    • , André Aleman
    • , Robert R. Althoff
    • , Andre Altmann
    • , Ole A. Andreassen
    • , David A. Baron
    • , Brenda L. Bartnik-Olson
    • , Janna Marie Bas-Hoogendam
    • , Arielle R. Baskin-Sommers
    • , Carrie E. Bearden
    • , Laura A. Berner
    • , Premika S. W. Boedhoe
    • , Rachel M. Brouwer
    • , Jan K. Buitelaar
    • , Karen Caeyenberghs
    • , Charlotte A. M. Cecil
    • , Ronald A. Cohen
    • , James H. Cole
    • , Patricia J. Conrod
    • , Stephane A. De Brito
    • , Sonja M. C. de Zwarte
    • , Emily L. Dennis
    • , Sylvane Desrivieres
    • , Danai Dima
    • , Stefan Ehrlich
    • , Carrie Esopenko
    • , Graeme Fairchild
    • , Simon E. Fisher
    • , Jean-Paul Fouche
    • , Clyde Francks
    • , Sophia Frangou
    • , Barbara Franke
    • , Hugh P. Garavan
    • , David C. Glahn
    • , Nynke A. Groenewold
    • , Tiril P. Gurholt
    • , Boris A. Gutman
    • , Tim Hahn
    • , Ian H. Harding
    • , Dennis Hernaus
    • , Derrek P. Hibar
    • , Frank G. Hillary
    • , Martine Hoogman
    • , Hilleke E. Hulshoff Pol
    • , Maria Jalbrzikowski
    • , George A. Karkashadze
    • , Eduard T. Klapwijk
    • , Rebecca C. Knickmeyer
    • , Peter Kochunov
    • , Inga K. Koerte
    • , Xiang-Zhen Kong
    • , Sook-Lei Liew
    • , Alexander P. Lin
    • , Mark W. Logue
    • , Eileen Luders
    • , Fabio Macciardi
    • , Scott Mackey
    • , Andrew R. Mayer
    • , Carrie R. McDonald
    • , Agnes B. McMahon
    • , Sarah E. Medland
    • , Gemma Modinos
    • , Rajendra A. Morey
    • , Sven C. Mueller
    • , Pratik Mukherjee
    • , Leyla Namazova-Baranova
    • , Talia M. Nir
    • , Alexander Olsen
    • , Peristera Paschou
    • , Daniel S. Pine
    • , Fabrizio Pizzagalli
    • , Miguel E. Rentería
    • , Jonathan D. Rohrer
    • , Philipp G. Sämann
    • , Lianne Schmaal
    • , Gunter Schumann
    • , Mark S. Shiroishi
    • , Sanjay M. Sisodiya
    • , Dirk J. A. Smit
    • , Ida E. Sønderby
    • , Dan J. Stein
    • , Jason L. Stein
    • , Masoud Tahmasian
    • , David F. Tate
    • , Jessica A. Turner
    • , Odile A. van den Heuvel
    • , Nic J. A. van der Wee
    • , Ysbrand D. van der Werf
    • , Theo G. M. van Erp
    • , Neeltje E. M. van Haren
    • , Daan van Rooij
    • , Laura S. van Velzen
    • , Ilya M. Veer
    • , Dick J. Veltman
    • , Julio E. Villalon-Reina
    • , Henrik Walter
    • , Christopher D. Whelan
    • , Elisabeth A. Wilde
    • , Mojtaba Zarei
    •  & Vladimir Zelman

    Translational Psychiatry (2020)