Abstract
Neuroimaging plays a crucial role in understanding brain structure and function, but the lack of transparency, reproducibility, and reliability of findings is a significant obstacle for the field. To address these challenges, there are ongoing efforts to develop reporting checklists for neuroimaging studies to improve the reporting of fundamental aspects of study design and execution. In this review, we first define what we mean by a neuroimaging reporting checklist and then discuss how a reporting checklist can be developed and implemented. We consider the core values that should inform checklist design, including transparency, repeatability, data sharing, diversity, and supporting innovations. We then share experiences with currently available neuroimaging checklists. We review the motivation for creating checklists and whether checklists achieve their intended objectives, before proposing a development cycle for neuroimaging reporting checklists and describing each implementation step. We emphasize the importance of reporting checklists in enhancing the quality of data repositories and consortia, how they can support education and best practices, and how emerging computational methods, like artificial intelligence, can help checklist development and adherence. We also highlight the role that funding agencies and global collaborations can play in supporting the adoption of neuroimaging reporting checklists. We hope this review will encourage better adherence to available checklists and promote the development of new ones, and ultimately increase the quality, transparency, and reproducibility of neuroimaging research.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 13 print issues and online access
$259.00 per year
only $19.92 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Carp J. The secret lives of experiments: methods reporting in the fMRI literature. Neuroimage. 2012;63:289–300.
Gorgolewski KJ, Poldrack RA. A practical guide for improving transparency and reproducibility in neuroimaging research. PLOS Biol. 2016;14:e1002506.
Poldrack RA, Baker CI, Durnez J, Gorgolewski KJ, Matthews PM, Munafò MR, et al. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat Rev Neurosci. 2017;18:115–26.
Fusar-Poli P, Radua J, Frascarelli M, Mechelli A, Borgwardt S, Di Fabio F, et al. Evidence of reporting biases in voxel-based morphometry (VBM) studies of psychiatric and neurological disorders: reporting biases in VBM Studies of Psychiatric and Neurological Disorders. Hum Brain Mapp. 2014;35:3052–65.
David SP, Naudet F, Laude J, Radua J, Fusar-Poli P, Chu I, et al. Potential reporting bias in neuroimaging studies of sex differences. Sci Rep. 2018;8:6082.
Etkin A. A reckoning and research agenda for neuroimaging in psychiatry. AJP. 2019;176:507–11.
Robbins KA, Touryan J, Mullen T, Kothe C, Bigdely-Shamlo N. How sensitive are EEG results to preprocessing methods: a benchmarking study. IEEE Trans Neural Syst Rehabil Eng. 2020;28:1081–90.
Gentili C, Cecchetti L, Handjaras G, Lettieri G, Cristea IA. The case for preregistering all region of interest (ROI) analyses in neuroimaging research. Eur J Neurosci. 2021;53:357–61.
Pernet C, Garrido MI, Gramfort A, Maurits N, Michel CM, Pang E, et al. Issues and recommendations from the OHBM COBIDAS MEEG committee for reproducible EEG and MEG research. Nat Neurosci. 2020;23:1473–83.
Carp J. Better living through transparency: improving the reproducibility of fMRI results through comprehensive methods reporting. Cogn Affect Behav Neurosci. 2013;13:660–6.
Klapwijk ET, van den Bos W, Tamnes CK, Raschle NM, Mills KL. Opportunities for increased reproducibility and replicability of developmental neuroimaging. Dev Cogn Neurosci. 2021;47:100902.
Hupalo S, Jordan CJ, Bowen T, Mahar J, Yepez E, Kunath L, et al. NPP’s approach toward improving rigor and transparency in clinical trials research. Neuropsychopharmacology. 2023;48:429–31.
Nichols TE, Das S, Eickhoff SB, Evans AC, Glatard T, Hanke M, et al. Best practices in data analysis and sharing in neuroimaging using MRI. Nat Neurosci. 2017;20:299–303.
Bossuyt, Reitsma PM, Bruns DE JB, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Clin Chem. 2015;61:1446–52.
Ekhtiari H, Zare-Bidoky M, Sangchooli A, Janes AC, Kaufman MJ, Oliver JA, et al. A methodological checklist for fMRI drug cue reactivity studies: development and expert consensus. Nat Protoc. 2022;17:567–95.
Ekhtiari H, Ghobadi-Azbari P, Thielscher A, Antal A, Li LM, Shereen AD, et al. A checklist for assessing the methodological quality of concurrent tES-fMRI studies (ContES checklist): a consensus study and statement. Nat Protoc. 2022;17:596–617.
Kousta S, Pastrana E, Swaminathan S. Three approaches to support reproducible research. Sci Editor. 2020;42:77–82.
The NPQIP Collaborative group, Study steering committee, Macleod M, Sena E, Howells D, Macleod M, et al. Did a change in Nature Journals’ editorial policy for life sciences research improve reporting? BMJ Open Sci [Internet]. 2019 Feb [cited 2024 Mar 27]; 3. Available from: http://access.portico.org/stable?au=phzq8gmxdp1.
Feng X, Park DS, Walker C, Peterson AT, Merow C, Papeş M. A checklist for maximizing reproducibility of ecological niche models. Nat Ecol Evol. 2019;3:1382–95.
de Jong Y, van der Willik EM, Milders J, Voorend CGN, Morton RL, Dekker FW, et al. A meta-review demonstrates improved reporting quality of qualitative reviews following the publication of COREQ- and ENTREQ-checklists, regardless of modest uptake. BMC Med Res Methodol. 2021;21:184.
Uddin MN, Figley TD, Kornelsen J, Mazerolle EL, Helmick CA, O’Grady CB, et al. The comorbidity and cognition in multiple sclerosis (CCOMS) neuroimaging protocol: Study rationale, MRI acquisition, and minimal image processing pipelines. Front Neuroimaging [Internet]. 2022 Aug [cited 2024 Mar 10];1. Available from: https://www.frontiersin.org/articles/10.3389/fnimg.2022.970385.
Appelbaum M, Cooper H, Kline RB, Mayo-Wilson E, Nezu AM, Rao SM. Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. Am Psychol. 2018;73:3–25.
Köhler T, González-Morales MG, Banks GC, O’Boyle EH, Allen JA, Sinha R, et al. Supporting robust, rigorous, and reliable reviewing as the cornerstone of our profession: introducing a competency framework for peer review. Ind Organ Psychol. 2020;13:1–27.
Higgins JPT, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
Gorgolewski KJ, Auer T, Calhoun VD, Craddock RC, Das S, Duff EP, et al. The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Sci Data. 2016;3:160044.
Rid A, Schmidt H. The 2008 Declaration of Helsinki - first among equals in research ethics? J Law Med Ethics. 2010;38:143–8.
Buch ER, Santarnecchi E, Antal A, Born J, Celnik PA, Classen J, et al. Effects of tDCS on motor learning and memory formation: a consensus and critical position paper. Clin Neurophysiol. 2017;128:589–603.
Choi I, Kreis R. Advanced methodology for in vivo magnetic resonance spectroscopy. NMR Biomed. 2021;34:e4504.
Marek S, Tervo-Clemmens B, Calabro FJ, Montez DF, Kay BP, Hatoum AS, et al. Reproducible brain-wide association studies require thousands of individuals. Nature. 2022;603:654–60.
Elyounssi S, Kunitoki K, Clauss JA, Laurent E, Kane K, Hughes DE, et al. Uncovering and mitigating bias in large, automated MRI analyses of brain development. bioRxiv. 2023 Jan;2023.02.28.530498.
Allen K, Geimer JL, Popp E. Context matters: developing peer reviewers to advance science and practice. Ind Organ Psychol. 2020;13:57–60.
Nieminen P. Ten points for high-quality statistical reporting and data presentation. Appl Sci. 2020;10:3885.
Eby LT, Shockley KM, Bauer TN, Edwards B, Homan AC, Johnson R, et al. Methodological checklists for improving research quality and reporting consistency. Ind Organ Psychol. 2020;13:76–83.
Garcia-Costa D, Squazzoni F, Mehmani B, Grimaldo F. Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals. PeerJ. 2022;10:e13539.
ALBA Network. Alba Network. [cited 2024 Mar 22]. ALBA Declaration on Equity and Inclusion. 2024 Available from: https://www.alba.network/declaration.
Tzovara A, Amarreh I, Borghesani V, Chakravarty MM, DuPre E, Grefkes C, et al. Embracing diversity and inclusivity in an academic setting: Insights from the Organization for Human Brain Mapping. NeuroImage. 2021;229:117742.
Silver JK. Is a lack of diversity among clinical practice guideline authors contributing to health inequalities for patients? BMJ. 2023;381:p1035.
Synnot A, Hill S, Jauré A, Merner B, Hill K, Bates P, et al. Broadening the diversity of consumers engaged in guidelines: a scoping review. BMJ Open. 2022;12:e058326.
Poldrack RA, Fletcher PC, Henson RN, Worsley KJ, Brett M, Nichols TE. Guidelines for reporting an fMRI study. Neuroimage. 2008;40:409–14.
Nichols TE, Das S, Eickhoff SB, Evans AC, Glatard T, Hanke M, et al. Best practices in data analysis and sharing in neuroimaging using MRI [Internet]. Neuroscience; 2016 May [cited 2024 Mar 18]. Available from: http://biorxiv.org/lookup/doi/10.1101/054262.
Pernet CR, Garrido M, Gramfort A, Maurits N, Michel C, Pang E, et al. Best practices in data analysis and sharing in neuroimaging using MEEG [Internet]. Open Science Framework; 2018 Aug [cited 2024 Mar 21]. Available from: https://osf.io/a8dhx.
Uddin LQ, Betzel RF, Cohen JR, Damoiseaux JS, De Brigard F, Eickhoff S, et al. Controversies and progress on standardization of large-scale brain network nomenclature [Internet]. Open Science Framework; 2022 Mar [cited 2024 Mar 21]. Available from: https://osf.io/25za6.
Voets N et al. COBIDAS Clinical fMRI for language mapping. [cited 2024 Mar 21]. COBIDAS Clinical fMRI for language mapping. 2023 Available from: https://cobidasclinicalfmriforlanguagemapping.wordpress.com/.
Oz G, Alger JR, Barker PB, Bartha R, Bizzi A, Boesch C, et al. Clinical proton MR spectroscopy in central nervous system disorders. Radiology. 2014;270:658–79.
Wilson M, Andronesi O, Barker PB, Bartha R, Bizzi A, Bolan PJ, et al. Methodological consensus on clinical proton MRS of the brain: review and recommendations. Magn Reson Med. 2019;82:527–50.
Peek AL, Rebbeck T, Puts NAJ, Watson J, Aguila MER, Leaver AM. Brain GABA and glutamate levels across pain conditions: a systematic literature review and meta-analysis of 1H-MRS studies using the MRS-Q quality assessment tool. NeuroImage. 2020;210:116532.
Öngür D. Making progress with magnetic resonance spectroscopy. JAMA Psychiatry. 2013;70:1265.
Lin A, Andronesi O, Bogner W, Choi I, Coello E, Cudalbu C, et al. Minimum Reporting Standards for in vivo Magnetic Resonance Spectroscopy (MRSinMRS): experts’ consensus recommendations. NMR Biomed. 2021;34:e4484.
Addiction Cue-Reactivity Initiative (ACRI) Network. Parameter Space and Potential for Biomarker Development in 25 Years of fMRI Drug Cue Reactivity: A Systematic Review. JAMA Psychiatry [Internet]. 2024 Feb [cited 2024 Feb 13]; Available from: https://doi.org/10.1001/jamapsychiatry.2023.5483.
Knudsen GM, Ganz M, Appelhoff S, Boellaard R, Bormans G, Carson RE, et al. Guidelines for the content and format of PET brain data in publications and archives: a consensus paper. J Cereb Blood Flow Metab. 2020;40:1576–85.
Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7:e1000217.
Waggoner J, Carline JD, Durning SJ. Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad Med. 2016;91:663–8.
Gratton C, Nelson SM, Gordon EM. Brain-behavior correlations: two paths toward reliability. Neuron. 2022;110:1446–9.
Kragel PA, Han X, Kraynak TE, Gianaros PJ, Wager TD. Functional MRI can be highly reliable, but it depends on what you measure: a commentary on Elliott et al. (2020). Psychol Sci. 2021;32:622–6.
Banks GC, Rogelberg SG, Woznyj HM, Landis RS, Rupp DE. Editorial: evidence on questionable research practices: the good, the bad, and the ugly. J Bus Psychol. 2016;31:323–38.
Ganz M, Poldrack RA. Data sharing in neuroimaging: experiences from the BIDS project. Nat Rev Neurosci. 2023;24:729–30.
Li X, Guo N, Li Q. Functional neuroimaging in the new era of big data. Genomics Proteom Bioinforma. 2019;17:393–401.
Webb-Vargas Y, Chen S, Fisher A, Mejia A, Xu Y, Crainiceanu C, et al. Big data and neuroimaging. Stat Biosci. 2017;9:543–58.
Sudlow C, Gallacher J, Allen N, Beral V, Burton P, Danesh J, et al. UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 2015;12:e1001779.
Casey BJ, Cannonier T, Conley MI, Cohen AO, Barch DM, Heitzeg MM, et al. The Adolescent Brain Cognitive Development (ABCD) study: Imaging acquisition across 21 sites. Dev Cogn Neurosci. 2018;32:43–54.
Markiewicz CJ, Gorgolewski KJ, Feingold F, Blair R, Halchenko YO, Miller E, et al. OpenNeuro: An open resource for sharing of neuroimaging data. bioRxiv. 2021.
Wilkinson MD, Dumontier M, Aalbersberg IJJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016;3:160018.
Saunders JB, Aasland OG, Babor TF, De La Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption-II. Addiction. 1993;88:791–804.
Grasby KL, Jahanshad N, Painter JN, Colodro-Conde L, Bralten J, Hibar DP, et al. The genetic architecture of the human cerebral cortex. Science. 2020;367:eaay6690.
Thompson PM, Jahanshad N, Ching CRK, Salminen LE, Thomopoulos SI, Bright J, et al. ENIGMA and global neuroscience: a decade of large-scale studies of the brain in health and disease across more than 40 countries. Transl Psychiatry. 2020;10:1–28.
Mackey S, Kan KJ, Chaarani B, Alia-Klein N, Batalla A, Brooks S. et al. Chapter 10 - Genetic imaging consortium for addiction medicine: From neuroimaging to genes. In: Ekhtiari H, Paulus MP, editors. Progress in Brain Research [Internet]. Elsevier; 2016. p. 203–23. https://www.sciencedirect.com/science/article/pii/S0079612315001326 [cited 2021 Aug 14](Neuroscience for Addiction Medicine: From Prevention to Rehabilitation - Methods and Interventions; vol. 224).
Mackey S, Allgaier N, Chaarani B, Spechler P, Orr C, Bunn J, et al. Mega-analysis of gray matter volume in substance dependence: general and substance-specific regional effects. AJP. 2019;176:119–28.
Cao Z, McCabe M, Callas P, Cupertino RB, Ottino-González J, Murphy A, et al. Recalibrating single-study effect sizes using hierarchical Bayesian models. Front Neuroimaging [Internet]. 2023 Dec [cited 2024 Mar 22];2. Available from: https://www.frontiersin.org/articles/10.3389/fnimg.2023.1138193.
Smit DJA, Andreassen OA, Boomsma DI, Burwell SJ, Chorlian DB, de Geus EJC, et al. Large-scale collaboration in ENIGMA-EEG: a perspective on the meta-analytic approach to link neurological and psychiatric liability genes to electrophysiological brain activity. Brain Behav. 2021;11:e02188.
Caeyenberghs K, Imms P, Irimia A, Monti MM, Esopenko C, de Souza NL, et al. ENIGMA’s simple seven: Recommendations to enhance the reproducibility of resting-state fMRI in traumatic brain injury. NeuroImage: Clin. 2024;42:103585.
Agha RA, Fowler AJ, Limb C, Whitehurst K, Coe R, Sagoo H, et al. Impact of the mandatory implementation of reporting guidelines on reporting quality in a surgical journal: A before and after study. Int J Surg. 2016;30:169–72.
Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Methodology Review Group, editor. Cochrane Database of Systematic Reviews [Internet]. 2012 Nov [cited 2024 Mar 31]; 2013. Available from: https://doi.org/10.1002/14651858.MR000030.pub2.
Vilaró M, Cortés J, Selva-O’Callaghan A, Urrutia A, Ribera JM, Cardellach F, et al. Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review. BMC Med Res Methodol. 2019;19:112.
Shamseer L, Hopewell S, Altman DG, Moher D, Schulz KF. Update on the endorsement of CONSORT by high impact factor journals: a survey of journal “Instructions to Authors” in 2014. Trials. 2016;17:301.
Taylor R, Kardas M, Cucurull G, Scialom T, Hartshorn A, Saravia E, et al. Galactica: A Large Language Model for Science [Internet]. arXiv; 2022 [cited 2024 Mar 22]. Available from: http://arxiv.org/abs/2211.09085.
Liu R, Shah NB ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing [Internet]. arXiv; 2023 [cited 2024 Mar 22]. Available from: http://arxiv.org/abs/2306.00622.
Schulz KF, Altman DG, Moher D. the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8:18.
Hamilton CM, Strader LC, Pratt JG, Maiese D, Hendershot T, Kwok RK, et al. The PhenX Toolkit: get the most from your measures. Am J Epidemiol. 2011;174:253–60.
Ekhtiari H, Zare-Bidoky M, Sangchooli A, Janes AC, Kaufman MJ, Oliver JA, et al. A methodological checklist for fMRI drug cue reactivity studies: development and expert consensus. Nat Protoc. 2022;17:567–95.
Duarte RV, Bresnahan R, Copley S, Eldabe S, Thomson S, North RB, et al. Reporting guidelines for clinical trial protocols and reports of implantable neurostimulation devices: protocol for the SPIRIT-iNeurostim and CONSORT-iNeurostim extensions. Neuromodulation Technol Neural Interface. 2022;25:1045–9.
Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010;8:24.
Sarafoglou A, Hoogeveen S, Matzke D, Wagenmakers EJ. Teaching good research practices: protocol of a research master course. Psychol Learn Teach. 2020;19:46–59.
Kohrs FE, Auer S, Bannach-Brown A, Fiedler S, Haven TL, Heise V, et al. Eleven strategies for making reproducible research and open science training the norm at research institutions. Zaidi M, editor. eLife. 2023;12:e89736.
Pownall M, Azevedo F, König LM, Slack HR, Evans TR, Flack Z, et al. Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes. R Soc Open Sci. 2023;10:221255.
van Viegen T, Akrami A, Bonnen K, DeWitt E, Hyafil A, Ledmyr H, et al. Neuromatch Academy: teaching computational neuroscience with global accessibility. Trends Cogn Sci. 2021;25:535–8.
Moher D, Altman DG, Schulz KF, Simera I. How to Develop a Reporting Guideline. In: Moher D, Altman DG, Schulz KF, Simera I, Wager E, editors. Guidelines for Reporting Health Research: A User’s Manual [Internet]. 1st ed. Wiley; 2014. p. 14–21. https://onlinelibrary.wiley.com/doi/10.1002/9781118715598.ch2.
Bastow R, Leonelli S. Sustainable digital infrastructure. EMBO Rep. 2010;11:730–4.
Zakaria S, Grant J, Luff J. Fundamental challenges in assessing the impact of research infrastructure. Health Res Policy Sys. 2021;19:119.
Barker M, Katz DS. Overview of research software funding landscape. 2022 Feb [cited 2024 Mar 22]; Available from: https://zenodo.org/records/6102487.
RFA-MH-22-145: BRAIN Initiative: Standards to Define Experiments Related to the BRAIN Initiative (R01 Clinical Trial Not Allowed) [Internet]. [cited 2024 Mar 26]. Available from: https://grants.nih.gov/grants/guide/rfa-files/RFA-MH-22-145.html.
Backhausen LL, Herting MM, Tamnes CK, Vetter NC. Best practices in structural neuroimaging of neurodevelopmental disorders. Neuropsychol Rev. 2022;32:400–18.
Wachinger C, Rieckmann A, Pölsterl S. Alzheimer’s Disease Neuroimaging Initiative. Detect and correct bias in multi-site neuroimaging datasets. Med Image Anal. 2021;67:101879.
Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:MR000030.
Altman DG, Simera I, Hoey J, Moher D, Schulz K. EQUATOR: reporting guidelines for health research. Lancet. 2008;371:1149–50.
Ros T, Enriquez-Geppert S, Zotev V, Young KD, Wood G, Whitfield-Gabrieli S, et al. Consensus on the reporting and experimental design of clinical and cognitive-behavioural neurofeedback studies (CRED-nf checklist). Brain. 2020;143:1674–85.
Davis KD, Flor H, Greely HT, Iannetti GD, Mackey S, Ploner M, et al. Brain imaging tests for chronic pain: medical, legal and ethical issues and recommendations. Nat Rev Neurol. 2017;13:624–38.
Cisek P. Making decisions through a distributed consensus. Curr Opin Neurobiol. 2012;22:927–36.
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.
Jorm AF. Using the Delphi expert consensus method in mental health research. Aust N Z J Psychiatry. 2015;49:887–97.
Eickhoff S, Nichols TE, van Horn JD, Turner JA. Sharing the wealth: neuroimaging data repositories. Neuroimage. 2016;124:1065.
Petersen RC, Aisen PS, Beckett LA, Donohue MC, Gamst AC, Harvey DJ, et al. Alzheimer’s disease Neuroimaging Initiative (ADNI) clinical characterization. Neurology. 2010;74:201–9.
Poline JB, Breeze JL, Ghosh S, Gorgolewski K, Halchenko YO, Hanke M, et al. Data sharing in neuroimaging research. Front Neuroinform. 2012;6:9.
van Essen DC, Ugurbil K. The future of the human connectome. Neuroimage. 2012;62:1299–310.
Zidane YJT, Olsson NOE. Defining project efficiency, effectiveness and efficacy. Int J Manag Proj Bus. 2017;10:621–41.
Roy A, Colpitts J, Becker K, Brewer J, van Lutterveld R. Improving efficiency in neuroimaging research through application of Lean principles. PloS One. 2018;13:e0205232.
Shapiro L, Staroswiecki E, Gold G. Magnetic resonance imaging of the knee: optimizing 3 Tesla imaging. Semin Roentgenol. 2010;45:238–49.
Khodyakov D, Mikesell L, Schraiber R, Booth M, Bromley E. On using ethical principles of community-engaged research in translational science. Transl Res. 2016;171:52–62.
Puri KS, Suresh KR, Gogtay NJ, Thatte UM. Declaration of Helsinki, 2008: implications for stakeholders in research. J Postgrad Med. 2009;55:131–4.
Rotstein HG, Santamaria F. Development of theoretical frameworks in neuroscience: a pressing need in a sea of data. arXiv preprint arXiv:220909953. 2022.
Poline JB, Kennedy DN, Sommer FT, Ascoli GA, van Essen DC, Ferguson AR, et al. Is neuroscience FAIR? A call for collaborative standardisation of neuroscience data. Neuroinformatics. 2022;20:507–12.
Poldrack RA, Whitaker K, Kennedy D. Introduction to the special issue on reproducibility in neuroimaging. NeuroImage. 2020;218:116357.
Goldfarb MG, Brown DR. Diversifying participation: The rarity of reporting racial demographics in neuroimaging research. NeuroImage. 2022;254:119122.
Schwab S, Janiaud P, Dayan M, Amrhein V, Panczak R, Palagi PM, et al. Ten simple rules for good research practice. PLoS Comput Biol. 2022;18:e1010139.
Am Smeets P, Dagher A, Hare TA, Kullmann S, van der Laan LN, Poldrack RA, et al. Good practice in food-related neuroimaging. Am J Clin Nutr. 2019;109:491–503.
Nakayama T. What are “clinical practice guidelines”? J Neurol. 2007;254:2–7.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.
Pomponio R, Erus G, Habes M, Doshi J, Srinivasan D, Mamourian E, et al. Harmonization of large MRI datasets for the analysis of brain imaging patterns throughout the lifespan. Neuroimage. 2020;208:116450.
Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet. 1999;354:1896–900.
Matshabane OP. Promoting diversity and inclusion in neuroscience and neuroethics. EBioMedicine. 2021;67:103359.
Noble S, Scheinost D, Constable RT. A decade of test-retest reliability of functional connectivity: a systematic review and meta-analysis. Neuroimage. 2019;203:116157.
Strickland JC. Guide to research techniques in neuroscience. J Undergrad Neurosci Educ. 2014;13:R1.
Gross J, Baillet S, Barnes GR, Henson RN, Hillebrand A, Jensen O, et al. Good practice for conducting and reporting MEG research. Neuroimage. 2013;65:349–63.
Shekari M, Verwer EE, Yaqub M, Daamen M, Buckley C, Frisoni GB, et al. Harmonization of brain PET images in multi-center PET studies using Hoffman phantom scan. EJNMMI Phys. 2023;10:68.
Sullivan JA. The multiplicity of experimental protocols: a challenge to reductionist and non-reductionist models of the unity of neuroscience. Synthese. 2009;167:511–39.
Lu H, Kashani AH, Arfanakis K, Caprihan A, DeCarli C, Gold BT, et al. MarkVCID cerebral small vessel consortium: II. Neuroimaging protocols. Alzheimer’s Dement. 2021;17:716–25.
Murphy A, Weerakkody Y. MRI protocols. In: Radiopaedia.org. Radiopaedia.org; 2005.
O’Boyle EH, Götz M, Questionable research practices. Jussim, LJ, Krosnick, JA, and Stevens, ST Research integrity: Best practices for the social and behavioral sciences. 2022;260–94.
Xie Y, Wang K, Kong Y. Prevalence of research misconduct and questionable research practices: A systematic review and meta-analysis. Sci Eng Ethics. 2021;27:41.
Siritzky EM, Cox PH, Nadler SM, Grady JN, Kravitz DJ, Mitroff SR. Standard experimental paradigm designs and data exclusion practices in cognitive psychology can inadvertently introduce systematic “shadow” biases in participant samples. Cogn Res: Princ Implic. 2023;8:66.
Barch DM, Yarkoni T. Introduction to the special issue on reliability and replication in cognitive and affective neuroscience research. Cogn Affect Behav Neurosci. 2013;13:687–9.
Plichta MM, Schwarz AJ, Grimm O, Morgen K, Mier D, Haddad L, et al. Test–retest reliability of evoked BOLD signals from a cognitive–emotive fMRI test battery. Neuroimage. 2012;60:1746–58.
Elliott ML, Knodt AR, Ireland D, Morris ML, Poulton R, Ramrakha S, et al. What is the test-retest reliability of common task-functional MRI measures? New empirical evidence and a meta-analysis. Psychol Sci. 2020;31:792–806.
Rudeck J, Vogl S, Banneke S, Schönfelder G, Lewejohann L. Repeatability analysis improves the reliability of behavioral data. PloS One. 2020;15:e0230900.
Sörös P, Wölk L, Bantel C, Bräuer A, Klawonn F, Witt K. Replicability, repeatability, and long-term reproducibility of cerebellar morphometry. Cerebellum. 2021;20:439–53.
Miłkowski M, Hensel WM, Hohol M. Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail. J Comput Neurosci. 2018;45:163–72.
Dienlin T, Johannes N, Bowman ND, Masur PK, Engesser S, Kümpel AS, et al. An agenda for open science in communication. J Commun. 2021;71:1–26.
Kenall A, Edmunds S, Goodman L, Bal L, Flintoft L, Shanahan DR, et al. Better reporting for better research: a checklist for reproducibility. Gigascience. 2015;4:32.
Weissgerber TL, Garovic VD, Winham SJ, Milic NM, Prager EM. Transparent reporting for reproducible science. J Neurosci Res. 2016;94:859.
Heßler N, Rottmann M, Ziegler A. Empirical analysis of the text structure of original research articles in medical journals. PloS One. 2020;15:e0240288.
Botvinik-Nezer R, Wager TD. Reproducibility in neuroimaging analysis: challenges and solutions. Biol Psychiatry Cogn Neurosci Neuroimaging. 2023;8:780–8.
Glatard T, Lewis LB, Ferreira da Silva R, Adalat R, Beck N, Lepage C, et al. Reproducibility of neuroimaging analyses across operating systems. Front Neuroinformatics. 2015;9:12.
Valkenburg G, Dix G, Tijdink J, de Rijcke S. Expanding research integrity: a cultural-practice perspective. Sci Eng Ethics. 2021;27:10.
Beauvais MJS, Knoppers BM, Illes J. A marathon, not a sprint–neuroimaging, Open Science and ethics. Neuroimage. 2021;236:118041.
Graham M, Hallowell N, Savulescu J. A just standard: the ethical management of incidental findings in brain imaging research. J Law Med Ethics. 2021;49:269–81.
Tedersoo L, Küngas R, Oras E, Köster K, Eenmaa H, Leijen Ä, et al. Data sharing practices and data availability upon request differ across scientific disciplines. Sci Data. 2021;8:192.
Ciric R, Thompson WH, Lorenz R, Goncalves M, MacNicol EE, Markiewicz CJ, et al. TemplateFlow: FAIR-sharing of multi-scale, multi-species brain models. Nat Methods. 2022;19:1568–71.
Hedge C, Powell G, Sumner P. The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behav Res Methods. 2018;50:1166–86.
Helwegen K, Libedinsky I, van den Heuvel MP. Statistical power in network neuroscience. Trends Cogn Sci. 2023;27:282–301.
Esteban O, Markiewicz CJ, Blair RW, Moodie CA, Isik AI, Erramuzpe A, et al. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat Methods. 2019;16:111–6.
Loss CM, Melleu FF, Domingues K, Lino-de-Oliveira C, Viola GG. Combining animal welfare with experimental rigor to improve reproducibility in behavioral neuroscience. Front Behav Neurosci. 2021;15:763428.
Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proc Natl Acad Sci USA. 2018;115:2600–6.
Abrams MB, Bjaalie JG, Das S, Egan GF, Ghosh SS, Goscinski WJ, et al. A standards organization for open and FAIR neuroscience: the international neuroinformatics coordinating facility. Neuroinformatics. 2022;20:25–36.
Barnes J, Conrad K, Demont-Heinrich C, Graziano M, Kowalski D, Neufeld J, et al. Understanding generalizability and transferability. Writing@ CSU. 2012.
Schleim S. Real neurolaw in the Netherlands: the role of the developing brain in the new adolescent criminal law. Front Psychol. 2020;11:549375.
Bradley SH, DeVito NJ, Lloyd KE, Richards GC, Rombey T, Wayant C, et al. Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps. J R Soc Med. 2020;113:433–43.
James S, Rao SV, Granger CB. Registry-based randomized clinical trials—a new clinical trial paradigm. Nat Rev Cardiol. 2015;12:312–6.
Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials. gov results database—update and key issues. N Engl J Med. 2011;364:852–60.
Namiot ED, Smirnovová D, Sokolov AV, Chubarev VN, Tarasov VV, Schiöth HB. The international clinical trials registry platform (ICTRP): data integrity and the trends in clinical trials, diseases, and drugs. Front Pharmacol. 2023;14:1228148.
Andrade C. Internal, external, and ecological validity in research design, conduct, and evaluation. Indian J Psychol Med. 2018;40:498–9.
Wrightson JJ. CONSORT_GPT [Internet]. [cited 2024 Apr 13]. Available from: https://chat.openai.com/g/g-jOiNJ3mhR-consort-gpt?utm_source=gptshunter.com.
Ekhtiari H, Soleimani G, Kuplicki R, Yeh H, Cha Y, Paulus M. Transcranial direct current stimulation to modulate fMRI drug cue reactivity in methamphetamine users: a randomized clinical trial. Hum Brain Mapp. 2022;43:5340–57.
Funding
HE is supported by funds from Laureate Institute for Brain Research and Medical Discovery Team on Addiction and Brain and Behavior Foundation (NARSAD Young Investigator Award 27305). JK is supported by the Department of Veterans Affairs (National Center for PTSD), NIAAA (Center for the Translational Neuroscience of Alcohol (2P50AA012870-23), and National Center for Translational Science Clinical and Translational Science Award (2UL1TR001863-06). He has stock, options, or has received consultation fees from the following companies Aptinyx, Biogen, Biohaven Pharmaceuticals, Bionomics, Boehringer Ingelheim, Cartego Therapeutics, Damona Pharmaceuticals, Epiodyne, Epivario, Freedom Biosciences, Janssen Research and Development, Jazz Pharmaceuticals, Neumora Therapeutics, Otsuka America, Response Therapeutics, Rest Therapeutics, Spring Care, Sumitomi America, Terran Biosciencews, Tetricus, inc. He is an inventor on patents licensed by Yale University to Biohaven Pharmaceuticals, Freedom Biosciences, Janssen Pharmaceuticals, and Novartis Pharmaceuticals. DO has received R01MH114982 funding and also an honorarium from Boehringer-Ingelheim in the past 12 months. CRP is supported by the Novo Nordisk Fonden NNF20OC0063277. MPP is partly supported by The William K. Warren Foundation, the National Institute of General Medical Sciences Center (Grant 2 P20 GM121312), and the National Institute on Drug Abuse (U01DA050989). He advises Spring Care, Inc., receives royalties from an article on methamphetamine in UpToDate, and has a compensated consulting agreement with Boehringer Ingelheim International GmbH. Other authors declare no conflicts of interest.
Author information
Authors and Affiliations
Contributions
All authors contributed to the conception of the manuscript first through two panels held at the American College of Neuropsychopharmacology Annual Meeting (ACNP2023) and then through online rounds of discussions. HE, MZB, AS, AV, DMC, HG, TEN, CRP, PMT, and MPP contributed to drafting the first version of the manuscript. HE supervised the process of receiving and implementing the comments. All authors contributed to the manuscript revision, read, and approved the submitted version.
Corresponding author
Ethics declarations
Competing interests
TPG is the co-editor of the NeuroPschychoPharmacology. JK also serves as the editor of Biological Psychiatry. Other authors declare no conflicts of interest.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ekhtiari, H., Zare-Bidoky, M., Sangchooli, A. et al. Reporting checklists in neuroimaging: promoting transparency, replicability, and reproducibility. Neuropsychopharmacol. (2024). https://doi.org/10.1038/s41386-024-01973-5
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41386-024-01973-5