Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Replication of a neuroimaging biomarker for striatal dysfunction in psychosis

Molecular Psychiatry (2024)Cite this article

Subjects

Abstract

To bring biomarkers closer to clinical application, they should be generalizable, reliable, and maintain performance within the constraints of routine clinical conditions. The functional striatal abnormalities (FSA), is among the most advanced neuroimaging biomarkers in schizophrenia, trained to discriminate diagnosis, with post-hoc analyses indicating prognostic properties. Here, we attempt to replicate its diagnostic capabilities measured by the area under the curve (AUC) in receiver operator characteristic curves discriminating individuals with psychosis (n = 101) from healthy controls (n = 51) in the Human Connectome Project for Early Psychosis. We also measured the test-retest (run 1 vs 2) and phase encoding direction (i.e., AP vs PA) reliability with intraclass correlation coefficients (ICC). Additionally, we measured effects of scan length on classification accuracy (i.e., AUCs) and reliability (i.e., ICCs). Finally, we tested the prognostic capability of the FSA by the correlation between baseline scores and symptom improvement over 12 weeks of antipsychotic treatment in a separate cohort (n = 97). Similar analyses were conducted for the Yeo networks intrinsic connectivity as a reference. The FSA had good/excellent diagnostic discrimination (AUC = 75.4%, 95% CI = 67.0–83.3%; in non-affective psychosis AUC = 80.5%, 95% CI = 72.1–88.0%, and in affective psychosis AUC = 58.7%, 95% CI = 44.2–72.0%). Test-retest reliability ranged between ICC = 0.48 (95% CI = 0.35–0.59) and ICC = 0.22 (95% CI = 0.06–0.36), which was comparable to that of networks intrinsic connectivity. Phase encoding direction reliability for the FSA was ICC = 0.51 (95% CI = 0.42–0.59), generally lower than for networks intrinsic connectivity. By increasing scan length from 2 to 10 min, diagnostic classification of the FSA increased from AUC = 71.7% (95% CI = 63.1–80.3%) to 75.4% (95% CI = 67.0–83.3%) and phase encoding direction reliability from ICC = 0.29 (95% CI = 0.14–0.43) to ICC = 0.51 (95% CI = 0.42–0.59). FSA scores did not correlate with symptom improvement. These results reassure that the FSA is a generalizable diagnostic – but not prognostic – biomarker. Given the replicable results of the FSA as a diagnostic biomarker trained on case-control datasets, next the development of prognostic biomarkers should be on treatment-response data.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overall study in context.
Fig. 2: Distribution of FSA and network intrinsic connectivity values between patients and controls.
Fig. 3: Receiver operating characteristic curves for prediction of diagnosis by FSA score and prediction of treatment response.
Fig. 4: Test-retest and phase encoding direction reliability of the FSA and network intrinsic connectivity.
Fig. 5: Accuracy of discrimination between cases and controls by scan acquisition time by the FSA and by intrinsic network connectivity.
Fig. 6: Phase encoding direction reliability by scan acquisition time for the FSA and by intrinsic network connectivity.

Similar content being viewed by others

Data availability

Analyses were conducted with custom scripts in R and python available on https://github.com/lorente01. Data for the HCP Early Psychosis is available at https://www.humanconnectome.org/study/human-connectome-project-for-early-psychosis and data for the ZHH cohort was accessed directly from the study Principal Investigator and is expected to be publicly available at https://nda.nih.gov/.

References

  1. Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372:793–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Stephan KE, Bach DR, Fletcher PC, Flint J, Frank MJ, Friston KJ, et al. Charting the landscape of priority problems in psychiatry, part 1: classification and diagnosis. Lancet Psychiatry. 2016;3:77–83.

    Article  PubMed  Google Scholar 

  3. Abi-Dargham A, Horga G. The search for imaging biomarkers in psychiatric disorders. Nat Med. 2016;22:1248–55.

    Article  CAS  PubMed  Google Scholar 

  4. Abi-Dargham A, Moeller SJ, Ali F, DeLorenzo C, Domschke K, Horga G, et al. Candidate biomarkers in psychiatric disorders: state of the field. World Psychiatry. 2023;22:236–62.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Li A, Zalesky A, Yue W, Howes O, Yan H, Liu Y, et al. A neuroimaging biomarker for striatal dysfunction in schizophrenia. Nat Med. 2020;26:558–65.

    Article  CAS  PubMed  Google Scholar 

  6. Scheinost D, Noble S, Horien C, Greene AS, Lake EM, Salehi M, et al. Ten simple rules for predictive modeling of individual differences in neuroimaging. Neuroimage. 2019;193:35–45.

    Article  PubMed  Google Scholar 

  7. Demro C, Mueller BA, Kent JS, Burton PC, Olman CA, Schallmo M-P, et al. The psychosis human connectome project: an overview. NeuroImage. 2021;241:118439.

    Article  PubMed  Google Scholar 

  8. Yeo BTT, Krienen FM, Sepulcre J, Sabuncu MR, Lashkari D, Hollinshead M, et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J Neurophysiol. 2011;106:1125–65.

    Article  PubMed  Google Scholar 

  9. Human Connectome Project for Early Psychosis Investigators. HCP Early Psychosis 1.1 Data Release https://www.humanconnectome.org/study/human-connectome-project-for-early-psychosis.

  10. First MB, Williams JBW, Karg RS, Spitzer RL. Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA: American Psychiatric Association; US; 2015.

  11. Woerner MG, Mannuzza S, Kane JM. Anchoring the BPRS: an aid to improved reliability. Psychopharmacol Bull. 1988;24:112–7.

    CAS  PubMed  Google Scholar 

  12. Sarpal DK, Robinson DG, Fales C, Lencz T, Argyelan M, Karlsgodt KH, et al. Relationship between duration of untreated psychosis and intrinsic corticostriatal connectivity in patients with early phase schizophrenia. Neuropsychopharmacology. 2017;42:2214–21.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Van Essen DC, Ugurbil K, Auerbach E, Barch D, Behrens TEJ, Bucholz R, et al. The Human Connectome Project: a data acquisition perspective. Neuroimage. 2012;62:2222–31.

    Article  PubMed  Google Scholar 

  14. Glasser MF, Sotiropoulos SN, Wilson JA, Coalson TS, Fischl B, Andersson JL, et al. The minimal preprocessing pipelines for the Human Connectome Project. Neuroimage. 2013;80:105–24.

    Article  PubMed  Google Scholar 

  15. Smith SM, Beckmann CF, Andersson J, Auerbach EJ, Bijsterbosch J, Douaud G, et al. Resting-state fMRI in the Human Connectome Project. Neuroimage. 2013;80:144–68.

    Article  PubMed  Google Scholar 

  16. Griffanti L, Salimi-Khorshidi G, Beckmann CF, Auerbach EJ, Douaud G, Sexton CE, et al. ICA-based artefact removal and accelerated fMRI acquisition for improved resting state network imaging. Neuroimage. 2014;95:232–47.

    Article  PubMed  Google Scholar 

  17. Salimi-Khorshidi G, Douaud G, Beckmann CF, Glasser MF, Griffanti L, Smith SM. Automatic denoising of functional MRI data: combining independent component analysis and hierarchical fusion of classifiers. Neuroimage. 2014;90:449–68.

    Article  PubMed  Google Scholar 

  18. Li J, Kong R, Liégeois R, Orban C, Tan Y, Sun N, et al. Global signal regression strengthens association between resting-state functional connectivity and behavior. NeuroImage. 2019;196:126–41.

    Article  PubMed  Google Scholar 

  19. Power JD, Barnes KA, Snyder AZ, Schlaggar BL, Petersen SE. Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. Neuroimage. 2012;59:2142–54.

    Article  PubMed  Google Scholar 

  20. Bing Liu Lab Github repository, https://github.com/BingLiu-Lab/FSA.

  21. Zou Q-H, Zhu C-Z, Yang Y, Zuo X-N, Long X-Y, Cao Q-J, et al. An improved approach to detection of amplitude of low-frequency fluctuation (ALFF) for resting-state fMRI: fractional ALFF. J Neurosci Methods. 2008;172:137–41.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Jia X-Z, Wang J, Sun H-Y, Zhang H, Liao W, Wang Z, et al. RESTplus: an improved toolkit for resting-state functional magnetic resonance imaging data processing. Sci Bull. 2019;64:953–4.

    Article  Google Scholar 

  23. Abraham A, Pedregosa F, Eickenberg M, Gervais P, Mueller A, Kossaifi J, et al. Machine learning for neuroimaging with scikit-learn. Front Neuroinform. 2014;8:14.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Dadi K, Varoquaux G, Machlouzarides-Shalit A, Gorgolewski KJ, Wassermann D, Thirion B, et al. Fine-grain atlases of functional modes for fMRI analysis. NeuroImage. 2020;221:117126.

    Article  PubMed  Google Scholar 

  25. Kraguljac NV, McDonald WM, Widge AS, Rodriguez CI, Tohen M, Nemeroff CB. Neuroimaging Biomarkers in Schizophrenia. AJP. 2021;178:509–21.

    Article  Google Scholar 

  26. Hedeker D, Gibbons RD. Longitudinal data analysis. Wiley-Interscience; Chicago, Illinois, US 2006.

  27. Noble S, Scheinost D, Constable RT. A guide to the measurement and interpretation of fMRI test-retest reliability. Curr Opin Behav Sci. 2021;40:27–32.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.

    Article  CAS  PubMed  Google Scholar 

  29. Cicchetti DV, Sparrow SA. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic. 1981;86:127–37.

    CAS  PubMed  Google Scholar 

  30. Cao H, Barber AD, Rubio JM, Argyelan M, Gallego JA, Lencz T, et al. Effects of phase encoding direction on test-retest reliability of human functional connectome. Neuroimage. 2023;277:120238.

    Article  CAS  PubMed  Google Scholar 

  31. Li J, Bzdok D, Chen J, Tam A, Ooi LQR, Holmes AJ, et al. Cross-ethnicity/race generalization failure of behavioral prediction from resting-state functional connectivity. Sci Adv. 2022;8:eabj1812.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Badhwar A, Collin-Verreault Y, Orban P, Urchs S, Chouinard I, Vogel J, et al. Multivariate consistency of resting-state fMRI connectivity maps acquired on a single individual over 2.5 years, 13 sites and 3 vendors. NeuroImage. 2020;205:116210.

    Article  PubMed  Google Scholar 

  33. Srirangarajan T, Mortazavi L, Bortolini T, Moll J, Knutson B. Multi‐band FMRI compromises detection of mesolimbic reward responses. NeuroImage. 2021;244:118617.

    Article  PubMed  Google Scholar 

  34. Abbosh C, Birkbak NJ, Wilson GA, Jamal-Hanjani M, Constantin T, Salari R, et al. Phylogenetic ctDNA analysis depicts early-stage lung cancer evolution. Nature. 2017;545:446–51.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749–62.

    Article  PubMed  Google Scholar 

  36. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–10.

    Article  PubMed  Google Scholar 

  37. Kane JM, Kishimoto T, Correll CU. Non-adherence to medication in patients with psychotic disorders: epidemiology, contributing factors and management strategies. World Psychiatry. 2013;12:216–26.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Alvarez-Jimenez M, Priede A, Hetrick SE, Bendall S, Killackey E, Parker AG, et al. Risk factors for relapse following treatment for first episode psychosis: a systematic review and meta-analysis of longitudinal studies. Schizophr Res. 2012;139:116–28.

    Article  CAS  PubMed  Google Scholar 

  39. Howes OD, McCutcheon R, Agid O, de Bartolomeis A, van Beveren NJM, Birnbaum ML, et al. Treatment-Resistant Schizophrenia: Treatment Response and Resistance in Psychosis (TRRIP) Working Group Consensus Guidelines on Diagnosis and Terminology. Am J Psychiatry. 2017;174:216–29.

    Article  PubMed  Google Scholar 

  40. Carbon M, Correll CU. Clinical predictors of therapeutic response to antipsychotics in schizophrenia. Dialogues Clin Neurosci. 2014;16:505–24.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Dong D, Wang Y, Chang X, Luo C, Yao D. Dysfunction of large-scale brain networks in schizophrenia: a meta-analysis of resting-state functional connectivity. Schizophrenia Bull. 2018;44:168–81.

    Article  Google Scholar 

  42. Mehta UM, Ibrahim FA, Sharma MS, Venkatasubramanian G, Thirthalli J, Bharath RD, et al. Resting-state functional connectivity predictors of treatment response in schizophrenia – a systematic review and meta-analysis. Schizophrenia Res. 2021;237:153–65.

    Article  Google Scholar 

  43. Wimberley T, Støvring H, Sørensen HJ, Horsdal HT, MacCabe JH, Gasse C. Predictors of treatment resistance in patients with schizophrenia: a population-based cohort study. Lancet Psychiatry. 2016;3:358–66.

    Article  PubMed  Google Scholar 

  44. Jauhar S, Johnstone M, McKenna PJ. Schizophrenia. Lancet. 2022;399:473–86.

    Article  CAS  PubMed  Google Scholar 

  45. Vivian-Griffiths T, Baker E, Schmidt KM, Bracher-Smith M, Walters J, Artemiou A, et al. Predictive modeling of schizophrenia from genomic data: comparison of polygenic risk score with kernel support vector machines approach. Am J Med Genet. 2019;180:80–85.

    Article  PubMed  Google Scholar 

  46. Zhang J-P, Robinson D, Yu J, Gallego J, Fleischhacker WW, Kahn RS, et al. Schizophrenia polygenic risk score as a predictor of antipsychotic efficacy in first-episode psychosis. AJP. 2019;176:21–28.

    Article  Google Scholar 

  47. Noble S, Spann MN, Tokoglu F, Shen X, Constable RT, Scheinost D. Influences on the test-retest reliability of functional connectivity MRI and its relationship with behavioral utility. Cereb Cortex. 2017;27:5415–29.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Liu J, Liao X, Xia M, He Y. Chronnectome fingerprinting: identifying individuals and predicting higher cognitive functions using dynamic brain connectivity patterns. Hum Brain Mapp. 2018;39:902–15.

    Article  PubMed  Google Scholar 

  49. Byrge L, Kennedy DP. Accurate prediction of individual subject identity and task, but not autism diagnosis, from functional connectomes. Hum Brain Mapp. 2020;41:2249–62.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Finn ES, Rosenberg MD. Beyond fingerprinting: choosing predictive connectomes over reliable connectomes. Neuroimage. 2021;239:118254.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

JR was supported by NIH grant K23MH127300.

Author information

Authors and Affiliations

  1. Donald and Barbara Zucker School of Medicine at Hofstra University – Northwell Health, New York, NY, USA

    Jose M. Rubio, Todd Lencz, Hengyi Cao, Elvisha Dhamala, Miklos Argyelan, Juan Gallego, Anita Barber, John M. Kane & Anil K. Malhotra

  2. Institute of Behavioral Science, Feinstein Institutes for Medical Research, Northwell Health, New York, NY, USA

    Jose M. Rubio, Todd Lencz, Hengyi Cao, Elvisha Dhamala, Miklos Argyelan, Juan Gallego, John Cholewa, Anita Barber, John M. Kane & Anil K. Malhotra

  3. Division of Psychiatry Research, Zucker Hillside Hospital, Northwell Health, New York, NY, USA

    Jose M. Rubio, Todd Lencz, Hengyi Cao, Elvisha Dhamala, Miklos Argyelan, Juan Gallego, John Cholewa, Anita Barber, John M. Kane & Anil K. Malhotra

  4. Department of Psychiatry and Behavioral Health, The Ohio State University, Columbus, OH, USA

    Nina Kraguljac

  5. Department of Psychiatry, Psychotherapy, and Psychosomatics, Psychiatric University Hospital Zurich, University of Zurich, 8032, Zurich, Switzerland

    Philipp Homan

  6. Neuroscience Center Zurich, University of Zurich and Swiss Federal Institute of Technology Zurich, 8057, Zurich, Switzerland

    Philipp Homan

  7. Department of Psychiatry, Columbia University, and New York State Psychiatric Institute, New York, NY, USA

    Guillermo Horga

  8. Department of Psychiatry, School of Medicine, University of Pittsburgh, Pittsburgh, PA, USA

    Deepak K. Sarpal

Authors
  1. Jose M. Rubio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Todd Lencz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hengyi Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nina Kraguljac
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elvisha Dhamala
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Philipp Homan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Guillermo Horga
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Deepak K. Sarpal
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Miklos Argyelan
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Juan Gallego
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. John Cholewa
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Anita Barber
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. John M. Kane
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Anil K. Malhotra
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JR conceptualized the initial plan for the study, conduct the analyses, and write the first draft of the manuscript. TL, HC, NK, ED, PH, GH, DS, MA, JG, AB, JK and AM provided feedback on the analysis strategy and manuscript. HC did secure the access to the HCP EP dataset and wrote some of the scripts that were used in the data analysis. JC co-wrote some of the scripts that were used in the analyses. The corresponding authors had access to all the data and takes full responsibility on the results presented in this manuscript.

Corresponding author

Correspondence to Jose M. Rubio.

Ethics declarations

Competing interests

JR has received research support from Alkermes, speaker bureau or advisory board compensation from TEVA, Karuna, Janssen, royalties from UpToDate. PH has received grants and honoraria from Novartis, Lundbeck, Mepha, Janssen, Boehringer Ingelheim, Neurolite outside of this work. JK has received funds for research support from H. Lundbeck, Janssen, and Otsuka; has received consulting fees or honoraria for lectures from Alkermes, Biogen, Boehringer Ingelheim, Cerevel, Dainippon Sumitomo, H. Lundbeck, HLS, Indivior, Intra-Cellular Therapies, Janssen, Johnson and Johnson, Karuna, LB Pharmaceuticals, Merck, Minerva, Neurocrine, Neumora, Newron, Novartis, Otsuka, Reviva, Roche, Saladax, Sunovion, Takeda, and Teva; and has ownership interest in HealthRhythms, North Shore Therapies, LB Pharmaceuticals, Medincell, and the Vanguard Research Group. The rest of the authors declare no conflict of interest.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rubio, J.M., Lencz, T., Cao, H. et al. Replication of a neuroimaging biomarker for striatal dysfunction in psychosis. Mol Psychiatry (2024). https://doi.org/10.1038/s41380-023-02381-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41380-023-02381-9

Search

Advanced search

Quick links