Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Unconsidered issues of measurement noninvariance in biological psychiatry: A focus on biological phenotypes of psychopathology

Molecular Psychiatry volume 27, pages 1281–1285 (2022)Cite this article

Subjects

Abstract

There is increasing appreciation that certain biological processes may not be equally related to all psychiatric symptoms in a given diagnostic category. Research on the biological phenotyping of psychopathology has begun examining the etiological and treatment implications of identified biotypes; however, little attention has been paid to a critical methodological implication of these results: measurement noninvariance. Measurement invariance is the ability of an instrument to measure the same construct, the same way, across different people, or across different time points for the same individual. If what a measure quantifies differs across different people (e.g., those with or without a particular biotype) or time points, then it is invalid to directly compare means on that measure. Using a running example of inflammatory phenotypes of depression, we first describe the biological phenotyping of psychopathology. Second, we discuss three types of measurement invariance. Third, we demonstrate how differential biology-symptom associations invariably creates measurement noninvariance using a theoretical example and simulated data (for which code is provided). We also show how this issue can lead to false conclusions about the broader diagnostic construct. Finally, we provide several suggestions for addressing these important issues to help advance the field of biological psychiatry.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Visual representation of a risk factor associated with a subset of symptoms.
Fig. 2: Visual representations of measurement noninvariance.

Similar content being viewed by others

References

  1. Majd M, Saunders EFH, Engeland CG. Inflammation and the dimensions of depression: a review. Front Neuroendocrinol. 2020;56.

  2. Fried EI, Nesse RM, Zivin K, Guille C, Sen S. Depression is more than the sum score of its parts: individual DSM symptoms have different risk factors. Psychol Med. 2014;44:2067–76.

    Article  CAS  Google Scholar 

  3. Slavich GM, Irwin MR. From stress to inflammation and major depressive disorder: a social signal transduction theory of depression. Psychol Bull. 2014;140:774–815.

    Article  Google Scholar 

  4. Moriarity DP. Building a replicable and clinically-impactful immunopsychiatry: methods, phenotyping, and theory integration. Brain Behav Immun-Heal. 2021;16.

  5. Mac Giollabhui N, Ng TH, Ellman LM, Alloy LB. The longitudinal associations of inflammatory biomarkers and depression revisited: systematic review, meta-analysis, and meta-regression. Mol Psychiatry. 2020:1–13.

  6. Fried EI, von Stockert S, Haslbeck JMB, Lamers F, Schoevers RA, Penninx BWJH. Using network analysis to examine links between individual depressive symptoms, inflammatory markers, and covariates. Psychol Med. 2019. 2019. https://doi.org/10.31234/osf.io/84ske.

  7. Moriarity DP, Horn SR, Kautz MM, Haslbeck JM, Alloy LB. How handling extreme C-reactive protein (CRP) values and regularization influences CRP and depression criteria associations in network analyses. Brain Behav Immun. 2021;91:393–403.

    Article  CAS  Google Scholar 

  8. Milaneschi Y, Kappelmann N, Ye Z, Lamers F, Moser S, Jones PB, et al. Association of inflammation with depression and anxiety: evidence for symptom-specificity and potential causality from UK Biobank and NESDA Cohorts. Mol Psychiatry. https://doi.org/10.1101/2021.01.08.20248710.

  9. Moriarity DP, Alloy LB. Beyond diagnoses and total symptom scores: diversifying the level of analysis in psychoneuroimmunology research. Brain Behav Immun. 2020;89:1–2.

    Article  Google Scholar 

  10. Putnick DL, Bornstein MH. Measurement invariance conventions and reporting: the state of the art and future directions for psychological research. Dev Rev. 2016;41:71–90.

    Article  Google Scholar 

  11. McNeish D, Wolf MG. Thinking twice about sum scores. Behav Res Methods. 2020;52:2287–305.

    Article  Google Scholar 

  12. Kroenke K, Spitzer RL, Williams JB. The PHQ‐9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–13.

    Article  CAS  Google Scholar 

  13. Bauer D. A more general model for testing measurement invariance and differential item functioning. Psychol Methods. 2017;22:507–26.

    Article  Google Scholar 

  14. Dooley LN, Kuhlman KR, Robles TF, Eisenberger NI, Craske MG, Bower JE. The role of inflammation in core features of depression: insights from paradigms using exogenously-induced inflammation. Neurosci Biobehav Rev. 2018;94:219–37.

    Article  Google Scholar 

  15. Isvoranu AM, Guloksuz S, Epskamp S, van Os J, Borsboom D. Toward incorporating genetic risk scores into symptom networks of psychosis. Psychol Med. 2020;50:636–43.

    Article  Google Scholar 

  16. van Loo HM, Van Borkulo CD, Peterson RE, Fried EI, Aggen SH, Borsboom D, et al. Robust symptom networks in recurrent major depression across different levels of genetic and environmental risk. J Affect Disord. 2018;227:313–22.

    Article  Google Scholar 

  17. Santos H, Fried EI, Asafu-Adjei J, Jeanne, Ruiz R. Network structure of perinatal depressive symptoms in Latinas: relationship to stress and reproductive biomarkers. Res Nurs Heal. 2017;40:218–28.

    Article  Google Scholar 

  18. Okada K, Nakao T, Sanematsu H, Murayama K, Honda S, Tomita M, et al. Biological heterogeneity of obsessive-compulsive disorder: a voxel-based morphometric study based on dimensional assessment. Psychiatry Clin Neurosci. 2015;69:411–21.

    Article  Google Scholar 

  19. Hilland E, Landrø NI, Kraft B, Tamnes CK, Fried EI, Maglanoc LA, et al. Exploring the links between specific depression symptoms and brain structure: a network study. Psychiatry Clin Neurosci. 2020;74:220–1.

    Article  Google Scholar 

  20. Kotov R, Krueger RF, Watson D, Bagby M, Carpenter WT, Caspi A. The hierarchical taxonomy Of Psychopathology (HiTOP). J Abnorm Psychol. 2017:1–83.

  21. Feczko E, Miranda-dominguez O, Marr M, Graham AM, Nigg JT, Fair DA. The heterogeneity problem: approaches to identify psychiatric subtypes. Trends Cogn Sci. 2019:1–18.

  22. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine D, Quinn K, et al. Research Domain Criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry Online. 2010;167:748–51.

    Article  Google Scholar 

  23. Ryff CD, Seeman T, Weinstein M. Midlife in the United States (MIDUS 2): Biomarker Project, 2004-2009. Ann Arbor, MI Inter-University Consort Polit Soc Res. [Distributor]. 2017:10.

  24. Allen NE, Sudlow C, Peakman T, Collins RUK. Biobank data: come and get it. Sci Transl Med. 2014;6:4–7.

    Article  Google Scholar 

  25. Volkow ND, Koob GF, Croyle RT, Bianchi DW, Gordon JA, Koroshetz WJ, et al. The conception of the ABCD study: from substance use to a broad NIH collaboration. Dev Cogn Neurosci. 2018;32:4–7.

    Article  Google Scholar 

Download references

Acknowledgements

DPM was supported by National Research Service Award F31 MH122116 and an APF Visionary Grant. KJJ was supported by a Ford Foundation Predoctoral Fellowship administered by the National Academy of Sciences, Engineering, and Medicine and National Institute of Drug Abuse R36 DA050049. GMS was supported by National Institutes of Health grant K08 MH103443 and by grant OPR21101 from the California Initiative to Advance Precision Medicine. LBA was supported by National Institute of Mental Health R01 MH101168.

Author information

Authors and Affiliations

  1. Department of Psychology, Temple University, Philadelphia, USA

    Daniel P. Moriarity & Lauren B. Alloy

  2. Department of Psychiatry, McLean Hospital/Harvard University Medical School, Boston, USA

    Daniel P. Moriarity

  3. Department of Psychology, Florida State University, Tallahassee, USA

    Keanan J. Joyner

  4. Cousins Center for Psychoneuroimmunology and Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, USA

    George M. Slavich

Authors
  1. Daniel P. Moriarity
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keanan J. Joyner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George M. Slavich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lauren B. Alloy
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DPM generated the idea for the manuscript, wrote the manuscript, and ran analyses. KJJ helped refine theoretical underpinning of the manuscript, consulted on code, and provided feedback on the manuscript. GMS and LBA provided feedback on the manuscript.

Corresponding author

Correspondence to Daniel P. Moriarity.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moriarity, D.P., Joyner, K.J., Slavich, G.M. et al. Unconsidered issues of measurement noninvariance in biological psychiatry: A focus on biological phenotypes of psychopathology. Mol Psychiatry 27, 1281–1285 (2022). https://doi.org/10.1038/s41380-021-01414-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41380-021-01414-5

This article is cited by

Search

Advanced search

Quick links