Abstract
Our capacity to measure diverse aspects of human biology has developed rapidly in the past decades, but the rate at which these techniques have generated insights into the biological correlates of psychopathology has lagged far behind. The slow progress is partly due to the poor sensitivity, specificity and replicability of many findings in the literature, which have in turn been attributed to small effect sizes, small sample sizes and inadequate statistical power. A commonly proposed solution is to focus on large, consortia-sized samples. Yet it is abundantly clear that increasing sample sizes will have a limited impact unless a more fundamental issue is addressed: the precision with which target behavioral phenotypes are measured. Here, we discuss challenges, outline several ways forward and provide worked examples to demonstrate key problems and potential solutions. A precision phenotyping approach can enhance the discovery and replicability of associations between biology and psychopathology.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 digital issues and online access to articles
$59.00 per year
only $4.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Perkins, E. R., Latzman, R. D. & Patrick, C. J. Interfacing neural constructs with the Hierarchical Taxonomy of Psychopathology: ‘why’ and ‘how’. Personal. Ment. Health 14, 106–122 (2020).
Insel, T. et al. Research Domain Criteria (RDoC): toward a new classification framework for research on mental disorders. Am. J. Psychiatry 167, 748–751 (2010).
Hyman, S. E. Can neuroscience be integrated into the DSM-V? Nat. Rev. Neurosci. 8, 725–732 (2007).
Singh, I. & Rose, N. Biomarkers in psychiatry. Nature 460, 202–207 (2009).
First, M. B. et al. Clinical applications of neuroimaging in psychiatric disorders. Am. J. Psychiatry 175, 915–916 (2018).
Marek, S. et al. Reproducible brain-wide association studies require thousands of individuals. Nature 603, 654–660 (2022).
Poldrack, R. A. et al. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat. Rev. Neurosci. 18, 115 (2017).
Saggar, M. & Uddin, L. Q. Pushing the boundaries of psychiatric neuroimaging to ground diagnosis in biology. eNeuro https://doi.org/10.1523/eneuro.0384-19.2019 (2019).
Sha, Z., Wager, T. D., Mechelli, A. & He, Y. Common dysfunction of large-scale neurocognitive networks across psychiatric disorders. Biol. Psychiatry 85, 379–388 (2019).
Smoller, J. W. et al. Psychiatric genetics and the structure of psychopathology. Mol. Psychiatry 24, 409–420 (2019).
Nour, M. M., Liu, Y. & Dolan, R. J. Functional neuroimaging in psychiatry and the case for failing better. Neuron 110, 2524–2544 (2022).
Sullivan, P. F. et al. Psychiatric genomics: an update and an agenda. Am. J. Psychiatry 175, 15–27 (2018).
Kundu, P., Inati, S. J., Evans, J. W., Luh, W.-M. & Bandettini, P. A. Differentiating BOLD and non-BOLD signals in fMRI time series using multi-echo EPI. NeuroImage 60, 1759–1770 (2012).
Parkes, L., Fulcher, B., Yücel, M. & Fornito, A. An evaluation of the efficacy, reliability, and sensitivity of motion correction strategies for resting-state functional MRI. NeuroImage 171, 415–436 (2018).
Kong, R. et al. Individual-specific areal-level parcellations improve functional connectivity prediction of behavior. Cereb. Cortex 31, 4477–4500 (2021).
Visscher, P. M. et al. 10 years of GWAS discovery: biology, function, and ranslation. Am. J. Hum. Genet. 101, 5–22 (2017).
Volkow, N. D. et al. The conception of the ABCD study: from substance use to a broad NIH collaboration. Dev. Cogn. Neurosci. 32, 4–7 (2018).
Lilienfeld, S. O. The Research Domain Criteria (RDoC): an analysis of methodological and conceptual challenges. Behav. Res. Ther. 62, 129–139 (2014).
Xing, X.-X. & Zuo, X.-N. The anatomy of reliability: a must read for future human brain mapping. Sci. Bull. 63, 1606–1607 (2018).
Zuo, X. N., Xu, T. & Milham, M. P. Harnessing reliability for neuroscience research. Nat. Hum. Behav. 3, 768–771 (2019).
Nikolaidis, A. et al. Suboptimal phenotypic reliability impedes reproducible human neuroscience. Preprint at bioRxiv https://doi.org/10.1101/2022.07.22.501193 (2022).
Falk, E. B. et al. What is a representative brain? Neuroscience meets population science. Proc. Natl. Acad. Sci. USA 110, 17615–17622 (2013).
Markon, K. E., Chmielewski, M. & Miller, C. J. The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psychol. Bull. 137, 856–879 (2011).
van der Sluis, S., Posthuma, D., Nivard, M. G., Verhage, M. & Dolan, C. V. Power in GWAS: lifting the curse of the clinical cut-off. Mol. Psychiatry 18, 2–3 (2013).
Fisher, J. E., Guha, A., Heller, W. & Miller, G. A. Extreme-groups designs in studies of dimensional phenomena: Advantages, caveats, and recommendations. J. Abnorm. Psychol. 129, 14–20 (2020).
Angold, A., Costello, E. J., Farmer, E. M. Z., Burns, B. J. & Erkanli, A. Impaired but undiagnosed. J. Am. Acad. Child Adolesc. Psychiatry 38, 129–137 (1999).
Preacher, K. J. in Extreme Groups Designs in the Encyclopedia of Clinical Psychology Vol. 2 (eds. Cautin, R. L. & Lilienfeld, S. O.) 1189–1192 (John Wiley and Sons, 2015).
Dong, H.-M. et al. Charting brain growth in tandem with brain templates at school age. Sci. Bull. 65, 1924–1934 (2020).
Peterson, R. E. et al. Genome-wide association studies in ancestrally diverse populations: opportunities, methods, pitfalls, and recommendations. Cell 179, 589–603 (2019).
Liu, S. et al. Chinese color nest project: an accelerated longitudinal brain-mind cohort. Dev. Cogn. Neurosci. 52, 101020 (2021).
Tobe, R. H. et al. A longitudinal resource for studying connectome development and its psychiatric associations during childhood. Sci. Data 9, 300 (2022).
Sanchez-Roige, S. & Palmer, A. A. Emerging phenotyping strategies will advance our understanding of psychiatric genetics. Nat. Neurosci. 23, 475–480 (2020).
Newson, J. J., Hunter, D. & Thiagarajan, T. C. The heterogeneity of mental health assessment. Front. Psychiatry https://doi.org/10.3389/fpsyt.2020.00076 (2020).
Feczko, E. et al. The heterogeneity problem: approaches to identify psychiatric subtypes. Trends Cogn. Sci. 23, 584–601 (2019).
Schnack, H. G. & Kahn, R. S. Detecting neuroimaging biomarkers for psychiatric disorders: sample size matters. Front. Psychiatry 7, 50 (2016).
Yang, Z. et al. Brain network informed subject community detection in early-onset schizophrenia. Sci. Rep. 4, 5549 (2014).
Hodgson, K., McGuffin, P. & Lewis, C. M. Advancing psychiatric genetics through dissecting heterogeneity. Hum. Mol. Genet. 26, R160–R165 (2017).
De Nadai, A. S., Hu, Y. & Thompson, W. K. Data pollution in neuropsychiatry—an under-recognized but critical barrier to research progress. JAMA Psychiatry 79, 97–98 (2022).
Reise, S. P., Bonifay, W. E. & Haviland, M. G. Scoring and modeling psychological measures in the presence of multidimensionality. J. Pers. Assess. 95, 129–140 (2013).
van der Sluis, S., Verhage, M., Posthuma, D. & Dolan, C. V. Phenotypic complexity, measurement bias, and poor phenotypic resolution contribute to the missing heritability problem in genetic association studies. PLoS ONE 5, e13929 (2010).
Caspi, A. & Moffitt, T. E. All for one and one for all: mental disorders in one dimension. Am. J. Psychiatry 175, 831–844 (2018).
Kotov, R. et al. The Hierarchical Taxonomy of Psychopathology (HiTOP): a quantitative nosology based on consensus of evidence. Annu. Rev. Clin. Psychol. 17, 83–108 (2021).
Clark, L. A. & Watson, D. Constructing validity: new developments in creating objective measuring instruments. Psychol. Assess. 31, 1412–1427 (2019).
Reise, S. P. The rediscovery of bifactor measurement models. Multivariate Behav. Res. 47, 667–696 (2012).
Reise, S. P. & Waller, N. G. Item response theory and clinical measurement. Annu. Rev. Clin. Psychol. 5, 27–48 (2009).
Rosopa, P. J., Schaffer, M. M. & Schroeder, A. N. Managing heteroscedasticity in general linear models. Psychol. Methods 18, 335–351 (2013).
Thomas, M. L. The value of item response theory in clinical assessment: a review. Assessment 18, 291–307 (2011).
Streiner, D. L. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J. Pers. Assess. 80, 99–103 (2003).
Saccenti, E., Hendriks, M. H. W. B. & Smilde, A. K. Corruption of the Pearson correlation coefficient by measurement error and its estimation, bias, and correction under different error models. Sci. Rep. 10, 438 (2020).
Vandenberg, R. J. & Lance, C. E. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ. Res. Methods 3, 4–70 (2000).
Miettunen, J., Nordstrom, T., Kaakinen, M. & Ahmed, A. O. Latent variable mixture modeling in psychiatric research: a review and application. Psychol. Med. 46, 457–467 (2016).
Achenbach, T. M. The Achenbach System of Empirically Based Assessment (ASEBA): Development, Findings, Theory, and Applications (University of Vermont, Research Center for Children, Youth and Families, 2009).
Kelly, E. L. Interpretation of Educational Measurements (World Book, 1927).
Fried, E. I. Moving forward: how depression heterogeneity hinders progress in treatment and research. Expert Rev. Neurother. 17, 423–425 (2017).
Fried, E. I. & Nesse, R. M. Depression is not a consistent syndrome: an investigation of unique symptom patterns in the STAR*D study. J. Affect. Disord. 172, 96–102 (2015).
Wager, T. D. & Woo, C.-W. Imaging biomarkers and biotypes for depression. Nat. Med. 23, 16–17 (2017).
Fried, E. I. & Nesse, R. M. Depression sum-scores don’t add up: why analyzing specific depression symptoms is essential. BMC Med. 13, 72 (2015).
Kendler, K. S., Aggen, S. H. & Neale, M. C. Evidence for multiple genetic factors underlying DSM-IV criteria for major depression. JAMA Psychiatry 70, 599–607 (2013).
Podsakoff, P. M., MacKenzie, S. B., Lee, J. & Podsakoff, N. P. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J. Appl. Psychol. 88, 879–903 (2003).
Podsakoff, P. M., MacKenzie, S. B. & Podsakoff, N. P. Sources of method bias in social science research and recommendations on how to control it. Annu. Rev. Psychol. 63, 539–569 (2012).
Haslam, N., Holland, E. & Kuppens, P. Categories versus dimensions in personality and psychopathology: a quantitative review of taxometric research. Psychol. Med. 42, 903–920 (2012).
Plomin, R., Haworth, C. M. & Davis, O. S. Common disorders are quantitative traits. Nat. Rev. Genet. 10, 872–878 (2009).
Cuthbert, B. N. The RDoC framework: facilitating transition from ICD/DSM to dimensional approaches that integrate neuroscience and psychopathology. World Psychiatry 13, 28–35 (2014).
Stanton, K., McDonnell, C. G., Hayden, E. P. & Watson, D. Transdiagnostic approaches to psychopathology measurement: Recommendations for measure selection, data analysis, and participant recruitment. J. Abnorm. Psychol. 129, 21–28 (2020).
Allen, N. et al. UK Biobank: current status and what it means for epidemiology. Health Policy Technol. 1, 123–126 (2012).
Strauss, M. E. & Smith, G. T. Construct validity: advances in theory and methodology. Annu. Rev. Clin. Psychol. 5, 1–25 (2009).
Karcher, N. R., Michelini, G., Kotov, R. & Barch, D. M. Associations between resting-state functional connectivity and a hierarchical dimensional structure of psychopathology in middle childhood. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 6, 508–517 (2021).
Tiego, J. et al. Dissecting schizotypy and its association with cognition and polygenic risk for schizophrenia in a nonclinical sample. Schizophr Bull. https://doi.org/10.1093/schbul/sbac016 (2023).
Conway, C. C., Forbes, M. K. & South, S. C. A Hierarchical Taxonomy of Psychopathology (HiTOP) primer for mental health researchers. Clin. Psychol. Sci. https://doi.org/10.1177/21677026211017834 (2021).
Yim, O. & Ramdeen, K. T. Hierarchical cluster analysis: comparison of three linkage measures and application to psychological data. Quant. Methods Psychol. 11, 8–21 (2015).
Goldberg, L. R. Doing it all bass-ackwards: the development of hierarchical factor structures from the top down. J. Res. Pers. 40, 347–358 (2006).
Michelini, G. et al. Delineating and validating higher-order dimensions of psychopathology in the Adolescent Brain Cognitive Development (ABCD) study. Transl. Psychiatry 9, 261 (2019).
Simms, L. J. et al. Computerized adaptive assessment of personality disorder: introducing the CAT–PD project. J. Pers. Assess. 93, 380–389 (2011).
Greven, C. U., Buitelaar, J. K. & Salum, G. A. From positive psychology to psychopathology: the continuum of attention‐deficit hyperactivity disorder. J. Child Psychol. Psychiatry 59, 203–212 (2018).
Stark, S., Chernyshenko, O. S. & Drasgow, F. Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy. J. Appl. Psychol. 91, 1292–1306 (2006).
van de Schoot, R. et al. Facing off with Scylla and Charybdis: a comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Front. Psychol. 4, 770 (2013).
Clark, S. L. et al. Models and strategies for factor mixture analysis: an example concerning the structure underlying psychological disorders. Struct. Equation Modell. 20, 681–703 (2013).
Eid, M., Lischetzke, T., Nussbeck, F. W. & Trierweiler, L. I. Separating trait effects from trait-specific method effects in multitrait-multimethod models: a multiple-indicator CT-C(M-1) model. Psychol. Methods 8, 38–60 (2003).
Eid, M., Geiser, C. & Koch, T. Measuring method effects: from traditional to design-oriented approaches. Curr. Dir. Psychol. Sci. 25, 275–280 (2016).
Aron, A. R. & Poldrack, R. A. The cognitive neuroscience of response inhibition: relevance for genetic research in attention-deficit/hyperactivity disorder. Biol. Psychiatry 57, 1285–1292 (2005).
Martinussen, R., Hayden, J., Hogg-Johnson, S. & Tannock, R. A meta-analysis of working memory impairments in children with attention-deficit/hyperactivity disorder. J. Am. Acad. Child Adolesc. Psychiatry 44, 377–384 (2005).
DeVellis, R. F. Classical test theory. Med. Care 44, S50–S59 (2006).
Nunnally, J. C. & Bernstein, I. Psychometric Theory 3rd edn (McGraw-Hill, 1994).
Antonakis, J., Bendahan, S., Jacquart, P. & Lalive, R. On making causal claims: a review and recommendations. Leadersh. Q. 21, 1086–1120 (2010).
Kendell, R. & Jablensky, R. Distinguishing between the validity and utility of psychiatric diagnoses. Am. J. Psychiatry 160, 4–12 (2003).
Kendler, K. S. The phenomenology of major depression and the representativeness and nature of DSM criteria. Am. J. Psychiatry 173, 771–780 (2016).
Kotov, R., Ruggero, C. J., Krueger, R. F., Watson, D. & Zimmerman, M. The perils of hierarchical exclusion rules: a further word of caution. Depress. Anxiety 35, 903–904 (2018).
Caspi, A. et al. The p factor: one general psychopathology factor in the structure of psychiatric disorders? Clin. Psychol. Sci. 2, 119–137 (2014).
Allsopp, K., Read, J., Corcoran, R. & Kinderman, P. Heterogeneity in psychiatric diagnostic classification. Psychiatry Res. 279, 15–22 (2019).
Cuthbert, B. N. & Insel, T. R. Toward the future of psychiatric diagnosis: the seven pillars of RDoC. BMC Med. 11, 126 (2013).
Simms, L. J. et al. Development of measures for the hierarchical taxonomy of psychopathology (HiTOP): a collaborative scale development project. Assessment 29, 3–16 (2021).
HiTOP Friendly Measures. HiTOP Clinical Network https://hitop.unt.edu/clinical-tools/hitop-friendly-measures (accessed 1 October 2022).
Lahey, B. B., Krueger, R. F., Rathouz, P. J., Waldman, I. D. & Zald, D. H. A hierarchical causal taxonomy of psychopathology across the life span. Psychol. Bull. 143, 142–186 (2017).
Michelini, G., Palumbo, I. M., DeYoung, C. G., Latzman, R. D. & Kotov, R. Linking RDoC and HiTOP: a new interface for advancing psychiatric nosology and neuroscience. Clin. Psychol. Rev. 86, 102025 (2021).
Hair, J. F., Black, W. C., Babin, B. J. & Anderson, R. E. Multivariate Data Analysis 7th edn (Pearson Education, 2014).
Grice, J. W. Computing and evaluating factor scores. Psychol. Methods 6, 430–450 (2001).
Devlieger, I., Mayer, A. & Rosseel, Y. Hypothesis testing using factor score regression: a comparison of four methods. Educ. Psychol. Meas. 76, 741–770 (2016).
Kim, J., Zhu, W., Chang, L., Bentler, P. M. & Ernst, T. Unified structural equation modeling approach for the analysis of multisubject, multivariate functional MRI data. Hum. Brain Mapp. 28, 85–93 (2007).
Kline, R. B. Principles and Practice of Structural Equation Modeling 4th edn (Guilford, 2015).
Reise, S. P. & Rodriguez, A. Item response theory and the measurement of psychiatric constructs: some empirical and conceptual issues and challenges. Psychol. Med. 46, 2025–2039 (2016).
de Ayala, R. J. The Theory and Practice of Item Response Theory (Guilford, 2009).
Acknowledgements
J.T. was supported by a Turner impact fellowship from the Turner Institute for Brain and Mental Health, Monash University, Australia. A.F. was supported by the Sylvia and Charles Viertel Foundation, the National Health and Medical Research Council (grant numbers: 1146292 and 1197431) and the Australian Research Council (grant number: DP200103509). A.J.S. was supported by the National Institute of Mental Health (grant numbers: AA030042, MH131264 and MH121409) and the University of Maryland.
Author information
Authors and Affiliations
Consortia
Corresponding author
Peer review
Peer review information
Nature Mental Health thanks Xi-Nian Zuo, Terrie Moffitt and Gregory Miller for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary information, tables and figures.
Rights and permissions
About this article
Cite this article
Tiego, J., Martin, E.A., DeYoung, C.G. et al. Precision behavioral phenotyping as a strategy for uncovering the biological correlates of psychopathology. Nat. Mental Health 1, 304–315 (2023). https://doi.org/10.1038/s44220-023-00057-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s44220-023-00057-5
This article is cited by
-
Functional connectivity uniqueness and variability? Linkages with cognitive and psychiatric problems in children
Nature Mental Health (2023)
-
Regional, circuit and network heterogeneity of brain abnormalities in psychiatric disorders
Nature Neuroscience (2023)