The replication crisis in the social, behavioural and life sciences has spurred a reform movement aimed at increasing the credibility of scientific studies. Many of these credibility-enhancing reforms focus, appropriately, on specific research and publication practices. A less often mentioned aspect of credibility is the need for intellectual humility or being transparent about and owning the limitations of our work. Although intellectual humility is presented as a widely accepted scientific norm, we argue that current research practice does not incentivize intellectual humility. We provide a set of recommendations on how to increase intellectual humility in research articles and highlight the central role peer reviewers can play in incentivizing authors to foreground the flaws and uncertainty in their work, thus enabling full and transparent evaluation of the validity of research.
This is a preview of subscription content, access via your institution
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
[Editorial]. Tell it like it is. Nat. Hum. Behav. https://doi.org/10.1038/s41562-020-0818-9 (2020).
Pashler, H. & De Ruiter, J. P. Taking responsibility for our field’s reputation. APS Letter/Observer Forum https://www.psychologicalscience.org/observer/taking-responsibility-for-our-fields-reputation (2017).
Whitcomb, D., Battaly, H., Baehr, J. & Howard-Snyder, D. Intellectual humility: owning our limitations. Phil. Phenom. Res. https://doi.org/10.1111/phpr.12228 (2015).
Alfano, M. et al. Development and validation of a multi-dimensional measure of intellectual humility. PLoS ONE 12, e0182950 (2017).
Leary, M. R. et al. Cognitive and interpersonal features of intellectual humility. Pers. Soc. Psych. Bull. 43, 793–813 (2017).
Krumrei-Mancuso, E. J. & Rouse, S. V. The development and validation of the Comprehensive Intellectual Humility Scale. J. Pers. Assess. 98, 209–221 (2015).
Van Tongeren, D. R., Davis, D. E., Hook, J. N. & Witvliet, C. V. Humility. Curr. Dir. Psych. Sci. 28, 463–468 (2019).
Merton, R. K. Science and technology in a democratic order. J. Leg. Pol. Soc. 1, 115–126 (1942).
Vinkers, C. H., Tijdink, J. K. & Otte, W. M. Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. BMJ 351, h6467 (2015).
Yarkoni, T. The generalizability crisis. Behav. Brain Sci. https://doi.org/10.1017/S0140525X20001685 (2020).
Riddle, T. Linguistic overfitting in empirical psychology. Preprint at https://doi.org/10.31234/osf.io/qasde (2018).
Anderson, M. S., Martinson, B. C. & De Vries, R. Normative dissonance in science: results from a national survey of US scientists. J. Empir. Res. Hum. Res. Ethics 2, 3–14 (2007).
Mitroff, I. I. Norms and counter-norms in a select group of the Apollo moon scientists: a case study of the ambivalence of scientists. Am. Soc. Rev. 39, 579–595 (1974).
Sessions, R. How a ‘difficult’ composer gets that way. New York Times (8 January 1950).
Bem, D. J. in The Compleat Academic: A Practical Guide for the Beginning Social Scientist (eds Darley, J. M. et al.) Ch. 10 (American Psychological Association, 2004).
Kail, R. V. Reflections on five years as editor. APS Observer https://www.psychologicalscience.org/observer/reflections-on-five-years-as-editor (2012).
Nosek, B. A. et al. Promoting an open research culture. Science 348, 1422–1425 (2015).
Davis, W. E. et al. Peer-review guidelines promoting replicability and transparency in psychological science. Adv. Methods Pract. Psychol. Sci. 1, 556–573 (2018).
Morey, R. D. et al. The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review. R. Soc. Open Sci. 3, 150547 (2016).
Simmons, J. P., Nelson, L. & Simonsohn, U. A 21 word solution. Soc. Pers. Soc. Psych. Dial. 26, 4–7 (2012).
Chambers, C. D. Registered Reports: a new publishing initiative at Cortex. Cortex 49, 609–610 (2013).
Loftus, G. R. A picture is worth a thousand P values: on the irrelevance of hypothesis testing in the microcomputer age. Behav. Res. Meth. Inst. Comp. 25, 250–256 (1993).
Dragicevic, P., Jansen, Y., Sarma, A., Kay, M. & Chevalier, F. Increasing the transparency of research papers with explorable multiverse analyses. In Proc. 2019 Conf. on Human Factors in Computing Systems (eds Brewster, S. et al.) Paper no. 65 (Association for Computing Machinery, 2019).
Steegen, S., Tuerlinckx, F., Gelman, A. & Vanpaemel, W. Increasing transparency through a multiverse analysis. Perspect. Psychol. Sci. 11, 702–712 (2016).
Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L. & Kievit, R. A. An agenda for purely confirmatory research. Perspect. Psychol. Sci. 7, 632–638 (2012).
Hoekstra, R., Finch, S., Kiers, H. A. L. & Johnson, A. Probability as certainty: dichotomous thinking and the misuse of P values. Psychon. Bull. Rev. 13, 1033–1037 (2006).
Van der Bles, A. M. et al. Communicating uncertainty about facts, numbers and science. R. Soc. Open Sci. 6, 181870 (2019).
Hoekstra, R. Risk as an explanatory factor for researchers’ inferential interpretations. Math. Enthus. 12, 103–112 (2015).
Wasserstein, R. L., Schirm, A. L. & Lazar, N. A. Moving to a world beyond ‘P < 0.05’. Am. Stat. 73, 1–19 (2019).
Etz, A. & Vandekerckhove, J. Introduction to Bayesian inference for psychology. Psychon. Bull. Rev. 25, 5–34 (2018).
Ly, A. et al. Bayesian reanalyses from summary statistics: a guide for academic consumers. Adv. Methods Pract. Psychol. Sci. 1, 367–374 (2018).
Vazire, S. Editorial. Soc. Psychol. Personal. Sci. 7, 3–7 (2016).
Simons, D. J., Shoda, Y. & Lindsay, D. S. Constraints on generality (COG): a proposed addition to all empirical papers. Perspect. Psychol. Sci. 12, 1123–1128 (2017).
IJzerman, H. et al. Use caution when applying behavioural science to policy. Nat. Hum. Behav. 4, 1092–1094 (2020).
Flake, J. K., & Fried, E. I. Measurement schmeasurement: questionable measurement practices and how to avoid them. Preprint at https://doi.org/10.31234/osf.io/hs7wm (2019).
Schubert, S. Hedge-drift and advanced motte-and-bailey. LessWrong Blog https://www.lesswrong.com/posts/oMYeJrQmCeoY5sEzg/hedge-drift-and-advanced-motte-and-bailey (2016).
Sumner, P. et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ 349, g7015 (2014).
Rohrer, J. M. et al. Putting the self in self-correction: findings from the loss-of-confidence project. Perspect. Psychol. Sci., https://doi.org/10.1177/1745691620964106 (2021).
Yarkoni, T. (2018). No, it’s not the incentives—it’s you. Talyarkoni Blog https://www.talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/ (2018).
We thank A. Allard, A. Holcombe, H. Kiers, L. King, S. Lindsay and D. Trafimow for valuable input for this paper.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hoekstra, R., Vazire, S. Aspiring to greater intellectual humility in science. Nat Hum Behav 5, 1602–1607 (2021). https://doi.org/10.1038/s41562-021-01203-8
This article is cited by
Nature Human Behaviour (2023)
Tighter nets for smaller fishes? Mapping the development of statistical practices in consumer research between 2008 and 2020
Marketing Letters (2023)
Nature Reviews Psychology (2022)