The replication crisis in the social, behavioural and life sciences has spurred a reform movement aimed at increasing the credibility of scientific studies. Many of these credibility-enhancing reforms focus, appropriately, on specific research and publication practices. A less often mentioned aspect of credibility is the need for intellectual humility or being transparent about and owning the limitations of our work. Although intellectual humility is presented as a widely accepted scientific norm, we argue that current research practice does not incentivize intellectual humility. We provide a set of recommendations on how to increase intellectual humility in research articles and highlight the central role peer reviewers can play in incentivizing authors to foreground the flaws and uncertainty in their work, thus enabling full and transparent evaluation of the validity of research.
This is a preview of subscription content, access via your institution
Subscribe to Nature+
Get immediate online access to Nature and 55 other Nature journal
Subscribe to Journal
Get full journal access for 1 year
only $9.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
[Editorial]. Tell it like it is. Nat. Hum. Behav. https://doi.org/10.1038/s41562-020-0818-9 (2020).
Pashler, H. & De Ruiter, J. P. Taking responsibility for our field’s reputation. APS Letter/Observer Forum https://www.psychologicalscience.org/observer/taking-responsibility-for-our-fields-reputation (2017).
Whitcomb, D., Battaly, H., Baehr, J. & Howard-Snyder, D. Intellectual humility: owning our limitations. Phil. Phenom. Res. https://doi.org/10.1111/phpr.12228 (2015).
Alfano, M. et al. Development and validation of a multi-dimensional measure of intellectual humility. PLoS ONE 12, e0182950 (2017).
Leary, M. R. et al. Cognitive and interpersonal features of intellectual humility. Pers. Soc. Psych. Bull. 43, 793–813 (2017).
Krumrei-Mancuso, E. J. & Rouse, S. V. The development and validation of the Comprehensive Intellectual Humility Scale. J. Pers. Assess. 98, 209–221 (2015).
Van Tongeren, D. R., Davis, D. E., Hook, J. N. & Witvliet, C. V. Humility. Curr. Dir. Psych. Sci. 28, 463–468 (2019).
Merton, R. K. Science and technology in a democratic order. J. Leg. Pol. Soc. 1, 115–126 (1942).
Vinkers, C. H., Tijdink, J. K. & Otte, W. M. Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. BMJ 351, h6467 (2015).
Yarkoni, T. The generalizability crisis. Behav. Brain Sci. https://doi.org/10.1017/S0140525X20001685 (2020).
Riddle, T. Linguistic overfitting in empirical psychology. Preprint at https://doi.org/10.31234/osf.io/qasde (2018).
Anderson, M. S., Martinson, B. C. & De Vries, R. Normative dissonance in science: results from a national survey of US scientists. J. Empir. Res. Hum. Res. Ethics 2, 3–14 (2007).
Mitroff, I. I. Norms and counter-norms in a select group of the Apollo moon scientists: a case study of the ambivalence of scientists. Am. Soc. Rev. 39, 579–595 (1974).
Sessions, R. How a ‘difficult’ composer gets that way. New York Times (8 January 1950).
Bem, D. J. in The Compleat Academic: A Practical Guide for the Beginning Social Scientist (eds Darley, J. M. et al.) Ch. 10 (American Psychological Association, 2004).
Kail, R. V. Reflections on five years as editor. APS Observer https://www.psychologicalscience.org/observer/reflections-on-five-years-as-editor (2012).
Nosek, B. A. et al. Promoting an open research culture. Science 348, 1422–1425 (2015).
Davis, W. E. et al. Peer-review guidelines promoting replicability and transparency in psychological science. Adv. Methods Pract. Psychol. Sci. 1, 556–573 (2018).
Morey, R. D. et al. The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review. R. Soc. Open Sci. 3, 150547 (2016).
Simmons, J. P., Nelson, L. & Simonsohn, U. A 21 word solution. Soc. Pers. Soc. Psych. Dial. 26, 4–7 (2012).
Chambers, C. D. Registered Reports: a new publishing initiative at Cortex. Cortex 49, 609–610 (2013).
Loftus, G. R. A picture is worth a thousand P values: on the irrelevance of hypothesis testing in the microcomputer age. Behav. Res. Meth. Inst. Comp. 25, 250–256 (1993).
Dragicevic, P., Jansen, Y., Sarma, A., Kay, M. & Chevalier, F. Increasing the transparency of research papers with explorable multiverse analyses. In Proc. 2019 Conf. on Human Factors in Computing Systems (eds Brewster, S. et al.) Paper no. 65 (Association for Computing Machinery, 2019).
Steegen, S., Tuerlinckx, F., Gelman, A. & Vanpaemel, W. Increasing transparency through a multiverse analysis. Perspect. Psychol. Sci. 11, 702–712 (2016).
Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L. & Kievit, R. A. An agenda for purely confirmatory research. Perspect. Psychol. Sci. 7, 632–638 (2012).
Hoekstra, R., Finch, S., Kiers, H. A. L. & Johnson, A. Probability as certainty: dichotomous thinking and the misuse of P values. Psychon. Bull. Rev. 13, 1033–1037 (2006).
Van der Bles, A. M. et al. Communicating uncertainty about facts, numbers and science. R. Soc. Open Sci. 6, 181870 (2019).
Hoekstra, R. Risk as an explanatory factor for researchers’ inferential interpretations. Math. Enthus. 12, 103–112 (2015).
Wasserstein, R. L., Schirm, A. L. & Lazar, N. A. Moving to a world beyond ‘P < 0.05’. Am. Stat. 73, 1–19 (2019).
Etz, A. & Vandekerckhove, J. Introduction to Bayesian inference for psychology. Psychon. Bull. Rev. 25, 5–34 (2018).
Ly, A. et al. Bayesian reanalyses from summary statistics: a guide for academic consumers. Adv. Methods Pract. Psychol. Sci. 1, 367–374 (2018).
Vazire, S. Editorial. Soc. Psychol. Personal. Sci. 7, 3–7 (2016).
Simons, D. J., Shoda, Y. & Lindsay, D. S. Constraints on generality (COG): a proposed addition to all empirical papers. Perspect. Psychol. Sci. 12, 1123–1128 (2017).
IJzerman, H. et al. Use caution when applying behavioural science to policy. Nat. Hum. Behav. 4, 1092–1094 (2020).
Flake, J. K., & Fried, E. I. Measurement schmeasurement: questionable measurement practices and how to avoid them. Preprint at https://doi.org/10.31234/osf.io/hs7wm (2019).
Schubert, S. Hedge-drift and advanced motte-and-bailey. LessWrong Blog https://www.lesswrong.com/posts/oMYeJrQmCeoY5sEzg/hedge-drift-and-advanced-motte-and-bailey (2016).
Sumner, P. et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ 349, g7015 (2014).
Rohrer, J. M. et al. Putting the self in self-correction: findings from the loss-of-confidence project. Perspect. Psychol. Sci., https://doi.org/10.1177/1745691620964106 (2021).
Yarkoni, T. (2018). No, it’s not the incentives—it’s you. Talyarkoni Blog https://www.talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/ (2018).
We thank A. Allard, A. Holcombe, H. Kiers, L. King, S. Lindsay and D. Trafimow for valuable input for this paper.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hoekstra, R., Vazire, S. Aspiring to greater intellectual humility in science. Nat Hum Behav 5, 1602–1607 (2021). https://doi.org/10.1038/s41562-021-01203-8
This article is cited by
Nature Reviews Psychology (2022)