The publication of our first two Registered Reports marks a major milestone for Nature Human Behaviour. These studies demonstrate what many researchers know, but is often hidden from the published literature: confirmatory research doesn’t always confirm the authors’ hypotheses.
When we launched Nature Human Behaviour in 2017, we were committed to making the journal a champion of robust scientific practices, and this commitment is reflected in our editorial criteria (see https://www.nature.com/nathumbehav/info/editorial-process) and publication formats. We adopted the Registered Report format from the outset (see our launch editorial: https://www.nature.com/articles/s41562-016-0034), and have since received a large number of excellent submissions. This month marks a special milestone in the history of Registered Reports at Nature Human Behaviour, as we are publishing our first two Stage 2 Registered Reports: articles that report on studies that were peer reviewed and accepted in principle before data collection began.
When we accepted in principle the Stage 1 versions of these Registered Reports, both we and our reviewers were persuaded that these papers were addressing important questions of broad significance and that they represented the most rigorous efforts to address these questions to date. We committed to publishing these studies before the results were known because we strongly believe that when the question is important and the methods robust, the results will be important no matter what they are.
The results of both studies are illuminating: they showcase how the results of meticulously designed, carefully executed studies, for which a commitment to publication was made results-blind, may differ from the type of work we usually see published, which typically presents ‘clean’ results that seemingly always confirm the researchers’ primary hypothesis.
He & Côté1 designed a large-scale online diary study to address the longstanding question of whether better insight into one’s abilities is associated with higher satisfaction with one’s life, career, and relationship. In a comprehensive and analytically sophisticated design, vetted down to each single line of code by the reviewers before data collection began, the authors tested five main hypotheses for how abilities might relate to life, career, and relationship satisfaction. Yet none of these hypotheses, all previously put forward in the literature, were supported by the evidence.
Brannon et al.2 conducted a study to address the much-debated question of whether testosterone changes how participants choose to react in moral dilemmas. Their hypothesis, based on considerable previous work, was that administering testosterone would increase participants’ willingness to take morally transgressive actions in those fictional dilemmas, for example by deciding to take active steps to sacrifice a single life to save others. However, the results contradicted the hypothesis, showing that participants became more sensitive to moral norms and were therefore less likely to take these steps.
In essence, the first two Registered Reports we are publishing consist of a null result and a result that contradicts the experimental hypothesis. These were obtained through highly powered, rigorously vetted protocols, lending significant credence to the results. The outcomes of our first two published Registered Reports amplify what has become increasingly apparent during the nascent credibility revolution in the social and behavioural sciences: we have to expect scientific results to be contradictory, unpredicted or even messy. This does not reduce their value; rather, by ensuring that studies are extremely robustly designed, we guarantee that their results offer significant, trustworthy insights, regardless of direction. We need these important contributions to be published.