Many PhD students are enthusiastic about robust scientific practices, but afraid that ‘doing good science’ will jeopardize their chances on the job market, argues Felix Schönbrodt, Managing Director of the LMU Open Science Center. Aligning incentives and preparing students for a job market that values contributions to Open Science will be key.
Over the last few years, many scientific fields have been experiencing a replication crisis, in which individual high-profile replication failures and systematic investigations have cast doubt on the robustness and credibility of the published literature. This led to an increased awareness of good (and bad) scientific practices, triggering reform movements. Often the umbrella term ‘open science’ is used to summarize these reforms aiming at improved replicability and credibility of research.
Both the replication crisis itself and the reforms impact PhD students and early career researchers (ECRs). Imagine you plan your PhD project as an extension of seemingly robust findings. Yet after three years of failed experiments, you realize that you are not standing on the shoulders of giants, but rather trying to build on quicksand. In the previously existing incentive structure, typically only significant and clean (too good to be true?) results were publishable, and some journals explicitly desk-rejected replication studies and ‘uninteresting’ null results. A lot has been written about the billions of euros and dollars of funders’ money that have been wasted on research that was not replicable. Further, these alarming numbers need to be understood in the context of (yet-unknown) wasted years and jeopardized careers of PhD students working on projects that were doomed from the start.
As a reaction to the replication crisis, the open science reform movement aims for a more transparent, reproducible and replicable science, with the prospect that more published research findings are true. But learning how to preregister a study, documenting open datasets and open materials, and doing more reliable studies with larger sample sizes are time consuming. This investment can slow the progress open ECRs are able to make, compared to competitors that practice the old style of closed science. Surveys show that ECRs are the group that shows the strongest support for open science reforms. At the same time, ECRs have more fears about risks for their careers, as research practices may change faster than incentive structures.
They are faced with a dilemma: on the one hand, ECRs want to do open science, and several journals and funders already require higher standards of research transparency; on the other hand, the incentive structure for academic success mostly capitalizes on the old metrics of journal prestige, impact factor and quantity of research output. Going the extra mile to do reproducible research rarely counts as academic achievement. Hence, the insecurities of the academic job market force ECRs to do a cost benefit analysis in which ‘doing good research’ may be at odds with ‘having a career in academia’.
How can we resolve this dilemma? Institutional change is difficult and slow, and the current reform movement is in a phase of organic growth without central coordination. During this time of transition, some friction is inevitable, but we must make sure that ECRs are not the pawns sacrificed to this reform process. Actions in three areas must be taken to avoid such a scenario: build necessary skills, change transparency norms and change the incentive structure.
At the psychology department at the Ludwig-Maximilians-University Munich, our goals are to prepare ECRs for the future academic system and to shape a new research environment with new incentive structures. First, starting at the undergraduate level, we equip students with the skills to do reproducible science through mandatory classes on study preregistration, open data and reproducible data analysis scripts. Second, the dissertation agreement between PhDs students and their supervisors requires an explicit statement whether and in what way open practices will be pursued. The submission of the final dissertation must include a disclosure form that either states which open practices have been used in each empirical study (i.e., open data, open material, preregistration, open access), including a link to each of those, or explains why not.
Third, we change the incentive structure by rewarding open science practices at the level of hiring. Since 2015 our department has required an open science statement from every applicant for a professorship position:
“Our department embraces the values of open science and strives for replicable and reproducible research. For this goal we support transparent research with open data, open materials, and study pre-registration. Candidates are asked to describe in what way they already pursued and plan to pursue these goals.”
This statement is now an official part of our hiring policy. By including this statement in job ads, our department aims to communicate core values of good scientific practice and to attract excellent researchers who aim for transparent and credible research. At least 17 further hiring committees from other universities have started to include this or similar statements in their academic job descriptions (https://osf.io/7jbnt/). Thus, these institutions send a strong signal to ECRs: engaging in open science practices is not a stumbling block for their career, but a boost.
Transitions create tensions. It would be a tragic loss of potential if those young researchers who push for good scientific practices were selected out of academia. Therefore, it is the duty of the senior researchers in funding organizations, hiring committees and editorial boards to solve this dilemma for ECRs, by (i) providing accessible training for the necessary skills, (ii) changing the norms by expecting transparency from all researchers and (iii) changing the incentive structure by valuing transparency in publishing, hiring and promotion.
The author is managing director of the LMU Open Science Center, a non-profit organization that aims to increase transparency and reproducibility of research.
About this article
Cite this article
Schönbrodt, F. Training students for the Open Science future. Nat Hum Behav 3, 1031 (2019). https://doi.org/10.1038/s41562-019-0726-z
PLOS Computational Biology (2020)
Annals of the American Association of Geographers (2020)
European Journal of Education (2020)
Education for Information (2020)