Florian Frommlet is an Associate Professor at the Institute of Medical Statistics at the Medical University of Vienna, Austria. His research interests include both theoretical and applied statistics, and he has worked extensively on the problems of high dimensional data analysis which occur in modern genetics. Florian has been serving as Editorial Board Member for Scientific Reports since 2014, and is the Guest Editor for our Collection on Improving reproducibility in animal research.
On our 10th anniversary, Elizabeth Mann (Senior Editor at Scientific Reports) talked to Florian about changing attitudes to reproducibility, the publishing and research landscape, and what lies ahead.
Could you tell us a little about yourself and your current research?
I studied mathematics, but I have been working as a statistician for 20 years, and my research is threefold. I do statistical methodological research, and this involves high dimensional data, model selection issues, and Bayesian statistics. I also collaborate a lot with medical researchers: consulting, study design, planning studies, and also helping with data analysis. And for the last seven years, I've been working in the Animal Ethics Committee of our university, and this has led me to be more and more interested in statistical issues in animal experiments, and particularly the question of reproducibility.
Do you think that attitudes about the importance of reproducibility are changing?
If I say anything with authority, it's about animal trials, and I think there is some movement, although it's rather slow. Each semester, I teach young researchers and PhD students who work with animal trials, and they are interested. I hope that the younger generation will make a change, but it's a slow process.
What changes would you like to see in preclinical research?
Well, ARRIVE 2.0 has just come out, but the first ARRIVE guidelines seem to have had little impact, so we will see if they actually make a difference.
A problem with preclinical science is that statisticians are not really seen as collaborators. For some reason, preclinical people seem to believe they can do the statistics themselves. In medical trials, it's completely clear that a statistician takes care of the statistics. This cooperation would be a good change to preclinical research.
When I started as statistician in the animal ethics committee, it was often difficult to be taken seriously when you said there is a problem with your study design. Maybe one thing I see now is that we're getting taken more seriously. It's easier now because you see what has gone wrong in the last decade, there are all these reproducibility problems. So what we are saying is not completely bogus.
How could journals help more?
The whole idea of pre-registration is very important, but amongst preclinical researchers the concept is not often known so the research community will have to develop it. If journals offer some incentives to move in this direction, it makes it clear that it's really important.
Do you think there are changes we can make which will benefit research across the board?
Even within preclinical science, you have so many different subcultures, and in each specific field there are different issues to look at.
One very specific issues that I came across in our work at the animal ethics committee is how certain researchers plan their experiment. They say "we need three times as many animals to replicate the experiment three times, because journals ask us to." Because this is how they see replication – if they replicate the whole experiment three times, then this will show that their experiment is reproducible.
We wrote about this, and the problem with this approach and why it's not good. We got one reviewer report where they told us: This problem doesn't exist.
And this is where I learned: Okay, so this problem only exists in a very narrow branch in immunology. So this tradition which is completely bogus from a statistics point of view, is how things are done. I don't think the paper changed anything, but, well, you have to start somewhere. So I would say you have to look at specific problems in different research fields.
Could researchers collaborating across different disciplines help overcome these kinds of challenges?
Yes, if people see different fields, I think it would help. There are people who work in a clinical setting who also collaborate on preclinical research. I don't know how common this was in the past, but this is something which I definitely see happening.
How have you found being the Guest Editor of your Collection?
I've found the whole experience quite nice. You have a little more control which papers are coming to you to handle, and of course improving reproducibility in animal research is something that I have a very personal interest in. For me, it was a very good experience, like having your own project.
What do you think sets Scientific Reports apart from other journals?
Well, of course it's very broad. The most important thing for articles to get published is not that the presented results are completely new and innovative, but rather that it's sound research. Ideally, this should help to work against publication bias.