Podcast: The super-sleuth who spots trouble in science papers

Host: Shamini Bundell

Welcome back to the Nature Podcast. This week, Elisabeth Bik, the scientific image detective…

Host: Nick Howe

And how air pollution can form in cities. I’m Nick Howe.

Host: Shamini Bundell

And I’m Shamini Bundell.

[Jingle]

Host: Shamini Bundell

First up, reporter Adam Levy has made contact with a scientific sleuth.

Interviewer: Adam Levy

Elisabeth Bik spends her days reading academic papers. Well, actually, reading might not be the right word.

Interviewee: Elisabeth Bik

I can indeed scan hundreds of papers sometimes a day because I just look at the images. I’m just like a four-year-old scanning a book for the pictures.

Interviewer: Adam Levy

Elisabeth is a science consultant based in Sunnyvale, California, and she’s staring at pictures for a reason – to look for patterns. She’s hunting for duplicated images, sometimes which are being flipped, rotated or digitally manipulated. These can suggest mistakes or malpractice on the part of researchers, and Elisabeth shares her finds with journals, as well as by posting publicly. Initially just a hobby, Elisabeth became hooked and is now a full-time image sleuth. I gave her a call to find out why and how she does what she does.

Interviewee: Elisabeth Bik

Once you see those patterns, it’s really hard to not see it anymore. You just see the same cell four, five or sometimes ten times in the same photo. Just to see the same thing over and over again in the same photo, it’s sometimes just hilarious to see it.

Interviewer: Adam Levy

What I’ve found when looking at examples of this kind of thing is that when I just see the example I am hopeless, and then when the suspect area is circled or indicated in some way, it just feels so obvious and I can’t believe I didn’t see it in the first place.

Interviewee: Elisabeth Bik

That is, I guess, my experience. But yeah, it is hard to explain because in the beginning I thought well, this is so obvious, and I would send it to a journal editor and they were like, ‘I don’t see it.’ I’m like, ‘Well, it’s obvious, right?’ And they’re like, ‘No, I don’t see it.’ And then I draw these boxes around it and then they’re like, ‘Oh, now I see it.’ So, I realise that maybe not everybody sees it the same way as I do. I need to point that out.

Interviewer: Adam Levy

And now I understand that you’re doing this not primarily as a job, just doing it for the work itself. So, what is your motivation to be going through, as you say, hundreds and hundreds of papers?

Interviewee: Elisabeth Bik

The reason I do that is because if a science paper has some problems, other people might try to replicate that result but they might find that it’s impossible to do, and if that is because the science paper was maybe fabricated or falsified, then that is a really good reason to flag that paper so that other people are warned that there’s a concern about that paper. So, I find that very rewarding. I feel I’m sending out a message that there’s potentially something wrong with papers and helping others to see that as well.

Interviewer: Adam Levy

And I understand that it’s not just standalone papers sometimes that you uncover issues with. You’ve sometimes uncovered problems in groups of papers by the same academic groups or authors.

Interviewee: Elisabeth Bik

I have found several clusters of papers that were all authored by the same research group, so sometimes I might find two or three or ten papers, and one time, I even found 100 papers all by the same author and they all had image problems, So, some of these cases can be quite big.

Interviewer: Adam Levy

Now, you’re not the only person who does this kind of work, but you’re unusual in that you do this quite openly under your own name.

Interviewee: Elisabeth Bik

Yes, I do. Most people who do this work are posting under a pseudonym, and that is because this work can be quite risky. I’m, of course, criticising other people, but a researcher might maybe not like that so much and decide to sue me, so I need to be very careful in how I word my concerns about a paper. So, I’ll just say, ‘These two images look remarkably similar. Can you please explain?’ And hopefully that will keep me safe.

Interviewer: Adam Levy

One thing that has been a source of criticism is that you publish your findings publicly. You go to certain social media to talk about the things you’ve uncovered. Is there a risk that this approach could be counterproductive, potentially causing actual problems further down the line for investigations?

Interviewee: Elisabeth Bik

It might be. So, when I flag papers then that could leave the authors with the opportunity to destroy the evidence. But on the other hand, of the 800-something papers that I’ve reported in 2014 and 2015, only 30% of them have been either corrected or retracted and the rest is just not touched upon, so they’re still out in the open with their duplicated images. So, it’s very frustrating that it seems that journals or institutes are not really acting upon these allegations, so I try to do it the official way but now I’m posting these things online because it’s faster. I feel I’m flagging these things and at least I’ve warned people in a fast way.

Interviewer: Adam Levy

Do you get, I guess, frustrated by how long these things sometimes take to resolve?

Interviewee: Elisabeth Bik

Yes, I do. It is very frustrating when I find these problems, sometimes in seconds, where I see a duplicated background or a duplicated photo, and it seems so obvious to me, and I report these things through the journals and then I just don’t hear anything back for years. And that’s why I take things sometimes to Twitter and say, ‘Guys, this is a problem, you should have looked at it.’ But yeah, some people have called that ‘trial by Twitter’, and I can see that, but sometimes nothing has been done, even though I reported it five years ago. This paper is still out there and everybody thought it was fine.

Interviewer: Adam Levy

Now, I think many people might hear about this and hear about someone who’s literally going through and looking at images on a personal level and think that’s quite an old-fashioned way to do it. We’ve got artificial intelligence and image analysis tools now. Why is this something that a human is still doing?

Interviewee: Elisabeth Bik

Because it’s really hard to computerise this. I don’t know. I like to think of the voice recognition of ten years ago. I would say, ‘Yes,’ and the computer said, ‘No, you said no.’ ‘No, I said yes.’ And image recognition is even more complex. It’s much harder than people think. But having said that, that will happen.

Interviewer: Adam Levy

Do you think as the software improves, you’ll be out of work, you won’t have anything to do?

Interviewee: Elisabeth Bik

No, I think it will take a while before such software is completely up and running, and there’s still going to be the need for a human reviewing the results, similarly to plagiarism checkers that are widely used in scientific publishing, a human is still needed to decide that this was an okay reuse of an image.

Host: Shamini Bundell

That was Elisabeth Bik. To read more about her work, check out the profile of her in this week’s Nature. You’ll also find a news piece detailing a brand-new, cross-industry initiative to try and catch image issues before publication. There’ll be a link to those in the show notes.

Host: Nick Howe

Coming up, we’ll be hearing about a chemical mystery surrounding air pollution particles. Now, though, it’s time for the Research Highlights, read to you this week by Benjamin Thompson.

[Jingle]

Benjamin Thompson

The Tully Monster was a bizarre creature that lived over 300 million years ago and looked, to me at least, like a cross between a fish, an ocarina and a sock puppet. This aquatic animal has been the source of debate among palaeontologists for decades as it’s been tricky to work out whether it was a vertebrate, an invertebrate or something in between. To get a better idea, a team of researchers have now studied the different molecular signals found in the fossils of vertebrates and invertebrates from where the Tully Monster was discovered. The team found that fossilised soft tissues from invertebrates had relatively high levels of nitrogen-containing compounds whereas the vertebrate soft tissue had high levels of sulphur-containing compounds. They showed that the molecular makeup of the Tully Monster’s fossil suggests that the creature was a vertebrate, adding yet another twist in the Tully Monster’s tale. Dig up that research over at Geobiology.

[Jingle]

Benjamin Thompson

New research has traced record-breaking air pollution levels in Chile to football fans firing up the barbecue during big games. Chile’s capital Santiago often faces high levels of air pollution in winter, but the city also sees brief spikes in particulate matter that sometimes reach tens time average levels. To get an idea of what was causing these spikes, a team of researchers looked at the type of particles they contained and when they were detected. They found that these pollution peaks coincided with massive levels of barbecuing taking place while football fans watched the Chilean national team play. While efforts to decrease pollution levels in Santiago are focused on emissions by traffic and industry, the team suggests that sporadic emission sources, such as those created by the cooking habits of football fans, need to be considered as well. Head over to Atmospheric Chemistry and Physics to read more.

[Jingle]

Interviewer: Nick Howe

Sticking with the pollution theme, one thing that you may have seen while scrolling through your social media in the past weeks are pictures of air pollution or, in fact, the lack of. Formally smog-filled cityscapes from São Paulo to New Delhi are apparently looking pristine thanks to the near-global lockdown, and it’s not just about improving the view. Air pollution is a serious health issue in cities across the world. Of all the various constituents of air pollution, there’s one thing called fine particles that are particularly problematic. These are particles between a few nanometres and a few micrometres in size.

Interviewee: Neil Donahue

They’re incredibly important for climate and they also kill an enormous number of people around the globe, so something of the order of 7 million deaths per year are attributed to inhalation of fine particle pollution, which is something like 15% of all the mortality around the planet.

Interviewer: Nick Howe

That’s Neil Donahue, an atmospheric chemist at Carnegie Mellon University in the US. Because of these damaging effects of fine particles, researchers like Neil are interested in finding out how they form. Fine particles can enter the air via smoke from burning fuel or take the form of dust particles blown into the air. But they can also be created in the atmosphere itself. Tiny molecules in the air can react chemically to form small particles which stick together, growing until they reach the dangerous fine particle size. But getting that big isn’t easy. Researchers have struggled to understand why these growing particles don’t end up bumping into things and breaking apart again.

Interviewee: Neil Donahue

So, most particles are formed and they’re very jittery, right, so they bounce around a lot like my little dog, and they tend to bump into stuff and what they bump into is bigger particles, and that little particle is gone.

Interviewer: Nick Howe

But the fine particles we see in our atmosphere clearly haven’t bumped themselves into oblivion. The small molecules that form fine particles are able to stay together and grow, and this growth is somewhat unusual.

Interviewee: Neil Donahue

They have a trick up their sleeves that lets them grow up a lot faster than they should.

Interviewer: Nick Howe

It’s this trick that Neil and his colleagues have uncovered and published in Nature this week. It turns out there was a missing puzzle piece – another chemical present in the air that was able to cause rapid growth of fine particles.

Interviewee: Neil Donahue

Fertiliser, it turns out it’s fertiliser, so ammonium nitrate.

Interviewer: Nick Howe

Ammonium nitrate forms in the air when ammonia interacts with nitric acid. Researchers knew that these pollutants were present in the air, but they didn’t realise the significance they might be having on fine particle formation and therefore on public health.

Interviewee: Neil Donahue

The traditional understanding here was that even though there’s ammonia and nitric acid around, it should be in a thermodynamic equilibrium, meaning that there should be very little going into particles or off of particles. It’s sort of in a happy balance.

Interviewer: Nick Howe

Under certain conditions like cold, the balance disappears and ammonium nitrate forms quickly and particles can rapidly grow. According to Neil, this likely explains a lot of air pollution in cities during winter.

Interviewee: Neil Donahue

Especially when it’s quite cold it’s very likely.

Interviewer: Nick Howe

The next steps for Neil are to better replicate the conditions found in an actual city with air turbulence and variable concentrations of molecules across the landscape to get a better sense of how this process occurs in the real world. But even without this, this new mechanism is an important part of the puzzle for scientists trying to study and decrease fine particle pollution in cities. For Neil, this work may also inform policy and help us predict what the future state of our atmosphere might be.

Interviewee: Neil Donahue

We know we need to get rid of the pollution. We just did it by shutting down the entire economy. That won’t persist, unfortunately, but we’ll learn something about ways that we can reduce particle levels and particle mass. But we actually don’t know that as we clean up the atmosphere that the number of particles is going to continue essentially back to the pre-industrial conditions. We’re not going back to the Shire and Hobbiton just because we shut down all of the fossil fuel combustion or whatever we do, and so that’s really where this fits in most to policy discussions, is trying to understand what the future atmosphere will look like.

Interviewer: Nick Howe

That was Neil Donahue from Carnegie Mellon University in the US. The paper he and I talked about can be found in the show notes.

Host: Shamini Bundell

Finally on the show, it’s Briefing chat time. So, for those who might not be familiar, the Nature Briefing is basically a daily pick of science news and stories delivered to you by email, and for the past few weeks, Nick and I have been combing the Briefing for non-corona science news. So, Nick, what have you found this time?

Host: Nick Howe

Well, this time I’ve been looking at black holes, and tell me, Shamini, how close do you think the closest black hole to Earth is?

Host: Shamini Bundell

Oh, not very close. The centre of the galaxy. The centre of the Milky Way.

Host: Nick Howe

Yeah, I’m thinking of one a bit closer than that and, to be honest, I was trying to trick you because whatever you were going to say, as of a few days ago, astronomers reckon they’ve found one even closer. There’s a black hole that’s been found in a constellation called Telescopium that is about 1,000 lightyears away from us.

Host: Shamini Bundell

But black holes are these massive things with a huge gravitational pull, how can we have not noticed that?

Host: Nick Howe

Yeah, I mean they are massive but they also absorb light so they’re kind of invisible. We can only see them if they have an effect on something, and this one is kind of surprising as well because the particular star system that it was found in – HR 6819 – has been studied since the 1980s, but we’ve never seen this black hole before.

Host: Shamini Bundell

So, how do we know it’s there?

Host: Nick Howe

So, in this star system there’s two stars and they move in sort of a puzzling way, and so astronomers started to wonder whether there was a third object there, but they couldn’t see anything, but through various measurements and things like that they’ve determined that there may be a third object there and it’s a very, very massive one, and so the only explanation, really, is that it’s a black hole.

Host: Shamini Bundell

Oh man, I love those kind of astronomical mysteries, but presumably this is still far enough away that we don’t need to panic about being swallowed into nothingness, right?

Host: Nick Howe

It’s definitely far away enough that I don’t think we need to worry about it eating us up tomorrow, but it may indicate that there are many more black holes in the galaxy than previously thought because if we assume that we’re not in any way special and there’s one that close to use, there’s probably many more throughout the galaxy that we’ve not noticed.

Host: Shamini Bundell

Well, that’s a lot more sort of grand and dramatic than my story for this week, which is about worms.

Host: Nick Howe

About worms… are they wriggling in the garden?

Host: Shamini Bundell

No, not garden worms. They’re little sludge worms that apparently, you can just buy them as fish food, and a bunch of scientists decided to use them to study fluid dynamics and the viscosity of active systems.

Host: Nick Howe

That’s not a bunch of words I ever thought would be in a sentence together. What is going on here?

Host: Shamini Bundell

Well, apparently, active systems basically means if you’ve got fluid with particles in it that are active, that are moving off their own steam, so for example, swimming bacteria or maybe inside the cell sort of cellular filaments that are moving around, the fluid behaves in a different way than if it was just sort of passive molecules getting carried along, and so researchers sort of need to study this and sort of need to understand this kind of fluid and how it moves and its viscosity, and so they’ve picked, as a sort of model, these worms that they can then put in fluid, measure the viscosity of this worm solution, and see what difference it having active wriggling things in it makes.

Host: Nick Howe

That does make sense if these active things are having more of an influence on viscosity, but still, I’d keep coming back to why worms?

Host: Shamini Bundell

I think just because they’re small and wriggly and cheap, but the important thing is that they tried it out with the worms and it seems to work in the sense that they found that when the worms were active, the viscosity of the fluid was measurably different than when the worms where passive, so this could be a useful model.

Host: Nick Howe

Oh, okay, so this could be like a way to study these active things in the future.

Host: Shamini Bundell

Exactly, so they can actually control the worms. So, for example, if they want them to stop wriggling, they add alcohol to the water that they’re in – these poor worms – and then they become temporarily motionless. So, they’re a lot bigger than cell filaments, they’re more controllable so they can use this to maybe study the equivalent of what’s happening inside cells.

Host: Nick Howe

I never thought I’d be discussing worm viscosity on the podcast but thanks for that, Shamini. And listeners, if you’d like more short snippets of science like that but instead delivered to your inbox, then make sure you check out the Nature Briefing. We’ll put a link to that and also the articles we discussed in the show notes.

Host: Shamini Bundell

That’s all for this week. There’s just time to tell you about our latest mini documentary. It’s all about how fake news and misinformation surrounding COVID-19 are spreading around the world and what we can do about it. We’ll put a link to that in the show notes. I’m Shamini Bundell.

Host: Nick Howe

And I’m Nick Howe. See you next time.