Podcast: Antibiotics in orchards, and rethinking statistical significance

Host: Benjamin Thompson

Welcome back to the Nature Podcast. This week, we’ll be hearing about a proposed plan to spray antibiotics on citrus crops…

Host: Nick Howe

And finding out why some statisticians think we should throw out statistical significance. I’m Nick Howe.

Host: Benjamin Thompson

And I’m Benjamin Thompson.

[Jingle]

Interviewer: Benjamin Thompson

Wednesday 20th March is the first day of spring here in the northern hemisphere, and many of our plants are waking up from their long winter slumber. For our first podcast piece this week, we’re going to head to Florida’s orange groves where the first flush of leaves will soon be emerging from the trees. However, all is not well, and many of these trees are at risk of a serious bacterial disease. The US Environmental Protection Agency (EPA) is in the process of allowing growers to use two antibiotics to treat this disease – a move that’s got some public health researchers concerned. This week, science journalist Maryn McKenna has written a feature for Nature describing what’s going on. I gave her a call and she gave me an overview of the state of orange farming in Florida.

Interviewee: Maryn McKenna

The situation is really quite desperate because the trees are under attack by a disease that’s usually called citrus greening, which is an invasive disease that’s been in the orange groves for more than a decade now, and it ruins the trees. It causes them to drop fruit early, it causes them to lose their leaves, and it’s taken hundreds of thousands of acres of orange cultivation out of production already.

Interviewer: Benjamin Thompson

And I imagine this is having severe economic impacts for the farmers involved?

Interviewee: Maryn McKenna

Absolutely, the growers have been really quite desperate, so finally now they have turned to this solution that looks sort of like a last-ditch effort to them, but is quite appalling to public health advocates, which is to spray the trees with antibiotics that are also used in human medicine, antibiotics that would be very familiar to an average physician which are streptomycin and oxytetracycline.

Interviewer: Benjamin Thompson

And this is what your feature is looking at, Maryn, but why do farmers want to do this in the first place?

Interviewee: Maryn McKennaSo, the disease that causes this citrus greening is caused by a bacterium and the bacterium moves through essentially the vascular tissue of the tree, the channels under the bark. So, it’s quite hard to get to with any kind of treatment and the growers have concluded that spraying the trees with antibiotics when the leaves are new and therefore somewhat permeable will get antibiotics into the trees’ circulatory systems in such a way that it will if not cure the trees, at least sort of beat back the amount of bacteria that’s in them and eke out a few more harvests for any tree.

Interviewer: Benjamin Thompson

Which I guess on the face of it makes sense, right? I mean is this the first time that this has been tried or have antibiotics been sprayed onto other fruits?

Interviewee: Maryn McKenna

Well, the interesting thing is that it’s not the first time, and it turns out that this is quite an under-recognised aspect of fruit agriculture. In the United States, apple and pear production have been under threat for many, many years from another bacterial disease that’s generally known as ‘fire blight’, and the solution to fire blight is the same as this potential solution for citrus greening, which is to spray trees with antibiotics. Now, it’s nowhere near as much antibiotic, but there is some history of using antibiotics that are also used in human medicine against plant diseases and so in that sense, the citrus growers do have kind of a leg to stand on because there’s a past precedent that they can point to and say, oh look, this has been done before.

Interviewer: Benjamin Thompson

And what evidence is there that this solution will be effective?

Interviewee: Maryn McKennaWell, this is the tricky part. In fact, there is not a large amount of research that has been peer reviewed and published on this use, not for citrus. That’s not to say there has been no research, it’s just that the research isn’t public. Now, there are some records of some field trials, which is not the same as a classically conducted experiment, that have either leaked out or have been presented to conferences and so forth, so it’s possible to find some research. And in those trials, it looks like the antibiotics are somewhat effective in slowing the progression of this disease, but they’re not 100% effective and that makes public health people looking at the planned use of these antibiotics say that the downside risk of this is really more than you ought to be taking on.

Interviewer: Benjamin Thompson

Well, what are the potential downsides then Maryn? What are public health researchers concerned about that may happen as a result of using these antibiotics?

Interviewee: Maryn McKennaSo, what’s inspiring the concern about this is the long-standing realisation that the use of antibiotics in animal agriculture caused antibiotic-resistant bacteria to arise on farms and then flow away from farms on a variety of pathways that were quite difficult to monitor. People who look at the use of antibiotics proposed in citrus groves are saying we are running very similar risks here and yet we don’t have the kind of detection mechanisms that were eventually set up for animal agriculture, so we won’t be able to know whether we’re making the same mistake again.

Interviewer: Benjamin ThompsonOne could argue though that an orange grove and, say, a pig farm are two very different things, so a detection system that works for one won’t necessarily work for the other.

Interviewee: Maryn McKenna

Right, that’s exactly correct, and for people who have researched this, that’s the contention that they make in response to concerns about this, is that the setting is so different that we shouldn’t expect the antibiotics and the bacteria they’re impacting to behave in the same way, that antibiotics sprayed on the surface of a tree and sliding off onto the ground or being rained off are in nowhere near the same concentration as antibiotics in a pig’s poop going into a manure lagoon holding a lot of other poop, and therefore there won’t be the same risk that the antibiotics that are present are going to turn the bacteria present toward a higher percentage of resistance.

Interviewer: Benjamin Thompson

And how will we know if spraying antibiotics onto trees has worked or if it has limited efficacy?

Interviewee: Maryn McKennaWell, we’ll know it works if the groves where the drugs are sprayed have, I suppose, a lower amount of the visible signs of this disease, but the problem is not just how will we know whether this remedy is working and to what degree it’s working, the larger problem is how will we know whether antibiotic-resistant bacteria are occurring in the ecosystem of the groves. Unless someone sets up some sort of surveillance projects to find that out, we won’t know and there’s no sign at this point that any such surveillance is going to be set up.

Interviewer: Benjamin ThompsonSo what’s to be done then? I mean if we want to protect these orange trees but also protect these precious antibiotics, where’s the middle ground? What future opportunities are there to do both?

Interviewee: Maryn McKenna

So, what the industry says it’s doing is looking for other remedies and it is viewing antibiotics just as a stopgap, and to be fair, citrus is a massive industry and it’s very heavily supported. Both the industry itself and the states where the industry is based really want to see continue because of the size of the market and the size of the labour force. So, there is money going into research for things like biological controls like wasps that you could send after the insect that transmits this disease, or other chemical compounds that are not antibiotics, or even probably the ultimate goal is breeding a tree that would have natural protection against this infection, but all of those things are 5-10 or more years down the road, and so the hope of the industry is that they can get into and out of antibiotic use without causing any unintended side effects with enough time left to achieve those newer remedies and get them into the field.

Interviewer: Benjamin Thompson

That was science journalist Maryn McKenna. You can read her feature over at nature.com/news.

Host: Nick Howe

Coming up in the show, we’ll be hearing about the latest winner of the Abel Maths Prize. But first, here’s the Research Highlights read this week by Anna Nagle.

[Jingle]

Anna Nagle

It’s said that parenting takes a village and that many people are needed to support new parents as they raise their children. In lots of species, however, that’s not the case. Among insects, for example, childcare is often left exclusively to the mum. But researchers in Prague have found a particular type of bee that’s bucked the solo parent trend. The scientists studied 460 nests that were home to a type of carpenter bee, and found that almost all of them held male-female pairs. Unlike honey bees, carpenter bees don’t have a queen, and any female can reproduce. While the female carpenter bees went out foraging, the males stayed home and protected the nest. The researchers also found that many of the stay at home males weren’t even related to the offspring — how very progressive. The females of this particular bee species are polyandrous and take several mates, but if a male sticks around for a little longer, they get more mating opportunities. The scientists even put a number on it – the male bees average 0.6 offspring for every 7 days of guard duty. Buzz on over to the Proceedings of the National Academy of Sciences for more on that research.

[Jingle]

Anna Nagle

Scientists have created liquid marbles that can self-assemble from a pile of powdered plastic. Liquid marbles are typically made by encasing water droplets in spherical particles and can be used as microscopic ball bearings, miniature vessels for chemical reactions or small sensors. Now, a team of researchers has come up with a different way of making liquid marbles using tiny hexagonal plates made from a powder of the kind of plastic used to make drinks bottles. When the researchers placed water droplets onto the plastic, the plates surrounded them, creating marbles that look like little lumpy disco balls. When the researchers then exposed the marbles to a vapour made of the main ingredient in superglue, they found that the marbles kept their shape, even once the water droplet inside had evaporated. Nudging their tiny liquid marbles with little external pressure made them shapeshift, with the scientists able to create cylinders, dumbbells and even letters. Roll on over to that research in Advanced Functional Materials.

[Jingle]

Host: Nick Howe

You have a science background, right Ben?

Host: Benjamin Thompson

Oh yes, I’ve got my PhD. I was very much a proper scientist.

Host: Nick Howe

Brilliant, so you must have done all sorts of statistical testing in your research then. I bet you know your ANOVAs from your ANCOVAs and your t-tests from your Mann-Whitney Us?

Host: Benjamin Thompson

Um, I recognise some of those words.

Host: Nick Howe

Well, don’t worry. Just to jog your memory, these are some of the statistical tests that are often used to calculate statistical significance. So, let’s say you’re looking at two sets of data and have found a difference between the two groups. How would you know that that difference is statistically significant?

Host: Benjamin Thompson

I know this! You’d need to calculate the P value to work it out, right?

Host: Nick Howe

Yeah, that’s right. The P value can be thought of as a measure of how likely you are to see this difference by chance, so a low P value means it’s unlikely to be coincidental. Results are often deemed significant if the P value is below a certain threshold.

Host: Benjamin Thompson

A threshold like 0.05?

Host: Nick Howe

Exactly, but what is so special about 0.05? I asked Regina Nuzzo, a statistician and writer from Gallaudet University.

Interviewee: Regina Nuzzo

Nothing, really. It’s an accident of history, believe it or not. The statistician in the early 1900s who’s most widely attributed for developing the P value, said 0.05 seems good enough to me, and it kind of stuck.

Host: Benjamin Thompson

Wait, so 0.05 is completely arbitrary?

Interviewer: Nick Howe

Well, it can be useful to have a threshold, but statisticians have been saying for a long time that whilst P values are a useful rule of thumb, they’re often misunderstood and misused when we talk about statistical significance. Now, there’s a Comment this week in Nature saying we should get rid of the concept of statistical significance altogether. I spoke to one of the authors of the Comment, Blake McShane, a statistician from Kellogg School of Management, about why he thinks statistical significance isn’t that significant.

Interviewee: Blake McShane

Basically, the idea is we’ll see some slide and usually there’ll be a graph and it’ll be comparing two or more groups, maybe a treatment group and a control group, and you’ll see the two measurements are not exactly equal. But the speaker will then tell you that in fact, these two things which the naked eye shows you are in fact different, there’s no difference. And the reason they do is because the difference failed to attain statistical significance.

Interviewer: Nick Howe

The problem here is that by crossing an arbitrary threshold and categorising results as significant or non-significant, researchers can be misled in their interpretation. They may consider differences to exist when they don’t, or vice versa.

Interviewee: Blake McShane

Even if the underlying statistics that led to the categorisation are perfectly in order, the very act of cutting them at some threshold, it then causes people to think that these items are in fact categorically different.

Interviewer: Nick Howe

Blake also tested scientists and undergraduates on their ability to decide whether there were differences in data. They were presented with significant and non-significant P values. When P values were non-significant, many respondents became blind to real differences. It could be that the way we’re being taught statistics leads to this categorisation.

Interviewee: Blake McShane

If you look at undergraduates and break them up by those who have taken a statistics class versus those who have not, those who have taken statistics classes make the error at the rate that the other scientists do, which means this is sort of deeply ingrained from the get-go in terms of statistical education. Interestingly, the undergraduates who have never taken a statistics class do not make this error because they don’t know what a P value is, they don’t know what statistical significance is, so when you present it, they ignore it and just say is 8.2 bigger than 7.3?

Interviewer: Nick Howe

Rather than dumping results into one of two categories and using that to decide whether or not an effect is interesting or even real, Regina and Blake think that statistics should be used thoughtfully. Reporting all the data, making clear the uncertainty in estimates and presenting P values precisely as a continuum, may help avoid the problems that come with categorisation. There are concerns, though, that humans are great groupers. We love putting things in boxes, so whatever subtle results are observed, humans will inevitably find a category to put them in. Regina, though, is optimistic that we can learn not to categorise.

Interviewee: Regina Nuzzo

I think as a society – and certainly many individuals – we’re now becoming more comfortable with acknowledging the shades of grey that exist in the real world. So, I wonder if now is a very good time, it’s the right time, for us to be looking at P values and saying you know what, it is continuous.

Interviewer: Nick Howe

So, what are the alternatives? Whilst Blake and his co-authors wish to put statistical significance out to pasture, there is utility in having some kind of threshold to assess the quality of publications. Regina has some thoughts on potential ways forward, in particular the idea of journals using ‘registered reports’.

Interviewee: Regina Nuzzo

Researchers can submit their entire study and data protocol before they’ve collected any data, submit that for peer review, and if accepted, the researchers get an in-principle publication no matter what their results are. So that means no matter what P value I get, if I’ve done it smartly then I get a publication.

Interviewer: Nick Howe

Many other solutions have been proposed, but so far, the concept of statistical significance has stuck. If categorisation is counter-productive then Regina thinks that we’ll need systemic change.

Interviewee: Regina Nuzzo

You can tell people what’s right and wrong and they can know that intellectually, but if they’re surrounded by a system where the incentives and rewards and all their peers are doing something different, then it’s very hard to shift them into doing the right thing. Journals and labs have been putting into place various policies and innovations that support using P values as they were originally intended and I think that’s very exciting when we can change the surrounding environment to make it easier to do the right thing.

Interviewer: Nick Howe

That was Regina Nuzzo from Gallaudet University in the US. You also heard from Blake McShane from the Kellogg School of Management, also in the US. You can read Blake’s Comment piece at nature.com/opinion.

Interviewer: Benjamin Thompson

Finally then this week, it’s time for the News Chat and I’m joined here in the studio by Davide Castelvecchi, senior reporter here at Nature. Davide, hi!

Interviewee: Davide Castelvecchi

Hello Ben.

Interviewer: Benjamin Thompson

Our first story today, Davide, is about this year’s winner of the Abel Prize in Mathematics.

Interviewee: Davide Castelvecchi

Indeed, and we’re very excited that Karen Uhlenbeck, a legend of mathematics, has won the Abel Prize. She is the first woman to have won it since it was first awarded in 2003. This is a prize for mathematics which is awarded by the Norwegian Academy of Science and Letters, and it’s modelled after the Nobel Prize.

Interviewer: Benjamin Thompson

Right, and what is her field of mathematical study?

Interviewee: Davide Castelvecchi

Uhlenbeck is considered a technical wizard in the field of partial differential equations which are the equations that link certain quantities to their rates of change, and they appear basically everywhere in maths and also in physics.

Interviewer: Benjamin Thompson

And I understand that bubbles are quite central to some of her work as well?

Interviewee: Davide Castelvecchi

Yes, so in fact when I talked to her, she told me that her favourite result in her long and distinguished career had to do with soap films and this phenomenon she discovered which is called bubbling.

Interviewer: Benjamin ThompsonWell, pray, tell Davide - what is bubbling?

Interviewee: Davide Castelvecchi

So, she discovered that when you have a soap film that is trying to minimise its energy, sometimes you have some points where the energy seems to become infinitely concentrated. So basically, the theory becomes intractable, but what she discovered is that what really happens is that there is a bubble splitting off from the soap film.

Interviewer: Benjamin Thompson

Right, I mean, I can conceptualise a bubble splitting off a soap film, but what context does this maybe have outside of that?

Interviewee: Davide CastelvecchiIt turns out that this idea had applications not just in the same context where she studied soap films but also, for example, in what physicists call field theory, and specifically, gauge field theory, which is kind of a generalisation of the classical theory of electromagnetism which is also used in the standard model of particle physics.

Interviewer: Benjamin Thompson

And as I understand it Davide, her work has been at the foundation of lots of other fields as well?

Interviewee: Davide Castelvecchi

Yes, and this is true in pure mathematics as well as in mathematical physics, and there was a time in the 1980s when physicists started to find the work of mathematicians such as Uhlenbeck useful, and vice versa.

Interviewer: Benjamin Thompson

Well, a worthy winner no doubt. What has she said about winning this award?

Interviewee: Davide Castelvecchi

Well, she was justifiably proud of it and also surprised, she said, because she’s retired and she hasn’t been very active. She was ill for a while, she was kind of out of the loop, but she still has an affiliation with the Princeton Institute for Advanced Study and in fact, she just released a new paper based on research she did with a co-author.

Interviewer: Benjamin ThompsonAnd also in your article Davide, you mentioned that at Princeton she does a lot of work to advocate for women in mathematics.

Interviewee: Davide CastelvecchiYeah, so Uhlenbeck was the founder of the Women in Mathematics Program at the Institute for Advanced Study and has been very active in promoting and inspiring women and girls who want to go on to become mathematicians. It’s not a role that she took on lightly. She was part of a generation of women who broke the glass ceiling in mathematics and academia, but then she and others realised that there was a lot more work to be done. But she said that she hopes that this prize, which has a lot of recognition, will help her to inspire even more women and girls in the future.

Interviewer: Benjamin ThompsonFor our second story today, Davide, we’re back on the hunt for dark matter, the universe’s missing mass.

Interviewee: Davide CastelvecchiThis has been a long ongoing saga for 20 years. An experiment based in Italy in an underground lab has been seeing this seasonal pattern in its data, this up and down, peaking in June and bottoming out in December and they’ve been saying this is dark matter, this is a sign of dark matter. And basically, at this stage nobody else believes them, but still researchers now are for the first time, trying to actually reproduce the exact same kind of experiment and get to the bottom of it.

Interviewer: Benjamin Thompson

And this is something you’ve been following. There’s been some admittedly not peer reviewed results from two groups trying to replicate the work – what can you tell me about them?

Interviewee: Davide Castelvecchi

So, one of the two experiments which is based in the Pyrenees in Spain has posted its first results of their search for this seasonal effect, and another one which is based in South Korea, also underground, has announced its first results at conferences.

Interviewer: Benjamin Thompson

And do these results back up the original finding that this seasonal dark matter pattern exists?

Interviewee: Davide Castelvecchi

The answer is a definite maybe. So far, there is no definite sign of the original signal being confirmed, but the good news is both of these experiments are basically working as advertised, as planned, and so hopefully within a year or so, we should get at least some hint of a result.

Interviewer: Benjamin Thompson

And what are the researchers involved in this hunt saying?

Interviewee: Davide Castelvecchi

The Italy-based experiment, which is an Italian and Chinese team, all along they’ve been saying we are absolutely sure that we are right and there’s nothing these other experiments can do to disprove it – basically, if they disagree with us it’s because they are wrong. And other people in the field – obviously, there’s a range of opinions – but I think it’s fair to say that a lot of people have strong doubts that dark matter has been discovered because there’s a number of other different kinds of experiments that should have seen it by now but they haven’t.

Interviewer: Benjamin Thompson

Well, hypothetically, if dark matter is discovered, what would this mean for physics as we know it?

Interviewee: Davide Castelvecchi

If it is there, it will be one of the biggest discoveries of the century. For almost 100 years, people have had hints that there’s this mysterious, invisible matter making galaxies go round, but all attempts so far have gone nowhere so it would be huge.

Interviewer: Benjamin Thompson

But if it’s not there?

Interviewee: Davide Castelvecchi

If it’s not there then we’re back to square zero. A lot of theoretical physicists are trying to reimagine what dark matter could be because there’s been so many negative results.

Interviewer: Benjamin Thompson

Well, Davide, we’ll have to get back to you when the results come out to see if we’re back to square zero or not. In the meantime, listeners, you can head over to nature.com/news to read more on these stories.

Host: Nick Howe

That’s all for this week’s show. If that hasn’t been enough to scratch your science itch, head over to youtube.com/NatureVideoChannel where there’s a new video on biologically inspired robots that move by mimicking cells. I’m Nick Howe.

Host: Benjamin Thompson

And I’m Benjamin Thompson. See you next time.