ChatGPT can write a paper in an hour — but there are downsides

Benjamin Thompson

Hi, Benjamin from the Nature Podcast here. We're going to change things up a little bit this week. And we're going to take a deep-dive into some stories that have been in the Nature Briefing over the past few days. And joining me to do so are Noah Baker. Noah, how're you doing today?

Noah Baker

I'm very well. Thank you. Looking forward to chatting about Briefing items.

Benjamin Thompson

And Dan Fox, Dan. Hello.

Dan Fox

Hi. How's it going?

Benjamin Thompson

Yeah, it's going very, very well. Well, listen, three stories to go through today. Noah, why don't you go first? You're talking about AI? Right?

Noah Baker

Yeah. So I think probably one of the hottest topics at the moment is generative AI. And this is a story all about ChatGPT, the poster child of said movement in the news. But specifically, this is about some researchers that have used ChatGPT to generate an entire paper from scratch. And so there's a news article in Nature that I've been reading about whether or not that paper is any good, what the limitations are. Should we be scared? Spoiler alert. No, not really yet. But maybe it could be useful as a tool in the future.

Benjamin Thompson

When you say an entire paper, do you mean what abstract, introduction, methods, conclusion? The whole business?

Noah Baker

Absolutely the whole thing. So it did the analysis with some prompts. And it created an article that was quote, fluent, insightful and presented in the expected structure for a scientific paper. But it did it with a lot of help. So the researchers wrote a program which prompted ChatGPT to do this in stages. And I can talk you through what those stages were from the sort of beginning of the research all the way through to the published paper in the end, but this entire process took less than an hour, which is I guess, a big part of why this is something that's worthy of talking about now.

Benjamin Thompson

Maybe we start off with, with what was this paper about? And how did the researchers go about making it?

Noah Baker

Yeah, so it's two researchers from Israel from Technion in Haifa. And they essentially started with the data, right. So they started with some freely available data, CDC telephone survey data about 250,000 people talking about their diabetes status. And then they wrote this program, which essentially prompted ChatGPT in an automated way. So it said, 'Hey, ChatGPT write some code that could uncover patterns in this data'. And so it wrote some code to uncover some patterns. And its first attempt, it was a nightmare. Loads of errors, things that didn't work. But the program took those error messages, fed it back into ChatGPT and said 'do better'. And eventually it did, it produced code that could be used to explore the dataset. And then they said, 'Okay, cool. You've got a structured data set now, look for some things'. And so it said, 'why don't you explore how physical activity and diet are related to diabetes risk', so it generated more code. And it found yep, sure, eating more fruit and vegetables and exercising is linked to a lower risk of diabetes according to these data. And then still via this program, it prompted ChatGPT to summarize those findings in a table, then write the whole results section. Then step-by-step, they got it to write the abstract, the introduction, the methods and the discussion section of the manuscript. And then after that, refine all the text so that it could go through and refine it all. So there was lots and lots of prompts. But what they ended up with is in less than an hour, they've essentially done some research and written a paper, or rather ChatGPT had done some research and written a paper.

Dan Fox

I mean, that's really interesting. But I guess diabetes, and correlation with eating fruit and veg and physical activity is not the most, not the most novel piece of research.

Noah Baker

Yeah, for sure. I mean, this is one of the things that the researchers quoted in this story have said, you know, this is not something that's going to surprise any medical expert. It's not close to being novel. But really importantly, one of the things that ChatGPT did do when it was generating the text for this paper is it used quotes like this "addresses a gap in the literature", which this very much isn't. There's very little novel about this finding, in terms of our understanding of diabetes and risk factors for diabetes. And this is one of the big concerns that scientists have about using ChatGPT for this kind of purpose.

Benjamin Thompson

So it's not necessarily what it found then. But it's the fact that it could be done that that is so interesting about this work.

Noah Baker

Yeah. I mean, researchers are trying to find ways that ChatGPT could be used as a tool, essentially, it's like, how could it help the scientific process? How could it be used to make things more efficient for researchers to be able to do more work, more insightful work, see things that perhaps aren't seen otherwise? But there are lots and lots of questions around whether or not you can do that. This approach of trying to get it to create an entire paper means that you end up with a paper that's not super novel or exciting, and equally has lots of made up citations. It's another thing that it did, it hallucinated concepts it didn't really understand, well, because ChatGPT doesn't understand anything, it doesn't really understand any of that context. And so it needs a lot of support from the researchers, however, a lot happened in an hour. So there's a discussion in this piece about how it might be useful and where the problems might lie. And there are suggestions that it could be very useful to just throw up a bunch of potential avenues for research, or perhaps things like summarization of results could be something that ChatGPT could do very quickly. Or perhaps even things like writing code is something that ChatGPT can do very quickly. So there are places it could be useful, but building an entire paper, perhaps not. And there are other reasons that this could be concerning. So one of the things that's that's mentioned here in this story is things like P-hacking. So P-hacking is when scientists just test a bunch of hypotheses on datasets, hoping that one of them's going to work and then they just only talk about that thing. And in a world where you can essentially say, 'Hey, ChatGPT, do some P-hacking for me' because it can look at so many things so quickly, then potentially you end up with a situation where you're using an AI to enhance this relatively dishonest practice. So concerns abound, but potentially a very powerful tool. There's just a lot of kinks to work out before we can really get there and use it appropriately. And that includes in publishing research as well.

Benjamin Thompson

Yeah, I mean, quite the quite the sort of two-way street and for a number of reasons, right, because you said you could do it in an hour. And I can almost hear some of the researchers listening to this podcast, rubbing their hands going 'writing papers is boring and takes ages, this will be super useful'. But the other side of that is, it could be used to just churn out absolute dross, right. And these kind of, we've heard about paper mills before publishing fake research.

Noah Baker

Yeah, absolutely. And I think there's a lot to be watched here, you know, there's a lot of onus on journals like Nature and other journals to continue to develop their process for how AI use can be declared. I mean, this is something that is very important to try to declare how AI tools are used in papers so that we can better see and spot places where there may have been biases or hallucinations, inputted by the generative AI. And that's something that editors can do. It's something that scientists need to do, because the stuff that's created can be so fluent ChatGPT can make stuff that sounds so convincing, that even experts in the field can sometimes struggle to see places where it's made stuff up, or it's seen things that aren't there. So, lots to watch, but perhaps smaller steps at the moment, so we can make things actually useful.

Benjamin Thompson

Well, I have a feeling that isn't the last we're going to hear about ChatGPT on the Nature Podcast. But let's move on to our second story today. And it's a story that I've brought, and I read about it in Bloomberg. And it's about some records that have been broken, but sadly, not in a good way. It's how data suggests that last week, the average global temperatures across the world broke the existing record, not once but twice.

Noah Baker

Yeah, this is the record in like human history, right, a really significant record to break twice in one year.

Benjamin Thompson

Yeah it's not a good one. And this is according to data from the US National Centers for Environmental Prediction. In terms of the numbers then, the average worldwide temperature reached 17.01 ° on Monday last week. And that was then shortly broken on Tuesday, when they reached 17.18 °C. This record isn't official, it has to be said. But I think it is quite a sobering indication of where things are right now.

Noah Baker

Yeah, I mean, point two-degree rise doesn't seem much on any given day. But across, you know, when you start to bring in averages, and also when you start to think about this as a global temperature, it's a really significant jump in temperature. Right?

Benjamin Thompson

Absolutely. And this comes against a backdrop of a lot that's going on across the world right now. I mean, there's been huge heatwaves in China with records are broken there. The hottest June in the UK on record, Antarctica had its highest temperature recorded in July, at 8.7 °C, heatwaves in Spain, you know, across the land and sea.

Dan Fox

So is there a sense of some of the reasons behind it being the hottest day on record sort of two, two days in a week?

Benjamin Thompson

WellI mean, you'll be you'll be shocked to know that climate change is clearly playing a part in the fact that the world is warming. But there's something else involved here as well, and that is El Niño, which is this sort of global weather phenomenon that happens every sort of two to seven years.

Noah Baker

Right, So we have a record broken twice in one week. And then at the same time, we have a new El Niño cycle starting, which we know is associated with particularly high temperatures. This doesn't sound like good maths, is this gonna be a particularly big El Niño cycle?

Benjamin Thompson

Well, that's a great question. And it's actually yet to be decided. Researchers don't quite know yet. But what I can say is last time, there was a strong El Niño that corresponded to the hottest year on record, and that was in 2016. So from what I've read, there is a sense that really, maybe 2023, or maybe even 2024, when El Niño really kicks in, could break the record for the for the hottest year.

Noah Baker

And with hot temperatures also comes more extreme weather and loads of other potential knock ons, right?

Benjamin Thompson

So this changes weather patterns across the globe, due to a slackening of the trade winds above the tropical Pacific Ocean. It could lead to droughts in Australia and southeast Asia, increased rainfall in places like the Horn of Africa. And that can lead to localized flooding, which can damage crops. But also other things as well, which maybe not entirely obvious. For example, strong rainfall could lead to an increase in the number of mosquitoes, which could then lead to more diseases being spread. Something similar was seen in Peru, weather pattern change was linked to a severe outbreak of the disease, dengue, which of course, is a viral disease spread by mosquitoes. So there's a great deal going on here, which could have really quite severe effects on life and livelihoods. And of course, you know, in the face of a warming world, one of the only ways we can get on top of this is reducing emissions, right. And of course, COP28 is coming up in November. And it'd be interesting to see in the face of of what is potentially going to happen, whether that will affect negotiations and discussions at that meeting at all.

Noah Baker

I wonder whether or not this will change the urgency with which people discuss things. Anyway. Dan, do you have anything a little bit more uplifting? Perhaps maybe something a little bit less doom-and-gloom to talk about as a final story for the show.

Dan Fox

Yeah. So this is I think it's quite a positive research development. So some researchers have published a paper that's come out in Nature Aging, where rhesus macaques were injected with a protein called klotho. And that increased their cognitive ability, quite significantly, for well over two weeks, and is potentially a treatment for degenerative diseases like Alzheimer's and Parkinson's and dementia.

Noah Baker

So, I know this is probably trivializing, but it sounds to me like scientists have injected monkeys with a super smart-serum.

Dan Fox

I mean, it's very much the plot of 2011's Rise of the Planet of the Apes, starring James Franco. But this is a bit more exciting in that...

Noah Baker

It's real?

Dan Fox

But I mean, it's probably less exciting. There's no talking monkeys. But they are they are 15% better at finding snacks.

Noah Baker

I was interested in that. So what is it that they've actually done to measure cognitive ability here? Is this a series of behavioral tasks?

Dan Fox

So the tests the researchers used, involved putting a treat reward into one of a variety of different wells, so different sort of geographical location, spatial locations. That experiment was hidden from the monkeys for a period of time. And then when they returned, they had to find the treat that they'd seen hidden previously. So the researchers say, this is similar to a task like finding your car in a busy car park, and it's something that specifically gets harder as you get older. So the rhesus macaques were older macaques as well. So the average age of these monkeys was around 22 years old, which the authors of the paper say corresponds to being about a 65 year old human. And before they were injected with this protein, they successfully found the treat in the correct well 45% of the time,after an injection with this protein klotho, they found that 60% of the time. So a 15%, increase in their ability to find their car in a car park, find their their tasty snack, and that lasted for two weeks.

Noah Baker

Klotho that is a protein, I have to say, I've never heard of. What is klotho, and what's the scientists know about it?

Dan Fox

So klotho is protein that's expressed in the body in a few different places, it comes in a few different varieties. And these researchers are looking at alpha klotho. So it's something that actually decreases in the body as you age. And the researchers have called it a 'longevity factor'.

Noah Baker

Hence it becoming a target to try to understand what's going on with aging, because it's something that seems to be correlated with aging.

Benjamin Thompson

Wow, what was going on in the brains of these monkeys then? Because it seems like fairly straightforward to inject them with protein, but I'm sure there's more to it than that.

Dan Fox

Well, I think that's one of the fascinating parts of the story is they don't really know. The actual mechanics of how this protein is working, is not very well understood. So it's previously been shown that they get a similar effect from injecting this protein into mice, increases in synaptic plasticity and cognition and in resilience against aging and Alzheimer's and Parkinson's disease. But they also know that this protein doesn't pass the blood-brain barrier. So there's some sort of intermediary that's communicating this effect into the brain. And that isn't well understood. And something else that complicates matters, and shows that there's kind of a more complicated process going on then the researchers initially thought is that, while a low-dose works very well, and gives this two-week effect, at higher doses, there's no cognitive improvement. And they think that possibly even higher doses, there'd be cognitive impairment. So it's not really clear exactly what's going on inside the body.

Noah Baker

Well, so you can improve a monkey memory using a protein without really understanding how the protein works. Could you improve a human memory by doing a similar thing?

Dan Fox

The short answer is, is yes. So it's already been shown that individuals that have elevated klotho levels due to natural variation show improved cognition and decreased risk of dementia and Alzheimer's. So there's already been a demonstrated benefit to klotho in humans. And the researchers behind this work have said that there's a very strong reason to jump into human clinical trials now.

Noah Baker

So lots of excitement in the field. But knowing scientists as I do, I can imagine there are certainly people that are saying, Sure, clinical trials will be interesting. But we do really need to understand this mechanism a little bit better, right?

Dan Fox

Absolutely. And there are research saying that a better understanding of the proteins mode of action will be crucial for realizing its clinical potential and the authors of the paper, and the authors also mentioned in the paper, that identifying these intermediaries that are taking the message from this klotho across the blood-brain barrier into the brain itself, is going to be key.

Benjamin Thompson

When they do work out what's going on with that intermediary, please come back on and let us know. But let's leave it there for this week's Briefing Chat and listeners, for more of these stories, look out for links in the show notes, where you'll also find a link where you can sign up for the Nature Briefing to get even more science stories like this delivered directly to your inbox. But for this week, all that's left to say is Noah and Dan, thank you so much for joining me.

Noah Baker

Cheers, Ben.

Dan Fox

Yeah, thanks for having me.