Podcast: An AI masters the video game StarCraft II, and measuring arthropod abundance

Host: Benjamin Thompson

Welcome back to the Nature Podcast. This week, an AI becomes a video game Grandmaster…

Host: Shamini Bundell

And the disappearance of our arthropods. I’m Shamini Bundell.

Host: Benjamin Thompson

And I’m Benjamin Thompson.

[Jingle]

Host: Benjamin Thompson

First up, reporter Nick Howe has been finding out about a new challenger in the world of online gaming.

Interviewer: Nick Howe

When you were young, how did you learn? School, of course, but also likely through playing games. Whether it was a friendly game of football or an unfriendly game of Monopoly, play allows humans to acquire skills that they’ll go on to use in later life. But it’s not just humans who need to learn life skills – artificial intelligences do as well. By getting them to learn by playing, researchers think that this could be a better way for AIs to understand the world around them. AIs have mastered board games like chess, Go and even video games like Super Mario, but these games have relatively simple rules. For researchers to develop a more complex AI, they need a bit more of a challenge. Enter StarCraft II. In basic terms, the aim of StarCraft II is to build an army to overwhelm your opponent, who is trying to do the same thing. To actually accomplish that, you need to direct your units to gather resources to allow you to build buildings, which in turn will allow you to build more units for your army. Now, beating an opponent in StarCraft is particularly difficult for AIs because there’s lots of choices that can made at any one time. Do you want to gather more resources, build a unit, move some of your army to attack an enemy, develop a new technology? The list goes on. At any one time, there’s around 10²⁶ different choices that could be made, and this is all happening in real-time. There’s no turns to think about things, like in chess. To further complicate matters, in a game of StarCraft, you don’t know what your enemy is up to unless you have eyes on them with a scouting unit. Also, whilst playing in real-time, you need to be planning – something you do in the first few minutes of the game may have impacts near the end.

Interviewee: Oriol Vinyals

And so, it was kind of somewhat clear that StarCraft would be a great grand challenge for us to tackle using the techniques we love, namely neural networks, deep reinforcement learning and so on and so forth.

Interviewer: Nick Howe

This is Oriol Vinyals of Google’s DeepMind Technologies. This week in Nature, Oriol is showcasing an AI called AlphaStar, which has been learning how to play StarCraft and it’s gotten pretty good at it. How good? Well, Oriol and the team tested it against real human players online.

Interviewee: Oriol Vinyals

We actually calibrated at a very kind of high level. So, StarCraft II has a few leagues that players get ranked as, from Bronze, Silver, Gold, up to Diamond, Master and Grandmaster. Grandmaster is the very top 200 players of a region, and we played in Europe because that’s where DeepMind is based, and we actually ranked amongst the Grandmasters.

Interviewer: Nick Howe

AlphaStar swept aside many top players to reach the pinnacle of the ranking leagues. But what does it mean to be a Grandmaster? Here’s professional StarCraft II player, Dario Wünsch, better known as TLO, with an explanation.

Interviewee: Dario Wünsch

So, yeah, that’s basically the crème-de-la-crème. Probably the top 50 Grandmaster players are people that are competing at international tournaments, so AlphaStar is very close to the professional level, looking at that.

Interviewer: Nick Howe

So, how did AlphaStar reach these lofty heights? Well, the first step was to train the AI by letting it watch human players playing against each other. By being fed in data from hundreds of human matches, AlphaStar got a sense of how to play the game. The next step was then for the AI to play against itself, to determine which strategies were the most effective. This made the AI improve, but as Oriol explained, it wasn’t quite enough to get the AI to Grandmaster level.

Interviewee: Oriol Vinyals

So, if you just take this agent that’s created from human behaviour or imitation learning and make it play against itself, it improves a little bit, but it loses its diversity in strategies. There’s so many different things that can happen in this game and essentially, self-play focuses too much on a single strategy and then it’s not robust against all the sorts of different creative ways that you can play the game. So, what we did to improve self-play was to create this notion of exploiters that are just agents whose sole purpose is to beat the main agent to show what you are weak against.

Interviewer: Nick Howe

These exploiters played in bizarre and unexpected ways, which exposed the holes in AlphaStar’s strategy.

Interviewee: Oriol Vinyals

And so, that, plus then some notions of keeping a bit of this imitation behaviour and just this human, if you will, prior knowledge about the strategies that exist, those two key components – the imitation and the league – then created AlphaStar final, the agent that was essentially Grandmaster.

Interviewer: Nick Howe

Now, you may be thinking okay, the AI was a Grandmaster, but isn’t a computer always going to be better at a computer game? And you have a point. While humans playing the game are physically limited to speed they can click the mouse or press the keyboard, the AI doesn’t have fingers that can only click so fast. To help make sure the AlphaStar played fair, Dario, who you heard from earlier, advised the DeepMind team.

Interviewee: Dario Wünsch

They basically put some limitations on the agents so they could act just about the same speed as a human with some reaction delay and so on. Then I was in the offices, I was assessing if now that they’ve put these restrictions in, if the agent is actually competing on a fair level to a human.

Interviewer: Nick Howe

After adding the limitations to the AI, Dario was happy with how the AI played. In the paper, he said: “Overall, it feels very fair”. This was important to the DeepMind team, as it meant that AlphaStar couldn’t brute force a victory by performing a superhuman number of actions simultaneously. Instead, it had to improve its skill and strategy, much like a human would have to do. But whilst hammering humans at video games is all well and good, how does it help develop AI technologies and move us towards more broad-ranging intelligences and even human-like Artificial General Intelligence or AGI? Well, Oriol hopes that understanding how AlphaStar got better at StarCraft could help train future AIs. But whilst Oriol is optimistic about how his team’s approach will help build AGI, Michael Rovatsos, an AI researcher not associated with this work, has concerns that we may miss something by designing AIs in this way. He thinks Oriol’s approach is great for solving problems that have an optimal solution, like being awesome at StarCraft II, but for more general-purpose applications it falls short.

Interviewee: Michael Rovatsos

So, it seems to me that, to some extent, in the current landscape, we’re very much kind of looking at very hard puzzle-like problems rather than what humans do, which they’re mostly rubbish at most things but they kind of do okay across a very broad range of tasks.

Interviewer: Nick Howe

To move past this narrow specialisation and reach that human-like level of general intelligence, Michael thinks that the next thing AIs need to do is to learn how to listen. AlphaStar basically tried millions of different strategies until it found something that worked, which isn’t really what humans do. If an architect had to build a million houses until they built one that didn’t fall down, well, they wouldn’t get a lot of work. Instead humans learn from each other, from books and schools, and that’s what Michael thinks AIs should do more of.

Interviewee: Michael Rovatsos

My personal view is that it has to be more about communication. The products of human intelligence, we accumulate them culturally over time as a society and through collective intelligence. No single person can get enough experience to solve the really big problems by themselves, and right now, even on the simplest level, you couldn’t, let’s say, take this system and easily tell it this whole area of strategies you’re considering is not useful – ignore it. There is no way for this agent or algorithm to actually consider what information it might be given.

Host: Benjamin Thompson

That was Michael Rovatsos from the University of Edinburgh in the UK. You also heard from Oriol Vinyals from DeepMind Technologies, also here in the UK, and Dario Wünsch from Team Liquid in the Netherlands. You can find Oriol’s paper over at nature.com, and we’ve also got a video showing the AI StarCraft army being put through its paces. You’ll find that over at youtube.com/NatureVideoChannel.

Host: Shamini Bundell

Later on, we’ll be hearing about a climate lawsuit in Canada – that’s coming up in the News Chat. Now though, it’s time for the Research Highlights, read this week by Dan Fox.

[Jingle]

Dan Fox

Although it may have only lasted for a few seconds, researchers have uncovered evidence of an enormous lightning bolt more than 500 kilometres long that blazed through American skies back in October 2017. The record-smashing flash was discovered by a team of researchers who were studying data from ground-based lightning detectors and an instrument aboard the GOES-16 satellite. The ‘megaflash’ originated in skies above northern Texas before travelling across the state of Oklahoma and into Kansas. In total, the lightning lit up an area of almost 68,000 square kilometres. And while this flash beats the previous record, a 321-kilometre-long bolt seen in 2007, researchers are already beginning to identify even bigger lightning bolts in data from GOES-16 and its twin satellite GOES-17. This suggests that these vast electrical discharges may be much more common than previously thought. Head over to Bulletin of the American Meteorological Society to read this illuminating research.

[Jingle]

Dan Fox

If you’re planning a career in pop music, a team of researchers in Montreal might have some song writing tips for you. They’ve used a new model to identify the songs most enjoyable to the human brain. By analysing a huge musical repertoire, from Canadian folk songs to the works of Bach, the team have developed a way to measure a trait the researchers call ‘complexity’. The researchers then asked people to rate various musical clips. Participants preferred songs of medium complexity to simple and highly complex tunes. The team also found that when listeners weren’t sure of how a song would unfold, they preferred fewer surprises, but if they thought they knew how a tune was going, they enjoyed a surprise. This result supports existing theories that in many types of art, audiences tend to most enjoy the middle-of-the-road option. Read the full paper in the Journal of Neuroscience.

[Jingle]

Host: Shamini Bundell

If you pick an animal on Earth at random, chances are it’s an insect. Some estimates suggest that more than 80% of all living species are insects and there are probably hundreds of millions of times more insects living on this planet than there are humans. But unfortunately, that doesn’t mean that insects or other arthropods, such as spiders and centipedes, are thriving. In fact, arthropod populations are facing serious declines. But just as it’s difficult to count their exact number, it’s also tough to put precise numbers on their decline. A team led by Sebastian Seibold at the Technical University of Munich in Germany have been taking on this sizeable challenge with one of the biggest studies of arthropod abundance and diversity. Reporter Izzie Clarke called Sebastian up and started by asking him what existing studies have shown.

Interviewee: Sebastian Seibold

We have seen a very interesting study from Germany a few years ago showing that insect biomass across many sites in northern Germany has declined massively over the least 25 years, but they have not identified insects to species level and therefore it was not possible to show which species are actually declining. Is it affecting all or is it only certain insect groups?

Interviewer: Izzie Clarke

So, we know insect levels are declining, but we don’t seem to have a big picture study, so how have you been trying to fill in that gap?

Interviewee: Sebastian Seibold

Our study now covers a broad spectrum of insects, arthropods – because it’s not only insects but also spiders are involved – and we recorded biomass but also the number of individuals, species numbers, so we can really talk about biodiversity more broadly. And the other thing that is new and which has been unclear so far is whether most studies so far have been focused on grasslands or other open habitats but it was unclear whether forests were affected by these massive declines.

Interviewer: Izzie Clarke

And to tackle this, you have done a pretty extensive study, so what did you set out to do?

Interviewee: Sebastian Seibold

So, the studies seen so far have usually focused on a few sites, but for this project, 150 sites in managed grasslands were installed in three regions in Germany and another 140 sites in forests, and at the moment, we are at the level of about 2,600 species that were sampled on our sites. But ours is, I would say, one of the first that combined these different meshes of diversity and our group has been collecting arthropod data for about ten years now.

Interviewer: Izzie Clarke

Gosh, ten years is quite a lot of time. So, how is it looking? What did you find by looking at all these different species over such a long period of time?

Interviewee: Sebastian Seibold

Biomass of insects, both in forests and grasslands, have been declining over the ten years of our study. Also, the number of individuals, so the abundance of insects, has been declining and also the number of species has been declining in both of these habitat types – grasslands and forests.

Interviewer: Izzie Clarke

Do we know why we’re seeing these declines?

Interviewee: Sebastian Seibold

Yeah, this is, of course, the most important question, and our data shows that drivers of the decline have to act somehow on landscape scale. It was not the local land-use intensity which we more or less suspected, but it must be something acting more on a larger spatial scale, and we found, at least for grasslands, that the decline was more pronounced, it was stronger on sites that were embedded in a landscape with a lot of arable fields in the surrounding. So, this points to drivers related to intensive agriculture, especially arable farming.

Interviewer: Izzie Clarke

Okay, so does that mean if you had a nature reserve in the area, if the surrounding areas aren’t looking so good, then that will also not be as helpful as we think it could be?

Interviewee: Sebastian Seibold

That’s exactly the point, and that’s what we find. We have some sites that are protected areas, that are managed mainly for conservation purposes, but still we observe declines within these protected areas and the drivers seem to be linked to land use in the surrounding region. We tried to collect more data on forest structure, for example. So, we would like to understand in more detail what’s driving insect dynamics in forests and we are, at the moment, collecting more data on this, but also for grasslands it would be nice to understand what is it that causes the decline. It’s not just ploughing over a field. It could be pesticides, it could be the loss of certain habitats or it could be a combination of all these things, and I think we really need to understand more clearly the contribution of these individual drivers to design better management practices.

Interviewer: Izzie Clarke

Now that we know that we’re seeing these declines, based on some of the policies that we have in place, does anything need to change now that we’ve got this study to show?

Interviewee: Sebastian Seibold

I think what really needs to change is that we have to stop thinking small-scale. It’s obviously not enough to just protect a small area or change farming practices within a small area. What we need is a large-scale approach, I would like to say. So, we really need to coordinate conservation activities, changes in land-use practices across regions and, for example, in the vicinity of protected areas and so on, to create buffer zones but also to really increase habitat reliability, habitat quality for insects across a continuous landscape.

Host: Shamini Bundell

That was Sebastian Seibold from the Technical University of Munich in Germany. You can find his paper over at nature.com.

Interviewer: Benjamin Thompson

Finally on the show, it’s time for the News Chat and I’m joined in the studio by Flora Graham, editor of the Nature Briefing. Flora, hi.

Interviewee: Flora Graham

Hi!

Interviewer: Benjamin Thompson

Three quick-fire stories today then, and for our first one, let’s head over to Canada where a lawsuit has just been filed.

Interviewee: Flora Graham

That’s right. This joins a wave of lawsuits all over the world, where climate activists, particularly young people, are turning to the courts to try to stop their governments from doing things that they think are contributing to global warming.

Interviewer: Benjamin Thompson

And yes, quite striking then that it is young people who have filed this suit.

Interviewee: Flora Graham

A lot of these cases hinge on the idea that young people are the ones who are really going to reap the whirlwind of climate change, and so they’re the ones that perhaps have the best chance of making this argument in the courts. In this case, it’s fifteen plaintiffs between the ages of 10 and 19. One, for example, Ira Reinhart-Smith, is fifteen. He said he first got involved with climate activism through the school-strike movement, and he’s saying that this lawsuit is helping him to express his anger and his fear that his generation and generations to come are going to be exposed to things the world has never been exposed to before.

Interviewer: Benjamin Thompson

And you say there’s a wave of these across the world – what’s happening in this case?

Interviewee: Flora Graham

Well, in this case, there’s no explicit right to a healthy environment under the Canadian constitution, so the case really hinges on an untested legal theory that global warming will impinge on other fundamental rights like equality, the rights to life, property, culture and those kinds of things.

Interviewer: Benjamin Thompson

If we’re in untested waters then, what are the plaintiffs hoping is going to be done as a result of this?

Interviewee: Flora Graham

I think what’s interesting here is, in contrast to some of the really high-profile cases in the United States, the government in Canada is, by and large, in agreement with the plaintiffs about many of the issues, including the underlying climate science. But the people who brought the case, they want to argue that a simple ruling ordering the government to take a certain action is not the outcome they’re looking for. They want the courts to really take control and oversee everything until the government has really implemented a strong climate policy.

Interviewer: Benjamin Thompson

Finally on this one then, this lawsuit has only just been filed. What are people saying about its chances of success?

Interviewee: Flora Graham

Well, legal experts that we’ve spoken to say the case probably does have legs and it may even be politically beneficial for the government in that it could give them cover to act more aggressively to curb greenhouse gas emissions.

Interviewer: Benjamin Thompson

Moving on to our next story in today’s News Chat then Flora, and it’s about an Alzheimer’s drug that’s had, well, it’s had a bit of a second chance.

Interviewee: Flora Graham

That’s right. This drug is called Aducanumab. It’s from a company called Biogen. What’s really interesting here is this drug was the last gasp for a very promising line of research into a potential treatment for Alzheimer’s and when it failed in clinical trials, this whole line of research was seen as dead. So, the fact that this drug now seems, after a new analysis of the dataset, to have some promise is a potentially very exciting change.

Interviewer: Benjamin Thompson

How is it then that this ‘failed’ drug has come back into prominence?

Interviewee: Flora Graham

Well, the company itself said the drug doesn’t work, but since then, they’ve evaluated new data from the same studies and found that over the very long term at very high doses, it does seem to significantly slow cognitive decline for people with early-stage Alzheimer’s disease. Now, if that pans out to be true, this will be the only treatment that can say it does that.

Interviewer: Benjamin Thompson

Right, and what pray tell does it do?

Interviewee: Flora Graham

Well, it attacks the amyloid protein plaques in the brain that we know are associated with Alzheimer’s, but really, we don’t know exactly how. We don’t know how these protein plaques are related to the symptoms and the causes of the disease, but they’re definitely present in the brains of people with Alzheimer’s.

Interviewer: Benjamin Thompson

Okay, so, some hope then that this drug might actually be useful. What happens next?

Interviewee: Flora Graham

The drug still has to go through the US Food and Drug Administration for approval, so it will be a long time before any patients outside the trial do get to try this treatment.

Interviewer: Benjamin Thompson

Well, the drug approvals process can take a fair while, so I’ll guess we’ll have to wait and see how this one pans out. For our final story today though, let’s talk about an ongoing outbreak of the viral disease African swine fever. What’s going on here?

Interviewee: Flora Graham

Well, anybody who’s been following this story will know that African swine fever is having an absolutely devastating effect on farms throughout Asia. Almost 2 million pigs have been culled in China alone.

Interviewer: Benjamin Thompson

So, it’s a serious disease then and it’s, as you say, been spreading across Asia.

Interviewee: Flora Graham

That’s right. It’s lethal for pigs and now there’s new signs that it’s appearing in South Korea. It’s been seen on farms near the northern border and it’s also been seen in wild pigs, so the country is taking some pretty extreme measures to try to stop it.

Interviewer: Benjamin Thompson

And what sort of measures are we talking about then?

Interviewee: Flora Graham

Well, South Korea has mobilised military snipers and drones along the demilitarised zone between South and North Korea to try to stave off the wild boars that can carry the disease.

Interviewer: Benjamin Thompson

I mean that seems, as you say, a very extreme way of controlling this disease which presumably is down, at least in part, to the lack of a vaccine. But there has been some research out that might help speed vaccine development along.

Interviewee: Flora Graham

Researchers are indeed under strong pressure to try to come up with some way to stop this disease. Veterinary researchers in China have announced that they have managed to image the virus to get a newly detailed look at all of its complex parts and they do have two vaccines under development.

Interviewer: Benjamin Thompson

Well, as with our other stories in today’s News Chat, it will be a while before we see any results, so one to keep an eye on there with what clearly is a huge story in Asia. Flora, thank you so much for joining me in today’s News Chat. Listeners, head over to nature.com/news for all the latest science news and head over to nature.com/briefing, where you can sign up for a daily science newsletter.

Host: Shamini Bundell

That’s all for this week, but don’t forget, if you want to see that StarCraft AI in action, you can check it out over at youtube.com/NatureVideoChannel. I’m Shamini Bundell.

Host: Benjamin Thompson

And I’m Benjamin Thompson. Thanks for listening.