Podcast: Ultrafast machine vision, and quicker crystal creation

Host: Benjamin Thompson

Welcome back to the Nature Podcast. This week, improving machine vision…

Host: Nick Howe

And a new way to grow crystals. I’m Nick Howe.

Host: Benjamin Thompson

And I’m Benjamin Thompson.

[Jingle]

Host: Benjamin Thompson

First up on the show, reporter Ali Jennings has been finding out how researchers are trying to speed up the way that machines interpret images.

Interviewer: Ali Jennings

Those of you with smarter smartphones might find that your camera has software that identifies faces in your pictures. You might also find there’s a little delay between pointing your phone at a face and the software recognising it’s a face. For machines trying to decode images, this is a common problem.

Interviewee: Thomas Mueller

Currently, machine vision systems are slow. Typically, they are based on a frame-based camera that takes images that’s connected to a standard computer that does the classification.

Interviewer: Ali Jennings

This is Thomas Mueller. He’s been rethinking the design of machine vision systems to speed up their classification of features in an image. This week in Nature, Thomas and his team present a new device that may be able to do just that.

Interviewee: Thomas Mueller

Our device takes the images and at the same time tells you what this image shows. It’s extremely fast, so what we have demonstrated is classification, for example, within 40 nanoseconds, 50 nanoseconds. But I think you could even speed it up by the effect of 1,000 or so.

Interviewer: Ali Jennings

This would be millions of times faster than most machine vision systems, which generally operate on the order of milliseconds. To get such rapid classification, Thomas’ device takes the software that conventional machine vision computers use and builds it into the hardware of the sensor itself. Conventional machine vision software uses artificial neural networks to identify features in an image. By tuning the strength of the connections between different parts of the network, these neural networks can learn to recognise important features, like your face, for example. Thomas’ team built a camera where the individual light sensors, which they call pixels, can be tuned in the same way.

Interviewee: Thomas Mueller

In the standard camera, you read out pixel by pixel. It’s not what we are doing in this chip. Instead, we just interconnect the pixels such that it forms a neural network itself.

Interviewer: Ali Jennings

When this pixel network is correctly tuned, it can recognise simple features in an image without having to transfer all the data to a computer for processing. This makes it much, much faster than existing machine vision systems. But it might not be ready for your phone camera just yet.

Interviewee: Thomas Mueller

The way it is now, it’s a very simple network that can detect simple objects. That’s the limitation currently.

Interviewer: Ali Jennings

Thomas thinks, though, that in the future, the sensor could be built with more pixels.

Interviewee: Thomas Mueller

Well, the next logical step would be, of course, to scale things up. It would allow to recognise much more complex images than what we have now. It would also make it more powerful to recognise more details in images as the complexity of this network increases.

Interviewer: Ali Jennings

So, will we see this new, superfast sensor appearing in smartphones? Thomas is sceptical.

Interviewee: Thomas Mueller

This system that we developed is not so much for the everyday life. It’s rather for the scientific life, I would think.

Interviewer: Ali Jennings

Thomas thinks his device would be best suited to imaging very fast-changing situations, such as turbulences in fluids, combustion processes or how materials break. But when I talked to Yang Chai, a researcher in artificial intelligence who was not involved in the study, he was excited about what such an invention could mean for machine vision technology.

Interviewee: Yang Chai

I would say that this is a new concept that people never proposed or demonstrated previously. I think this work actually opens up new computing paradigm. Basically, computing is far away from the data source, but this work, it moves the computing very close to the data source.

Interviewer: Ali Jennings

Yang also thought that these kinds of devices could one day be used in surveillance systems, medical imaging and even unmanned drones. And it’s not just machine vision. The novel concept of having a sensor built as a neural network could be applied to many other technologies.

Interviewee: Yang Chai

This concept can be extended to many, many other physical senses. It could be audio, it could be some chemical, it could be natural chemical, even pressure.

Interviewer: Ali Jennings

To get to this point, there are some challenges that will have to be overcome. For example, integrating these new sensor chips with conventional electronics could be problematic, as the chips output is very new and very different to what standard computer processes typically receive. But Thomas’ device may have given us a glimpse of a faster, more efficient kind of computing. Now, if only it could help me take the perfect selfie.

Host: Benjamin Thompson

That was Ali Jennings speaking to Yang Chai from Hong Kong Polytechnic University and Thomas Mueller from the University of Vienna. You can find the full paper along with a News and Views article over at nature.com.

Host: Nick Howe

At the end of the show, we’ll have an update on the contribution of climate change to the Australian bushfires – that’s coming up in the News Chat. Before that, it’s time for the Research Highlights, read this week by Tristan Varela.

[Jingle]

Tristan Varela

Dieting is not fun. But it’s been known for a long time that drastically restricting calories can increase lifespan. Why this is, is a bit of an unknown though. Now, researchers in California reveal that certain changes in cell behaviour may play a significant part in this anti-ageing effect. To understand how calorie restriction wards off ageing, the scientists studied 26 middle-aged rats that, for nine months, ate 30% fewer calories than their siblings. After this long spell of calorie restriction, the authors found that the dieting rats had fewer cells associated with inflammation and had a genetic profile that was more like that of younger rats. The researchers think that this could lead to the development of strategies to live better for longer. Chew on that research at Cell.

[Jingle]

Tristan Varela

The traditional seaside summer holiday may soon become a thing of the past. New research suggests that beaches around the world are in grave danger of disappearing due to climate change. Shorelines are naturally expected to change over time, but after studying satellite pictures of Earth going back over 30 years, researchers have determined that rising sea levels combined with the destructive power of storms could result in the loss of almost half of the world’s sandy beaches before the end of the century. Australia would be one of the hardest countries hit, with nearly 12,000 kilometres of their coast line at risk of severe erosion. Curbing our greenhouse gas emissions would be a huge help in preventing tis beach-less future, but we have to act fast. Take a trip to Nature Climate Change for more on that research.

[Jingle]

Host: Benjamin Thompson

There’s a reason that crystallography has led to more than 20 Nobel Prize wins. It’s critical for determining the structure of many materials. But to do crystallography, you need to convert the material you are studying into to a crystal. And not just that, this crystal has to be the right size and of high enough quality. That’s not always easy to accomplish, but this week in Nature, there’s a new, unconventional way to cultivate crystals that could help. Nick’s been finding out more.

Interviewer: Nick Howe

Cool, so, we’re in the lab now, show me around.

Interviewee: Naomi Chayen

This here, that is the robot which was one of the…

Interviewer: Nick Howe

That’s Naomi Chayen. I went to her crystallisation lab at Imperial College London to understand more about how to grow crystals.

Interviewee: Naomi Chayen

Crystallisation used to be known as an art rather than a science, and it was a lot of trial and error. And people said it’s only people who have got green fingers that could do it. So, people did all sorts of things, even sending to space.

Interviewer: Nick Howe

Nowadays, crystallisation is less trial and error, but many materials are still incredibly difficult to crystallise. In broad terms, to make a crystal from a material, you have to have it dissolve in a solvent, then take the solvent away, forcing the material into a solid, crystal state. But simply growing a crystal is not enough. If you want to determine its structure using crystallography, there are specific requirements.

Interviewee: Naomi Chayen

It’s got to be an ordered molecule. If the material is not ordered, you cannot X-ray it. Really, you need something that you can X-ray, and X-ray will go through and be able to analyse it, and you need it a certain size that the X-ray beam can go through it.

Interviewer: Nick Howe

Once a material is dissolved, the first challenge is kick-starting the crystallisation process, but this can be tricky. Stirring the solution has been shown to help, but this can make it more difficult to get that all-important ordered structure, as it’s easier for an ordered crystal to grow when the molecules are still. But Bartosz Grybowski thinks he has found a way to stir a solution and still create ordered crystals.

Interviewee: Bartosz Grybowski

Here, we also have an additive, a polymer, that is in most cases actually charged, and we started observing at some point a very strange phenomenon. All of a sudden when you start stirring this, you crystallise the substance better.

Interviewer: Nick Howe

In fact, stirring the crystal with the charged polymer seemed to grow crystals that were even better than those grown by conventional techniques.

Interviewee: Bartosz Grybowski

In terms of size, this offers significant benefits. Now, in terms of quality, the quality seems very, very nice.

Interviewer: Nick Howe

Bartosz’s crystal grew with that vital ordered structure. As with all techniques, the size of the crystal depends on how long you grow it for, but Bartosz’s spinning method allowed the crystals to grow to a lot more quickly. So, how did the system actually work?

Interviewee: Bartosz Grybowski

Physics! Physics! Yeah, the physics is actually quite non-trivial.

Interviewer: Nick Howe

Quite. Like any other crystallisation method, Bartosz’s technique works by drawing the solvent away from the material of interest, and this is where the polymer comes in. Because it is charged, it acts like a sponge attracting and binding the solvent. What’s more, as the solution is spun around, the polymer stretches which pushes the process forward.

Interviewee: Bartosz Grybowski

When you stretch and expose more charged groups to the solvent, well, it needs more solvent. So, it needs to take the solvent away from somewhere else. Well, it’s taking it from the something that we want to crystallise, so this something that we want to crystallise now sees itself sort of naked. It doesn’t have enough solvent to stay in solution.

Interviewer: Nick Howe

As the crystal grows larger, it stretches the polymer more, which exposes more charged groups, which make the crystal grow larger, which stretches the polymer more. You can see where this is heading.

Interviewee: Bartosz Grybowski

The more extended it is, the more spongy it becomes for solvent, and it becomes more extended, more deformed around bigger crystals. It’s like a feedback mechanism.

Interviewer: Nick Howe

At the moment, only a few types of materials have been tried with Bartosz’s technique, and it’s unlikely to work for everything. But, so far, barring a little tweaking, Bartosz hasn’t had a problem making this technique work for a range of materials, from proteins to scientific reagents. Even if this method doesn’t upend the entire crystal growing community, Bartosz believes it’s another thing to add to chemists’ toolkits.

Interviewee: Bartosz Grybowski

And I will be very happy if this simple system is on the bench of some chemist, let’s say, in industry. If my substance is not crystallising by traditional methods, hey, maybe I should try spinning it around. And it’s going to broaden the repertoire of methods that are available.

Interviewer: Nick Howe

Naomi Chayen, who you heard from earlier and who also wasn’t associated with Bartosz’s study welcomed any new method which could help her with the dark art of crystallisation.

Interviewee: Naomi Chayen

When you want a crystal of a target protein and you’re desperate to do it, you’ll use anything. And if that works for you, then that’s... one of the methods may lead to big drug now against coronavirus. You never know. So, you need methodology and it needs to keep coming because there’s more and more targets, more problems to get crystals and the problem is getting more acute rather than less.

Interviewer: Nick Howe

That was Naomi Chayen from Imperial College London, right here in the UK. You also heard from Bartosz Grybowski from Ulsen National Institute of Science and Technology in South Korea. You can find Bartosz’s paper over at nature.com.

Interviewer: Benjamin Thompson

Finally on the show, it is, of course, time for the News Chat, and I’m joined here in the studio by Nisha Gaind and Nicky Phillips, Nature’s European Bureau Chief and Asia-Pacific Bureau Chief respectively. Hi, both.

Interviewee: Nisha Gaind

Hello.

Interviewee: Nicky Phillips

Hello.

Interviewer: Benjamin Thompson

First today, let’s talk about coronavirus as we so often do, and the usual caveat yet again applies – this is a developing story and we’re recording this on Tuesday morning, so, listeners, go to nature.com/news for all the latest breaking news on this one. Nisha, what’s been happening since the last time we were on air?

Interviewee: Nisha Gaind

So, since last week, the coronavirus has spread. The coronavirus has spread quite quickly to quite a lot of other countries and at first blush that seems quite alarming but it’s probably worth saying that many of these countries – I think there are now 60 or more that have reported cases of coronavirus – many of them just have one or a couple of cases. Nevertheless, the number of infections worldwide has increased. It’s now more than 90,000 in total, but the vast majority of those are still in China. About 80,000 of them are in China. The number of deaths as well has also risen to more than 3,000.

Interviewer: Benjamin Thompson

Well, a couple of big announcements from the WHO. One is that they still haven’t labelled this outbreak a pandemic, and the other one is a big report that came out of a meeting in China last week.

Interviewee: Nisha Gaind

Yes, that’s right. So, the discussions around pandemic continue. They still haven’t labelled it a pandemic, but they have raised the global alert level for this outbreak to very high, which is the highest possible level they can, short of calling it a pandemic. But some of the most interesting things that have come out of the World Health Organization over the past week are a large report that has looked at cases in China – a big analysis of the many tens of thousands of cases that we’ve seen in China – and from that we have gleaned some details about how the virus is transmitted and also whether it’s mutating, and it seems not to be mutating that quickly.

Interviewer: Benjamin Thompson

Yeah, and samples taken from people between December and February are 99.9% similar, I understand.

Interviewee: Nisha Gaind

Yes, that’s right, and we also know some other things. The median age of people infected is 51. Most cases of the virus that have spread from person to person in China are within hospitals and jails and households, and that implies that close contact is required for this virus to spread between people. And the other interesting thing that came out is this question of airborne spread, which has been something that scientists have been trying to figure out, and the China report said that airborne spread is not believed to be a major driver of transmission.

Interviewer: Benjamin Thompson

And as I understand it, this report has also looked at how China has tried to contain this outbreak.

Interviewee: Nisha Gaind

Yes, that’s right, and one of the interesting things that has happened in the past week is the kind of shift in the way that this thing is spreading. So, cases which were, at first, rising rapidly in China, the number of new daily cases now seems to be declining in the country. So, whereas a week ago or perhaps two weeks ago China was reporting new cases daily in the high hundreds, I think it’s now reporting cases in the low hundreds, and the WHO attributes much of that to China implementing these unprecedented measures, these lockdowns, that quarantined effectively tens of millions of people in their city. And meanwhile, it’s now cases outside China that seem to be rising more rapidly on a daily basis.

Interviewer: Benjamin Thompson

Outside of China then, efforts are stepping up to prevent transmission, and we’ve got a story about one effort in particular that is impacting researchers.

Interviewee: Nisha Gaind

Yes, this is something that directly affects the scientific community – it’s the cancellation of conferences, and in one case, the cancellation of the American Physical Society’s March meeting and that is the biggest physics conference in the world, so that was a pretty big deal.

Interviewer: Benjamin Thompson

Yeah, and our colleague Davide Castelvecchi was en route to the conference. In fact, he made it as far as Denver where it was due to be held, and we can hear from him now, actually. He chatted to Nature earlier this week about the cancellation and explained how some of the 11,000 delegates to the conference are adapting to the situation. Here’s a little clip from the interview.

Interviewee: Davide Castelvecchi

Some of the people who ended up sort of stranded in Denver decided to have informal gatherings and what people call ‘unconferences’, which means you get together without any prearranged schedule and whoever wants to give a talk gives a talk. And at the same time, there’s people who are going to record their own talks as videos and upload them. Some groups of physicists went further and they basically tried to recreate the schedule of the conference as a series of webinars. So, they set up an online spreadsheet where people could post links to their live talks that will happen at the same time that they were supposed to give the talks in Denver.

Interviewer: Benjamin Thompson

So, lots going on then, Nisha.

Interviewee: Davide Castelvecchi

This is a particularly serious case of it because it’s such a huge conference, but physicists are trying to do their best and this is not the only conference that we have heard of that’s been cancelled. Of course, many had been postponed in China in the past few months, but now, many conferences and meetings are being cancelled. So, we’ve heard that some will be trying to go virtual or online only, or that some will simply be postponed until we hope that this thing abates. But this is an interesting experiment for scientists to see whether they can do meetings without the travel which is relentless for them and, of course, creates a large carbon footprint.

Interviewer: Benjamin Thompson

Well, listeners, to read more about the ongoing outbreak and to hear the full interview with Davide, head over to nature.com/news. In the meantime, though, let’s move on to the next story in today’s News Chat, and Nicky, it’s about the bushfires in Australia. You appeared on the show a few weeks ago to talk about this, but in case somebody missed that, could you give us a quick overview of what’s been going on?

Interviewee: Nicky Phillips

So, listeners might remember that earlier this year and at the end of last year, Australia experienced what was a fairly unprecedented bushfire event. We had hundreds of fires which burned probably in excess of 11 million hectares over several months. It destroyed thousands of homes, at least 34 people lost their lives, and the toll on wildlife is hard to really assess but probably in the order of millions, maybe even a billion native animals. So, yeah, it was a pretty devastating event.

Interviewer: Benjamin Thompson

And understandably, researchers were keen to try and work out the role that climate change was playing in this fire season – was it making it worse or not – and it seems like we’ve got an update on that.

Interviewee: Nicky Phillips

Yeah, the results are in, actually, of this big study that was trying to work out whether and to what extent human induced climate change played a role in the bushfire season this year. So, researchers found that climate change actually contributed to the conditions of the bushfires by about 30%, and that sounds like quite a lot but, actually, they think that’s quite a conservative estimate.

Interviewer: Benjamin Thompson

When you say 30% then, is that in terms of it made it 30% more severe or increased the likelihood of it by 30%?

Interviewee: Nicky Phillips

Yeah, so that’s 30% more likely.

Interviewer: Benjamin Thompson

And in terms of what the researchers were looking at, I think we learnt before that there are maybe lots of individual factors that could feed in to the risk of a fire. Did they find that one was more likely than another, and what sort of things were they looking at?

Interviewee: Nicky Phillips

Yeah, so overall, they were looking at this thing called fire risk, which is measured by something called a fire-weather index which takes into account temperature, wind, rainfall and some other variables. So, they were looking at that measure as a whole, and that’s what they found increased by 30%. They also looked at individual factors, so temperature and drought conditions, so rainfall. So, they were able to find a very clear signal in the temperature, that temperature has increased because of climate change, and that factor has contributed to an increased fire risk. They weren’t actually able to see whether climate change has influenced the rainfall measure that they were looking at, so that was quite interesting and I think there’s a lot of debate in the community about how we go about accurately measuring the impact that climate change could have on rainfall and therefore droughts in Australia.

Interviewer: Benjamin Thompson

Well, what are the researchers involved in this work saying about this then, Nicky?

Interviewee: Nicky Phillips

So, overwhelmingly, they’re saying that this sort of attribution study will help us understand what the risks are in the future, and that could feed into the sorts of things that planning agencies like emergency services and government organisations and not-for-profit organisations might need to prepare for future fire seasons. I mean the other thing to say is that they also think this estimate is quite conservative. Models aren’t very good at modelling local heatwave conditions, so the increase in temperature that we see observed, so the chances are that the contribution of climate change to this year’s season could have been bigger than 30%.

Interviewer: Benjamin Thompson

Well, sobering stuff indeed. Listeners, head over to nature.com/news for more on that story. And all that remains to say is, Nisha and Nicky, thank you so much for joining me.

Interviewee: Nisha Gaind

Thanks.

Interviewee: Nicky Phillips

Thanks for having me.

Host: Nick Howe

That’s all for now, but before you skip to the next podcast you’ve got lined up, we’ve got a video you might want to hear about. It’s all about keas – a type of parrot – and how they’ve got a surprising aptitude for statistics. Head over to youtube.com/NatureVideoChannel for that. I’m Nick Howe.

Host: Benjamin Thompson

And I’m Benjamin Thompson. See you next time.