The brain implant that turns thoughts into text

Host: Shamini Bundell

Welcome back to the Nature Podcast. This week, the implant that turns thoughts into text.

Host: Benjamin Thompson

And what craftspeople can teach scientists about materials. I'm Benjamin Thompson.

Host: Shamini Bundell

And I'm Shamini Bundell.

[Jingle]

Interviewer: Shamini Bundell

Brain-computer interfaces or BCIs are, as the name suggests, interfaces between the brain and a computer. They can read electrical signals in the brain and translate those signals via an algorithm into computer actions like moving a cursor or typing. This week, researcher Frank Willett from Stanford University and his team have demonstrated a faster way to use BCIs for writing, which could allow people who've been paralysed to communicate more efficiently. They recruited a volunteer who couldn't move his hand due to paralysis and then asked him to imagine writing with a pen. The algorithm was trained on his brain patterns while he thought about writing. And after a while, he was able to type text on a computer screen with impressive speed. I called Frank and started by asking him how brain-computer interfaces work.

Interviewee: Frank Willett

So BCI works by recording the neural activity in some way, so here it was recorded with an implanted array of electrodes. Then there's some algorithm that finds the relevant patterns in that activity and translates it.

Interviewer: Shamini Bundell

And what kind of routes have people tried to develop so far in trying to translate brain signals into written text and how did your approach differ?

Interviewee: Frank Willett

Yeah, I guess there's a lot of different approaches. I mean, the one that was most on our minds was this point-and-click method, which we think was the prior best, where you're moving this computer cursor on a screen to each key that you want to hit, and you click on that key and it types that letter. So, this is different from that. So, instead of moving a single cursor from key to key, instead you're trying to handwrite something. You're just quickly trying to write this string of letters, and we show that on the screen.

Interviewer: Shamini Bundell

And how does that work?

Interviewee: Frank Willett

Yeah, it's, well, that's the magic, that's the secret sauce of the algorithm, and that's basically just pattern recognition. So, it's just looking at the patterns of neural activity and it's remembering. It knows what kind of pattern is associated with each letter and then when it sees that in the neural recordings it types that letter out.

Interviewer: Shamini Bundell

So, the neurons that are firing in my head now when I try and handwrite something, if I then subsequently become paralysed, those same neurons are still firing – the patterns are still there for you to observe.

Interviewee: Frank Willett

Yeah, exactly.

Interviewer: Shamini Bundell

It does sound incredibly complicated because it sounds like you would need to know the inner workings of the brain, this like in-depth neuroscientific understanding, but actually, you're kind of avoiding that by using pattern recognition. So, is that like a machine learning system?

Interviewee: Frank Willett

Yeah, exactly. It just learns based on having seen it many times before, so that's part of the calibration process where we collect data of the participant trying to write all these different letters multiple times and then the algorithm is able to then, through those many repetitions, kind of form an image of what each one looks like.

Interviewer: Shamini Bundell

So, you've so far tried this out on one participant, and they've basically had to do a load of training to show a computer what trying to write ‘a’ looks like and ‘b’. So, at the moment, it's personalised to their brain.

Interviewee: Frank Willett

Definitely in the future we want to look towards ways of making that process a lot faster. And we also hope that when we translate this to additional people, we'll be able to leverage that so that it won't take as long on those additional people, right, that there'll be some shared structure.

Interviewer: Shamini Bundell

And there's a video online of this actually working – so comparing your participant using your new handwriting-based BCI system compared to a previous method – and the cursor method is very impressive but it's relatively slow, so I can see the moving the cursor around to ‘y’, ‘o’, ‘u’, ‘space’, ‘m’. Whereas your participant here, it's noticeably and impressively faster. What's the comparison there?

Interviewee: Frank Willett

The original point-and-click typing device peaked at around 40 characters per minute, whereas this new method does 90 characters per minute. So, 90 characters per minute is about 18 words per minute, which is kind of exciting because it starts to get into the range that's maybe comparable to normal handwriting speeds or, in this case, kind of comparable to how you would type on a smartphone.

Interviewer: Shamini Bundell

So, the technology that you're using of implanting an electrode array into the brain and then communicating that, that's kind of the same as what's been done previously, but the novelty is then using handwriting. So, what's been the benefit of that?

Interviewee: Frank Willett

The reason why we think that handwriting was much more effective than the point-and-click cursor movements is because when you try to handwrite each different character, that evokes a very different pattern of neural activity for each character, which is great for BCIs because that makes things easy to distinguish in the neural activity. When you're doing like a point-and-click cursor, right, when you're going to nearby keys, that evokes very similar patterns.

Interviewer: Shamini Bundell

And given that at the moment it does require quite a lot of setup, is this going to be widely usable?

Interviewee: Frank Willett

I mean, obviously, we hope that yeah, it can one day be turned into a product that anyone who's has severe paralysis who can't speak or communicate could get something like this implanted. I think in the future, you could imagine if things really develop along really nicely maybe this could be part of a general-purpose device that lets you control a computer. So, even if you're, let's say, your spinal cord injured and you can move your head and face and still talk, well, maybe this could be part of a general device that lets you type on a computer and click things more easily.

Interviewer: Shamini Bundell

So, what is the next step now for you in this research?

Interviewee: Frank Willett

Yeah, well, with the handwriting stuff I think one big theme that emerged for future work is kind of making it much more streamlined for practical use and in particular the calibration times, making that faster, and then also, similarly, when things change across days. So, sometimes you get different patterns of neural activity on different days because the implant device maybe moves around a little bit so you record from different neurons. So, instead of having to retrain it every day, it would be great if instead you were able to seamlessly in the background kind of keep track of these changes, so basically minimising this training time and making it more streamlined.

Interviewer: Shamini Bundell

That was Frank Willett from Stanford University. Head over to the show notes, where you can find a link to Frank's paper and the video I described of the system in action.

Host: Benjamin Thompson

Coming up in the show, we'll be hearing how a scientist’s journey into craft helped shape her materials research. Right now, Noah Baker is here with this week's Research Highlights.

[Jingle]

Noah Baker

Flatworms can see where they're going without a head, thanks to light-sensitive cells throughout their bodies. Like most animals, flatworms have eyes that respond to light, but previous research has suggested that the worms don't rely entirely on their eyes to see. To test the theory, researchers cut the heads of a type of freshwater-dwelling flatworm that's able to survive in its decapitated state. Then they exposed the worms’ bodies to ultraviolet light and watched as they turned away, moving away from the light just like intact worms do. By looking at gene expression throughout the animal's tissue, the researchers found that the flatworms’ bodies are lined with networks of light-sensitive cells that coordinate this kind of movement. These cells contained a new type of light-sensitive protein which was also found in pigment cells in the worm. Newly hatched worms don't have this light-sensing ability, suggesting that it develops in adulthood. The researchers suggest that because flatworms are nocturnal, this system evolved to help them quickly hide from the Sun, even if they're resting and not using their eyes or brains. You can read more about that study in the Proceedings of the National Academy of Sciences USA.

[Jingle]

Noah Baker

How itchy would you say you were? It's a tough thing to quantify. And yet now a new wearable device knows the answer precisely. Atopic dermatitis, also known as eczema, causes chronic itching. It can be so severe in children that they scratch their itchy skin at night instead of sleeping, leading to stunted growth. Treatments do exist to reduce this nocturnal itching, but until now the only reliable method to measure their effectiveness was time-consuming analysis of infrared-camera recordings. But now researchers have developed a sensor which can be worn on the back of a child's hand. It uses acoustic and mechanical signals to measure scratches initiated from the arm, wrist, fingers and fingertips. By wiring up healthy volunteers, the researchers trained the sensor to detect which movements constitute scratching and which don't. Then they tested the device in 11 children with moderate to severe eczema over many nights. When compared to the infrared-camera method, the sensor performed well, correctly identifying 84% of scratching movements and 99% of non-scratching movements. The researchers hope their device could provide a new way to help tackle this common condition. Read more in Science Advances.

Host: Benjamin Thompson

Have you ever wondered why phone screens are made of glass when it is so breakable, or why porcelain and terracotta start life as similar lumps, but one ends up as fine china and the other has heavy duty plant pots? Materials scientist Anna Ploszajski realised that despite all her hours in the lab with glass instruments in her hands, there were major cracks in her understanding of that material and many others. In her new book, Handmade, she spoke to ten artisans, makers and craftspeople about the materials they know and love. Our reporter Lorna Stewart spoke to Anna from her London home.

Interviewee: Anna Ploszajski

So it all started about four years ago when I experienced this sort of crushing realisation that although I was trained in materials science and I knew all of the theories and formulae behind materials and how they sort of exist and operate in the world on paper, I realised that I knew nothing really about how they behave in the sort of real handmade world. I'd never sort of forged an iron bar in a blacksmith’s workshop or thrown a pot on the potter's wheel. And so that's what I wanted to explore in this book is this world of handmaking and craft and sort of rediscovering materials through that.

Interviewer: Lorna Stewart

You were super hands-on for the book, so trying all kinds of crafts, like you've mentioned, glassblowing, blacksmithing, pottery and even wool tasting, and I think we should start by talking about wool tasting.

Interviewee: Anna Ploszajski

So wool tasting was an experience that I did a workshop called Wild and Woolly up in North London in Clapton. And it much appealed to me as a materials scientist, actually, because we were presented with an egg box full of six different wools that were unnamed, and we had to sort of knit up a small sort of swatch of them, and sort of experience how they behaved under hand and the different qualities to them. And listeners who aren't knitters might be surprised to hear that there are all sorts of different qualities to wool but there really are. You can have sort of different plies, some of them were shiny, some of them are very matte, some of them had bits of hair sort of sticking out of them at weird angles, so some really sort of big personalities in in this collection of wools. And then at the end of the session, we were sort of introduced to exactly the type of sheep that they came from, what the breed were like, and what sort of objects you could expect to knit from these wools. So, that was wool tasting.

Interviewer: Lorna Stewart

What does feeling the wool and understanding all those characteristics tell a materials scientist about the structure of it or about the properties of it that you wouldn't know?

Interviewee: Anna Ploszajski

So this is what was really fascinating to me because wool is not a material that we study in materials science, so it was a complete sort of new substance for me, and as I was writing the book and looking into what is wool, how is it constructed, I discovered that it's obviously made of atoms, and these atoms are sort of configured into amino acids at the very sort of smallest levels. And then these are sort of twisted round, not only down their own lengths, but also around each other, so they form these sort of coiled ropes. And the coiledness of those tiny, tiny, tiny atomic ropes is what gives wool it's stretchiness at the kind of human scale that we interact with it and it's kind of wrinkle recovery. And so, that was an example of a property that has its origins in atomic structures, which is my comfort zone, but is looked for by knitters and craftspeople as a property for the materials that they want to work with. So, that was a really delightful kind of crossover, really, of the science that I was comfortable with, with the craft side.

Interviewer: Lorna Stewart

I mean, it's so interesting to me, you talk there about how the wool is made, how it’s spun together and twisted together, and a lot of what you went and did was making things, glassblowing, blacksmithing pottery, when I think of materials science as so much about breaking things, I suppose.

Interviewee: Anna Ploszajski

It’s so true.

Interviewer: Lorna Stewart

How was it to sit down as somebody who is normally breaking and make instead?

Interviewee: Anna Ploszajski

Yeah, it was a whole new experience for me, and at the start of the book, I talk about my sort of lifelong reluctance to really get crafty and get hands-on and make things. But then as I went through the materials and gained confidence in using my hands to make with them and I suppose learning the process of thinking about making, if that makes sense. So, the things I would learn from craftspeople were the skills, but then observing their approach to applying those skills to the materials really did increase my confidence, I suppose, in terms of being able to make these really quite simple objects. Now, I do feel like a crafter and able to make things that people actually seem to genuinely appreciate.

Interviewer: Lorna Stewart

Were you able to teach any of the artisans something about materials science that they didn't know?

Interviewee: Anna Ploszajski

I wouldn't go so far as to think that I would be able to teach them, I don't think, because if we take the example of the blacksmith, Agnes Jones, who I met, we had a go at blacksmithing in her forge. She is able to tell the temperature of her steels just by looking at them and the colour that they incandesce at, and although I don't think she would necessarily know the atomic origins of the incandescence and why hot atoms give off electromagnetic waves in the visible spectrum and all the stuff that I would know about, to her, I don't think that atomic explanation is what's important when you're blacksmithing. So, although, yeah, I suppose I was able to kind of provide that information, I wouldn't say that… for me, that's kind of an awkward hierarchy, because I wasn't trying to teach them anything. I was trying to learn from them, if that makes sense.

Interviewer: Lorna Stewart

Yeah, it makes total sense. Do you feel like you're going to reshape your academic practice in light of all the things that you've learned?

Interviewee: Anna Ploszajski

Yeah, so at the time that I was writing the book, I was doing postdoc research in 4D printing and 4D materials, so sort of substances that move in reaction to their environment when you 3D print with them, and yeah, I did start applying what I learned in the world of craft to that research. For example, I was wanting to start 3D printing on to sort of pre-existing textiles and fabrics. And so, I got my microscope out and I looked at these textiles under the microscope, and I noticed familiar knitted constructions that I'd been knitting with and knitting blankets with in these textiles, and so then I could understand their mechanical properties and how to manipulate those in my experiments. So, yeah, it definitely did influence my research direction, yeah.

Interviewer: Lorna Stewart

It's a cliché that maths geniuses sort of see numbers everywhere. I wondered whether you see atoms and structures everywhere when you look at materials? Do you look at cake and see the structure of sugar? How do materials scientists, how do you as a materials scientist, view the world?

Interviewee: Anna Ploszajski

I think how I view the world has changed throughout the process of writing this book because at the start, I would go about the world and yeah, see atomic structures in things. But now, having had my eyes open to the world of craft through this book, I look through the eyes of ten different craftspeople with ten different areas of expertise. And through looking through their lenses, I think I have gained an appreciation for how our world is made, not just how it's made by atoms, but how it's made by people's hands.

Host: Benjamin Thompson

Anna Ploszajski there. Head over to the show notes where you can find a link to Nature's review of her book.

Host: Shamini Bundell

Time now for the Briefing Chat, where we discuss a couple of stories highlighted in the Nature Briefing. Ben, what have you got this week?

Host: Benjamin Thompson

Well, Shamini, I've got a microbiology story that was reported in Nature. But let's start, as we often do in the Briefing Chat with a short quiz. Shamini, tell me about the base pairs in DNA.

Host: Shamini Bundell

Ah, okay, I love the quizzes. Base pairs, okay. There are four base pairs – A, G, T, and C. So, G binds with C and A always binds with T, and then that's the code that the DNA is sort of written in

Host: Benjamin Thompson

Bingo, I'm very pleased you got that right, to be honest with you, given that we've been hosting a science show for many, many years. But what's neat about this story is that there are a bunch of viruses that infect bacteria. Now, I'd call them phages. You might call them phages, but let's stick with phages for the time being. So, these viruses infect bacteria, but instead of using adenine or A, they use a base called 2-aminoadenine or Z.

Host: Shamini Bundell

There’s a Z base? I've never heard of a Z base.

Host: Benjamin Thompson

Yeah, so it turns out it's been known about for quite a long time. It was first discovered about 40 years ago. So, some scientists in the Soviet Union in the 1970s showed that Z was being used in a phage called S-2L that infects photosynthetic bacteria. And for a long time, it wasn't really known whether this was kind of a fluke, if this was a one-off, if this was literally n = 1.

Host: Shamini Bundell

So, is it just like maybe a weird mutation in sort of one obscure virus potentially?

Host: Benjamin Thompson

Well, Shamini, that is a great question and the answer is it does seem to be pretty widespread, but it's taken a little while to get to that point. So, back in the 2000s, researchers sequenced the genome of this S-2L phage and found a gene that looked like it was involved in making this Z base, but couldn't find any similar genes in the databases, and fast forward ten years or so and some more were found. And now there's two papers out in Science, where two groups have expanded that out and published the genetic pathway that shows how the Z base is made and, as I say, it's pretty widespread, it turns out.

Host: Shamini Bundell

And what does the fact that there's, yeah, a whole group of phages using a completely different base? What does that mean?

Host: Benjamin Thompson

Well, I think what it means in the first instance is that hiding in plain sight, I guess, there is a sort of DNA that uses a different system to the one that you and I learnt at school. And to answer the ‘What is it for?’ I mean, I think the best guess at the moment is that this kind of non ATCG DNA maybe protects these phages from some of the mechanisms that bacteria use to break them down.

Host: Shamini Bundell

That's cool, yeah, that this might not just be a random thing, but there might be some sort of advantage for the phage. Wow.

Host: Benjamin Thompson

Yeah, I mean, it may be an advantage and it may be that researchers piggyback that advantage and use it for different things. So the Z-T bond is actually stronger than the A-T bond in DNA, so it takes more energy to break them apart. So, there’s talk about maybe this can be used to make more stable DNA molecules when you're trying to store information, which we know that some researchers are trying to do. It might make DNA fold differently, which could be used in DNA origami, which has been used for a variety of different things. So, yeah, people are looking to work out what this means and what could be done with it. Right now, one of the teams involved in one of these papers is looking to try and incorporate this Z DNA system not into a virus but into a bacterium, so into E. coli to make that make this kind of different sort of DNA.

Host: Shamini Bundell

Wow. Next time someone asks me about DNA bases, I shall sort of update my knowledge and mention Z in there as well.

Host: Benjamin Thompson

Yeah, absolutely right. I mean, a few years ago on the podcast, I covered some researchers who were making synthetic bases, X and Y, so it turns out that this is a fairly big area of research. And, well, I mean, who knows quite where it's going to go, but something I'm sure that we'll keep an eye on. But, Shamini, what about you? what have you got this week for the Briefing Chat?

Host: Shamini Bundell

So, I've been reading an article in Science this week, all about the Chernobyl Nuclear Power Plant disaster, which was just over 35 years ago, but it seems like there are still some sort of ongoing worries and ongoing hazards remaining even after all this time, at the site of the plant of the disaster.

Host: Benjamin Thompson

Well, you're right, Shamini, disaster is the right word to use. And I know that a couple of years ago, I think it was a big sort of shell, a big carapace put over the top of this sort of wreckage. Is this related to that in any way?

Host: Shamini Bundell

Well, that's still there and the potential risks that might be there are still being contained by that. It was called the New Safe Confinement, so that's still there sort of keeping everything in. But one of the aims of that big protective structure was to stop rainwater getting in. So, after the disaster, there was all this melted nuclear fuel in there and they put a temporary shelter over everything. But rainwater would leak in and it seemed that the rainwater was basically affecting the neutrons and accelerating the fission reactions in this fuel. So basically, there's still kind of nuclear reactions going on in there. This article describes it as the sort of embers in a barbecue pit smouldering. So, the New Safe Confinement was supposed to stop the rain. However, for some reason, the scientists who've been monitoring it have recently seen, from a particular point in this building, an increase in the number of neutrons, which suggests that fission is once again sort of happening and increasing in there somewhere.

Host: Benjamin Thompson

I mean, I'm no nuclear physicist, Shamini, but that doesn't sound too good. I mean, what are the researchers saying about this situation?

Host: Shamini Bundell

Well, they're quite puzzled by the mechanism by which it's happening because the stopping of the rain water was supposed to stop exactly this happening. There is a theory that maybe it's the opposite problem. Maybe as the fuel mass is drying out, that’s somehow causing this reaction to keep going as well. And there is a chance that it could go critical and it could cause some sort of explosion. Now, this isn't going to be able to break free of the New Safe Confinement, but the people who are sort of managing this plant and trying to make it all safer absolutely do not want that to happen. They are trying to remove the remains of the original temporary shelter. It’s still there but it's sort of unstable, it's resting on the remains of the original building, which is itself kind of unstable. They don't want that all sort of collapsing down. It would release loads of radioactive dust into the middle of it. So, they're keeping quite a close eye on this rising neutron count.

Host: Benjamin Thompson

So, is it just at the moment, keep an eye on it and see where it goes?

Host: Shamini Bundell

Yeah, so this isn't like they suddenly noticed this yesterday. This has been slowly increasing and they think they have a few years to get this under control. Unfortunately, the particular place where these neutrons are coming from is kind of in a collapsed room, so it's kind of buried under concrete so it’s really hard to get to and of course a really unsafe environment. So, the one idea that this article mentions is to develop a robot that can withstand the radiation in there that could drill holes and insert boron cylinders, so they would mop up the extra neutrons, similar to control rods that you have in a nuclear power plant. Hopefully they will have time to sort of work on these ideas, but they're also being careful to monitor this one and other areas where there's this fuel-containing material which could go critical. Again, it's not going to be a sort of huge explosion – it will be contained – but they definitely don't want even within the New Safe Confinement any sort of explosion.

Host: Benjamin Thompson

Well, absolutely right, Shamini, and that's a really, really interesting story. Thank you for bringing it to the Briefing Chat this week. And listeners, if you're interested in more stories like this but instead as an email then make sure you check out the Nature Briefing, and we'll put a link in the show notes where you can sign up.

Host: Shamini Bundell

That's all for this edition of the Nature Podcast. Don't forget to follow us on Twitter – we’re @NaturePodcast – and head over there to see a video of the brain-computer interface we heard about earlier in the show.

Host: Benjamin Thompson

And also, don't forget you can reach out to us on email – we’re podcast@nature.com. I'm Benjamin Thompson.

Host: Shamini Bundell

And I'm Shamini Bundell. Thanks for listening.