Host: Benjamin Thompson
Hello and welcome to Backchat. If the Nature Podcast is an assignment smugly turned in two weeks ahead of the deadline, then Backchat is doing your homework on the bus on the way to school. Yes, Backchat is back. And listeners if you’re new to the show, it’s a bit different to the regular Nature Podcast and it’s more of a personal take on the latest stories from our team of reporters and editors. In today’s roundtable discussion, we’ll be crunching the numbers and looking at data journalism, finding out the best way to squeeze a science story into a single sentence, and learning how CRISPR gene editing is editing stock market prices. I’m Benjamin Thompson, and joining me on today’s show are Heidi Ledford...
Reporter: Heidi Ledford
Hi, I’m Heidi Ledford. I’m a reporter with Nature.
Host: Benjamin Thompson
Richard Van Noorden...
Editor: Richard Van Noorden
Hello, I’m Richard Van Noorden and I edit Nature’s Features.
Host: Benjamin Thompson
And making her Backchat debut, it’s Flora Graham.
Editor: Flora Graham
Hi, I’m Flora Graham and I’m editor of Nature Briefing.
Host: Benjamin Thompson
Coming up in the show, we’ll be talking about the results of a Nature survey on lab health. What did it reveal about research groups around the world? And when we’re deluged with data, how do we pick out what’s important for a story? We’ll also be looking at email briefings – are they having a golden age? What makes a good one and how much can you compress a story before it loses all meaning? Finally, seemingly rather technical research findings are having sizeable waves in the stock market. What’s going on? Firstly then, let’s talk about lab health, and I must say that I’ve been rather lucky in my career and have worked in some very supportive research environments, but that’s not necessarily the case for everyone, and it can be difficult to speak up about it. Richard, I know this is something that you’ve been looking into.
Editor: Richard Van Noorden
Yeah, so for quite a few months we’ve been trying to find out what do scientists around the world think is going on when it comes to the working environment of their lab. And this was a classic set-up in my view, and in the view of Monya Baker who led a lot of this work from our San Francisco office, to survey a lot of scientists and find out what’s going on. So we ended up with survey responses from more than 3,000 scientists to figure out what’s happening in labs. And the major thing that we found is that the people who run the labs, the PIs or Principal Investigators, have a much rosier picture of the dynamics in their research groups than the staff members who are working in the trenches under them.
Host: Benjamin Thompson
Yeah, and that’s what maybe struck me, is this kind of dissonance then. The people at the top think that everything is pretty rosy, but those a bit further down maybe not so much.
Editor: Richard Van Noorden
And this wasn’t just, hey, things are going great/things are not going so great, it was factual things. So, we asked the PIs, how often do you consistently check the raw data of your research group, and more than 90% of them said yes. We asked the non-PIs – just over half. So, these are disagreements on factual matters, not just perceptions of how wonderful you feel this morning.
Host: Benjamin Thompson
Well, that’s something that rather sort of stuck out to me Richard as well, is that this is based on you know, anonymous survey results, and I don’t know about the three of you, but I think we live in a world where things can often be one-star terrible or five-star amazing. So does that then maybe introduce some bias into this sort of research then, like if you’re having a bad day you know, you’ll say things are bad and if you’re having a good day things are the best.
Editor: Richard Van Noorden
Yeah, I mean it absolutely does. And also, we’re sending out emails to hundreds of thousands of people – we’re not going to hear from everyone. The people we’re going to hear from are going to be perhaps the people who have an axe to grind. So, I think all that we can do is mitigate against that by reaching as wide a sample as possible. We need to be offering some kind of incentive for doing this survey, and making sure that our questions have a factual basis to them and aren’t just about opinions. But we still find that people answer the survey in very surprising ways. We did test this survey extensively with people beforehand, and we asked people to describe their lab and then we said how much does the atmosphere in your lab seriously hinder your ability to produce good quality research? Because there’s been research in to how people feel about their labs and sometimes people do feel negative. But it’s not actually clear that this means that this hinders the lab’s research, so we were trying to link the two together. Unfortunately, we failed miserably because one respondent said my lab culture is great, its friendly, it’s collaborative but lab culture seriously hinders our ability to do good quality research.
Host: Benjamin Thompson
Hmm…
Editor: Richard Van Noorden
So, I then emailed nearly a hundred people to follow up, find out what’s going on, and many people either misread the question or thought that we were asking in general, how important is lab culture for hindering… very important, very important. So, having realised this we decided to completely discard that question in our analysis which, you know, limited what we could say. So, you have to put in a lot of work for these kinds of surveys and I am going to be a real survey snob here and say that when I read these surveys in newspapers, we at Nature usually instantly find three or four problems before we’ve got past the second paragraph. So, we’re trying really hard to make sure that we are very honest about the limitations of these surveys and what they can and can’t tell you.
Reporter: Heidi Ledford
Do you consult with, I don’t know, social scientists? Can you, you know, get a few to just look over the questions or say oh, be sure to do this or that?
Editor: Richard Van Noorden
Yeah, absolutely. So, for this lab survey one we consulted with Brian Martinson among others, he studies research integrity, and they went over our survey with a fine-tooth comb and we tested it on 10-20 scientists.
Host: Benjamin Thompson
Well let’s zoom about a little bit then, and a look at data journalism as a bit more of a whole. To my mind, it seems to be becoming a lot more prevalent – is it? And if so, why? I mean, is it access to more tools, more data, you know and so on?
Editor: Richard Van Noorden
Yes, well I think there’s always been data journalism going back hundreds and hundreds of years and just computers on the internet and the development of tools that make it easy for even journalists to do analysis on large data sets is why we’re seeing more and more of it. There’s also, not so much hype, but it takes a lot of resources to do it well and news organisations are undoubtedly keen to highlight when they’ve poured a lot of effort into doing something like this. And so, it’s not that they’re hyping it, but that they are sort of parading it I think, or saying look, this is its own kind of journalism. But it’s always been done as anyone who watched the movie Spotlight will know – it didn’t take Python and R and modern accoutrements of coding to land that story. I think that it’s just becoming easier and easier for novices to crunch through huge datasets, but I think that almost no data journalism stories were landed by someone unloading a dataset without knowing what was in it, searching through and discovering amazing results. Almost all of these stories come from old-fashioned work where a source will tell you that something's going on, you’ll report that story, maybe they’ll point you to a dataset, and then you’ll find the evidence in there. But it’s almost impossible to find a data journalism story by just looking through a dataset without any idea of what you were going to find.
Host: Benjamin Thompson
Well let’s move on to our second topic of discussion then everyone, and I’d like to talk about emails, and specifically daily briefing emails. Now Flora, you’re the Senior Editor of Nature’s Briefing which wraps up the science news and opinions of the day. So, my first question is to you: what is it about them that makes them good at what they are?
Editor: Flora Graham
I think the real benefit of a Briefing is that hopefully it comes from someone you really trust. So, you’re getting that person or that editorial team’s best judgement of what you really need to know. Of course, we’re all undergoing this kind of information tsunami all the time, and there was a time when we thought oh well, my Facebook feed will do it for me or my Twitter feed but even then, we start to feel like really, we’re just in a bubble of kind of randomly appearing information. And people, I think, have really started to come back to the point where they want an expert to say look, I’ve read all the stuff and I can tell you that what’s great and what you really need to know.
Host: Benjamin Thompson
I mean one of the keys things for this, and I think you mentioned it there, is you’re almost compressing it down, right? You’re giving that kind of concentrated burst of information. I mean, how does one go about doing that without giving away the punchline, like I mean if you want someone to read the full article but you tell them what the full article is in a sentence, I mean how do you square that circle?
Editor: Flora Graham
Well, different newsletters have different purposes – some are definitely to drive people to read the full article. Now, I’m kind of lucky as the editor of Nature Briefing that I don’t need to have to worry about that. What I’m trying to do is give people hopefully all the information they need. I want them to be able to go into their next meeting or meet their next colleague or go down the pub and be able to have to hand the salient facts about that news story.
Reporter: Heidi Ledford
How many words do you have to summarise the whole gist of the story?
Editor: Flora Graham
Well there’s no set limit, but actually it is surprising. I would generally keep it down around 60-80 words. I mean it just goes to show how quickly you can get information across when you genuinely are trying to summarise things in as effective way as possible.
Host: Benjamin Thompson
I mean how on Earth does one go about taking a 10,000-word feature and getting it down into those 60 words? I mean it just seems like a massive headache.
Editor: Flora Graham
Well I should say that I tend to summarise fully news stories. So, just like a headline can kind of give you the gist, I go a little bit further. But when it comes to those super long reads, those amazing multimedia features, maybe the really opinionated blog posts, I definitely do try to set them up and let the reader click through because of course, if you’re talking about a long, personal story, of course you can’t wrap that up simply by reporting just the facts and let’s move on to the next thing. But when it comes to news in briefs I think that’s why these email summaries are really coming back, because people just don’t want to visit every website that they’re interested in. I can give them those signposts.
Host: Benjamin Thompson
And throwing it open then to the rest of the panel, obviously Richard you’re involved in editing ‘regular’ stories if I use those air quotes, and Heidi you’re of course involved in writing them. I mean, when I started out I was very much of the opinion like this isn’t just the best thing I’ve ever written, this is the best thing I’ve ever read when I finished an article and very quickly that gets beaten out of you, right? Like when it comes back covered in tracked changes and red pen and what have you. But Heidi, what about you? When you file a story, what tends to come back?
Reporter: Heidi Ledford
What comes back? It varies a lot from story to story. I have to say, I don’t typically think this is the best thing I’ve ever read, I think this is the worst thing I’ve ever read, this is awful but I’m out of time and I’ve got to hand it in so then I hand it in. But yeah, I don’t know, it varies a lot depending on the story and how complicated it was and how I interpreted it and how Richard, for example, might interpret it. But when it comes to writing short, I guess, which is what Flora really has to do a lot of, I found that very difficult at first. We write these Research Highlights which are you know 120-130 word summaries of papers. And I remember when I first started at Nature, I mean every first draft I wrote was 250 words or 300, and it was just agony for me to cut it down, and now I don’t know, 10-12 years into this I can’t write them anything but 120-130 words. I think you just learn to find the key bits of information and just focus on those.
Editor: Richard Van Noorden
Just to give away some editorial secrets. So, I mean, once you’ve edited a lot you just think of the story in terms of the number of words it will be. So, someone says oh this has happened, and you think okay, so is that a 60-word NIB, or is that a 600-word story, or is that a 1,500-worder or are we talking 3,000 words for this? And you just get an instinct for what is a meaty story and what is a canapé of a story, and you just know what can be told to its natural length.
Editor: Flora Graham
Yeah, for me it’s like is this a one-sentence story, a three-sentence story or sometimes I mean, literally if it’s like neutron stars have revealed gravitational waves and I’m like five sentences, maybe even a bullet point!
Reporter: Heidi Ledford
But you know, the reporter’s perspective on what Richard just said is that I think all my stories deserve at least twice as many words as the editor thinks that it deserves. And it’s often painful for me to, you know, I often write them a little bit over and then I have to cut it down before I hand it to the editor or else they get very cranky. And then sometimes what happens though, is that if it’s going into print, you’ll write it a certain length, and then you’ll find out at the end oh, we have a couple lines we can add back. And a lot of times I’ll look at the story that I agonised over every cut and I’ll think hmm, no actually it’s good. It’s good the length it is.
Editor: Flora Graham
That is one of the truest things of journalism, is that the first edit is so painful and every word is precious, and then you have to take that deep breath and say oh, thank goodness that all came out at the end, it ended up so much better. But I also find that often there’s a disagreement on what’s the most important point, and that’s something that I really struggle with in the Briefing is for these stories I really have to choose. Of course, there’s all kinds of angles – is it the fact that this is a new discovery, is it the fact that it’s going to have a knock-on effect on research, is it the fact that it’s going to have a day-to-day effect on people’s lives? So, there’s always the challenge of picking that particular angle that I feel is the absolute most important angle.
Host: Benjamin Thompson
Okay then, well it’s time to move on to our final story, and I’d like to talk about something that came to our attention a couple of weeks back with a bunch of headlines talking about CRISPR, cancer and stock prices. Heidi, before we start, maybe you can tell us what CRISPR is?
Reporter: Heidi Ledford
CRISPR is a method that many research labs are using to make targeted changes to the genome. But in addition to research labs, there are also several companies out there who are hoping to design gene therapies essentially, that would one day be used to treat genetic diseases as well.
Host: Benjamin Thompson
And what’s the story that’s come up then in the last couple of weeks?
Reporter: Heidi Ledford
So, a couple of papers came out in Nature Medicine from two groups who were trying to use CRISPR, in particular cell lines, and they found that the process was really inefficient for the change that they were trying to make. And so, they looked a little more deeply, and they found that if you mutated a gene, encoding a protein called p53, you could getter a higher efficiency of this gene editing. And that sounds great because now you have a way to get this higher efficiency, but actually it’s terrible because you don’t want mutations in p53 if you’re trying to design a therapy and you’re going to put those cells back into a patient, because mutations in p53 are very strongly associated with developing cancer.
Host: Benjamin Thompson
So, these papers have come out then, and they’ve had a bit of a knock-on effect.
Reporter: Heidi Ledford
Yeah, that’s right. So, there was a lot of media coverage right after the papers came out, a lot of headlines saying CRISPR may cause cancer, CRISPR could cause cancer and so forth. I didn’t see any that said CRISPR will cause cancer, but still you know, the implication was pretty clear. So that spooked a number of investors, and we saw stock prices in some of these companies that are trying to develop therapies drop.
Host: Benjamin Thompson
And from what I understand, this isn’t actually the first time this has happened?
Reporter: Heidi Ledford
No at all. So, I would say over the past year or so it’s happened a number of times. There was a paper that came out last year in Nature Methods that was later retracted, but the paper said that this group had found many off-target effects of using CRISPR, so many unwanted genetic changes, and that caused stock prices to drop as well. There was another paper or a couple of papers earlier this year that talked about how many people have pre-existing antibodies against some of the components of CRISPR which may then make some sort of gene therapy based on CRISPR ineffective in those people – that also made stock prices bottom out for a while.
Host: Benjamin Thompson
But why CRISPR though? This is happening repeatedly, what is it about this technology that is maybe having such a sort of roller coaster effect?
Reporter: Heidi Ledford
It’s a number of things. I think there’s so much interest in this technology and it is so unproven when it comes to you know, gene therapy and human therapies and so forth. And yet these companies are out there, they have a heavy level of investment, quite a bit of money in them. So, everyone I think kind of recognises that this is hyped. It has been hyped, it has a lot of promise it is a very interesting technology, it has a lot of useful applications, but it has been overhyped and so everybody’s kind of waiting for that bubble to burst, I think. When you know, something comes along and oh, there’s a potential problem with this, it gets a lot of attention and then that attention spooks some of the investors and they drop the stock. Now, the stock – if you look at it over time though – is doing fine, so you get these dips, and then typically it comes back and in some cases, it goes higher and so forth.
Editor: Richard Van Noorden
So Heidi, I remember in news meetings you looking at this stuff and sort of sighing as if you know, here we go again! And we didn’t cover the latest findings, so how do you react to this as a science journalist when you see sensational headlines and stocks plunging on not-that-exciting papers?
Reporter: Heidi Ledford
I definitely have a level of fatigue, I think, with this cycle. And I do, in fact we were talking the other day, I do feel a bit of regret for not covering the Nature Medicine papers that just came out and they were perfectly fine papers. You know, they point out a perfectly legitimate issue that companies should be looking at, researchers should be looking at. It’s just that I would have thought they would have been looking at that anyway. I think what happened for me is that I felt like we would end up covering it the same way that we covered the papers that talked about pre-existing antibodies which was to do an Explainer and to say this result came out, here’s what it means, here’s what it doesn’t mean, here’s why you know, people should pay attention, here’s why people maybe shouldn’t panic, and you know, that kind of thing. I thought oh we just did that for these other papers and I don’t want to do this every time a paper comes out and says there’s this potential problem with this one potential application of CRISPR so, yeah. But in the end, you know, there’s so much reader interest that maybe I should have done it.
Host: Benjamin Thompson
Well, final question from me then. Heidi, you talked about headlines earlier and their implications, which perhaps links back a little bit to our previous conversation about compressing the news. I mean, there’s even less words I suppose in a headline – where does the responsibility lie to try to get that nuance in there? I mean can you even do it at all? Is it more just about getting eyeballs on an article itself?
Editor: Flora Graham
Well, that’s one of the great questions of modern journalism. I mean, of course, we all do our best to write factually accurate and compelling headlines. It’s a constant challenge and a constant balancing act. I think at Nature we have the benefit that our readers are looking for extremely accurate, precise, detailed, scientifically-valid coverage. Not every outlet has that benefit. If you’re a science reporter for a daily, you might have to argue with the editor that you deserve any space at all for that type of coverage. Well, the only thing you have to offer is that it’s of interest to your readers, so you have to present it to your readers in a way that they find compelling. I think that’s very valid, I mean, it’s always dealt with as a negative, but actually at the same time I think many of us are in agreement that people need to read about science and they need to, it needs to be on par with the latest reality show contestant to a certain extent. So, if we’re not getting it out there in a way that people find interesting then the whole enterprise is fairly pointless.
Host: Benjamin ThompsonWell there we have it. Many thanks to my guests: Flora Graham, Heidi Ledford and Richard Van Noorden for joining me here today. You can read their work and more stories from the world of science over at nature.com/news. And you can sign up for the Nature Briefing over at nature.com/briefing. If you want to get in contact with us, you can reach us on Twitter @NaturePodcast, or on email, we are . This has been Backchat, I’ve been Benjamin Thompson. Thanks for listening, see you all next time.
