Editorial


Nature Neuroscience 10, 1 (2007)
doi:10.1038/nn0107-1

Setting the record straight


The discovery of serious errors in two recent papers in the journal leads to lessons for authors, referees and editors.


Mistakes happen in all areas of life, including scientific research. Peer review is intended to catch errors before they are made permanent in the public record, but occasionally errors slip through and are published. Though rare cases involve scientific misconduct, most errors result from unintentional problems with experimental design, execution or analysis. Regardless of the mistake's origin, journals bear a responsibility to acknowledge errors when they occur and to set the record straight for the scientific community. In that spirit, we have two important corrections to report in this issue.

The first correction involves a Brief Communication1 reporting that inhibition of the enzyme that breaks down the endocannabinoid 2-arachidonoylglycerol enhances retrograde signaling in the hippocampus. The authors concluded that 2-arachidonoylglycerol is important for synaptic plasticity and that the enzyme is a possible drug target, in part because one of the putative inhibitors tested appeared to be specific for the enzyme. They subsequently discovered that the commercial preparation of this drug was contaminated. When the contaminant was eliminated, the effect disappeared. The contaminant affects several neurotransmitter systems and thus, unlike the drug, is not of therapeutic interest. The authors contacted us upon discovering the error and carried out experiments to verify that inhibition of the enzyme was indeed responsible for the effect on retrograde signaling2, as others have also found3. It remains unclear whether the enyzme holds promise as a drug target, as the known inhibitors are not very specific in their effects.

The second correction is more complex. The original article4 reported high-resolution fMRI measurements in the fusiform face area (FFA), a region of the visual cortex that responds more to faces than to other visual stimuli. The authors drew two conclusions: that the FFA is heterogeneous, in that the degree of selectivity varies over the region, and—more remarkably—that the FFA contains some voxels that are highly selective for object categories other than faces. After the paper was published, two groups wrote to point out flaws in the analysis. One letter5 noted that the authors used a formula for selectivity that erroneously assigns high selectivity values to voxels with negative responses to nonpreferred categories, causing a substantial overestimate in selectivity for all object categories.

Another group6 spotted a more subtle flaw: the analysis used to demonstrate selectivity for particular categories did not distinguish between random variation and replicable effects reflecting neural tuning. Random variation can cause some voxels to respond more to some categories than to others. To demonstrate that such differences reflect neural selectivity requires an appropriate statistical analysis, for instance cross-validation across independent datasets. The original paper seemed to report the results of such an analysis—that voxel selectivity was highly correlated between even and odd scans. However, communication with the authors revealed that this analysis had excluded voxels whose responses were negatively correlated across the two sets of scans, a detail that was omitted from the paper. This restriction could falsely increase consistency across scans. Indeed, when the authors redid their analysis without it, the selectivity for nonface objects was not replicated from one set of scans to the next.

The authors of the article acknowledge both errors in their correction7. When these errors are fixed, the most interesting conclusion of the paper—that the FFA contains voxels highly selective for nonface objects—is no longer supported. The re-analyzed data from the article fail to reveal voxels in the FFA that respond significantly more to nonface objects than to faces. The other conclusion of the paper, that selectivity in the FFA is heterogeneous, remains valid; some voxels are more selective than others when examined at high resolution.

In both cases, after considerable discussion with colleagues, we have decided to publish a correction2, 7 to the original paper rather than a retraction, even though it seems likely that neither paper would have been published in Nature Neuroscience had the errors been identified and corrected during the review process. Retractions were deemed inappropriate because they would have removed from the record some valid data and conclusions that are likely to be useful to specialists in the field, and it seemed unlikely that the authors would be able to publish these data elsewhere.

Should the problems with these papers have been recognized before publication? The contamination of a commercially produced drug would have been very difficult for referees, or even the authors, to detect. On the other hand, the error in the formula for selectivity probably should have been discovered during the review process. The incorrect formula was given in the paper, and the selectivity histograms show many values clustered around one, which should have been a red flag to a mathematically sophisticated referee. The cross-validation issue would have been harder to detect, as part of what made this analysis flawed is a detail of the methods that was omitted from the paper. Of course, the ultimate responsibility for recruiting referees with appropriate expertise lies with the editors, and in this case we clearly should have consulted referees with stronger mathematical expertise.

Nonetheless, it is common practice in functional imaging (and indeed in other areas of neuroscience) to analyze experiments by selecting data according to some criteria and then plotting the average response, without testing an independent data set to ensure that the selection criteria have not merely picked out random variation in a particular direction. We hope that one positive outcome of this correction will be to give authors, referees and editors an increased awareness of the hazards of this approach, so that such mistakes may be avoided in future studies.

Top

References

  1. Makara, J.K. et al. Nat. Neurosci. 8, 1139–1141 (2005). | Article | PubMed | ISI | ChemPort |
  2. Makara, J.K. et al. Nat. Neurosci. 10, 134 (2007). | Article | ISI | ChemPort |
  3. Szabo, B. et al. J. Physiol. (Lond.) 577, 263–280 (2006). | Article | PubMed | ChemPort |
  4. Grill-Spector, K., Sayres, R. & Ress, D. Nat. Neurosci. 9, 1177–1185 (2006). | Article | PubMed | ChemPort |
  5. Simmons, W.K., Bellgowan, P.S.F. & Martin, A. Nat. Neurosci. 10, 4–5 (2007). | Article | PubMed | ChemPort |
  6. Baker, C.I., Hutchinson, T.L. & Kanwisher, N. Nat. Neurosci. 10, 3–4 (2007). | Article | PubMed | ChemPort |
  7. Grill-Spector, K, Sayres, R. & Ress, D. Nat. Neurosci. 10, 133 (2007). | Article | ChemPort |

MORE ARTICLES LIKE THIS

These links to content published by NPG are automatically generated.

NEWS AND VIEWS

Fine structure in representations of faces and objects

Nature Neuroscience News and Views (01 Sep 2006)