Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Disputed results a fresh blow for social psychology

17 May 2013 We would like to clarify the following points about this News story: • Contrary to the story, unconscious thought theory, which is concerned with unconscious decision making, is not the same as intelligence priming. • The story refers to the study by David Shanks (D. R. Shanks et al. PLoS ONE 8, e56515; 2013) in which he was unable to replicate the ‘professor/hooligan’ intelligence-priming effect reported by Ap Dijksterhuis. We note that other researchers have observed Dijksterhuis’s intelligence-priming effect. • Contrary to the story, Dijksterhuis has stated that he will provide a protocol to assist in replicating results in the field. He had not, however, provided such a protocol at the time that the story was published. • As the story states, there is no suggestion of fraud in Dijksterhuis’s research, and we are happy to emphasize that there is no suggestion that he has been involved in any misconduct.

A Correction to this article was published on 22 May 2013

This article has been updated

Failure to replicate intelligence-priming effects ignites row in research community.

Social psychologist Ap Dijksterhuis. Credit: Flip Franssen/Hollandse Hoogte/eyevine

Thinking about a professor just before you take an intelligence test makes you perform better than if you think about football hooligans. Or does it? An influential theory that certain behaviour can be modified by unconscious cues is under serious attack.

A paper published in PLoS ONE last week1 reports that nine different experiments failed to replicate this example of ‘intelligence priming’, first described in 1998 (ref. 2) by Ap Dijksterhuis, a social psychologist at Radboud University Nijmegen in the Netherlands, and now included in textbooks.

David Shanks, a cognitive psychologist at University College London, UK, and first author of the paper in PLoS ONE, is among sceptical scientists calling for Dijksterhuis to design a detailed experimental protocol to be carried out indifferent laboratories to pin down the effect. Dijksterhuis has rejected the request, saying that he “stands by the general effect” and blames the failure to replicate on “poor experiments”.

An acrimonious e-mail debate on the subject has been dividing psychologists, who are already jittery about other recent exposures of irreproducible results (see Nature 485, 298–300; 2012). “It’s about more than just replicating results from one paper,” says Shanks, who circulated a draft of his study in October; the failed replications call into question the under­pinnings of ‘unconscious-thought theory’.

Dijksterhuis published that theory in 2006 (ref. 3). It fleshed out more general, long-held claims about a ‘smart unconscious’ that had been proposed over the past couple of decades — exemplified in writer Malcolm Gladwell’s best-selling book Blink (Penguin, 2005). The theory holds that behaviour can be influenced, or ‘primed’, by thoughts or motives triggered unconsciously — in the case of intelligence priming, by the stereotype of a clever professor or a stupid hooligan. Most psychologists accept that such priming can occur consciously, but many, including Shanks, are unconvinced by claims of unconscious effects.

In their paper, Shanks and his colleagues tried to obtain an intelligence-priming effect, following protocols in Dijksterhuis’s papers or refining them to amplify any theoretical effect (for example, by using a test of analytical thinking instead of general knowledge). They also repeated intelligence-priming studies from independent labs. They failed to find any of the described priming effects in their experiments.

The e-mail debate that Shanks joined was kicked off last September, when Daniel Kahneman, a Nobel-prizewinning psychologist from Princeton University in New Jersey who thinks that unconscious social priming is likely to be real, circulated an open letter warning of a “train wreck looming” (see Naturehttp://doi.org/mdr;2012) because of a growing number of failures to replicate results. Social psychology “is now the poster child for doubts about the integrity of psychological research”, he told psychologists, “and it is your responsibility” to deal with it.

“ It’s about more than just replicating the results from one paper. ”

Other high-profile social psychologists whose papers have been disputed in the past two years include John Bargh from Yale University in New Haven, Connecticut. His claims include that people walk more slowly if they are primed with age-related words.

Bargh, Dijksterhuis and their supporters argue that social-priming results are hard to replicate because the slightest change in conditions can affect the outcome. “There are moderators that we are unaware of,” says Dijksterhuis.

But Hal Pashler, a cognitive psychologist at the University of California, San Diego — a long-time critic of social priming — notes that the effects reported in the original papers were huge. “If effects were that strong, it is unlikely they would abruptly disappear with subtle changes in procedure,” he says.

No one is suggesting that there is anything fraudulent about the results, but the charges that some of Dijksterhuis’s key papers may report false positives is a particular embarrassment for the Netherlands. It comes close on the heels of exposures of scientific misconduct by two other Dutch social psychologists: in 2011, Diederik Stapel of Tilburg University admitted to inventing data, and in June 2012, an investigation committee concluded that Dirk Smeesters from the Erasmus University in Rotterdam had cherry-picked data in some papers.

Shanks’s replication failures cannot be dismissed, says Eric-Jan Wagenmakers, a mathematical psychologist at the University of Amsterdam who last year published a series of studies that failed to lend support4 to unconscious-thought theory. He is disappointed that Dijksterhuis has declined “repeated requests” to help to generate a definitive answer.

Dijksterhuis says that “focusing on a single phenomenon is not that helpful and won’t solve the problem”. He adds that social psychology needs to get more rigorous, but that the rigour should be applied to future, not historical, experiments. The social-priming debate will rumble on, he says, because “there is an ideology out there that doesn’t want to believe that our behaviour can be cued by the environment”.

Others remain concerned. Kahneman wrote in the e-mail debate on 4 February that this “refusal to engage in a legitimate scientific conversation … invites the interpretation that the believers are afraid of the outcome”.

figurea

Flip Franssen/Hollandse Hoogte/eyevine

Change history

  • 17 May 2013

    We would like to clarify the following points about this News story: • Contrary to the story, unconscious thought theory, which is concerned with unconscious decision making, is not the same as intelligence priming. • The story refers to the study by David Shanks (D. R. Shanks et al. PLoS ONE 8, e56515; 2013) in which he was unable to replicate the ‘professor/hooligan’ intelligence-priming effect reported by Ap Dijksterhuis. We note that other researchers have observed Dijksterhuis’s intelligence-priming effect. • Contrary to the story, Dijksterhuis has stated that he will provide a protocol to assist in replicating results in the field. He had not, however, provided such a protocol at the time that the story was published. • As the story states, there is no suggestion of fraud in Dijksterhuis’s research, and we are happy to emphasize that there is no suggestion that he has been involved in any misconduct.

  • 22 May 2013

    A Correction to this paper has been published: https://doi.org/10.1038/497423a

References

  1. 1

    Shanks, D. R. et al. PLoS ONE 8, e56515 (2013).

    ADS  CAS  Article  Google Scholar 

  2. 2

    Dijksterhuis, A. & van Knippenberg, A. J. Pers. Soc. Psychol. 74, 865–877 (1998).

    CAS  Article  Google Scholar 

  3. 3

    Dijksterhuis, A. & Nordgren, L. F. Persp. Psychol. Sci. 1, 95–109 (2006).

    Article  Google Scholar 

  4. 4

    Huizenga, H. M., Wetzels, R., van Ravenzwaaji, D. & Wagenmakers, E.-J. Organ. Behav. Hum. Decis. Proc. 117, 332–340 (2012).

    Article  Google Scholar 

Download references

Authors

Additional information

See Correspondence ‘Reproducibility: Priming-effect author responds’

Related links

Related links

Related links in Nature Research

Announcement: Reducing our irreproducibility 2013-Apr-24

Nobel laureate challenges psychologists to clean up their act 2012-Oct-03

Replication studies: Bad copy 2012-May-16

Related external links

PsychFileDrawer

Kahneman’s open letter from 26 September (PDF)

Perspectives in Psychological Science register replication reports

Cortex registered reports (PDF)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Abbott, A. Disputed results a fresh blow for social psychology. Nature 497, 16 (2013). https://doi.org/10.1038/497016a

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing