Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Non-naïvety may reduce the effect of intuition manipulations

Main

To the Editor — In an experiment run on Amazon Mechanical Turk (MTurk), Camerer et al.1 failed to replicate the finding in study 8 of Rand et al.2 that promoting intuition via a recall induction increased cooperation. Before Camerer et al.1 began collecting data to replicate our study2, I cited prior evidence2,3 that experienced participants would not exhibit the original effect, and that many MTurkers are now experienced. At my request, Camerer et al.1 collected data on prior experience. As described in more detail below, examining these data shows that a large majority of their participants were experienced — and that analysing only inexperienced participants yields an effect size comparable to the original effect.

MTurk participants have become much more experienced with economic games since 2010, when our study 8 was run (this was before MTurk became widely used by academics). My colleagues and I have argued that experienced participants are typically more aware of — and attentive to — the strategic details of the game they are playing2,3. This attentiveness may enhance treatment effects based on variation in experimental details (like some MTurk experiments4,5 successfully replicated by Camerer et al.1) — but has been theorized to reduce the application of daily-life intuitions to atypical (one-shot anonymous) lab experiments2,3.

Accordingly, we showed that such experience undermines the intuition–cooperation effect in study 9 of our original paper2, which used a similar procedure to study 8 but was run nearly two years later. Study 9 found a non-significant overall effect of promoting intuition, but a significant interaction with experience: intuition increased cooperation only among participants who lacked prior experience with economic games. Subsequent work3 provided further evidence, showing that the time-pressure effect on cooperation decreased over two years as MTurk participants became increasingly experienced. It is therefore not surprising — and actually consistent with past results — that Camerer et al.1 failed to replicate study 8’s results using the original methods.

Before their data collection, I informed Camerer et al.1 that “[I]t’s been well documented that MTurkers are now highly experienced with economic game paradigms”, and “Study 9 of our paper shows that non-naïvety specifically undermines the treatment effect you are replicating.” I therefore asked that they include our standard question assessing prior experience with economic games, enabling them to compare “people that answered 1 [naïve] to everyone else [non-naïve]”. Camerer et al.1 collected these data, but did not analyse them.

These data show that 82.8% of the participants in Camerer et al.1 had prior experience with economic games. When analysing only the 367 naïve participants, the effect size was similar to our original study: Tobit regression coefficient on a ‘promote intuition’ dummy of b = 9.37, compared with b = 10.95 in the original study. (The naïve-only effect does not reach statistical significance, z = 1.61, P = 0.054 one-tailed given clear directional hypothesis; however, when including only naïve participants, the analysis is under-powered to detect an effect of the anticipated (and observed) size.)

Importantly, I am not arguing that it is now impossible to study intuition and cooperation on MTurk. It just requires more powerful manipulations. For example, Levine et al.6 conceptually replicate the present results by directly instructing participants to decide using emotion versus reason (rather than our more indirect recall induction); and Everett et al.7 and Isler et al.8 reproduce our original time-constraint results using improved methods that avoid non-compliance. In sum, the weight of the evidence confirms our initial conclusion that intuition promotes cooperation in social dilemmas, but reliably reproducing these effects requires updated methods.

References

  1. 1.

    Camerer, C. F. et al. Nat. Hum. Behav. https://doi.org/10.1038/s41562-018-0399-z (2018).

    Article  Google Scholar 

  2. 2.

    Rand, D. G., Greene, J. D. & Nowak, M. A. Nature 489, 427–430 (2012).

    CAS  Article  Google Scholar 

  3. 3.

    Rand, D. G. et al. Nat. Commun. 5, 3677 (2014).

    CAS  Article  Google Scholar 

  4. 4.

    Nishi, A., Shirado, H., Rand, D. G. & Christakis, N. A. Nature 526, 426–429 (2015).

    CAS  Article  Google Scholar 

  5. 5.

    Hauser, O. P., Rand, D. G., Peysakhovich, A. & Nowak, M. A. Nature 511, 220–223 (2014).

    CAS  Article  Google Scholar 

  6. 6.

    Levine, E. E., Barasch, A., Rand, D., Berman, J. Z. & Small, D. A. J. Exp. Psychol. Gen. 147, 702–719 (2018).

    Article  Google Scholar 

  7. 7.

    Everett, J., Ingbretsen, Z., Cushman, F. A. & Cikara, M. J. Exp. Soc. Psychol. 73, 76–81 (2017).

    Article  Google Scholar 

  8. 8.

    Isler, O., Maule, J., Starmer, C. & Georgantzis, N. PLoS One 13, e0190560 (2018).

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to David G. Rand.

Ethics declarations

Competing interests

The author declares no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rand, D.G. Non-naïvety may reduce the effect of intuition manipulations. Nat Hum Behav 2, 602 (2018). https://doi.org/10.1038/s41562-018-0404-6

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing