To the Editor — We applaud the Social Sciences Replication Project (SSRP) team’s1 effort in replicating multiple experiments, including ours, and appreciate their communication throughout the process. This Correspondence addresses a methodological deviation of the replication from the original experiment.
In the original experiment2, participants examined covers of 30 music albums; picked 10 they would like to own; wrote down these 10 albums’ titles and artist names; and ranked them in order of preference. After a filler task, they chose between their fifth and sixth ranked album as a token of appreciation and worked on an allegedly unrelated task that included examining a bottle of liquid soap or testing it by washing their hands. Meanwhile, the experimenter prepared a different form by listing the 10 albums in alphabetical order of artist names (rather than album titles), thus ensuring that the presentation order of albums would differ at t1 (pre-choice) and t2 (post-washing) rankings. We considered this different ordering necessary because changes in ranking from t1 to t2 become less likely the easier participants can recall their t1 rankings.
In the replication, the experimenter “inadvertently neglected [this step]. Instead, the presentation order of albums in the post-manipulation evaluation was the same as what was in the pre-manipulation evaluation” (https://osf.io/yhx49/). This may have reduced the SSRP study’s chances of finding a significant effect of washing away postdecisional dissonance. Postdecisional dissonance involves multiple processes, from an aversive feeling and its expression and attribution3,4,5 to memory and performance monitoring6,7. Theoretically, postdecisional dissonance can be ‘washed away’ because physical cleansing reduces residual feelings from recent behaviours, as observed in many labs across contexts, domains and cultures (for a comprehensive review, see ref. 8). Whenever an evaluation can be based on multiple inputs, the impact of any given input decreases as the accessibility of competing inputs increases. Hence, the SSRP’s inadvertent methodological deviation may have increased reliance on memory of the earlier evaluation at the expense of reliance on the affective processes that motivated the original experiments in the first place.
This possibility is difficult to evaluate with the available data. The SSRP team’s exploratory analysis does not resolve these concerns. That analysis merely identified whether participants transcribed album titles at t1 in order of ranks or not, based on the assumption that those who transcribed would have better memory than those who did not. This is possible, but sheds no light on participants’ memory once the t2 form facilitated recall by presenting all albums in exactly the same order as the t1 form. Independent experiments that did not inadvertently introduce memory cues supported the original findings, indicating that postdecisional dissonance can be washed away (see ref. 9, experiment 2 of ref. 2, and M. Marotta and G. Bohner, presentation at the Tagung der Fachgruppe Sozialpsychologie, Hagen, Germany, 2013). A meta-analysis of all available studies, including the SSRP replication, is available at https://go.nature.com/2uJfpdu. Future research may fruitfully address the impact of affective and cognitive processes in cleansing effects on postdecisional dissonance.
Camerer, C. F. et al. Nat. Hum. Behav. https://doi.org/10.1038/s41562-018-0399-z (2018).
Lee, S. W. S. & Schwarz, N. Science 328, 709 (2010).
Brehm, J. W. J. Appl. Soc. Psychol. 52, 384–389 (1956).
Pyszczynski, T., Greenberg, J., Solomon, S., Sideris, J. & Stubing, M. J. J. Pers. Soc. Psychol. 64, 177–186 (1993).
Zanna, M. P. & Cooper, J. J. Pers. Soc. Psychol. 29, 703–709 (1974).
Colosio, M., Shestakova, A., Nikulin, V. V., Blagovechtchenski, E. & Klucharev, V. J. Neurosci. 37, 5074–5083 (2017).
Van Overwalle, F. & Jordens, K. Pers. Soc. Psychol. Rev. 6, 204–231 (2002).
Lee, S. W. S. & Schwarz, N. in Purity and Danger Now: New Perspectives (eds Duschinsky, R., Schnall, S. & Weiss, D. H.) 136–161 (Routledge, New York, NY, 2016).
De Los Reyes, A., Aldao, A., Kundey, S. M. A., Lee, B. G. & Molina, S. J. Clin. Psychol. 68, 1–7 (2012).
The authors declare no competing interests.
About this article
Cite this article
Lee, S.W.S., Schwarz, N. Methodological deviation from the original experiment. Nat Hum Behav 2, 605 (2018). https://doi.org/10.1038/s41562-018-0403-7
This article is cited by
Nature Human Behaviour (2022)
European Journal for Philosophy of Science (2021)