Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

A 2 million-person, campaign-wide field experiment shows how digital advertising affects voter turnout

Nature Human Behaviour volume 7, pages 332–341 (2023)Cite this article

Subjects

Abstract

We present the results of a large, US$8.9 million campaign-wide field experiment, conducted among 2 million moderate- and low-information persuadable voters in five battleground states during the 2020 US presidential election. Treatment group participants were exposed to an 8-month-long advertising programme delivered via social media, designed to persuade people to vote against Donald Trump and for Joe Biden. We found no evidence that the programme increased or decreased turnout on average. We found evidence of differential turnout effects by modelled level of Trump support: the campaign increased voting among Biden leaners by 0.4 percentage points (s.e. = 0.2 pp) and decreased voting among Trump leaners by 0.3 percentage points (s.e. = 0.3 pp) for a difference in conditional average treatment effects of 0.7 points (t1,035,571 = −2.09; P = 0.036; \(\widehat{{\rm{DIC}}}=0.7\) points; 95% confidence interval = −0.014 to 0). An important but exploratory finding is that the strongest differential effects appear in early voting data, which may inform future work on early campaigning in a post-COVID electoral environment. Our results indicate that differential mobilization effects of even large digital advertising campaigns in presidential elections are likely to be modest.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Average treatment effects and CATEs of treatment on 2020 turnout under three inverse probability-weighted regression specifications.
Fig. 2: Heterogeneous effects of treatment on voting, in-person voting and early voting by Trump support under three inverse probability-weighted regression specifications.
Fig. 3: 2020 turnout rates by one-point bins of Trump support score and condition.
Fig. 4: Treatment Assignment Flow Chart.
Fig. 5: Balance on pre-treatment covariates.
Fig. 6: Examples of typical advertisement content run in Acronym’s persuasion program.

Data availability

An anonymized replication dataset is available via Dataverse at https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YMKVA1. TargetSmart generously agreed to make replication data available for this paper. By downloading replication data, researchers agree to use the data only for academic research, agree not to share the data with outside parties and agree not to attempt to re-identify individuals in the dataset in order to download the data.

Code availability

Replication scripts are available via Dataverse at https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YMKVA1.

References

  1. Romer, D., Kenski, K., Winneg, K., Adasiewicz, C. & Jamieson, K. H. Capturing Campaign Dynamics, 2000 & 2004: The National Annenberg Election Survey (Univ. Pennsylvania Press, 2006).

  2. Sides, J., Vavreck, L. & Warshaw, C. The effect of television advertising in United States elections. Am. Polit. Sci. Rev. 116, 702–718 (2022).

    Article  Google Scholar 

  3. Druckman, J. N. The implications of framing effects for citizen competence. Polit. Behav. 23, 225–256 (2001).

    Article  Google Scholar 

  4. Druckman, J. N. & Holmes, J. W. Does presidential rhetoric matter? Priming and presidential approval. Pres. Stud. Q. 34, 755–778 (2004).

    Article  Google Scholar 

  5. Coppock, A., Hill, S. J. & Vavreck, L. The small effects of political advertising are small regardless of context, message, sender, or receiver: evidence from 59 real-time randomized experiments. Sci. Adv. 6, 36 (2020).

    Article  Google Scholar 

  6. Chong, D. & Druckman, J. N. A theory of framing and opinion formation in competitive elite environments. J. Commun. 57, 99–118 (2007).

    Google Scholar 

  7. Zaller, J. R. The Nature and Origins of Mass Opinion (Cambridge Univ. Press, 1992).

  8. Vavreck, L. The Message Matters: The Economy and Presidential Campaigns (Princeton Univ. Press, 2009).

  9. Gerber, A. S., Gimpel, J. G., Green, D. P. & Shaw, D. R. How large and long-lasting are the persuasive effects of televised campaign ads? Results from a randomized field experiment. Am. Polit. Sci. Rev. 105, 135–150 (2011).

    Article  Google Scholar 

  10. Kalla, J. L. & Broockman, D. E. The minimal persuasive effects of campaign contact in general elections: evidence from 49 field experiments. Am. Polit. Sci. Rev. 112, 148–166 (2018).

    Article  Google Scholar 

  11. Mellman, M. Are we doing it all wrong? The Hill https://thehill.com/opinion/campaign/353720-mark-mellman-are-we-doing-it-all-wrong (2017).

  12. Spenkuch, J. L. & Toniatti, D. Political advertising and election results. Q. J. Econ. 133, 1981–2036 (2018).

    Article  Google Scholar 

  13. Ansolabehere, S., Iyengar, S., Simon, A. & Valentino, N. Does attack advertising demobilize the electorate? Am. Polit. Sci. Rev. 88, 829–838 (1994).

    Article  Google Scholar 

  14. Wattenberg, M. P. & Brians, C. L. Negative campaign advertising: demobilizer or mobilizer?. Am. Polit. Sci. Rev. 93, 891–899 (1999).

    Article  Google Scholar 

  15. Ansolabehere, S. D., Iyengar, S. & Simon, A. Replicating experiments using aggregate and survey data: the case of negative advertising and turnout. Am. Polit. Sci. Rev. 93, 901–909 (1999).

    Article  Google Scholar 

  16. Kahn, K. F. & Kenney, P. J. Do negative campaigns mobilize or suppress turnout? Clarifying the relationship between negativity and participation. Am. Polit. Sci. Rev. 93, 877–889 (1999).

    Article  Google Scholar 

  17. Freedman, P. & Goldstein, K. Measuring media exposure and the effects of negative campaign ads. Am. J. Polit. Sci. 43, 1189–1208 (1999).

    Article  Google Scholar 

  18. Goldstein, K. & Freedman, P. Campaign advertising and voter turnout: new evidence for a stimulation effect. J. Polit. 64, 721–740 (2002).

    Article  Google Scholar 

  19. Finkel, S. E. & Geer, J. G. A spot check: casting doubt on the demobilizing effect of attack advertising. Am. J. Polit. Sci. 42, 573–595 (1998).

    Article  Google Scholar 

  20. Krupnikov, Y. When does negativity demobilize? Tracing the conditional effect of negative campaigning on voter turnout. Am. J. Polit. Sci. 55, 797–813 (2011).

    Article  Google Scholar 

  21. Lau, R. R. & Rovner, I. B. Negative campaigning. Annu. Rev. Polit. Sci. 12, 285–306 (2009).

    Article  Google Scholar 

  22. Schuirmann, D. J. A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. J. Pharmacokinet. Biopharm. 15, 657–680 (1987).

    Article  CAS  PubMed  Google Scholar 

  23. Lakens, D. Equivalence tests: a practical primer for t tests, correlations, and meta-analyses. Soc. Psychol. Pers. Sci. 8, 355–362 (2017).

    Article  Google Scholar 

  24. Samuelsohn, D. Facebook: Russian-linked accounts bought $150,000 in ads during 2016 race. Politico https://www.politico.com/story/2017/09/06/facebook-ads-russia-linked-accounts-242401 (2017).

  25. Redlawsk, D. Feeling Politics: Emotion in Political Information Processing (Springer, 2006).

  26. Albertson, B., Dun, L. & Kushner Gadarian, S. in The Oxford Handbook of Electoral Persuasion (eds Suhay, E. et al.) 169–183 (Oxford Univ. Press, 2020).

  27. Broockman, D. E. & Kalla, J. L. When and why are campaigns’ persuasive effects small? Evidence from the 2020 U.S. presidential election. Am. J. Pol. Sci. https://doi.org/10.1111/ajps.12724 (2022).

  28. Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B Methodol. 57, 289–300 (1995).

    Google Scholar 

  29. Fowler, E.F., Franz, M. & Ridout, T.N. Political advertising in the United States (Routledge, 2021).

  30. Broockman, D. & Kalla, J. The impacts of selective partisan media exposure: a field experiment with Fox News viewers. Preprint at OSF https://doi.org/10.31219/osf.io/jrw26 (2022).

  31. House GOP campaign arm created misleading fake news sites. Talking Points Memo https://talkingpointsmemo.com/livewire/nrcc-fake-news-sites (2014).

  32. Shaw, D. Blue dog-affiliated website blurs line between newsroom and political advertiser. Sludge https://readsludge.com/2021/09/17/blue-dog-affiliated-website-blurs-line-between-newsroom-and-political-advertiser/ (2021).

  33. Fischer, S. Facebook says new pre-election political ad rules apply to boosted posts. Axios https://www.axios.com/facebook-says-new-pre-election-political-ad-rules-apply-to-boosted-posts-ecbab5e5-48c8-42b4-878f-aa7581ed6186.html (2020).

  34. 2020: Working America. Fight for a Better America (2020). https://www.fightforbetter.org/workingamerica

  35. Levy, A., Rodriguez, S. & Graham, M. Why political campaigns are flooding facebook with ad dollars. CNBC https://www.cnbc.com/2020/10/08/trump-biden-pacs-spend-big-on-facebook-as-election-nears.html (2020).

  36. R Core Development Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2022).

  37. Wickham, H. et al. Welcome to the tidyverse. J. Open Source Softw. 4, 1686 (2019).

    Article  Google Scholar 

  38. Blair, G., Cooper, J., Coppock, A., Humphreys, M. & Sonnet, L. estimatr: Fast estimators for design-based inference. R package version 0.30.6 https://CRAN.R-project.org/package=estimatr (2022).

  39. Fox, J. & Weisberg, S. An R Companion to Applied Regression 3rd edn (Sage, 2019).

Download references

Acknowledgements

We thank D. Green, F. Sävje, M. Michelson, B. Sinclair, E. Porter, L. Vavreck, Y. Velez, J. Kalla and D. Broockman for generous early feedback. We also thank E. Franklin Fowler and the Wesleyan Media Project for generously sharing data. We received no specific external funding for this work.

Author information

Author notes

  1. These authors contributed equally: Minali Aggarwal, Jennifer Allen, Alexander Coppock, Dan Frankowski, Solomon Messing, Kelly Zhang.

Authors and Affiliations

  1. Department of Political Science, Yale University, New Haven, CT, USA

    Minali Aggarwal & Alexander Coppock

  2. MIT Sloan School of Management, Boston, MA, USA

    Jennifer Allen

  3. Boxydog, Minneapolis, MN, USA

    Dan Frankowski

  4. McCourt School of Public Policy, Georgetown University, Washington, DC, USA

    Solomon Messing

  5. Busara Center for Behavioral Economics, Nairobi, Kenya

    Kelly Zhang

  6. Condorsay, New York, NY, USA

    James Barnes

  7. Macmillan Publishers, New York, NY, USA

    Andrew Beasley

  8. Greenlight, New York, NY, USA

    Harry Hantman

  9. Department of Politics, New York University, New York, NY, USA

    Sylvan Zheng

Authors
  1. Minali Aggarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennifer Allen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Coppock
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dan Frankowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Solomon Messing
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kelly Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. James Barnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Andrew Beasley
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Harry Hantman
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sylvan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.M. and J.B. conceived of the initial research. S.M. designed and implemented the original experiment. A.B. and H.H. administered the experiment. D.F., M.A., K.Z., S.Z., J.A. and S.M. contributed to data collection and curation. M.A., J.A. and A.C. analysed the results of the experiment. A.C. and S.M. drafted the initial manuscript and figures. All authors contributed to the revision and editing of the manuscript.

Corresponding author

Correspondence to Solomon Messing.

Ethics declarations

Competing interests

All researchers were employed by Acronym or were contractors thereof during the 2020 election cycle. J.A., S.Z. and H.H. have a substantial financial interest in Facebook. Acronym played a role in conceptualizing and designing the study presented here: all of the authors except A.C. contributed to the design and/or implementation of the experiment while employed by Acronym from January 2020 to January 2021. After the termination of their employment in January 2021, but before working on this manuscript, the other primary authors signed explicit third-party data-sharing agreements on 8 February 2021 to allow data access while they conducted the scientific analysis and reporting presented here, with other institutions. Acronym and the authors agreed in writing to the publication of this manuscript in advance of manuscript analysis and preparation. The authors agreed to provide Acronym with a draft of the manuscript before publication.

Peer review

Peer review information

Nature Human Behaviour thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Matching of treatment voters to Facebook users, a description of the advertisement content, a description of the advertisement testing programme, analysis of the overall Facebook advertisement environment, an overview of the PAP, PAP deviations, randomization code and Supplementary Tables 1–3 and Figs. 1–4.

Reporting Summary

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aggarwal, M., Allen, J., Coppock, A. et al. A 2 million-person, campaign-wide field experiment shows how digital advertising affects voter turnout. Nat Hum Behav 7, 332–341 (2023). https://doi.org/10.1038/s41562-022-01487-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-022-01487-4

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing