If popular online platforms systematically expose their users to partisan and unreliable news, they could potentially contribute to societal issues such as rising political polarization1,2. This concern is central to the ‘echo chamber’3,4,5 and ‘filter bubble’6,7 debates, which critique the roles that user choice and algorithmic curation play in guiding users to different online information sources8,9,10. These roles can be measured as exposure, defined as the URLs shown to users by online platforms, and engagement, defined as the URLs selected by users. However, owing to the challenges of obtaining ecologically valid exposure data—what real users were shown during their typical platform use—research in this vein typically relies on engagement data4,8,11,12,13,14,15,16 or estimates of hypothetical exposure17,18,19,20,21,22,23. Studies involving ecological exposure have therefore been rare, and largely limited to social media platforms7,24, leaving open questions about web search engines. To address these gaps, we conducted a two-wave study pairing surveys with ecologically valid measures of both exposure and engagement on Google Search during the 2018 and 2020 US elections. In both waves, we found more identity-congruent and unreliable news sources in participants’ engagement choices, both within Google Search and overall, than they were exposed to in their Google Search results. These results indicate that exposure to and engagement with partisan or unreliable news on Google Search are driven not primarily by algorithmic curation but by users’ own choices.
This is a preview of subscription content, access via your institution
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
Owing to privacy concerns and IRB limitations, visit-level data will not be released, but aggregated data are available at https://doi.org/10.7910/DVN/WANAX3. The domain scores and classifications we used are available at https://github.com/gitronald/domains, but the NewsGuard classifications are not included because of their proprietary nature.
Wagner, C. et al. Measuring algorithmically infused societies. Nature 595, 197–204 (2021).
Rahwan, I. et al. Machine behaviour. Nature 568, 477–486 (2019).
Sunstein, C. R. Republic.com (Princeton Univ. Press, 2001).
Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W. & Starnini, M. The echo chamber effect on social media. Proc. Natl Acad. Sci. 118, e2023301118 (2021).
Lewandowsky, S., Ecker, U. K. H. & Cook, J. Beyond misinformation: understanding and coping with the ‘post-truth’ era. J. Appl. Res. Mem. Cogn. 6, 353–369 (2017).
Pariser, E. The Filter Bubble: What the Internet is Hiding From You (Penguin, 2011).
Bakshy, E., Messing, S. & Adamic, L. A. Exposure to ideologically diverse news and opinion on Facebook. Science 348, 1130–1132 (2015).
Cardenal, A. S., Aguilar-Paredes, C., Galais, C. & Pérez-Montoro, M. Digital technologies and selective exposure: how choice and filter bubbles shape news media exposure. Int. J. Press/Politics 24, 465–486 (2019).
Fletcher, R., Kalogeropoulos, A. & Nielsen, R. K. More diverse, more politically varied: how social media, search engines and aggregators shape news repertoires in the United Kingdom. New Media & Society https://doi.org/10.1177/14614448211027393 (2021).
Ekström, A. G., Niehorster, D. C. & Olsson, E. J. Self-imposed filter bubbles: selective attention and exposure in online search. Comput. Hum. Behav. Reports https://doi.org/10.1016/j.chbr.2022.100226 (2022).
Hosseinmardi, H. et al. Examining the consumption of radical content on YouTube. Proc. Natl Acad. Sci. 118, e2101967118 (2021).
Chen, W., Pacheco, D., Yang, K.-C. & Menczer, F. Neutral bots probe political bias on social media. Nat. Commun. 12, 5580 (2021).
Guess, A. M., Nyhan, B. & Reifler, J. Exposure to untrustworthy websites in the 2016 US election. Nat. Hum. Behav. 4, 472–480 (2020).
Allen, J., Howland, B., Mobius, M., Rothschild, D. & Watts, D. J. Evaluating the fake news problem at the scale of the information ecosystem. Sci. Adv. 6, eaay3539 (2020).
Muise, D. et al. Quantifying partisan news diets in Web and TV audiences. Sci. Adv. 8, eabn0083 (2022).
Garimella, K., Smith, T., Weiss, R. & West, R. Political polarization in online news consumption. Proc. International AAAI Conference on Web and Social Media 15, 152–162 (2021).
Hannak, A. et al. Measuring personalization of web search. In Proc. 22nd International Conference on World Wide Web, 527–538 https://doi.org/10.1145/2488388.2488435 (ACM, 2013).
Metaxa, D., Park, J. S., Landay, J. A. & Hancock, J. Search media and elections: a longitudinal investigation of political search results. Proceedings of the ACM on Human–Computer Interaction 3, 129 (2019).
Robertson, R. E. et al. Auditing partisan audience bias within Google Search. Proc. ACM on Human–Computer Interaction 2, 148 (2018).
Trielli, D. & Diakopoulos, N. Partisan search behavior and Google results in the 2018 U.S. midterm elections. Inf. Commun. Soc. 25, 145–161 (2020).
Fischer, S., Jaidka, K. & Lelkes, Y. Auditing local news presence on Google News. Nat. Hum. Behav. 4, 1236–1244 (2020).
Kawakami, A., Umarova, K. & Mustafaraj, E. The media coverage of the 2020 US presidential election candidates through the lens of Google’s top stories. In Proc. International AAAI Conference on Web and Social Media 14, 868–877 (AAAI, 2020).
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B. & Lazer, D. Fake news on Twitter during the 2016 U.S. presidential election. Science 363, 374–378 (2019).
Huszár, F. et al. Algorithmic amplification of politics on Twitter. Proc. Natl Acad. Sci. 119, e2025334119 (2022).
Lazer, D. M. J. et al. The science of fake news. Science 359, 1094–1096 (2018).
Watts, D. J., Rothschild, D. M. & Mobius, M. Measuring the news and its impact on democracy. Proc. Natl Acad. Sci. 118, e1912443118 (2021).
Gillespie, T. The politics of ‘platforms’. New Media & Society 12, 347–364 (2010).
Taber, C. S. & Lodge, M. Motivated skepticism in the evaluation of political beliefs. Am. J. Political Sci. 50, 755–769 (2006).
Iyengar, S. & Hahn, K. S. Red media, blue media: evidence of ideological selectivity in media use. J. Commun. 59, 19–39 (2009).
Introna, L. D. & Nissenbaum, H. Shaping the web: why the politics of search engines matters. The Information Society 16, 169–185 (2000).
Lawrence, S. & Giles, C. L. Accessibility of information on the web. Nature 400, 107–107 (1999).
Metaxas, P. T. & DeStefano, J. Web spam, propaganda and trust. In Proc. 2005 World Wide Web Conference (2005).
Vaidhyanathan, S. The Googlization of Everything (And Why We Should Worry) (Univ. of California Press, 2011).
Allcott, H. & Gentzkow, M. Social media and fake news in the 2016 election. J. Econ. Perspect. 31, 211–236 (2017).
Newman, N., Fletcher, R., Kalogeropoulos, A. & Neilsen, R. K. Digital News Report 2019 (Reuters Institute for the Study of Journalism, 2019).
Mitchell, A., Gottfried, J., Shearer, E. & Lu, K. How Americans Encounter, Recall and Act Upon Digital News (Pew Research Center, 2017).
Edelman. Edelman Trust Barometer 2021 (2021).
Golebiewski, M. & Boyd, D. Data Voids: Where Missing Data can Easily be Exploited (Data & Society, 2019).
Tripodi, F. Searching for Alternative Facts: Analyzing Scriptural Inference in Conservative News Practices (Data & Society, 2018).
Epstein, R. & Robertson, R. E. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proc. Natl Acad. Sci. 112, E4512–E4521 (2015).
Epstein, R., Robertson, R. E., Lazer, D. & Wilson, C. Suppressing the search engine manipulation effect (SEME). Proc. ACM on Human–Computer Interaction 1, 42 (2017).
Wojcieszak, M., Menchen-Trevino, E., Goncalves, J. F. F. & Weeks, B. Avenues to news and diverse news exposure online: comparing direct navigation, social media, news aggregators, search queries, and article hyperlinks. Int. J. Press/Politics 27, 860–886 (2022); https://doi.org/10.1177/19401612211009160.
Peterson, E., Goel, S. & Iyengar, S. Partisan selective exposure in online news consumption: evidence from the 2016 presidential campaign. Political Sci. Res. Methods 9, 242–258 (2021).
Guess, A. M., Nagler, J. & Tucker, J. Less than you think: prevalence and predictors of fake news dissemination on Facebook. Sci. Adv. 5, eaau4586 (2019).
Yin, L. Local news dataset. Zenodo https://doi.org/10.5281/ZENODO.1345145 (2018).
Rating Process and Criteria. NewsGuard https://www.newsguardtech.com/ratings/rating-process-criteria/ (2021).
Mustafaraj, E., Lurie, E. & Devine, C. The case for voter-centered audits of search engines during political elections. In Proc. 2020 Conference on Fairness, Accountability, and Transparency 559–569 (ACM, 2020).
van Hoof, M., Meppelink, C. S., Moeller, J. & Trilling, D. Searching differently? How political attitudes impact search queries about political issues. New Media & Society https://doi.org/10.1177/14614448221104405 (2022).
Zuckerman, E. Why study media ecosystems? Inf. Commun. Soc. 24, 1495–1513 (2021).
Hobbs, W. R. Text scaling for open-ended survey responses and social media posts. SSRN https://doi.org/10.2139/ssrn.3044864 (2019).
Flaxman, S., Goel, S. & Rao, J. M. Filter bubbles, echo chambers, and online news consumption. Public Opin. Q. 80, 298–320 (2016).
Klar, S. & Krupnikov, Y. Independent Politics: How American Disdain for Parties Leads to Political Inaction (Cambridge Univ. Press, 2016).
Noble, S. U. Algorithms of Oppression: How Search Engines Reinforce Racism (New York Univ. Press, 2018).
Guess, A. M., Barberá, P., Munzert, S. & Yang, J. The consequences of online partisan media. Proc. Natl Acad. Sci. 118, e2013464118 (2021).
Kobayashi, T., Taka, F. & Suzuki, T. Can ‘Googling’ correct misbelief? Cognitive and affective consequences of online search. PLoS ONE 16, e0256575 (2021).
Gillespie, T. Algorithmically recognizable: Santorum’s Google problem, and Google’s Santorum problem. Inf. Commun. Soc. 20, 63–80 (2017).
Diakopoulos, N. Algorithmic accountability: journalistic investigation of computational power structures. Digit. Journal. 3, 398–415 (2015).
Parry, D. A. et al. A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nat. Hum. Behav. 5, 1535–1547 (2021).
Walker, M. Americans favor mobile devices over desktops and laptops for getting news. Pew Research Center https://www.pewresearch.org/short-reads/2019/11/19/americans-favor-mobile-devices-over-desktops-and-laptops-for-getting-news/ (2019).
Trielli, D. & Diakopoulos, N. Search as news curator: the role of Google in shaping attention to news information. In Proc. 2019 CHI Conference on Human Factors in Computing Systems https://doi.org/10.1145/3290605.3300683 (ACM Press, 2019).
Nanz, A. & Matthes, J. Democratic consequences of incidental exposure to political information: a meta-analysis. J. Commun. 72, 345–373 (2022).
Bechmann, A. & Nielbo, K. L. Are we exposed to the same ‘news’ in the news feed? An empirical analysis of filter bubbles as information similarity for Danish Facebook users. Digit. Journal. 6, 990–1002 (2018).
Guess, A. M. (Almost) everything in moderation: new evidence on Americans’ online media diets. Am. J. Political Sci. 65, 1007–1022 (2021).
Reeves, B. et al. Screenomics: a framework to capture and analyze personal life experiences and the ways that technology shapes them. Hum. Comput. Interact. 36, 150–201 (2021).
Stier, S., Mangold, F., Scharkow, M. & Breuer, J. Post post-broadcast democracy? News exposure in the age of online intermediaries. Am. Political Sci. Rev. 116, 768–774 (2021).
Pan, B. et al. In Google we trust: users’ decisions on rank, position, and relevance. J. Comput.-Mediat. Commun. 12, 801–823 (2007).
Kulshrestha, J. et al. Search bias quantification: investigating political bias in social media and web search. Inf. Retr. J. 22, 188–227 (2019).
Joachims, T. et al. Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search. ACM Trans. Inf. Syst. 25, 7–es (2007).
Ebbinghaus, H. Memory: A Contribution to Experimental Psychology https://doi.org/10.1037/10011-000 (Teachers College Press, 1913).
Fishkin, R. New jumpshot 2018 data: where searches happen on the web (Google, Amazon, Facebook, & beyond). SparkToro https://sparktoro.com/blog/new-jumpshot-2018-data-where-searches-happen-on-the-web-google-amazon-facebook-beyond/ (2018).
Desktop search engine market share United States of America. StatCounter Global Stats https://gs.statcounter.com/search-engine-market-share/desktop/united-states-of-america/2020 (2020).
Brown, N. E. Political participation of women of color: an intersectional analysis. J. Women Polit. Policy 35, 315–348 (2014).
Hyde, J. S., Bigler, R. S., Joel, D., Tate, C. C. & van Anders, S. M. The future of sex and gender in psychology: five challenges to the gender binary. Am. Psychol. 74, 171–193 (2019).
Early versions of this work were presented at the 2019 International Conference on Computational Social Science (IC2S2), the 2019 Conference on Politics and Computational Social Science (PaCCS) and the 2020 annual meeting of the American Political Science Association (APSA). We are grateful to the New York University Social Media and Political Participation (SMaPP) lab, the Stanford Internet Observatory and the Stanford Social Media Lab for feedback, and to Muhammad Ahmad Bashir for development on the 2018 extension. This research was supported in part by the Democracy Fund, the William and Flora Hewlett Foundation and the National Science Foundation (IIS-1910064).
The authors declare no competing interests.
Peer review information
Nature thanks Homa Hosseinmardi and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
Extended Data Fig. 1 Strong partisans are exposed to similar rates of partisan and unreliable news, but asymmetrically follow and engage with such news.
This figure complements Fig. 1 in the main text by displaying, for all 7-point PID groups, average exposure, follows and overall engagement with partisan (a, c) and unreliable news (b, d) by study wave and 7-point PID clustered at the participant-level. Data are presented as participant-level means grouped by 7-point PID in each subplot, all error bars indicate 95% confidence intervals (CI), and results from bivariate tests of differences in partisan and unreliable news by 7-point PID are available in Extended Data Table 2. A score of zero does not imply neutrality in the scores we used, so left-of-zero scores do not imply a left-leaning bias (Methods, ‘Partisan News Scores’).
Extended Data Fig. 2 Partisans who engage with more identity-congruent news also tend to engage with more unreliable news.
This figure complements Fig. 3 in the main text by displaying all 7-point PID groups, highlighting the relationship between partisan and unreliable news for participants’ exposure on Google Search (a, d), follows from Google Search (b, e), and overall engagement (c, f). These subplots show that the relationship between partisan and unreliable news varies across data types, and within data types when taking partisan identity into account (Extended Data Table 3).
Extended Data Fig. 3 Partisan news distributions at the participant level for each dataset and study wave.
Each line represents the distribution of partisan news sources that a single participant was exposed to in their Google Search results (a, d), followed from those results (b, e), or engaged with overall (c, f). Partisan news scores have been binned in 0.1 point intervals (e.g. −1 to −0.9, −0.9 to −0.8, etc.) along the x-axis, with tick labels showing the midpoints of those bins.
This file contains text that introduces several Supplementary Information tables and provides our IRB study procedures. The Supplementary Information tables include extensive demographics for each study (Supplementary Information Tables 1 and 2), example news domains and their partisan audience bias scores (Supplementary Information Table 3), comparisons of participants’ popular search engine usage (Supplementary Information Table 4), participant-level averages for each dataset in each study wave (Supplementary Information Table 5), the average proportion of news and unreliable news we found in each dataset and study wave (Supplementary Information Table 6) and detailed results for each of the regressions we ran (Supplementary Information Tables 7–18).
About this article
Cite this article
Robertson, R.E., Green, J., Ruck, D.J. et al. Users choose to engage with more partisan news than they are exposed to on Google Search. Nature 618, 342–348 (2023). https://doi.org/10.1038/s41586-023-06078-5
This article is cited by
Nature Computational Science (2023)