Introduction

Democracies function best when citizens are up-to-date on current affairs and knowledgeable about policies, their representatives, and the political system at large (e.g., Delli Carpini and Keeter, 1996 and so news exposure has long been seen as normatively desirable (de Tocqueville, 2000). Thus, scholars worry about decreasing news use (Putnam, 2000) and observers express concerns about the under-funding of media organizations (Mitchelstein and Boczkowski, 2009, Picard, 2011). Survey respondents also overreport news exposure (Prior, 2009), underscoring its perceived importance among the general public. The belief that news media are democratically beneficial is confirmed by empirical research. By covering the issues of the day, news media use increases knowledge and informs prospective voters (Chaffee et al., 1994) and by providing information about opportunities for political involvement, news exposure stimulates participation (Colwell Quarles, 1979; Lemert, 1984; Norris, 2000; Shah et al., 2005; Strömbäck, 2005; Wolfinger and Rosenstone, 1980). These positive links hold for both offline and digital media and across countries (Beckers et al., 2021; Kenski and Stroud, 2006; Ohme, 2020). And yet, despite these documented benefits, there are reasons to believe that news exposure, in general, may have a range of adverse effects that have been largely overlooked in past theorizing and research, as we note below.

To offer a comprehensive portrayal of individual-level news effects, both desirable and negative, political and nonpolitical, we embed two pre-registered experiments in a larger international project that combines over-time surveys with behaviorally tracked online exposure among non-probability but representative on key census demographics samples in the United States (US) and Poland (see Supplementary Information SI E.1 and E.2 for descriptions of the samples). The first experiment examines the effects of taking a 7-day ‘news vacation’ (N = 803) in the US, where we incentivized participants to not consume any news (i.e., The second experiment tests the effects of ‘news binging’ in Poland (N = 939), where we incentivized participants to increase their news consumption for 14 days. Because the participants were encouraged to change their news consumption patterns in general, not only online, we assess compliance using self-reports and also participants’ online browsing data collected via the open-source tool Web Historian, as detailed below and in SI A.1). Additionally, we consider whether the tested effects are contingent on how much news subjects typically consume and on the extent to which their typical news diet is like-minded, as determined by a combination of pre-wave survey and their actual web browsing behavior. From over 88 million visits, we analyze 10 million visits from the month before the experiments and use these data in conjunction with a comprehensive list of news websites and machine learning algorithms to construct the measures of (ideologically like-minded) prior news consumption (see SI B.2).

This project advances prior work both theoretically and methodologically. Contributions of news use to political engagement notwithstanding, we aim to challenge the normative notion that news exposure is inherently beneficial. We propose that news content, whether centrist or partisan, congenial or not, has two defining—potentially harmful—traits: it focuses on conflict and often activates citizens’ political identity. Hence, news exposure may have various political and nonpolitical outcomes that have received little attention in scholarly literature, such as exacerbating polarization, worsening individual perception of the political system, as well as negatively affecting personal well-being and health. After all, negativity is one of the core journalistic values (Galtung and Ruge, 1965) and so news media tend to focus on clashes between political groups, and feature uncivil debates (Levendusky, 2013; Martin and Yurukoglu, 2017; Mutz, 2006), and cover politics as a game or a horse-race (Cappella and Jamieson, 1996). In addition, the fact that journalists turn to social media posts to represent public opinion in the news gives a disproportionate voice to social media users, who are more extreme and polarized than the average citizen (McGregor, 2019). These tendencies—woven into journalistic routines—may lead people to see the system at large as failing, the elites as evil, and society as sharply divided. Such negativity and conflict may also make individuals anxious, worried, or angry. These tendencies—woven into journalistic routines—may lead people to see the system at large as failing, the elites as evil, and society as sharply divided. Such negativity and conflict may also make individuals anxious, worried, or angry.

Furthermore, news use in general (regardless of whether it is partisan or not) can reinforce people’s attitudes and affective polarization. Theories on public opinion formation emphasize that exposure to elite cues can distort citizens’ political preferences, such that party attachments outweigh substantive information (Mullinix, 2016; Nicholson, 2012). Because political parties have become more polarized and convey clear information about their policy stances, exposure to elite cues and party communication should further polarize the views of the electorate (Druckman et al., 2013; Levendusky, 2013) and intensify out-group hostility (Finkel et al., 2020; Levendusky and Malhotra, 2016). Because few citizens are exposed to these cues directly, it is the news media that convey this information to ordinary voters (Zaller, 1992). Lastly, media coverage of (polarized) politics highlights the existence of inter-group conflicts, activates viewers’ partisan identity, encourages partisans to perceive politics through the us-versus-them lens, and ultimately exacerbates people’s dislike towards the political out-group (Levendusky and Malhotra, 2016).

Methodologically, realistic estimates of positive and negative media effects in naturalistic settings are missing. Most past work focuses on the contributions of news use, testing knowledge, efficacy, engagement, and/or informed voting as the core outcomes of interest. When negative outcomes are studied, scholars almost exclusively examine partisan news in the US (Arceneaux et al., 2013; Levendusky, 2013). Needless to say, this focus is rather limited given that partisan outlets attract a small fraction of the population (Prior, 2013; Wojcieszak et al., 2021) and the US is far from representative of other countries, media, and political systems globally. In addition, past work on the benefits of news use tends to rely on largely unreliable survey self-reports Prior, 2009). Experimental designs, in turn, often ’‘force” people to watch very specific—typically partisan—content (Arceneaux et al., 2013) or allow them to select from a limited content pool (Arceneaux et al., 2013; de Benedictis-Kessner et al., 2019; Stroud et al., 2019). These designs cannot approximate actual media consumption contexts, where people can use a nearly unlimited number of sources, and are likely to overestimate media effects on the tested outcomes (Gerber et al., 2011). The more externally valid experiments using encouragement designs zoom on the use of explicitly partisan news online (Casas et al., 2022; Guess, 2021), offline (Broockman and Kalla, 2022), or on social media (Levy, 2021) (see also Allcott et al. (2020) for Facebook deactivation design).

Our work examines news exposure in general, encouraging shifts in overall news consumption, and maximizes ecological validity by embedding the treatments in participants’ real life rather than in a controlled and isolated context. This approach comes with imperfect control over treatment, so we test compliance and estimate the Intention to Treat (ITT) and also the Complier Average Treatment Effect (CATE) relying on self-reported news media use (to capture offline and multi-platform news use) and on online behavioral data to assess actual exposure to news domains during the experiment. Also, because (news) media effects depend on a multitude of boundary conditions (Slater, 2015; Valkenburg and Peter, 2013), we use survey and online behavioral data to examine heterogeneity in treatment effects by prior levels of news consumption and also the ideological congeniality thereof.

We find that neither taking a week-long news vacation nor increasing news consumption for two weeks influenced the tested outcomes, beneficial (e.g., political engagement) or not (e.g., polarization, attribution of malevolence to out-party). With a few isolated exceptions, these null effects emerged regardless of one’s compliance with the treatment, prior levels of news exposure, and the extent to which one’s news diet was ideologically congenial, and held in additional Bayesian analyses. Because our designs had the sufficient statistical power to detect effects and these effects emerged in two different countries, we see them as accurate representations of reality. News media have a central role in society. Yet, our evidence suggests that their individual-level contributions may be more limited than generally believed, at least during a 1- or 2-week-long treatment. In fact, in participants’ browsing data news domains comprised only 3% of the overall online visits.

Research design

Overall project

Figure 1 provides an overview of the project and our pre-registered design. The pre-registrations are available at https://osf.io/6rphw/ and https://osf.io/6rphw/. Both experiments were embedded in a larger international three-wave panel study, in which, every 3 months, the same participants completed 20-min surveys and submitted their browsing data. The project was approved by the Ethical Board of the European Research Council (ERC) and the University at Amsterdam (for details see SI A.2).

Fig. 1: Overview of the research design.
figure 1

Information on sample sizes per wave, exposure statistics, experimental assignment, and compliance with treatment.

Full size image

Sampling and recruitment in the US were done by Lucid, an aggregator of survey respondents from many sources that collect demographic information on the panelists, facilitating quota sampling to match the US Census margins. In Poland, the sample was drawn from Panel Ariadna’s database, which consists of 286,000 adults recruited through multiple strategies (e.g., telephone, face-to-face, and online). Membership in the panel is by invitation only to ensure panel quality and representativeness and awards are sent to panelists by courier to assure that there are no bots in the panel and that an individual does not register multiple times. Quotas on age, gender, and education were enforced (and on ethnicity in the US). SI E.1 and E.2 describe the demographic characteristics of the samples.

At each wave, participants submitted their online browsing data via our open-source tool that allows for transparent data sharing, Web Historian (Menchen-Trevino, 2016). Web Historian is a web browser plug-in that accesses respondents’ browser history stored on their computers, displays it to them using visualizations (e.g. network graph of websites visited, word cloud of used search terms), and allows them to submit it to researchers following an extensive informed consent shown in SI A.1, which also shows the tool, data visualizations, and the steps taken by the participants. We use those online behavioral data to measure compliance and prior (congenial) news exposure, as detailed below.

Experimental design

The ‘news vacation’ experiment was embedded in Wave 3 of the US panel survey. The 872 respondents who completed Wave 3 were invited to take part in the experiment and 803 agreed to participate (92%). They were assigned to an experimental or control condition via simple randomization (probability of assignment to treatment: 60%). The treatment participants (N = 457) were incentivized to stop following news for one week. They were told they are expected to actively avoid the news on all devices and platforms (TV, radio, podcasts, news apps, newspapers, magazines, phone, tablet, laptop, etc.) and to minimize their conversations about current events to not receive news get at them via this two-step communication flow second-hand spread of news (Druckman et al., 2013; Katz, 1957) (see SM D.2 for instructions). The control (N = 346) received no instructions. The ‘news binging’ experiment was embedded in Wave 2 of the Polish part of the project. Out of 976 Wave 2 participants, 939 (96%) opted into the experiment (probability of assignment to treatment: 50%). Those in the treatment group (N = 445) were instructed to consume more news than usual for two weeks, with examples of reading newspapers, visiting news websites, watching TV news channels or newscasts, and listening to radio news more often; see SM D.2 for instructions). The control (N = 494) received no instructions. Given the challenges of defining news—as well as other media genres—in the current hybrid media environment (Edgerly and Vraga, 2020), we did not give the participants a specific definition and instead relied on their understanding and the examples provided, as we also note below.

Both groups were asked to complete the post-test (after one week in the US: treatment N = 378; control N = 288; after 2 weeks in Poland: treatment N = 421, control N = 402). Power analyses show that the sample sizes suffice to observe small effects (see SI Fig. F.1). We do not observe any concerning attrition bias when comparing the samples that completed the preceding waves, those who opted into the experiments, and those who completed the post-test (SI E.3 and E.4). In addition, SI E shows that randomization did not create any imbalances: the control and the treated groups do not differ significantly from each other, both when opting in and at the post-test.

Compliance

Because participants were incentivized to increase/decrease their news intake in general, across both studies, we recorded compliance using both self-reported and behavioral data, as pre-registered. The post-survey asked experimental participants whether they consumed less or more news than usual and how much they consumed news via a number of channels, platforms, and devices (11 items, self-reported measure). These measures were combined into an index from 0 to 1 indicating how much a subject reported changing consumption in the direction of the treatment. As the visualizations of compliance in Fig. 1 show, in the US, the no-news treatment group reported much lower news consumption than the control (0.099 vs. 0.378), a difference that did not emerge in Poland (treatment 0.489 vs. control 0.494).

We also rely on participants’ browsing data to measure compliance. Here, we operationalize news as visits to domains categorized as news (pre-registered) and also as visits to political content (non-pre-registered). Because Web Historian records data at the visit level, which allows us to calculate how often participants visited news before and during the experiments. We match the visited domain (e.g., nytimes.com) to identifiable news domains as determined by our comprehensive open-source lists of news sites per country. The U.S. list contains a total of 5400 local, national, and international news organizations (of which 911 were visited in the data used in this paper); the Polish list 298 organizations (208 actually visited). SI B.1 shows how the lists were created. The lists are made publicly available on Github for the US and Poland. We additionally account for the fact that users can access news on social media platforms. While we do not have access to the content of one’s Facebook News Feed or Twitter timeline, we can see if participants visited the Facebook page of a news organization, a tweet from the Twitter handle of a news organization, or a video from a YouTube channel of a news organization. We thus identified the YouTube channels, Facebook pages, and Twitter handles of all the news media organizations on our lists and assigned these exposures accordingly. By this behavioral measurement, the behavioral compliance measure compares the average visits to the news domains per day in online trace data before and during the experiment. The US subjects in the no-news treatment had fewer daily average visits (3.86) than the control group (5.22), and the Polish subjects in the more news treatment had a higher daily average than the control (5.54 vs. 4.61).

Furthermore, because participants’ understanding of online ”news” exposure may not map perfectly with our domain-level approach, we explore two alternative online compliance measures that capture the content visited during the experiments. We measure whether subjects consumed political content within the news domains on our list (i.e., hard news) or instead visited the news domains to read about sports or weather, and also whether they consumed political content in general, whether within or outside news domains (e.g., reading about political issues on websites, not on our lists). For this purpose, we apply our multilingual BERT-based natural language model, trained on website titles, that allows us to categorize the visits as related to politics or not with high accuracy (93%, Precision 0.92, Recall 0.91, F1 0.915; see SM C for details and performance metrics). By these exploratory behavioral measures of compliance, US subjects in the no-news treatment had fewer daily average visits to political news and any political content than the control group (political news: 2.20 in treatment, 1.50 in control; any political: 5.50 in treatment, 7.03 in control). In Poland, subjects in the more-news treatment had more visits to political (news) content (political news: 1.66 in treatment, 1.26 in control; any political: 3.79 in treatment, 3.48 in control).

Given the imperfect compliance for the self-reported measures and the rather moderate shifts in the behavioral measures, we provide both the Intention to Treat (ITT) and also the Complier Average Treatment Effect (CATE) estimates to test whether the effects were especially pronounced among those who did comply with the treatment using both the self-report and behavioral data. For this purpose, we construct two main compliance measures (self-reported and behavioral) that indicate an individual’s change of news consumption from before to during the experiment. We also run CATE models with two alternative behavioral compliance indicators, namely, change in political news exposure and change in exposure to any political content.

To further determine if a subject’s prior news diet was ideologically congenial, we match 755 of the visited US news domains with a list of validated ideology scores (Robertson et al., 2018) and also develop an open-source ideology categorization for 132 of the domains in Poland (see SI B.2 for details).

Results

News exposure in perspective

Do our respondents consume news, as measured prior to the experiment? Regarding self-reported exposure, subjects in the US say they consume news on average 3.25 days per week and the Polish subjects report 2.9 days of news exposure per week (averaging across a range of items, e.g., TV, newspapers, radio, see SI D.1). Regarding the trace data, there were roughly 4.7 million visits during the month prior to the experiment in the US, and about 3.4 million for the same time window in Poland. Only 3.10% of these visits across both countries were to news domains (US 2.72%, Poland 3.72%). That is, the average participant encountered only one news domain for every 33 sites they visited. We return to this important descriptive finding in the discussion.

News exposure effects

We examine a range of outcomes, each measured using multiple indicators: political knowledge (both self-reported and actual, assessed with questions about current events), political participation, support for compromise, attitude and affective polarization, attribution of malevolence to the out-party, perceived polarization, and general well-being (both psychological and physical). SI Table D.1 lists all items used in this study, as well as key statistics and reliability measures.

Figure 2 shows the results for the ‘No News’ experiment in yellow and for the ‘More News’ experiment in gray. Each panel represents an outcome. In all the models, we control for the pre-wave outcome measure to increase precision. Note that for the mental and physical well-being outcomes, as well as the actual knowledge outcome in the US, we do not have a pre-wave measure. Our first model tests the intention-to-treat (ITT) effect by regressing the outcome of the treatment. The other models test whether the effects differ according to self-reported and behavioral compliance—focusing on visits to news domains in the main paper, as pre-registered, but exploring the two alternative compliance measures—and for different levels and types of prior news exposure, also assessed via surveys and online trace data. We apply FDR adjustment to the entire set of models and show coefficients that do not reach 95% significance in opaque. These regression results can also be found on the Harvard Dataverse.

Fig. 2: Results: Estimates from OLS regression models.
figure 2

Dots represent coefficient estimates from OLS regressions (see legend for different model specifications). Horizontal bars represent 95% confidence intervals around the point estimates. Insignificant coefficients (after FDR adjustment) are shown in opaque. The dependent variables were rescaled between 0 and 100 so that the coefficients denote the percentage point change in the dependent variable as the result of one unit increase in the independent variable.

Full size image

Examining the ITT models, we first address the beneficial outcomes: political knowledge, participation, and support for political compromise. Unlike hypothesized, participants who consumed more news were not any more knowledgeable (Facet b)—or felt they were (Facet a)—than the control. In addition, those in the No News condition were not any less knowledgeable than those in the control, nor did they feel as such. Similar null effects from ‘news vacation’ and ‘news binging’ emerge for participants’ engagement in a range of civic and political activities, from signing a petition to protesting (Facet c) and also for one’s support for politicians crossing the aisle and reaching compromise (Support for compromise, (Facet d).

We turn to the negative outcomes, testing if news exposure increases attitude polarization (i.e., attitude importance and strength on five salient issues per country) and affective polarization (i.e., hostility toward out-ideologues, out-partisans, and citizens with opposite policy beliefs, each measured in three ways) (see SI D.1). Using multiple measures ensures that the detected patterns are not due to any specific measurement or out-groups alone. The treatment—whether decreasing or increasing news use—had no significant effects on attitude (Facets f for importance and e for strength) and affective (Facets g–i) polarization, with the exception of an unexpected reduction of affective polarization (measured with a feeling thermometer toward the out-party) in the more news condition in Poland. However, this small effect is not robust against alternative measures of affective polarization and other out-groups tested and did not emerge in the US where we pre-registered that it was the no news condition that should reduce affective polarization. We also note that the effects do not surpass the size of 2 percentage points independently of which indicator and which political out-group we examine.

Adding to this pattern, news exposure had near-zero effects on whether people think the out-party wants to harm the country (Attribution of malevolence, Facet j). We do find a small significant effect on perceived polarization: Decreasing news consumption decreases the perception that society is divided, Facet k). Even though this pre-registered outcome is consistent with our expectations, given the media’s focus on a horse-race and in-your-face debates (Levendusky and Malhotra, 2016; Martin and Yurukoglu, 2017; Mutz, 2006), the opposite effect did not ensue from consuming more news.

Lastly, we predicted that news exposure would reduce individual well-being. Studies find links between news consumption and stress, anxiety, fatigue, or sleep loss (Heid, 2020), especially when news is personally relevant (de Hoog and Verboon, 2020). One over-time survey also shows the negative effects of hard news exposure on one’s mental well-being (Boukes and Vliegenthart, 2017). These emotional responses may trigger unhealthy behaviors aimed to alleviate the stress. Yet, our tests find no significant new effects on well-being, psychological (e.g., anxiety, anger) or physical (e.g., consuming alcohol, desire to hit someone), during the treatment period (Facetsland m).

As mentioned, in maximizing ecological validity by embedding treatments in a larger project and testing new effects in naturalistic settings, we lose some control over treatment. To account for the extent to which participants complied with the treatment, we rely on the two aforementioned compliance measures, self-reported and behavioral. We use these measures to estimate complier average treatment effects (CATE) with an instrumental variable regression. Footnote 1 As Fig. 2 shows, the estimates are nearly identical to those already presented. We do find that the effect of no news on perceived polarization holds for self-reported compliance, but not for behavioral compliance. We further estimated CATE models with two alternative compliance measures, namely whether respondents changed, first, their consumption of hard news, as classified by a natural language model, and second, their consumption of any political content, whether in or outside of news domains. These analyses yield substantively identical results, as shown in SI H).

Heterogeneous treatment effects

We additionally test whether our treatments have different effects depending on one’s prior new diet. For instance, some participants in the No News experiment may consume no news in general, and thus be unable to reduce their intake. Also, heavy news consumers in the More News experiment may already have reached a saturation point before the experiment. Likewise, those who mostly consume like-minded partisan news may experience greater effects (e.g., on reduced polarization) than those with ideologically diverse news diets.

Four models tested heterogeneous treatment effects for different levels of prior news exposure (Models 3a, 3b) and prior congenial news exposure (Models 4a, 4b) by interacting our treatment with these moderators. Models a use a self-reported pre-treatment measure of how often participants consume news via nine different channels (e.g., TV, newspapers, etc.) and a self-reported measure of news diet ideology. Models b rely on behavioral measures. We averaged the number of visits to news websites per day for the month prior to the treatment.

To create a person-level average of behavioral news diet ideology, we used validated machine learning models to classify the ideology of news domains (see SI B.2 for detailed information on the ideology scores). Most generally, we observe that news consumption among our samples was concentrated on centrist domains. In the US, where the most extreme domains have scores of −0.91 (left) and 0.91 (right), respectively, 90% of participants have an average ideological exposure between −0.4 and 0.22. In Poland, where the spectrum of domain ideology reaches from −1.82 (left) to 1.90 (right), 90% of subjects have an average ideology of their news diet between −0.35 and 0.2. We do not observe different effects for heavy or light news consumers, nor for those whose media diet is primarily like-minded. The one exception is a small significant interaction for political knowledge (Facet ii): The effect of consuming more news decreases the more congenial one’s prior news diet.

Lastly, we provide Bayesian analyses to assure that the null effects detected are robust. Bayes factors quantify the evidence in favor of the null hypothesis relative to the alternative. Using the BayesFactor package in R and its default priors, for each model shown in Fig. 2, we compare the model that includes the parameter of interest with a baseline model without this parameter (BF01). The results are reported in SI Table G.1. For all the models, about 57% of all BF01 values are >10 (i.e., strong evidence for the null) and 90 percent are >3 (i.e., moderate evidence for the null, cf. Wetzels et al., 2014). The only two main effects (ITT) models with a BF01 <3 are the significant effects shown in Fig. 2. Although not pre-registered, we additionally explored heterogeneity along ideology and education in SI I, finding largely null effects. These supplementary analyses provide further evidence that reducing or increasing news consumption has very limited effects on the tested outcomes, with few minor and weak exceptions.

Discussion

Most scholars agree that news exposure is normatively desirable. In this project, we aimed to provide a new perspective on the role of news media in society. We argued that tuning in to news can generate a wide range of adverse outcomes, polarizing attitudes, exacerbating out-party hostility, worsening perceptions of the political system, or making people more anxious or angry. We tested these potential pitfalls in concert with three beneficial outcomes, i.e., political knowledge, political participation, and support for inter-party compromise.

Across two experimental designs combining participants’ survey and behavioral browsing data in two distinct countries, prolonged decreases or increases in news consumption had no effects on the positive or negative individual-level outcomes. Two exceptions to this null pattern emerged: increasing news intake made the Polish participants feel warmer toward the out-party and decreasing news use led the American participants to see the system as less polarized. Because these effects are not very robust, we caution against putting too much weight on these results. These largely null patterns did not depend on whether people more clearly complied with the treatments, assessed using self-reported as well as behavioral measures based on online traces, and also accounting for whether participants visited hard news and/or saw political content outside news during the treatment. Similarly, although we used both self-reported and behavioral indicators of prior levels of news consumption and its ideological congeniality, news effects did not depend on an individual’s typical news diet. That is, the decrease in news use was not less impactful for avid news consumers or the increase in news use did not affect those rarely exposed to the news. The one exception—those whose prior news diet was ideologically congenial became less knowledgeable about current events when consuming more news – is small in magnitude. Testing our hypotheses in two distinct contexts assures that the results are not due to idiosyncrasies of any particular media or party system alone.

Although we offer a comprehensive examination of various individual-level effects of news exposure, these null effects are not precise estimates of population average treatment effects because our samples are not perfect cross-sections of the populations. This limitation is common to most work relying on data from online samples willing to share their behavioral traces, in that no such work can claim representativeness. More importantly, we note a few considerations regarding compliance. Our participants complied with the experimental treatments, apart from the subjects in Poland who did not self-report greater news consumption (perhaps due to the aforementioned biases in self-reports). These shifts, however, were small in magnitude, as indicated in the variations in behaviorally tracked exposures to online news and in the US subjects’ self-reports of overall news diet on the post-survey. There are no established theoretical and empirical benchmarks for determining when exactly news exposure should influence individual attitudes, cognitions, or behaviors, and how large the shifts in exposure need to be to notice these effects. Research that systematically varies the amounts of news in people’s media diets is needed to identify such minimal thresholds. In our project, the detected increases or decreases were likely insufficient to generate noticeable impacts on the tested outcomes.

In addition, the participants were instructed to increase or decrease their news consumption overall, not only online. While our trace data can ascertain desktop visits to (congenial) news websites for behavioral compliance and prior exposure measures, parallel behavioral indicators of these exposures on mobile and offline are missing. For most people, television remains the dominant news source (Allen et al., 2020) and we do not have behavioral data on these exposures occurring offline. Again, the self-reported measures that did ask about news use across devices and modalities (e.g., television, radio, mobile, social media, and so forth) have known limitations, and so the totality of changes in news exposures cannot be reliably determined. If the participants did not comply with our treatments on sources from which we could not collect behavioral data and did not accurately report compliance, the effects of the detected changes in online news consumption may have been minimized, leading to the null effects observed.

In a similar vein, although we do account for visits to social media pages of news organizations on Facebook, Twitter, and YouTube, we cannot get at news people encounter elsewhere on social media (e.g., friends’ posts, headlines, or embedded news videos). Instead, we capture a more direct engagement with news (i.e., landing on the URL of a news domain or a social media news page). It is thus possible that No News study participants nevertheless encountered news inadvertently when going to social media for other purposes. These encounters may have counteracted the effects of the US participants avoiding news in other contexts. At the same time, we note that most people do not come across news and public affairs information on social media platforms. Online behavioral data suggest that only 4% of News Feed on Facebook are news (Zuckerberg, 2018) and public affairs comprise 1.8% of the average News Feed of college students (Wells and Thorson, 2017, see also Karnowski et al., 2017; Vermeer et al., 2020). In our data, although social media browsing from Twitter, Facebook, and YouTube made up 7.7% of all visits during the overall study in the US and Poland, only 0.4% of those visits were to the identified news organizations, and these made up only 1.2% of all news visits overall. It is thus unlikely that social media exposure to the news would bias our effects in any meaningful way. That said, exposure to political memes or friends’ posts about current events also carries political information that can have effects above and beyond any news exposure, as we also note below. In order to test the prevalence and the effects of such encounters with politics, scholars should dedicate (and ideally pull) resources to developing tracking and data donation tools that work across devices and platforms.

Furthermore, (news) media effects depend on a host of factors, and various personal, content- or medium-level characteristics likely moderate the effects. Our pre-registered models accounted for prior levels of news exposure and our exploratory analyses additionally tested education and ideology as moderators, yet other socio-demographics (Yang and Grabe, 2011), the medium itself (e.g., newspapers, television, or internet; Althaus and Tewksbury, 2002), different digital outlets (e.g., online news sites, blogs, or online video sites, Dimitrova et al., 2014), and/or the mode of consumption (e.g., on apps or mobile Ohme, 2020) may also matter to whether, when, and for whom (news) media exposure has effects. Instead, our approach incentivized shifts in news consumption across devices, outlets, and modalities, potentially obscuring some nuances and contingencies.

These considerations aside, our results challenge the popular narrative that news media contribute to a healthy citizenry. These results also counter our expectations that news use should have a range of adverse effects. We speculate about three reasons for these patterns. In the current polarized climate in many countries, when citizens’ political identities are constantly activated (Settle, 2018) and when numerous ostensibly non-political issues and events become associated with politics (DellaPosta, 2020), it may be increasingly difficult to shift individual opinions and beliefs. Feelings toward out-groups, political elites, and the system at large may be too deeply ingrained in citizens’ overarching social and political identities (Mason, 2018) to be noticeably affected by (again minor) increases or decreases in one’s news consumption.

Second, despite the long-standing theoretical centrality of news, sizable proportions of the American and international public see news as complex or boring, are averse to partisan politics (Klar et al., 2018), and avoid news (Newman, 2019). As such, news accounts for only a small part of citizens’ overall information and communication ecology and is overshadowed by sports, entertainment, socializing, among other content categories that are not related to politics. Online, only between 2% (Wojcieszak et al., 2021) and 7–9% (Guess et al., 2021) of all URLs visited by large samples of Americans are news domains (Stier et al., 2022), and news comprised around 14% of total daily media diets when additionally accounting for mobile and television (Allen et al., 2020). In our trace data, visits to news sites comprised 3.01% of the overall browsing. Given that citizens’ time and attention are not consumed by current affairs and their attitudes, cognitions, and behaviors are also shaped by other factors (e.g., family, community context), whatever shifts in the very low baselines would have to be massive in strength or duration and/or small increases in news use would have to have a massive influence on the tested outcomes.

Third, today’s hybrid media environment may require a reconsideration of what is news, how to define and measure it, and how to identify the sources, contents, or textual and visual messages that can be considered news (or at least fulfill the democratic role of news). Our project did not define news consumption for the participants and—when measuring it behaviorally—narrowly focused on domain-level conceptualization (e.g., visiting cnn.com or foxnews.com). However, news sites feature not only hard news but also non-political content, so users may indeed visit “news" domains but only to read about sports, weather, or food recipes, not about politics. Others, in contrast, may visit political websites, not on our list (e.g., blogs) and/or go to ostensibly non-political outlets to read about politics (e.g., an article about abortion in Women’s Health), and learn about public affairs from such sources and contents. In our exploratory analyses relying on the classification of titles as related to politics, we accounted for the fact that citizens may conceptualize news in different ways and see each of the above scenarios as ”news” exposure. Yet even after accounting for these political contents within and outside news domains, the null effects remained unchanged. Nevertheless, myriad other sources and media messages may be seen as news or having news value by audiences (e.g., a celebrity tweeting about the U.S. Supreme Court overturning Roe vs. Wade abortion; see Edgerly, 2017; Edgerly and Vraga, 2019 for key evidence). To the extent that these distinct outlooks on what is news shape what sources audiences use, how they process information, and what they learn, scholars may need to expand their understanding and definitions when theorizing and studying news use and its democratic effects, positive and adverse.

Naturally, news media are important. They keep other powers in check by investigating and publicizing the truth and bind citizens together around shared events, values, or concerns (Dayan and Katz, 1992; Delli Carpini and Keeter, 1996; de Tocqueville, 2000), a function that is proving increasingly difficult in the fragmented media environment. In fact, much democratic theorizing concerns these macro-level effects of news media on society and democracy at large, effects that are challenging to study using social scientific methods. In addition, news media may play a paramount role in the gradual development of attitudes and participatory habits during political socialization (Moeller and de Vreese, 2019) and have a cumulative influence on people’s perceptions of (political) reality over the years (Gerbner, 1998), subtle effects that can only be tested with longitudinal designs that collect data over much longer periods. Nevertheless, this project, the first to rely on incentivized over-time designs in naturalistic settings and using both self-reported and online behavioral indicators of general news exposure across two countries, suggests that direct individual-level contributions of news media may be more limited than typically hoped or assumed.