Abstract
Despite AI-driven recommendation algorithms being widely adopted to counter information overload, substantial evidence suggests that they are building cocoons of homogeneous contents and viewpoints, further aggravating social polarization and prejudice. Curbing these perils requires a deep insight into the origin of information cocoons. Here we investigate information cocoons in the real world using two large datasets and find that a large number of users are trapped in information cocoons. Further empirical analysis suggests that two ingredients, each corresponding to a fundamental mechanism in human–AI interaction systems, are correlated with the loss of information diversity. Grounded on the empirical findings, we derive a mechanistic model for the adaptive information dynamics in complex human–AI interaction systems governed by these fundamental mechanisms. It allows us to predict critical transitions between three states: diversification, partial information cocoons, and deep information cocoons. Our work not only empirically traces real-world information cocoons in two representative scenarios, but also theoretically unearths basic mechanisms governing the emergence of information cocoons. We provide a theoretical method for understanding major social issues resulting from adaptive information dynamics in complex human–AI interaction systems.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The news dataset5 is available at https://msnews.github.io/. For commercial reasons, we anonymize the specific name of the video platform. We present the video dataset at https://github.com/tsinghua-fib-lab/Adaptive-Information-Dynamic-Model (refs. 39,40). In the GitHub repository, we provide the behavioural data aggregated to individual granularity and the processed data for Figs. 1–4. Source data are provided with this paper.
Code availability
The code used in this research is available at https://github.com/tsinghua-fib-lab/Adaptive-Information-Dynamic-Model (refs. 39,40).
References
Tagliabue, J. et al. A challenge for rounded evaluation of recommender systems. Nat. Mach. Intell. 5, 181–182 (2023).
Ricci, F., Rokach, L. & Shapira, B. Recommender Systems Handbook (Springer, 2022).
Zhang, S., Yao, L., Sun, A. & Tay, Y. Deep learning based recommender system: a survey and new perspectives. ACM Comput. Surv. 52, 1–38 (2019).
Bakshy, E., Messing, S. & Adamic, L. A. Exposure to ideologically diverse news and opinion on Facebook. Science 348, 1130–1132 (2015).
Wu, F. et al. Mind: a large-scale dataset for news recommendation. In Proc. 58th Annual Meeting of the Association for Computational Linguistics (eds Jurafsky, D. et al.) 3597–3606 (Association for Computational Linguistics, 2020).
Covington, P., Adams, J. & Sargin, E. Deep neural networks for YouTube recommendations. In RecSys '16: 10th ACM Conference on Recommender Systems 191–198 (Association for Computing Machinery, 2016).
Davidson, J. et al. The YouTube video recommendation system. In Proc. Fourth ACM Conference on Recommender Systems, RecSys ’10 293–296 (Association for Computing Machinery, 2010).
Santos, F. P., Lelkes, Y. & Levin, S. A. Link recommendation algorithms and dynamics of polarization in online social networks. Proc. Natl Acad. Sci. USA 118, e2102141118 (2021).
Sunstein, C. R. Infotopia: How Many Minds Produce Knowledge (Oxford University Press, 2006).
Nguyen, T. T., Hui, P.-M., Harper, F. M., Terveen, L. & Konstan, J. A. Exploring the filter bubble: the effect of using recommender systems on content diversity. In WWW '14 Companion: Proc. 23rd International Conference on World Wide Web 677–686 (Association for Computing Machinery, 2014).
Chaney, A. J. B., Stewart, B. M. & Engelhardt, B. E. How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. In Proc. 12th ACM Conference on Recommender Systems, RecSys ’18 224–232 (Association for Computing Machinery, 2018).
Algorithmic recommendations, anyone? Nat. Mach. Intell. 5, 95 (2023).
Liu, J., Huang, S., Aden, N. M., Johnson, N. F. & Song, C. Emergence of polarization in coevolving networks. Phys. Rev. Lett. 130, 037401 (2023).
Baumann, F., Lorenz-Spreen, P., Sokolov, I. M. & Starnini, M. Modeling echo chambers and polarization dynamics in social networks. Phys. Rev. Lett. 124, 048301 (2020).
Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W. & Starnini, M. The echo chamber effect on social media. Proc. Natl Acad. Sci. USA 118, e2023301118 (2021).
Leonard, D. & Sensiper, S. The role of tacit knowledge in group innovation. Calif. Manag. Rev. 40, 112–132 (1998).
Munson, S. A. & Resnick, P. Presenting diverse political opinions: how and how much. In Proc. SIGCHI Conference on Human Factors in Computing Systems 1457–1466 (Association for Computing Machinery, 2010).
Garimella, K., De Francisci Morales, G., Gionis, A. & Mathioudakis, M. Political discourse on social media: echo chambers, gatekeepers, and the price of bipartisanship. In WWW '18: Proc. 2018 World Wide Web Conference 913–922 (International World Wide Web Conferences Steering Committee, 2018).
Schmidt, A. L. et al. Anatomy of news consumption on Facebook. Proc. Natl Acad. Sci. USA 114, 3035–3039 (2017).
Kitchens, B., Johnson, S. L. & Gray, P. Understanding echo chambers and filter bubbles: the impact of social media on diversification and partisan shifts in news consumption. MIS Q. 44, 1619–1649 (2020).
Kalimeris, D., Bhagat, S., Kalyanaraman, S. & Weinsberg, U. Preference amplification in recommender systems. In Proc. 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining 805–815 (Association for Computing Machinery, 2021).
Korbel, J., Lindner, S. D., Pham, T. M., Hanel, R. & Thurner, S. Homophily-based social group formation in a spin glass self-assembly framework. Phys. Rev. Lett. 130, 057401 (2023).
Lorenz-Spreen, P., Oswald, L., Lewandowsky, S. & Hertwig, R. A systematic review of worldwide causal and correlational evidence on digital media and democracy. Nat. Hum. Behav. 7, 74–101 (2023).
Flamino, J. et al. Political polarization of news media and influencers on Twitter in the 2016 and 2020 US presidential elections. Nat. Hum. Behav. 7, 904–916 (2023).
Levy, R. Social media, news consumption, and polarization: evidence from a field experiment. Am. Econ. Rev. 111, 831–870 (2021).
Bail, C. A. et al. Exposure to opposing views on social media can increase political polarization. Proc. Natl Acad. Sci. USA 115, 9216–9221 (2018).
Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019).
Castelvecchi, D. Can we open the black box of AI?. Nature 538, 20–23 (2016).
Kunaver, M. & Požrl, T. Diversity in recommender systems—a survey. Knowl.-Based Syst. 123, 154–162 (2017).
Liu, P., Shivaram, K., Culotta, A., Shapiro, M. A. & Bilgic, M. The interaction between political typology and filter bubbles in news recommendation algorithms. In WWW '21: Proc. Web Conference 2021 3791–3801 (Association for Computing Machinery, 2021).
Rendle, S., Freudenthaler, C., Gantner, Z. & Schmidt-Thieme, L. BPR: Bayesian personalized ranking from implicit feedback. In UAI ’09: Proc. 25th Conference on Uncertainty in Artificial Intelligence 452–461 (AUAI Press, 2009).
Ding, J., Quan, Y., He, X., Li, Y. & Jin, D. Reinforced negative sampling for recommendation with exposure data. In Proc. 28th International Joint Conference on Artificial Intelligence, IJCAI-19 (ed. Kraus, S.) 2230–2236 (International Joint Conferences on Artificial Intelligence, 2019).
Su, X. & Khoshgoftaar, T. M. A survey of collaborative filtering techniques. Adv. Artif. Intell. 2009, 421425 (2009).
Kobayashi, M. & Takeda, K. Information retrieval on the web. ACM Comput. Surv. 32, 144–173 (2000).
König, M. D., Levchenko, A., Rogers, T. & Zilibotti, F. Aggregate fluctuations in adaptive production networks. Proc. Natl Acad. Sci. USA 119, e2203730119 (2022).
Itô, K. On Stochastic Differential Equations (American Mathematical Society, 1951).
Clifford, P. & Sudbury, A. A model for spatial conflict. Biometrika 60, 581–588 (1973).
Holley, R. A. & Liggett, T. M. Ergodic theorems for weakly interacting infinite systems and the voter model. Ann. Probab. 3, 643–663 (1975).
Piao, J. et al. Open code for in-review natmachintell-a23038004 “Human–AI adaptive dynamics drive emergence of information cocoons”. Code Ocean https://doi.org/10.24433/CO.6503936.v1 (2023).
Piao, J. et al. tsinghua-fib-lab/Adaptive-Information-Dynamic- Model: NMI. Zenodo https://doi.org/10.5281/zenodo.8265474 (2023).
Acknowledgements
We thank J. Ding, Z. Chen and C. Song for discussions and comments on the manuscript. This work was supported in part by the National Key Research and Development Program of China under grant 2020AAA0106000 to Y.L., the National Natural Science Foundation of China under grants 72104126 to F.Z., 71721002 to J.S., U1936217 and U22B2057 to Y.L. The funders had no role in study design, data collection, data analysis, decision to publish, or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
J.P., J.L. and Y.L. designed the model. J.P. performed the experiments and prepared the figures. J.L. conducted the theoretical analysis. F.Z., J.S. and Y.L. provided critical revisions. All authors jointly participated in the writing of the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work. Primary Handling Editor: Liesbeth Venema, in collaboration with the Nature Machine Intelligence team.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Information.
Source data
Source Data Fig. 1
Statistical source data.
Source Data Fig. 2
Statistical source data.
Source Data Fig. 3
Statistical source data.
Source Data Fig. 4
Statistical source data.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Piao, J., Liu, J., Zhang, F. et al. Human–AI adaptive dynamics drives the emergence of information cocoons. Nat Mach Intell 5, 1214–1224 (2023). https://doi.org/10.1038/s42256-023-00731-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s42256-023-00731-4
This article is cited by
-
How to break information cocoons
Nature Machine Intelligence (2023)