Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

How large language models can reshape collective intelligence

Nature Human Behaviour volume 8, pages 1643–1655 (2024)Cite this article

Subjects

Abstract

Collective intelligence underpins the success of groups, organizations, markets and societies. Through distributed cognition and coordination, collectives can achieve outcomes that exceed the capabilities of individuals—even experts—resulting in improved accuracy and novel capabilities. Often, collective intelligence is supported by information technology, such as online prediction markets that elicit the ‘wisdom of crowds’, online forums that structure collective deliberation or digital platforms that crowdsource knowledge from the public. Large language models, however, are transforming how information is aggregated, accessed and transmitted online. Here we focus on the unique opportunities and challenges this transformation poses for collective intelligence. We bring together interdisciplinary perspectives from industry and academia to identify potential benefits, risks, policy-relevant considerations and open research questions, culminating in a call for a closer examination of how large language models affect humans’ ability to collectively tackle complex problems.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Development of information environments over time.

Similar content being viewed by others

References

  1. Hu, K. ChatGPT sets record for fastest-growing user base—analyst note. Reuters (2 February 2023).

  2. Mitchell, M. & Krakauer, D. C. The debate over understanding in AI's large language models. Proc. Natl Acad. Sci. USA 120, e2215907120 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. On the dangers of stochastic parrots: can language models be too big? In Proc. 2021 ACM Conference on Fairness, Accountability, and Transparency 610–623 (Association for Computing Machinery, 2021).

  4. Weidinger, L. et al. Taxonomy of risks posed by language models. In Proc. 2022 ACM Conference on Fairness, Accountability, and Transparency 214–229 (Association for Computing Machinery, 2022).

  5. Bubeck, S. et al. Sparks of artificial general intelligence: early experiments with GPT-4. Preprint at arXiv https://doi.org/10.48550/arXiv.2303.12712 (2023).

  6. Hilbert, M. What is the content of the world’s technologically mediated information and communication capacity: how much text, image, audio, and video? Inf. Soc. 30, 127–143 (2014).

    Article  Google Scholar 

  7. Malone, T. W. & Bernstein, M. S. Handbook of Collective Intelligence (MIT Press, 2015).

  8. Flack, J., Ipeirotis, P., Malone, T. W., Mulgan, G. & Page, S. E. Editorial to the inaugural issue of Collective Intelligence. Collect. Intell. 1, 263391372211141 (2022).

    Article  Google Scholar 

  9. Malone, T. W., Laubacher, R. & Dellarocas, C. The collective intelligence genome. IEEE Eng. Manage. Rev. 38, 38–52 (2010).

    Article  Google Scholar 

  10. Suran, S., Pattanaik, V., Yahia, S. B. & Draheim, D. Exploratory analysis of collective intelligence projects developed within the EU-Horizon 2020 framework. In Computational Collective Intelligence: 11th International Conference, ICCCI 2019, Hendaye, France, September 4–6, 2019: Proceedings Pt 2 (eds Nguyen, N. T. et al.) 285–296 (Springer, 2019).

  11. Suran, S., Pattanaik, V. & Draheim, D. Frameworks for collective intelligence: a systematic literature review. ACM Comput. Surv. 53, 1–36 (2021).

    Article  Google Scholar 

  12. Woolley, A. W. & Gupta, P. Understanding collective intelligence: investigating the role of collective memory, attention, and reasoning processes. Perspect. Psychol. Sci. 19, 344–354 (2024).

    Article  PubMed  Google Scholar 

  13. Salminen, J. Collective intelligence in humans: a literature review. Preprint at arXiv https://doi.org/10.48550/arXiv.1204.3401 (2012).

  14. Smith, A. An Inquiry into the Nature and Causes of the Wealth of Nations (W. Strahan and T. Cadell, 1776).

  15. Hayek, F. A. The use of knowledge in society. Am. Econ. Rev. 35, 519–530 (1945).

    Google Scholar 

  16. Hastie, R. & Kameda, T. The robust beauty of majority rules in group decisions. Psychol. Rev. 112, 494–508 (2005).

    Article  PubMed  Google Scholar 

  17. Surowiecki, J. The Wisdom of Crowds (Anchor Books, 2005).

  18. Dasgupta, P. & Maskin, E. On the robustness of majority rule. J. Eur. Econ. Assoc. 6, 949–973 (2008).

    Article  Google Scholar 

  19. Arrow, K. J. et al. The promise of prediction markets. Science 320, 877–878 (2008).

    Article  CAS  PubMed  Google Scholar 

  20. Simon, H. A. Rational Decision-Making in Business Organizations (American Economic Association, 1978).

  21. Vélez, N., Christian, B., Hardy, M., Thompson, B. D. & Griffiths, T. L. How do humans overcome individual computational limitations by working together? Cogn. Sci. 47, e13232 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  22. Gupta, P., Nguyen, T. N., Gonzalez, C. & Woolley, A. W. Fostering collective intelligence in human–AI collaboration: laying the groundwork for COHUMAIN. Top. Cogn. Sci. https://doi.org/10.1111/tops.12679 (2023).

  23. Riedl, C., Kim, Y. J., Gupta, P., Malone, T. W. & Woolley, A. W. Quantifying collective intelligence in human groups. Proc. Natl Acad. Sci. USA 118, e2005737118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Hong, L. & Page, S. E. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proc. Natl Acad. Sci. USA 101, 16385–16389 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Bang, D. & Frith, C. D. Making better decisions in groups. R. Soc. Open Sci. 4, 170193 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  26. Harrison, D. A., Price, K. H., Gavin, J. H. & Florey, A. T. Time, teams, and task performance: changing effects of surface- and deep-level diversity on group functioning. Acad. Manage. J. 45, 1029–1045 (2002).

    Article  Google Scholar 

  27. Mohammed, S. & Angell, L. C. Surface‐ and deep‐level diversity in workgroups: examining the moderating effects of team orientation and team process on relationship conflict. J. Organ. Behav. 25, 1015–1039 (2004).

    Article  Google Scholar 

  28. Phillips, K. W. & Loyd, D. L. When surface and deep-level diversity collide: the effects on dissenting group members. Organ. Behav. Hum. Decis. Process. 99, 143–160 (2006).

    Article  Google Scholar 

  29. Condorcet, N. Essai sur l’Application de l’Analyse à la Probabilité des Décisions Rendues à la Pluralité des Voix (Chelsea, 1785).

  30. Grofman, B., Owen, G. & Feld, S. L. Thirteen theorems in search of the truth. Theory Decis. 15, 261–278 (1983).

    Article  Google Scholar 

  31. Page, S. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies New Edn (Princeton Univ. Press, 2008).

  32. Hong, L. & Page, S. E. in Collective Wisdom (eds Landemore, H. & Elster, J.) 56–71 (Cambridge Univ. Press, 2012).

  33. Ladha, K. K. The Condorcet jury theorem, free speech, and correlated votes. Am. J. Polit. Sci. 36, 617–634 (1992).

    Article  Google Scholar 

  34. Kameda, T., Toyokawa, W. & Tindale, R. S. Information aggregation and collective intelligence beyond the wisdom of crowds. Nat. Rev. Psychol. 1, 345–357 (2022).

    Article  Google Scholar 

  35. Laan, A., Madirolas, G. & De Polavieja, G. G. Rescuing collective wisdom when the average group opinion is wrong. Front. Robot. AI 4, 56 (2017).

    Article  Google Scholar 

  36. Lyon, A. & Pacuit, E. in Handbook of Human Computation (ed. Michelucci, P.) 599–614 (Springer New York, 2013).

  37. Landemore, H. & Page, S. E. Deliberation and disagreement: problem solving, prediction, and positive dissensus. Polit. Phil. Econ. 14, 229–254 (2015).

    Article  Google Scholar 

  38. List, C. The theory of judgment aggregation: an introductory review. Synthese 187, 179–207 (2012).

    Article  Google Scholar 

  39. Centola, D. The network science of collective intelligence. Trends Cogn. Sci. 26, 923–941 (2022).

    Article  PubMed  Google Scholar 

  40. Baron, R. S. So right it’s wrong: groupthink and the ubiquitous nature of polarized group decision making. Adv. Exp. Soc. Psychol. 37, 219–253 (2005).

    Article  Google Scholar 

  41. Hahn, U., Von Sydow, M. & Merdes, C. How communication can make voters choose less well. Top. Cogn. Sci. 11, 194–206 (2019).

    Article  PubMed  Google Scholar 

  42. Hahn, U., Hansen, J. U. & Olsson, E. J. Truth tracking performance of social networks: how connectivity and clustering can make groups less competent. Synthese 197, 1511–1541 (2020).

    Article  Google Scholar 

  43. Jönsson, M. L., Hahn, U. & Olsson, E. J. The kind of group you want to belong to: effects of group structure on group accuracy. Cognition 142, 191–204 (2015).

    Article  PubMed  Google Scholar 

  44. Becker, J., Brackbill, D. & Centola, D. Network dynamics of social influence in the wisdom of crowds. Proc. Natl Acad. Sci. USA 114, E5070–E5076 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Zollman, K. J. S. The communication structure of epistemic communities. Phil. Sci. 74, 574–587 (2007).

    Article  Google Scholar 

  46. Zollman, K. J. S. The epistemic benefit of transient diversity. Erkenntnis 72, 17–35 (2010).

    Article  Google Scholar 

  47. Zollman, K. J. S. Network epistemology: communication in epistemic communities. Phil. Compass 8, 15–27 (2013).

    Article  Google Scholar 

  48. Smaldino, P. E., Moser, C., Pérez Velilla, A. & Werling, M. Maintaining transient diversity is a general principle for improving collective problem solving. Perspect. Psychol. Sci. 19, 454–464 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Bak-Coleman, J. B. et al. Stewardship of global collective behavior. Proc. Natl Acad. Sci. USA 118, e2025764118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Suran, S. et al. Building global societies on collective intelligence: challenges and opportunities. Digit. Gov. Res. Pract. 3, 1–6 (2022).

    Article  Google Scholar 

  51. Tsvetkova, M., Yasseri, T., Pescetelli, N. & Werner, T. Human-machine social systems. Nat. Hum. Behav. https://doi.org/10.48550/arXiv.2402.14410 (in press).

  52. Cui, H. & Yasseri, T. AI-enhanced collective intelligence. Patterns https://doi.org/10.48550/arXiv.2403.10433 (in press).

  53. Ovadya, A. ‘Generative CI’ through collective response systems. Preprint at arXiv https://doi.org/10.48550/arXiv.2302.00672 (2023).

  54. Zaremba, W. et al. Democratic inputs to AI. OpenAI https://openai.com/blog/democratic-inputs-to-ai (2023).

  55. Lorenz-Spreen, P., Oswald, L., Lewandowsky, S. & Hertwig, R. A systematic review of worldwide causal and correlational evidence on digital media and democracy. Nat. Hum. Behav. 7, 74–101 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  56. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B. & Mordatch, I. Improving factuality and reasoning in language models through multiagent debate. Preprint at arXiv https://doi.org/10.48550/arXiv.2305.14325 (2023).

  57. Wu, Q. et al. AutoGen: enabling next-gen LLM applications via multi-agent conversation. Preprint at arXiv https://doi.org/10.48550/arXiv.2308.08155 (2023).

  58. Yoffe, L., Amayuelas, A. & Wang, W. Y. DebUnc: mitigating hallucinations in large language model agent communication with uncertainty estimations. Preprint at arXiv https://doi.org/10.48550/arXiv.2407.06426 (2024).

  59. Bommasani, R. et al. On the opportunities and risks of foundation models. Preprint at arXiv http://arxiv.org/abs/2108.07258 (2022).

  60. Törnberg, P. ChatGPT-4 outperforms experts and crowd workers in annotating political Twitter messages with zero-shot learning. Preprint at arXiv http://arxiv.org/abs/2304.06588 (2023).

  61. Rathje, S. et al. GPT is an effective tool for multilingual psychological text analysis. Proc. Natl Acad. Sci. USA 131, e2308950121 (2024).

    Article  Google Scholar 

  62. Goldstein, D. G., McAfee, R. P. & Suri, S. The wisdom of smaller, smarter crowds. In Proc. 15th ACM Conference on Economics and Computation 471–488 (Association for Computing Machinery, 2014).

  63. Mannes, A. E., Soll, J. B. & Larrick, R. P. The wisdom of select crowds. J. Pers. Soc. Psychol. 107, 276–299 (2014).

    Article  PubMed  Google Scholar 

  64. Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019).

    Article  CAS  PubMed  Google Scholar 

  65. Hahn, U. Collectives and epistemic rationality. Top. Cogn. Sci. 14, 602–620 (2022).

    Article  PubMed  Google Scholar 

  66. Lafont, C. Deliberation, participation, and democratic legitimacy: should deliberative mini-publics shape public policy? J. Polit. Phil. 23, 40–63 (2015).

    Article  Google Scholar 

  67. Landemore, H. Can AI Bring Deliberation To The Masses? (Stanford Institute for Human-Centered Artificial Intelligence, 2022).

  68. Cohen, R. L. Procedural justice and participation. Hum. Relat. 38, 643–663 (1985).

    Article  Google Scholar 

  69. Greenberg, J. & Folger, R. in Basic Group Processes (ed. Paulus, P. B.) 235–256 (Springer New York, 1983).

  70. El Zein, M., Bahrami, B. & Hertwig, R. Shared responsibility in collective decisions. Nat. Hum. Behav. 3, 554–559 (2019).

    Article  PubMed  Google Scholar 

  71. Fishkin, J. S. When the People Speak: Deliberative Democracy and Public Consultation (Oxford Univ. Press, 2009).

  72. Steiner, I. D. Models for inferring relationships between group size and potential group productivity. Behav. Sci. 11, 273–283 (1966).

    Article  CAS  PubMed  Google Scholar 

  73. Steiner, I. D. Group Process and Productivity (Academic Press, 1972).

  74. Hill, G. W. Group versus individual performance: are N + 1 heads better than one? Psychol. Bull. 91, 517–539 (1982).

    Article  Google Scholar 

  75. Almaatouq, A., Alsobay, M., Yin, M. & Watts, D. J. Task complexity moderates group synergy. Proc. Natl Acad. Sci. USA 118, e2101062118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  76. Straub, V. J., Tsvetkova, M. & Yasseri, T. The cost of coordination can exceed the benefit of collaboration in performing complex tasks. Collect. Intell. 2, 263391372311569 (2023).

    Article  Google Scholar 

  77. Zhu, W. et al. Multilingual machine translation with large language models: empirical results and analysis. In Findings of the Association for Computational Linguistics: NAACL 2024 (eds. Duh, K., Gomez, H. & Bethard, S.) 2765–2781 (Association for Computational Linguistics, 2024).

  78. Bawden, R. & Yvon, F. Investigating the translation performance of a large multilingual language model: the case of BLOOM. In Proc. 24th Annual Conference of the European Association for Machine Translation (eds. Nurminen, M. et al.) 157–170 (European Association for Machine Translation, 2023).

  79. Berdejo-Espinola, V. & Amano, T. AI tools can improve equity in science. Science 379, 991 (2023).

    Article  CAS  PubMed  Google Scholar 

  80. Katsnelson, A. Poor English skills? New AIs help researchers to write better. Nature 609, 208–209 (2022).

    Article  CAS  PubMed  Google Scholar 

  81. Romero-Olivares, A. L. Reviewers, don’t be rude to nonnative English speakers. Science https://doi.org/10.1126/science.caredit.aaz7179 (2019).

  82. Ramírez-Castañeda, V. Disadvantages in preparing and publishing scientific papers caused by the dominance of the English language in science: the case of Colombian researchers in biological sciences. PLoS ONE 15, e0238372 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  83. Brabham, D. C. Crowdsourcing as a model for problem solving: an introduction and cases. Converg. Int. J. Res. N. Media Technol. 14, 75–90 (2008).

    Google Scholar 

  84. von Hippel, E. in Handbook of the Economics of Innovation (eds Hall, B. H. & Rosenberg, N.) Vol. 1, 411–427 (Elsevier, 2010).

  85. Boussioux, L., Lane, J. N., Zhang, M., Jacimovic, V. & Lakhani, K. R. The crowdless future? Generative AI and creative problem solving. Organ. Sci. 0, 1–19 (2024).

  86. Girotra, K., Meincke, L., Terwiesch, C. & Ulrich, K. T. Ideas are dimes a dozen: large language models for idea generation in innovation. SSRN Electron. J. https://doi.org/10.2139/ssrn.4526071 (2023).

  87. Argyle, L. P. et al. Out of one, many: using language models to simulate human samples. Polit. Anal. 31, 337–351 (2023).

    Article  Google Scholar 

  88. Jiang, H., Zhang, X., Cao, X., Breazeal, C., Roy, D. & Kabbara, J. PersonaLLM: investigating the ability of large language models to express personality traits. In Findings of the Association for Computational Linguistics: NAACL 2024 (eds. Duh, K. et al.) 3605–3627 (Association for Computational Linguistics, 2024).

  89. Salewski, L., Alaniz, S., Rio-Torto, I., Schulz, E. & Akata, Z. In-context impersonation reveals large language models’ strengths and biases. In Adv. Neur. Inf. Process. Syst. 36 (NeurIPS 2023) (eds Oh, A. et al.) 72044–720579 (2023).

  90. Ashkinaze, J., Mendelsohn, J., Qiwei, L., Budak, C. & Gilbert, E. How AI ideas affect the creativity, diversity, and evolution of human ideas: evidence from a large, dynamic experiment. Preprint at arXiv https://doi.org/10.48550/arXiv.2401.13481 (2024).

  91. Fink, A. et al. Stimulating creativity via the exposure to other people’s ideas. Hum. Brain Mapp. 33, 2603–2610 (2012).

    Article  PubMed  Google Scholar 

  92. Doshi, A. R. & Hauser, O. Generative artificial intelligence enhances creativity. SSRN Electron. J. https://doi.org/10.2139/ssrn.4535536 (2023).

  93. Herzog, S. M. & Hertwig, R. The wisdom of many in one mind: improving individual judgments with dialectical bootstrapping. Psychol. Sci. 20, 231–237 (2009).

    Article  PubMed  Google Scholar 

  94. Herzog, S. M. & Hertwig, R. Harnessing the wisdom of the inner crowd. Trends Cogn. Sci. 18, 504–506 (2014).

    Article  PubMed  Google Scholar 

  95. Schilling, M. A. & Green, E. Recombinant search and breakthrough idea generation: an analysis of high impact papers in the social sciences. Res. Policy 40, 1321–1331 (2011).

    Article  Google Scholar 

  96. Porciello, J., Ivanina, M., Islam, M., Einarson, S. & Hirsh, H. Accelerating evidence-informed decision-making for the Sustainable Development Goals using machine learning. Nat. Mach. Intell. 2, 559–565 (2020).

    Article  Google Scholar 

  97. Weitzman, M. L. Recombinant growth. Q. J. Econ. 113, 331–360 (1998).

    Article  Google Scholar 

  98. Hibbing, J. R. & Theiss-Morse, E. Stealth Democracy: Americans’ Beliefs about How Government Should Work (Cambridge Univ. Press, 2002).

  99. Rosenberg, S. W. in Deliberative Democracy (eds Elstub, S. & McLaverty, P.) 98–117 (Edinburgh Univ. Press, 2014).

  100. Achen, C. H. & Bartels, L. M. Democracy for Realists: Why Elections Do Not Produce Responsive Government (Princeton Univ. Press, 2017).

  101. Sunstein, C. R. On a danger of deliberative democracy. Daedalus 131, 120–124 (2002).

    Google Scholar 

  102. Neblo, M. A., Esterling, K. M., Kennedy, R. P., Lazer, D. M. J. & Sokhey, A. E. Who wants to deliberate—and why? Am. Polit. Sci. Rev. 104, 566–583 (2010).

    Article  Google Scholar 

  103. Maćkowiak, B., Matějka, F. & Wiederholt, M. Rational inattention: a review. J. Econ. Lit. 61, 226–273 (2023).

    Article  Google Scholar 

  104. Shim, J. P. et al. Past, present, and future of decision support technology. Decis. Support Syst. 33, 111–126 (2002).

    Article  Google Scholar 

  105. Donohoe, H., Stellefson, M. & Tennant, B. Advantages and limitations of the e-Delphi technique. Am. J. Health Educ. 43, 38–46 (2012).

    Article  Google Scholar 

  106. Dalkey, N. & Helmer, O. An experimental application of the Delphi method to the use of experts. Manage. Sci. 9, 458–467 (1963).

    Article  Google Scholar 

  107. Tetlock, P. E., Mellers, B. A., Rohrbaugh, N. & Chen, E. Forecasting tournaments: tools for increasing transparency and improving the quality of debate. Curr. Dir. Psychol. Sci. 23, 290–295 (2014).

    Article  Google Scholar 

  108. McAndrew, T. et al. Early human judgment forecasts of human monkeypox, May 2022. Lancet Digit. Health 4, e569–e571 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  109. Lin, X. Exploring the role of ChatGPT as a facilitator for motivating self-directed learning among adult learners. Adult Learn. 35, 56–166 (2023).

    Google Scholar 

  110. Argyle, L. P. et al. AI chat assistants can improve conversations about divisive topics. Preprint at arXiv https://doi.org/10.48550/arXiv.2302.07268 (2023).

  111. Hadfi, R. et al. Conversational agents enhance women’s contribution in online debates. Sci. Rep. 13, 14534 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  112. Small, C. T. et al. Opportunities and risks of LLMs for scalable deliberation with polis. Preprint at arXiv https://doi.org/10.48550/arXiv.2306.11932 (2023).

  113. Fishkin, J. et al. Deliberative democracy with the online deliberation platform. In 7th AAAI Conference on Human Computation and Crowdsourcing, https://www.humancomputation.com/2019/assets/papers/144.pdf (Association for the Advancement of Artificial Intelligence, 2019).

  114. Miller, K. A moderator ChatBot for civic discourse. Stanford HAI https://hai.stanford.edu/news/moderator-chatbot-civic-discourse (2020).

  115. Jackson, M. O. & Xing, Y. Culture-dependent strategies in coordination games. Proc. Natl Acad. Sci. USA 111, 10889–10896 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  116. Coavoux, M., Elsahar, H. & Gallé, M. Unsupervised aspect-based multi-document abstractive summarization. In Proc. 2nd Workshop on New Frontiers in Summarization (eds Wang, L. et al.) 42–47 (Association for Computational Linguistics, 2019).

  117. Angelidis, S., Amplayo, R. K., Suhara, Y., Wang, X. & Lapata, M. Extractive opinion summarization in quantized transformer spaces. Trans. Assoc. Comput. Linguist. 9, 277–293 (2021).

    Article  Google Scholar 

  118. Suhara, Y., Wang, X., Angelidis, S. & Tan, W.-C. OpinionDigest: a simple framework for opinion summarization. In Proc. 58th Annual Meeting of the Association for Computational Linguistics (eds Jurafsky, D. et al.) 5789–5798 (Association for Computational Linguistics, 2020).

  119. Bakker, M. et al. Fine-tuning language models to find agreement among humans with diverse preferences. In Adv. Neur. Inf. Process. Syst. 35 (NeurIPS 2022) (eds Koyejo, S. et al.) 38176–38189 (2022).

  120. Huang, S. & Siddarth, D. Generative AI and the digital commons. Preprint at arXiv https://doi.org/10.48550/arXiv.2303.11074 (2023).

  121. Veselovsky, V., Ribeiro, M. H. & West, R. Artificial artificial artificial intelligence: crowd workers widely use large language models for text production tasks. Preprint at arXiv https://doi.org/10.48550/arXiv.2306.07899 (2023).

  122. del Rio-Chanona, M., Laurentsyeva, N. & Wachs, J. Are large language models a threat to digital public goods? Evidence from activity on Stack Overflow. Preprint at arXiv https://doi.org/10.48550/arXiv.2307.07367 (2023).

  123. Farič, N. & Potts, H. W. Motivations for contributing to health-related articles on Wikipedia: an interview study. J. Med. Internet Res. 16, e260 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  124. Javanmardi, S., Ganjisaffar, Y., Lopes, C. & Baldi, P. User contribution and trust in Wikipedia. In Proc. 5th International ICST Conference on Collaborative Computing: Networking, Applications, Worksharing (eds Joshi, J. & Zhang, T.) https://doi.org/10.4108/ICST.COLLABORATECOM2009.8376 (Institute of Electrical and Electronics Engineers, 2009).

  125. Adaji, I. & Vassileva, J. in Social Informatics (eds Spiro, E. & Ahn, Y.-Y.) 3–13 (Springer International, 2016).

  126. Blincoe, K., Sheoran, J., Goggins, S., Petakovic, E. & Damian, D. Understanding the popular users: following, affiliation influence and leadership on GitHub. Inf. Softw. Technol. 70, 30–39 (2016).

    Article  Google Scholar 

  127. Franceschelli, G. & Musolesi, M. Copyright in generative deep learning. Data Policy 4, e17 (2022).

    Article  Google Scholar 

  128. Desai, S. C., Xie, B. & Hayes, B. K. Getting to the source of the illusion of consensus. Cognition 223, 105023 (2022).

    Article  Google Scholar 

  129. Noelle-Neumann, E. The spiral of silence: a theory of public opinion. J. Commun. 24, 43–51 (1974).

    Article  Google Scholar 

  130. Wikipedia: five pillars. Wikipedia https://en.wikipedia.org/w/index.php?title=Wikipedia:Five_pillars (2023).

  131. Yasseri, T. & Kertész, J. Value production in a collaborative environment: sociophysical studies of Wikipedia. J. Stat. Phys. 151, 414–439 (2013).

    Article  Google Scholar 

  132. Hecht, B. & Gergle, D. The tower of Babel meets Web 2.0: user-generated content and its applications in a multilingual context. In Proc. SIGCHI Conference on Human Factors in Computing Systems 291–300 (Association for Computing Machinery, 2010).

  133. Graham, M. The problem with Wikidata. Atlantic (6 April 2012).

  134. Clemen, R. T. & Winkler, R. L. Limits for the precision and value of information from dependent sources. Oper. Res. 33, 427–442 (1985).

    Article  Google Scholar 

  135. Davis-Stober, C. P., Budescu, D. V., Dana, J. & Broomell, S. B. When is a crowd wise?. Decision 1, 79–101 (2014).

    Article  Google Scholar 

  136. Herzog, S. M., Litvinova, A., Yahosseini, K. S., Tump, A. N. & Kurvers, R. H. J. M. in Taming Uncertainty (eds Hertwig, R. et al.) 245–262 (MIT Press, 2019).

  137. Kurvers, R. H. J. M. et al. How to detect high-performing individuals and groups: decision similarity predicts accuracy. Sci. Adv. 5, eaaw9011 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  138. Palley, A. B. & Soll, J. B. Extracting the wisdom of crowds when information is shared. Manage. Sci. 65, 2291–2309 (2019).

    Google Scholar 

  139. Walzner, D. D., Fuegener, A. & Gupta, A. Managing AI advice in crowd decision-making. In ICIS 2022 Proceedings https://aisel.aisnet.org/icis2022/hci_robot/hci_robot/7 (Association for Information Systems, 2022).

  140. Padmakumar, V. & He, H. Does writing with language models reduce content diversity? Preprint at arXiv https://doi.org/10.48550/arXiv.2309.05196 (2023).

  141. Kleinberg, J. & Raghavan, M. Algorithmic monoculture and social welfare. Proc. Natl Acad. Sci. USA 118, e2018340118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  142. Campbell, C. M., Izquierdo, E. J. & Goldstone, R. L. Partial copying and the role of diversity in social learning performance. Collect. Intell. 1, 263391372210818 (2022).

    Article  Google Scholar 

  143. Toyokawa, W., Whalen, A. & Laland, K. N. Social learning strategies regulate the wisdom and madness of interactive crowds. Nat. Hum. Behav. 3, 183–193 (2019).

    Article  PubMed  Google Scholar 

  144. Almaatouq, A., Rahimian, M. A., Burton, J. W. & Alhajri, A. The distribution of initial estimates moderates the effect of social influence on the wisdom of the crowd. Sci. Rep. 12, 16546 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  145. Brinkmann, L. et al. Machine culture. Nat. Hum. Behav. 7, 1855–1868 (2023).

    Article  PubMed  Google Scholar 

  146. OpenAI. GPT-4 technical report. Preprint at arXiv https://doi.org/10.48550/arXiv.2303.08774 (2023).

  147. Semnani, S., Yao, V., Zhang, H. & Lam, M. WikiChat: stopping the hallucination of large language model chatbots by few-shot grounding on wikipedia. In Findings of the Association for Computational Linguistics: EMNLP 2023 (eds. Bouamor, H., Pino, J. & Bali, K.) 2387–2413 (Association for Computational Linguistics, 2023).

  148. Goldstein, J. A. et al. Generative language models and automated influence operations: emerging threats and potential mitigations. Preprint at arXiv https://doi.org/10.48550/arXiv.2301.04246 (2023).

  149. Simon, F. M., Altay, S. & Mercier, H. Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harv. Kennedy Sch. Misinform. Rev. https://doi.org/10.37016/mr-2020-127 (2023).

  150. Musser, M. A cost analysis of generative language models and influence operations. Preprint at arXiv https://doi.org/10.48550/arXiv.2308.03740 (2023).

  151. Kapoor, S. & Narayanan, A. How to Prepare for the Deluge of Generative AI on Social Media (Knight First Amendment Institute, 2023).

  152. Solaiman, I. The gradient of generative AI release: methods and considerations. In 2023 ACM Conference on Fairness, Accountability, and Transparency 111–122 (Association for Computing Machinery, 2023).

  153. Warso, Z. & Keller, P. Open source AI and the paradox of open. Open Future https://openfuture.eu/blog/open-source-ai-and-the-paradox-of-open (2023).

  154. Kapoor, S. et al. On the societal impact of open foundation models. Preprint at arXiv https://doi.org/10.48550/arXiv.2403.07918 (2024).

  155. Widder, D. G., West, S. & Whittaker, M. Open (for business): big tech, concentrated power, and the political economy of open AI. SSRN Electron. J. https://doi.org/10.2139/ssrn.4543807 (2023).

  156. National Artificial Intelligence Initiative https://www.ai.gov/nairrtf/ (National Artificial Intelligence Research Resource Task Force, 2024).

  157. Artificial Intelligence Act (European Parliament, 2023).

  158. Kapoor, S. & Narayanan, A. Generative AI Companies Must Publish Transparency Reports (Knight First Amendment Institute, 2023).

  159. Mökander, J., Schuett, J., Kirk, H. R. & Floridi, L. Auditing large language models: a three-layered approach. AI Ethics https://doi.org/10.1007/s43681-023-00289-2 (2023).

  160. Chang, Y. et al. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol. 15, 39 (2024).

    Article  Google Scholar 

  161. Knott, A. et al. Generative AI models should include detection mechanisms as a condition for public release. Ethics Inf. Technol. 25, 55 (2023).

    Article  Google Scholar 

  162. Berditchevskaia, A., Malliaraki, E. & Peach, K. Participatory AI for Humanitarian Innovation: A Briefing Paper (Nesta, 2021).

  163. Meta’s Third-Party Fact-Checking Program https://www.facebook.com/formedia/mjp/programs/third-party-fact-checking (Meta Journalism Project, accessed 29 March 2024).

  164. Porter, E. & Wood, T. J. The global effectiveness of fact-checking: evidence from simultaneous experiments in Argentina, Nigeria, South Africa, and the United Kingdom. Proc. Natl Acad. Sci. USA 118, e2104235118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  165. Walter, N., Cohen, J., Holbert, R. L. & Morag, Y. Fact-checking: a meta-analysis of what works and for whom. Polit. Commun. 37, 350–375 (2020).

    Article  Google Scholar 

  166. Carnahan, D. & Bergan, D. E. Correcting the misinformed: the effectiveness of fact-checking messages in changing false beliefs. Polit. Commun. 39, 166–183 (2022).

    Article  Google Scholar 

  167. Ecker, U. K. H. et al. The psychological drivers of misinformation belief and its resistance to correction. Nat. Rev. Psychol. 1, 13–29 (2022).

    Article  Google Scholar 

  168. Cai, A. et al. DesignAID: using generative AI and semantic diversity for design inspiration. In Proc. ACM Collective Intelligence Conference (eds Bernstein, M. et al.) 1–11 (Association for Computing Machinery, 2023).

  169. Griebel, M., Flath, C. & Friesike, S. Augmented creativity: leveraging artificial intelligence for idea generation in the creative sphere. ECIS 2020 Research-in-Progress Papers https://aisel.aisnet.org/ecis2020_rip/77 (Association for Information Systems, 2020).

  170. Wittenberg, C., Tappin, B. M., Berinsky, A. J. & Rand, D. G. The (minimal) persuasive advantage of political video over text. Proc. Natl Acad. Sci. USA 118, e2114388118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  171. Radford, A. et al. Language models are unsupervised multitask learners. OpenAI Blog (2019).

  172. Hoffmann, J. et al. Training compute-optimal large language models. In 36th Conf Neural Information Processing, https://proceedings.neurips.cc/paper_files/paper/2022/file/c1e2faff6f588870935f114ebe04a3e5-Paper-Conference.pdf (NeurIPS, 2022).

  173. Ouyang, L. et al. Training language models to follow instructions with human feedback. In Adv. Neur. Inf. Process. Syst. 35 (NeurIPS 2022) (eds Koyejo, S. et al.) 27730–27744 (2022).

  174. Lee, A., Miranda, B. & Koyejo, S. Beyond scale: the diversity coefficient as a data quality metric demonstrates LLMs are pre-trained on formally diverse data. Preprint at arXiv http://arxiv.org/abs/2306.13840 (2023).

  175. Atari, M., Xue, M. J., Park, P. S., Blasi, D. E. & Henrich, J. Which humans? Preprint at PsyArXiv https://doi.org/10.31234/osf.io/5b26t (2023).

  176. Cao, Y. et al. Assessing cross-cultural alignment between ChatGPT and human societies: an empirical study. In Proc. First Workshop on Cross-Cultural Considerations in NLP (C3NLP) 53–67 (Association for Computational Linguistics, 2023).

  177. Haller, P., Aynetdinov, A. & Akbik, A. OpinionGPT: modelling explicit biases in instruction-tuned LLMs. Preprint at arXiv https://doi.org/10.48550/arXiv.2309.03876 (2023).

  178. Levy, S. et al. Comparing biases and the impact of multilingual training across multiple languages. In Proc. 2023 Conference on Empirical Methods in Natural Language Processing (eds Bouamor, H. et al.) 10260–10280 (Association for Computational Linguistics, 2023).

  179. Arora, A., Kaffee, L.-A. & Augenstein, I. Probing pre-trained language models for cross-cultural differences in values. Preprint at arXiv https://doi.org/10.48550/arXiv.2203.13722 (2023).

  180. Dietterich, T. G. in Multiple Classifier Systems. MCS 2000. Lecture Notes in Computer Science, vol 1857, https://doi.org/10.1007/3-540-45014-9_1 (Springer Berlin Heidelberg, 2000).

  181. Grossmann, I. et al. AI and the transformation of social science research. Science 380, 1108–1109 (2023).

    Article  CAS  PubMed  Google Scholar 

  182. Bail, C. A. Can generative AI improve social science? Proc. Natl Acad. Sci. USA 121, e2314021121 (2024).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  183. Aher, G. V., Arriaga, R. I. & Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In Proc. 40th International Conference on Machine Learning 337–371 (Proceedings of Machine Learning Research, 2023).

  184. Dillion, D., Tandon, N., Gu, Y. & Gray, K. Can AI language models replace human participants? Trends Cogn. Sci. 27, 597–600 (2023).

    Article  PubMed  Google Scholar 

  185. Epstein, J. M. & Axtell, R. Growing Artificial Societies: Social Science from the Bottom Up (Brookings Institution Press, 1996).

  186. Bonabeau, E. Agent-based modeling: methods and techniques for simulating human systems. Proc. Natl Acad. Sci. USA 99, 7280–7287 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  187. Park, J. S. et al. Social simulacra: creating populated prototypes for social computing systems. In Proc. 35th Annual ACM Symposium on User Interface Software and Technology (eds Agrawala, M. et al.) 1–18 (Association for Computing Machinery, 2022).

  188. Gao, C. et al. S3: social-network simulation system with large language model-empowered agents. Preprint at arXiv https://doi.org/10.48550/arXiv.2307.14984 (2023).

  189. Horton, J. J. Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? Working Paper 31122 (National Bureau of Economic Research, 2023).

  190. Chen, L., Zaharia, M. & Zou, J. How is ChatGPT’s behavior changing over time? Harvard Data Sci. Rev. 6, https://doi.org/10.1162/99608f92.5317da47 (2024).

  191. Burton, J. W., Stein, M. & Jensen, T. B. A systematic review of algorithm aversion in augmented decision making. J. Behav. Decis. Mak. 33, 220–239 (2020).

    Article  Google Scholar 

  192. Glikson, E. & Woolley, A. W. Human trust in artificial intelligence: review of empirical research. Acad. Manage. Ann. 14, 627–660 (2020).

    Article  Google Scholar 

Download references

Acknowledgements

We thank D. Ain for her meticulous editing. We also thank all participants of the summer retreat at the Center for Adaptive Rationality, Max Planck Institute for Human Development, who provided helpful feedback on the original conceptualization of this work. J.W.B. was supported by an Alexander von Humboldt Foundation Research Fellowship. T.Y. was funded by the Irish Research Council (grant no. IRCLA/2022/3217). S.M.H. and A.B. are funded by the European Union’s Horizon Europe Programme (grant agreement ID 101070588) and UKRI (project no. 10037991). E.L.-L., S.M.H. and U.H. were funded by the Deutsche Forschungsgemeinschaft (project no. 458366841). S.L. was supported by Science Foundation Ireland (grant no. 12/RC/2289_P2). R.H.J.M.K. is funded by the Deutsche Forschungsgemeinschaft (project no. 45836684).

Author information

Authors and Affiliations

  1. Department of Digitalization, Copenhagen Business School, Frederiksberg, Denmark

    Jason W. Burton

  2. Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany

    Jason W. Burton, Ezequiel Lopez-Lopez, Shahar Hechtlinger, Zoe Rahwan, Samuel Aeschbach, Julian Berger, Stefan M. Herzog, Pietro Nickl, Ralf H. J. M. Kurvers, Dirk U. Wulff & Ralph Hertwig

  3. Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany

    Shahar Hechtlinger, Julian Berger & Pietro Nickl

  4. Center for Cognitive and Decision Sciences, University of Basel, Basel, Switzerland

    Samuel Aeschbach & Dirk U. Wulff

  5. Google DeepMind, London, UK

    Michiel A. Bakker

  6. UCL School of Management, University College London, London, UK

    Joshua A. Becker

  7. Centre for Collective Intelligence Design, Nesta, London, UK

    Aleks Berditchevskaia

  8. Center for Humans and Machines, Max Planck Institute for Human Development, Berlin, Germany

    Levin Brinkmann, Anne-Marie Nussberger & Iyad Rahwan

  9. Bonn-Aachen International Center for Information Technology, University of Bonn, Bonn, Germany

    Lucie Flek

  10. Lamarr Institute for Machine Learning and Artificial Intelligence, Bonn, Germany

    Lucie Flek

  11. Collective Intelligence Project, San Francisco, CA, USA

    Saffron Huang & Divya Siddarth

  12. Center for Information Technology Policy, Princeton University, Princeton, NJ, USA

    Sayash Kapoor & Arvind Narayanan

  13. Department of Computer Science, Princeton University, Princeton, NJ, USA

    Sayash Kapoor & Arvind Narayanan

  14. School of Sociology, University College Dublin, Dublin, Ireland

    Taha Yasseri

  15. Geary Institute for Public Policy, University College Dublin, Dublin, Ireland

    Taha Yasseri

  16. Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA

    Abdullah Almaatouq

  17. Department of Psychological Sciences, Birkbeck, University of London, London, UK

    Ulrike Hahn

  18. Science of Intelligence Excellence Cluster, Technical University Berlin, Berlin, Germany

    Ralf H. J. M. Kurvers

  19. School of Information and Communication, Insight SFI Research Centre for Data Analytics, University College Dublin, Dublin, Ireland

    Susan Leavy

  20. Oxford Internet Institute, Oxford University, Oxford, UK

    Divya Siddarth

  21. Deliberative Democracy Lab, Stanford University, Stanford, CA, USA

    Alice Siu

  22. Tepper School of Business, Carnegie Mellon University, Pittsburgh, PA, USA

    Anita W. Woolley

Authors
  1. Jason W. Burton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ezequiel Lopez-Lopez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shahar Hechtlinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zoe Rahwan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Samuel Aeschbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michiel A. Bakker
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joshua A. Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Aleks Berditchevskaia
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Julian Berger
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Levin Brinkmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Lucie Flek
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Stefan M. Herzog
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Saffron Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Sayash Kapoor
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Arvind Narayanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Anne-Marie Nussberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Taha Yasseri
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Pietro Nickl
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Abdullah Almaatouq
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Ulrike Hahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Ralf H. J. M. Kurvers
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Susan Leavy
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Iyad Rahwan
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Divya Siddarth
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Alice Siu
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Anita W. Woolley
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Dirk U. Wulff
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Ralph Hertwig
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jason W. Burton.

Ethics declarations

Competing interests

M.A.B. is employed by Google DeepMind. S. Huang and D.S. are employed by the Collective Intelligence Project. L.F. is affiliated with the Lamarr Institute of ML and AI, an associated partner in the OpenGPT-X project via Fraunhofer IAIS. The other authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Giovanni Luca Ciampaglia and John Meluso for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Burton, J.W., Lopez-Lopez, E., Hechtlinger, S. et al. How large language models can reshape collective intelligence. Nat Hum Behav 8, 1643–1655 (2024). https://doi.org/10.1038/s41562-024-01959-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-024-01959-9

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing