Wisdom of stakeholder crowds in complex social–ecological systems


Sustainable management of natural resources requires adequate scientific knowledge about complex relationships between human and natural systems. Such understanding is difficult to achieve in many contexts due to data scarcity and knowledge limitations. We explore the potential of harnessing the collective intelligence of resource stakeholders to overcome this challenge. Using a fisheries example, we show that by aggregating the system knowledge held by stakeholders through graphical mental models, a crowd of diverse resource users produces a system model of social–ecological relationships that is comparable to the best scientific understanding. We show that the averaged model from a crowd of diverse resource users outperforms those of more homogeneous groups. Importantly, however, we find that the averaged model from a larger sample of individuals can perform worse than one constructed from a smaller sample. However, when averaging mental models within stakeholder-specific subgroups and subsequently aggregating across subgroup models, the effect is reversed. Our work identifies an inexpensive, yet robust way to develop scientific understanding of complex social–ecological systems by leveraging the collective wisdom of non-scientist stakeholders.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Fig. 1: Centrality profiles of different groups (in colour) and the expert reference model (in black/grey).
Fig. 2: Agreement on strong causal patterns in the FCM of stakeholder-specific groups, the crowd and the experts.
Fig. 3: Eigenvalue similarity index.
Fig. 4: The dynamic distance between the experts model and the stakeholder-derived models based on 10,000 randomly generated scenarios (experiments).
Fig. 5: The sampling and averaging effect on performance error in crowds built by drawing and aggregating mental models using two aggregation methods.

Data availability

All data supporting the findings of this study including data for obtaining the FCM of individuals are available and can be downloaded as Excel spreadsheets (Supplementary Datasets 1,2,3). Raw data files are available from the corresponding author on reasonable request.

Code availability

Codes for mental model aggregation and FCM analyses are publically available and can be obtained on GitHub at https://github.com/payamaminpour/PyFCM/wiki.


  1. 1.

    Liu, J. et al. Complexity of coupled human and natural systems. Science 317, 1513–1516 (2007).

  2. 2.

    Levin, S. et al. Social–ecological systems as complex adaptive systems: modeling and policy implications. Environ. Dev. Econ. 18, 111–132 (2013).

  3. 3.

    Johannes, R. E. The case for data-less marine resource management: examples from tropical nearshore finfisheries. Trends Ecol. Evol. 13, 243–246 (1998).

  4. 4.

    Arlinghaus, R. & Krause, J. Wisdom of the crowd and natural resource management. Trends Ecol. Evol. 28, 8–11 (2013).

  5. 5.

    Post, J. et al. Canada’s recreational fisheries: the invisible collapse? Fisheries 27, 6–17 (2002).

  6. 6.

    Walters, C. J. Adaptive Management of Renewable Resources (Macmillan, 1986).

  7. 7.

    Arlinghaus, R., Tillner, R. & Bork, M. Explaining participation rates in recreational fishing across industrialised countries. Fish. Manag. Ecol. 22, 45–55 (2015).

  8. 8.

    Carruthers, T. R. et al. Landscape-scale social and ecological outcomes of dynamic angler and fish behaviours: processes, data, and patterns. Can. J. Fish. Aquat. Sci. 76, 970–988 (2018).

  9. 9.

    Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N. & Malone, T. W. Evidence for a collective intelligence factor in the performance of human groups. Science 330, 686–688 (2010).

  10. 10.

    Irwin, A. Citizen Science: A Study of People, Expertise and Sustainable Development (Routledge, 1995).

  11. 11.

    Bonney, R. Citizen science: a developing tool for expanding science knowledge and scientific literacy. BioScience 59, 977–984 (2009).

  12. 12.

    Galton, F. Vox populi (the wisdom of crowds). Nature 75, 450–451 (1907).

  13. 13.

    Surowiecki, J. The Wisdom of Crowds: Why the Many are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations (Doubleday, 2004).

  14. 14.

    Yi, S. K. M., Steyvers, M., Lee, M. D. & Dry, M. J. The wisdom of the crowd in combinatorial problems. Cogn. Sci. 36, 452–470 (2012).

  15. 15.

    Bernstein, E., Shore, J. & Lazer, D. How intermittent breaks in interaction improve collective intelligence. Proc. Natl Acad. Sci. USA 115, 8734–8739 (2018).

  16. 16.

    Worm, B. et al. Rebuilding global fisheries. Science 325, 578–585 (2009).

  17. 17.

    Arlinghaus, R. et al. Opinion: governing the recreational dimension of global fisheries. Proc. Natl Acad. Sci. USA 116, 5209–5213 (2019).

  18. 18.

    Craik, K. J. W. The Nature of Explanation (Cambridge Univ. Press, 1967).

  19. 19.

    Jetter, A. & Kok, K. Fuzzy cognitive maps for futures studies—a methodological assessment of concepts and methods. Futures 61, 45–57 (2014).

  20. 20.

    Gray, S., Chan, A., Clark, D. & Jordan, R. Modeling the integration of stakeholder knowledge in social–ecological decision-making: benefits and limitations to knowledge diversity. Ecol. Model. 229, 88–96 (2012).

  21. 21.

    Gray, S., Hilsberg, J., McFall, A. & Arlinghaus, R. The structure and function of angler mental models about fish population ecology: the influence of specialization and target species. J. Outdoor Recreat. Tour. 12, 1–13 (2015).

  22. 22.

    Page, S. E. Making the difference: applying a logic of diversity. Acad. Manag. Perspect. 21, 6–20 (2007).

  23. 23.

    Hong, L. & Page, S. E. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proc. Natl Acad. Sci. USA 101, 16385–16389 (2004).

  24. 24.

    Boudreau, K. J., Lacetera, N. & Lakhani, K. R. Incentives and problem uncertainty in innovation contests: an empirical analysis. Manag. Sci. 57, 843–863 (2011).

  25. 25.

    Kao, A. B. & Couzin, I. D. Decision accuracy in complex environments is often maximized by small group sizes. Proc. R. Soc. B. 281, 20133305 (2014).

  26. 26.

    Kao, A. B. & Couzin, I. D. Modular structure within groups causes information loss but can improve decision accuracy. Phil. Trans. R. Soc. Lond. B. 374, 20180378 (2019).

  27. 27.

    Lorenz, J., Rauhut, H., Schweitzer, F. & Helbing, D. How social influence can undermine the wisdom of crowd effect. Proc. Natl Acad. Sci. USA 108, 9020–9025 (2011).

  28. 28.

    Navajas, J., Niella, T., Garbulsky, G., Bahrami, B. & Sigman, M. Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds. Nat. Hum. Behav. 2, 126 (2018).

  29. 29.

    Hooker, R. H. Mean or median. Nature 75, 487–488 (1907).

  30. 30.

    Perry-Coste, F. H. The ballot-box. Nature 75, 509–509 (1907).

  31. 31.

    Becker, J., Brackbill, D. & Centola, D. Network dynamics of social influence in the wisdom of crowds. Proc. Natl Acad. Sci. USA 114, 5070–5076 (2017).

  32. 32.

    Jentoft, S. Fisheries co-management: delegating government responsibility to fishermen’s organizations. Mar. Policy 13, 137–154 (1989).

  33. 33.

    Kosko, B. Fuzzy cognitive maps. Int. J. Man. Mach. Stud. 24, 65–75 (1986).

  34. 34.

    Daedlow, K., Beard, T. D. & Arlinghaus, R. A property rights-based view on management of inland recreational fisheries: contrasting common and public fishing rights regimes in Germany and the United States. Am. Fish. Soc. Symp. 75, 13–38 (2011).

  35. 35.

    von Lukowicz, M. in Recreational Fisheries: Social, Economic, and Management Aspects (eds Hickley, P. & Tompkins, H.) 287–293 (Blackwell Scientific, 1998).

  36. 36.

    Fujitani, M., McFall, A., Randler, C. & Arlinghaus, R. Participatory adaptive management leads to environmental learning outcomes extending beyond the sphere of science. Sci. Adv. 3, e1602516 (2017).

  37. 37.

    Gray, S. A., Zanre, E. & Gray, S. R. in Fuzzy Cognitive Maps for Applied Sciences and Engineering (ed. Papageorgiou, E. I.) 29–48 (Springer, 2014).

  38. 38.

    Recreational Fisheries, Technical Guidelines for Responsible Fisheries No. 13 (FAO, 2012); http://www.fao.org/docrep/016/i2708e/i2708e00.htm

  39. 39.

    Townsley, P. Social Issues in Fisheries Fisheries Technical Paper No. 375 (FAO, 1998).

  40. 40.

    Folke, C., Pritchard, L., Berkes, F., Colding, J. & Svedin, U. The problem of fit between ecosystems and institutions: ten years later. Ecol. Soc. 12, 30 (2007).

  41. 41.

    Ostrom, E. Governing the Commons (Cambridge Univ. Press, 1990).

  42. 42.

    Ostrom, E. A diagnostic approach for going beyond panaceas. Proc. Natl Acad. Sci. USA 104, 15181–15187 (2007).

  43. 43.

    Worm, B. & Branch, T. A. The future of fish. Trends Ecol. Evol. 27, 594–599 (2012).

  44. 44.

    Arlinghaus, R., Mehner, T. & Cowx, I. Reconciling traditional inland fisheries management and sustainability in industrialized countries, with emphasis on Europe. Fish Fish. 3, 261–316 (2002).

  45. 45.

    Lorenzen, K., Beveridge, M. C. M. & Mangel, M. Cultured fish: integrative biology and management of domestication and interactions with wild fish. Biol. Rev. 87, 639–660 (2012).

  46. 46.

    Cooke, S. J., Arlinghaus, R., Johnson, B. M. & Cowx, I. G. in Freshwater Fisheries Ecology (ed. Craig, J. F.) 449–465 (Wiley-Blackwell, 2016).

  47. 47.

    Sandström, A. Navigating a complex policy system: explaining local divergences in Swedish fish stocking policy. Mar. Policy 35, 419 (2011).

  48. 48.

    Dorow, M., Beardmore, B., Haider, W. & Arlinghaus, R. Using a novel survey technique to predict fisheries stakeholders’ support for European eel (Anguilla anguilla L.) conservation programs. Biol. Conserv. 142, 2973–2982 (2009).

  49. 49.

    Gray, S. A. et al. Using fuzzy cognitive mapping as a participatory approach to analyze change, preferred states, and perceived resilience of social–ecological systems. Ecol. Soc. 20, 11 (2015).

  50. 50.

    Craig, J. F. A short review of pike ecology. Hydrobiologia 601, 5–16 (2008).

  51. 51.

    Giabbanelli, P. J., Gray, S. A. & Aminpour, P. Combining fuzzy cognitive maps with agent-based modeling: frameworks and pitfalls of a powerful hybrid modeling approach to understand human–environment interactions. Environ. Model. Softw. 95, 320–325 (2017).

  52. 52.

    Ozesmi, U. & Ozesmi, S. Ecological models based on people’s knowledge: a multi-step fuzzy cognitive mapping approach. Ecol. Model. 176, 43–64 (2004).

  53. 53.

    Dorow, M. & Arlinghaus, R. The relationship between personal commitment to angling and the opinions and attitudes of German anglers towards the conservation and management of the European eel Anguilla anguilla. N. Am. J. Fish. Manag. 32, 466–479 (2012).

  54. 54.

    Baer, J., Blasel, K. & Diekmann, M. Benefits of repeated stocking with adult, hatchery-reared brown trout, Salmo trutta, to recreational fisheries? Fish. Manag. Ecol. 14, 51–59 (2007).

  55. 55.

    Mannes, A., Soll, J. & Larrick, R. The wisdom of select crowds. J. Pers. Soc. Psychol. 107, 276–299 (2014).

  56. 56.

    Budescu, D. & Chen, E. Identifying expertise to extract the wisdom of crowds. Manag. Sci. 61, 267–280 (2015).

  57. 57.

    van Dolder, D. & van den Assem, M. J. The wisdom of the inner crowd in three large natural experiments. Nat. Hum. Behav. 2, 21 (2018).

  58. 58.

    Katz, L. A. New status index derived from sociometric analysis. Psychometrika 18, 39–43 (1953).

  59. 59.

    Lavin, E. A., Giabbanelli, P. J., Stefanik, A. T., Gray, S. A. & Arlinghaus, R. Should we simulate mental models to assess whether they agree? In Proc. Annual Simulation Symposium Article no. 6 (Society for Computer Simulation International, 2018).

  60. 60.

    Koutra, D., Shah, N., Vogelstein, J., Gallagher, B. & Faloutsos, C. DELTACON: principled massive-graph similarity function with attribution. ACM Trans. Knowl. Discov. Data 10, 1–43 (2016).

  61. 61.

    Koutra, D., Ramdas, A. & Xiang, J. Algorithms for Graph Similarity and Subgraph Matching (Carnegie-Mellon Univ., 2011).

  62. 62.

    Yoon, B. S. & Jetter, A. J. Comparative analysis for fuzzy cognitive mapping. In Portland International Conference on Management of Engineering and Technology (2016); https://doi.org/10.1109/PICMET.2016.7806755

Download references


We thank A. McFall and J. Hilsberg for their assistance in data collection. Funding was granted to R.A. by the German Ministry for Education and Research (grant nos. 01UU0907, 033W046A and 01LC1826E), the European Union through the European Maritime and Fisheries Fund and the State of Mecklenburg-Vorpommern (grant nos. MV-I.18-LM-004, B 730117000069). Funding was granted to S.A.G. from the Socio-Environmental Synthesis Center from National Science Foundation (grant no. DBI-1052875) and to A.J.J. from National Academies of Sciences, Engineering and Medicine (grant no. 200007350).

Author information

P.A., S.A.G., A.J.J. and J.E.I. conceived the study given data collected by R.A. All authors were involved in theoretical development. P.A., S.A.G. and A.J.J. analysed the data. All authors were involved in interpretation and writing.

Correspondence to Payam Aminpour.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary methods, discussion, references, Figs. 1–3 and Tables 1–3.

Supplementary Dataset 1

Excel file of stakeholders’ adjacency matrices of fuzzy cognitive maps.

Supplementary Dataset 2

Excel file of experts’ adjacency matrices of fuzzy cognitive maps.

Supplementary Dataset 3

Excel file of participants’ identification codes and positions.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aminpour, P., Gray, S.A., Jetter, A.J. et al. Wisdom of stakeholder crowds in complex social–ecological systems. Nat Sustain (2020) doi:10.1038/s41893-019-0467-z

Download citation