Perspective | Published:

The evolution of citation graphs in artificial intelligence research

Nature Machine Intelligencevolume 1pages7985 (2019) | Download Citation


As artificial intelligence (AI) applications see wider deployment, it becomes increasingly important to study the social and societal implications of AI adoption. Therefore, we ask: are AI research and the fields that study social and societal trends keeping pace with each other? Here, we use the Microsoft Academic Graph to study the bibliometric evolution of AI research and its related fields from 1950 to today. Although early AI researchers exhibited strong referencing behaviour towards philosophy, geography and art, modern AI research references mathematics and computer science most strongly. Conversely, other fields, including the social sciences, do not reference AI research in proportion to its growing paper production. Our evidence suggests that the growing preference of AI researchers to publish in topic-specific conferences over academic journals and the increasing presence of industry research pose a challenge to external researchers, as such research is particularly absent from references made by social scientists.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


  1. 1.

    Brynjolfsson, E. & Mitchell, T. What can machine learning do? Workforce implications. Science 358, 1530–1534 (2017).

  2. 2.

    Kirilenko, A., Kyle, A. S., Samadi, M. & Tuzun, T. The flash crash: high-frequency trading in an electronic market. J. Finance 72, 967–998 (2017).

  3. 3.

    Brogaard, J. et al. High Frequency Trading and its Impact on Market Quality Working Paper No. 66 (Northwestern University Kellogg School of Management, 2010).

  4. 4.

    Verghese, A., Shah, N. H. & Harrington, R. A. What this computer needs is a physician: humanism and artificial intelligence. J. Am. Med. Assoc. 319, 19–20 (2018).

  5. 5.

    Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H. & Aerts, H. J. Artificial intelligence in radiology. Nat. Rev. Cancer 18, 500–510 (2018).

  6. 6.

    Bonnefon, J.-F., Shariff, A. & Rahwan, I. The social dilemma of autonomous vehicles. Science 352, 1573–1576 (2016).

  7. 7.

    The Road to Zero: A Vision of Achieving Zero Roadway Deaths by 2050 (National Safety Council and the RAND Corporation, 2018).

  8. 8.

    Russell, S., Hauert, S., Altman, R. & Veloso, M. Ethics of artificial intelligence. Nature 521, 415–416 (2015).

  9. 9.

    Rahwan, I. Society-in-the-loop: programming the algorithmic social contract. Ethics Inf. Technol. 20, 5–14 (2018).

  10. 10.

    Crandall, J. W. et al. Cooperating with machines. Nat. Commun. 9, 233 (2018).

  11. 11.

    Taddeo, M. & Floridi, L. How AI can be a force for good. Science 361, 751–752 (2018).

  12. 12.

    Miller, A. P. Want less-biased decisions? Use algorithms. Harvard Business Review (2018).

  13. 13.

    Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J. & Mullainathan, S. Human decisions and machine predictions. Q. J. Econ. 133, 237–293 (2017).

  14. 14.

    Naik, N., Kominers, S. D., Raskar, R., Glaeser, E. L. & Hidalgo, C. A. Computer vision uncovers predictors of physical urban change. Proc. Natl Acad. Sci. USA 114, 7571–7576 (2017).

  15. 15.

    Erel, S. L. H. T. C., Isil & Weisbach, M. S. Could machine learning help companies select better board directors? Harvard Business Review (2018).

  16. 16.

    Buolamwini, J. & Gebru, T. Gender shades: intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency 77–91 (MLR, 2018).

  17. 17.

    Buolamwini, J. How I’m fighting bias in algorithms. TED Talks (2016).

  18. 18.

    Frank, M. R., Sun, L., Cebrian, M., Youn, H. & Rahwan, I. Small cities face greater impact from automation. J. R. Soc. Interface 15, 20170946 (2018).

  19. 19.

    Frey, C. B. & Osborne, M. A. The future of employment: how susceptible are jobs to computerisation? Technol. Forecast. Soc. Change 114, 254–280 (2017).

  20. 20.

    Acemoglu, D. & Restrepo, P. Robots and jobs: evidence from US labor markets (National Bureau of Economic Research, 2017).

  21. 21.

    Klinger, J., Mateos-Garcia, J. C. & Stathoulopoulos, K. Deep learning, deep change? Mapping the development of the artificial intelligence general purpose technology. Preprint at (2018).

  22. 22.

    Sinha, A. et al. An overview of Microsoft Academic Service (MAS) and applications. In Proc. 24th International Conference on World Wide Web 243–246 (ACM, 2015).

  23. 23.

    Effendy, S. & Yap, R. H. Analysing trends in computer science research: a preliminary study using the microsoft academic graph. In Proceedings of the 26th International Conference on World Wide Web Companion, 1245–1250 (International World Wide Web Conferences Steering Committee, 2017).

  24. 24.

    Hug, S. E. & Brändle, M. P. The coverage of Microsoft academic: analyzing the publication output of a university. Scientometrics 113, 1551–1571 (2017).

  25. 25.

    Burd, R. et al. GRAM: global research activity map. In Proc. 2018 International Conference on Advanced Visual Interfaces 31 (ACM, 2018).

  26. 26.

    Fiala, D. & Tutoky, G. Computer science papers in web of science: a bibliometric analysis. Publications 5, 23 (2017).

  27. 27.

    Russell, S. J. & Norvig, P. Artificial Intelligence: A Modern Approach (Pearson Education Limited, London, 2016).

  28. 28.

    McCorduck, P. Machines Who Think: A Personal Inquiry into the History and Prospects of Artificial Intelligence (CRC, Natik, 2009).

  29. 29.

    Kieval, H. J. Pursuing the Golem of Prague: Jewish culture and the invention of a tradition. Mod. Jud. 17, 1–20 (1997).

  30. 30.

    Pollin, B. R. Philosophical and literary sources of Frankenstein. Comp. Lit. 17, 97–108 (1965).

  31. 31.

    Floridi, L. Distributed morality in an information society. Sci. Eng. Ethics 19, 727–743 (2013).

  32. 32.

    Plant, S. Zeros and ones (Doubleday Books, 1997).

  33. 33.

    David, A. H. Why are there still so many jobs? The history and future of workplace automation. J. Econ. Perspect. 29, 3–30 (2015).

  34. 34.

    Sinatra, R., Deville, P., Szell, M., Wang, D. & Barabási, A.-L. A century of physics. Nat. Phys. 11, 791 (2015).

  35. 35.

    Morgan, A. C., Economou, D., Way, S. F. & Clauset, A. Prestige drives epistemic inequality in the diffusion of scientific ideas. EPJ Data Sci. 7, 40 (2018).

  36. 36.

    Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484 (2016).

  37. 37.

    Bergholz, R., Timm, K. & Weisser, H. Autonomous vehicle arrangement and method for controlling an autonomous vehicle. US patent 6,151,539 (2000).

  38. 38.

    Pilutti, T. E., Rupp, M. Y., Trombley, R. A., Waldis, A. & Yopp, W. T. Autonomous vehicle identification. US patent 9,552,735 (2017).

  39. 39.

    Herbach, J. S. & Fairfield, N. Detecting that an autonomous vehicle is in a stuck condition. US patent 8,996,224 (2015).

  40. 40.

    Pham, M. C., Klamma, R. & Jarke, M. Development of computer science disciplines: a social network analysis approach. Soc. Netw. Anal. Min. 1, 321–340 (2011).

  41. 41.

    Barabási, A.-L. & Albert, R. Emergence of scaling in random networks. Science 286, 509–512 (1999).

  42. 42.

    Page, L., Brin, S., Motwani, R. & Winograd, T. The PageRank Citation Ranking: Bringing Order to the Web (Stanford InfoLab, 1999).

  43. 43.

    Larivière, V., Macaluso, B., Mongeon, P., Siler, K. & Sugimoto, C. R. Vanishing industries and the rising monopoly of universities in published research. PLoS ONE 13, 1–10 (2018).

  44. 44.

    Freyne, J., Coyle, L., Smyth, B. & Cunningham, P. Relative status of journal and conference publications in computer science. Commun. ACM 53, 124–132 (2010).

  45. 45.

    Newman, M. E. Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74, 036104 (2006).

Download references


The authors would like to thank E. Moro and Z. Epstein for their comments.

Author information


  1. Media Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA

    • Morgan R. Frank
    • , Manuel Cebrian
    •  & Iyad Rahwan
  2. Kellogg School of Management, Northwestern University, Evanston, IL, USA

    • Dashun Wang
  3. Northwestern Institute on Complex Systems, Northwestern University, Evanston, IL, USA

    • Dashun Wang
  4. Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA, USA

    • Iyad Rahwan
  5. Center for Humans and Machines, Max Planck Institute for Human Development, Berlin, Germany

    • Iyad Rahwan


  1. Search for Morgan R. Frank in:

  2. Search for Dashun Wang in:

  3. Search for Manuel Cebrian in:

  4. Search for Iyad Rahwan in:


M.R.F. and D.W. processed data and produced figures. All authors wrote the manuscript.

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Iyad Rahwan.

Supplementary information

  1. Supplementary Information

    Supplementary notes and figures

About this article

Publication history




Issue Date