As artificial intelligence (AI) applications see wider deployment, it becomes increasingly important to study the social and societal implications of AI adoption. Therefore, we ask: are AI research and the fields that study social and societal trends keeping pace with each other? Here, we use the Microsoft Academic Graph to study the bibliometric evolution of AI research and its related fields from 1950 to today. Although early AI researchers exhibited strong referencing behaviour towards philosophy, geography and art, modern AI research references mathematics and computer science most strongly. Conversely, other fields, including the social sciences, do not reference AI research in proportion to its growing paper production. Our evidence suggests that the growing preference of AI researchers to publish in topic-specific conferences over academic journals and the increasing presence of industry research pose a challenge to external researchers, as such research is particularly absent from references made by social scientists.
Subscribe to Journal
Get full journal access for 1 year
only $8.67 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Brynjolfsson, E. & Mitchell, T. What can machine learning do? Workforce implications. Science 358, 1530–1534 (2017).
Kirilenko, A., Kyle, A. S., Samadi, M. & Tuzun, T. The flash crash: high-frequency trading in an electronic market. J. Finance 72, 967–998 (2017).
Brogaard, J. et al. High Frequency Trading and its Impact on Market Quality Working Paper No. 66 (Northwestern University Kellogg School of Management, 2010).
Verghese, A., Shah, N. H. & Harrington, R. A. What this computer needs is a physician: humanism and artificial intelligence. J. Am. Med. Assoc. 319, 19–20 (2018).
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H. & Aerts, H. J. Artificial intelligence in radiology. Nat. Rev. Cancer 18, 500–510 (2018).
Bonnefon, J.-F., Shariff, A. & Rahwan, I. The social dilemma of autonomous vehicles. Science 352, 1573–1576 (2016).
The Road to Zero: A Vision of Achieving Zero Roadway Deaths by 2050 (National Safety Council and the RAND Corporation, 2018).
Russell, S., Hauert, S., Altman, R. & Veloso, M. Ethics of artificial intelligence. Nature 521, 415–416 (2015).
Rahwan, I. Society-in-the-loop: programming the algorithmic social contract. Ethics Inf. Technol. 20, 5–14 (2018).
Crandall, J. W. et al. Cooperating with machines. Nat. Commun. 9, 233 (2018).
Taddeo, M. & Floridi, L. How AI can be a force for good. Science 361, 751–752 (2018).
Miller, A. P. Want less-biased decisions? Use algorithms. Harvard Business Review https://hbr.org/2018/07/want-less-biased-decisions-use-algorithms (2018).
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J. & Mullainathan, S. Human decisions and machine predictions. Q. J. Econ. 133, 237–293 (2017).
Naik, N., Kominers, S. D., Raskar, R., Glaeser, E. L. & Hidalgo, C. A. Computer vision uncovers predictors of physical urban change. Proc. Natl Acad. Sci. USA 114, 7571–7576 (2017).
Erel, S. L. H. T. C., Isil & Weisbach, M. S. Could machine learning help companies select better board directors? Harvard Business Review https://hbr.org/2018/04/research-could-machine-learning-help-companies-select-better-board-directors (2018).
Buolamwini, J. & Gebru, T. Gender shades: intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency 77–91 (MLR, 2018).
Buolamwini, J. How I’m fighting bias in algorithms. TED Talks https://www.ted.com/talks/joy_buolamwini_how_i_m_fighting_bias_in_algorithms (2016).
Frank, M. R., Sun, L., Cebrian, M., Youn, H. & Rahwan, I. Small cities face greater impact from automation. J. R. Soc. Interface 15, 20170946 (2018).
Frey, C. B. & Osborne, M. A. The future of employment: how susceptible are jobs to computerisation? Technol. Forecast. Soc. Change 114, 254–280 (2017).
Acemoglu, D. & Restrepo, P. Robots and jobs: evidence from US labor markets (National Bureau of Economic Research, 2017).
Klinger, J., Mateos-Garcia, J. C. & Stathoulopoulos, K. Deep learning, deep change? Mapping the development of the artificial intelligence general purpose technology. Preprint at https://arxiv.org/abs/1808.06355 (2018).
Sinha, A. et al. An overview of Microsoft Academic Service (MAS) and applications. In Proc. 24th International Conference on World Wide Web 243–246 (ACM, 2015).
Effendy, S. & Yap, R. H. Analysing trends in computer science research: a preliminary study using the microsoft academic graph. In Proceedings of the 26th International Conference on World Wide Web Companion, 1245–1250 (International World Wide Web Conferences Steering Committee, 2017).
Hug, S. E. & Brändle, M. P. The coverage of Microsoft academic: analyzing the publication output of a university. Scientometrics 113, 1551–1571 (2017).
Burd, R. et al. GRAM: global research activity map. In Proc. 2018 International Conference on Advanced Visual Interfaces 31 (ACM, 2018).
Fiala, D. & Tutoky, G. Computer science papers in web of science: a bibliometric analysis. Publications 5, 23 (2017).
Russell, S. J. & Norvig, P. Artificial Intelligence: A Modern Approach (Pearson Education Limited, London, 2016).
McCorduck, P. Machines Who Think: A Personal Inquiry into the History and Prospects of Artificial Intelligence (CRC, Natik, 2009).
Kieval, H. J. Pursuing the Golem of Prague: Jewish culture and the invention of a tradition. Mod. Jud. 17, 1–20 (1997).
Pollin, B. R. Philosophical and literary sources of Frankenstein. Comp. Lit. 17, 97–108 (1965).
Floridi, L. Distributed morality in an information society. Sci. Eng. Ethics 19, 727–743 (2013).
Plant, S. Zeros and ones (Doubleday Books, 1997).
David, A. H. Why are there still so many jobs? The history and future of workplace automation. J. Econ. Perspect. 29, 3–30 (2015).
Sinatra, R., Deville, P., Szell, M., Wang, D. & Barabási, A.-L. A century of physics. Nat. Phys. 11, 791 (2015).
Morgan, A. C., Economou, D., Way, S. F. & Clauset, A. Prestige drives epistemic inequality in the diffusion of scientific ideas. EPJ Data Sci. 7, 40 (2018).
Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484 (2016).
Bergholz, R., Timm, K. & Weisser, H. Autonomous vehicle arrangement and method for controlling an autonomous vehicle. US patent 6,151,539 (2000).
Pilutti, T. E., Rupp, M. Y., Trombley, R. A., Waldis, A. & Yopp, W. T. Autonomous vehicle identification. US patent 9,552,735 (2017).
Herbach, J. S. & Fairfield, N. Detecting that an autonomous vehicle is in a stuck condition. US patent 8,996,224 (2015).
Pham, M. C., Klamma, R. & Jarke, M. Development of computer science disciplines: a social network analysis approach. Soc. Netw. Anal. Min. 1, 321–340 (2011).
Barabási, A.-L. & Albert, R. Emergence of scaling in random networks. Science 286, 509–512 (1999).
Page, L., Brin, S., Motwani, R. & Winograd, T. The PageRank Citation Ranking: Bringing Order to the Web (Stanford InfoLab, 1999).
Larivière, V., Macaluso, B., Mongeon, P., Siler, K. & Sugimoto, C. R. Vanishing industries and the rising monopoly of universities in published research. PLoS ONE 13, 1–10 (2018).
Freyne, J., Coyle, L., Smyth, B. & Cunningham, P. Relative status of journal and conference publications in computer science. Commun. ACM 53, 124–132 (2010).
Newman, M. E. Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E 74, 036104 (2006).
The authors would like to thank E. Moro and Z. Epstein for their comments.
The authors declare no competing interests.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Frank, M.R., Wang, D., Cebrian, M. et al. The evolution of citation graphs in artificial intelligence research. Nat Mach Intell 1, 79–85 (2019). https://doi.org/10.1038/s42256-019-0024-5
AAPS PharmSciTech (2020)
Autonomous Vehicles and Avoiding the Trolley (Dilemma): Vehicle Perception, Classification, and the Challenges of Framing Decision Ethics
Cybernetics and Systems (2020)
Validity of machine learning in biology and medicine increased through collaborations across fields of expertise
Nature Machine Intelligence (2020)
The AI techno-economic complex System: Worldwide landscape, thematic subdomains and technological collaborations
Telecommunications Policy (2020)
WIREs Data Mining and Knowledge Discovery (2020)