Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Skill levels and gains in university STEM education in China, India, Russia and the United States

Nature Human Behaviour volume 5, pages 892–904 (2021)Cite this article

Subjects

An Author Correction to this article was published on 06 April 2021

This article has been updated

Abstract

Universities contribute to economic growth and national competitiveness by equipping students with higher-order thinking and academic skills. Despite large investments in university science, technology, engineering and mathematics (STEM) education, little is known about how the skills of STEM undergraduates compare across countries and by institutional selectivity. Here, we provide direct evidence on these issues by collecting and analysing longitudinal data on tens of thousands of computer science and electrical engineering students in China, India, Russia and the United States. We find stark differences in skill levels and gains among countries and by institutional selectivity. Compared with the United States, students in China, India and Russia do not gain critical thinking skills over four years. Furthermore, while students in India and Russia gain academic skills during the first two years, students in China do not. These gaps in skill levels and gains provide insights into the global competitiveness of STEM university students across nations and institutional types.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Critical thinking skill levels and gains (s.d. units) across China, India and Russia with benchmarks from the United States.
Fig. 2: Maths and physics skill levels and gains from the start of the first year to the end of the second year (s.d. units).

Data availability

Data have been deposited at the Open Science Framework (https://osf.io/4t8cu/).

Code availability

Stata do-files used to perform the analyses have been deposited at the Open Science Framework (https://osf.io/4t8cu/).

Change history

References

  1. National Academies of Sciences, Engineering, and Medicine. Indicators for Monitoring Undergraduate STEM Education (National Academies Press, 2017).

  2. Mayhew, M. J. et al. How College Affects Students Volume 3: 21st Century Evidence that Higher Education Works (Jossey-Bass, 2016).

  3. Criteria for Accrediting Engineering Technology Programs (ABET, 2015); www.abet.org/wp-content/uploads/2015/10/T001-16-17-ETAC-Criteria-10-16-15.pdf

  4. Passow, H. J. Which ABET competencies do engineering graduates find most important in their work? J. Eng. Educ. 101, 95–118 (2012).

    Article  Google Scholar 

  5. National Research Council. Education for Life and Work: Developing Transferable Knowledge and Skills in the Twenty-First Century (National Academies Press, 2012).

  6. Arum, R. & Roksa, J. Academically Adrift: Limited Learning on College Campuses (Univ. Chicago Press, 2011).

  7. AAC&U. How Should Colleges Assess and Improve Student Learning? Employers’ Views on the Accountability Challenge (AAC&U, 2008).

  8. National Academy of Science. Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future (National Academy of Science, 2005).

  9. Goldin, C. D. & Katz, L. F. The Race Between Education and Technology (Harvard Univ. Press, 2009).

  10. Autor, D. H., Levy, F. & Murnane, R. J. The skill content of recent technological change: an empirical exploration. Q. J. Econ. 118, 1279–1333 (2003).

    Article  Google Scholar 

  11. Bresnahan, T. F., Brynjolfsson, E. & Hitt, L. M. Information technology, workplace organization, and the demand for skilled labor: firm-level evidence. Q. J. Econ. 117, 339–376 (2002).

    Article  Google Scholar 

  12. Katz, L. F. & Krueger, A. B. Computer science inequality: have computers changed the labor market? Q. J. Econ. 113, 1169–1213 (1998).

    Article  Google Scholar 

  13. Lucas, R. E. On the mechanics of economic development. J. Monetary Econ. 22, 3–42 (1988).

    Article  Google Scholar 

  14. Romer, P. M. Endogenous technological change. J. Polit. Econ. 98, S71–S102 (1990).

    Article  Google Scholar 

  15. Mankiw, N. G., Romer, D. & Weil, D. N. A contribution to the empirics of economic growth. Q. J. Econ. 107, 407–437 (1992).

    Article  Google Scholar 

  16. Murnane, R. J., Willett, J. B. & Levy, F. The growing importance of cognitive skills in wage determination. Rev. Econ. Stat. 77, 251–266 (1995).

    Article  Google Scholar 

  17. Hanushek, E. A. & Kimko, D. D. Schooling, labor-force quality, and the growth of nations. Am. Econ. Rev. 90, 1184–1208 (2000).

    Article  Google Scholar 

  18. Hanushek, E. A. & Woessmann, L. The role of cognitive skills in economic development. J. Econ. Lit. 46, 607–668 (2008).

    Article  Google Scholar 

  19. Hanushek, E. & Woessmann, L. Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation. J. Econ. Growth 17, 267–321 (2012).

    Article  Google Scholar 

  20. Hanushek, E. & Woessmann, L. Schooling, educational achievement, and the Latin American growth puzzle. J. Dev. Econ. 99, 497–512 (2012).

    Article  Google Scholar 

  21. Kaarsen, N. Cross-country differences in the quality of schooling. J. Dev. Econ. 107, 215–224 (2014).

    Article  Google Scholar 

  22. Heckman, J., Stixrud, J. & Urzua, S. The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. J. Labor Econ. 24, 411–482 (2006).

    Article  Google Scholar 

  23. Deming, D. & Kahn, L. B. Skill requirements across firms and labor markets: evidence from job postings for professionals. J. Labor Econ. 36, S337–S369 (2018).

    Article  Google Scholar 

  24. World Bank. World Development Report 2018: Learning to Realize Education’s Promise (World Bank, 2018).

  25. Loyalka, P. et al. Factors affecting the quality of engineering education in the four largest emerging economies. High. Educ. 68, 977–1004 (2014).

    Article  Google Scholar 

  26. Musekamp, F. & Pearce, J. Assessing engineering competencies: the conditions for educational improvement. Stud. High. Educ. 40, 505–524 (2015).

    Article  Google Scholar 

  27. Zlatkin-Troitschanskaia, O., Shavelson, R. J. & Kuhn, C. The international state of research on measurement of competency in higher education. Stud. High. Educ. 40, 393–411 (2015).

    Article  Google Scholar 

  28. OECD. Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study Report—Volume 2 Data Analysis and National Experiences (OECD, 2012).

  29. Loyalka, P. et al. Computer science skills across China, India, Russia, and the United States. Proc. Natl Acad. Sci. USA 116, 6732–6736 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. OECD. Education at a Glance 2018 (OECD, 2018).

  31. Montenegro, C. E. & Patrinos, H. A. Comparable Estimates of Returns to Schooling Around the World (World Bank, 2014).

  32. Spence, M. Job market signaling. Q. J. Econ. 87, 355–374 (1973).

    Article  Google Scholar 

  33. Moretti, E. Estimating the social return to higher education: evidence from longitudinal and repeated cross-sectional data. J. Econ. 121, 175–212 (2004).

    Article  Google Scholar 

  34. Aghion, P., Howitt, P., Howitt, P. W., Brant-Collett, M. & García-Peñalosa, C. Endogenous Growth Theory (MIT press, 1998).

  35. Nelson, R. R. & Phelps, E. S. Investment in humans, technological diffusion, and economic growth. Am. Econ. Rev. 56, 69–75 (1966).

    Google Scholar 

  36. Benhabib, J. & Spiegel, M. M. Human capital and technology diffusion. Handb. Econ. Growth 1, 935–966 (2005).

    Article  Google Scholar 

  37. Katz, L. F. & Murphy, K. M. Changes in relative wages, 1963–1987: supply and demand factors. Q. J. Econ. 107, 35–78 (1992).

    Article  Google Scholar 

  38. Autor, D. H., Katz, L. F. & Krueger, A. B. Computing inequality: have computers changed the labor market? Q. J. Econ. 113, 1169–1213 (1998).

    Article  Google Scholar 

  39. National Science Board. Science and Engineering Indicators 2016 (National Science Foundation, 2016).

  40. Bourdieu, P. Homo Academicus (Stanford Univ. Press, 1988)

  41. Shavit, Y., Arum, R. & Gamoran, A. (eds) Stratification in Higher Education: A Comparative Study. (Stanford Univ. Press, 2007).

  42. Altbach, P. G. & Salmi, J. (eds) The Road to Academic Excellence: The Making of World-Class Research Universities (The World Bank, 2011).

  43. Carnoy, M. et al. University Expansion in a Changing Global Economy: Triumph of the BRICs? (Stanford Univ. Press, 2013).

  44. World Bank. World Development Indicators 2018 (World Bank, 2018); http://databank.worldbank.org/data/reports.aspx?source=world-development-indicators

  45. Muralidharan, K., Das, J., Holla, A. & Mohpal, A. The fiscal cost of weak governance: evidence from teacher absence in India. J. Publ. Econ. 145, 116–135 (2017).

    Article  Google Scholar 

  46. Burdett, N. Review of High Stakes Examination Instruments in Primary and Secondary School in Developing Countries Research on Improving Systems of Education (RISE) Working Paper 17/018 (RISE Programme, 2017).

  47. Cheney, G. R., Ruzzi, B. B. & Muralidharan, K. A Profile of the Indian Education System (National Center on Education and Economy, 2005).

  48. Guo, C. B., Tsang, M. & Ding, X. H. Gender difference in the education and employment of science & engineering students in HEIs. J. High. Educ. 11, 89–101 (2007).

    Google Scholar 

  49. Dobryakova, M. & Froumin, I. Higher engineering education in Russia: incentives for real change. Int. J. Eng. Educ. 26, 1032–1041 (2010).

    Google Scholar 

  50. Pal, Y. Report of the Committee to Advise on Renovation and Rejuvenation of Higher Education (Department of Human Resources, Government of India, 2009).

  51. Zha, Q., Hu, L. & Wang, Y. The characteristics of university students’ extracurricular time assignment and its impact on learning outcome: based on China college student survey in J University in 2016. High. Educ. Explor. 7, 44–49 (2017).

    Google Scholar 

  52. Lv, L. & Zhang, H. The characteristics of undergraduates’ learning engagement of Chinese research-oriented university: based on the comparison of 12 research-oriented universities in the world. Educ. Res. 36, 51–63 (2015).

    Google Scholar 

  53. Mandel, H. & Semyonov, M. Gender pay gap and employment sector: sources of earnings disparities in the United States, 1970-2010. Demography 51, 1597–1618 (2014).

    Article  PubMed  Google Scholar 

  54. Bound, J., Lovenheim, M. F. & Turner, S. Why have college completion rates declined? An analysis of changing student preparation and collegiate resources. Am. Economic J. Appl. Econ. 2, 129–157 (2010).

    Article  Google Scholar 

  55. Bettinger, E. P., Long, B. T., Oreopoulos, P. & Sanbonmatsu, L. The role of application assistance and information in college decisions: results from the H&R Block FAFSA experiment. Q. J. Econ. 127, 1205–1242 (2012).

    Article  Google Scholar 

  56. Stinebrickner, T. & Stinebrickner, R. Learning about academic ability and the college dropout decision. J. Labor Econ. 30, 707–748 (2012).

    Article  Google Scholar 

  57. Glewwe, P. & Kremer, M. Schools, teachers, and education outcomes in developing countries. Handb. Econ. Educ. 2, 945–1017 (2006).

    Article  Google Scholar 

  58. McEwan, P. Improving learning in primary schools of developing countries: a meta-analysis of randomized experiments. Rev. Educ. Res. 85, 353–394 (2013).

    Article  Google Scholar 

  59. Hanushek, E. A. Will more higher education improve economic growth? Oxford Rev. Econ. Pol. 32, 538–552 (2016).

    Article  Google Scholar 

  60. Clark, D. & Martorell, P. The signaling value of a high school diploma. J. Polit. Econ. 122, 282–318 (2014).

    Article  Google Scholar 

  61. Li, H., Meng, L., Shi, X. & Wu, B. Does attending elite colleges pay in China? J. Comp. Econ. 40, 78–88 (2012).

    Article  Google Scholar 

  62. Muralidharan, K. Priorities for Primary Education Policy in India’s 12th Five-Year Plan India Policy Forum Vol. 9, 1–61 (National Council of Applied Economic Research and Brookings Institution, 2013).

  63. Li, H., Loyalka, P., Rozelle, S. & Wu, B. Human capital and China’s future growth. J. Econ. Perspect. 31, 25–48 (2017).

    Article  Google Scholar 

  64. Kouzminov, Y., Ovcharova, L. & Yakobson, L. (eds) How to Increase Human Capital and its Impact on Economic and Social Development (HSE Publishing House, 2018).

  65. Bound, J., Braga, B., Golden, J. & Khanna, G. Recruitment of foreigners in the market for computer scientists in the United States. J. Labor Econ. 33, S187–S223 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  66. Bound, J., Khanna, G. & Morales, N. Reservoir of foreign talent. Science 356, 697 (2017).

    Article  CAS  PubMed  Google Scholar 

  67. Hanson, G. H. & Slaughter, M. J. in Education, Skills, and Technical Change: Implications for Future US GDP Growth (eds Hulten, C. R. & Ramey, V. A.) 465–494 (Univ. Chicago Press, 2017).

  68. Liu, O. L., Frankel, L. & Roohr K. C. Assessing Critical Thinking in Higher Education: Current State and Directions for Next‐Generation Assessment ETS Research Report Series (ETS, 2014).

  69. Kardanova, E. et al. Developing instruments to assess and compare the quality of engineering education: the case of China and Russia. Assess. Eval. High. Educ. 41, 770–786 (2016).

    Article  Google Scholar 

Download references

Acknowledgements

We thank M. Carnoy, J. Cohen, T. Dee, B. Domingue, A. Eble, R. Fairlie, E. Hanushek, B. Kim, S. Loeb, K. Muralidharan, S. Reardon, S. Rozelle, D. Schwartz, S. Sylvia and C. Wieman, and participants at the demography workshop at the University of Chicago, the economics of education workshop at the Teachers College, the South Asia Region Knowledge Exchange Group at the World Bank, KDI School and technical reviewers at ETS for their feedback. We appreciate research funding from E. Li, the Basic Research Program of the National Research University Higher School of Economics and Russian Academic Excellence Project 5–100, and the All India Council for Technical Education. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Graduate School of Education, Stanford University, Stanford, CA, USA

    Prashant Loyalka & Zhaolei Shi

  2. Freeman Spogli Institute for International Studies, Stanford University, Stanford, CA, USA

    Prashant Loyalka, Huan Wang, Saurabh Khanna & Ashutosh Bhuradia

  3. Educational Testing Service, Princeton, NJ, USA

    Ou Lydia Liu, Lin Gu & Guangming Ling

  4. International Center for Action Research on Education, School of Education, Henan University, Henan, China

    Guirong Li & Yanyan Li

  5. Institute of Education, National Research University Higher School of Economics, Moscow, Russia

    Elena Kardanova, Igor Chirikov & Denis Federiakin

  6. Center for Studies in Higher Education, Goldman School of Public Policy, University of California Berkeley, Berkeley, CA, USA

    Igor Chirikov

  7. Faculty of Education, Sichuan Normal University, Sichuan, China

    Shangfeng Hu

  8. Institute of Higher Education Research, University of Jinan, Jinan, China

    Ningning Yu

  9. Graduate School of Education, Peking University, Beijing, China

    Liping Ma

  10. Institute of Education, Tsinghua University, Beijing, China

    Fei Guo

  11. World Bank, Washington, DC, USA

    Tara Beteille & Namrata Tognatta

Authors
  1. Prashant Loyalka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ou Lydia Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guirong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elena Kardanova
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Igor Chirikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shangfeng Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ningning Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Liping Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Fei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Tara Beteille
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Namrata Tognatta
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Lin Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Guangming Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Denis Federiakin
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Huan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Saurabh Khanna
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Ashutosh Bhuradia
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Zhaolei Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Yanyan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

P.L., O.L.L., G.Li, I.C., E.K., N.Y., F.G., L.M., S.H., A.B., T.B. and N.T. designed research. P.L., O.L.L., G.Li, I.C., E.K., N.Y., F.G., L.M., S.H., H.W., Y.L., A.B. and S.K. performed research. P.L., O.L.L., E.K., D.F., L.G., G.Ling, S.K. and Z.S. analysed data. P.L., O.L.L. and I.C. wrote the paper.

Corresponding authors

Correspondence to Prashant Loyalka or Igor Chirikov.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Human Behaviour thanks Alex Eble, Thomas Luschei and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editor: Charlotte Payne.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary References and Supplementary Tables 1–7.

Reporting Summary

Peer Review Information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Loyalka, P., Liu, O.L., Li, G. et al. Skill levels and gains in university STEM education in China, India, Russia and the United States. Nat Hum Behav 5, 892–904 (2021). https://doi.org/10.1038/s41562-021-01062-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-021-01062-3

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing