Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • NEWS

AI now beats humans at basic tasks — new benchmarks are needed, says major report

Stanford University’s 2024 AI Index charts the meteoric rise of artificial-intelligence tools.
  1. Nicola Jones
    View author publications

    You can also search for this author in PubMed  Google Scholar

A woman wearing traditional Chinese clothing is playing Go with a robot at the Mobile World Congress, 2024.

A woman plays Go with an AI-powered robot developed by the firm SenseTime, based in Hong Kong. Credit: Joan Cros/NurPhoto via Getty

Artificial intelligence (AI) systems, such as the chatbot ChatGPT, have become so advanced that they now very nearly match or exceed human performance in tasks including reading comprehension, image classification and competition-level mathematics, according to a new report (see ‘Speedy advances’). Rapid progress in the development of these systems also means that many common benchmarks and tests for assessing them are quickly becoming obsolete.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 628, 700-701 (2024)

doi: https://doi.org/10.1038/d41586-024-01087-4

References

  1. Rein, D. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2311.12022 (2023).

  2. Li, P., Yang, J., Islam, M. A. & Ren, S. Preprint at arXiv https://doi.org/10.48550/arXiv.2304.03271 (2023).

Download references

Subjects

Latest on:

Nature Careers

Jobs

  • Research Associate - Metabolism

    Houston, Texas (US)

    Baylor College of Medicine (BCM)

  • Postdoc Fellowships

    Train with world-renowned cancer researchers at NIH? Consider joining the Center for Cancer Research (CCR) at the National Cancer Institute

    Bethesda, Maryland

    NIH National Cancer Institute (NCI)

  • Faculty Recruitment, Westlake University School of Medicine

    Faculty positions are open at four distinct ranks: Assistant Professor, Associate Professor, Full Professor, and Chair Professor.

    Hangzhou, Zhejiang, China

    Westlake University

  • PhD/master's Candidate

    PhD/master's Candidate    Graduate School of Frontier Science Initiative, Kanazawa University is seeking candidates for PhD and master's students i...

    Japan (JP)

    Kanazawa University

  • Senior Research Assistant in Human Immunology (wet lab)

    Senior Research Scientist in Human Immunology, high-dimensional (40+) cytometry, ICS and automated robotic platforms.

    Boston, Massachusetts (US)

    Boston University Atomic Lab

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing

Search

Advanced search

Quick links